Shulou(Shulou.com)05/31 Report--

This article is about how to use Sqoop. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Sqoop is an open source tool mainly used for Hadoop (Hive) and traditional databases (mysql, postgresql...) Data transfer is carried out between.

Sqoop is a tool for transferring data between Hadoop and relational database. It can import data from a relational database (such as Mysql,Oracle,Postgres, etc.) into the HDFS of Hadoop, or data from HDFS into a relational database.

Sqoop imports mysql database running process depends on mysql-connector-java-*.jar

List all databases in the mysql database

Sqoop list-databases-connect jdbc:mysql://localhost:3306/-username root-password admin

Connect mysql and list the tables in the data

Sqoop list-tables-connect jdbc:mysql://localhost:3306/test-username root-password admin

Copying the table structure of a relational database to hive only copies the structure of the table, and the contents of the table are not copied.

Sqoop create-hive-table-connect jdbc:msql://localhost:3306/test-table sqoop_test-username root-password admin

Import data from rdbms to hdfs using sqoop

Sqoop import-connect jdbc:mysql://hadoop:3306/test-username root-password admin-table user-fields-terminated-by'\ t'- m

Note: test is the name of the database

Export data from hdfs to mysql using sqoop

Sqoop export--connect jdbc:mysql://hadoop0:3306/test-- username root-- password admin-- table user-- fields-terminated-by'\ t'--export-dir'/ user/root/user'

Note: there is a data file under / user/root/user: the data in sort is related to the user fields in this database table.

Thank you for reading! This is the end of the article on "how to use Sqoop". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.