Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to import and export Sqoop1, the introduction in the article is very detailed, has a certain reference value, interested friends must read!

"the difference between Sqoop1 and Sqoop2

The Sqoop1 is directly connected to the Hadoop by the Client side, and the task is executed by parsing to generate the corresponding MR.

Sqoop1 has only one client, the architecture is simple and clear, the deployment is ready to use, the threshold for use is relatively low, but the coupling is strong, and the exposure of user passwords is not secure.

Sqoop2 server deploys, runs, and provides Cli, rest api, webui and other portals. Connector centralized management, RDBMS control account is more secure, but Sqoop2 is only responsible for data read and write operations, and the architecture is relatively complex.

"Sqoop1 Import (MySQL Import to HDFS)

Note: start MySQL, ZK, Hadoop first

# demo1:table# sqoop1 Import Command mysql= > hdfsbin/sqoop import # mysql driver Class-- driver com.mysql.cj.jdbc.Driver# connection string-- which table does connect jdbc:mysql://192.168.56.1/dbName?serverTimezone=GMT%2B8# import data from-- table tbName# mysql account-- username root# mysql password-- password 123456-- verbose # destination directory (HDFS)-- target-dir=/sqoop/a_users# demo2:query# Import data #-- query and-- table are mutually exclusive Bin/sqoop import-- connect jdbc:mysql://192.168.56.1/dbName?serverTimezone=GMT%2B8--username root-- password 123456-- verbose # get the number of records-- fetch-size 30 # SQL statement # $CONDITIONS sql statement execution condition controlled by sqoop-- query 'select * from a_users where $CONDITIONS' # split according to id #-- split-by The split column cannot be repeated, usually the primary key-split-by id # target directory (HDFS)-target-dir=/sqoop/a_users# demo3: encapsulate the sqoop script in a file # create a new file; (one per line, use # comments) Write import--connect jdbc:mysql://192.168.56.1/dbName?serverTimezone=GMT%2B8--username root--password 123456--verbose--fetch-size 30--query 'select * from a_users where $CONDITIONS'--split-by id--target-dir=/sqoop/a_users casually in the file name

Execute commands for Sqoop script files

Bin/sqoop-options-file / root/fileName.txt above are all the contents of the article "how to Import and Export Sqoop1". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.