Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install Sqoop 1.4.6. It is very detailed and has certain reference value. If you are interested, you must finish it.

1. Sqoop data migration

1.1 Overview

Sqoop is an apache tool for transferring data between Hadoop and relational database servers.

Import data: MySQL,Oracle imports data into Hadoop's HDFS, HIVE, HBASE and other data storage systems

Export data: export data from Hadoop's file system to a relational database

1.2 working mechanism

Translate import or export commands into mapreduce programs to implement

In the translated mapreduce, inputformat and outputformat are mainly customized.

1.3 sqoop practice and principle

1.3.1 sqoop installation

The prerequisite for installing sqoop is that you already have an environment for java and hadoop

1. Download and decompress

The latest download address is http://ftp.wayne.edu/apache/sqoop/1.4.6/

For example: sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

[root@hadoop1 sqoop] # tar-zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

[root@hadoop1 sqoop] # mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop

[root@hadoop1 sqoop] # ls

Apache-hive-1.2.1-bin hadoop-2.7.4 hdfs pig-0.17.0 pig_1517170893185.log sqoop tmp

2. Modify the configuration file

Configure sqoop_home in / etc/profile as follows:

Vim / etc/profile

Export SQOOP_HOME=/usr/local/hadoop/sqoop

Append path

Export PATH=$PATH:$SQOOP_HOME/bin

[root@hadoop1 sqoop] # source / etc/profile

$cd $SQOOP_HOME/conf

$mv sqoop-env-template.sh sqoop-env.sh

Open sqoop-env.sh and edit the following lines: # # remove the previous # #

Export HADOOP_COMMON_HOME=/usr/local/hadoop/hadoop-2.7.4/

Export HADOOP_MAPRED_HOME=/usr/local/hadoop/hadoop-2.7.4/

Export HIVE_HOME=/usr/local/hadoop/apache-hive-1.2.1-bin/

The effect of the configured interface is as follows:

Write the picture description here.

3.1 add the driver package to oracle

Put ojdbc6.jar under $SQOOP_HOME/lib/.

3.2 jdbc driver package added to mysql

Put mysql-connector-java-5.1.38.jar under $SQOOP_HOME/lib/.

4. Verify startup

$cd $SQOOP_HOME/bin

$sqoop-version

Expected output:

[root@hadoop1 sqoop] # sqoop-version

Warning: / usr/local/hadoop/sqoop/../hbase does not exist! HBase imports will fail.

Please set $HBASE_HOME to the root of your HBase installation.

Warning: / usr/local/hadoop/sqoop/../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: / usr/local/hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: / usr/local/hadoop/sqoop/../zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

18-01-29 19:09:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

Sqoop 1.4.6

Git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25

Compiled by root on Mon Apr 27 14:38:36 CST 2015

[root@hadoop1 sqoop] #

The above is all the contents of the article "how to install Sqoop 1.4.6". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.