Shulou(Shulou.com)06/01 Report--

Download the installation package

Http://archive.apache.org/dist/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

I installed it under the hadoop user.

The hadoop directory is / home/hadoop/hadoop

The hive directory is / home/hadoop/hive

The sqoop directory is / home/hadoop/sqoop

The current directory is / home/hadoop

Tar-zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

Mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop

Add the sqoop environment variable to / etc/profile

Export SQOOP_HOME=/home/hadoop/sqoop

Export PATH=$PATH:$SQOOP_HOME/bin

Create a sqoop-env.conf file

Export HADOOP_COMMON_HOME=/home/hadoop/hadoop

Export HADOOP_MAPRED_HOME=/home/hadoop/hadoop

Export HIVE_HOME=/home/hadoop/hive

Modify $SQOOP_HOME/bin/configure-sqoop

# if [- z "${HCAT_HOME}"]; then

# if [- d "/ usr/lib/hive-hcatalog"]; then

# HCAT_HOME=/usr/lib/hive-hcatalog

# elif [- d "/ usr/lib/hcatalog"]; then

# HCAT_HOME=/usr/lib/hcatalog

# else

# HCAT_HOME=$ {SQOOP_HOME} /.. / hive-hcatalog

# if [!-d ${HCAT_HOME}]; then

# HCAT_HOME=$ {SQOOP_HOME} /.. / hcatalog

# fi

# fi

# fi

# if [- z "${ACCUMULO_HOME}"]; then

# if [- d "/ usr/lib/accumulo"]; then

# ACCUMULO_HOME=/usr/lib/accumulo

# else

# ACCUMULO_HOME=$ {SQOOP_HOME} /.. / accumulo

# fi

# fi

# # Moved to be a runtime check in sqoop.

# if [!-d "${HCAT_HOME}"]; then

# echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."

# echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'

# fi

#

# if [!-d "${ACCUMULO_HOME}"]; then

# echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."

# echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'

# fi

Comment out the HCatalog,Accumulo check (unless you are going to use components on HADOOP such as HCatalog,Accumulo)

My sqoop will import the oracle database into hive.

So we need to use oracle jdbc to drive ojdbc6.jar

Download this driver from oracle's official website and put it at

/ home/hadoop/sqoop/lib directory.

View sqoop version

[hadoop@bi-hadoop01 sqoop] $sqoop version

Warning: / home/hadoop/sqoop/../hbase does not exist! HBase imports will fail.

Please set $HBASE_HOME to the root of your HBase installation.

Warning: / home/hadoop/sqoop/../zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

16-10-13 14:33:44 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

Sqoop 1.4.6

Git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25

Compiled by root on Mon Apr 27 14:38:36 CST 2015

View sqoop-import help

Execute sqoop-import-help

Test oracle Import hive data

Sqoop import-hive-import-conect jdbc:oracle:thin:@192.168.106.35:1521:cyporcl-username CYP-password cyp-verbose-M1-table CYP_ETL_PROC_CONFIG

Note that user names and table names must be capitalized.

Import succeeded

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.