Shulou(Shulou.com)06/03 Report--

Perform the export of data from the MySQL database to HDFS.

Preparation is required before execution:

The code written in eclipse, so a new sqoop-1.4.6-cdh6.5.2.jar is added to the original hadoop project, otherwise there will be compilation errors.

At the same time, on the cluster, add this package under HADOOP_HOME/share/hadoop/yarn, otherwise an error of missing Jar package will be reported.

Error display:

[hadoop@fa01 ~] $hadoop jar SqoopTest1.jar

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/sqoop/util/OptionsFileUtil

Code:

Import org.apache.hadoop.conf.Configuration

Import org.apache.sqoop.Sqoop

Import org.apache.sqoop.tool.SqoopTool

Import org.apache.sqoop.util.OptionsFileUtil

Public class SqoopTest1 {

Private static int importDataFromMysql () throws Exception {

String [] args = new String [] {

/ / parameters can be added.

"--connect", "jdbc:mysql://192.20.1.54:3306/tbname_test"

"--driver", "com.mysql.jdbc.Driver"

"--username", "username"

"- password", "123456"

"--table", "abtest_data"

"- m", "1"

"--target-dir", "/ user/hadoop/java_sqoop_test1"

}

String [] expandArguments = OptionsFileUtil.expandArguments (args)

SqoopTool tool = SqoopTool.getTool ("import")

Configuration conf = new Configuration ()

Conf.set ("fs.default.name", "hdfs://192.168.20.32:9000")

Configuration loadPlugins = SqoopTool.loadPlugins (conf)

Sqoop sqoop = new Sqoop ((com.cloudera.sqoop.tool.SqoopTool) tool, loadPlugins)

Return Sqoop.runSqoop (sqoop, expandArguments)

}

Public static void main (String [] args) throws Exception {

ImportDataFromMysql ()

}

}

Reference:

Http://blog.csdn.net/shantaodaoshik852/article/details/77266831

Http://blog.csdn.net/guzicheng/article/details/41519947

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.