Shulou(Shulou.com)06/03 Report--

August 27, 2016, Resolved SQOOP

First download is sqoop 1.99.7, the result configuration time, there is no server/conf directory, and startup can not configure error

./ bin/sqoop.sh server start

yq@ubuntu:/opt/sqoop-1.99.7$ ./ bin/sqoop.sh server start

Setting conf dir: ./ bin/../ conf

Sqoop home directory: /opt/sqoop-1.99.7

Starting the Sqoop2 server...

Sqoop2 server started.

dyq@ubuntu:/opt/sqoop-1.99.7$ 0 [main] INFO org.apache.sqoop.core.SqoopServer - Initializing Sqoop server.

58 [main] INFO org.apache.sqoop.core.PropertiesConfigurationProvider - Starting config file poller thread

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration

at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem. (DefaultMetricsSystem.java:38)

at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem. (DefaultMetricsSystem.java:36)

at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:120)

at org.apache.hadoop.security.UserGroupInformation. (UserGroupInformation.java:236)

at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:39)

at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98)

at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)

at org.apache.sqoop.server.SqoopJettyServer. (SqoopJettyServer.java:67)

at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)

Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 9 more

The solution is to download sqoop 1.99.3, and then find the following configuration, and fix it:

2. Unzip files to working directory:

hadoop@hadoopMaster:$ sudo tar -xvf /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200.tar.gz

hadoop@hadoopMaster:mv /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200 /usr/local/sqoop

3. Modify environment variables:

hadoop@hadoopMaster:~$ vim /etc/profile

Add the following:

#sqoop

export SQOOP_HOME=/usr/sqoop-1.99.3

export PATH=$SQOOP_HOME/bin:$PATH

export CATALINA_HOME=$SQOOP_HOME/server

export LOGDIR=$SQOOP_HOME/logs

Save Exit with immediate effect:

source /etc/profile

4. Modify sqoop configuration:

hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/sqoop.properties

#Modify the profile directory pointing to my hadoop

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop-2.6.2/etc/hadoop

#Import jar packages from hadoop directory

hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/catalina.properties

common.loader=/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop/share/hadoop/httpfs/tomcat/lib/*.jar

5. Download mysql driver package

mysql-connector-java-5.1.16-bin.jar

6. Start/Stop SQOOP200

hadoop@hadoopMaster:/usr/local/sqoop/bin$ ./ sqoop.sh server start/stop

Try running:

dyq@ubuntu:/opt/sqoop-1.99.3$ ./ bin/sqoop.sh server start

Sqoop home directory: /opt/sqoop-1.99.3

Setting SQOOP_HTTP_PORT: 12000

Setting SQOOP_ADMIN_PORT: 12001

Using CATALINA_OPTS:

Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001

Using CATALINA_BASE: /opt/sqoop-1.99.3/server

Using CATALINA_HOME: /opt/sqoop-1.99.3/server

Using CATALINA_TMPDIR: /opt/sqoop-1.99.3/server/temp

Using JRE_HOME: /usr/java/jdk1.7.0_79/jre

Using CLASSPATH: /opt/sqoop-1.99.3/server/bin/bootstrap.jar

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.