Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how to omit entering MySQL to save password when executing Job". In the operation process of actual cases, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

1. Delete Job

sqoop job \

--delete testjob

II. Creating a Job

Note: When creating a Job, we use the--password-file parameter, and not--passowrd. The main reason is that using the--password parameter when executing a Job will have warnings and requires entering a password to execute the Job. When we use the--password-file parameter, we execute Job without entering the database password.

sqoop job \

--create testjob \

-- \

import \

--connect jdbc:mysql://192.168.137.130:3306/jepsondb \

--username root \

--password-file /input/sqoop.pwd \

--table abc \

--target-dir /input/abc \

--delete-target-dir \

-m 1

Note: There must be a space between import and "--"

III. Execution of Job

sqoop job \

--exec testjob

4. Create the password file required by the--password-file parameter (Note: sqoop stipulates that the password file must be placed on HDFS and the permission must be 400)

1. Operation command:

echo -n "123456" > sqoop.pwd

hdfs dfs -rm sqoop.pwd /input/sqoop.pwd

hdfs dfs -put sqoop.pwd /input

hdfs dfs -chmod 400 /input/sqoop.pwd

hdfs dfs -ls /input

-r-------- 1 hadoop supergroup 6 2018-01-15 18:38 /input/sqoop.pwd

2. Check whether the following configuration exists in the sqoop configuration file (sqoop-site.xml), and the value value is true.

sqoop.metastore.client.record.password

true

If true, allow saved passwords in the metastore.

Note: The password file format is incorrect error prompt as follows:

arning: /app/sqoop-1.4.6-cdh6.7.0/../ hbase does not exist! HBase imports will fail.

Please set $HBASE_HOME to the root of your HBase installation.

Warning: /app/sqoop-1.4.6-cdh6.7.0/../ hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /app/sqoop-1.4.6-cdh6.7.0/../ accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /app/sqoop-1.4.6-cdh6.7.0/../ zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

18/01/15 18:33:50 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh6.7.0

18/01/15 18:33:53 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

18/01/15 18:33:53 INFO tool.CodeGenTool: Beginning code generation

18/01/15 18:33:54 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: Access denied for user 'root'@'spark220' (using password: YES)

java.sql.SQLException: Access denied for user 'root'@'spark220' (using password: YES)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)

at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)

at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)

at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:873)

at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1710)

at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)

at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2188)

at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2219)

at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2014)

at com.mysql.jdbc.ConnectionImpl. (ConnectionImpl.java:776)

at com.mysql.jdbc.JDBC4Connection. (JDBC4Connection.java:47)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)

at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)

at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)

at java.sql.DriverManager.getConnection(DriverManager.java:664)

at java.sql.DriverManager.getConnection(DriverManager.java:247)

at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)

at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)

at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)

at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)

at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)

at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)

at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1846)

at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1646)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)

at org.apache.sqoop.Sqoop.run(Sqoop.java:143)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

18/01/15 18:33:54 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter

at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1652)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)

at org.apache.sqoop.Sqoop.run(Sqoop.java:143)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)

at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

"How to omit entering MySQL save password when executing Job" is introduced here. Thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.