Summary of frequently asked questions for HIVE operation 01/19 Update SLTechnology News&Howtos

Summary of frequently asked questions for HIVE operation

2025-01-19 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Shulou(Shulou.com)06/01 Report--

This article mainly explains "the summary of frequently asked questions about HIVE operation". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Now let the editor to take you to learn the "summary of frequently asked questions about HIVE operation"!

1 、 Terminal initialization failed; falling back to unsupported

[ERROR] Terminal initialization failed; falling back to unsupported

Java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

At jline.TerminalFactory.create (TerminalFactory.java:101)

At jline.TerminalFactory.get (TerminalFactory.java:158)

At jline.console.ConsoleReader. (ConsoleReader.java:229)

At jline.console.ConsoleReader. (ConsoleReader.java:221)

At jline.console.ConsoleReader. (ConsoleReader.java:209)

At org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader (CliDriver.java:787)

At org.apache.hadoop.hive.cli.CliDriver.executeDriver (CliDriver.java:721)

At org.apache.hadoop.hive.cli.CliDriver.run (CliDriver.java:681)

At org.apache.hadoop.hive.cli.CliDriver.main (CliDriver.java:621)

At sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethod)

At sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57)

At sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

At java.lang.reflect.Method.invoke (Method.java:606)

At org.apache.hadoop.util.RunJar.run (RunJar.java:221)

At org.apache.hadoop.util.RunJar.main (RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

At jline.console.ConsoleReader. (ConsoleReader.java:230)

At jline.console.ConsoleReader. (ConsoleReader.java:221)

At jline.console.ConsoleReader. (ConsoleReader.java:209)

At org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader (CliDriver.java:787)

At org.apache.hadoop.hive.cli.CliDriver.executeDriver (CliDriver.java:721)

At org.apache.hadoop.hive.cli.CliDriver.run (CliDriver.java:681)

At org.apache.hadoop.hive.cli.CliDriver.main (CliDriver.java:621)

At sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethod)

At sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57)

At sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

At java.lang.reflect.Method.invoke (Method.java:606)

At org.apache.hadoop.util.RunJar.run (RunJar.java:221)

At org.apache.hadoop.util.RunJar.main (RunJar.java:136)

Solution:

The main reason for this problem is that the lib of yarn in Hadoop contains a jar package of jline that is inconsistent with the version of jline in hive's lib directory. You only need to synchronize the jline package in hive, delete the jline package in yarn's lib, and then copy it.

For example: cp / u01/apache-hive-1.2.1-bin/lib/jline-2.12.jar / u01/hadoop-2.6.1/share/hadoop/yarn/lib

2. Hive uses derby as the reason why the created table cannot be found in the Metabase

Many beginners install hive using the default Metabase derby, which is prone to the following problems:

After installing the hive environment, close the connection window after building the table, and then re-enter the table and disappear. What is wrong with the configuration?

1. Create a table after typing hive on the command line

two。 Display a list of tables in show tables;, showing the table you just created

3. Close the window, reopen a window to connect to the server, type hive, and then show table, the table can't be seen.

Why the created table can not be found, this does not disappear, but you are in the wrong position.

a. If we enter hive under home, then our metastore_db will be generated in the home directory.

b. If we enter hive in the / usr directory, then our metastore_db will be generated in the usr directory.

So if you change the directory to start hive, you can't find the table you created before.

What is a different path to enter hive, for example

Experiment 1: create a table under the directory, view the table, and do not switch directory operations

Check the current directory, here is the / home/bdata/data directory

[bdata@bdata4 data] $pwd

/ home/bdata/data

To view the content, there are only IY02_C.txt files.

[bdata@bdata4 data] $ls

IY02_C.txt

Execute the hive command to enter the client

[bdata@bdata4 data] $hive

Logging initialized using configuration in JarVel fileVlue, U01 Universe, ApacheMurhiveMou1.2.1, LIBUBINGUBINGUBUBING, CommonMutual, 1.2.1.jarbank bank, hiveMusi log4j.properties

Hive > show tables

Time taken: 0.64 seconds

# create a table structure and split it according to |

Hive > CREATE TABLE YHJHK_IY02 (AAB301 String,AAE043 String,AAB001 String,AAC001 String,AAC003 String,AAC004 String,AAC002 String,AAC005 String,AAC006 String,AAC009 String,AAC012 String,AAC028 String,AAC064 String,AAC087 String,AAC007 String,AAC032 String,AIC001 int, AIC090 int, AAC031 String,AAE160 String,AIC104 int, AIC164 int, AIC020 int, AAC095 int, AAC096 int, AAC097 int, AAC098 int, AAC099 int, AAC103 int, AAC104 int, AIC268 int, AIC115 int, AIC116 int, AIC117 int, AAB000 int, SUBJOB_ID int, VZE105 String VZE003 String) ROW FORMAT DELIMITED FIELDS TERMINATED BY'| 'STORED AS TEXTFILE

# load the files in HDFS, where / data/IY02_C.txt is stored in the hdfs file system in advance

Hive > load data inpath'/ data/IY02_C.txt' into table YHJHK_IY02

# check the table again to make sure that the table yhjhk_iy02 exists

Hive > show tables

Yhjhk_iy02

Time taken: 0.686 seconds, Fetched: 1 row (s)

# exit

Hive > quit

If you look at the directory again, you will have two more files, metastore_db and derby.log.

[bdata@bdata4 data] $ls

Derby.log IY02_C.txt metastore_db

Experiment 2: switch directories, enter hive, and view table information

Go back to the home directory

[bdata@bdata4 data] $cd

Check the current path to confirm that the directory has been switched

[bdata@bdata4 ~] $pwd

/ home/bdata

Check the current directory and confirm that it is a new directory. There is no metastore_db or derby.log, only one data directory.

[bdata@bdata4 ~] $ls

Data

Enter the hive client

[bdata@bdata4 ~] $hive

Logging initialized using configuration in JarVel fileVlue, U01 Universe, ApacheMurhiveMou1.2.1, LIBUBINGUBINGUBUBING, CommonMutual, 1.2.1.jarbank bank, hiveMusi log4j.properties

# check the table information. No table information is found here.

Hive > show tables

Time taken: 0.64 seconds

Hive > quit

Look at the current directory and the metastore_db and derby.log files have been generated

[bdata@bdata4 ~] $ls

Data metastore_db derby.log

It is determined in the above way that when we are using derby, if the data is not queried when we suddenly execute the hive query table, then, do not worry, the table data does not disappear, you only need to switch to the directory where the hive table was created before to perform the client operation, or switch to MYSQL in advance to store the original information.

3. Execute the table creation command and report an error: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException (message:For direct MetaStore DB connections, we don't support retries at the client level.)

This is because my mysql is no longer local (using a local database by default), so you need to configure a remote metadata server and then start the metadata storage service on the hive server. Metastore is used to communicate with mysql when the table structure is created or updated.

First configure the configuration of metastore in the configuration file, as follows

Hive.metastore.uris

Thrift://192.168.10.34:9083

Jdbc/odbc connection hive,if mysql must set

Once configured, start the communication service:

[bdata@bdata4 bin] $. / hive-- service metastore-hiveconf hbase.zookeeper.quorum=bdata1,bdata2,bdata3-hiveconf hbase.zookeeper.property.clientPort=2181 &

4. Execute the HIVE command and report an error: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

At org.apache.hadoop.hive.ql.session.SessionState.start (SessionState.java:522)

At org.apache.hadoop.hive.cli.CliDriver.run (CliDriver.java:677)

At org.apache.hadoop.hive.cli.CliDriver.main (CliDriver.java:621)

At sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethod)

At sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:57)

At sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)

At java.lang.reflect.Method.invoke (Method.java:606)

At org.apache.hadoop.util.RunJar.run (RunJar.java:221)

At org.apache.hadoop.util.RunJar.main (RunJar.java:136)

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

At org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance (MetaStoreUtils.java:1523)

At org.apache.hadoop.hive.metastore.RetryingMetaStoreClient. (RetryingMetaStoreClient.java:86)

At org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy (RetryingMetaStoreClient.java:132)

At org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy (RetryingMetaStoreClient.java:104)

At org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient (Hive.java:3005)

At org.apache.hadoop.hive.ql.metadata.Hive.getMSC (Hive.java:3024)

At org.apache.hadoop.hive.ql.session.SessionState.start (SessionState.java:503)

... 8 more

Caused by: java.lang.reflect.InvocationTargetException

At sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)

At sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:57)

At sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)

At java.lang.reflect.Constructor.newInstance (Constructor.java:526)

At org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance (MetaStoreUtils.java:1521)

... 14 more

Caused by: MetaException (message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

At org.apache.thrift.transport.TSocket.open (TSocket.java:187)

At org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open (HiveMetaStoreClient.java:420)

At org.apache.hadoop.hive.metastore.HiveMetaStoreClient. (HiveMetaStoreClient.java:236)

At org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient. (SessionHiveMetaStoreClient.java:74)