Shulou(Shulou.com)06/02 Report--

The main content of this article is to explain "how to solve the abnormal problem in Chinese when Fayson uses the impala-shell command to export data." interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Fayson uses the impala-shell command to export data there is a Chinese abnormal problem how to solve it"!

When Fayson uses impala-shell-I hosts-d default-f test.sql-o test.txt to export the result data, it finds that SQL reports an error and prompts the following Unknown Exception: 'ascii' codec can't encode characters in position 61-62: ordinal not in range (128). Seeing this exception, friends who must be familiar with python know that this is the problem of garbled codes in python. Fayson will describe the problem and solution in detail below.

Test environment:

1.RedHat7.4

2.CDH5.16.1

Problem recurrence

First we use impala-shell to create a table and insert a piece of data containing Chinese characters, and then look at it

[hadoop2.macro.com:21000] > create table testimpala_shell (S1 string,s2 string)

[hadoop2.macro.com:21000] > insert into testimpala_shell values ("hello", "Hello")

[hadoop2.macro.com:21000] > select * from testimpala_shell

Then execute the following command and check the log and find that the exception is as follows:

Impala-shell-I hadoop2.macro.com-d default-f test.sql-o test1.log

Or

Impala-shell-I hadoop2.macro.com-d default-f test.sql > test.log 2 > & 1

Or

Impala-shell-I hadoop2.macro.com-d default-Q "select * from testimpala_shell"-o test1.log

The same statement can execute and export the result data without Chinese characters.

Problem solving

Because there is no problem with the query in impala-shell, there is a problem when exporting the data, which is written by python on the client side of impala-shell, and Python cannot automatically write unicode objects to the output stream that does not have a default encoding set, resulting in this exception. For this problem, changing the default code of impala-shell to utf-8 can solve the problem.

Vim / opt/cloudera/parcels/CDH/lib/impala-shell/impala_shell.py

# add the following code:

Import sys

Reload (sys)

Sys.setdefaultencoding ('utf-8')

Then execute it again, and find that it can be executed normally.

At this point, I believe you have a deeper understanding of "how to solve the problem of Chinese anomalies when Fayson uses the impala-shell command to export data". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.