Shulou(Shulou.com)06/01 Report--

Sqoop import--hive-import-- hive-overwrite-- connect jdbc:oracle:thin:@192.168.92.136:1521:cyporcl-- username ODS-- password 'od154DScards! ('- M1-- hive-database ODS-- table Q_TRA_DISPUTESTATUS-- fields-terminated-by'\ 001'-- hive-drop-import-delims-- null-string'\ N'-null-non-string'\\ N'-map-column-java DISPUTERESULT=String

The DISPUTERESULT field is of type Oracleclob and contains carriage returns, which results in an increase in the number of records when loaded into hive.

-- both map-column-java and-- map-column-hive parameters need to be displayed; you also need to set-- hive-drop-import-delims so that the data can be successfully exported.

. How to deal with newline characters in imported columns

I thought everything would be all right, but I never thought that the imported column had a nclob type, which stored articles and was bound to have line breaks. Sure enough, the hive query data is all out of order, and the query data is found-- hive-delims-replacement (replace / n-direction-r and other symbols with the characters you specify) and-- hive-drop-import-delims parameters. Can be added to find that there is no egg, no effect. Baidu goole for a long time, finally found a post, http://stackoverflow.com/questions/28076200/hive-drop-import-delims-not-removing-newline-while-using-hcatalog-in-sqoop

It means to use the-- map-column-java parameter to display the specified column String type. My solution is-- map-column-java my CLOB column = String sure enough, the problem is solved, and the newline characters are all removed. However, I have heard that changing the storage format may also avoid this question. I hope an experienced god can answer it. That's all to sum up. I hope I can help you.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.