Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to convert UTF8 codes into GB2312 codes in batches under Linux". In daily operation, I believe many people have doubts about how to convert UTF8 codes into GB2312 codes in batches under Linux. The editor consulted all kinds of data and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "how to convert UTF8 codes into GB2312 codes in batches under Linux". Next, please follow the editor to study!

Background

I use oracle's sqlplus to import UTF8-encoded sql scripts in batches, because I don't know how to set up to let sqlplus recognize the UTF8 format, resulting in garbled, wrong lines and other errors, so that the work can not continue, in the case of google without results, I have to find a way to convert the code.

Due to the large number of files, manual conversion is too troublesome, so I thought of using scripts to convert in batches. Fortunately, there are many related scripts on the Internet, and the only trouble to achieve is the BOM tag of UTF8.

Content:

The code is as follows:

#! / bin/bash

For loop in `find. -type f-name "* .sql"-print`do

Echo $loop

Mv-f $loop $loop.tmp

Dos2unix $loop.tmp

File_check_utf8='file_check_utf8.log'

Sed-n '1l' $loop.tmp "$file_check_utf810. If grep'^\ 357\ 273\ 277'$file_check_utf8 "/ dev/null 2" & 111. Then

Echo 'UTF-8 BOM'

Sed-n-e '1s/ ^. / /'- e'w intermediate.txt' $loop.tmp14. Iconv-f UTF-8-t GB2312-o $loop intermediate.txt15. Rm-rf intermediate.txt

Rm-rf $loop.tmp

Elif iconv-f UTF-8-t GB2312 $loop.tmp "/ dev/null 2" & 118. Then

Echo 'UTF-8'

Iconv-f UTF-8-t GB2312-o $loop $loop.tmp21. Rm-rf $loop.tmp

Else

Echo 'ANSI'

Mv-f $loop.tmp $loop

Fi

Rm-rf $file_check_utf8

# simulate unix2dos, which requires that the last line of the text file must have a newline character 28. Sed-n-e's _ Mv-f $loop.tmp $loop

Done

#! / bin/bash

For loop in `find. -type f-name "* .sql"-print`do

Echo $loop

Mv-f $loop $loop.tmp

Dos2unix $loop.tmp

File_check_utf8='file_check_utf8.log'

Sed-n '1l' $loop.tmp "$file_check_utf810. If grep'^\ 357\ 273\ 277'$file_check_utf8 "/ dev/null 2" & 111. Then

Echo 'UTF-8 BOM'

Sed-n-e '1s/ ^. / /'- e'w intermediate.txt' $loop.tmp14. Iconv-f UTF-8-t GB2312-o $loop intermediate.txt15. Rm-rf intermediate.txt

Rm-rf $loop.tmp

Elif iconv-f UTF-8-t GB2312 $loop.tmp "/ dev/null 2" & 118. Then

Echo 'UTF-8'

Iconv-f UTF-8-t GB2312-o $loop $loop.tmp21. Rm-rf $loop.tmp

Else

Echo 'ANSI'

Mv-f $loop.tmp $loop

Fi

Rm-rf $file_check_utf8

# simulate unix2dos, which requires that the last line of the text file must have a newline character 28. Sed-n-e's _ Mv-f $loop.tmp $loop

Done

explain

1. I did not find a good way to deal with the BOM of UTF8. Finally, I judged with sed+grep that if the first three bytes are\\ 357\\ 273\\ 277, then the file must be UTF8. Remove these three bytes with sed and then convert them.

two。 In order to avoid repetition or omission, the script uses iconv to convert a file without BOM. The successful conversion indicates that the file is UTF8, otherwise it means ANSI or GB2312.

3. With regard to the final sed command, it is because there is no unix2dos command on my system, so it is simulated in order to facilitate your viewing and editing under windows.

At this point, the study on "how to convert UTF8 codes into GB2312 codes in batches under Linux" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.