Shulou(Shulou.com)06/01 Report--

This article shows you how to use AWK to organize KyLin log improvements, the content is concise and easy to understand, can definitely brighten your eyes, through the detailed introduction of this article, I hope you can get something.

On the basis of the previous article, some improvements have been added.

It is mainly the adjustment of the detail format.

It takes 7s to process 100m data.

Http://blog.itpub.net/29254281/viewspace-2156713/

Time awk'

Function ltrim (s) {sub (/ ^ [\ t\ r\ n] + /, "", s); return s}

Function rtrim (s) {sub (/ [\ t\ r\ n] + $/, ", s); return s}

Function trim (s) {return rtrim (ltrim (s));

/ ^ Query Id:/ {split ($0gamma rowdata, ":");}

/ ^ SQL:/ {s=trim ($0); next;}

/ ^ User:/ {print "19700101 000000000 INFO [Query" trim (rowdata [2]) "" s = ""; print $0;}

/ ^ = / {delete rowdata;}

{

If (rowdata [2]! = "")

If (slots = "") s = (s "" trim ($0))

Else

Print "19700101 000000000 INFO [Query" trim (rowdata [2]) "$0

Else gsub ("[,: -]", ", $1); gsub (" [,: -] ", $2); $5=substr ($5d0d0, 36); print ($0)

}

'kylinbig.log | grep "\ [Query" | nl-n ln | tr "\ t" | tr-s' | sort-k6 Using project:/ 6-k1n t1.txt 1 | cut-d''- f 2 Leng 3 Leng 4 6-| sed'/ Using project:/ {xttertx;}'> t1.txt

The above is the improvement of using AWK to organize KyLin logs. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

The above content is how to use AWK to organize KyLin log improvements, have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.