Shulou(Shulou.com)06/01 Report--

How to use apache pig to deal with the data collected by chukwa, many novices are not very clear about this. In order to help you solve this problem, the following editor will explain it in detail. People with this need can come and learn. I hope you can get something.

The company uses chukwa to collect data from each server, and the hdfs collected from the data is later written as sequencefile, and wants to use pig to deal with the data. Pig itself does not support this data structure. Looking at the chukwa source code, it is found that there is a script with pig, which is used by hicc to do data statistics. Hicc has been difficult to use and doesn't bother to pay attention to this thing.

1 、 chukwa/script/pig/DF.pig

Define chukwaLoader org.apache.hadoop.chukwa.pig.ChukwaLoader ()

The source code is under: chukwa/contrib/chukwa-pig

The version of pig supported by chkwa is relatively old, use the pig script to load the jar package, the execution is not good at all, put the code under pig to compile, you can use it directly.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.