Shulou(Shulou.com)06/01 Report--

This article is about how to install pig. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

View the pig version http://www.aboutyun.com/blog-61-62.html corresponding to Hadoop

We are asked to use start-dfs.sh and start-yarn.sh to start Hadoop

Add the following red information to the / home/hadoop/.bashrc file

# set java environment

PIG_HOME=/home/hadoop/pig-0.9.2

HBASE_HOME=/home/hadoop/hbase-0.94.3

HIVE_HOME=/home/hadoop/hive-0.9.0

HADOOP_HOME=/home/hadoop/hadoop-1.1.1

JAVA_HOME=/home/hadoop/jdk1.7.0

PATH=$JAVA_HOME/bin:$PIG_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$HADOOP_HOME/bin:$PATH

CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$HBASE_HOME/lib:$PIG_HOME/lib:$HIVE_HOME/lib:$JAVA_HOME/lib/tools.jar

Export PIG_HOME

Export HBASE_HOME

Export HADOOP_HOME

Export JAVA_HOME

Export HIVE_HOME

Export PATH

Export CLASSPATH

Restart the machine or use the source command to make the file effective.

Switch to the .bashrc file path and then

Source .bashrc

If you execute the pig command permission denied, chmod + x pig makes it an executable file

Run the Pig program example

To run a simple example, the first column of the / etc/passwd file under linux is extracted and output, and run in MapReduce mode. The effect is to enter all user names.

First put the / etc/passwd file to the hdfs of hadoop, with the following command

Hadoop fs-put / etc/passwd / user/root/passwd

Then go to Pig shell, run the command, extract A separated by':', and then put the first column of An into BMagne dump and type out B.

[root@hadoop-namenodenew] # piggrunt > A = load 'passwd' using PigStorage (':'); grunt > B = foreach A generate $0 as id; grunt > dump B

The output is as follows:

Input (s): Successfully read 29 records (1748 bytes) from: "hdfs://192.168.12.67:8020/user/root/passwd" Output (s): Successfully stored 29 records (325 bytes) in: "hdfs://192.168.12.67:8020/tmp/temp1558767875/tmp-1327634226" Counters:Total records written: 29Total bytes written: 325Spillable Memory Manager spill count: 0Total bags proactively spilled: 0Total records proactively spilled: 0Job DAG:job_1401631066126_ 0005 (omit tens of thousands of words....) (root) (bin) (daemon) (adm) (lp) (sync) (shutdown) (halt) (mail) (uucp) (games) (gopher) (ftp) (nobody) (dbus) (vcsa) (rpc) (abrt) (rpcuser) (nfsnobody) (haldaemon) (ntp) (saslauth) (postfix) (sshd) (tcpdump) (oprofile) (riak) Thank you for reading! This is the end of the article on "how to install pig". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.