Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the pig operation method", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let Xiaobian take you to learn "what is the pig operation method"!

Pig is a program that runs as a client and you need to connect it to local Hadoop or to a cluster. When Pig is installed, there are three ways to execute a pig program: pig scripting (writing the program to a.pig file), Grunt (an interactive shell environment that runs Pig commands), and embedded mode.

records = Load 'sample.txt' as (year:chararray, temperature:int, quality:int);

filter_records = FILTER records BY temperature != 9999 AND quality == 0;

group_records = GROUP filter_records BY year;

max_temp = FOREACH group_records GENERATE group, MAX(filter_records.temperature);

DUMP max_temp;

Generate the data set structure created by the above program: grunt> ILLUSTRATE max_temp;

Comparison of Pig and Database:

Pig is a data flow programming language, while SQL is a descriptive programming language. A Pig is a step-by-step operation relative to the input, where each step is a simple transformation on the data, while an SQL statement is a collection of constraints that combine to define the output. Pig is more like a query planner in an RDBMS.

2) RDBMS stores data in tables with strictly defined schemas, but pig is more relaxed about data, and schemas can be defined at runtime, and it is optional.

3) pig has stronger support for complex, nested data structures;

4) Pig does not support transactions and indexes, nor does it support random reads and queries at the level of tens of milliseconds. It is for batch processing of data.

Hive is a system between Pig and RDBMS. Hive uses HDFS for storage, but the query language is SQL-based, and Hive requires that all data must be stored in tables,

Tables must have schemas, and schemas are managed by Hive. But Hive allows a pattern to be associated with data pre-stored in HDFS, so the data loading step is optional.

At this point, I believe that everyone has a deeper understanding of "what is the pig operation method", so let's actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.