Shulou(Shulou.com)05/31 Report--

This article is about what Pig is on the Internet. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Pig is a large-scale data analysis platform based on Hadoop. The SQL-LIKE language it provides is called Pig Latin. The compiler of this language converts SQL-like data analysis requests into a series of optimized MapReduce operations. Pig provides a simple operation and programming interface for complex massive data parallel computing.

Pig features:

1. Focus on mass data analysis (ad-hoc analysis,ad-hoc representative: a solution that has been custom designed for a specific problem).

2. Running on the computing architecture of the cluster, Yahoo Pig provides multiple levels of abstraction to simplify parallel computing for ordinary users; these abstractions automatically translate user requests for queries into effective parallel evaluation plans, and then execute these plans on the physical cluster

3. Provide operation syntax similar to SQL

4. Open source

About Pig and Hive:

For developers, using Java APIS directly can be tedious or error-prone, and it also limits the flexibility of Java programmers to program on Hadoop. So Hadoop provides two solutions that make Hadoop programming easier.

Pig is a programming language that simplifies the common tasks of Hadoop. Pig can load data, express transformed data, and store the final results. Pig's built-in operations make semi-structured data meaningful (such as log files), while Pig can be extended to use custom data types added in Java and support data conversion.

Hive acts as a data warehouse in Hadoop. Hive adds data structures in HDFS (hive superimposes structure on data in HDFS) and allows data queries similar to SQL syntax. Like Pig, the core functions of Hive are extensible.

Pig and Hive are always confusing. Hive is better suited for data warehouse tasks, and Hive is mainly used for static structures and work that requires frequent analysis. The acquaintance of Hive and SQL makes it an ideal intersection of Hadoop and other BI tools. Pig rich developers have more flexibility in the big data collection area and allow concise scripts to transform data streams for embedding into larger applications. Pig is relatively lightweight compared to Hive, and its main advantage is that it can significantly reduce the amount of code compared to the direct use of Hadoop Java Apis. Because of this, Pig still attracts a large number of software developers.

Thank you for reading! This is the end of this article on "what is Pig in the Internet?". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.