Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the Hadoop exercises of big data's processing architecture?" interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next, let the editor take you to learn "what are the Hadoop exercises for big data to deal with the architecture?"

1. On the relationship between hadoop and Google's mapreduce, gfs and other technologies

A:

The core of Hadoop is the distributed file system HDFS and MapReduce,HDFS are the open source implementation of Google file system GFS, and MapReduces is the open source implementation for Google MapReduce.

two。 Try to describe the characteristics of Hadoop.

A:

High reliability, high efficiency, high scalability, high fault tolerance, low cost, running on Linux platform, supporting multiple programming languages

3. Try to describe the application of Hadoop in various fields.

A: in 2007, Yahoo established M45muri, a Hadooop cluster system with 4000 processors and 1.5PB capacity, at its Sunnyvale headquarters.

Facebook mainly uses Hadoop platform for log processing, recommendation system and data warehouse.

Baidu mainly uses Hadoop in log storage and statistics, web data analysis and mining, business analysis, online data feedback, web page clustering and so on.

4. Try to describe the project structure of Hadoop and the specific functions of each part.

A:

Commeon is a common tool for supporting other Hadoop subprojects, including the file system, RPC, and serialization libraries.

Avro is a subproject of Hadoop and a system for data serialization. It provides rich data structure types, fast compressible binary data format, file set for storing persistent data, remote call function and simple dynamic language integration function.

HDFS, one of the two cores of the Hadoop project, is an open source implementation of Google's file system.

HBase is a column database with high reliability, high performance, scalability, real-time reading and writing, and distributed. Generally, HDFS is used as its underlying data storage.

MapReduce is an open source implementation for Google MapReduce, which is used for parallel computing of large data sets.

Zoookepper is an open source implementation of Google Chubby, which is an efficient and reliable cooperative work system. It provides basic services such as distributed locks, which are used to build distributed applications and reduce the coordination tasks undertaken by distributed applications.

Hive is a Hadoop-based data warehouse tool, which can be used for data collation, special query and distributed storage of data sets in Hadoop files.

Pig is a data flow language and running environment, which is suitable for querying large semi-structured data sets on Hadoop and MapReducce platforms.

Sqoop can improve the interoperability of data and is mainly used to exchange data between Hadoop and relational databases.

Chukwa is an open source data collection system for monitoring large distributed systems. Various types of data can be collected into files suitable for Hadoop processing and saved in HDFS for Hadoop to carry out various MapReduce operations.

At this point, I believe you have a deeper understanding of "what are the Hadoop exercises of big data's processing architecture?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.