Shulou(Shulou.com)06/01 Report--

This article mainly introduces "which two parts Hadoop contains". In daily operation, I believe many people have doubts about which two parts Hadoop contains. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "which two parts Hadoop contains"! Next, please follow the editor to study!

Hadoop consists of two parts:

1.Hadoop Distributed File System (Hadoop distributed file system)

HDFS has high fault tolerance and can be deployed on low-cost hardware devices. HDFS is suitable for applications with large data sets and provides high throughput for reading and writing data. HDFS is the structure of a master/slave, and for a typical deployment, only one Namenode is run on master and one Datanode is run on each slave.

HDFS supports the traditional hierarchical file organization structure, which is similar to some existing file systems in operation, such as you can create and delete a file, move a file from one directory to another, rename and so on. Namenode manages the entire distributed file system, and operations on the file system (such as creating and deleting files and folders) are controlled by Namenode.

2. The realization of MapReduce

MapReduce is an important technology of Google. It is a programming model for computing large amounts of data. For the calculation of a large amount of data, the usual processing method is parallel computing. At least at this stage, parallel computing is still a distant thing for many developers. MapReduce is a programming model that simplifies parallel computing. It allows developers who do not have much experience in parallel computing to develop parallel applications.

MapReduce is named after two core operations in this model: Map and Reduce. To put it simply, Map maps one set of data one to one to another, and the mapping rules are specified by a function, for example, the mapping of [1, 2, 3, 4] by 2 becomes [2, 4, 6, 8]. Reduce is a reduction of a set of data. The reduction rule is specified by a function. For example, the result of summation of [1, 2, 3, 4] is 10, while the reduction result of quadrature is 24.

At this point, the study of "which two parts Hadoop contains" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.