Shulou(Shulou.com)05/31 Report--

This article mainly introduces what the HDFS architecture has, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

The architecture of HDFS, as shown in the figure above, adopts master/slave architecture as a whole, which is mainly composed of Client, NameNode, DataNode and SecondaryNameNode.

1 、 Client

Client represents the user and accesses files in HDFS by interacting with NameNode and DataNode. Client provides a POSIX-like file system interface for users to call.

2 、 NameNode

There is only one NameNode in the entire Hadoop cluster. Responsible for managing the metadata information of HDFS directory tree and related files.

NameNode is also responsible for monitoring the health of each DataNode, moving the DataNode out of the HDFS and re-backing up the data on it once a data Node is found to be down.

3 、 Secondary NameNode

The most important task of Secondary NameNode is not to make a hot backup of NameNode metadata, but to merge fsimage and edits logs periodically and transfer them to NameNode. It is important to note that in order to reduce the pressure on NameNode, NameNode itself does not merge fsimage and edits and save the files to disk, but to make friends with Secondary Namenode.

4 、 DataNode

A DataNode is installed on each Slave node, which is responsible for the actual data storage and reports the data information to NameNode on a regular basis.

When a user uploads a file to HDFS, the file is split into several block and stored in different DataNode. To ensure that the data is reliable, the same block is pipelined to several different DataNode.

Thank you for reading this article carefully. I hope the article "what is the structure of HDFS" shared by the editor will be helpful to everyone? at the same time, I also hope that you will support and pay attention to the industry information channel, and more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.