Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what are the hadoop mechanisms?". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. A brief introduction of checkpoint Mechanism

Here is mainly used the role of SN, through the SN to copy the log files in the original NN and the original image files in SN combined, through this principle, SN is equivalent to a copy of NN, when the NN server encountered failure restart, you can use SN to restore data.

two。 How to restart Namenode separately

Dfs-sh.start

3. A brief introduction to the principle of HDFS data Reading

The client sends the request to the NN, and then the NN agrees to read the data, and then the NN uses the request information of the client to find the location of the data from the metadata in it (the block block storing the data to be read is in the three DN), and then returns the information to the client, and the client uses this information to get the data from the DN.

4. How to start the zookeeper cluster, how to determine the success of the zookeeper cluster, and how to stop the zookeeper cluster

ZkCli.sh,zkCli.sh status,zkCli.sh stop

5. A brief introduction to the principle of HA High availability Cluster of HDFS

At least two NN servers are needed. When one of the NN servers encounters a failure, hadoop will send a heartbeat to another server at short intervals to tell the other server if there is something wrong with the server it supervises. When it detects that the server is broken, it detects whether the server is broken. It will tell it to connect to another server in standby, and then standby's server will start.

About how the data on standby's server is consistent with that of the running server:

There is a data file supervision system between the two servers, which is managed by the zookeeper cluster. It ensures its data consistency and carries out context scheduling management through zookeeper. It is used to save the log files of the changes of the runing server, and then send them to the standby server. The standby server will form a server consistent with the running server according to these log files and its own metadata.

About cerebral fissure: sometimes for network reasons, the false inspection tool still sends a request for standby to start, so that there are two NN servers at the same time, and the data will be confused.

The treatment of cerebral fissure: 1. Kill-9 2. Use shell script

This is the end of the content of "what are the hadoop mechanisms?". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.