Shulou(Shulou.com)05/31 Report--

This article mainly explains "what are the knowledge points related to the hadoop security model". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what are the knowledge points related to the hadoop security model".

1. What actions does the namenode node do when namenode starts? What are the changes in fsimage and edits? 2. When does namenode start listening for RPC and HTTP requests?

When namenode starts, it first loads the image file (fsimage) into memory and performs the operations in the editing log (edits).

Once you have successfully created an image of the file system metadata in memory, create a new fsimage file (which does not require a secondary namenode) and an empty edit log. At this point, namnode starts listening for RPC and HTTP requests.

3. At what stage will namenode be in safe mode?

In the start-up phase.

4. Namenode runs in safe mode. Can the file system be read and written to the client?

But at the moment, namenode is running in safe mode, meaning that the file system of namenode is read-only to the client.

Strictly speaking, in safe mode, only file system operations that access file system metadata can be successful, such as displaying a list of directories.

For file read operations, files can be read only if blocks on the current datanode in the cluster are available. However, file modification operations, including write, delete, or rename, fail.

5. Who maintains the location of the data blocks in the system?

It is important to emphasize that the location of blocks in the system is not maintained by namenode, but is stored in datanode as a list of blocks.

6. Where does namenode keep the mapping information for all block locations?

During normal operation of the system, namenode retains mapping information for all block locations in memory.

7. What actions have been done by namenode and datanode in safe mode?

1), datanode checks the block list information with namenode.

2) namenode does not issue any block copy or block delete commands to datanode. That is, the copy or deletion of the block failed.

The detailed process is as follows:

1) in safe mode, each datanode checks the block list information to the namenode (that is, the latest status of the block list is sent to namenode)

2) after namenode knows enough block location information, it can run the file system efficiently.

3), but if namenode does not check enough datanode, you need to copy the block to another datanode, which in most cases is unnecessary (because you only need to wait for a number of datanode check-ins to be checked in) and is a huge waste of cluster resources.

4) in fact, in safe mode, namenode does not issue any block copy or block deletion commands to datanode.

8. When do I exit safe mode?

When the minimum copy condition (minimal replication condition) is met, namenode exits safe mode after 30 seconds.

9. What is the minimum copy condition?

The minimum replica condition is that 99.9% of the blocks in the entire file system meet the minimum replica level (the default is 1, which is set by the dfs.replication.min attribute).

10. Safety mode related attributes 1), dfs.replication.min2), dfs.safemode.threshold.pct3), dfs.safemode.extension

The table is as follows:

11. Will namenode enter safe mode when starting a newly formatted HDFS cluster? Why?

Because there are no blocks in the system, namenode does not enter safe mode.

12. How do I enter and leave safe mode? 1) # check whether namenode is in safe mode?

Hadoop dfsadmin-safemode get

The web interface of HDFS can also show whether namenode is in safe mode.

2) # wait for namenode to exit security mode

What happens when a user expects namenode to exit safe mode before executing a command?

Sometimes users expect namenode to exit safe mode before executing a command, especially in scripts. You can do this by using the wait option:

Hadoop dfsadmin-safemode wait

3) # enter safe mode

The administrator can allow namenode to enter or leave safe mode at any time. This feature is critical when maintaining and upgrading clusters because you need to ensure that the data is read-only for a specified period of time.

Hadoop dfsadmin-safemode enter

4) # leave safe mode

Hadoop dfsadmin-safemode leave

13. How to keep namenode in safe mode forever? 1), namenode startup phase, run hadoop dfsadmin-safemode enter

Namenode will be in safe mode during the startup phase. In the meantime, you can also use this command to ensure that namenode does not leave safe mode after startup.

2), set the dfs.safemode.threshold.pct value to be greater than 1

Another way to keep namenode in safe mode forever is to set the value of the property dfs.safemode.threshold.pct to greater than 1.

Thank you for your reading. The above is the content of "what are the knowledge points related to hadoop security mode". After the study of this article, I believe you have a deeper understanding of what are the knowledge points related to hadoop security mode, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.