Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

What is the Hadoop architecture?

2025-01-16 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)05/31 Report--

Editor to share with you what the Hadoop architecture is, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to understand it!

one。 Namenode

1. Introduction

1) is the management node of the entire file system

2) it maintains the file directory tree of the entire file system, the meta-information of files / directories and the list of data blocks corresponding to each file.

3) receive the user's operation request

4) the meta-information of a file / directory refers to the name, size, modification time and other information of the file.

2. Hdfs file view

1) View the path: hdfs-site.xml 's dfs.replication attribute hadoop.tmp.dir

Hadoop.tmp.dir / usr/local/hadoop/tmp

View Fil

[root@chaoren hadoop] # cd / usr/local/hadoop/tmp/dfs/name/current [root@chaoren current] # lsedits fsimage fstime VERSION

2) fsimage: metadata image file. Store NameNode memory metadata information for a certain period of time

3) edits: operation log file

4) fstime: the time when the last checkpoint was saved

two。 Datanode

1. Provide real file data storage service.

two。 Block: the most basic unit of storage. For the content of a file, the length of a file is size, so starting from the zero offset of the file, the file is divided and numbered according to the fixed size and order, and each block is called a Block. HDFS default Block size is 64MB, to a 256MB file, a total of 256 Block 64 Block.

3. Unlike ordinary file systems, in HDFS, if a file is less than the size of a block, it does not occupy the entire block storage space.

4. Replication . Multiple copies. The default is three. View the dfs.replication properties of hdfs-site.xml

three。 SecondaryNameNode

1. A solution of HA. But hot backup is not supported. Just configure it.

two。 Execution process: download metadata information (fsimage,edits) from NameNode, then merge the two to generate a new fsimage, save it locally, and push it to NameNode, while resetting the edits of NameNode.

3. It is installed on the NameNode node by default, but this way. Unsafe

4. Other

These are all the contents of the article "what is the Hadoop Architecture?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report