Shulou(Shulou.com)06/03 Report--

1.NameNode:

Equivalent to a leader, responsible for scheduling, for example, if you need to save a 1280m file, if it is divided into 128m blocks, then namenode will allocate these 10 blocks (regardless of the copy here) to the datanode in the cluster and record the relationship. When you download this file, namenode will know which nodes to fetch the data for you. It mainly maintains two map, one is the correspondence from file to block and the other is from block to node.

2. Secondarynamenode:

It is a snapshot of namenode, which determines how long it takes to cp-namenode periodically according to the values set in configuration, and records metadata and other data in namenode.

3. NodeManager (NM):

1. An agent on each node in YARN, which manages a single compute node in a Hadoop cluster

2. Including maintaining communication with ResourceManger and supervising the life cycle management of Container

3. Monitor the resource usage (memory, CPU, etc.) of each Container and track the node health.

4. Health status, management logs and ancillary services used by different applications (auxiliary service)

4.DataNode:

The first task that a.DataNode needs to complete is Kmuri V storage.

b. Complete communication with namenode, which is achieved through IPC heartbeat connection, in addition to exchanging information with other datanode on the client side.

c. To complete large-scale communication with the client and other nodes, this needs to be achieved directly through the socket protocol.

5.ResourceManager:

In YARN, ResourceManager is responsible for the unified management and allocation of all resources in the cluster. It receives resource report information from each node (NodeManager) and assigns this information to each application (actually ApplicationManager) according to a certain strategy. RM works with NodeManagers (NMs) of each node and ApplicationMasters (AMs) of each application.

A.NodeManagers follows instructions from ResourceManager to manage available resources on a single node.

B.ApplicationMasters is responsible for negotiating resources with ResourceManager and working with NodeManagers to launch the container.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.