Shulou(Shulou.com)06/01 Report--

This article mainly explains the "use of hadoop", the content of the explanation is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn "the use of hadoop" bar!

Introduction to the framework

Hadoop uses the master / slave (Master/Slave) architecture, and the main role is made up of NameNode,DataNode,secondary NameNode,JobTracker,TaskTracker.

NameNode,secondary NameNode,JobTracker runs on the Master node, and DataNode and TaskTracker run on the Slave node.

1,NameNode

NameNode is the HDFS daemon that records how the file is divided into blocks and on which data nodes those blocks are stored. Its function is to centralize the management of memory and Iswap O.

2,DataNode

Each slave server in the cluster runs a DataNode daemon, which is responsible for reading and writing HDFS blocks to the local file system. When you need to read and write data, NameNode tells the client which DataNode to go to for specific read and write operations.

3,Secondary NameNode

Secondary NameNode is a secondary daemon used to monitor the status of HDFS. If there is a problem with NameNode, you can use Secondary NameNode as a backup NameNode.

4,JobTracker

The JobTracker daemon is used to connect the application with the Hadoop. After the user application is submitted to the cluster, it is up to JobTracker to decide which file processes which task execution. If a task fails, JobTracker will automatically open the task.

5,TaskTracker

TaskTracker is responsible for the combination of DataNode for storing data, located in the slave node, and is responsible for their respective task.

Using hadoop under windows

Start SSH service with Open SSH under 1.Cygwin

two。 Run command

Cd. / bin./hadoop namenode-format./start-all.sh# started successfully after the test. / hadoop fs-mkdir In./hadoop dfs-ls / Thank you for reading, the above is the content of "the use of hadoop", after the study of this article, I believe you have a deeper understanding of the use of hadoop, the specific use of the situation also needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.