Shulou(Shulou.com)06/02 Report--

I. Common ports

Component node default port configuration purpose description HDFSDataNode50010dfs.datanode.addressdatanode service port, port for data transmission HDFSDataNode50075dfs.datanode.http.addresshttp service port HDFSDataNode50020dfs.datanode.ipc.addressipc service port HDFSNameNode50070dfs.namenode.http-addresshttp service port HDFSNameNode50470dfs.namenode.https-addresshttps service port HDFSNameNode8020fs.defaultFS receives the Client connection RPC port, which is used to obtain file system metadata information. HDFSjournalnode8485dfs.journalnode.rpc-addressRPC Service HDFSjournalnode8480dfs.journalnode.http-addressHTTP Service HDFSZKFC8019dfs.ha.zkfc.portZooKeeper FailoverController Applications manager (ASM) port YARNResourceManager8030yarn.resourcemanager.scheduler.addressscheduler component for NN HAYARNResourceManager8032yarn.resourcemanager.addressRM IPC port YARNResourceManager8031yarn.resourcemanager.resource-tracker.addressIPCYARNResourceManager8033yarn.resourcemanager.admin.addressIPCYARNResourceManager8088yarn.resourcemanager.webapp.addresshttp service port YARNNodeManager8040yarn.nodemanager.localizer.addresslocalizer IPCYARNNodeManager8042yarn.nodemanager.webapp.addresshttp service port container manager port in YARNNodeManager8041yarn.nodemanager.addressNM YARNJobHistory Server10020mapreduce.jobhistory.addressIPCYARNJobHistory Server19888mapreduce.jobhistory.webapp.addresshttp service port HBaseMaster60000hbase.master.portIPCHBaseMaster60010hbase.master.info.porthttp service port HBaseRegionServer60020hbase.regionserver.portIPCHBaseRegionServer60030hbase .regionserver.info.porthttp service port HBaseHQuorumPeer2181hbase.zookeeper.property.clientPortHBase-managed ZK mode Using a separate ZooKeeper cluster does not enable the port. HBaseHQuorumPeer2888hbase.zookeeper.peerportHBase-managed ZK mode, using a separate ZooKeeper cluster will not enable this port. HBaseHQuorumPeer3888hbase.zookeeper.leaderportHBase-managed ZK mode, using a separate ZooKeeper cluster will not enable this port. Export PORT= in HiveMetastore9083/etc/default/hive-metastore to update the default port

Export HIVE_SERVER2_THRIFT_PORT= in HiveHiveServer10000/etc/hive/conf/hive-env.sh to update the default port

In ZooKeeperServer2181/etc/zookeeper/conf/zoo.cfg, the port where clientPort= provides services to clients is server.x= [hostname]: nnnnn [: nnnnn] in ZooKeeperServer2888/etc/zookeeper/conf/zoo.cfg. The blue part of follower is used to connect to leader, and only listen on this port on leader. Server.x= [hostname]: nnnnn [: nnnnn] in ZooKeeperServer3888/etc/zookeeper/conf/zoo.cfg, the blue part is used for leader election. Only required if the electionAlg is 1 and 2 or 3 (the default).

All port protocols are based on TCP.

For more information on some ports, please see: http://blog.csdn.net/xiaolang85/article/details/8647602

2. Viewing method

For NameNode and DataNode, you can view their information through the following Web page:

I. NameNode: http://{NameNodeServer}:50070/

/ dfshealth.jsp HDFS information page with links to view the file system

/ dfsnodelist.jsp?whatNodes= (DEAD | LIVE) datanode that displays the status of DEAD or LIVE

/ fsck runs the fsck command, which is not recommended when the cluster is busy!

II. DataNode: http://{DataNodeServer}:50075/

/ blockScannerReport each datanode specifies an interval to verify block information

iii. For both NameNode and DataNode, the following Web URL exists:

/ logs log file list for downloading and viewing

/ logLevel allows you to set the logging level of log4j, similar to hadoop daemonlog

/ stacks the stack trace of all threads, which is very helpful for debug

/ Metrics of the jmx server, output in JSON format.

/ jmx?qry=Hadoop:* returns all hadoop-related metrics.

The / jmx?get=MXBeanName::AttributeName query specifies the value of the property specified by bean, for example, / jmx?get=Hadoop:service=NameNode,name=NameNodeInfo::ClusterId returns ClusterId. The processing class for this request: org.apache.hadoop.jmx.JMXJsonServlet

3. Configure Eclipse

1. In the Define Hadoop Location interface, for pseudo-distributed deployments with default configuration:

Map/Reduce (V2) Master: localhost:50020

DFS Master: localhost:8020

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.