Shulou(Shulou.com)06/01 Report--

Editor to share with you where the pid file in HDFS is stored, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. The pid file is stored in the / tmp directory by default, and the content of the pid file is the process number

[hadoop@hadoop002 ~] $cd / tmp

[hadoop@hadoop002 tmp] $pwd

/ tmp

[hadoop@hadoop002 tmp] $ll

Total 132

Drwxrwxr-x. 4 hadoop hadoop 4096 May 22 12:46 hadoop-hadoop

-rw-rw-r--. 1 hadoop hadoop 5 May 23 11:00 hadoop-hadoop-datanode.pid

-rw-rw-r--. 1 hadoop hadoop 5 May 23 11:00 hadoop-hadoop-namenode.pid

-rw-rw-r--. 1 hadoop hadoop 5 May 23 11:00 hadoop-hadoop-secondarynamenode.pid

Drwxr-xr-x. 2 hadoop hadoop 4096 May 23 13:17 hsperfdata_hadoop

Drwxrwxr-x. 3 hadoop hadoop 4096 May 23 11:00 Jetty_0_0_0_0_50070_hdfs____w2cu08

Drwxrwxr-x. 4 hadoop hadoop 4096 May 23 11:05 Jetty_0_0_0_0_8042_node____19tj0x

Drwxrwxr-x. 3 hadoop hadoop 4096 May 23 11:01 Jetty_hadoop002_50090_secondary____.xoi5bj

Drwxrwxr-x. 4 hadoop hadoop 4096 May 23 11:05 Jetty_hadoop002_8088_cluster____jy43fd

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 12:43 Jetty_localhost_33164_datanode____.86mfre

Drwxrwxr-x. 3 hadoop hadoop 4096 May 21 16:31 Jetty_localhost_34452_datanode____57ccs7

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 13:28 Jetty_localhost_35801_datanode____1320ou

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 11:14 Jetty_localhost_41633_datanode____.reuty8

Drwxrwxr-x. 3 hadoop hadoop 4096 May 21 17:17 Jetty_localhost_43514_datanode____do382o

Drwxrwxr-x. 3 hadoop hadoop 4096 May 23 11:00 Jetty_localhost_54437_datanode____.slqxa8

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 12:24 Jetty_localhost_55741_datanode____.cj1ayt

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 12:18 Jetty_localhost_55901_datanode____.91pz3

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 13:36 Jetty_localhost_59457_datanode____.gif7tz

Drwxrwxr-x. 3 hadoop hadoop 4096 May 22 11:28 Jetty_localhost_59865_datanode____dj2ere

Drwx-. 2 root root 4096 May 23 10:55 keyring-dOEsWO

Drwx-. 2 root root 4096 Apr 21 02:20 keyring-fR5f6f

Drwx-. 2 root root 4096 Apr 21 02:13 keyring-jZCkKD

Drwx-. 2 root root 4096 Apr 20 20:07 keyring-RWPvsu

Drwx-. 2 root root 4096 May 21 16:18 keyring-Rythlk

Drwx-. 2 root root 4096 Apr 25 14:13 keyring-zrMZWC

Srwxrwxrwx. 1 mysql mysql 0 Apr 25 14:17 mysql.sock

Drwx-. 2 gdm gdm 4096 May 23 10:55 orbit-gdm

Drwx-. 2 root root 4096 May 23 10:56 orbit-root

Drwx-. 2 gdm gdm 4096 May 23 10:56 pulse-aQWRK1os9K8n

Drwx-. 2 root root 4096 May 23 10:55 pulse-Zd56fb7r0ESa

Drwx-. 2 root root 4096 May 23 10:55 virtual-root.OV7MJo

Drwx-. 2 root root 4096 May 21 16:18 virtual-root.q1Shey

-rw-rw-r--. 1 hadoop hadoop 5 May 23 11:05 yarn-hadoop-nodemanager.pid

-rw-rw-r--. 1 hadoop hadoop 5 May 23 11:05 yarn-hadoop-resourcemanager.pid

[hadoop@hadoop002 tmp] $

[hadoop@hadoop002 tmp] $cat hadoop-hadoop-datanode.pid

2619

[hadoop@hadoop002 tmp] $

two。 Modify the path of the HADOOP_PID_DIR file

Normally, pid files cannot be placed in the / tmp directory in production, because the pid files under / tmp are automatically deleted once a month. So we built the tmp directory under the / data/hadoop directory:

2.1 looking at the / opt/software/hadoop/sbin/hadoop-daemon.sh file, it is found that the hadoop lookup pid is obtained according to the HADOOP_PID_DIR environment variable.

[hadoop@hadoop002 tmp] $cat / opt/software/hadoop/sbin/hadoop-daemon.sh | grep pid

# HADOOP_PID_DIR The pid files are stored. / tmp by default.

Pid=$HADOOP_PID_DIR/hadoop-$HADOOP_IDENT_STRING-$command.pid

If [- f $pid]; then

If kill-0 `cat $pid` > / dev/null 2 > & 1; then

Echo $command running as process `cat $pid`. Stop it first.

Echo $! > $pid

If [- f $pid]; then

TARGET_PID= `cat $pid`

Rm-f $pid

[hadoop@hadoop002 tmp] $

2.2 create the / data/hadoop/tem directory with root

[hadoop@hadoop002 hadoop] $exit # exit hadoop

Logout

[root@hadoop002 software] # cd /

[root@hadoop002 /] #

[root@hadoop002 /] # mkdir data

[root@hadoop002 /] # cd data/

[root@hadoop002 data] # mkdir hadoop

[root@hadoop002 data] # cd hadoop/

[root@hadoop002 hadoop] # mkdir tmp

[root@hadoop002 hadoop] # ll

Total 4

Drwxr-xr-x. 2 root root 4096 May 23 14:11 tmp

2.3 / data/hadoop/tem Directory Authorization

[root@hadoop002 hadoop] # chmod 777 tmp

[root@hadoop002 hadoop] # ll

Total 4

Drwxrwxrwx. 2 root root 4096 May 23 14:11 tmp

2.4 modify the environment variable of HADOOP_PID_DIR in the / opt/software/hadoop/etc/hadoop/hadoop-env.sh file

[hadoop@hadoop002 hadoop] $pwd

/ opt/software/hadoop/etc/hadoop

[hadoop@hadoop002 hadoop] $vi hadoop-env.sh

# export HADOOP_PID_DIR=$ {HADOOP_PID_DIR} # comment this line

Export HADOOP_PID_DIR=/data/hadoop/tmp # add a new path

After editing, save and exit.

2.5 restart sbin/start-dfs.sh

# stop the hadoop service first

[hadoop@hadoop002 sbin] $sh stop-dfs.sh

Stopping namenodes on [hadoop002]

Hadoop002: no namenode to stop

Hadoop002: no datanode to stop

Stopping secondary namenodes [hadoop002]

Hadoop002: no secondarynamenode to stop

# start the hadoop service again

[hadoop@hadoop002 sbin] $sh start-dfs.sh

Starting namenodes on [hadoop002]

Hadoop002: starting namenode, logging to / opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-hadoop002.out

Hadoop002: starting datanode, logging to / opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-hadoop002.out

Starting secondary namenodes [hadoop002]

Hadoop002: starting secondarynamenode, logging to / opt/software/hadoop-2.8.1/logs/hadoop-hadoop- secondarynamenode-hadoop002.out

[hadoop@hadoop002 sbin] $

# there are no files in the tmp directory before reboot

[root@hadoop002 ~] # cd / data/hadoop/tmp/

[root@hadoop002 tmp] # ll

Total 0

# after restart, the pid file is generated in the tmp directory

[root@hadoop002 tmp] # ll

Total 12

-rw-rw-r--. 1 hadoop hadoop 5 May 23 14:17 hadoop-hadoop-datanode.pid

-rw-rw-r--. 1 hadoop hadoop 5 May 23 14:17 hadoop-hadoop-namenode.pid

-rw-rw-r--. 1 hadoop hadoop 5 May 23 14:17 hadoop-hadoop-secondarynamenode.pid

The above is all the contents of the article "where are the pid files stored in HDFS"? thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.