Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "how to realize Hadoop cluster synchronization". In the operation process of actual cases, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to handle these situations! I hope you can read carefully and learn something!

I. Synchronous mode

Select a machine as the time server (here select hadoop01), and all machines synchronize with this cluster time on a regular basis, for example, every ten minutes.

II. Tools required

There are two time synchronization servers: ntpd and ntpdatp. Although both can be used to achieve the purpose of time synchronization, but before using a clear question, ntpd and ntpdate update time when there is any difference. ntpd is not only a time synchronization server, it can also synchronize time with the standard time server through the client, and it is smooth synchronization, not ntpdate immediate synchronization, ntpdate should be used with caution in the production environment, and just like this, both cannot run at the same time.

III. Specific operation steps

Time Server Configuration

(1)Check if ntp is installed. [root@hadoop01 Desktop]# rpm -qa| grep ntp if ntp-4.2.6p5-1.el6.centos.x86_64 is displayed. fontpackages-filesystem-1.41-1.1.el6.noarch

ntpdate-4.2.6p5-1.el6.centos.x86_64 proves that ntp has been installed, otherwise ntp service must be installed first. [root@hadoop01 Desktop]# yum install -y ntp.

(2)Modify the ntp configuration file [root@hadoop01 Desktop]# vi /etc/ntp.conf. First configure the authorization of the client, that is, set the permission to access NTP Server for the specified machine (client), which is realized by restricting the configuration item. Release #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap. Change to restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap. If the cluster is in a closed local area network, you can mask the default server.

Server 0.centos.pool.ntp.org iburst.server 1.centos.pool.ntp.org iburst.

server 2.centos.pool.ntp.org iburst.server 3.centos.pool.ntp.org iburst shield changed to

#server 0.centos.pool.ntp.org iburst.# server 1.centos.pool.ntp.org iburst.# server 2.centos.pool.ntp.org iburst.# server 3.centos.pool.ntp.org iburst. uses its local clock as a time source so that it can continue to serve the network even if it loses network connectivity. add server 127.127.1.0 fudge 127.127.1.0 stratum. modify the/etc/sysconfig/ntpd file

[root@hadoop01 Desktop]# vim /etc/sysconfig/ntpd Restart ntpd after adding SYNC_HWCLOCK=yes,[root@hadoop01 Desktop]# service ntpd restart, set ntpd boot default startup [root@hadoop01 Desktop]# chkconfig mtpd on. After completing the above operations, use the date command to check whether the system time corresponds to the current time. If it does not correspond, it should be a time zone problem. Change time zone [root@hadoop01 Desktop]# tzselect. Choose the correct time zone as prompted and set cp /usr/share/zoneinfo/Asia/Shanghai as the link to/etc/localtime

[root@hadoop01 Desktop]# cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

IV. Other machine configurations

Configure synchronization with the time server every ten minutes

You need to write time synchronization scripts in other machines in the cluster. Take hadoop02 as an example,[root@hadoop02 Desktop]# crontab -e. */10 * * * * /usr/sbin/ntpdate hadoop01

Restart crond service. [root@hadoop02 Desktop]# service crond restart。If necessary, also to modify the time zone, the operation is the same as above.

"How to achieve Hadoop cluster synchronization" content is introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.