Shulou(Shulou.com)06/03 Report--

1. Demand:

Combined with the company environment hadoop1.0 and 2.0 cluster, Ganglia is selected as the cluster monitoring project of our company.

II. Introduction to Ganglia

1.Ganglia is an open source cluster monitoring project launched by UC Berkeley, designed to measure thousands of nodes. The core of Ganglia consists of gmond, gmetad, and a Web front end. It is mainly used to monitor system performance, such as: cpu, mem, hard disk utilization, Imax O load, network traffic and so on. It is easy to see the working status of each node through the curve. It plays an important role in reasonably adjusting and allocating system resources and improving the overall performance of the system.

two。 Each computer runs a daemon named gmond that collects and sends metrics data. The host that receives all the measurement data can display the data and pass a compact form of the data into the hierarchy. It is precisely because of this hierarchical pattern that Ganglia can be well extended.

3.gmond brings very little system load, which makes it a piece of code running on each computer in the cluster without affecting user performance. All this data is collected multiple times, which can affect the performance of nodes. The "jitter" in the network occurs when a large number of small messages appear at the same time, which can be avoided by keeping the node clock consistent.

The 4.Ganglia monitoring suite consists of three main parts: gmond,gmetad, and the web interface, often referred to as ganglia- web.

Gmond is a daemon that runs on every node that needs to be monitored, collects monitoring statistics, sends and receives statistics on the same multicast or unicast channel.

Gme tad is also a daemon that periodically checks the gmonds, pulls data from there, and stores their metrics in the RRD storage engine. It can query multiple clusters and aggregate metrics. RRD is also used to generate the web front end of the user interface.

4.3 ganglia- web, as the name implies, should be installed on a machine with gmetad running in order to read RRD files.

Generally speaking, gmond needs to be installed on every machine in a cluster, as long as the machine responsible for displaying the web interface is installed with gmetad.

Third, installation step by step

1. Preparation in advance

1.1. Node preparation

Modify hostname, ip, iptables off; time synchronization: ntpdate-s time.windows.com

1.2. Software preparation

Ganglia-3.7.1.tar.gz and ganglia-web-3.7.0.tar.gz address: http://ganglia.info/?page_id=66confuse-2.7.tar.gz address: http://www.nongnu.org/confuse/Python-2.7.10.tgz address: https://www.python.org/downloads/release/python-2710/httpd-2.2.31.tar.gz address: http://httpd.apache.org/downloadphp-5 .6.12.tar.gz address: http://php.net/downloads.php

two。 Cluster planning

2.1. Cluster information

Cat / etc/issue host name

IP system version

Function dchdmaster2192.168.1.193CentOS 6.5 x86 slave 64 monitoring service master node dchadoop206192.168.1.199CentOS 6.5 x86 slave 64 monitoring service slave node dchadoop207192.168.1.216CentOS 6.5 x86twin 64 monitoring service slave node

2.2. Software planning

The master node of Ganglia monitoring service needs to be installed: ganglia,ganglia-web,php,apacheGanglia is monitored and slave node needs to be installed: ganglia

2.3. Installation path

Ganglia-core installation path: / usr/local/gangliaphp installation path: / usr/local/phpapache installation path: / usr/local/apache2ganglia-web installation path: / opt/ganglia/ganglia-webrrds data path: / opt/ganglia/rrds

3. Installation framework

3.1. Check for dependencies [or find a lack of reinstallation during installation]

# rpm-q gcc glibc glibc-common rrdtool rrdtool-devel expat expat-devel pcre pcre-devel dejavu-lgc-sans-mono-fonts dejavu-sans-mono-fonts zlib-devel pcre pcre-devel apr apr-devel apr-util apr-util-devel

Yum installation:

# yum-y install gcc glibc glibc-common rrdtool rrdtool-devel expat expat-devel dejavu-lgc-sans-mono-fonts dejavu-sans-mono-fonts zlib-devel pcre pcre-devel apr apr-devel apr-util apr-util-devel#yum-y install perl libxml2 libxml2-devel

Annotation

Pcre (Perl Compatible Regular Expressions) regular expression processing

Apr Apache Portable Runtime

3.2. Install ganglia

Transfer the source files to be installed to a node under / opt/src

# cd / opt/src

3.2.1. Confuse

/ / libconfuse is a configuration file parser library implemented in C #

# tar zvxf confuse-2.7.tar.gz#cd confuse-2.7#./configure CFLAGS=-fPIC-disable-nls

/ /-fPIC acts in the compilation phase, telling the compiler to generate location-independent code (Position-Independent Code). There is no absolute address in the code, and all the relative addresses are used, so the code can be loaded into any location in memory by the loader and executed correctly. This is exactly what the shared library requires. When the shared library is loaded, the location in memory is not fixed.

/ /-- disable-nls: forbids support for internationalization

Make & & make install

3.2.2. Python

/ / ganglia-gmond-modules-python

/ / very slow

Python [default python.so path / usr/lib64/]

# cd / opt/src#tar-zvxf Python-2.7.10.tgz#cd / opt/src/Python-2.7.10#./configure-- prefix=/usr/local-- enable-shared make & & make install

/ / libpython2.7.so is under / usr/local/lib after installation

Configure shared libraries

# vi / etc/ld.so.conf

-- add the following content: the location of # python.so

/ usr/local/lib

Enable configuration

# ldconfig

The purpose of the / / ldconfig command is to search for shareable dynamic link libraries (format as described above, lib*.so*) under the default search directory (/ lib and / usr/lib) and the directory listed in the dynamic library configuration file / etc/ld.so.conf

Check whether it is effective or not

# ldconfig-v | grep "libpython2.7.so"

3.2.3. Ganglia

3.2.3.1. Installation

# cd / opt/src# tar zvxf ganglia-3.7.1.tar.gz# cd ganglia-3.7.1#. / configure-- prefix=/usr/local/ganglia-- with-gmetad-- enable-gexec-- with-python=/usr/local# make & & make install

3.2.3.2. Configure gmond (all nodes)

# cd / opt/src/ganglia-3.7.1

/ / set boot to boot

# cp. / gmond/gmond.init / etc/init.d/gmond#vi / etc/init.d/gmond#GMOND=/usr/local/ganglia/sbin/gmond

/ / initialize the gmond configuration file

. / gmond/gmond-t > / usr/local/ganglia/etc/gmond.confvi / usr/local/ganglia/etc/gmond.conf-- modify the following cluster {name = "cloud-cluster" owner = "nobody" latlong = "unspecified" url = "unspecified"} udp_send_channel {# bind_hostname = yes # Highly recommended, soon to be default. # This option tells gmond to use a source address # that resolves to the machine's hostname. Without # this, the metrics may appear to come from any # interface and the DNS names associated with # those IPs will be used to create the RRDs.mcast_join = 239.2.11.71port = 8649ttl = 1} / * You can specify as many udp_recv_channels as you like as well. * / udp_recv_channel {mcast_join = 239.2.11.71port = 8649bind = X.X.X.Xretry_bind = true# Size of the UDP buffer. If you are handling lots of metrics you really# should bump it up to e.g. 10MB or even higher.# buffer = 10485760}

# # adding python Metric Begin

Copy python module to the ganglia deployment directory

# mkdir / usr/local/ganglia/lib64/ganglia/python_modules#cd / opt/src/ganglia-3.7.1#cp. / gmond/python_modules/*/*.py / usr/local/ganglia/lib64/ganglia/python_modules

The installer ganglia-3.7.1 provides some python module configuration files by default, which only need to be deployed to the / usr/local/ganglia/etc/conf.d directory to take effect.

# cp. / gmond/python_modules/conf.d/*.pyconf / usr/local/ganglia/etc/conf.d

# Why create python_modules and copy to the specified folder? Convention due to configuration, check / usr/local/ganglia/etc/conf.d/modpython.conf

# # adding python Metric End

Synchronize [ganglia, gmond configuration, python module] other nodes

# scp-r / usr/local/lib/libpython2.7.so* root@dchadoop206:/usr/local/lib#scp-r / usr/local/lib/python2.7/ root@dchadoop206:/usr/local/lib/#scp / etc/ld.so.conf root@dchadoop206:/etc/ld.so.conf # scp-r / usr/local/ganglia/ root@dchadoop206:/usr/local/ganglia/#scp-r / etc/init.d/gmond root@dchadoop206:/etc/init.d/gmond

/ / copy to all slave nodes

# for i in {206.. 212}; do scp-r / usr/local/lib/libpython2.7.so* root@dchadoop$i:/usr/local/lib;done;#for I in {206.. 212}; do scp-r / usr/local/lib/python2.7 root@dchadoop$i:/usr/local/lib;done;#for I in {206.. 212}; do scp-r / etc/ld.so.conf root@dchadoop$i:/etc;done; # for i in {206.. 212} Do scp-r / usr/local/ganglia/ root@dchadoop$i:/usr/local/ganglia/;done;#for I in {206.. 212}; do scp-r / etc/init.d/gmond root@dchadoop$i:/etc/init.d/gmond;done

Start the gmond service and set it to boot to run # ssh root@dchdmaster2 'cmd' automatically

Ssh root@dchadoop206 'ldconfig-v | grep python2.7'

/ / add boot boot

# cd / opt/src/ganglia-3.7.1#cp. / gmond/gmond.init / etc/init.d/gmond # vi / etc/init.d/gmond-- modify the following GMOND=/usr/local/ganglia/sbin/gmondchkconfig-- add gmond

Activate the service:

Service gmond start/restart

Test it on a gmond machine.

Telnet localhost 8649

Check: gmond did not start successfully

Add ganglia users to each node and restart gmond

# for i in {208.. 212}; do ssh dchadoop$i 'useradd ganglia;groups ganglia';done;#for I in {208.. 212}; do ssh dchadoop$i' hostname;service gmond restart;service gmond status';done

3.2.3.3. Configure gmetad (primary node)

# cd / opt/src/ganglia-3.7.1#vi / usr/local/ganglia/etc/gmetad.conf-- modify the following

# # if this file does not have cp. / gmetad/gmetad.conf / usr/local/ganglia/etc/

-- collecting status data from the first response host data_source "cloud-cluster" 10 dchdmaster2 dchadoop206 crxy162.crxydata_source "cloud-cluster" 10 dchadoop207 dchadoop208 dchadoop209 dchadoop210 dchadoop211 dchadoop212gridname "cloud-grid"-- gmetad listening port xml_port 8651--gmetad interactive listening port interactive_port 8652rrd_rootdir "/ opt/ganglia/rrds" case_sensitive_hostnames 0

Create and modify rrds data directory owner

# mkdir-p / opt/ganglia/rrds#chown-R nobody:nobody / opt/ganglia/rrds

Start the gmetad service

# service gmetad restart

# # setting Boot

# cd / opt/src/ganglia-3.7.1#cp. / gmetad/gmetad.init / etc/init.d/gmetadvi / etc/init.d/gmetad

Modify the following

GMETAD=/usr/local/ganglia/sbin/gmetadchkconfig-add gmetad

Test on a gmetad machine

Telnet localhost 8651

4. Ganglia debugging skills

Http://blog.csdn.net/xxd851116/article/details/25109043

4.0.0.1. Install ganglia-web (primary node)

4.0.0.1.1. Install apache

Cd / opt/srctar zvxf httpd-2.2.31.tar.gzcd httpd-2.2.31./configure-- prefix=/usr/local/apache2make & & make install

Question:

[APR is required]

[configure: error: Cannot use an external APR with the bundled APR-util]

4.0.0.1.2. Install php (slow installation)

# vi / usr/local/apache2/bin/apxs

# some versions want to change the first line to the perl location: #! / usr/bin/perl-w

# cd / opt/src#tar zvxf php-5.6.12.tar.gz#cd php-5.6.12#./configure-- prefix=/usr/local/php-- with-apxs2=/usr/local/apache2/bin/apxs#make & & make install

[Perl is not installed]

[xml2-config not found]

4.0.0.1.3. Ganglia-web

# cd / opt/src#tar zvxf ganglia-web-3.7.0.tar.gz-C / opt/ganglia#mv / opt/ganglia/ganglia-web-3.7.0 / opt/ganglia/ganglia-web#cd / opt/ganglia/ganglia-web#cp conf_default.php conf.phpvi conf.php$conf ['gweb_confdir'] = "/ opt/ganglia/ganglia-web"; $conf [' gmetad_root'] = "/ opt/ganglia"; vi header.php

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.