Shulou(Shulou.com)06/02 Report--

Graphite installation notes

Experimental environment: CentOS7.5 x64

Programs that deploy python recommend using virtual environments such as virtualenv to prevent contamination of the python package that comes with the system.

Install some necessary rpm packages

Yum install cairo--the other 2 packages cairo-devel pycairo can be installed without installation

Install virtualenv

Yum install python-pip

Pip install virtualenv

Activate the virtual environment

Virtualenv / opt/graphite

Source / opt/graphite/bin/activate

Install the full set of graphite components

Export PYTHONPATH= "/ opt/graphite/lib/:/opt/graphite/webapp/"

Pip install-- no-binary=:all: https://github.com/graphite-project/whisper/tarball/master

Pip install-- no-binary=:all: https://github.com/graphite-project/carbon/tarball/master

Pip install-- no-binary=:all: https://github.com/graphite-project/graphite-web/tarball/master

If this step indicates that the version of a dependency package is wrong, we can manually execute pip to install the dependency package. Here is a list of pip packages seen on a functioning graphite:

Attrs==19.1.0

Automat==0.7.0

Cachetools==3.1.1

Cairocffi==0.9.0

Cffi==1.12.3

Constantly==15.1.0

Django==1.11.24

Django-tagging==0.4.3

Gunicorn==19.9.0

Hyperlink==19.0.0

Idna==2.8

Incremental==17.5.0

Pycparser==2.19

PyHamcrest==1.9.0

Pyparsing==2.4.2

Pytz==2019.2

Scandir==1.10.0

Six==1.12.0

Twisted==19.7.0

TxAMQP==0.8.2

Urllib3==1.25.3

Whisper==1.2.0

Zope.interface==4.6.0

Initialization

Cd / opt/graphite/webapp/graphite

Cp local_settings.py.example local_settings.py

Vim local_settings.py modifies the following three lines of code:

CONF_DIR ='/ opt/graphite/conf'

STORAGE_DIR ='/ opt/graphite/storage'

STATIC_ROOT ='/ opt/graphite/webapp/content'

LOG_DIR ='/ opt/graphite/storage/log/webapp'

Initialize the data, otherwise an error such as no such user will be prompted when graphite is started.

Cd / opt/graphite/webapp

PYTHONPATH=/opt/graphite/webapp django-admin.py migrate-- settings=graphite.settings-- run-syncdb # # if there is no yum install cairo in the first step above, this step will report an error

Copy the configuration file for graphite

Cd / opt/graphite/conf/

Cp carbon.conf.example carbon.conf

Cp storage-schemas.conf.example storage-schemas.conf will generally change the configuration in this file to customize the key storage cycle.

Cp relay-rules.conf.example relay-rules.conf

Cp dashboard.conf.example dashboard.conf

Cp graphTemplates.conf.example graphTemplates.conf

Cp blacklist.conf.example blacklist.conf

Cp aggregation-rules.conf.example aggregation-rules.conf

Cp storage-aggregation.conf.example storage-aggregation.conf

Start the graphite suite

/ opt/graphite/bin/carbon-relay.py start

/ opt/graphite/bin/carbon-cache.py start

Ss-lntp | egrep carbon

LISTEN 0 50 *: 7002 *: * users: ("carbon-cache.py", pid=48433,fd=11)

LISTEN 0 50 *: 2013 *: * users: ("carbon-relay.py", pid=37683,fd=13)

LISTEN 0 50 *: 2014 *: * users: ("carbon-relay.py", pid=37683,fd=15)

LISTEN 0 50 *: 2003 *: * users: ("carbon-cache.py", pid=48433,fd=13)

LISTEN 0 50 *: 2004 *: * users: ("carbon-cache.py", pid=48433,fd=14)

/ opt/graphite/bin/run-graphite-devel-server.py-- port=8085-- libs=/opt/graphite/webapp / opt/graphite foreground launch

Start the dotting test process

Python / opt/graphite/examples/example-client.py

Install nginx

Yum install nginx

Cd / etc/nginx/conf.d

The cat graphite.conf content is as follows:

Upstream graphite {

Server 127.0.0.1:8085 fail_timeout=0

}

Server {

Listen 80 default_server

Server_name _

Root / opt/graphite/webapp

Access_log / var/log/nginx/graphite.access.log

Error_log / var/log/nginx/graphite.error.log

Location = / favicon.ico {

Return 204

}

# serve static content from the "content" directory

Location / static {

Alias / opt/graphite/webapp/content

Expires max

}

Location / {

Try_files $uri @ graphite

}

Location @ graphite {

Proxy_pass_header Server

Proxy_set_header Host $http_host

Proxy_redirect off

Proxy_set_header X-Real-IP $remote_addr

Proxy_set_header X-Scheme $scheme

Proxy_connect_timeout 10

Proxy_read_timeout 10

Proxy_pass http://graphite;

}

}

Create nginx related files

Touch / var/log/nginx/graphite.access.log

Touch / var/log/nginx/graphite.error.log

Chmod 640 / var/log/nginx/graphite.*

Chown nginx.nginx / var/log/nginx/graphite.*

Start nginx

Nginx-t

Systemctl start nginx

Visit 192.168.2.4 / the following figure

You can see that the data has been collected.

We can match the graphite data source in grafana, and then display the data, similar to the following:

Launch brubeck (foreground) [a tool similar to statsd, but does not require a nodejs environment]

Yum install jansson jansson-devel

Yum install libmicrohttpd libmicrohttpd-devel

Git clone https://github.com/github/brubeck.git

Cd brubeck/

. / script/bootstrap

Mkdir / usr/local/brubeck/

Cp brubeck/ usr/local/brubeck/

The added configuration file for vim / usr/local/brubeck/brubeck.json is as follows:

{

"sharding": false

"server_name": "prod-devops-graphite-01"

"dumpfile": "/ var/lib/brubeck.dump"

"capacity": 15

"expire": 20

"http": "8000"

"backends": [

{

"type": "carbon"

"address": "localhost"

"port": 2003

"frequency": 10

}

]

"samplers": [

{

"type": "statsd"

"address": "0.0.0.0"

"port": 8135

"workers": 8

"multisock": true

"multimsg": 8

}

]

}

[brubeck listens on port 8135 and is used to receive management from the business side. Port 2003 refers to the port of carbon-cache]

. / brubeck-- config brubeck.json foreground launch process

Test call to brubeck

Echo "db.dba.dble2.front_conn:11111 | g" | nc-u-w1 127.0.0.1 8135

Modify the period in which graphite stores data

Cd / opt/graphite/conf

Vim storage-schemas.conf

# Schema definitions for Whisper files. Entries are scanned in order

# and first match wins. This file is scanned for changes every 60 seconds.

#

# [name]

# pattern = regex

# retentions = timePerPoint:timeToStore, timePerPoint:timeToStore,...

# Carbon's internal metrics. This entry should match what is specified in

# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings

[carbon]

Pattern = ^ carbon\.

Retentions = 60RU 90d

[service-sla]

Pattern = ^ service.sla-pct.*

Retentions = 10sVR 2drect 1minR 8d 10minRom 365d

[service1-rule]

Pattern = ^ service1\ .timers\ .Rule. *

Retentions = 10s v v 5min pm 1min v r r 1h r r 10min r r 7d

[service1-ic]

Pattern = ^ service1\ .counters\ .IC. *

Retentions = 10sVR 24hpm 1minRU 7d

[logstash]

Pattern = service1\ .counters\ .logstash. *

Retentions = 10sVR 24hpm 1minRU 7d

[service1-timers]

Pattern = ^ service1\ .timers\. *

Retentions = 10sbank 5min, 1min, 1min, 1min, 10min, 1d, 1h, 365d

[service]

Pattern = ^ service1.*

Retentions = 10sVR 24hwork 1minRU 6d 10minRom 365d

[counter]

Pattern = ^. *\ .counters\.

Retentions = 10sVR 24hwork 1minRU 7d 10minJu 30d

[timers]

Pattern = ^. *\ .timers\.. *

Retentions = 10sbank 5min, 1min, 1min, 10min, 10min, 1d, 1h, 30d.

[db]

Pattern = ^ db\. *

Retentions = 1min:30d

[default_10s_for_1hour]

Pattern =. *

Retentions = 10s:1h

The vim storage-aggregation.conf content is as follows:

[min]

Pattern =\ .min$

XFilesFactor = 0.01,

AggregationMethod = min

[max]

Pattern =\ .max$

XFilesFactor = 0.01,

AggregationMethod = max

[all_sum]

Pattern =\ .sum$

XFilesFactor = 0.01,

AggregationMethod = sum

[all_count]

Pattern =\ .count$

XFilesFactor = 0.01,

AggregationMethod = sum

[service1_gauges]

Pattern = ^ service1.gauges

XFilesFactor = 0.01,

AggregationMethod = last

[service1_counters]

Pattern = ^ service1\ .counters

XFilesFactor = 0.01,

AggregationMethod = sum

[service1]

Pattern = ^ service1.*

XFilesFactor = 0.01,

[default_average]

Pattern =. *

XFilesFactor = 0.01,

AggregationMethod = average

Note that after changing to the above two files, we need to restart the carbon-cache process to take effect.

Recommendations:

1. Graphite-web process. It is recommended to use supervisor to control start and stop.

2. Personally, I think brubeck is better than statsd. For example, it may not be convenient to deploy nodejs environment on some machines.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.