Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to use Puppet correctly. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

1. Overview

Puppet is an open source software automation configuration and deployment tool, which is easy to use and powerful, and is getting more and more attention. now many large IT companies are using puppet to manage and deploy the software in the cluster. for example, google uses puppet to manage more than 6000 mac desktops (2007 data).

This paper mainly introduces the installation method, design architecture and use of puppet.

two。 Design architecture

Puppet is based on the CMARS architecture. The server side holds all the configuration code for the client server, which is called manifest in puppet. After the client downloads manifest, the server can be configured according to manifest, such as software package management, user management, file management and so on.

As shown in the figure above, the workflow of puppet is as follows: (1) the client puppetd calls facter,facter to detect some variables of the host, such as hostname, memory size, ip address, and so on. Pupppetd sends this information to the server through a ssl connection; (2) the puppetmaster on the server detects the hostname of the client, then finds the corresponding node configuration in manifest, and parses this part of the content. The information sent by facter can be processed as variables, the code involved in node is parsed, and other codes not involved are not parsed. Parsing is divided into several stages, grammar checking, and reporting errors if there is a grammatical error. If the syntax is correct, continue parsing, the parsing result generates an intermediate "pseudo code", and then sends the pseudo code to the client; (3) the client receives the "pseudo code" and executes it. The client sends the execution result to the server; (4) the server writes the execution result of the client to the log.

There are two points worth noting in the working process of puppet. First, in order to ensure security, the relationship between client and master is based on ssl and certificate, and only client authenticated by master certificate can communicate with master. Second, puppet will keep the system in a certain state you want and maintain it all the time, such as detecting a file and ensuring that it always exists, ensuring that the ssh service is always on, and if the file is deleted or the ssh service is closed, the next time puppet executes (default 30 minutes), it will recreate the file or start the ssh service.

3. Software installation

The apt-get command is not recommended for installation because bug exists in the puppet downloaded by this command. Can be installed directly from the source code, the software that needs to be installed are ruby,facter and puppet.

3.1 installation steps

Edit / etc/host to modify the hostname because puppet is certificate-based and the certificate contains the hostname

Install ruby, facter, and puppet on master and slave, and use ruby install.rb when installing facter and puppet.

3.2 directory structure after installation

(1) installation directory

The installation directory is saved as / etc/puppet by default, and the manifests in this directory stores the manifest file.

Other executable files are under / user/sbin, mainly:

Puppet: used to execute independent mainfests files written by users, such as:

Puppet-l / tmp/manifest.log manifest.pp

Puppetd: a client program that runs on a managed host, such as:

Puppet-server servername-waitforcert 60

Puppetmasterd: server programs that run on the management machine, such as:

Puppetmasterd-debug

Puppetca puppet authentication program, mainly used to authenticate slave certificates, such as:

Check the slave:puppetca-list to be authenticated

Authenticate these slave: puppetca-s-a

Puppetrun is used to connect clients and force local configuration files to run, such as:

Puppetrun-p 10-host host1-host host2-t remotefile-t webserver

(2) configuration file

Puppet.conf

The main configuration file of Puppet. If it is a root user, the profile is / etc/puppet/puppet.conf, ordinary user, and the profile is: ~ user/.puppet/puppet.conf

For more information on configuration parameters, see:

Http://docs.puppetlabs.com/references/stable/configuration.html#configuration-files

Fileserver.conf

Configuration file for the puppet file server. Configure access permissions with path configuration file path and allow/deny. For more information, please see: http://docs.puppetlabs.com/guides/file_serving.html

3.3 verify that the installation is successful

Select a slave and master for verification. Assume that the host of slave is slave00,master and the host of masterhost is masterhost, and enter on slave00:

Puppetd-test-server servername

Then check the slave that looks at the certification on masterhost:

Puppetca-list

If there is no problem, you can see slave00 at this time and sign the certificate of the slave:

Puppetca-s-a

In this way, the slave00 has passed the certificate verification and can interact with master further.

Write the site.pp file in the / etc/puppet/manifests directory of masterhost, as follows:

The code is as follows:

Node default {

File {

"/ tmp/test":

Content= > "hello\ n"

Mode = > 0644

}

}

At the same time, type: puppetd-test-server servername on slave00, look at the / tmp folder of slave00, and generate a new file test, which contains hello, and the permission of this file is-rw-r-r-. This proves that the puppet installation is successful, and if there is an error, see section 6.

4. Configuration scripting

This section introduces the puppet configuration script writing method, mainly refers to the puppet manifest writing method. Puppet abstracts the content that needs to be managed into resources, and each resource has different attributes, so puppet language is the language that describes the attributes of these resources and the relationship between resources.

For ease of management, puppet modularizes resources, meaning that the manifest for each functional module is placed in a separate directory. Each module contains a main manifest file (init.pp, which is the entry of the module, similar to the main function in C language), which contains a number of class to encapsulate the resources of the module, common resources are file,package,service, etc., each kind of resource has its own attributes, such as file has attribute name,owner,mode and so on.

This section mainly introduces the writing method of manifest in puppet, which in turn introduces the writing methods of resource attributes, resources, node management, functions and modules.

4.1 Resource Properties

There are two kinds of resource attributes, one is resource-specific attributes, the other is resource-specific attributes, which are described in the next section, while resource common attributes are common to all resources, which mainly include:

Before

Used to control the order of execution of different objects (resources), indicating that one object (resource) occurs after another object (require, by contrast, indicates that it occurred before). Such as:

The code is as follows:

File {"/ var/nagios/configuration":

Source = > "…"

Recurse = > true

Before = > Exec ["nagios-rebuid"]

}

Exec {"nagios-rebuild":

Command = > "/ usr/bin/make"

Cwd = > "/ var/nagios/configuration"

}

This code ensures that all code is up to date before it is compiled with make. You can also before multiple resources, such as:

Before = > [File ["/ usr/local"], File ["/ usr/local/scripts"]

Subscribe

Detect a resource that reloads when it changes, such as:

The code is as follows:

Class nagios {

File {"/ etc/nagios/nagios.conf":

Source = > "puppet://server/module/nagios.conf"

Alias = > nagconf # just to make things easier for me

}

Service {nagios:

Ensure = > running

Subscribe = > File [nagconf]

}

}

When the file nagconf is detected to be modified, the service nagios will be updated accordingly. It is important to note that currently the only resources that support subscribe are exec,service and mount.

For more information, see http://docs.puppetlabs.com/references/latest/metaparameter.html

4.2 Resources

The main resources commonly used are as follows:

File: file management

Package: package management

Service: system service management

Cron: configuring periodic tasks

Exec: running the shell command

(1) file resources

For more details, see http://puppet.wikidot.com/file

(2) package resources

For more details, see http://puppet.wikidot.com/package

(3) service resources

For more details, see http://puppet.wikidot.com/srv

(4) exec resources

For more details, see http://puppet.wikidot.com/exec

(5) cron resources

For more details, see http://puppet.wikidot.com/cron

4.3 Node Management

How does puppet distinguish between different clients and assign manifest to different servers? Puppet uses node resources to do this, and node is followed by the hostname of the client, for example:

Node 'slave00' {

Include ssh

}

Node 'slave11' {

$networktype= "tele"

$nagioscheckport= "80pr 22pr 3306"

Include apache, mysql, php

}

Variables can be used in the resource node, or other manifest can be included directly through include.

For more details, see http://docs.puppetlabs.com/references/latest/type.html

4.4 classes and functions

A class can define multiple related resources together to form a class. Classes can be inherited. For more information, please see: http://docs.puppetlabs.com/guides/language_guide.html#resource-collections

Functions (called "defination" in puppet) can wrap multiple resources into a single resource, or wrap a resource into a model for ease of use. For example, managing an apache virtual machine in debian is very simple, put the configuration file of a virtual host in / etc/sites-available/, and then make a symbolic link to the / etc/sites-enabled directory. You can copy the same configuration code for each of your virtual hosts, but it will be better and easier if you use the following code:

Define virtual_host ($docroot, $ip, $order = 500, $ensure = "enabled") {

$file = "/ etc/sites-available/$name.conf"

# The template fills in the docroot, ip, and name.

File {$file:

Content = > template ("virtual_host.erb")

Notify = > Service [apache]

}

File {"/ etc/sites-enabled/$order-$name.conf":

Ensure = > $ensure? {

Enabled = > $file

Disabled = > absent

}

}

}

You can then use this definition to manage an apache virtual host, as shown in the following code:

Virtual_host {"reductivelabs.com":

Order = > 100

Ip = > "192.168.0.100"

Docroot = > "/ var/www/reductivelabs.com/htdocs"

}

4.5 module

A module is a directory and its subdirectories under a / etc/puppet/modules directory. Modules can be inserted with import modulename in the main file site.pp of puppet. The new version of puppet can be automatically inserted into modules in the / etc/puppet/modules directory. The introduction of modules, you can structure the code, easy to share and manage. For example, all configurations for apache are written under the apache module. There are usually three directories under a module directory: files,manifests,templates. Manifests must include an init.pp file, which is the initial (entry) file of the module. When a module is imported, it will be executed from init.pp. You can write all the code into init.pp, or you can divide it into multiple pp files, and init will include other files. The files directory is the file distribution directory for the module, and puppet provides a file distribution mechanism, similar to rsync modules. The templates directory contains the erb model file, which is related to the template property of the file resource.

After puppet is installed, the modules directory is not available, just create one yourself, and then you can add your module to it.

5. Programming example

5.1 Hello World

This section introduces a very simple programming example: a slave gets its manifest from master, and the maniftest requires slave to do the following: install gcc, create a folder / home/dxc/test, download the file hello.c program, and compile hello.c.

(1) Code structure organization

The directory structure of the code on Master is as follows:

| |-auth.conf |

| |-fileserver.conf # puppet File Server configuration file |

| |-manifests # puppet main file directory |

| |-modules.pp # puppet summary of each module |

| |-nodes # the module to be processed by each slave |

| `- execHello.pp # hello module is processed by which slave |

| | `- site.pp # puppet master file (entry file) |

| |-modules # the file where each module of puppet resides |

| | `- hello # hello module |

| |-files # the file resources corresponding to the module, which may be the configuration files to be sent to slave, etc. |

| | `- hello.c |

| | `- manifests # manifest file of module |

| | `- init.pp # module entry file |

`- ssl # puppet certificate file directory

(2) Program execution process

The sequence of code calls is:

Slave initiates a connection request à site.pp à nodes à modules à init.pp

First, slave initiates a master connection request to verify the certificate

Then, after the certificate verification is passed, master will directly find the site.pp file in the entry file manifests directory, which may contain some global variables, parameter defaults (when these parameters are not set by each module, their default values), and calls to other pp files (in this case, various pp files under modules.pp and nodes will be called)

Then, through each pp file under nodes, master locates the module to be executed by the slave (init.pp is the entry of each module), and summarizes these module codes and returns them to slave

Finally, slave based on the manifest, configuration information sent by master.

(3) Code interpretation

Download the code directly here.

5.2 A more complex example

This section describes a more complex example that a company is using, and the puppet code layout is the same as the previous example, except that this example involves more modules and more complex dependency management. The details of the code will not be explained in this section, see the code for details.

6. Problems that may be encountered

Q: certificate mechanism of puppet

A: the problem of puppet certificate is the most common problem for beginners. Here's how to deal with it. When the puppet server is installed or started for the first time, it will automatically produce a root certificate and a server certificate, which is related to the host name, so if you change the host name after the certificate is generated, there will be a problem. The puppet client also automatically generates a certificate when it starts for the first time, but the certificate needs to be signed by the puppet server, so the puppet client sends a certificate request when it connects to the server for the first time; the server needs to sign the certificate. The puppet client downloads the signed certificate the next time it connects to the server.

There is an error in the certificate under Q:Ubuntu. How to solve it?

A: this method is a test environment for beginners, which is not recommended in the build environment. First delete the / var/lib/puppet/ssl directory on the puppetmaster (server side); then start puppetmasterd; and then delete the / var/lib/puppet/ssl directory on the client side as well. Write the hostname and corresponding ip address of the puppetmaster machine to the / etc/hosts of the client machine.

Then execute: puppetd-test-server server.example.com. Replace server.example.com

Change to your own server host name. When you execute this command, there will be a prompt, ignore it.

Then log in to the puppetmaster server machine and execute the puppetca-list command to see if there is a certificate request from the client; if not, check that the previous steps are performed correctly and that the network connection is working. If puppetca-list can see the request, execute the puppetca-s-a command; sign all certificate requests. Finally, go back to the puppet client machine and execute

Puppetd-test-server server.example.com.

You can establish a connection if your site.pp is ready. You can test puppet.

Add: if the time between the client and the server is not the same, it will also cause the certificate verification to fail, so when there is a certificate problem, you need to check whether the time of the two machines is the same. If not, use the date command or the ntpdate command to keep the time of the two machines the same.

Q: error [Puppet Users] err: Could not retrieve catalog; skipping run

A: it may be due to the installation of two versions of ruby or facter. The solution can be found in:

Https://projects.puppetlabs.com/issues/5279

7. Summary

As the server cluster becomes larger and larger, automating the configuration and deployment of these servers can make it very easy to manage and greatly reduce the cost of management and deployment, so it is highly valued by IT.

This document introduces puppet, a new type of software automation configuration and deployment tool. This paper mainly deals with the architecture, installation and use of puppet, and gives two examples.

In a large-scale generation environment, if there is only one puppetmaster, it will be too busy, because puppet is written in ruby, ruby is a parsing language, and each client has to be parsed once. When there are too many clients, it is too busy, so it needs to be expanded into a server group. Puppetmaster can be thought of as a web server, which is actually done by the web server module provided by ruby. Therefore, you can use web agent software to cooperate with puppetmaster to do cluster settings, see: http://puppet.wikidot.com/puppetnginx.

After reading the above, do you have any further understanding of how to use Puppet correctly? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.