Shulou(Shulou.com)06/01 Report--

This article introduces how to use the configuration management automation tool Puppet and its working principle, the content is very detailed, interested friends can refer to, hope to be helpful to you.

The characteristics of high scalability and fault tolerance in big data era put forward higher requirements for operation and maintenance. System management is no longer an era when we are tired of installing operating systems, configuring and optimizing system parameters one by one, patching, installing software, configuring software, and adding certain services. In order to improve efficiency, avoid repetitive work, reduce errors and accumulate knowledge, system administrators have begun to do some local automation work. But these are far from enough, in order to meet the needs of operation and maintenance, we need to apply automated operation and maintenance tools more thoroughly.

The editor will introduce how to use the configuration management automation tool Puppet to complete the system installation, monitor and alarm work, dissect the convenience that Puppet brings to the system administrator, and also introduce the architecture and working principle of Puppet.

From system installation to automatic deployment software, configuration, rollback, and then to server availability, performance, security maintenance, operation and maintenance managers need to fully master, in order to effectively complete the work, it is necessary to be familiar with several excellent open source software. As shown in Table 1.

Table 1 Classification of common operation and maintenance tools

For me, the most popular toolkits are Kickstart, Puppet, Zabbix, and Cacti.

Difficulties in operation and maintenance

Operation and maintenance work flow

Common operation and maintenance workflow includes: installation of system → optimization system and configuration of → installation software → configuration software → add monitoring → check. In the future, there may be adding service → configuration changes, → patching to fix vulnerabilities, and so on. Do you think it's cumbersome? Especially if you are responsible for a lot of equipment and can't do it on your own, you need some tools to help.

The uncertainties faced by the operation and maintenance work are even more troublesome. It is easy to change the application on 10 machines, but it will become very complicated if it rises to thousands or tens of thousands. Repetitive labor can also make people feel tired and boring, and over time it may also lead to boredom of work. These problems can be easily solved by using Puppet.

Automate on your own

In order to improve work efficiency and reduce the probability of making mistakes. Many companies are gradually adopting automation to achieve the above work. Some companies choose to develop their own set of tools because they can be customized according to their needs, but is this really necessary? Let's look at the disadvantages of doing so.

1. Building wheels from scratch: the challenge and hassle of building scripts.

two。 The maintainability of the program cannot be guaranteed (language).

3. Support different platforms.

4. Consideration after reinstallation of the system.

It takes a long time to co-ordinate and plan the whole system, and with the flow of personnel, different levels of skills will bring new problems. And independently developed systems can't just support one platform-cross-platform development means more uncertainty.

Comparison of automatic configuration tools

Table 2 compares the two most commonly used automated operation and maintenance tools, Puppet and Cfengine.

Table 2 functional comparison between Puppet and Cfengine

But what I really want to say is that the above comparison doesn't make much sense, the tool is how you use it, how you use it, play to its strengths, and combine it with your business. We don't need to be busy choosing a tool, but we should study it in depth.

Dissecting Puppet

Before using any software, we need to understand how it works, otherwise it will bring a lot of inconvenience to the subsequent use. Puppet uses the very simple Cramp S architecture, and all data interactions are carried out through SSL to ensure security. Its workflow is shown in figure 1.

Figure 1 Puppet workflow

1. The client Puppetd initiates an authentication request to Master or uses a signed certificate.

2. Master told Client that you were legal.

3. The client Puppetd calls Facter,Facter to detect some variables of the host, such as hostname, memory size, IP address, and so on. Puppetd sends this information to the server over a SSL connection.

4. The Puppet Master on the server side detects the hostname of the client, then finds the node configuration corresponding to manifest, and parses this part of the content. The information sent by Facter can be handled as variables, the code involved in node will be parsed, and other codes that are not involved will not be parsed. Parsing is divided into several stages, the first is syntax checking, if the syntax error is reported; if the syntax is correct, continue parsing, the result of parsing into an intermediate "pseudo code" (catelog), and then send the pseudo code to the client.

5. The client receives the "pseudo code" and executes it.

6. The client determines whether there is a File file during execution, and if so, initiates a request to the fileserver.

7. The client determines whether Report is configured or not, and if so, sends the execution result to the server.

8. The server writes the execution result of the client into the log and sends it to the reporting system.

When there are more than a thousand servers

As you have more and more servers, you may find that the efficiency of Puppet execution begins to decline, and the servers can no longer meet your needs. Here are several solutions to solve this kind of problem.

LoadBlancer

This is to improve the ability of Master to calculate "pseudo code" through a very simple Master expansion scheme. Typically, this architecture can support about 1000 servers. Of course, it also depends on whether your system is "complex" enough.

Figure 2 LoadBlancer scheme

There are two common implementations of this architecture: Apache+Passenger and Nginx+Mongrel. This article will take the latter as an example to briefly introduce its working mode.

1. Puppet Master runs multiple processes:

Puppet Master+Mongrel,port 18140

Puppet Master+Mongrel,port 18141

Puppet Master+Mongrel,port 18142

Puppet Master+Mongrel,port 18143

2. Nginx achieves load balancing to Puppet Master through Upstream. Nginx listens to port 8140 to forward requests other than file issuance to one of the above four Puppet Master instances. Nginx will verify the client certificate, but you need to configure the certificate issued by CA to allow the request. We can also configure 8141 to provide certificate signing.

3. If you use fileserver,Nginx, you can also handle it directly.

Puppet certified load balancer

Is multiple Master sufficient? A machine still has risks, so we need to be fault-tolerant and distribute Master on different machines, and CA authentication is also a very important part. We can use the following architecture to do a hot backup. As shown in figure 3.

This architecture can also be extended. Let's review how Puppet works again; there is an HTTPS connection between the Puppet client and the Nginx, and a HTTP connection between the Nginx and each Mongrel. Nginx is responsible for the verification of the client certificate, while Nginx only needs the public key of CA to do the verification. The advantage of this is that there is no need to synchronize the client's certificate and other settings between multiple management machines, only the public key of CA is needed, and the public key can be copied. However, there is a drawback: it is not convenient to delete client certificates. However, it is possible to use a master management machine, from which other management machines can synchronize certificates in real time.

Figure 3 Puppet authentication load balancing scheme

The idea of Puppet management machine cluster is as follows:

1. Synchronize the CA configuration to each machine, including public and private keys

two。 Use CA to issue a certificate to each management machine

3. Each management machine is equipped with LoadBalancer. 8140 provides configuration management and 8141 provides certificate issuance.

4. Keeplived can be used between management machines to achieve high availability and failover, including HA, etc. The architecture can be expanded at will

5. The configuration of each management machine is divided into Production and Development, which can be simply published to the management machine through Git.

6. Only modify the Development part during the test, specify it for individual clients, and push it to Production after success.

7. Configure a master CA management machine to solve the problem of deleting authentication.

Rational planning

All the post-rescue plans are not as good as reasonable planning before use, you need to be very clear about the status of the current business, the status of operations and maintenance. Know what problem you need to solve, then break it down and break it down step by step.

Git is recommended to manage Puppet

Standardize HostName and adopt DNS management

FileServer is independent, putting infrequent changes in fileserver and frequent changes in templates

Communicate with custom OS.

Many people may not understand why to customize OS, its advantage is that it can help you make some software packages needed by Puppet during system initialization and installation. Through the SN number you get when purchasing equipment, register the information of this machine in the WebUI system, and the machine can complete all the configuration after startup. If your WebUI is better, you can call the API of the monitoring system to complete the monitoring.

I believe that after reading this article, you will not only have an overall understanding of Puppet, but also be more familiar with the focus of automated operation and maintenance work. It may make you start to think about using Puppet to solve a lot of repetitive work in your operation and maintenance work, or to solve the architectural problems you now face.

I think to many system administrators who want to learn Puppet or are using Puppet, the working principle is very important, and many people just don't understand how it works, so they are in a hurry when they encounter problems in the process of using it. Readers must solve the problem by thinking more.

On how to use the configuration management automation tool Puppet and its working principle is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.