Shulou(Shulou.com)06/02 Report--

Optimize ansible speed

Briefly introduce and explain how to optimize the execution speed of ansible, as follows

The ingenious use of the-t option of the conventional optimization method ansible

The "- t" or "--tree" options of ansible save the execution results of ansible in a file under the specified directory by hostname.

Ansible test-m ping-o-f 6-t / tmp/tree

What is the content saved? In fact, it just saves the normal output.

# pwd/tmp/tree# cat 192.168.246.187 {"ansible_facts": {"discovered_interpreter_python": "/ usr/bin/python"}, "changed": false, "ping": "pong"} set ansible to open ssh persistent connection

To open a ssh persistent connection, the openssh version of the ansible side is required to be higher than or equal to 5.6. Use ssh-V to view the version number. Then set the connection parameters for ansible to connect to the controlled side using ssh, modify / et c/ans ible/ansible.cfg here, and start the following connection options in this file, where ControlPersist=5d is to control the duration of the ssh connection session for 5 days.

Ssh_args =-C-o ControlMaster=auto-o ControlPersist=5d

In addition, it is also possible to directly set the corresponding persistent connection item in / etc/ssh/ssh_config (not sshd_config, because the ssh command is a client command).

When a persistent connection is enabled, a connection will be established until the session expires. In the result of netstat, you will see that the ssh connection is always in established state, and some socket files will be generated in the .ans ible/cp directory of the current user's home directory, one file per session.

For example, perform an ad-hoc operation:

Ansible test-m ping

Looking at the netstat, it is found that the session of the ssh process has always been in the established state.

Netstat-lnpta

And the corresponding socket file will be generated under ~ / .ansible / cp/ in the home directory.

⚠️ 's method of maintaining a long connection in ssh:

(1) modify the etc/ssh/sshd_config on server

ClientAliveInterval 60 # server sends a request to client every 60 seconds, and then client responds, thus keeping the connection. After ClientAliveCountMax 3 # server sends a request, the client will automatically disconnect when the number of times of no response reaches 3. Under normal circumstances, client will not fail to respond to systemctl reload sshd.

(2) modify the etc/ssh/ssh_config on client and add the following: (if you don't have permission to change server configuration)

ServerAliveInterval 60 # client sends a request to server every 60 seconds, and then server responds, thus keeping the connection. After ServerAliveCountMax 3 # client sends a request, the server disconnects automatically when the number of times of no response reaches 3. Under normal circumstances, server will not fail to respond.

(3) temporary modification of the command line

In the command parameters, ssh-o ServerAliveInterval=60 will only maintain a persistent connection in the required connection. After all, not all connections have to be persistent.

Turn on pipelining

Pipeline is also a feature of openssh. In ansible's process of performing each task, there is a process of put the temporary task file to a temporary file on an ansible side, then sftp to the remote end, and then remotely execute the task over a ssh connection.

If pipelining is enabled, all the actions of a task are done in a ssh session, and the process of sftp to the remote is omitted, and the task it is about to perform is performed directly in the ssh session.

The way to enable pipelining is to set pipelining=true in the configuration file (such as ansible.cf g). The default is false.

Note, however, that if you use the sudo command in ansible (ssh user@host sudo cmd), you need to disable "requiretty" in the / etc/sudoers of the controlled node.

The reason for setting requiretty in / etc/sudoers is that when ssh executes commands remotely, its environment is non-login non-interactive shell, tty is not assigned by default, and sudo without tty,ssh cannot turn off password echo (use the "- tt" option to force SSH to assign tty).

Therefore, for security reasons, requiretty is enabled by default in / etc/sudoers, which requires that only users with tty can use sudo, that is, ssh connections were not allowed to execute sudo in the past. You can edit the configuration file through visudo/vim and annotate this option to disable it.

Grep requiretty / etc/sudoers # Defaults requiretty

The modification of the setting / etc/sudoers is done on the controlled node (or the ansible connection has been modified in the past). In fact, the sudo problem can also be solved on the ansible side, just add the "- tt" option to the ssh parameter of the ansible.

Ssh_args =-C-o ControlMaster=auto-o ControlPersist=5d-tt

When pipelining is enabled, the number of ssh connections is greatly reduced when each task is executed (only one ssh connection is needed), and the process of transferring task files by sftp is eliminated, so the execution efficiency is greatly improved when managing a large number of nodes.

Modify ansible execution policy

By default, ansible executes tasks remotely in parallel, and a batch controls how many hosts are controlled by the "- f" or "--forks" options on the command line. For example, the default number of parallel processes is 5, and if there are 20 controlled hosts, the next batch of five will not continue to execute the task until every five have completed one task, even if one of the intermediate machines performs better and completes faster, it will also wait there idle until all 20 hosts have completed the task before continuing with the next task in the same way. As follows:

H 2 h 3 h 4 h 5 h 6 (T1)-- > h 7 h 7 h 8 h 9 h 20 (T 1).-- > h 26 h 27 h 28 h 29 h 30 (T 1)-- > h 2 h 3 h 4 h 5 h 6 (T 2)-- >.

In ansible 2.0, a policy control option, strategy, has been added, and the default value is "linear", that is, the above is processed in batch parallelism. You can also set the value of strategy to "free".

In free mode, ansible cuts to the next host as quickly as possible. Also in the above example, first, every five machines execute a task in parallel. When one of the machines completes the task ahead of schedule due to good performance, it will not wait for the other four to complete, but will jump out of the task and let ansible cut into the next machine to execute the task. In other words, in this mode, after one host completes a task, the other host immediately executes the task, which is a "one after another" way. As follows:

H 2 h 3 h 4 h 5 h 6 (T1)-- > h 2 h 3 h 4 h 4 h 5 h 5 h 7 (T1)-- > h 27 h 28 h 29 h 30 (T1) h 2 (T2)-- > h 28 h 29 h 30 (T1) h 2 (T 2)-- >...

The settings are as follows:

-hosts: all strategy: free tasks:... Set up facts cach

Ansible or ansible-playbook always collects facts information first by default. In the case of a small number of controlled hosts, the collection of information can be tolerated, if the number of controlled hosts is very large, collecting facts information will consume a lot of time.

For example, collect facts when you are idle, cache it, and directly read the cache for reference when needed.

You can change the value of 'gathering'' in the configuration file of ansible to smart, implicit, or explicit.

(1) smart means to collect facts by default, but it will not be collected if facts already exists. Cache facts is used.

(2) implicit means to collect facts by default. To disable collection, you must use gather_facts: False.

(3) explicit means no collection by default. To collect explicitly, you must use gather_facts: Ture

When using facts caching (that is, set to smart), ansible supports two types of facts caching: redis and jsonfile.

For example, the following is the cache configuration method for jsonfile format in / etc/ansible/ansible.cfg.

[defaults] gathering = smartfact_caching_timeout = 86400 # # the cache expiration time set here is 86400 seconds, that is, one day of cache. Fact_caching = jsonfile fact_caching_connection = / path/to/cachedir # # cached json files are placed in the / path/to/cachedir directory, and the cache files for each host are named after the hostname.

The cache file is a json file. To view the cache file, such as the contents in / path/to/cachedir/192.168.100.59, use the following statement:

Cat / path/to/cachedir/192.168.100.59 | python-m json.tool

Description

This blog is a reference to Ma long Shuai boss article collation and generation, belongs to the blogger reading notes, if there is infringement, please contact me, delete!

Finally, thank open source, embrace open source ~

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.