Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to use Docker Compose and Ansible to improve this design. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have some understanding of the relevant knowledge after reading this article.

We have implemented the automatic construction of the image through Jenkins server. We use a script to encapsulate the Docker command, store it in GitHub, and set the version. We are currently taking steps to achieve sustained deployment by gradually improving existing processes. However, there are still some pain points that need to be addressed. We'll look at how to use Docker Compose and Ansible to improve this design.

When deploying the image, the engineer needs to log in to the server and run our Docker wrapper script from shell. This is not a good solution because it also requires developers to wait. No one will benefit from this way (as an engineer, how many times do you get interrupted when you do something you know well and are easy to automate? Because each deployment is performed through a SSH session on the operator's computer, the deployment process is not visible.

If you still remember our deployment script, you will find that it looks like the following code snippet:

In fact, what we do is abstract the Docker run command statement so that the engineer will not need to know the exact parameters needed for each image to run successfully. While this improves the situation where you have to remember all the Docker parameters and enter them manually, it also creates new problems:

The logic of each container is stored in the same file, which makes changes to the application deployment logic more difficult to track

When developers need to test or modify parameters, they need to be forced to sort out the logic in the script, rather than being able to easily read and modify parameters in a particular program.

Docker Compose is a more suitable tool in our workflow. It can also encode deployment parameters and specify it in the YAML file, which is docker-compose.yml. Docker Compose not only helps us solve the difficulties mentioned above, but also enables us to benefit from the future work of the community. Let's sort out the deployment script and create a Compose file for our JAVA program example. First, we need to create a docker-compose.yml file based on the original deployment logic:

For now, the deployment container only needs to enter the following command in the same directory as the docker-compose.yml file:

Docker-compose up

It starts a container based on the parameters set in the compose file. An important variable in the compose file is ${VERSION}. Docker Compose can insert the parameters listed in the compose file from the current shell environment. We can set the parameters by simply running the following statement:

VERSION=1.0.0 docker-compose up

It will start the java-service-1 program by picking out the image marked 1.0.0 from our private image repository. If the VERSION variable is not set, Docker Compose will generate a warning message and replace the variable value with an empty string, thus the image with the latest version label will be singled out. Therefore, it is very important to set variables correctly.

As part of the development process, we want developers to be able to build services locally and test their services. However, because docker-compose.yml points to the image of the private image repository, running docker-compose will open the service from the recently built image rather than from the local resource. Ideally, developers can use a typical docker-compose workflow by running the following code:

Docker Compose allows us to do this without modifying the docker-compose.yml file. We can use multiple files to override any parameters we want to change in the local test. In docker-compose.override.yml, we specify a key instead of a mirror, and remove the need for the VERSION variable. Since this is an overwrite file, we do not need to copy any additional settings, such as port settings:

After using Docker Compose instead of deploying the script, we can:

Store each compose file in source code, similar to Dockerfile

Complex deployment scripts are no longer required

Allows developers to easily test and modify applications locally.

Now that we have the compose file for the java-service-1 program, we can remove it from our deployment script, so the file organization is similar to the following:

At this point, we still haven't solved the problem between image build and deployment. All the deployment logic is included in the docker-compose.yml file, but how does it run to the end in the environment? Now that we are running the Docker daemons related to UNIX and TPC socket, it is time to discuss some security-related issues.

In our case, the engineer logs in to the server and manually runs the deployment script for the container required by each server. By default, when the Docker command is run locally, it will use UNIX socket / var/run/docker.sock to connect to the Docker daemon, or let the daemon listen on the TCP socket, which allows the user to connect remotely to each Docker daemon, allowing the engineer to run the command as if it were logged in to the host. This provides more flexibility for connections, but does not take into account some overhead and security issues:

Increased security risks through network connections

Increased demand for host-based or network-based ACLs

Protecting daemons requires distributed CA and client authentication.

Another possible approach is not to run the Docker daemon in a UNIX socket-based manner, but to use SSH to run commands. The established ACLs protects the SSH port, and it allows only specific users authorized through SSH to use the Docker daemon. Although this is not the simplest approach, it helps to keep running overhead low and minimize security risks. This is important, especially for fine-grained, sparse task queues.

To facilitate running Docker commands through SSH, we can use Ansible--, a popular choreography and configuration management tool. It is agentless and allows the script (collection of server tasks) to be run over a SSH connection. A simple script to run the docker-compose command is as follows:

If you don't know much about Ansible, you may be able to get a general idea of what we want to do through the script above. They are executed step by step in order, as follows:

Ansible will connect to the target server through SSH (allows hosts to be specified by using the DESTINATION variable)

In each server, Ansible logs in to the company's private image repository by executing the shell command

Ansible copies the docker-compose.yml files located in Jenkins (the server running the ansible script) to / tmp/docker-compose.yml in each target server

Run the docker-compose command on each target server

Clean up by deleting the remote / tmp/docker-compose.yml file

A shell script can be used in the same event. In Ansible, however, it is easy to parallelize tasks and get well-tested modules, and by combining Ansible with the new deployment script, we can launch the container remotely, which is an important step forward compared to engineers logging in to the host and running commands manually. To provide greater visibility into the deployment process and state, we will set up Jenkins tasks to run Ansible code. By using Jenkins, we can easily integrate build and deployment tasks in the future, resulting in additional benefits.

The Jenkins task requires two parameters: the target host (passed to the DESTINATION variable in the script) and the version of the deployment image (insert the VERSION variable in the docker-compose.yml file). The build part of most tasks is a shell builder that will try to find the docker-compose.yml file in the program and then run the ansible-playbook command by passing the variable (with-e) to the script:

Although it seems that we have made only minor changes to the workflow, we are moving step by step towards building a continuous deployment model:

Deployment can be reviewed. We use logs to record what output, when output, and which hosts are target hosts, all thanks to Jenkins.

The program deployment logic has been dispersed from a single script to a separate docker-compose.yml file stored in the program source code, which means that we can easily change the program deployment logic through git. We can also easily build and deploy when the program source file or deployment file changes.

Although these improvements have solved some problems, the new problems they bring have also become the focus:

Which version of which container will be deployed to where?

What state will the container be in after it is deployed?

How do we determine which host is the target host of the program?

In the following articles in this series, we will explore how to run Rancher and why it is used, especially how it solves the above problems. At the same time, we also discuss its unexpected role as a bridge between the business and the development team.

On how to use Docker Compose and Ansible to improve this design is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.