Shulou(Shulou.com)06/01 Report--

Deploy 3-instance MGR environment on macOS through Docker technology

Since the advent of Docker, the deployment of various environments has become simple and leisurely, but in fact, the environment of MySQL is also easy to deploy.

Prerequisite: you have successfully set up a 3-node MySQL Group Replication cluster running locally in your macOS operating system. For more information on how to set it, please refer to the official documentation.

Assuming that a 3-node MGR cluster with the following architecture has been implemented on our macOS, our goal is to move all three databases to the docker container to run, and can communicate with each other, still a MGR cluster.

Docker on MacOS

Next, let's put MySQL aside and first introduce the docker architecture in macOS. In macOS, the implementation of docker is slightly different from that in other Linux systems. In other Linux systems, the operating system itself is the host of the docker container, and the docker images are directly stored in the file system of the host. For example, we can see the root directory of docker through the docker info command:

$docker info | grep "Docker RootDir"

Docker Root Dir: / var/lib/docker

But under macOS, we look directly at this directory, which actually doesn't exist at all.

$ls / var/lib/docker

Ls: / var/lib/docker: No such file or directory

Who is the Docker host?

So where on earth is this directory? In fact, in a QEMU virtual machine, when we install docker in macOS and start it, we start a virtual machine. The whole contents of the virtual machine are all in one file. You can see the path to this file in the docker program properties interface. For example, on my machine, the path is / Users/Kamus/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2, which is usually large. In particular, after installing multiple docker images, it will easily take up dozens of GB of space.

As shown in the figure above, this is the location of the Disk image location.

You can also see from the figure above that this virtual machine is limited to 2 CPU and 2GB memory.

So, can we log in to this virtual machine to confirm this? You can log in to the virtual machine using the screen command that comes with macOS. As shown below, you can see that this is a virtual machine with a very new Linux version 4.9.38 kernel. In this virtual machine, there is a / var/lib/docker directory, only 2 CPU, and a total of 2GB memory.

Press control+a d (first control+a, then d) in the screen window to temporarily dettach the screen,screen-r to reopen the window. More screen commands can be viewed on your own man screen.

So now we can make it clear that macOS itself is not the host of the docker container that will be run in the future, and this Linux virtual machine is the real host. The host name of this machine is moby, which is the name of the community version of the docker project.

Directory sharing between Docker Host and macOS operating system

Due to the following shared folder function, the local directory of the macOS operating system can be accessed and updated in this virtual machine, which plays an important role in moving the local MySQL database to the docker container.

Let's take a look at these shared directories in screen.

As you can see, it is true that the directory in the macOS operating system is directly accessible in the virtual machine, and what is more convenient is that the directory path that is automatically mounted in the virtual machine is exactly the same as the path in macOS, for example, my personal home directory is / Users/Kamus whether in macOS or in this virtual machine.

Now our idea is basically to move the data files and configuration files of the MySQL database originally running in the macOS operating system to a directory that can be accessed by the Docker host, and then start the MySQL instance in the docker container.

Install the MySQL docker image

First install the MySQL docker image, which is the basis for running the MySQL docker container. Installing the image is extremely simple and only takes one step, and the following command is executed in the macOS operating system.

The standard steps are:

$docker pull mysql

However, because this will install the latest version, and my original MySQL database in macOS is version 5.7.17, in order to avoid possible version upgrade problems, I have done a specified version of the mirror pull.

$docker pull mysql:5.7.17

It only takes a few minutes to pull the MySQL image from the official docker store. After that, you can check it with the docker images command.

The first line is the newly pulled MySQL image, which was created by the creator 4 months ago.

Preparation before running the MySQL docker container

Design the directory structure of docker host

Now that we have a docker image and a MySQL MGR cluster that works well in the macOS operating system, we are starting to migrate. The first step, of course, is to cleanly close the three MySQL databases in the macOS operating system, and then we design the following directory structure to move all the data files of the three databases in.

$mkdir / Users/Kamus/mysql_data

$mkdir / Users/Kamus/mysql_data/s1-docker

$mkdir / Users/Kamus/mysql_data/s2-docker

$mkdir / Users/Kamus/mysql_data/s3-docker

According to the shared folder function described earlier, we can know that these directories can be accessed through exactly the same path in the docker host. We will move the data files of the three databases originally in the macOS operating system to the s1murdocker directory respectively. As for which files need to be moved, this is the basic knowledge of MySQL DBA, which will not be discussed here.

Since the goal is to be able to run the MGR cluster, some database initialization parameters need to be set extra, and the my.cnf content in the docker container cannot be changed, so we design a special directory to store the my.cnf files of all databases.

$mkdir / Users/Kamus/mysql_data/conf.d

$mkdir / Users/Kamus/mysql_data/conf.d/s1-docker

$mkdir / Users/Kamus/mysql_data/conf.d/s2-docker

$mkdir / Users/Kamus/mysql_data/conf.d/s3-docker

Then copy the original my.cnf files of each database to conf.d/s1-docker and other three directories, and finally form the following directory structure.

The purpose of designing these directory structures is to mount the data file directory into the / var/lib/mysql directory inside the container through the volume option when running the docker container, and mount the directory where the my.cnf parameter files are located into the / etc/mysql/conf.d directory inside the container, so that all the key files needed to start the MySQL database instance are stored in the host, while the docker container itself only provides MySQL software images. This is more flexible than storing all files inside the docker image.

Design the host name and IP address of the docker container

The hostname used in the various parameters configured for MGR in the MySQL instance running locally in macOS is often the localhost,IP address, and the communication port between nodes in the 127.0.0.1 Magr cluster is often specified on different ports on the native IP. If we refer to the construction guide in the official documentation, we use 24901MGR 24902MGR 24903.

Group_replication_group_seeds= "127.0.0.1VOR 24901127.0.0.1VOF 24902127.0.1VOF 24903"

Now that we have moved the database to the container, we can redesign more meaningful hostnames and separate IP addresses.

When you start the Docker container, you can specify the hostname and static IP through the hostname and ip options. You can refer to the complete command that ran the Docker container at the end.

If you want to specify a static IP for the Docker container, you must use a manually created network, which can be created with the following command, such as here we created a network named mynet.

$docker network create--subnet=192.168.100.0/24 mynet

Once created, you can view the newly created network through the network ls command. You can see that the default is bridging.

Design docker container resource footprint limit

Resource restriction is a necessary requirement for running multiple docker containers on the same host. Although in this article we are only testing the relocation of MySQL instances to docker containers, container resource limits are still planned. Since the host itself only has 2 CPU and 2GB memory, make the following planning.

When you start the Docker container, you can specify resource limits for CPU and memory through the cpus and memory and memory-swap options. You can refer to the complete command that ran the Docker container at the end.

Modify the my.cnf configuration of each database according to the above settings

Because we designed the container to have different IP addresses at startup, we need to specify these IP addresses in the MGR section of the initialization parameters of the MySQL instance running in the container. The following are the settings for the my.cnf file.

The contents of the my.cnf files of the three nodes are exactly the same except for the loose-group_replication_local_address parameter.

For the my.cnf file used by the first container s1-container, this parameter is:

Loose-group_replication_local_address= "192.168.100.11 purl 24901"

For the my.cnf file that will be used by the second container s2-container, this parameter is:

Loose-group_replication_local_address= "192.168.100.12 24901"

For the my.cnf file that will be used by the third container s3-container, this parameter is:

Loose-group_replication_local_address= "192.168.100.13purl 24901"

Run the MySQL docker container

Since the MGR cluster requires that each database can directly access the hostnames of other nodes at the host layer, on ordinary hosts, you only need to modify the / etc/hosts file to add entries corresponding to the hostnames and IP addresses of other nodes, but the / etc/hosts in the container cannot be modified manually. Even if the entry is manually added, the entry will be lost as long as the container is restarted.

To deal with this problem, you can use the add_host option when running the container. In a real production environment, we can choose to configure a dedicated DNS server to correspond to the IP and hostname (the DNS server can also be a Docker container, which will be added in our subsequent tests. This article will temporarily solve the problem by means of add_host).

The final complete command to run the three docker containers is as follows:

After all the containers are started, you can check the running status of the container through the docker ps command, or view the output of the MySQL database log through the docker logs command. It's very convenient.

Start MGR

Log in to the operating system of the Docker container with the following command, then enter the MySQL instance and start MGR. We are currently setting up MGR in Single Primary mode. Start the first Primary instance first.

Then start the second read-only instance in turn.

Start the third read-only instance.

At this point, we have moved the entire set of MGR clusters that were originally running in macOS to the docker container. Finally, the following system architecture is realized.

Enmo College, a subsidiary of Yunhe Enmo (Beijing) Information Technology Co., Ltd., is committed to providing professional and high-level oracle database and big data training services, mining and training big data and database talents. Enmo College provides a full range of big data and database technology training, including individual practical skills training, personal certification training, and enterprise internal training. ACE-level super teachers, equipped with professional laboratories, immersion learning and training, professional laboratories, equipped with professional teaching assistants to guide training. Can quickly integrate into the circle of experts, rich in resources in the industry, and quickly accumulate workplace contacts. Oracle database courses include: Oracle DBA practical class, Oracle OCM examination, Oracle OCP examination and so on.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.