Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of how to deploy Docker virtualization, the content is detailed and easy to understand, the operation is simple and fast, and has a certain reference value. I believe you will gain something after reading this article on how to deploy Docker virtualization. Let's take a look at it.

Docker Virtualization about Docker

This section describes some of the features of Docker virtualization.

Docker is an open source software project that automates application containerization deployment, thus providing an additional layer of software abstraction and an automatic management mechanism for operating system layer virtualization on the Linux operating system. -From wiki

In the process of coming into contact with Docker, you will learn more or less about the virtualization of Docker. The most common way of introduction is to compare the differences between Docker and virtual machines. The author also gives a comparison table of the two, so as to expand on them in detail later.

Virtual machine Docker container isolation hardware-level process isolation operating system-level process isolation system each virtual machine has a separate operating system each container can share the operating system (shared operating system kernel) startup time takes a few minutes, size virtual machine mirror GB container is lightweight (KB/MB) boot virtual machine image is difficult to find pre-built docker capacity The virtual machine can be easily migrated to a new host container to be destroyed and recreated instead of moving creation speed to create a VM that takes a relatively long time to create container resources in a matter of seconds using the GB level MB level

Virtualization in Docker depends on Windows and Linux kernels. Hyper-V will be enabled on Windows, and namespace and cgroups will be relied on on Linux. Therefore, Docker will not be introduced here, and the virtualization technology on Linux will be mainly introduced later.

Traditional virtualized deployment

The traditional way of virtualization is virtualization at the level of hardware abstraction, which is characterized by a high degree of virtualization.

The advantages of traditional virtualization are:

1. Virtual machines are isolated from each other through virtualization technology.

2. Multiple virtual machines can be deployed on physical machines to improve resource utilization

3. Application resource allocation and capacity expansion can be configured directly through the virtual manager

4. Support snap, virtual machine cloning, rapid deployment, disaster recovery and mitigation

Disadvantages of traditional virtualized deployment:

1. High resource consumption, which requires additional operating system images, GB-level memory and dozens of GB storage space.

2. The startup speed is slow, and the operating system in the virtual machine needs to be started before the application can be started.

3. Great impact on performance, application = > virtual machine operating system = > physical machine operating system = > hardware resources

Linux Virtualization

This section briefly explains the principles of Docker implementation, from which readers can learn how Linux isolates resources and how Docker is isolated.

As we know, the operating system schedules resources in a single process, and modern operating systems set resource boundaries for processes, and each process uses its own memory area, and there is no memory mixing between processes. In the Linux kernel, there are cgroups and namespaces that define boundaries for processes and isolate them from each other.

Linux-Namespace

In the container, when we use the top command or the ps command to look at the machine's processes, we can see the Pid of the process, each process has a Pid, and all the containers of the machine have a basis of Pid = 1, but why is there no conflict? Processes in containers can use all ports at will, while different containers can use the same port. Why is there no conflict? These are all manifestations that resources can set boundaries.

In Linux, namespace is a resource isolation technology provided by the Linux kernel, which can isolate the network and process environment in the system, so that the system resources in each namespace are no longer global. Currently, there are six kinds of resource isolation, and Docker basically isolates the container environment on these six kinds of resources.

Readers can memorize this table a little bit, which will be used later.

Namespace system call parameters isolate content UTSCLONE_NEWUTS hostname and domain name IPCCLONE_NEWIPC semaphore, message queue, shared memory PIDCLONE_NEWPID process number NetworkCLONE_NEWNET network device, network stack, port MountCLONE_NEWNS file system mount UserCLONE_NEWUSER users and user groups

[info] about Mount

Namespace's Mount enables you to mount subdirectories as root directories.

Unshare

In Linux, the unshare command line program can create a namespace and isolate various resources in the namespace based on parameter creation, here we can simply create a namespace using this tool.

To gain a deeper understanding of namespace in Linux, we can execute it in Linux:

Unshare-pid / bin/sh

-- pid isolates only processes.

This command is similar to docker run-it {image}: {tag} / bin/sh. When we execute the command, the terminal enters a namespace and executes the top command to view the list of processes.

PID USER PR NI VIRT RES SHR S% CPU% MEM TIME+ COMMAND 1 root 20 0 160188 5488 S 0.0 0.4 9 systemd 2 root 20 00 00 S 0.0 0.00: 00.08 kthreadd 3 root 0-20 000 I 0.0 0.00: 00.00 rcu_gp 4 root 0-20 000 I 0.0 0.00: 00.00 rcu_par_gp

As you can see, the process PID starts at 1, indicating that in this namespace, it is isolated from the host process.

In this command, only the process is isolated, because the network is not isolated, so when we execute the netstat-tlap command, the network of this namespace is connected to the network of other namespaces.

Before executing the unshare command, use the pstree command to view the process tree:

Init ─┬─ 2 * [init ─── init ─── bash] ├─ init ─── init ─── bash pstree ├─ init ─── init ─── fsnotifier-wsl ├─ init ─── init ─── server ─── 14 * [{server}] └─ 2 * [{init}]

To facilitate the comparison, we use unshare-- pid top to create a namespace, and compare after executing the unshare command:

$> pstree-lhainit ├─ init │ └─ init │ └─ bash │ └─ sudo unshare-pid top │ └─ top ├─ init │ └─ init │ └─ bash │ └─ pstree-lha ├─ init │ └─ init fsnotifier-wsl init init │ └─ bash ├─ init │ └─ init │ └─ server-- port 29687-- instance WSL-Ubuntu │ └─ 14 * [{server}] └─ 2 * [{init}]

In namespace, look at what the top displays and find:

PID USER PR NI VIRT RES SHR S CPU MEM TIME+ COMMAND 1 root 20 0 1904 1136 1020 S 0.0 MEM TIME+ COMMAND 08.38 init

Through the process tree, we can see that the processes in different namespace are in different tree branches, and their process PID is independent of each other. Its function is similar to runc in Docker.

In the unshare command, the-- pid parameter creates a namespace that isolates the process. In addition, you can isolate a variety of system resources:

Mount: the namespace has a separate mount file system

Ipc:Inter-Process Communication (interprocess communication) namespace with independent semaphores, shared memory, etc.

Uts: namespaces have independent hostname, domainname

Net: independent network, for example, each docker container has a virtual network card

Pid: independent process space. All processes in the space Pid start from 1.

User: there is an independent user system in the namespace. For example, the root in Docker is different from the host user.

Cgroup: independent user grouping

Simple implementation of process isolation with Go

Earlier we used unshare to create namespaces, where we can try to use Go to call the namespace of the Linux kernel to create isolated resource spaces through programming code.

Examples of Go code are as follows:

Package mainimport ("log"os"os/exec"syscall") func main () {cmd: = exec.Command ("sh") cmd.SysProcAttr = & syscall.SysProcAttr {Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWIPC | syscall.CLONE_NEWNS | Syscall.CLONE_NEWNET | syscall.CLONE_NEWPID | syscall.CLONE_NEWUSER } cmd.Stdin = os.Stdin cmd.Stdout = os.Stdout cmd.Stderr = os.Stderr if err: = cmd.Run () Err! = nil {log.Fatalln (err)}}

[info] prompt

Resource isolation such as UTS has been mentioned earlier, and readers can refer to the instructions in the table to understand the role of Cloneflags against the code.

In this code, we start the sh command in Linux to start a new process that will use the new IPC, PID, and so on isolation.

Readers can enter the new namespace by executing go run main.go in Linux.

So much for the introduction to namespace.

Cgroups hardware resource isolation

Namepace mentioned earlier is a logical form that makes processes invisible to each other, resulting in environment isolation, which is the same as the daily use of Docker containers, such as isolating root directories, isolating networks, isolating process PID, and so on.

Of course, in addition to the isolation of the Docker processing environment, it can also limit the physical resources used by each container, such as CPU, memory, and so on. This limitation of hardware resources is based on the cgroups of the Linux kernel.

Example of a parameter that limits the amount of resources that a container can use in Docker:

-m 4G-- memory-swap 0-- cpu-period=1000000-- cpu-quota=8000000

Cgroups, which stands for control groups, is a mechanism provided by the Linux kernel for physical resources that can be used by processes.

Cgroups can control a variety of resources. In cgroups, each resource restriction function corresponds to a subsystem. You can use the command to view:

Mount | grep cgroup

[info] prompt

The functions of each subsystem are summarized as follows:

Blkio-this subsystem sets restrictions on input / output access to inbound and outbound block devices, such as USB, etc.

Cpu-this subsystem uses a scheduler to provide cgroup task access to CPU.

Cpuacct-this subsystem generates automatic reports on CPU resources used by tasks in cgroup.

Cpuset-this subsystem assigns a single CPU and memory node to tasks in cgroup.

Devices-this subsystem allows or denies tasks in cgroup access to the device.

Freezer-this subsystem suspends or resumes tasks in cgroup.

Memory-the subsystem sets limits on the memory used by tasks in cgroup and generates automatic reports about it.

Net_cls- allows the Linux traffic controller (tc) to identify packets originating from a specific cgroup task.

Net_prio-this subsystem provides a way to dynamically prioritize network traffic for each network interface.

Ns- Namespace subsystem.

Perf_event-this subsystem identifies the cgroup membership of the task and can be used for performance analysis.

For more information, please refer to redhat documentation.

We can also use the lssubsys command to view the subsystems supported by the kernel.

$> lssubsys-acpusetcpucpuacctblkiomemorydevicesfreezernet_clsperf_eventnet_priohugetlbpidsrdma

[info] prompt

Ubuntu can use the apt install cgroup-tools installation tool.

In order to avoid too much space, readers only need to know that Docker limits the usage of container resources, the number of CPU cores and other operations. The principle is cgroups in the Linux kernel, which I will not repeat here.

Talk about Virtualization

This section will talk about virtualization from an underlying point of view.

Theoretical basis computer hierarchy

From the point of view of language, a general-purpose computer system composed of software and hardware can be regarded as a hierarchical structure composed of multi-tier machine level according to function.

From a language point of view, the hierarchical structure of the computer system can be shown in the following figure.

We usually use laptops, Android phones, tablets, Linux servers, etc., although different machines have different systems and some hardware, but their system structure is the same. From transistors and registers in CPU to CPU instruction sets, to operating systems and assemblers, general-purpose computers in use today basically have this structure.

Let's explain the main characteristics of different levels.

The lowest level of the computer is the hard-connected logic level, which is composed of gate circuits, triggers and other logic circuits, which is characterized by the use of very small components, representing 0,1 in the computer.

Microprograms are written using microinstructions. A microprogram is a machine instruction, which is usually executed directly by hardware. It can represent the simplest operation. For example, an addition instruction consists of a plurality of logic elements to form an adder, whose components are composed as shown in the following figure (an 8-bit full adder in the figure).

The traditional machine language machine level is the instruction set of the processor, and the instruction set that we are familiar with, such as X86, ARM, MIPS, RISC-V and so on, is at this level. A program written by a programmer using instructions in an instruction set, which is interpreted by a lower-level microprogram.

The machine layer of the operating system is seen from the basic functions of the operating system. The operating system needs to be responsible for managing the software and hardware resources in the computer, such as memory, devices, files and so on. It is the interface between software and hardware. The common operating systems are Windows, Linux, Unix and so on. The language used at this level is the machine language, that is, the binary code composed of 0 and 1, which can be directly recognized and executed by the computer.

As the name implies, the assembly language machine layer is the location of the assembly language, which is related to the processor, and the assembly language set used by the same type of processor is consistent. The assembly language needs to be transformed into the equivalent binary code object program by the assembly language program. Because the resources in the computer are managed by the operating system, assembly language needs to be carried out under the control of the operating system.

To the high-level language machine layer, we use C, C++ and other programming languages, high-level languages are close to human thinking.

Software and hardware are equivalent.

Some functions of the computer can be realized either by hardware or by software. That is, software and hardware are functionally equivalent.

Is a function implemented in hardware or software?

Hardware implementation: high speed, high cost, poor flexibility, low memory consumption.

Software realization: low speed, low replication cost, good flexibility and high memory consumption.

Virtualization technology uses software to realize the functions realized by the original hardware, and they are different in performance, price and difficulty. A function can be realized either by hardware or by software, or by a combination of both, which may be considered according to a variety of manpower costs, R & D difficulties, R & D cycle and so on.

Virtualization

Virtualization (technology) or virtual technology is a kind of resource management technology, which abstracts and transforms various physical resources of a computer (CPU, memory, disk space, network adapter, etc.) and can be divided and combined into one or more computer configuration environments.

Different levels of virtualization

In many books and articles, we should learn about the comparison between virtual machines and Docker, and the advantages of Docker. After packaging images through Docker, we can run elsewhere at any time without worrying about machine compatibility. But the virtualization of Docker does not make Linux run Windows containers, nor does Windows run Linux containers, let alone x86 machines run arm instruction set binaries. However, VMware can run mirrors of Linux and Mac on Windows, but WMWare cannot be built by MIPS instructions on Linux systems.

Both Docker and VMware can achieve varying degrees of virtualization, but they are not arbitrary. Their degrees of virtualization are very different because they are virtualized at different levels.

[Info] prompt

Many virtualization software may have multiple levels of virtualization capabilities, not just at one level.

In instruction set-level virtualization, from the point of view of the instruction system, it is to implement the instruction system of another machine on one machine. For example, QEMU can emulate processors such as ARM32/64, Godson, MIPS, etc., on X64 machines.

The degree of virtualization lies in the ratio of hardware implementation to software implementation. Generally speaking, the greater the proportion of hardware, the stronger the performance, and the greater the proportion of software, the greater the flexibility, but the performance will decline. Different levels of implementation will also affect performance, compatibility, and so on. With the increasingly fierce computer performance, to a large extent, resulting in excess performance; coupled with the increasingly difficult hardware research and development, more and more difficult to break through, non-hardware virtualization will be more and more widespread.

This is the end of the article on "how to deploy Docker Virtualization". Thank you for reading! I believe you all have a certain understanding of the knowledge of "how to deploy Docker Virtualization". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.