Shulou(Shulou.com)06/02 Report--

Link: https://blog.51cto.com/14320361/2457143

1.docker introduction

1.1 what is docker

Docker is an open source application container engine based on the Go language and open source according to the Apache2.0 protocol.

Docker allows developers to package their applications and dependencies into a lightweight, portable container and publish them to any popular Linux machine.

What problem can 1.2docker solve?

1.2.1 efficient and orderly use of resources

Machine resources are limited

Multiple applications have to be deployed on a single machine

Applications are isolated from each other

No preemption of resources can occur between applications, and each application can only use the resources applied for in advance.

1.2.2 compile once, run everywhere

Similar to java code, applications and dependent environments are built once and can be run everywhere.

Introduction to the underlying principle of 1.2.docker

Simple understanding of namespace and cgroup of 1.2.1Linux

Namespace: a namespace similar to JAVA

Controll groups: controll (system resource) (for) (process) groups

Namespace in 1.2.2Linux

In the Linux system, there can be multiple users and multiple processes at the same time, so the coordination and management of their operation can be solved through process scheduling and schedule management, but the overall resources are limited, how to reasonably allocate the limited resources (process number, network resources, etc.) to each user's process?

The Linux Namespaces mechanism provides a resource isolation scheme. PID,IPC,Network and other system resources are no longer global, but belong to a specific Namespace. Resources under each namespace are transparent and invisible to resources under other namespace. So at the operating system level, there will be multiple processes with the same pid. In the system, there can be two processes with the process number 0 namespace 1 and 2, and there is no conflict between them because they belong to different processes. At the user level, you can only see the resources that belong to the user's own namespace. For example, using the ps command can only list the processes under your own namespace. This makes each namespace look like a separate Linux system.

Namespaces establish different views of the system. For each namespace, the user should look like a separate Linux computer with its own init process (PID is 0). The PID of other processes increases successively. Both An and B spaces have init processes with PID 0. The processes of the child container are mapped to the processes of the parent container, and the parent container can know the running status of each child container. The child container is isolated from the child container.

| |

Namespace

| |

Relevant kernel versions introduced

| |

Isolated global system resources

| |

Isolation effect in Container context

Mount namespacesLinux 2.4.19 file system mount point

Mount the top-level directory of one file system to a subdirectory of another file system, making them a whole, called mount. This subdirectory is called the mount point.

Mount namespace is used to isolate the mount point of the file system, so that different mount namespace have their own mount point information, and different namespace will not affect each other, which is very useful for building users or containers' own file system directories.

| |

| | UTS namespaces | Linux 2.6.19 | nodename and domainname | |

UTS,UNIX Time-sharing System namespace provides isolation of hostnames and domain names. The ability to make a child process have an independent hostname and domain name (hostname) is used in Docker container technology, so that the docker container is regarded as an independent node on the network, not just a process on the host.

| |

| | IPC namespaces | Linux 2.6.19 | specific inter-process communication resources, including System V IPC and POSIX message queues | |

IPC, whose full name is Inter-Process Communication, is a way of inter-process communication under Unix/Linux. IPC has shared memory, semaphores, message queues and other methods. So, in order to isolate, we also need to isolate the IPC so that only processes under the same Namespace can communicate with each other. If you are familiar with the principle of IPC, you will know that IPC needs to have a global ID, that is, it is global, which means that our Namespace needs to isolate this ID from other Namespace processes.

| |

| | PID namespaces | Linux 2.6.24 | process ID digital space (process ID number space) |

PID namespaces is used to isolate the ID space of processes, so that the ID of processes in different pid namespace can be repeated and do not affect each other.

PID namespace can be nested, that is, there is a parent-child relationship, and all new namespace created in the current namespace are child namespace of the current namespace. The process information in the namespace of all descendants can be seen in the parent namespace, while the process information in the ancestor or brother namespace cannot be seen in the child namespace.

| |

| Network namespaces | started from Linux 2.6.24 and completed on Linux 2.6.29 | Network-related system resources | each container has its own independent network device, IP address, IP routing table, / proc/net directory, port number and so on. This also allows the same application in multiple containers on a host to be bound to port 80 of their respective containers. | |

| | User namespaces | starts from Linux 2.6.23 and completes on Linux 3.8) | ID space for users and groups |

User namespace is used to isolate Linux resources related to user permissions, including user IDs and group IDs.

This is the most complex namespace implemented so far, because user is closely related to permissions, and permissions are also related to the security of the container, so security problems will occur if you are not careful.

In different user namespace, the user ID and group ID of the same user can be different. In other words, a user can be an ordinary user in the parent user namespace and a superuser in the child user namespace.

1.3Namespace (namespace) is used to isolate the container [root@localhost] # docker run-it-- name test centos / bin/bash// into the container

[root@41052cceb473 /] # ls// check that it is similar to the host, all linked from the host.

[root@41052cceb473 /] # uname-r bind / check the kernel, it's the same as the host

If the virtual machine service requires the kernel version, this service is not suitable to be implemented in docker, because docker is the kernel of the shared host and can use virtual machines such as kvm.

[root@localhost ~] # docker pull ubuntu// downloads a Ubuntu using docker

[root@localhost ~] # docker images// check

[root@localhost] # docker run-it ubuntu:latest / bin/bash// enter the ubuntu environment root@afbee6750865:/# ls / check it out

Root@48c8dd7b098e:/# uname-r _ paw / check the kernel

Docker itself does not occupy any port, it usually runs in the background, no matter what operation (system, service) is done in docker, for docker, they are just a process.

Run-  centos system (nginx,web)

Busybox: deception layer. Spoofing virtual machines in docker is in their own independent environment.

Decoupling: uncoupling, conflict.

Coupling: conflict phenomenon.

1.4 Namespace operation

/ proc / sys: virtual file system, pseudo directory file

[root@localhost ~] # cd / proc/ [root@localhost proc] # ls

[root@localhost proc] # echo $$/ / the current process number 3864 [root@localhost proc] # cd 3864 [root@localhost 3864] # cd ns [root@localhost ns] # ll// can be seen in a flash.

[root@localhost ns] # ls

IPC: shared memory, message queuing

MNT: Mount point, file system

NET: network stack

PID: process number

USER: users, groups

UTS: hostname, domain name

The six isolations of namespec realize the isolation between container and host, container and container.

/ / create a user and set the password

[root@localhost ns] # useradd bdqn [root@localhost ns] # echo 123.com | passwd-- stdin bdqn [root@localhost ns] # id bdqn

View docker processes

[root@localhost ns] # docker ps-a

[root@localhost ns] # docker start test// starts centos [root @ localhost ns] # docker exec-it test/ bin/bash// to enter the docker container [root@41052cceb473 /] # id dbqn

[root@41052cceb473 /] # echo $$

2.1linux cgroup introduction

2.1.1 Why do you need cgroup when you have namespace:

The Docker container uses linux namespace to isolate its running environment, making the processes in the container look as if they were running in a separate environment. However, runtime environment isolation is not enough, because these processes still have unlimited access to system resources, such as network, disk, CPU, and memory. As for its purpose, on the one hand, it is to prevent it from consuming too many resources and affecting other processes; on the other hand, when the system resources are exhausted, the linux kernel will trigger OOM, which will make some killed processes become innocent scapegoats. Therefore, to make the processes in the container more manageable, Docker uses Linux cgroups to limit the system resources that processes in the container are allowed to use.

2.1.2 principle

Linux Cgroup allocates resources to user-defined groups of tasks (processes) running in the system-such as CPU time, system memory, network bandwidth, or a combination of these resources. You can monitor the cgroup configured by the administrator, deny cgroup access to certain resources, and even dynamically configure cgroup on the running system. Therefore, controll groups can be understood as controller (system resource) (for) (process) groups, that is, it allocates and controls system resources with a set of processes as the target. It mainly provides the following functions:

Resource limitation: limit resource usage, such as memory usage limits and file system cache limits.

Prioritization: priority control, such as CPU utilization and disk IO throughput.

Accounting: some audits or some statistics, the main purpose is for billing.

Controll: suspend the process and resume the execution process.

With cgroup, system administrators have more specific control over the allocation, prioritization, rejection, management, and monitoring of system resources. It can better allocate hardware resources according to tasks and users, and improve the overall efficiency.

In practice, system administrators typically use CGroup to do the following:

Isolate a set of processes (for example, all processes in nginx) and limit the resources they consume, such as the core that binds the CPU.

Allocate enough memory for this group of processes

Assign appropriate network bandwidth and disk storage limits to this set of processes

Restrict access to certain devices (by setting a whitelist of devices)

2.1.3Cgroup (control group) operation resource limit, docker's occupation of resources [root@localhost] # cd / sys/fs/cgroup/// to cpu, memory limit directory [root@localhost cgroup] # ls

[root@localhost cgroup] # cd cpu [root@localhost cpu] # ls

Cpu.shares: weight

Tasks: the number in this file records the process number. PID

[root@localhost cpu] # cd docker/ [root@localhost docker] # ls

[root@localhost docker] # cat tasks// is empty [root@localhost docker] # cd 41052cceb4739fa8e0ddd2ffa733a78cd1043b3fdff874cd266c009391a34d70/ [root@localhost41052cceb4739fa8e0ddd2ffa733a78cd1043b3fdff874cd266c009391a34d70] # ls

[root@localhost41052cceb4739fa8e0ddd2ffa733a78cd1043b3fdff874cd266c009391a34d70] # cat tasks

[root@localhost] # docker ps

Four major functions:

1) Resource restrictions: cgroup can limit the total amount of resources used by the process group

2) priority allocation: through the number of cpu time slices allocated and the size of the hard disk IO bandwidth, it is actually equivalent to controlling the priority of the process.

3) Resource statistics: group can count the usage of system resources, such as gpu usage time, memory usage, etc., for postpaid billing. At the same time, it also supports suspending kinetic energy, that is to say, all resources are limited by cgroup, and resources can not be used. Note that it does not mean that our program cannot be used, knowledge cannot use resources, and is in a waiting state.

4) process control: suspend, resume and other operations can be performed on the process group.

2.1.4 memory limit

Container memory consists of two parts: physical memory and swap

You can control the amount of container memory used by parameters:

-m or-- memory: set the memory usage limit

-- memory-swap: set the usage limit of memory + swap

For example:

Run a container and limit the container to 200m of memory and 100m of swap

[root@localhost] # docker run-it-m 200m-- memory-swap 300m centos:7 [root@fba67fec2718 ~] # cd / sys/fs/cgroup/ [root@fba67fec2718 cgroup] # ls

[root@fba67fec2718 cgroup] # cd memory/ [root@fba67fec2718 memory] # ls [root@fba67fec2718 memory] # cat memory.limit_in_bytes// View memory usage limits (in bytes)

[root@fba67fec2718 memory] # cat memory.memsw.limit_in_bytes// View swap Partition, memory + swap limit

Run a new container without restricting it

[root@localhost ~] # docker run-it centos:7 [root@5be901bfb093 /] # cd / sys/fs/cgroup/memory/ [root@5be901bfb093 memory] # cat memory.limit_in_bytes// View memory limit

[root@5be901bfb093 memory] # cat memory.memsw.limit_in_bytes// View swap Partition, memory + swap limit

Comparing an unlimited container, we will find that if there is no limit to memory after running the container, it means that there is no limit.

2.1.5 CPU usage

Set the weight of the container's use of cpu through-c or-- cpu-shares. If it is not set, the default is 1024.

For example:

There is no limit

[root@localhost ~] # docker run-it-- name containerA centos:7// has no limit, 1024 [root@8683d8ff8234 /] # cd / sys/fs/cgroup/cpu [root@8683d8ff8234 cpu] # cat cpu.shares

Limit the use weight of CPU to 512

[root@localhost] # docker run-it-- name containerB-c 512 centos:7// limits the weight of CPU to 512 [root@d919d906295d /] # cd / sys/fs/cgroup/cpu//. You can see that cpu has been restricted.

2.1.6 Block IO of the container

Read and write to the disk.

In Docker, you can control the IO of the read and write disk of the container by setting weights and limiting bps and iops.

Bps: the amount of data read and written per second byte per second

IopS: the number of IO per second io per second.

By default, all containers can read and write disks equally, or you can change the container's blocklO priority through the-blkig-weight parameter.

-- device-read-bps: displays the bps that reads a device.

-- device-write-bps: displays the bps written to a device.

-- device-read-iops: displays the iops that reads a device.

-- device-write-iops: displays the iops written to a device.

Limit testA to the container, and write to / dev/sda the bps of this disk is 30MB

[root@localhost ~] # docker run-it-- name testA-- device-write-bps / dev/sda:30MB centos:7 [root@60e59e96fc16 /] # time dd if=/dev/zero of=test.out bs=1M count=800 oflag=direct// is imported from / dev/zero and then exported to the test.out file, each time the size is 1m, a total of 800times. Oflag=direct is used to specify the directlQ method to write the file, so that-- device-write-bps will take effect.

[root@60e59e96fc16 /] # du-h test.out

There are no restrictions on docker

[root@localhost ~] # docker run-it-- name testc centos:7 [root@5bf5f3d60d0e /] # time dd if=/dev/zero of=test.out bs=1M count=800 oflag=direct

[root@5bf5f3d60d0e /] # du-h test.out

What is the difference between 3.Docker virtualization and normal virtualization?

Virtual machines:

Our traditional virtual machine needs to simulate the whole machine, including hardware, and each virtual machine needs its own operating system. Once the virtual machine is turned on, all the resources pre-allocated to it will be occupied. Each virtual machine includes applications, necessary binaries and libraries, and a complete user operating system

Docker:

Container technology is to share hardware resources with our host and the operating system can realize the dynamic allocation of resources.

The container contains the application and all its dependent packages, but shares the kernel with other containers. The container runs as a separate process in user space in the host operating system.

| |

The virtual machine and container are above the hardware and operating system, the virtual machine has Hypervisor layer, and Hypervisor is the core of the whole virtual machine. He provides a virtual running platform for the virtual machine and manages the operating system running of the virtual machine. Each virtual machine has its own system and system libraries as well as applications.

The container does not have a Hypervisor layer, and each container shares hardware resources and operating system with the host, so the performance loss caused by Hypervisor does not exist in the linux container.

However, virtual machine technology also has its advantages, which can provide a more isolated environment for applications, and will not cause any problems to the host because of the loopholes in the application. It also supports virtualization across operating systems, for example, you can run windows virtual machines under the linux operating system.

From the perspective of virtualization, the traditional virtualization technology is the virtualization of hardware resources, while the container technology is the virtualization of processes, which can provide lighter virtualization and realize the isolation of processes and resources.

From the perspective of architecture, Docker has two layers less than virtualization, eliminates the hypervisor layer and GuestOS layer, uses Docker Engine for scheduling and isolation, and all applications share the host operating system, so Docker is lighter than virtual machines in volume, superior to virtualization in performance and close to bare-metal performance. From the perspective of application scenarios, Docker and virtualization have their own areas of expertise, and they have their own advantages and disadvantages in software development, testing scenarios and production operation and maintenance scenarios.

Specific comparison:

Docker startup fast belongs to the second level. Virtual machines usually take a few minutes to start. Docker requires fewer resources, docker virtualizes at the operating system level, docker containers interact with the kernel, with little performance loss, and performance is better than virtualization through the Hypervisor layer and the kernel layer. Docker is more lightweight, and the architecture of docker can share a kernel and a shared application library, taking up very little memory. In the same hardware environment, Docker runs far more images than virtual machines. Very high utilization of the system compared with virtual machines, docker isolation is weaker, docker belongs to the isolation between processes, virtual machines can achieve system-level isolation; security: the security of docker is also weaker. The tenant root of Docker is the same as the host root. Once the user in the container is promoted from ordinary user permission to root permission, it directly has the root permission of the host, and then it can perform unlimited operations. The root permission of the virtual machine tenant and the root virtual machine permission of the host are separated, and the virtual machine uses the ring-1 hardware isolation technology such as VT-d of Intel and VT-x, which can prevent the virtual machine from breaking through and interacting with each other, but the container does not have any form of hardware isolation so far, which makes the container vulnerable. Manageability: docker's centralized management tools are not yet mature. Various virtualization technologies have mature management tools, such as VMware vCenter to provide complete virtual machine management capabilities. High availability and recoverability: docker's high availability support for business is achieved through rapid redeployment. Virtualization has proven mature guarantee mechanisms such as load balancing, high availability, fault tolerance, migration and data protection. VMware can guarantee 99.999% high availability of virtual machines to ensure business continuity. Fast creation and deletion: virtualization creation is minute-level, Docker container creation is second-order, and the fast iteration of Docker determines that a lot of time can be saved in development, testing and deployment. Delivery and deployment: virtual machines can achieve the consistency of environment delivery through images, but image distribution cannot be systematized. Docker records the container construction process in Dockerfile, which enables rapid distribution and rapid deployment in clusters.

3.1.1 introduction to docker structure

Infrastructure (Infrastructure).

The main operating system (Host Operating System). All major Linux distributions can run Docker. For MacOS and Windows, there are also ways to "run" Docker.

Docker daemon (Docker Daemon). Instead of Hypervisor, the Docker daemon is a background process that runs on the operating system and is responsible for managing the Docker container.

All kinds of dependencies. For Docker, all dependencies of the application are packaged in the Docker image, and the Docker container is created based on the Docker image.

Application. The source code of the application and its dependencies are packaged in a Docker image, and different applications require different Docker images. Different applications run in different Docker containers, and they are isolated from each other.

The Docker daemon can communicate directly with the main operating system and allocate resources to individual Docker containers; it can also isolate containers from the main operating system and separate containers from each other. It takes several minutes for the virtual machine to start, while the Docker container can start in milliseconds. Since there is no bloated slave operating system, Docker can save a lot of disk space and other system resources; virtual machines are better at complete isolation of resources.

Link: https://blog.51cto.com/14320361/2457143

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.