Shulou(Shulou.com)06/01 Report--

How to compare the docker container cluster management platform, many novices are not very clear about this. In order to help you solve this problem, the following editor will explain it in detail. People with this need can come and learn. I hope you can get something.

Containerization and micro-services are the hottest topics at present. Not long ago, the author (it is said that the title "author" is no longer appropriate because the pen is no longer needed, because the keyboard is more appropriate, it is more appropriate to call "key person") to participate in a micro-service monitoring Session in QCon Shanghai. I had to listen to the whole session in the crowded aisle. With more and more containers to be managed, the cluster management platform for containers has become a rigid requirement!

Docker Swarm

Swarm is a new container cluster management tool released by Docker in early December 2014. It can turn multiple hosts into a virtual Docker host to manage. Swarm is developed in the GE language and is open source, and all of its source code can be found on github. Swarm uses standard Docker API to give Docker users a seamless cluster experience. In July 2016, Swarm has been integrated into Docker Engine.

Function

Docker Swarm provides API and CLI to manage clusters running Docker, and its functionality is not fundamentally different from using native Docker. But you can bring good scalability by adding Node. In theory, you can have an infinite Docker host by adding Node.

Swarm does not provide UI, if you need UI, you can install UCP, but unfortunately, this UCP is paid for.

Architecture

The architecture of Swarm is not complex and can be said to be very simple. Manager is responsible for the scheduling of the container, Node is responsible for the operation of the container, and Node runs the communication between Docker Daemon and Manager through HTTP. Docker Client uses the functionality of Docker through the standard Docker API exposed on Manager.

Swarm's cluster coordination and business discovery can support different third-party components, including:

Consul

Etcd

ZooKeeper

Docker Hub

If you are not familiar with the concept of cluster coordination, you can refer to my other blog, "using Python for distributed system Coordination (ZooKeeper,Consul, etcd)"

There are three basic container scheduling strategies for Swarm:

Spread containers are distributed on different nodes as much as possible

Binpack containers are distributed on the same node as much as possible

Random containers are distributed on random nodes

The highly available configuration of a Swarm cluster is easy, as shown in the following figure:

Manager is configured in different AWS AZ, and Primary manager is elected by leader election.

Multiple nodes are distributed in different AZ, and the Consul/etcd/ZooKeeper also needs to be redundant.

Characteristics

The characteristic of Docker Swarm is that the configuration and architecture are very simple. Using Docker native API, you can well integrate the ecosystem of Docker.

Kubernetes

Kubernetes is an open source container application management system developed by Google, which is used to manage the deployment, maintenance and expansion of applications. Using Kubernetes, you can easily manage applications that run across machines in containerization. Kubernetes is also developed in the go language, and the source code can be found on github.

Kubernetes originates from Google's internal container management system Borg, after years of production environment, so the function is very powerful.

Function

Kubernetes mainly provides the following functions:

Use Docker to wrap (package), instantiate (instantiate), and run (run) the application.

Run and manage containers across machines in a clustered manner.

Solve the communication problem between Docker cross-machine containers.

Kubernetes's self-healing mechanism makes the container cluster always run in the desired state of the user.

High availability and expansion of applications

Support for online upgrade of applications (Rolling Update)

Support for cross-cloud platform (IaaS) deployment

In order to support these functions, Kubernetes has done a lot of abstract concepts, so when you start using Kubernetes, you need to learn a lot of new concepts, including:

Pod

Pod is the basic operation unit of Kubernetes, which combines one or more related containers into a Pod. Usually the containers in Pod run the same application, or related applications. Pod contains containers that run on the same Minion (Host) as a unified snap-in, sharing the same volumes and network namespace/IP and Port spaces.

Job

Job is an application with a short life cycle. Generally, it will only restart in the event of an error. You can extend Job by configuring the concurrency and number of runs of Job.

Service

Service is an application with a long life cycle and will be restarted at any exit. High scalability and high availability can be achieved by configuring Service replication (Replica).

Recplica Controller

Replication Controller ensures that there are a specified number of pod replicas (replicas) running in the Kubernetes cluster at any time. If there are less than the specified number of pod replicas (replicas), Replication Controller will start a new Container, otherwise it will kill the excess to keep the number unchanged. Replication Controller uses a predefined pod template to create a pods. Once created, the pod template does not have any association with the created pods. You can modify the pod template without any impact on the created pods, or you can directly update the pods created through Replication Controller.

In addition to these core concepts, Kubernetes provides other concepts to support the operation and maintenance of applications, including:

Label

The objects in the system are managed by Label.

Namespace

Grouping objects and resources that can be used to support multi-tenancy

Config Map

Provides global configuration data storage

In short, the function is powerful, the system concept is various, and it is more complex.

Kubernetes supports the installation of UI's addon to manage the entire system:

Architecture

The following figure shows the basic architecture of Kubernetes:

Master

Master defines the main declaration of the Kubernetes cluster Master/API Server, and Client (Kubectl) calls Kubernetes API to manage the entry of Pods, Services, Minions, and containers, the main components of Kubernetes. Master is composed of API Server, Scheduler and Registry.

Scheduler collects and analyzes the resource (memory, CPU) load of all Minion nodes in the current Kubernetes cluster, and then distributes the newly created Pod to the available nodes in the Kubernetes cluster accordingly. Since once the resources of the Minion node are allocated to the Pod, these resources cannot be allocated to other Pod unless the Pod is deleted or exited, so the Kubernetes needs to analyze the resource usage of all Minion in the cluster to ensure that the workload distributed does not exceed the scope of the resources currently available to the Minion node.

Minion

Minion is responsible for running Pod,Service,Jobs, and Minion communicates with Master through Kubelet. Proxy solves the problem that external networks can access application services provided by containers across machine clusters.

Etcd

Responsible for cluster coordination and service discovery

Characteristics

Kubernetes provides a lot of application-level management capabilities, including high availability and high scalability, of course, in order to support these functions, its architecture and concepts are more complex, of course, I think in order to get these functions, value!

Apache Mesos

Mesos is an operating system for software-defined data centers in which resources across data centers are uniformly managed. The original intention of Mesos is not to manage containers, but with the development of containers, Mesos has added the functions of containers. Mesos can uniformly manage the computing resources of different machines, just like the same operating system, for running distributed applications.

Mesos originated from Google's data center resource management system Borg. You can learn more about the origin of Borg and its impact on Mesos from this article in WIRED magazine.

Function

The main features of Mesos include:

Highly scalable and highly available

Customizable two-level scheduling

Provide extension of API for application

Cross platform

The following figure shows the management interface of Mesos:

Architecture

The basic structure of Mesos is as follows

Master

Master is responsible for the coordination of resources and the management of Slave. Master coordinates all Slave and determines available resources for each node, aggregates reports of all available resources across nodes, and then issues resource invitations to Framework registered to Master (as a client of Master). Mesos implements a two-level scheduling architecture that can manage many types of applications. The first level of scheduling is the Master daemon, which manages the Slave daemon running on all nodes in the Mesos cluster. The cluster consists of physical or virtual servers for tasks that run applications, such as Hadoop and MPI jobs. The second level of scheduling consists of "components" called Framework.

Slave

Salve runs the executor (Executor) process to run tasks.

Framework

Framework includes Scheduler and Executor processes, where executors run on each node. Mesos can communicate with different types of Framework, and each Framework is managed by the corresponding application cluster.

Framework can choose to accept or reject resource invitations from master according to the needs of the application. Once the invitation is accepted, Master coordinates Framework and Slave, dispatches tasks on the participating node, and executes them in the container to enable multiple types of tasks

ZooKeeper

Zookeeper is responsible for the coordination of the cluster, the leadership election of Master, etc.

Characteristics

Mesos is more mature than Kubernetes and Swarm, but Mesos mainly deals with operating system-level abstraction, not specifically designed for containers. If users are out of the container, they have to integrate other applications, such as Hadoop,Spark,Kafka, and Mesos is more appropriate. Mesos is a more heavyweight cluster management platform with richer functions, of course, many functions are based on various Framework.

Mesos scalability is very good, supporting up to 50000 nodes, if the scalability requirements are very high, Mesos is the best choice.

AWS ECS

ECS (Amazon EC2 Container Service) is a highly scalable and high-performance container cluster service developed by Amazon. It is easy to run container applications and services on a managed cluster of Amazon EC2 instances. His biggest advantage is that he does not need to manage the machines and networks of the data center on his own in the cloud.

Function

ECS inherits the high scalability and high availability of AWS services, and security can be guaranteed.

Based on basic container management, ECS uses the concepts of Task and Service to manage applications.

Task is similar to Docker Compose in that it uses a JSON to describe the application to run. Service is a higher level of abstraction, which contains multiple running instances of task. By modifying the number of task instances, you can control the scaling of the service. At the same time, Service can guarantee that a specified number of Task is running, and restart the failed Task when an error occurs.

Architecture

The following is an architectural diagram of ECS:

Using the familiar concepts of AWS such as EC2,ELB and security groups, AWS users can easily manage your container cluster. And there is no need to pay fees other than the initial basic resources.

You can connect to the ECS cluster through ECS agent as a Container. ECS Agent is developed using Go and has been opened.

We are not clear about ECS's scheduling policy, but AWS provides an example if you inherit a third-party scheduling policy.

Through Cloud Watch Log, we can easily monitor the whole cluster.

Characteristics

If you are a heavy user of AWS, ECS is a good choice, because you can run your container cluster on the cloud of AWS, which is very easy to manage.

Compare

Here is a simple comparison of several platforms:

SwarmKubernetesMesosECS basic functions Docker native cluster management tool open source container cluster tool to provide application-level deployment, maintenance and expansion functions data center-based operating system cloud-based high availability, high expansion container cluster high-level abstraction without

Pod

Job

Service

No Task

Service application scalability without support Framework support for application high availability no support for Framework support for cluster coordination and service discovery

Etcd

ZooKeeper

Consul

EtcdZooKeeper/ scheduling policy (Schedule) built-in, scalable built-in two levels, scalable monitoring Logging DriverHeapter,ELK Addon built-in CloudWatch user interface

CLI

API

UCP UI

CLI

API

UIAPI

UI

CLI

API

AWS Console

Does the development language GoGoJavaNA open source support the maximum number of management nodes

Official 1000

Unofficial 5000

Official 1000

Unofficial 2000

10000/

Of the four platforms, Swarm is the lightest and simplest, suitable for users with a large number of Docker instance environments. Kubernetes adds a lot of application-level functions, which are suitable for rapid application deployment and maintenance. Mesos is the heaviest and is suitable for environments where there is already a considerable mix of applications and containers. ECS is recommended to AWS users or cloud users who do not want to manage their own data centers.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.