Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to analyze the Swarm Kubernetes Marathon choreography engine, the content of the article is of high quality, so the editor will share it for you as a reference. I hope you will have some understanding of the relevant knowledge after reading this article.

Docker Native Orchestration

Basic structure

Docker Engine 1.12 integrates a native orchestration engine to replace previously stand-alone Docker Swarm projects. The Docker native cluster (Swarm) also includes (Docker Engine\ / Daemons), which allows the native docker to act as a manager or worker node of the cluster at will. The work node (worker) will be responsible for running the container task you started, while the management node will get the task description from the user, integrate the existing cluster and maintain the cluster state. And you can have a recommended number of no more than seven people to manage nodes to support high availability. The management nodes will ensure the strong consistency of the internal environment state of the cluster. They achieve state consistency and backup based on the Raft protocol. Like all mandatory algorithms, having more manager management nodes (manager) means more performance overhead. At the same time, the fact that you maintain consistency within the manager nodes makes it easier for you to manage the cluster without external dependence on the native orchestration of Docker.

Usability

Docker extends the concept of using single-node Docker to Swarm clusters. If you are familiar with docker, it is quite easy for you to learn swarm. You can also migrate the docker environment to a running swarm cluster fairly easily. If the existing system uses Docker Engine, you can smoothly cut the Docker Engine to Swarm without changing the existing system.

You only need to run on one of the docker nodes to create a cluster using docker swarm init, and add any other nodes you want to add to the cluster you just created via docker swarm join. You can then use the same Docker Compose templates and the same docker command-line tools as you would in a separate docker environment.

Feature set

Docker Native Orchestration is supported with Docker Engine and Docker Compose. You can still create volumes (volume) and define open ports (expose) functions using the same links (link) as before on a single node. In addition to these, there are two new concepts, service (services) and network (networks).

A docker services (service) is a set of containers that manage a certain number of containers that are always running on a node, and it supports automatic replacement if one of the containers dies. There are two types of services, replication (replicated) or global (global). The replicated service maintains a specified number of containers in the cluster (meaning the service is free to scale), and the global service runs an instance of the container on each cluster node. To create a replication (replicated) service, use the following command.

Docker service create\-name frontend\-replicas 5\-network my-network\-p 80:80/tcp nginx:latest.

You can use docker network create-driver overlay NETWORK_NAME to create a named overylay network. Using a specified overylay network, you can create an isolated, flat, encrypted, cross-node host virtual network within your container.

You can use constraints plus labels to do some basic container scheduling. Using the constraints parameter, you can add the launch container on the node associated with the specified label to the service.

Docker service create\-name frontend\-replicas 5\-network my-network\-- constraint engine.labels.cloud==aws\-- constraint node.role==manager\-p 80:80/tcp nginx:latest.

In addition, you can use reserve CPU and reserve memory tags to define the amount of resources used by each container in the service, so that when multiple services are started on the cluster, containers are placed to minimize resource contention.

-- limit-cpu value Limit CPUs (default 0.000)\-- limit-memory value Limit Memory (default 0B)\-- reserve-cpu value Reserve CPUs (default 0.000)\-- reserve-memory value Reserve Memory (default 0B)

You can use the following command for basic rolling deployment. This will update the container image of the service, but do so, 2 containers at a time, with an interval of 10 seconds between the two groups. However, health check and automatic rollback are not supported.

Docker service update\-name frontend\-replicas 5\-network my-network\-- update-delay 10s\-- update-parallelism 2\-p 80:80/tcp nginx:other-version.

Docke supports persistent volume mounts using volume-driven (volume drivers) programs, and supports the mount option for its service create command extension using Native orchestration. The command that adds the following code snippet to the above will mount the NFS into your container. Note that this requires that NFS has been set up on a host outside of Docker, and some other drivers have added support for Amazon EBS volume drivers or Google container engine volume drivers to work without host settings. In addition, because these features are not well documented, you may need to test and refer to github issue to run in the docker project.

-- mount type=volume,src=/path/on/host,volume-driver=local,\ dst=/path/in/container,volume-opt=type=nfs,\ volume-opt=device=192.168.1.1:/your/nfs/path

Kubernetes

Basic structure

Conceptually, the only thing that Kubernetes is similar to Swarm is that it also uses a Master node of RAFT to ensure strong oneness. At the same time, ETCD is used in Kubernetes to achieve the goal. In addition, it will use an external extended network layer, such as overlay, weave network, etc. Using these external tools, you can start the main components of Kubernetes; such as the API server (API Server), controller manager (Controller Manager), and scheduler (Scheduler), which typically run as Kubernetes pod on the main (master) node. In addition to these, you need to run kubelet and kubeproxy on each node (node). The Worker nodes runs only Kubelet and Kubeproxy and a network layer provider such as flanneld.

In the above settings, the main function of kubelet is to obtain the desired state of pod\ / container on the node (what container to run, the number of copies to run, how to configure the network or storage, and so on). Kubelet will use the controller manager operation on the master node to pod\ / containers on the specified node. The scheduler is responsible for allocating and balancing resources, placing containers on the work node (worker node) with the most available resources. The API controller is responsible for your local kubectl CLI to set commands to. Finally, the kubeproxy component is used to provide load balancing and high availability for the services defined in Kubernetes.

Usability

Setting up Kubernetes from scratch is a difficult process because it requires setting up etcd, network plug-ins, DNS servers and certification authorities for a system operation. You can browse here for detailed documentation for building Kubernetes from scratch, but fortunately you have integrated all these settings in Rancher. In the previous article, we have described how to set up kubernetes in Rancher.

In addition to the initial setup, Kubernetes still has some steep learning curves because it uses its own terminology and concepts. Kubernetes uses resource types, such as Pods, Deployments, Replication Controllers, Services, Daemon sets, and so on, to define deployment. None of these concepts are part of the Docker dictionary, so you need to be familiar with them before you start creating your first deployment. In addition, some of these concepts conflict with Docker, such as Kubernetes Services is not conceptually equivalent to Docker Services (Docker Services maps more closely to Deployments in the Kubernetes world), and you use kubectl instead of docker CLI for cluster interaction, and you must also use Kubernetes configuration files instead of docker compose files.

It is not a bad thing for Kubernetes to have such a set of concepts independent of the core Docker. Kubernetes provides a richer feature set than Docker. However, Docker is also rapidly adding more features to compete with Kubernetes, which have different implementations and conflicting concepts. This will certainly lead to a heavy phenomenon of similar functional but competitive solution situations between CoreOS and rkt. But today, the goals of Docker Swarm and Kubernetes are still very different use cases (Kubernetes is better suited for large production deployments with a service-oriented architecture with dedicated cluster management teams), but as Docker Native Orchestration matures, the time will surely come to compete with it.

Feature set

The full feature set of Kubernetes is too large to be covered in this article, but we will discuss some basic concepts and some interesting differences. First, Kubernetes uses the concept of Pods as its basic unit of scaling, rather than a single container. Each pod is a set of containers (the setting can be 1) that they always start on the same node, share the same volume and assign a virtual IP (VIP) so that they can be addressed in the cluster. The Kubernetes specification file for a single pod is shown below.

Kind: Pod metadata: name: mywebservice spec: containers:-name: web-1-10 image: nginx:1.10 ports:-containerPort: 80

Next, expand the Deployment settings, which loosely map services to the Docker native orchestration. You can extend the deployment of containers that run the number of requests like services in Docker Native. It is important to note that Deployment is only similar to replication services locally in Docker, for example, Kubernetes uses the daemon assembly concept (Daemon Set) to support its equivalent global scheduling (globally scheduled) service. The deployment also supports health checks using HTTP or TCP reachability or custom exec commands to determine whether the container\ / pod is running properly, as well as automatic rollback deployment using health checks to ensure the success of each pod deployment.

Kind: Deployment metadata: name: mywebservice-deployment spec: replicas: 2 # We want two pods for this deployment template: metadata: labels: app: mywebservice spec: containers:-name: web-1-10 image: nginx:1.10 ports:-containerPort: 80

Next, it provides simple load balancing for deployment. All pod in the deployment will be registered with the service (service) when the service enters and exits, and the service abstracts out multiple Deployment, so if you want to run a rolling deployment, you will register two Kubernetes Deployment with the same service, and then gradually add pod to one of them while reducing the pod from the other Deployment. You can even deploy blue and green, where you can point the service to the new Kubernetes Deployment at once. Finally, services are also useful for service discovery in your Kubernetes cluster, where all services in the cluster get VIP, and well-integrated DNS servers like docker link-style environment variables are exposed to all pod in the cluster.

In addition to basic services, Kubernetes supports Jobs, Scheduled Jobs, and and Pet Sets,Jobs to create one or more pod and wait until they are terminated. Homework make sure you have a specified number of pod to complete your homework. For example, you can start a job (job) and start processing an hour of business intelligence data on the last day. It will start a job containing 24 pod from the previous day, and once all of them have been run, the job will complete. The scheduled job name implies that a scheduled job is a job that runs automatically on a given schedule. In our example, we may make our BI processor a daily scheduled job. Scheduled jobs are ideal for assembling batch-style workloads that are not services that are always started, but tasks that need to be run and then cleaned up automatically.

Another extension that Kubernetes provides to basic services is Pet Sets, which is an object type introduced by Kubernetes 1.3 to improve support for stateful services. Pet Sets supports workloads of stateful services that are often very difficult to containment. This includes databases and applications with real-time connectivity requirements. Pet Sets provides a stable hostname setting for each "Pet" in the collection. Pet is indexable; for example, pet5 will be addressable independent of pet3, and if the pet3 container\ / pod dies, it will restart on a new host with the same index information (index) and hostname (hostname).

Pet Sets also provides stable storage of persistent volumes, that is, if PET1 dies, it will restart on another node and remount the original data volume. In addition, you can use NFS or other network file systems to share volumes between containers, even if they are started on different hosts. This solves one of the most difficult problems in the transition from a single host to a distributed Docker environment.

Pet Sets also provides peer discovery (peer-discovery), the usual service that you can find other services (through Docker link, etc.), however, it is not possible to find other containers within the service. This makes services based on the gossip protocol, such as Cassandra and Zookeeper, very difficult to start.

Finally, Pet Sets provides startup and sorting, which is a necessary condition for persistent, extensible services such as Cassandra. Cassandra relies on a set of seed nodes, and when you extend the service, you must make sure that the seed node is the first to start and the last to delete. At the time of this writing, Pet Sets is a feature of Kubernetes, because without this support, it is almost impossible for a persistent stateful workload to run on a production scale of Docker.

Kubernetes also provides namespace (namespaces), security management (secrets management) and automatic extension (auto-scaling) support for isolated workloads at the cluster level. All of these features mean that Kubernetes also supports large, diverse workloads, which Docker Swarm currently does not have.

Marathon (Marathon)

Basic structure

Another common choreography setting choice for large clusters is to run Marathon on top of Apache Mesos. Mesos is an open source cluster management system that supports workloads on a variety of hosts. Mesos is a Mesos agent that runs on each host in the cluster and reports its available resources to the master host. Mesos uses multiple Mesos master to achieve high availability (high-availability), including an active master (called leader or leading master) and several backup master to avoid downtime. The active leader is elected through Apache ZooKeeper, and other nodes in the cluster, including other Master,slave nodes and scheduler driver, are notified. At any time, one of the master nodes is active using the main election process. Master can send tasks to any Mesos agent and report the status of those tasks. Although you can issue tasks through API, the normal approach is to use a framework framework on top of Mesos. Marathon is one such framework framework that provides support for running Docker containers (as well as local Mesos containers).

Usability

Compared to Swarm, Marathon has a rather steep learning curve because it does not share most of the concepts and terminology with Docker. However, the marathon (Marathon) is not rich in features, so it is easier to learn than Kubernetes. The complexity of managing Marathon deployments mainly comes from the fact that it is structured on top of Mesos, so there are two layers of tools to manage. In addition, some of the more advanced features of Marathon, such as load balancing, are only supported by additional frameworks that run on top of Marathon. Some features, such as authentication, are only available when running a marathon on DC\ / OS (when Marathon), which in turn runs on Mesos-which adds another layer of abstraction to the stack.

Feature set

To define a service in Marathon, you need to use its internal JSON format, as shown below. A simple definition, such as the one below, creates a service that runs two nginx container instances.

{"id": "MyService"instances": 2, "container": {"type": "DOCKER", "docker": {"network": "BRIDGE", "image": "nginx:latest"}

A slightly more complete version is shown below, and we now add port mapping and health checks. In port mapping, we specify a container port, which is the port exposed by the docker container. The host port defines which port on the host public interface maps to the container port. If you specify 0 for the host port, a random port is assigned at run time. Similarly, we can optionally specify the service port. Service ports are used for service discovery and load balancing, as described later in this section. With health check, we can now do both scrolling (default) and blue-green deployment

{"id": "MyService"instances": 2, "container": {"type": "DOCKER", "docker": {"network": "BRIDGE", "image": "nginx:latest"portMappings": [{"containerPort": 8080, "hostPort": 0, "servicePort": 9000, "protocol": "tcp"},]}}, "healthChecks": [{"protocol": "HTTP" "portIndex": 0, "path": "/", "gracePeriodSeconds": 5, "intervalSeconds": 20, "maxConsecutiveFailures": 3}]}

In addition to a single service, you can also define services that the Marathon application group (Application Groups) uses for nested tree structures. The advantage of defining an application in a group is the ability to scale the entire group together. This is very useful in micro-service stack scenarios because it is relatively difficult to adjust each service individually. At this point, we further assume that all services will expand at the same rate, and if you need "n" instances of a service, you will get "n" instances of all services.

{"id": "/ product", "groups": [{"id": "/ product/database", "apps": [{"id": "/ product/mongo",...}, {"id": "/ product/mysql",...}]}, {"id": "/ product/service", "dependencies": ["/ product/database"] "apps": [{"id": "/ product/rails-app",...}, {"id": "/ product/play-app",...}]}]}

In addition to defining basic services, Marathon can also schedule constraints based on specified containers, as detailed here, including specifying that each instance of the service must be on a different physical host "constraints": [["hostname", "UNIQUE"]]. You can use the CPU and mem tags to specify the resource utilization of the container. Each Mesos agent reports its overall resource availability, so the scheduler can intelligently place workloads on the host.

By default, Mesos relies on traditional Docker port mapping and external service discovery and load balancing mechanisms. Recent beta features have added support for Mesos DNS based on DNS service discovery or load balancing using Marathon LB.

Mesos DNS is an application running on top of Mesos that queries Mesos API for a list of all running tasks and applications. It then creates DNS records for the nodes on which these tasks are run. All Mesos agents then need to manually update using the Mesos DNS server as their primary Mesos DNS server. Mesos DNS uses a hostname or IP address for Mesos agent to register with master hosts, and port mappings can be queried as SRV records.

Marathon DNS uses the hostname of agent, so you must ensure that the appropriate ports on the host network are open and that conflicts do not occur. Mesos DNS provides a different way to reference stateful payloads, for example, we will be able to use Kubernetes pet sets together. In addition, unlike Kubernetes's VIP mechanism, which has any container addressable within the cluster, Mesos must manually update\ / etc\ / resolve.conf to the Mesos DNS server set and update the configuration when the DNS server changes. Marathon-lb uses Marathon Event bus to track the startup and revocation status of all services. It then starts the HAProxy instance on the agent node to relay traffic to the required service nodes.

The beta version of the marathon (Marathon) supports persistent volumes to external persistent volumes (external persistent volumes). However, both features are in a very primitive state. Persistent volumes only support persistent data volumes when the container is restarted on a single node, but volumes will be deleted. If applications that use them are deleted, the actual data on the disk will not be deleted, and volume must delete them manually. Support for external persistence volumes is limited to having to run on DC\ / OS, and currently only allows your service to extend to a single instance.

Summary

Now we look at the three options for Docker container orchestration: Docker Native (Swarm), Kubernetes, and Mesos\ / Marathon. It is difficult to choose a system to recommend, because the best system is highly dependent on your specific use case requirements, deployment size, and application history. In addition, all three systems are still under extensive development, and some of the features summarized above include test versions that may soon be changed, deleted, or replaced.

Docker Native provides you with the fastest way to adapt, and it has little or no vendor locking problems other than its dependence on Docker. The only dependence on Docker itself is not a big problem, because docker has become the de facto container standard. Given that there is no clear winner in the choreography engine market competition, and Docker native is the most flexible approach, it is a good choice for simple Web and stateless applications, but Docker Native is currently the initial state, if you need to get complex, large-scale applications to production, you need to choose a Mesos\ / Marathon or Kubernetes.

It is not an easy choice between Mesos\ / Marathon and Kubernetes, because both have their own advantages and disadvantages. Of the two, Kubernetes is certainly richer and more mature, but it is also a very stubborn software (translator's note: Kubernetes has some stubborn opinions about how the container is organized and the network enforces some concepts), and we think many of these opinions make sense. At the same time, Kubernetes doesn't have the flexibility of a marathon, especially if you consider the rich history of applications without Docker and containerization that can run on Mesos (such as Hadoop clusters). If you are doing a green area implementation and have no strong intention on how to layout the cluster, or if you have the same needs ideas as Google, then Kubernetes is a better choice. Conversely, if you have large, complex legacy workloads that will gradually move to containerization, then Mesos\ / Marathon is the way to go.

Another problem is scale: Kubernetes has tested thousands of nodes, while Mesos has tested thousands of nodes. If you are starting a cluster with tens of thousands of nodes, you need to use Mesos to achieve the scalability of the underlying infrastructure-but note that extending advanced features (such as load balancing) to this scope will still be retained. However, at that scale, there are few off-the-shelf solutions, and if any, they need to be carefully adjusted and constantly modified.

On how to analyze the Swarm Kubernetes Marathon choreography engine to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.