Shulou(Shulou.com)06/01 Report--

From the perspective of micro-services, the infrastructure services of containers can be divided into three layers:

Micro-service basic layer

Micro-service construction layer

Micro service access layer

Rancher's service discovery is based on rancher-dns, and the created stack&service will generate corresponding DNS records, which can be accessed by users through the corresponding rules, so that micro-services can be connected directly with their service names without knowing their IP addresses.

The basic layer of micro-service mainly provides computing, storage, network and other basic resources for the container. Host computing resources are mainly encapsulated with docker-machine to provide related services; container storage is accessed through Convoy components, and currently has the best storage adaptation to NFS protocol; the network between containers is implemented through rancher-net components, which currently supports ipsec overlay and CNI standard network plug-ins in the Rancher1.2 version.

In the micro-service construction layer, in addition to the main program of the micro-service itself, it also needs some additional auxiliary tools to improve the architecture of the corresponding micro-service. Rancher-dns to implement the service discovery mechanism; rancher-metadata can flexibly and dynamically inject some configuration data into the micro-service container; healthcheck to ensure the high availability of micro-service; at the same time, we also need a micro-service packaging tool to ensure that micro-service can be pulled up and run in any environment.

Micro-service access layer. Currently, service exposure access is mainly based on DNS binding or load balancer VIP. Rancher provides external-dns and external-lb framework that allows advanced users to hack their own scenario requirements. External-dns supports internal DNS servers (such as bind9) as well as common DNS services (such as route53), while external-lb currently supports F5 devices. In addition, Rancher's built-in load balancing is based on Haproxy and supports L4-L7.

This time, I will share it by explaining the principles of concept introduction and interspersing with some practical cases.

Rancher's metadata service rancher-metadata is very flexible, and more complex micro-service architectures can be decoupled to a certain extent through metadata. In particular, confd+metadata will be unexpectedly useful. Please refer to http://niusmallnan.github.io/_build/html/_templates/rancher/confd_metadata.html for this part.

Rancher's healthcheck implementation is based on Haproxy and supports TCP/HTTP. When unhealthy is triggered, it is executed according to the preset policy:

Do nothing

Rebuild based on the number of containers in scale

Guarantee the number of containers with at least x healthy

When a healthcheck policy is created for a service, the Haproxy service is started on the agent node where the container is located in the service, and the configuration of the healthcheck is converted into the configuration of the Haproxy. Backend is added as shown in the figure, and the corresponding ip is the ip of container. In addition, the stats scket of the Haproxy is exposed so that the status information of the backend can be read.

We can communicate with Haproxy sock in external programs and get the status information of the relevant backend. Because we set the check mechanism in the Haproxy, the status of the backend will be updated automatically.

The host-api component running on Rancher Agent reads the backend state information through Haproxy sock, and at the same time, through the rancher event mechanism, the state information push is sent to rancher-server,rancher-server according to the previously set healthcheck policy to control the relevant rancher agent to perform container recreate operations.

If the microservice itself is a built-in service port (TCP/HTTP), then the healthcheck rules are easy to set, as long as you fill in the form items normally. However, in practical applications, some micro-services do not have ports exposed, it may just be a program that interacts with DB, so we will consider that the service itself does not have a big code modification, so we need to use some gadgets to assist.

In addition to the well-known LB method, the access entry of micro-services can also be achieved by binding DNS. Especially in private cloud scenarios, the use of internal DNS is actually more concise than simply using LB to expose IP+Port, because this does not need to take into account the service IP changes caused by the container drift of micro-services.

Rancher provides an external-dns framework, https://github.com/rancher/external-dns, which can translate service service addresses into DNS records.

In private cloud scenarios, many industry users internally use F5 hardware load balancer to expose service access addresses. We try our best to control the transformation of micro-services at the program architecture level, while the original network structure should not be changed as far as possible, which will lead to the problem of how to integrate F5 devices in micro-service scenarios.

Let's take an application scenario as an example. Four micro-service exposure ports in the production environment are 9070, 9071, 9072 and 9073, respectively. For disaster recovery, you need to deploy two sets of environment master environment and standby environment, with three hosts in each environment. All database layers are placed in a non-container environment, and all services are finally exposed and accessed through F5.

Implement this application scenario based on Rancher: create two environment belonging to the master environment and the slave environment. Because they are different ENV, the two environments are isolated from the computing storage network level. Create a label of 4 service,service plus global=true under one stack,stack in each environment to ensure that the service is running on each host. At the same time, the service port of the service is directly exposed to the host through portmap, and the external F5 devices configure VIP on these HostIP+Port.

For the key F5 settings, we should consider that it is best to be able to set them dynamically. Rancher provides an external-lb framework https://github.com/rancher/external-lb to solve this problem, and the driver of F5 is also included in it. It also obtains the IP+Port information of micro-services through rancher-metadata components.

Floating IP is originally the product of Iaas, but Caas is still in the process of continuous evolution, and the internal network structure of enterprises still needs the mechanism of floating IP. The main scenario is the rule setting of the firewall, which is usually aimed at a certain IP, and this IP means that no matter how the back-end service changes, it requires that the IP cannot be changed, otherwise the firewall rules will be constantly modified, which is the most unacceptable to the enterprise operation and maintenance staff.

In essence, we need to solve the problem that after the microservice-related container drifts, the exposed IP remains the same.

Ruiyun Zhihe, a partner of Rancher, has proposed a floating IP solution, which is a very good idea.

Of course, we can also use Cattle's own mechanism to solve this problem flexibly.

The access entry of micro-service uses the built-in rancher-lb method. You can use the label scheduling method to make the rancher-lb container only fall on the fixed host, and the relevant firewall only needs to configure the fixed host IP.

Finally, let's take a look at a more appropriate general-purpose micro-service deployment structure.

The sidekick container is used to separate the functions of the main service. Configuration files and logs are handled by different containers, while ensuring integrity, and can be fully expanded and cloned. The configuration files are uniformly placed in the NFS storage of the convoy connection to ensure the consistency of the configuration files. The logging container sends the logs to the ELK log system, which is convenient for centralized query and management. Healthcheck is essential to ensure the availability of services. Externally, the built-in Rancher LB is used to expose access.

Q & A

Does the Q:convoy plug-in now have catalog that supports ceph or gluster?

A:gluster 's catalog was there before, but there have been some problems, and now it has been removed. Convoy does not currently support ceph.

Q: in the last architecture, does it mean that the log is stored in a volume, and then the application and the log service are mounted at the same time?

A: logs are sent to ELK through the logging container for collection.

Q: did you send it directly with the log plug-in?

A:log driver can only send standard input and output, while the architecture in the figure is more suitable for the traditional form of writing log files, sending the contents of log files to elk.

Q: is the log stored in a sidekick easy to parse and send by the logging container?

A: that's right.

Q: so does this volume need to be uploaded to the mount local directory? Or is it already in the form of a container?

A: one container is enough.

Q: is there no other way for convoy to use the local host disk of a cluster for the time being?

A:Rancher has a longhorn that you call a scenario that is still in the process of iteration.

Original source: Rancher Labs

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.