Shulou(Shulou.com)06/03 Report--

Author | Xu Di, Zhang Xiaoyu

Introduction: this article is based on the sharing of Xu Di and Zhang Xiaoyu at KubeCon NA 2019. Sharing will start from the following aspects: first, a brief introduction to what a Sidecar container is; second, a common scenario for several Alibaba economies and how they address these challenges.

Introduction to Sidecar

Sidecar containers are nothing new. It is a design pattern, mainly used to do some auxiliary work, such as network connectivity, download and copy files, and so on; if you are familiar with Docker Swarm, you will find that Docker Ambassador is actually Sidecar.

As shown above, Service Consumer and Redis Provider are strongly coupled and deployed on the same node. If there is a problem with Redis Provider at this time, you need to connect to another Redis instance, reconfigure and restart Service Provider.

So after the introduction of Ambassador, the problem becomes relatively simple, you just need to restart Redis Ambassador here, and you don't need to make any changes to Service Consumer.

Of course, in this mode, you can also communicate across nodes, as shown in the following figure. This allows Service Consumer and Redis Provider to be deployed on different nodes. To some extent, it is easy to decouple the two services.

Sidecar case sharing 1. What can Sidecar containers be used for?

Generally speaking, Sidecar containers can:

Log agents / forwards, such as fluentd;Service Mesh, such as Istio,Linkerd; agents, such as Docker Ambassador; probing: check that some components are working properly; other auxiliary work, such as copying files, downloading files, etc. That's all?

In fact, Sidecar is more and more accepted and used more and more widely. Sidecar containers are usually deployed in the same Pod as business containers (non-Sidecar containers), share the same lifecycle, and provide auxiliary functions for business containers. This is a very good model that can greatly decouple applications, support heterogeneous components and reduce technical barriers.

But at present, the management of Sidecar by Kubernetes is not perfect, and it is more and more unsatisfying for our use, especially in the production environment using Sidecar.

3. Sequence dependence of several typical cases

Suppose we inject multiple Sidecar into a Pod, but there are interdependencies between the Sidecar or between the Sidecar and the business container. In the following example, we need to start the proxy Sidecar container to establish a network connection so that mysql client can connect to the remote mysql cluster and expose the service locally. The later master's business container can work properly.

# 1 proxy_container (sidecar) # 2 mysql_client#3 svc_container

Of course, some people will think that this place can be solved by changing the image startup script to delay startup and so on. However, these methods are too intrusive and not conducive to expansion, and it is difficult to configure them accurately.

Sidecar management

Let's take a look at another case. The Sidecar container and the business container are coupled in the same Pod and share the same life cycle. Therefore, it is very inappropriate to control the Sidecar container alone, such as updating the image of Sidecar.

For example, we have injected many Pod with Sidecar containers such as Istio Proxy, which are currently running well. But what if we want to upgrade the Proxy image at this time?

If we follow the official documentation of the Istio community, we need to re-inject these Sidecar containers. Specifically, you need to delete the original Pod and generate a new Pod (some Pod associated with the workload will be automatically generated by the corresponding workload controller).

What if we have a lot of such Pod to deal with? Through the command line, it is too inconvenient and error-prone. Scalability is a problem with your own code, which needs to be changed frequently.

And there is another problem, we certainly will not upgrade all the Sidecar at once, there must be a grayscale process, that is, only part of the Sidecar should be upgraded, what should we do at this time?

Community Progress 1. Upstream community

Here we are very grateful to Joseph Irving (@ Joseph-Irving) for proposing a Sidecar kep that distinguishes whether it is a Sidecar container by defining a LifecycleType.

In the future, you only need to mark it in Pod Spec as follows:

Name: sidecarContainerimage: foolifecycle: type: Sidecar

The startup order of containers in Pod is in the order of initialization container-> Sidecar container-> business container.

The implementation of the kubelet side of the above kep is in progress.

In order to support more usage scenarios of Sidecar, we propose PreSidecar and PostSidecar, which are used to launch before and after the business container, respectively.

For specific usage scenarios, please see our PR.

Why do we think Sidecar should distinguish between front and rear?

This is because in some scenarios, we need the Sidecar container to take precedence over the application container to start to help with some preparation. For example, distribute certificates, create shared volumes, or copy and download other files.

In other scenarios, we need some Sidecar containers to start after the application container. Considering the factors of decoupling and version management, we divide the application into two parts, the application container focuses on the business itself, and some data and personalized configuration are placed in the Sidecar container. Typically, the two containers will share a storage volume, and the rear Sidecar container will update and replace some default and outdated data.

Of course, considering the more complex scenarios in the future, we may also do DAG choreography for the startup sequence of containers, which depends on the actual needs of production.

two。 How to deal with Ant Financial Services Group and Alibaba

In order to solve the management of Sidecar, we need a finer-grained workload to facilitate our management. This workload, which we call SidecarSet, is now open source and available for production. You can visit the OpenKruise project and learn about some of our new developments in the roadmap of the project. The OpenKruise project currently has three production workload available, namely, Advanced StatefulSet, BroadcastJob, and SidecarSet. The other two workload (AdvancedHPA and PodHealer) are under development and will be available soon. Please look forward to it. Related to the use of Demo, you can watch the video of Lachlan Evenson.

The definition of SidecarContainer in spec is the definition of corev1.Container in the Kubernetes code base. With an additional labelSelector, you can easily manipulate the specified container group. We support rolling upgrade (RollingUpdate), which allows users to upgrade Sidecar in batches, and also provides pause function to pause Sidecar upgrade in case of emergency.

If you simply upgrade the image of Sidecar, the SidecarSet controller will only patch the original pod, which is very convenient to upgrade the image with one click.

Other challenges

In the process of production practice, we have also found some other challenges, and we are still looking for a better solution.

1. Resource management of Sidecar container

Generally speaking, the resources occupied by Sidecar containers are relatively small, so should this resource be counted into the whole pod? Or can you just share the resources of the business container directly? The same Sidecar is used with different application containers, and how to accurately allocate resources to the Sidecar container needs to be considered.

2. Fault tolerance of Sidecar containers

Generally speaking, Sidecar containers are non-primary containers, so when there is a problem with such containers, such as liveness live detection, will it also affect the state of the main container or the state of the entire pod? Or, when there is a problem with the update of the Sidecar image, do you want to directly mark the problem with the entire pod? Of course, there are some other challenges, and we just listed a few generic ones. For these challenges, we need everyone to pool their collective wisdom and find a more reasonable solution.

Summary

As Sidecar is more and more widely used in production environment, more and more attention should be paid to its management. Although Sidecar and the business container are deployed in the same Pod, they are essentially auxiliary containers. This article introduces the current typical use cases of Sidecar and the challenges it faces. At the same time, it works with the upstream community to implement Ali economy's technology solutions in the community to help more users.

Brief introduction of the author:

Xu Di Ant Financial Services Group technical expert: responsible for ant financial cloud PaaS platform construction, Kubernetes community veteran, core code base contribution community top 50

Zhang Xiaoyu Aliyun technical expert: responsible for the ecological construction of Alibaba Cloud native application container platform, mainly designing and developing solutions related to node stability and resource utilization, and is also an enthusiastic member and contributor to the Kubernetes community.

"Alibaba Cloud Native focus on micro-services, Serverless, containers, Service Mesh and other technology areas, focus on cloud native popular technology trends, cloud native large-scale landing practice, to be the best understanding of cloud native developers of the technology circle."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.