Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you a case study of Serverless and OpenKruise deployment optimization. The article is rich in content and analyzed and described from a professional point of view. I hope you can get something after reading this article.

Project background

The underlying layer of SAE uses Kubernetes architecture and uses two kinds of resource pools: Shenlong naked metal security container and ECI. The applications that users run in SAE will be mapped to the corresponding resources in Kubernetes.

By collecting all the K8s events online, the creation life cycle of Pod is analyzed by node-by-node and phase-by-stage time-consuming statistical analysis. Take Dragon Node as an example, the proportion of each stage is shown in the figure:

As can be seen from the figure, the entire creation life cycle of pod includes scheduling, pulling and creating init containers, pulling user business images, creating and starting containers, and so on. Its time-consuming is mainly focused on scheduling and pulling user images. The reason is that the overall scheduling link of SAE Dragon node takes a long time, and the image time is mainly due to the time of pulling and decompressing the image, especially in the case of large-capacity image deployment.

Implementation plan

The SAE team investigates and analyzes the scheme from many aspects such as long-term architecture planning and using scenario versatility, and considers using the deployment strategy of upgrading in place instead of rebuilding and upgrading strategy, so as to avoid rescheduling in the deployment process and reduce the overall time-consuming.

The so-called in-place upgrade means that only one or more container versions in Pod are updated without affecting the upgrade of the entire Pod object and other containers, and ensure that ip and node do not change during the upgrade process. Within Alibaba, most e-commerce applications are released using in-place upgrade in the native cloud environment. This in-place upgrade mode greatly improves the efficiency of application release and saves scheduling and network allocation. It takes time to mount disks and pull images. Through the analysis of online SAE user history deployment records, it is found that most of them only update image / package deployment applications, that is to say, the ability to upgrade in place is very suitable for landing in SAE products.

The advantages of upgrading in place to SAE are:

Avoid rescheduling and sidecar container reconstruction. The whole deployment takes time only by pulling and creating business containers.

No scheduling is required, and new images can be cached on Node in advance to improve resiliency efficiency.

Ip can be kept unchanged to avoid latency perceived by dependent components such as registries due to ip changes

Reduce the pressure of rebuilding pod on schedulers, registries, upstream and downstream business.

At the same time, the OpenKruise project has contributed in-place upgrade capabilities to open source through CloneSet / AdvanceStatefulSet. CloneSet is one of the core workload provided in OpenKruise, which is mainly for stateless applications. It provides more efficient, definite and controllable application management and deployment capabilities, and supports rich strategies such as elegant in-place upgrade, specified deletion, configurable release order, parallel / grayscale release, and so on, which can meet more diversified application scenarios. The comparison between CloneSet and native Kubernetes workload features is shown in the figure:

SAE decided to adopt CloneSet as the new application load. On the one hand, it makes use of its ability to upgrade in place to improve the overall deployment efficiency of the application, on the other hand, it also combines the power of OpenKruise open source to create a common standard large-scale use of stateless application load. For incremental applications, SAE uses CloneSet to deploy user applications by default, and combines the maximum number of unavailable instances and elegant upgrade duration to ensure that the published traffic is lossless, while for inventory applications, SAE will use rolling upgrades based on finite state machines for online migration.

The solution has achieved remarkable results after it was launched. Within a month, nearly 1000 applications have been deployed using CloneSet, and the number of in-situ upgrades is twice that of reconstruction upgrades, and the deployment efficiency is 42% higher than that of native K8s. Combined with image cache, users deploy applications to the container within seconds. In the future, SAE will produce more advanced capabilities of OpenKruise, while combining with user scenarios, constantly polish stability and best practices to give back to open source.

Figure: native K8s deployment application reconstruction upgrade strategy VS SAE deployment application native upgrade strategy

The above is the example analysis of Serverless and OpenKruise deployment optimization shared by Xiaobian. If you happen to have similar doubts, you might as well refer to the above analysis. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.