Shulou(Shulou.com)06/02 Report--

Click to download "different double 11 Technologies: cloud Native practice in Alibaba economy"

This article is excerpted from the book "different double 11 Technologies: cloud Native practice in Alibaba economy". Click on the picture above to download it!

Author

Xie Yuning (Yuqi) Senior Development engineer of Aliyun Container Services

Senior Product Manager, Aliyun Container Services, Luo Jing (Yao Jing)

Deng Jun Aliyun container service technical expert

Introduction: in 2019, Tmall double 11, Alibaba core system for the first time 100% on the cloud. In the face of the world's largest trading peak, Ali Yun withstood the peak of 544000 transactions per second, which is a perfect joint name of "Yunyuan" and "Tmall Global Carnival".

(the picture shows the turnover of Tmall double 11 in 2019)

As one of the important infrastructure in the cloud native field of Alibaba economy, container image service has been faced with large-scale distribution demand as early as the preparation for Singles' Day. In order to better support this requirement, the product is planned and updated in advance, which comprehensively improves the performance, observability and stability in large-scale distribution scenarios. Before the arrival of the new Singles Day holiday, the container mirroring service has added a number of PB mirrored data, with an average of hundreds of millions of mirror pulls per month. At the same time, the product provides functions such as cloud native application delivery chain, fully covering the needs of Alibaba economy and cloud users in the cloud native era.

This article describes how the container image service can improve the product capability to cope with the new developments and challenges in the cloud native application distribution scenario.

New developments and new challenges

With the rapid popularity of cloud native technology, Kubernetes has become the standard for de facto application containerization platform and a "first-class citizen" in the cloud native field.

With a declarative container orchestration and management system, Kubernetes makes software delivery more and more standardized. Kubernetes provides a unified mode of API, which can define resources in the Kubernetes cluster in YAML format files. These resource definitions in YAML format enable Kubernetes to be easily integrated by upstream and downstream systems to complete a series of operations that would otherwise need to be done by non-standardized scripts and manually. At the same time, according to the application delivery scenarios and requirements, the community has derived more series of cloud native application delivery standards in addition to the native YAML format resource definition files, such as Helm Chart, Opeartor, Open Application Model and so on.

(the picture shows the evolution of cloud native application delivery standards)

In addition to cloud native application delivery standards, users also put forward higher requirements for delivery methods. More and more users expect to deliver cloud native applications in a streamlined, automated and more secure way, so the simple 10-node distribution scenario has evolved into ten-thousand-node minute-level multi-link collaborative distribution. Coupled with the global business development, this means that after completing each link in a minute-level time, it is necessary to complete the global distribution, which puts forward higher requirements for the platform to support the distribution of Yunsheng applications.

New practice

By controlling the image size of the container, using P2P to distribute the mirror layer, and optimizing the Registry server, we have greatly optimized the performance of large-scale distribution, and finally achieved the goal of minute-level distribution of ten thousand nodes:

Optimize the container image size and reduce the image transmission cost

Make the basic image, reuse the frequently used applications or environments into the basic image, reduce the number of layers of the image as much as possible, control the number of layers of each change to use multi-stage image construction, and separate the intermediate products from the final products in the image production process to form the most streamlined application image.

Optimize server-side processing performance and improve request response rate

The server uses various methods such as identifying hot images and caching hot data to deal with large-scale mirror Manifest concurrent pull to optimize the client container image layer download mode, reduce the image transmission time for the client to download the container image using dragonfly, and greatly reduce the image Layer download time based on P2P.

(the picture shows the optimization strategy for large-scale distribution of images)

In order to enable enterprise customers with the same needs to enjoy the same distribution capabilities and experiences as above, container image service products officially launched Container Image Service Enterprise Edition (ACR Enterprise Edition) in March 2019. Container Image Service Enterprise Edition provides enterprise-level cloud native asset hosting capability and cloud native application global synchronization and large-scale distribution capability. It is suitable for enterprise-level container customers with high security requirements, multi-regional business deployment and large-scale cluster nodes. In addition, Container Image Service Enterprise Edition also further enhances the ten-node-level distribution collaborative experience of cloud native applications in several aspects, such as hosting, delivery and distribution of cloud native assets.

Cloud native application hosting

At the application delivery level, Container Image Service Enterprise Edition currently supports full lifecycle management of two types of cloud native application assets: container image and Helm Chart.

At the level of access security, the product provides independent network access control function, which can finely control the access policy of public network and VPC network, and only allow sources who meet the policy to access assets, thus further ensuring the access security of cloud native assets.

At the access experience level, the product provides transparent pull plug-ins for container clusters and supports transparent pull of container images to ensure that business pulls images quickly in elastic scenarios without abnormal business update or expansion due to incorrect credential configuration.

(the picture shows that Container Image Service Enterprise Edition supports cloud native application delivery)

Cloud native application delivery

In the process of cloud native application production, users can upload hosted container images, Helm Chart and other cloud native assets directly, or intelligently build container images from source codes (Github, Aliyun Code, GitLab, etc.) through the build feature. At the same time, in order to meet the need of process, automation and more secure way to deliver cloud native applications, CCM Enterprise Edition introduces cloud native application delivery chain. Cloud native application delivery chain starts with cloud native application hosting and ends with cloud native application distribution, and the full link is observable, traceable and independently set. It can realize one application change and globalize multi-scene automatic delivery, which greatly improves the efficiency and security of cloud native application distribution at the process level.

(the picture shows the console creating a cloud native application delivery chain)

Cloud native application delivery step, which supports automatically initiating static security scans and customizing security blocking policies. Once high-risk vulnerabilities in static applications are identified, subsequent deployment links can be automatically blocked. Users can update and optimize to build a new mirror version based on the fix recommendations in the vulnerability report, and initiate delivery again.

Cloud native application distribution

After the distribution of cloud native applications is completed without blocking, cloud native applications have officially entered the global distribution and large-scale distribution link. To ensure the collaborative completion of minute-level distribution of ten thousand nodes, container image services, container services, elastic container instances and other cloud products provide the ultimate end-to-end distribution experience. For global distribution, due to optimization methods such as fine-grained synchronization policy scheduling and synchronization link optimization, the global synchronization efficiency of cloud native applications is 7 times higher than that of manual synchronization.

(the picture shows the global distribution of cloud native applications)

In the aspect of P2P large-scale distribution, the product optimizes the Dragonfly-based distribution scheme for many times for the cloud environment, and finally solves various file distribution problems such as large-scale file download and cross-network isolation through a number of innovative technologies, and greatly improves the ability of large-scale container image distribution. The efficiency of large-scale distribution of average images is several times higher than that of the normal method, which is suitable for scenarios where the number of nodes in a single cluster of container clusters reaches 100 or more.

(the picture shows a P2P-based distribution process)

In addition to P2P large-scale distribution, in order to better meet the needs of large-scale distribution in specific scenarios, the product also supports large-scale distribution based on mirror snapshots. The distribution method based on mirror snapshots can avoid or reduce the download of mirror layer, and greatly improve the speed of elastic container instance creation. In the joint use scenario of container cluster (ASK) and elastic container instance (ECI), the product can support 500-node image pull in seconds to achieve rapid expansion in business burst scenarios.

New platform

In addition to meeting the requirements of ten-thousand-node minute-level distribution collaboration of cloud native applications, container mirroring service also improves and optimizes the capability of the platform to ensure the observability and stability of the distribution process. At the same time, the platform provides integration capabilities to further extend the use scenarios and value of cloud native application distribution.

Stability.

The specific improvement and optimization of stability are carried out from several aspects, such as monitoring and alarm, fault tolerance and disaster tolerance, dependence governance, current limitation and degradation, capacity planning and so on.

In terms of dependency governance, the platform uniformly manages the relevant key links and external dependencies in the cloud native application delivery chain, improves the overall delivery capability of the delivery chain, and helps users identify hot warehouses and track the implementation results of the delivery chain.

In terms of current limitation and degradation, the platform analyzes and identifies the primary and secondary business functions of the core link of cloud native application distribution, giving priority to ensuring the completion of the main business logic, while the secondary business logic can be degraded and delayed.

In terms of capacity planning, the platform expands resources on demand according to the upstream and downstream business changes to ensure the normal delivery of cloud native applications.

(the picture shows the stability guarantee strategy of the platform)

Ecological integration

Based on the rich integration capabilities provided by the platform, users can also use CCS Enterprise Edition as an infrastructure for hosting and distributing cloud native assets, providing their users with cloud native application distribution capabilities.

Among them, the Container Mirror Service Enterprise Edition supports Alibaba Cloud Market to build a container application market, supports container commodity hosting and commercial distribution in the container application market, and builds a closed loop of cloud native ecology. ISV service providers, such as Intel, Fortinet and Aoze, quickly put containerized goods on the shelf in the cloud market in the form of container image or Helm Chart, realizing standardized delivery and commercial realization. Market customers can also obtain high-quality Aliyun official and ISV container images from the container application market, and quickly deploy to the container service container cluster to enjoy Aliyun's rich cloud native ecology.

(the picture shows the process of container application market)

Write at the end

From supporting the large-scale distribution needs of Alibaba double 11, to the hosting and distribution needs of cloud native assets that fully cover Alibaba's economy and cloud users, to supporting the construction of an ecological closed loop of containers on the cloud, Aliyun container imaging service has become one of the core infrastructure of the cloud native era and an important accelerator for releasing cloud native value. Container image service will also continue to bring users more excellent cloud native application distribution features, performance and experience.

The highlight of this book

In the practice of Shuang 11 super large K8s cluster, the problems and solutions encountered are described in detail. The best combination of Yunyuan biochemistry: Kubernetes+ container + Shenlong, to achieve the technical details of the core system 100% on the cloud. Double 11 Service Mesh super large-scale landing solution

"Alibaba Cloud Native focus on micro-services, Serverless, containers, Service Mesh and other technology areas, focus on cloud native popular technology trends, cloud native large-scale landing practice, to be the best understanding of cloud native developers of the technology circle."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.