Shulou(Shulou.com)06/03 Report--

author| Xianwei Alibaba Senior Technical Expert

Introduction: Serverless Kubernetes is an exploration of the future evolution direction of Kubernetes by Alibaba Cloud Kubernetes Engine team. By subtracting Kubernetes, it reduces the burden of operation and maintenance management, simplifies cluster management, and makes Kubernetes from complex to simple.

background

Kubernetes, as a general container orchestration system, carries a wide range of applications and scenarios, including CI/CD, data computing, online applications, AI, etc. However, due to its versatility and complexity, managing a Kubernetes cluster is still challenging for many users, mainly reflected in:

High learning costs; high cluster O & M costs, including node management, capacity planning, and location of various node anomalies; computation costs are not optimal in many scenarios, such as a cluster running Jobs regularly, and long-term holding of resource pools is wasteful behavior for users, and resource utilization is not high. Subtracting Kubernetes Cluster Nodeless Management

We believe future users will focus more on app development than infrastructure maintenance. Reflected in Kubernetes cluster, we want users to focus on application orchestration semantics such as pod/service/ingress/job, and reduce attention to the underlying node.

The O & M cost of the cluster can be significantly reduced without managing nodes. According to statistics, most of the common exceptions in Kubernetes are related to nodes, such as Node NotReady, and there is no need to worry about Node security issues and upgrade and maintenance of basic system software.

In ASK cluster, we use virtual node virtual-kubelet instead of ecs node. The capacity of virtual node can be regarded as "infinite", so users do not need to worry about the capacity of cluster and do not need to do capacity planning in advance.

No Master management

Just like the ACK hosted version, ASK's Master(apiserver, ccm, kcm, etc.) resources are hosted by the Kubernetes Engine platform, so users do not need to manage the upgrade and operation and maintenance of these core components, nor do they have to pay any cost.

Minimalist K8s base operating environment

In addition to eliminating the need to manage nodes and masters, we've made a number of simplifications to Kubernetes cluster management, including hosting a lot of addons by default, eliminating the need for users to manage some basic addons and paying for them. Relying on Alibaba Cloud's native network and storage capabilities, as well as unique hosting architecture design, we provide an extremely simplified but fully functional Kubernetes infrastructure operating environment.

Features ACKASK storage requires deployment aliyun-disk-controller/flexvolume No deployment required (support is in progress) CNI network requires deployment terway/flannel daemonset No deployment required, vpc-based network communication Coredns service discovery requires deployment of 2 copies of Coredns No deployment required, kube-proxy access based on privatezone requires deployment of kube-proxy daemonset No deployment required, privatezone access to Ingress requires deployment of nginx-ingress-controller No deployment required ACR image extraction based on SLB layer 7 forwarding requires deployment of acr-credential-helper without deployment, supports sls log collection by default requires deployment of logtail daemonset without deployment, supports metrics statistics by default requires deployment of metrics-server without deployment, out-of-the-box mount eip requires deployment of terway without deployment, uses annotaion to specify cloud disk creation with pod Mount dependency aliyun-disk-controller without deployment, Auto Scaling is supported by default cluster-autoscaler deploymentNo GPU plug-ins deploymentNivida-docker deploymentNo deployment, out-of-the-box

In summary, ACK clusters require at least 2 ECS machines to run these basic Addons, while ASK clusters make these basic Addons invisible and can create a Kubernetes cluster out of the box at zero cost.

Simplified Auto Scaling

Because there is no need to manage nodes and capacity planning, when the cluster needs to be expanded, it is not necessary to consider node level expansion, only need to pay attention to pod expansion, which greatly improves the speed and efficiency of expansion. At present, some customers specify to use ASK/ECI to quickly respond to business traffic peaks.

ASK/ECI supports 500 pods (running state) in 30 seconds, and a single pod can be started within 10 seconds.

lower cost

In addition to the low-cost creation of ASK clusters themselves, the on-demand use of pods also allows resource utilization to be optimized in many scenarios. For many Jobs or data computing scenarios, users do not need to maintain a fixed resource pool for a long time, and ASK/ECI can support these demands very well.

It has been verified that ASK/ECI is more cost-effective than maintaining ECS resource pools when pods run less than 16 hours a day.

ECI: Elastic Computing Services for Rapid Delivery of Container Resources

When you talk about ASK, you will definitely talk about the resource base ECI of ASK. ECI is a stable, efficient and highly elastic container instance service provided by Alibaba Cloud based on ECS IaaS resource pool. ECI enables containers to become first-class citizens of the public cloud. Users can deploy container applications directly without purchasing and managing ecs. This simplified container instance product form and ASK form a perfect combination.

Users can directly use ECI Open API to create container instance resources, but in container scenarios users generally need an orchestration system to be responsible for container scheduling, high availability orchestration and other capabilities, and ASK is such Kubernetes orchestration layer.

For ASK, ECI allows ASK Kubernetes Engine to eliminate the need to build a pool of back-end computing resources, let alone worry about the capacity of the underlying computing resource pool. Based on ECI, it means that the large-scale resource pool based on the entire Alibaba Cloud IaaS naturally has the advantages of inventory and elasticity (for example, the ecs specification corresponding to the underlying eci can be specified through Annotation, and most ecs specifications can be used in ASK to meet the needs of various computing scenarios). In addition, ECI and ECS reuse resource pools mean that we can maximize the release of scale dividends and provide users with lower cost computing services.

Container ecological support

ASK provides complete support for the Kubernetes container ecosystem, and a large number of customers have used ASK to support the following scenarios:

CI/CD: gitlab-runner, jenkins/jenkins-x Data calculation: spark/spark-operator, flink, presto, argoAI: tensorflow/arenaServiceMesh: istio, knotive Test: locus, selenium

ASK cluster does not support Helm v2, ACK/ASK will release Helm v3 support in the near future, after which users can easily deploy Charts in ASK cluster.

More ASK Reference Documentation Rapid deployment of jenkins environments and execution of pipelined builds Create Service based on privatezone Service discovery Create Ingress use nginx ingresssls Log collection use kaniko Automated build mirroring use virtual nodes use GPU container instances pod mount eip use mirror snapshots Accelerate startup collect pod logs to sls Run Argo workflow use vk-autoscaler ASK Examples

"Alibaba Cloud focuses on microservices, Serverless, containers, Service Mesh and other technical fields, focuses on cloud native popular technology trends, cloud native large-scale landing practices, and is the technical circle that understands cloud native developers best. "

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.