Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about the principle and evolution of HPA, the most commonly used component of elastic scaling in Kubernetes. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

Preface

Today we will introduce HPA (Horizontal Pod Autoscaler), the most commonly used component of auto scaling in Kubernetes. HPA is a component that replans the capacity by calculating the actual workload of Pod. Under the premise that the resource pool meets the requirements, HPA can well achieve the elastic scaling model. HPA has evolved into three major versions so far. This article will analyze in detail the underlying principles of HPA and the evolution of the concept of elastic scaling in Kubernetes.

Basic principles of HPA

HPA is a component that scales the number of containers according to the actual workload level, from which two very important keywords can be extracted: load and number. We can use a very simple mathematical formula to sum up: here is a practical example to illustrate the above formula. Suppose there is a Deployment called A that contains 3 Pod, and the Request value of each copy is 1 core. The CPU utilization of the current three Pod is 60%, 70% and 80%, respectively. In this case, we set the HPA threshold to 50%, the minimum copy is 3, and the maximum copy is 10. Next, we bring the above data into the formula:

The total utilization of Pod is 60% "70%" 80% = 210%.

The current Target is 3

The result of the formula is 70%, which is greater than the 50% threshold, so the current number of Target is too small and needs to be expanded.

Reset the Target value to 5, and the result of the formula is 42% less than 50%. It is determined that two containers need to be expanded.

At this point, HPA sets Replicas to 5 to expand the horizontal capacity of Pod.

The above deduction can help developers quickly understand the core principles of HPA, but the above deduction results are different from the actual situation. If developers experiment, they will find that the final result of Replicas is 6 instead of 5. This is due to the handling of some details in HPA, which mainly includes the following three main aspects:

Noise processing

From the above formula, it can be found that the number of Target affects the final result to a large extent, while in Kubernetes, whether it is a change or upgrade, it is more likely to use Recreate rather than Restart. As a result, there may be a time in the life cycle of Deployment that the Target will become very large due to the calculation of the Pod of Starting or Stopping. This will bring a lot of noise to the calculation of HPA. In the calculation of HPA Controller, if it is found that there is a Starting in the current object or the Pod of Stopping will directly skip the current calculation cycle, the waiting state will be changed to Running before calculation.

Cooling cycle

In auto scaling, the cooling cycle is an inescapable topic. Most of the time, we expect fast pop-up and fast recovery, but on the other hand, we do not want the cluster to shake, so what is the specific value of an auto scaling cooling cycle? has been challenged by developers. In HPA, the default cooling period for capacity expansion is 3 minutes, and the cooling period for capacity reduction is 5 minutes.

Boundary value calculation

Let's go back to the previous calculation formula. For the first time, we calculated that the number of containers that need to be popped up is 5. At this time, the overall load after expansion is 42%. But we seem to ignore a question: will a new Pod boot take up some resources on its own? In addition, can an 8% buffer relieve the overall load? You should know that when an elastic expansion is completed, the next expansion will have to wait at least 3 minutes before it can be further expanded. To solve these problems, HPA introduces the boundary value △, which automatically adds a 10% buffer when calculating the boundary conditions, which is why the final result is 6 in the example just now.

The Evolution of HPA

After understanding the basic principles of HPA, let's talk about the evolution of HPA. At present, HPA supports three major versions: autoscaling/v1, autoscaling/v1beta1 and autoscaling/v1beta2. At present, most developers are familiar with the autoscaling/v1 version, which only supports the auto scaling of CPU. The general yaml content is as follows:

ApiVersion: autoscaling/v1kind: HorizontalPodAutoscalermetadata: name: php-apache namespace: defaultspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: php-apache minReplicas: 1 maxReplicas: 10 targetCPUUtilizationPercentage: 50

Next, if we take a look at the yaml of v2beta1 and v2beta2, we will find that the supported metrics types have increased a lot and the structure has become much more complex.

ApiVersion: autoscaling/v2beta1kind: HorizontalPodAutoscalermetadata: name: php-apache namespace: defaultspec: scaleTargetRef: apiVersion: apps/v1 kind: php-apache minReplicas: 1 maxReplicas: 10 metrics:-type: Resource resource: name: cpu target: kind: AverageUtilization averageUtilization: 50-type: Pods pods: metric: name: packets-per-second targetAverageValue: 1k-type: Object object: metric: Name: requests-per-second describedObject: apiVersion: extensions/v1beta1 kind: Ingress name: main-route target: kind: Value value: 10k---apiVersion: autoscaling/v2beta2kind: HorizontalPodAutoscalermetadata: name: php-apache namespace: defaultspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: php-apache minReplicas: 1 maxReplicas: 10 metrics:-type: Resource resource: name: cpu target : type: Utilization averageUtilization: 50

The emergence of these changes has to be mentioned is the understanding and transformation of monitoring and monitoring indicators in the Kubernetes community. In Kubernetes, there are two core monitoring components, Heapster and Metrics Server. Heapster is the only monitoring component in the early Kubernetes community. It contains powerful functions, such as collecting the metrics interface provided by kubelet, and supporting offline and archiving of monitoring data.

The general architecture diagram is as follows: the part of source is from different data sources, mainly the common api of kubelet and the summary api;processor provided later are used to process the collected data, aggregate the data at namespace level and cluster level respectively, and create new aggregation type of monitoring data; the role of sink is data offline and archiving, and common archiving methods include influxdb, kafka, and so on. Heapster components have become the only source of monitoring data in the Kubernetes community for quite a long time, so there are a lot of monitoring-related components monitoring data consumption through Heapster links. Later, however, the Kubernetes community discovered several very serious problems with Heapster:

Powerful and diverse Sink is maintained by different Maintainer, more than 50% of the Heapster Issues is related to Sink can not be used, and due to the different activity of Maintainer, there are a large number of issues in the Heapster community can not solve.

For developers, the type of monitoring data is no longer simple metrics such as CPU and Memory. More and more developers need monitoring metrics within the application or at the access layer, such as the QPS of ingress, the number of online active applications, and so on. However, the acquisition of these indicators can not be achieved by Heapster.

The maturity of Prometheus keeps squeezing the living space of Heapster. Since Prometheus was included as an incubation project by CNCF, the irreplaceable status of Heapster has been officially removed.

After reflection, the community decided to divide the boundary of the monitored indicators into three different Metrics: Resource, Custom and External, while the positioning of Heapster (Metrics Server) is only concerned about the type of Resource. In order to solve the problem of code maintainability, Metrics Server tailored Heapster, and the structure after tailoring is as follows:

The mechanism of Sink is removed and the calling mode is changed to the standard API registration mode, which not only simplifies the logic of the core code but also provides the possibility of substitution, that is to say, Metrics Server can also be replaced at this time, as long as the same API interface is implemented and registered on API Server, Metrics Server can be replaced.

Next, let's analyze three different Metrics and usage scenarios:

API annotates the resource metrics of Resourcemetrics.k8s.ioPod. When calculating, divide by the number of Pod and then compare the threshold to determine the monitoring metrics of objects such as Customcustom.metrics.k8s.ioObject: CRD. Directly calculate the threshold Pods: custom metrics of each Pod, and divide the calculation by the number of Pods: the monitoring metrics of cluster metrics, usually implemented by cloud vendors.

Among them, autoscaling/v2beta1 supports Resource and Custom, while autoscaling/v2beta2 adds the support of External.

At present, HPA has entered the GA stage, and there will not be too many changes in the general function. At present, the main focus of the community is how to configure and adjust the detailed parameters, enrich the implementation of monitoring adapter, and so on.

This is the principle and evolution of the most commonly used auto-scaling component HPA in Kubernetes shared by Xiaobian. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.