Shulou(Shulou.com)05/31 Report--

How to use HPA and what are the details that need to be paid attention to? I believe many inexperienced people are at a loss about this. Therefore, this article summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Below we will show you how to use HPA and some details to pay attention to.

Autoscaling/v1 practice

The template of v1 is probably the most frequently seen and the simplest, and the v1 version of HPA supports only one indicator-CPU. Traditionally, auto scaling supports at least two metrics: CPU and Memory, so why only release CPU in Kubernetes? In fact, the earliest HPA was planned to support both metrics, but the actual development tests found that memory is not a very good auto-scaling condition. Unlike CPU, many memory-based applications do not quickly reclaim memory because HPA pops up new containers, and the memory of many applications has to be managed by VM at the language level. In other words, memory recovery is determined by the GC of VM. It is possible that the difference in GC time causes the HPA to oscillate at an inappropriate point in time, so in the v1 version, HPA only supports CPU.

A standard v1 template is roughly as follows:

ApiVersion: autoscaling/v1kind: HorizontalPodAutoscalermetadata: name: php-apache namespace: defaultspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: php-apache minReplicas: 1 maxReplicas: 10 targetCPUUtilizationPercentage: 50

Where scaleTargetRef indicates who the scaling object is currently being operated on. In this case, the scaled object is an apps/v1 version of Deployment. TargetCPUUtilizationPercentage said: when the overall resource utilization exceeds 50 per cent, capacity will be expanded. Next, let's do a simple Demo to practice.

Log in to the CCS console, first create an application deployment, and select to use a template. The template content is as follows:

ApiVersion: apps/v1beta1 kind: Deployment metadata:name: php-apachelabels: app: php-apache spec:replicas: 1selector: matchLabels: app: php-apachetemplate: metadata: labels: app: php-apache spec: containers:-name: php-apache image: registry.cn-hangzhou.aliyuncs.com/ringtail/hpa-example:v1.0 ports:-containerPort: 80 resources: requests: memory: "300Mi" cpu: "250m"-apiVersion: v1 kind: Service metadata:name: php-apachelabels: app: php-apache spec:selector: app: php-apacheports:- protocol: TCP name: http port: 80 targetPort: 80type: ClusterIP

Deploy the stress test module HPA template

ApiVersion: autoscaling/v1kind: HorizontalPodAutoscalermetadata:name: php-apachenamespace: defaultspec:scaleTargetRef: apiVersion: apps/v1beta1 kind: Deployment name: php-apacheminReplicas: 1maxReplicas: 10targetCPUUtilizationPercentage: 50

Turn on the pressure test

ApiVersion: apps/v1beta1 kind: Deployment metadata: name: load-generator labels: app: load-generator spec: replicas: 1 selector: matchLabels: app: load-generator template: metadata: labels: app: load-generator spec: containers:-name: load-generator image: busybox command: -"sh"-"- c"-"while true" Do wget-Q-O-http://php-apache.default.svc.cluster.local; done "

Check the capacity expansion status

Close the pressure test application

Check the capacity reduction status

Such a HPA using autoscaling/v1 is done. Relatively speaking, this version of HPA is by far the simplest, regardless of whether you upgrade Metrics-Server or not.

Autoscaling/v2beta1 practice

In the previous content for you to explain HPA and autoscaling/v2beta1 and autoscaling/v2beta2 two versions. The difference between the two versions is that autoscaling/v1beta1 supports Resource Metrics and Custom Metrics. Additional support for External Metrics has been added to the version of autoscaling/v2beta2. External Metrics will not be discussed too much in this article, because there are not many mature implementations of External Metrics in the community, and the more mature implementation is Prometheus Custom Metrics.

The figure above shows how HPA uses different types of Metrics when Metrics Server is enabled. If you need to use Custom Metrics, you need to configure and install the corresponding Custom Metrics Adapter. In the following, we will mainly introduce an example of auto scaling based on QPS.

Install Metrics Server and open it in kube-controller-manager

At present, the default Kubernetes cluster of Ali Cloud CCS is still Heapster. CCS plans to update Metrics Server in 1.12. It needs to be noted that although the community has gradually begun to abandon Heapster, there are still a large number of components in the community that are strongly dependent on Heapster's API. Therefore, Aliyun has carried out complete Heapster compatibility based on Metrics Server, which allows developers to use the new features of Metrics Server. You don't have to worry about the downtime of other components.

Before deploying the new Metrics Server, we first need to back up some startup parameters in the Heapster, because these parameters will be used directly in the Metrics Server template later. The focus is on the two Sink. If you need to use Influxdb, you can keep the first Sink;. If you need to retain the cloud monitor integration capability, you can keep the second Sink.

Copy these two parameters to the startup template of Metrics Server, in this case both are compatible, and distribute the deployment.

ApiVersion: v1kind: ServiceAccountmetadata: name: metrics-server namespace: kube-system---apiVersion: v1kind: Servicemetadata: name: metrics-server namespace: kubernetes.io/name: "Metrics-server" spec: selector: k8s-app: metrics-server ports:-port: 443 protocol: TCP targetPort: 443---apiVersion: apiregistration.k8s.io/v1beta1kind: APIServicemetadata: name: v1beta1.metrics.k8s.iospec: service: name: metrics-server namespace : kube-system group: metrics.k8s.io version: v1beta1 insecureSkipTLSVerify: true groupPriorityMinimum: 100versionPriority: 100---apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-serverspec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-serverspec: serviceAccountName: admin containers:- Name: metrics-server image: registry.cn-hangzhou.aliyuncs.com/ringtail/metrics-server:1.1 imagePullPolicy: Always command:-/ metrics-server -'--source=kubernetes: https://kubernetes.default' -'--sink=influxdb: http://monitoring-influxdb:8086' -'--sink=socket:tcp:// monitor.csk.region _ id] .aliyuncs.com: 8093 cluster clusterId = [cluster _ id] & public=true'

Next, we modify the Service of Heapster to transfer the back end of the service from Heapster to Metrics Server.

If the monitoring information on the right can be obtained from the node page of the console, Metrics Server is fully compatible with Heapster.

At this point, through kubectl get apiservice, if you can see the api of the registered v1beta1.metrics.k8s.io, it means that the registration has been successful.

Next we need to switch the data sources of Metrics on kube-controller-manager. Kube-controller-manger is deployed on each master and is hosted to kubelet through Static Pod. So you only need to modify the configuration file of kube-controller-manager, and kubelet will update it automatically. The path of kube-controller-manager on the host is / etc/kubernetes/manifests/kube-controller-manager.yaml.

You need to change-- horizontal-pod-autoscaler-use-rest-clients=true, here is a note, because if you use vim for editing, vim will automatically generate a cache file to affect the final result, so the recommended way is to move this configuration file to another directory to modify, and then move back to the original directory. Now that Metrics Server is ready to serve HPA, let's do the part of customizing metrics.

Deploy Custom Metrics Adapter

If Prometheus is not deployed in the cluster, you can deploy Prometheus first by referring to "Ali Cloud Container Kubernetes Monitoring (7)-Prometheus Monitoring solution deployment". Next we deploy Custom Metrics Adapter.

Kind: NamespaceapiVersion: v1metadata: name: custom-metrics---kind: ServiceAccountapiVersion: v1metadata: name: custom-metrics-apiserver namespace: custom-metrics---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: custom-metrics:system:auth-delegatorroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegatorsubjects:- kind: ServiceAccount name: custom-metrics-apiserver namespace: custom-metrics---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata: name : custom-metrics-auth-reader namespace: kube-systemroleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-readersubjects:- kind: ServiceAccount name: custom-metrics-apiserver namespace: custom-metrics---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: custom-metrics-resource-readerrules:- apiGroups:-"" resources:-namespaces-pods-services verbs:-get-list---apiVersion: rbac.authorization.k8s. Io/v1kind: ClusterRoleBindingmetadata: name: custom-metrics-apiserver-resource-readerroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: custom-metrics-resource-readersubjects:- kind: ServiceAccount name: custom-metrics-apiserver namespace: custom-metrics---apiVersion: rbac.authorization.k8s.io/v1kind: name: custom-metrics-getterrules:- apiGroups:-custom.metrics.k8s.io resources:-"*" verbs:-"*"-- apiVersion: rbac. Authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: hpa-custom-metrics-getterroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: custom-metrics-gettersubjects:- kind: ServiceAccount name: horizontal-pod-autoscaler namespace: kube-system---apiVersion: apps/v1kind: Deploymentmetadata: name: custom-metrics-apiserver namespace: custom-metrics labels: app: custom-metrics-apiserverspec: replicas: 1 selector: matchLabels: app: custom-metrics-apiserver template: metadata: Labels: app: custom-metrics-apiserver spec: tolerations:-key: beta.kubernetes.io/arch value: arm effect: NoSchedule-key: beta.kubernetes.io/arch value: arm64 effect: NoSchedule serviceAccountName: custom-metrics-apiserver containers:-name: custom-metrics-server image: luxas/k8s-prometheus-adapter:v0.2.0-beta.0 Args:-prometheus-url= http://prometheus-k8s.monitoring.svc:9090-metrics-relist-interval=30s-rate-interval=60s-- Vroom10-logtostderr=true ports:-containerPort: 443 securityContext: runAsUser: 0---apiVersion: v1kind: Servicemetadata: name: api namespace: custom-metricsspec: ports:- Port: 443 targetPort: 443 selector: app: custom-metrics-apiserver---apiVersion: APIServicemetadata: name: v1beta1.custom.metrics.k8s.iospec: insecureSkipTLSVerify: true group: custom.metrics.k8s.io groupPriorityMinimum: 1000 versionPriority: 5 service: name: api namespace: custom-metrics version: v1beta1---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: custom-metrics-server-resourcesrules:- apiGroups:-custom-metrics.metrics. K8s.io resources: ["*"] verbs: ["*"]-apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: hpa-controller-custom-metricsroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: custom-metrics-server-resourcessubjects:- kind: ServiceAccount name: horizontal-pod-autoscaler namespace: kube-system

Deploy the manual stress test application and HPA template

ApiVersion: apps/v1kind: Deploymentmetadata: labels: app: sample-metrics-app name: sample-metrics-appspec: replicas: 2 selector: matchLabels: app: sample-metrics-app template: metadata: labels: app: sample-metrics-appspec: tolerations:-key: beta.kubernetes.io/arch value: arm effect: NoSchedule-key: beta.kubernetes.io/arch value : arm64 effect: NoSchedule-key: node.alpha.kubernetes.io/unreachable operator: Exists effect: NoExecute tolerationSeconds: 0-key: node.alpha.kubernetes.io/notReady operator: Exists effect: NoExecute tolerationSeconds: 0 containers:-image: luxas/autoscale-demo:v0.1.2 name: sample-metrics-app ports:-name: web ContainerPort: 8080 readinessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 3 periodSeconds: 5 livenessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 3 periodSeconds: 5---apiVersion: v1kind: Servicemetadata: name: sample-metrics-app labels: app: sample- Metrics-appspec: ports:-name: web port: 80 targetPort: 8080 selector: app: sample-metrics-app---apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata: name: sample-metrics-app labels: service-monitor: sample-metrics-appspec: selector: matchLabels: app: sample-metrics-app endpoints:-port: web---kind: HorizontalPodAutoscalerapiVersion: autoscaling/v2beta1metadata: name: sample-metrics-app-hpaspec: scaleTargetRef: apiVersion : apps/v1 kind: Deployment name: sample-metrics-app minReplicas: 2 maxReplicas: 10 metrics:-type: Object object: target: kind: Service name: sample-metrics-app metricName: http_requests targetValue: 100---apiVersion: extensions/v1beta1kind: Ingressmetadata: name: sample-metrics-app namespace: default annotations: traefik.frontend.rule.type: PathPrefixStripspec: rules:-http: paths:-path: / sample-app backend: serviceName: sample-metrics-app servicePort: 80

This stress test application exposes a Prometheus interface. The data in the API is as follows, where the http_requests_total metric is the custom metric we will use for scaling.

[root@iZwz99zrzfnfq8wllk0dvcZ manifests] # curl 172.16.1.160:8080/metrics# HELP http_requests_total The amount of requests served by the server in total# TYPE http_requests_total counterhttp_requests_total 3955684

Deploy stress test application

ApiVersion: apps/v1beta1kind: Deploymentmetadata: name: load-generator labels: app: load-generatorspec: replicas: 1 selector: matchLabels: app: load-generator template: metadata: labels: app: load-generatorspec: containers:-name: load-generator image: busybox command:-"sh"-"- c"-"while true Do wget-Q-O-http://sample-metrics-app.default.svc.cluster.local; done "

Check the status and scaling of HPA. Wait a few minutes. Pod has been scaled successfully.

Workspace kubectl get hpaNAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGEphp-apache Deployment/php-apache 0% frame 50% 1 10 1 21dsample-metrics-app-hpa Deployment/sample-metrics-app 538133m/100 2 10 10 15 h finish reading the above content Do you know how to use HPA and what details you need to pay attention to? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.