Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how to use Kubernetes Critical Pod". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Is there any way to identify a Pod as Critical Pod

Rule 1:

Enable Feature Gate ExperimentalCriticaPodAnnotation

Must belong to kube-system namespace

Must add Annotation scheduler.alpha.kubernetes.io/critical-pod= ""

Rule 2:

Enable Feature Gate ExperimentalCriticaPodAnnotation, PodPriority

The Priority of Pod is not empty and is not less than 2 * 10 ^ 9

System-node-critical priority = 10 ^ 9 + 1000

System-cluster-critical priority = 10 ^ 9

If one of rule 1 or rule 2 is satisfied, the Pod is considered to be Critical Pod

Schedule Critical Pod

During the predicate phase of pod scheduling by default scheduler, GeneralPredicates is registered as one of the default predicates, and it is not judged that critical Pod uses EssentialPredicates to predicate process the critical Pod. What does that mean?

Let's just look at the relationship between GeneralPredicates and EssentialPredicates. In GeneralPredicates, first call noncriticalPredicates, then call EssentialPredicates. So if you mark Deployment/StatefulSet (except DeamonSet) as Critical, then when you schedule scheduler, you will still follow the GeneralPredicates process and call noncriticalPredicates, but you want it to go directly to EssentialPredicates.

/ / GeneralPredicates checks whether noncriticalPredicates and EssentialPredicates pass. NoncriticalPredicates are the predicates// that only non-critical pods need and EssentialPredicates are the predicates that all pods, including critical pods, needfunc GeneralPredicates (pod * v1.Pod, meta algorithm.PredicateMetadata, nodeInfo * schedulercache.NodeInfo) (bool, [] algorithm.PredicateFailureReason, error) {var predicateFails [] algorithm.PredicateFailureReason fit, reasons, err: = noncriticalPredicates (pod, meta, nodeInfo) if err! = nil {return false, predicateFails Err} if! fit {predicateFails = append (predicateFails, reasons...)} fit, reasons, err = EssentialPredicates (pod, meta, nodeInfo) if err! = nil {return false, predicateFails, err} if! fit {predicateFails = append (predicateFails, reasons...)} return len (predicateFails) = = 0, predicateFails Nil}

NoncriticalPredicates is meant to do extra predicate logic on non-critical pod, which is called PodFitsResources checking.

Pkg/scheduler/algorithm/predicates/predicates.go:1076// noncriticalPredicates are the predicates that only non-critical pods needfunc noncriticalPredicates (pod * v1.Pod, meta algorithm.PredicateMetadata, nodeInfo * schedulercache.NodeInfo) (bool, [] algorithm.PredicateFailureReason, error) {var predicateFails [] algorithm.PredicateFailureReason fit, reasons, err: = PodFitsResources (pod, meta, nodeInfo) if err! = nil {return false, predicateFails Err} if! fit {predicateFails = append (predicateFails, reasons...)} return len (predicateFails) = = 0, predicateFails, nil}

PodFitsResources does the following to check whether the resource meets the requirements:

Allowed Pod Number

CPU

Memory

EphemeralStorage

Extended Resources

That is to say, if you mark Deployment/StatefulSet (except DeamonSet) as Critical, then the corresponding Pod scheduling will still check whether Allowed Pod Number, CPU, Memory, EphemeralStorage,Extended Resources are sufficient, if not, it will trigger pre-selection failure, and in the Preempt phase, the normal preemption logic is only carried out according to the corresponding PriorityClass, and there is no special processing for Critical Pod, so it may end up because the Node that meets the resource requirements cannot be found. Causes the Critical Pod scheduling to fail and has been in the Pending state.

Users do not want to cause scheduling failure because of insufficient resources when setting Critical Pod. What if I just want to deploy critical services using Deployment/StatefulSet, etc. (except DeamonSet) marked as Critical Pod? There are two ways:

According to rule 2 mentioned earlier, set system-cluster-critical or system-node-critical Priority Class to Pod, so that resources will be preempted in scheduler's normal Preempt process to complete the scheduling.

Follow rule 1 mentioned earlier, and modify the code of GeneralPredicates as follows to check whether it is Critical Pod, and if so, no noncriticalPredicates logic is executed, that is, the predicate phase does not check Allowed Pod Number, CPU, Memory, and EphemeralStorage,Extended Resources resources.

Func GeneralPredicates (pod * v1.Pod, meta algorithm.PredicateMetadata, nodeInfo * schedulercache.NodeInfo) (bool, [] algorithm.PredicateFailureReason, error) {var predicateFails, resons [] algorithm.PredicateFailureReason var fit bool var err error / / * * Modify**: check whether the pod is a Critical Pod, don't invoke noncriticalPredicates if false. IsCriticalPod: = utilfeature.DefaultFeatureGate.Enabled (features.ExperimentalCriticalPodAnnotation) & & kubelettypes.IsCriticalPod (newPod) if! isCriticalPod {fit, reasons, err = noncriticalPredicates (pod, meta, nodeInfo) if err! = nil {return false, predicateFails, err}} if! fit {predicateFails = append (predicateFails Reasons...)} fit, reasons, err = EssentialPredicates (pod, meta, nodeInfo) if err! = nil {return false, predicateFails, err} if! fit {predicateFails = append (predicateFails, reasons...)} return len (predicateFails) = = 0, predicateFails, nil}

Method 1, in fact, Kubernetes has already done it for you during the Admission Priority check.

/ / admitPod makes sure a new pod does not set spec.Priority field. It also makes sure that the PriorityClassName exists if it is provided and resolves the pod priority from the PriorityClassName.func (p * priorityPlugin) admitPod (an admission.Attributes) error {... If utilfeature.DefaultFeatureGate.Enabled (features.PodPriority) {var priority int32 if len (pod.Spec.PriorityClassName) = = 0 & & utilfeature.DefaultFeatureGate.Enabled (features.ExperimentalCriticalPodAnnotation) & & kubelettypes.IsCritical (a.GetNamespace () Pod.Annotations) {pod.Spec.PriorityClassName = scheduling.SystemClusterCritical}.}

The Priority of Pod will be checked during Admission. If you find that you have:

Enable PriorityClass Feature Gate

Enable ExperimentalCriticalPodAnnotation Feature Gate

Added ExperimentalCriticalPodAnnotation to Pod

Deployed in kube-system namespace

Custom PriorityClass is not set manually

Then, the Admisson Priority phase automatically adds a SystemClusterCritical (system-cluster-critical) PriorityClass to the Pod

Best practic

Through the above analysis, the following best practices are given: in Kubernetes clusters, when deploying critical services through non-DeamonSet methods (such as Deployment, RS, etc.), in order to ensure the success of preemptive scheduling even when the cluster resources are insufficient, please ensure the following:

Enable PriorityClass Feature Gate

Enable ExperimentalCriticalPodAnnotation Feature Gate

Added ExperimentalCriticalPodAnnotation to Pod

Deployed in kube-system namespace

Never set a custom PriorityClass manually

That's all for the content of "how to use Kubernetes Critical Pod". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.