Shulou(Shulou.com)05/31 Report--

This article focuses on "how to enable Initializers". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to enable Initializers.

The best configuration of Admission Controll

Students who have configured kube-apiserver must remember this configuration-- admission-control or-- admission-control-config-file, where you can configure the desired admission controller sequentially. The default is AlwaysAdmit.

In Kubernetes 1.9, as many as 32 of all allowed controllers are already supported:

AlwaysAdmit

AlwaysDeny

AlwaysPullImages

DefaultStorageClass

DefaultTolerationSeconds

DenyEscalatingExec

DenyExecOnPrivileged

EventRateLimit

ExtendedResourceToleration

ImagePolicyWebhook

InitialResources

Initializers

LimitPodHardAntiAffinityTopology

LimitRanger

MutatingAdmissionWebhook

NamespaceAutoProvision

NamespaceExists

NamespaceLifecycle

NodeRestriction

OwnerReferencesPermissionEnforcement

PVCProtection

PersistentVolumeClaimResize

PersistentVolumeLabel

PodNodeSelector

PodPreset

PodSecurityPolicy

PodTolerationRestriction

Priority

ResourceQuota

SecurityContextDeny

ServiceAccount

ValidatingAdmissionWebhook

Note that when I write this blog, the Dynamic Admission Controll official document has not yet had time to update to 1.9 corresponding content, the official document is still written in GenericAdmissionWebhook, in fact, the Webhook class has been divided into MutatingAdmissionWebhook and ValidatingAdmissionWebhook, and there is no GenericAdmissionWebhook this item, in fact, it is ValidatingAdmissionWebhook in Kubernetes 1.9 after the rename.

If you don't want to know so much about so many access controllers (although I don't recommend it, please refer to the admission-controllers official documentation for the specific meaning of each), it doesn't matter, Kubernetes also has recommendations for you.

If you use Kubernetes 1.61.8, the official recommended configuration is as follows:

-- admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds

If you use Kubernetes 1.9, the official recommended configuration is as follows:

-- admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ValidatingAdmissionWebhook,ResourceQuota,DefaultTolerationSeconds,MutatingAdmissionWebhook

Again, the list of controllers configured by admission-control is ordered, and the higher the controller is, the first to execute. Once the result returned by a controller is reject, the whole admission control phase ends immediately, so the configuration order here is also fastidious. Poor configuration order will lead to poor performance.

Defects in built-in admission Control

Even if Kubernetes provides so many admission controllers, it is impossible to meet the needs of all enterprises, so Kubernetes provides three Dynamic Admission Controller:

Initializers (Alpha, Default disable in 1.9)

MutatingAdmissionWebhook (Belta, Default enable in 1.9)

ValidatingAdmissionWebhook (Alpha in 1.8, Belta in 1.9, Default enable in 1.9)

All three Dynamic Admission Controller are designed to address two shortcomings of the other built-in plug-in admission controllers:

Packaged at kube-apiserver compilation time, kube-apiserver needs to be recompiled if there are customized changes.

If you need to modify the list of controllers in admission-controll (including the order), you need to restart kube-apiserver.

If you do not do Kubernetes Master HA, it will cause Kubernetes Master to interrupt its service.

If you do Kubernetes Master HA, will it be all right? Of course, not exactly, the service will not be interrupted, but there will be different kube-apiserver with different admission-controll configuration for a period of time, so that if the same request is distributed to a kube-apiserver with different configuration, it cannot be idempotent. Of course, it doesn't seem to have much impact.

What is the use of Initializers working mechanism Initializers

When do we need to use Initializers? When the cluster administrator needs to force some or all requests to be verified or modified, you can consider using Initializers.

With Initializers, you can insert a SideCar container into each Pod you are about to create.

With Initializers, insert a volume with test data into all Pod for business testing.

Through Initializers, check whether the length of Secret meets the requirements, so as to ensure the complexity of the password, and reject the create pod request if not.

In addition, I was thinking about the security of Harbor images: in a multi-tenant environment, a user pull an image with sensitive data on a Node and boot it to Pod. At this point, as long as another user knows the image name and sets the imagePullPolicy to IfNotPresent, the user's Pod may be dispatched to this node (if the scheduler is configured with ImageLocalityPriority priority policy, which is not the default configuration, but is often configured to improve the pod startup speed), and then run other people's sensitive images, which is not acceptable in the public cloud.

How can we solve this problem? In the private cloud, permissions are controlled through the DevOps platform. Users can only choose their own app for deployment, and cannot specify other people's image names. At the Kubernetes level, is there a way to solve this problem? Well, it can be well solved with Initializers (fortunately, Kubernetes already provides AlwaysPullImages as an Admission Controller). All user-created Pod requests are checked and modified by your Initializers and forced to change the Pod ImagePullPolicy to Always.

How to enable Initializers

As mentioned earlier, you need to add Initializers in each kube-apiserver instance (taking into account Kubernetes Master HA)-- admission-controll.

In addition, you need to add admissionregistration.k8s.io/v1alpha1 to the-- runtime-config of each kube-apiserver instance.

How Initializers works

First deploy the Initializers controller you wrote yourself. This controller watch the resource type you want, capture and modify the POST requests of these resource. Let's take envoy-initializer as an example:

ApiVersion: apps/v1beta1kind: Deploymentmetadata: initializers: pending: [] labels: app: envoy-initializer name: envoy-initializerspec: replicas: 1 template: metadata: labels: app: envoy-initializer name: envoy-initializerspec: containers:-name: envoy-initializer image: gcr.io/hightowerlabs/envoy-initializer:0.0.1 imagePullPolicy: Always args: -"- annotation=initializer.kubernetes.io/envoy"-"- require-annotation=true"

When deploying envoy-initializer, be sure to set metadata.initializers.pending to empty to prevent the deployment of envoy-initializer from being stuck by yourself.

Then you need to create your initializerConfigurationAPI Object, for example, you want to inject an envoy proxy sidecar container into each later created Deployment through Initializers:

ApiVersion: admissionregistration.k8s.io/v1alpha1kind: InitializerConfigurationmetadata: name: envoyinitializers:-name: envoy.initializer.kubernetes.io rules:-apiGroups:-"*" apiVersions:-"*" resources:-deployments

After initializerConfiguration is created, you need to wait a few seconds before deploying your application through Deployment. At this time, the corresponding Initializers will automatically append to the metadata.initializers.pending array of Deployment. Take the above example as an example, append metadata.initializers.pending [0] = envoy.initializer.kubernetes.io

ApiVersion: apps/v1beta1kind: Deploymentmetadata: annotations: "initializer.kubernetes.io/envoy": "true" labels: app: helloworld envoy: "true" name: helloworld-with-annotationspec: replicas: 1 template: metadata: labels: app: helloworld envoy: "true" name: helloworld-with-annotationspec: containers:-name: helloworld image: gcr.io/hightowerlabs/ Helloworld:0.0.1 imagePullPolicy: Always args:-"- http=127.0.0.1:8080"

Note: when metadata.initializers.pending is not null, the deployment object cannot be obtained through api by default, so you need to add the parameter-includeUninitialized=true to the request url when you Initializers controller list&wath the object.

Then the event that creates the Deployment object is captured by your custom Initializers controller, and Initializers controller modifies the Deployment according to your logic, such as injecting sidecar container and volume, and removes its corresponding Initializers controller from the object's metadata.initializers.pending.

If there is more than one Initializers mapped to this object, it will be processed sequentially according to the above logic. Therefore, if it is an Admission Controller that does not need to modify the Object, it is recommended to deal with it in a webhook way (parallel), which will have higher performance. Initializers's serial approach is doomed to poor performance, so it's best not to create too many initializers.

When the metadata.initializers.pending of the Object is null, the initialization process is considered complete, and then the scheduler and the controllers managed by controller-managers can see the Object and continue with the following scheduling and autopilot logic.

Note: when you submit a request to create an object through kubectl or rest api, if the object has a corresponding Initializers, then the object will remain in uninitialized state and will not be returned until Initializers Controllers executes the corresponding logic, and there is a timeout of 30s.

Considerations for Initializers

Based on the above understanding of the working mechanism of Initializers, we find that it also has defects or considerations:

If the Initializers Controllers you deploy does not work properly or the performance is very low, a large number of related objects will stay in the uninitialized state in high concurrency scenarios and cannot be scheduled later. This may affect your business, for example, when you use HPA to flexibly expand the relevant Deployment objects, when the debt comes up, your Initializers Controllers will not work properly, which will cause your application to fail to scale flexibly, and the consequences can be imagined! So writing a high-performance and stable Initializers Controllers is a necessary skill.

At present, Initializers admission control still belongs to Alpha, you know.

The Initializers Controllers you deploy is so important that it is recommended that you deploy it in kube-system or a separate namespace and assign it enough ResourceQuota and LimitRanger to ensure its stability.

If you have multiple Initializers Controllers associated with a certain type of resource, then each time you create a resource, the order of the elements in the generated metadata.initializers.pending array may be different, so it is recommended that these Initializers Controllers should not be interdependent.

Again, when deploying your Initializers Controllers, be sure to set metadata.initializers.pending to empty to prevent the deployment of Initializers Controllers from being stuck.

How to develop a custom Initializers

Please refer to the project kubernetes-initializer-tutorial of kelseyhightower, the code is less than 200 lines, it is very simple.

... type config struct {Containers [] corev1.Container Volumes [] corev1.Volume} func main () {... / / Watch uninitialized Deployments in all namespaces. RestClient: = clientset.AppsV1beta1 (). RESTClient () watchlist: = cache.NewListWatchFromClient (restClient, "deployments", corev1.NamespaceAll, fields.Everything ()) / / Wrap the returned watchlist to workaround the inability to include / / the `IncludeUninitialized` list option when setting up watch clients. IncludeUninitializedWatchlist: = & cache.ListWatch {ListFunc: func (options metav1.ListOptions) (runtime.Object, error) {options.IncludeUninitialized = true return watchlist.List (options)}, WatchFunc: func (options metav1.ListOptions) (watch.Interface Error) {options.IncludeUninitialized = true return watchlist.Watch (options)},} resyncPeriod: = 30 * time.Second _, controller: = cache.NewInformer (includeUninitializedWatchlist, & v1beta1.Deployment {}, resyncPeriod Cache.ResourceEventHandlerFuncs {AddFunc: func (obj interface {}) {err: = initializeDeployment (obj. (* v1beta1.Deployment), c Clientset) if err! = nil {log.Println (err)}},} ) stop: = make (chan struct {}) go controller.Run (stop) signalChan: = make (chan os.Signal, 1) signal.Notify (signalChan, syscall.SIGINT, syscall.SIGTERM)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.