Shulou(Shulou.com)06/01 Report--

What is the internal implementation principle and source code analysis of Kubernetes ResourceQuotaController? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

ResoureQuota introduction

For an introduction and use of ResoureQuota and ResourceController, please see the following official documentation. This is the basis on which you understand this blog.

Https://kubernetes.io/docs/admin/resourcequota/

Https://kubernetes.io/docs/admin/resourcequota/walkthrough/

Https://kubernetes.io/docs/user-guide/compute-resources/

Https://kubernetes.io/docs/admin/admission-controllers/

Https://github.com/kubernetes/community/blob/master/contributors/design-proposals/admission_control_resource_quota.md

Analysis of directory structure of ResourceQuota Controller source code

ResourceQuota Controller is one of the many Controller managed by Kubernetes Controller Manager, and its main source code is located in the directories k8s.io/kubernetes/pkg/quota and k8s.io/kubernetes/pkg/controller/resourcequota. The specific analysis is as follows:

K8s.io/kubernetes/pkg/quota. ├── evaluator / / responsible for the implementation of │ └── core │ ├── configmap.go / / ConfigMapEvaluator for the use of various resources, and │ ├── doc.go │ ├── persistent_volume_claims.go / / PVCEvaluator for the statistics of ConfigMap resources Responsible for the implementation of │ ├── persistent_volume_claims_test.go │ ├── pods.go / / PodEvaluator of PVC resources, │ ├── pods_test.go │ ├── registry.go / / of Pod resources, registering all Evaluators │ ├── replication_controllers.go / / RCEvaluator when Registry is created Responsible for the implementation of │ ├── resource_quotas.go / / ResourceQuotaEvaluator for statistics of ReplicationController resources, │ ├── secrets.go / / SecretEvaluator for statistics of ResourceQuota resources, and │ ├── services.go / / ServiceEvaluator for statistics of Secret resources Definition and implementation of │ └── services_test.go ├── generic / / genericEvaluator responsible for statistics of Service resources │ ├── evaluator.go / / implements the interface of genericEvaluator Including the most important CalculateUsageStats interface │ └── registry.go / / definition GenericRegistry ├── install │ └── registry.go / / when startResourceQuotaController is defined, the method that creates ResourceQuota Registry will be called ├── interfaces.go / / defines the collection operation of Registry and Evaluator Interface ├── resources.go / / defines Resources and the CalculateUsage method └── resources_test.gok8s.io/kubernetes/pkg/controller/resourcequota. ├── doc.go ├── replenishment_controller.go / / define replenishmentControllerFactory Used to create replenishmentController ├── replenishment_controller_test.go ├── resource_quota_controller.go / / define ResourceQuotaController and its Run method, syncResourceQuota method, etc., belong to the core file. └── resource_quota_controller_test.goResourceQuota Controller internal implementation schematic

Please download it to your local enlarged view.

For the function and interaction of each module, please see the source code analysis below.

ResourceQuota Controller source code analysis

The above internal implementation schematic diagram shows that ResourceQuotaController is when Kubenetes Controller Manager starts to initialize a large number of Controllers, by calling startResourceQuotaController to complete ResourceQuotaController startup.

# start with the startResourceQuotaController of kube-controller-manager

Cmd/kube-controller-manager/app/core.go:76func startResourceQuotaController (ctx ControllerContext) (bool, error) {resourceQuotaControllerClient: = ctx.ClientBuilder.ClientOrDie ("resourcequota-controller") resourceQuotaRegistry: = quotainstall.NewRegistry (resourceQuotaControllerClient, ctx.InformerFactory) / / define the resource objects that ReplenishmentController needs to monitor groupKindsToReplenish: = [] schema.GroupKind {api.Kind ("Pod") Api.Kind ("Service"), api.Kind ("ReplicationController"), api.Kind ("PersistentVolumeClaim"), api.Kind ("Secret"), api.Kind ("ConfigMap"),}. Go resourcequotacontroller.NewResourceQuotaController (resourceQuotaControllerOptions,) .run (int (ctx.Options.ConcurrentResourceQuotaSyncs), ctx.Stop) return true, nil}

StartResourceQuotaController starts a goroutine, creates a ResourceQuotaController through NewResourceQuotaController and executes its Run method to start providing ResourceQuotaController.

Here are the definitions of ResourceQuotaController and ResourceQuotaControllerOptions structures. Several key Entity are defined in ResourceQuotaController, namely rqController, queue, missingUsageQueue, registry, and replenishmentControllers, which can also be seen in the schematic diagram in the previous section.

# ResourceQuotaController definition

Pkg/controller/resourcequota/resource_quota_controller.go:40// ResourceQuotaControllerOptions holds options for creating a quota controllertype ResourceQuotaControllerOptions struct {/ / Must have authority to list all quotas, and update quota status KubeClient clientset.Interface / / Controls full recalculation of quota usage ResyncPeriod controller.ResyncPeriodFunc / / Knows how to calculate usage Registry quota.Registry / / Knows how to build controllers that notify replenishment events ControllerFactory ReplenishmentControllerFactory / / Controls full resync of objects monitored for replenihsment. ReplenishmentResyncPeriod controller.ResyncPeriodFunc / / List of GroupKind objects that should be monitored for replenishment at / / a faster frequency than the quota controller recalculation interval GroupKindsToReplenish [] schema.GroupKind} / / ResourceQuotaController is responsible for tracking quota usage status in the systemtype ResourceQuotaController struct {/ / Must have authority to list all resources in the system And update quota status kubeClient clientset.Interface / / An index of resource quota objects by namespace rqIndexer cache.Indexer / / Watches changes to all resource quota rqController * cache.Controller / / ResourceQuota objects that need to be synchronized queue workqueue.RateLimitingInterface / / missingUsageQueue holds objects that are missing the initial usage informatino missingUsageQueue workqueue.RateLimitingInterface / / To allow injection of syncUsage for testing. SyncHandler func (key string) error / / function that controls full recalculation of quota usage resyncPeriod controller.ResyncPeriodFunc / / knows how to calculate usage registry quota.Registry / / controllers monitoring to notify for replenishment replenishmentControllers [] cache.ControllerInterface} NewRegistry

Next, let's take a look at the NewRegistry, NewResourceQuotaController, and ResourceQuotaController Run methods that startResourceQuotaController calls.

Pkg/quota/evaluator/core/registry.go:29// NewRegistry returns a registry that knows how to deal with core kubernetes resources// If an informer factory is provided, evaluators will use them.func NewRegistry (kubeClient clientset.Interface, f informers.SharedInformerFactory) quota.Registry {pod: = NewPodEvaluator (kubeClient) F) service: = NewServiceEvaluator (kubeClient) replicationController: = NewReplicationControllerEvaluator (kubeClient) resourceQuota: = NewResourceQuotaEvaluator (kubeClient) secret: = NewSecretEvaluator (kubeClient) configMap: = NewConfigMapEvaluator (kubeClient) persistentVolumeClaim: = NewPersistentVolumeClaimEvaluator (kubeClient, f) return & generic.GenericRegistry {InternalEvaluators: map [schema.GroupKind] quota.Evaluator {pod.GroupKind (): pod Service.GroupKind (): service, replicationController.GroupKind (): replicationController, secret.GroupKind (): secret, configMap.GroupKind (): configMap, resourceQuota.GroupKind (): resourceQuota PersistentVolumeClaim.GroupKind (): persistentVolumeClaim,},}}

It can be seen that NewRegistry is responsible for the creation and registration of the Evaluator of these resource objects (pod,service,rc,secret,configMap,resourceQuota,PVC) for later execution of quota.CalculateUsage (...) in Worker. Use statistics on these resource objects.

NewResourceQuotaController

After the NewRegistry is executed, start creating the ResourceQuotaController with the following code.

Pkg/controller/resourcequota/resource_quota_controller.go:78func NewResourceQuotaController (options * ResourceQuotaControllerOptions) * ResourceQuotaController {/ / build the resourcequota controller rq: = & ResourceQuotaController {kubeClient: options.KubeClient, queue: workqueue.NewNamedRateLimitingQueue (workqueue.DefaultControllerRateLimiter (), "resourcequota_primary"), missingUsageQueue: workqueue.NewNamedRateLimitingQueue (workqueue.DefaultControllerRateLimiter () "resourcequota_priority"), resyncPeriod: options.ResyncPeriod, registry: options.Registry, replenishmentControllers: [] cache.ControllerInterface {},}. / / set the synchronization handler rq.syncHandler = rq.syncResourceQuotaFromKey / / build the controller that observes quota rq.rqIndexer Rq.rqController = cache.NewIndexerInformer (& cache.ListWatch {ListFunc: func (options v1.ListOptions) (runtime.Object, error) {return rq.kubeClient.Core (). ResourceQuotas (v1.NamespaceAll) .List (options)}, WatchFunc: func (options v1.ListOptions) (watch.Interface Error) {return rq.kubeClient.Core (). ResourceQuotas (v1.NamespaceAll) .watch (options)},}, & v1.ResourceQuota {}, rq.resyncPeriod (), cache.ResourceEventHandlerFuncs {AddFunc: rq.addQuota UpdateFunc: func (old, cur interface {}) {oldResourceQuota: = old. (* v1.ResourceQuota) curResourceQuota: = cur. (* v1.ResourceQuota) if quota.V1Equals (oldResourceQuota.Spec.Hard CurResourceQuota.Spec.Hard) {return} rq.addQuota (curResourceQuota)}, DeleteFunc: rq.enqueueResourceQuota,}, cache.Indexers {"namespace": cache.MetaNamespaceIndexFunc} ) for _, groupKindToReplenish: = range options.GroupKindsToReplenish {controllerOptions: = & ReplenishmentControllerOptions {GroupKind: groupKindToReplenish, ResyncPeriod: options.ReplenishmentResyncPeriod, ReplenishmentFunc: rq.replenishQuota,} replenishmentController Err: = options.ControllerFactory.NewController (controllerOptions) if err! = nil {glog.Warningf ("quota controller unable to replenish% s due to% v, changes only accounted during full resync", groupKindToReplenish, err)} else {rq.replenishmentControllers = append (rq.replenishmentControllers, replenishmentController)}} return rq}

NewResourceQuotaController is responsible for creating Entity padding for ResourceQuotaController, including queue, missingUsageQueue, syncHandler,rqIndexer, and rqController,replenishmentControllers. Focus on rq.rqIndexer, rq.rqController = cache.NewIndexerInformer (...) Register ResourceEventHandlerFuncs:addQuota and enqueueResourceQuota in rqController. In addition, replenishmentController, err: = options.ControllerFactory.NewController (controllerOptions) is responsible for the creation of replenishmentController. Six kinds of replenishmentSource are registered in NewRegistry, so here replenishmentControllers will add the corresponding six replenishmentController.

# ResourceQuotaController.Run

After the ResourceQuotaController is created, the task processing begins by executing the Run method.

Pkg/controller/resourcequota/resource_quota_controller.go:227// Run begins quota controller using the specified number of workersfunc (rq * ResourceQuotaController) Run (workers int, stopCh)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.