Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces "how to achieve API Basics" in detail, the content is detailed, the steps are clear, and the details are handled properly. I hope this "how to achieve API Basics" article can help you solve your doubts.

Introduction

So-called cloud native applications can clearly run on K8s and use K8s api and resources as their extensions.

On the one hand, cloud native applications can easily migrate to different clouds. On the other hand, benefit from the concise and definable declarative api provided by K8s.

Extension Patterns

Cloud providers: in-tree controller manager:

Kubelet: network, devices, storage, container runtimes

Extend kubectl with plugins

Extensions in the API server, dynamic admission control with webhook

Custom resources and custom controllers.

Custom apiserver

Scheduler extensions.

Controllers and operators

Controller: implements a control loop that monitors the cluster state from API server and adjusts the current state to the expected state.

Operator: refers to the way controller+ customizes resources to do things like application lifecycle management.

The controller loop

Read the state of the resource, more event-driven.

Change the state of objects in a cluster or external cluster.

Update the status of resources stored in etcd through apiserver.

Repeat the cycle. Go back to step 1.

These three steps are consistent regardless of whether the implementation of controller is complex or not. Read resource status-> change the world- > update resource status.

Informers:

Responsible for the desired state of watch, implementing the resync mechanism to enhance periodic tuning, which is usually used to ensure that the cluster state and the desired state in memory do not flutter.

Work queues:

It is implemented in client-go 's workqueue package to provide event handler with work queues to "store" queues for status change operations and their various retry operations. When there is an error updating the status of the current resource object, the resource needs to be reordered.

Events

The control plane of K8s is decoupled by event-driven, and other distributed systems trigger various behaviors through remote procedure calls. Controller requests for adding, deleting and changing APISERVER through changes in watch K8s objects.

Example:

When a user creates a dp, dp controller creates a rs through dp informer.

Rs controller creates a new rs through rs informer and creates a pod object.

Scheduler observes a new pod through pod informer, and its spec.nodeName is empty, so it puts the pod in the scheduling queue.

At the same time, kubelet sees a new pod through pod informer, and its spec.nodeName is empty. So there is no match-to-kubelet nodeName, ignoring this pod and waiting for the next event.

Scheduler takes the pod from the workqueue and, by updating the spec.nodeName field, dispatches to the specified node with sufficient resources, and writes it to apiserver.

Kubelet continues to compare spec.NodeName with its own nodeName because of the pod update event. After the match, the container of the pod is started and the status information started by the container is written to the pod status, which is returned and stored to the apiserver.

Rs controller noticed a change in pod.

When the pod is finally destroyed, the kubelet watch goes to the pod event and the pod state is set to the "terminated" state and updated to apiserver.

Rs controller observes the ending pod and decides that the pod must be replaced. So delete the pod of the terminated state through apiserver and create a new one.

A series of independent control loops through changes to objects in watch apiserver, and these change events trigger changes through informers.

Watch events and event objects are two things:

The former is used to drive informers by establishing a link of http streaming between apiserver and controllers.

The latter is a resource, such as pod,dp,services, with special aging properties and automatically disappears from the etcd.

Edge-driven triggers or Level-driven triggers

The former triggers handler based on the instant of a state value change, while the latter triggers handler if the state is verified to match the expected value within a certain period of time.

The latter is more like a kind of polling polling, where the capacity can be expanded to a specified number, and the delay that the controller notices the change depends on the interval of polling polling and the response time of apiserver. Related to many asynchronous controllers, it takes a while for the system to reach the desired state.

The former is more efficient, and the delay depends on the worker thread that runs when the controller processes the event. Therefore, K8s is based on event-driven, namely edge-driven triggers.

Reconciliation with resync: tune continuously every 5 minutes.

Edge-triggered, level-driven

Level-triggering-and-reconciliation

Changing Cluster Objects or the External worldOptimistic Concurrency

Monasana:

The second stage:

Sharing status:

Parallel scheduling architecture:

The new generation of concurrent scheduling architecture relies on shared state and uses optimistic concurrency control to achieve scalability and performance scalability.

The scene of Lok View Lock in K8s is:

In retry loop, after getting the latest state of the foo object, try to update real world and foo status based on foo spec, and the actual modification is performed before the Update event.

The foo object returned through client.Get, containing the version of the resource, the ObjectMeta structure. This structure updates the data of etcd through a write operation when client.Update is called. The so-called resource version is actually stored in etcd as a key-value pair. Etcd maintains a counter counter, which is incremented by one each time the value of key is modified.

In API machinery code, the resource version is handled like an arbitrary string, but it actually follows some rules. The actual implementation details are on the etcd storage backend.

Essence: collision errors in controllers are common and elegant in handling and expectation.

Operators

Operator is an application-specific controller that extends K8s API to create, configure, and manage complex examples of stateful applications.

An operator includes a CRD+ controller.

Kubernetes API Basics

As the only component blocking the storage of etcd, apiserver interacts directly with etcd. The main responsibility is to act as server for k8s api and proxy for cluster components.

Servo API means reading status and operating status.

Declarative state management

By comparing the spec of the desired state with the value of the current actual state

For example, when you specify 20 copies in dp's declaration yaml, dp controller reads dp spec and creates rs, actually taking over rs-, that is, a specific number of pods. If any copy is kneeling, dp controller will let you feel it in the state.

APIServer Processes Requests

DefaultBuildHandlerChain ()

WithPanicRecovery (): handles recovery and logging panic

Basics of client-go

Repos

ClientLibrary

The K8s project provides client-go (https://github.com/kubernetes/client-go) as the user's development toolset, provides many types of API support, and contains a lot of common library code.

K8S API Types

Client-go manages client interfaces, such as PODS,services,deployments, which are maintained in this repo (types such as https://github.com/kubernetes/api) https://github.com/kubernetes/api) pod are defined in k8s.io/api/core/v1/types.go, and these files are automatically generated and maintained by code gen.

/ / Pod is a collection of containers that can run on a host. This resource is created// by clients and scheduled onto hosts.type Pod struct {metav1.TypeMeta `json: ", inline" `/ / Standard object's metadata. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata / + optional metav1.ObjectMeta `json: "metadata,omitempty" protobuf: "bytes,1,opt,name=metadata" `/ / Specification of the desired behavior of the pod. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status / + optional Spec PodSpec `json: "spec,omitempty" protobuf: "bytes,2,opt,name=spec" `/ / Most recently observed status of the pod. / / This data may not be up to date. / / Populated by the system. / / Read-only. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status / / + optional Status PodStatus `json: "status,omitempty" protobuf: "bytes,3,opt,name=status" `

}

API machinery

The third repo (https://github.com/kubernetes/apimachinery) contains all the common build modules, not limited to container management, but also other build APIs. You can see many definitions of API types under pkg/api/meta/v1, including the following:

ObjectMeta: Name + Namespaces + ResourceVersion + Labels + Annotations

TypeMeta: Kind + APIVersion

GetOptions

ListOptions

The build module is used to create k8s client objects that represent resources that can be accessed in the k8s cluster.

Version and compatibility

The api of K8s is differentiated by version. On the one hand, different versions of client access API server, and if the version does not match, the request will fail. Client is bound to a specific version, and application developers need to choose the correct one.

The corresponding relationship between its version and K8s version is shown in the figure:

| | Branch | Canonical source code location | Maintenance status | | `release- 1.4` | Kubernetes main repo, 1.4 branch | =-| | `release- 1.5` | Kubernetes main repo, 1.5 branch | =-| | `release- 2.0` | Kubernetes main repo, 1.5 branch | =-| | `release- 3.0` | Kubernetes main repo, 1.6branch | =-| `release- 4.0` | Kubernetes main repo, 1.7branch | =-| `release- 5.0` | Kubernetes main repo, 1.8branch | =-| `release- 6.0` | Kubernetes main repo | 1.9 branch | =-| | `release- 7.0` | Kubernetes main repo, 1.10 branch | =-| | `release- 8.0` | Kubernetes main repo, 1.11 branch | =-| | `release- 9.0` | Kubernetes main repo, 1.12 branch | =-| | `release- 10.0` | Kubernetes main repo, 1.13 branch | =-| `release- 11.0` | Kubernetes main repo, 1.14 branch | ✓ | | `release- 12.0` | Kubernetes main repo, 1.15 branch | ✓ | | `release- 13.0` | Kubernetes main repo | 1.16 branch | ✓ | | `release- 14.0` | Kubernetes main repo, 1.17 branch | ✓ | | client-go HEAD | Kubernetes main repo, master branch | ✓ |

Key:

✓ Changes in main Kubernetes repo are actively published to client-go by a bot

= Maintenance is manual, only severe security bugs will be patched.

-Deprecated; please upgrade.

What common libraries are used by the code: sharedInformer, workqueue, event, leaderelection, rest, scheme, flowcontrol, codec

K8S objects in Go

K8s resource is an instance of kind that provides services as a resource structure in API server.

Object in K8s implements the runtime.Object interface, which comes from the k8s.io/apimachinery/pkg/runtime package. Among them, another interface that schema.ObjectKind mainly implements comes from k8s.io/apimachinery/pkg/runtime/schema.

These Go implementations of object as data structures are mainly used to return and set GroupVersionKind, as well as for deep copy.

Type Object interface {GetObjectKind () schema.ObjectKind DeepCopyObject () Object} type ObjectKind interface {/ / SetGroupVersionKind sets or clears the intended serialized kind of an / / object. Passing kind nil should clear the current setting. SetGroupVersionKind (kind GroupVersionKind) / / GroupVersionKind returns the stored group, version, and kind of an / / object, or nil if the object does not expose or provide these fields. GroupVersionKind () GroupVersionKind} TypeMeta

The getter and setter types of the k8s object schema.ObjectMeta (k8s.io/api) are implemented by nesting metav1.TypeMeta (k8s.io/apimachinery/meta/v1).

/ / TypeMeta describes an individual object in an API response or request// with strings representing the type of the object and its API schema version.// Structures that are versioned or persisted should inline TypeMeta.//// + k8s:deepcopy-gen=falsetype TypeMeta struct {/ / Kind is a string value representing the REST resource this object represents. / / Servers may infer this from the endpoint the client submits requests to. / / Cannot be updated. / / In CamelCase. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds / + optional Kind string `json: "kind,omitempty" protobuf: "bytes,1,opt,name=kind" `/ / APIVersion defines the versioned schema of this representation of an object. / / Servers should convert recognized schemas to the latest internal value, and / / may reject unrecognized values. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources / / + optional APIVersion string `json: "apiVersion,omitempty" protobuf: "bytes,2,opt,name=apiVersion" `} / / Pod is a collection of containers that can run ona host. This resource is created// by clients and scheduled onto hosts.type Pod struct {metav1.TypeMeta `json: ", inline" `/ / Standard object's metadata. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata / + optional metav1.ObjectMeta `json: "metadata,omitempty" protobuf: "bytes,1,opt,name=metadata" `/ / Specification of the desired behavior of the pod. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status / + optional Spec PodSpec `json: "spec,omitempty" protobuf: "bytes,2,opt,name=spec" `/ / Most recently observed status of the pod. / / This data may not be up to date. / / Populated by the system. / / Read-only. / / More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status / / + optional Status PodStatus `json: "status,omitempty" protobuf: "bytes,3,opt,name=status" `

}

Client-go-based applications these fields are empty in memory and will not have actual values until they are actually serialized to json or pb. These conventions are implemented automatically in the version sequence.

ObjectMeta

With TypeMeta, most top-level objects will also include metav1.ObjectMeta, which comes from k8s.io/apimachinery/pkg/meta/v1.

(staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go)

Type ObjectMeta struct {Name string `json: "name,omitempty" `Namespace string `json: "namespace,omitempty" `UID types.UID `json: "uid,omitempty" `ResourceVersion string `json: "resourceVersion,omitempty" `CreationTimestamp Time `json: "creationTimestamp,omitempty" `DeletionTimestamp * Time `json: "deletionTimestamp,omitempty" `Labels map [string] string `json: "labels,omitempty" `Annotations map [string] string `json: "annotations,omitempty" `.}

Metav1.ObjectMeta basically contains all metadata information, such as name,namespace,resource version, some timestamps, as well as labels and annotation. Client-go can not read or write resource version, but it is the important information of the core work in the whole K8s code.

As part of the ObjectMeta information, it comes from the key field of all the data in the etcd.

Client Sets

Kubernetes.NewForConfig (config) returns a client set that allows you to access most of the API groups and resources defined in k8s.io/api, except for APIServices (aggregated API servers) and CRD.

The main interface of clientSet is as follows:

Type Interface interface {Discovery () discovery.DiscoveryInterface AdmissionregistrationV1alpha1 () admissionregistrationv1alpha1.AdmissionregistrationV1alpha1Interface AdmissionregistrationV1beta1 () admissionregistrationv1beta1.AdmissionregistrationV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Admissionregistration () admissionregistrationv1beta1.AdmissionregistrationV1beta1Interface AppsV1beta1 () appsv1beta1.AppsV1beta1Interface AppsV1beta2 () appsv1beta2.AppsV1beta2Interface AppsV1 () appsv1.AppsV1Interface / / Deprecated: please explicitly pick a version if possible. Apps () appsv1.AppsV1Interface AuthenticationV1 () authenticationv1.AuthenticationV1Interface / / Deprecated: please explicitly pick a version if possible. Authentication () authenticationv1.AuthenticationV1Interface AuthenticationV1beta1 () authenticationv1beta1.AuthenticationV1beta1Interface AuthorizationV1 () authorizationv1.AuthorizationV1Interface / / Deprecated: please explicitly pick a version if possible. Authorization () authorizationv1.AuthorizationV1Interface AuthorizationV1beta1 () authorizationv1beta1.AuthorizationV1beta1Interface AutoscalingV1 () autoscalingv1.AutoscalingV1Interface / / Deprecated: please explicitly pick a version if possible. Autoscaling () autoscalingv1.AutoscalingV1Interface AutoscalingV2beta1 () autoscalingv2beta1.AutoscalingV2beta1Interface BatchV1 () batchv1.BatchV1Interface / / Deprecated: please explicitly pick a version if possible. Batch () batchv1.BatchV1Interface BatchV1beta1 () batchv1beta1.BatchV1beta1Interface BatchV2alpha1 () batchv2alpha1.BatchV2alpha1Interface CertificatesV1beta1 () certificatesv1beta1.CertificatesV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Certificates () certificatesv1beta1.CertificatesV1beta1Interface CoreV1 () corev1.CoreV1Interface / / Deprecated: please explicitly pick a version if possible. Core () corev1.CoreV1Interface EventsV1beta1 () eventsv1beta1.EventsV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Events () eventsv1beta1.EventsV1beta1Interface ExtensionsV1beta1 () extensionsv1beta1.ExtensionsV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Extensions () extensionsv1beta1.ExtensionsV1beta1Interface NetworkingV1 () networkingv1.NetworkingV1Interface / / Deprecated: please explicitly pick a version if possible. Networking () networkingv1.NetworkingV1Interface PolicyV1beta1 () policyv1beta1.PolicyV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Policy () policyv1beta1.PolicyV1beta1Interface RbacV1 () rbacv1.RbacV1Interface / / Deprecated: please explicitly pick a version if possible. Rbac () rbacv1.RbacV1Interface RbacV1beta1 () rbacv1beta1.RbacV1beta1Interface RbacV1alpha1 () rbacv1alpha1.RbacV1alpha1Interface SchedulingV1alpha1 () schedulingv1alpha1.SchedulingV1alpha1Interface SchedulingV1beta1 () schedulingv1beta1.SchedulingV1beta1Interface / / Deprecated: please explicitly pick a version if possible. Scheduling () schedulingv1beta1.SchedulingV1beta1Interface SettingsV1alpha1 () settingsv1alpha1.SettingsV1alpha1Interface / / Deprecated: please explicitly pick a version if possible. Settings () settingsv1alpha1.SettingsV1alpha1Interface StorageV1beta1 () storagev1beta1.StorageV1beta1Interface StorageV1 () storagev1.StorageV1Interface / / Deprecated: please explicitly pick a version if possible. Storage () storagev1.StorageV1Interface StorageV1alpha1 () storagev1alpha1.StorageV1alpha1Interface}

There are some unversioned methods in this interface: appsv1beta1.AppsV1beta1Interface.

> in the past, K8s had so-called internal clients, in order to make objects more common in memory and follow the specifications that were defined when needed. This specification is designed to abstract controller code from the version of API actually used, while making it easy to switch between different versions. In fact, in order to follow this specification, it adds a lot of complexity and pays an unworthy price. > in addition, there is no automatic negotiation mechanism for the interaction between client and APIServer. Although there are internal versions and clients, controller hard-coding to the specified version results in not being very compatible. > in recent versions, k8s code avoids using these builds as much as possible.

All clientset can access discovery client and are used as RESTMappers

Type AppsV1beta1Interface interface {RESTClient () rest.Interface ControllerRevisionsGetter DeploymentsGetter ScalesGetter StatefulSetsGetter}

Under each GroupVersion (AppsV1beta1), we find that there is a general RESTClient for the resources of API group.

After reading this, the article "how to achieve API Basics" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.