Shulou(Shulou.com)06/01 Report--

How to access Kubernetes CRD in client-go? for this question, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

Install kubernetes

The easiest way, of course, is to use sealos, whether in a stand-alone or production environment or in the cloud.

Wget https://github.com/fanux/sealos/releases/download/v2.0.7/sealos & & chmod + x sealos & & mv sealos / usr/bin sealos init-- passwd YOUR_SERVER_PASSWD\-- master 192.168.0.2-- master 192.168.0.3-- master 192.168.0.4\-node 192.168.0.5\-- pkg-url https://sealyun.oss-cn-beijing.aliyuncs.com/ Cf6bece970f6dab3d8dc8bc5b588cc18-1.16.0/kube1.16.0.tar.gz\-- version v1.16.0

The above two commands set up a kubernetes 1.16.0 cluster, as do other versions

This paper is very practical, and it is difficult to find more complete and perfect relevant documents.

Take a chestnut:

When we implement the virtual machine CRD, the agent on the node needs to query the virtual machine CRD. Obviously, we will not operate through controller, so we need to know how to operate CRD directly with client-go.

Create CRDapiVersion: "apiextensions.k8s.io/v1beta1" kind: "CustomResourceDefinition" metadata: name: "projects.example.sealyun.com" spec: group: "example.sealyun.com" version: "v1alpha1" scope: "Namespaced" names: plural: "projects" singular: "project" kind: "Project" validation: openAPIV3Schema: required: ["spec"] properties: spec: Required: ["replicas"] properties: replicas: type: "integer" minimum: 1

This can be generated using kubebuilder or operator-framework. I'm too tired to write it myself.

To define a custom resource definition, you need to consider the API group name (in this case, example.sealyun.com). By convention, this is usually the domain name of the domain you control (for example, your organization's domain) to prevent naming conflicts. Then the name of the CRD follows the pattern., so in this case projects.example.sealyun.com.

Typically, you want to validate the data that users store in custom resources against a specific schema. This is what spec.validation.openAPIV3Schema is for: it contains an JSON schema that describes the format that resources should have.

Use kubectl to create a resource definition. If you use kubebuilder, you can directly make & & make deploy:

> kubectl apply-f projects-crd.yamlcustomresourcedefinition "projects.example.sealyun.com" created

You can create an instance of this new resource type:

ApiVersion: "example.sealyun.com/v1alpha1" kind: "Project" metadata: name: "example-project" namespace: "default" spec: replicas: 1 > kubectl apply-f project.yamlproject "example-project" created > kubectl get projectsNAME AGEexample-project 2m create Golang client

Next, we will use the client-go package to access these custom resources.

Define the type

Kubebuilder, etc., will be automatically generated for you. I also add the relevant instructions of this piece here to explain everything clearly.

First define the type of custom resource. It is a good practice to organize these types by the API group version; for example, api/types/v1alpha1/project.go:

Package v1alpha1import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" type ProjectSpec struct {Replicas int `json: "replicas" `} type Project struct {metav1.TypeMeta `json: ", inline" `metav1.ObjectMeta `json: "metadata,omitempty" `Spec ProjectSpec `json: "spec"`} type ProjectList struct {metav1.TypeMeta `json: ", inline" `metav1.ListMeta `json: "metadata,omitempty" `Items [] Project `json: "items" `}

This metav1.ObjectMeta type contains the properties of a typical metadata

Define the DeepCopy method

Each type provided by Kubernetes API (in this case, Project and ProjectList) needs to implement the k8s.io/apimachinery/pkg/runtime.Object interface. The interface defines two methods, GetObjectKind () and DeepCopyObject (). The first method is already provided by the embedded metav1.TypeMeta architecture; the second one you must implement on your own.

The DeepCopyObject method is designed to make a deep copy of the object. Because this involves a lot of boilerplate code, many tools usually generate these methods automatically. For the purposes of this article, we will do it manually. Continue to add a second file to the same package for deepcopy.go:

Package v1alpha1import "k8s.io/apimachinery/pkg/runtime" / / DeepCopyInto copies all properties of this object into another object of the// same type that is provided as a pointer.func (in * Project) DeepCopyInto (out * Project) {out.TypeMeta = in.TypeMeta out.ObjectMeta = in.ObjectMeta out.Spec = ProjectSpec {Replicas: in.Spec.Replicas }} / / DeepCopyObject returns a generically typed copy of an objectfunc (in * Project) DeepCopyObject () runtime.Object {out: = Project {} in.DeepCopyInto (& out) return & out} / / DeepCopyObject returns a generically typed copy of an objectfunc (in * ProjectList) DeepCopyObject () runtime.Object {out: = ProjectList {} out.TypeMeta = in.TypeMeta out.ListMeta = in.ListMeta if in.Items! = nil {out.Items = make ([] Project Len (in.Items)) for I: = range in.Items {in.Items [I] .DeepCopyInto (& out.Items [I])}} return & out}

Registration type

Next, you need to make the client library aware of the new type. Allows the client to automatically handle new types when communicating with the API server.

To do this, register.go, add a new file to the package:

Package v1alpha1import (metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"k8s.io/apimachinery/pkg/runtime"k8s.io/apimachinery/pkg/runtime/schema") const GroupName = "example.sealyun.com" const GroupVersion = "v1alpha1" var SchemeGroupVersion = schema.GroupVersion {Group: GroupName, Version: GroupVersion} var (SchemeBuilder = runtime.NewSchemeBuilder (addKnownTypes) AddToScheme = SchemeBuilder.AddToScheme) func addKnownTypes (scheme * runtime.Scheme) error {scheme.AddKnownTypes (SchemeGroupVersion, & Project {} & ProjectList {}) metav1.AddToGroupVersion (scheme, SchemeGroupVersion) return nil}

This code doesn't actually do anything (except to create a new runtime.SchemeBuilder instance). The important part is the AddToScheme function, which is the type created in runtime.SchemeBuilder. Once the Kubernetes client is initialized to register your type, you can call this function later from any part of the client code.

Build HTTP client

After defining types and adding methods to register them in the global schema builder, you can now create HTTP clients that can load custom resources.

To do this, add the following code to the package's main.go file:

Package mainimport ("flag"log"time"k8s.io/apimachinery/pkg/runtime/schema"k8s.io/apimachinery/pkg/runtime/serializer"github.com/martin-helmich/kubernetes-crd-example/api/types/v1alpha1"k8s.io/client-go/kubernetes/scheme"k8s.io/client-go/rest"k8s.io/client-go/tools/clientcmd") var kubeconfig stringfunc init () {flag.StringVar (& kubeconfig) "kubeconfig", "", "path to Kubernetes config file") flag.Parse ()} func main () {var config * rest.Config var err error if kubeconfig = = "" {log.Printf ("using in-cluster configuration") config, err = rest.InClusterConfig ()} else {log.Printf ("using configuration from'% s'", kubeconfig) config, err = clientcmd.BuildConfigFromFlags (") Kubeconfig)} if err! = nil {panic (err)} v1alpha1.AddToScheme (scheme.Scheme) crdConfig: = * config crdConfig.ContentConfig.GroupVersion = & schema.GroupVersion {Group: v1alpha1.GroupName, Version: v1alpha1.GroupVersion} crdConfig.APIPath = "/ apis" crdConfig.NegotiatedSerializer = serializer.DirectCodecFactory {CodecFactory: scheme.Codecs} crdConfig.UserAgent = rest.DefaultKubernetesUserAgent () exampleRestClient Err: = rest.UnversionedRESTClientFor (& crdConfig) if err! = nil {panic (err)}}

You can now query all custom resources in the example.sealyun.com/v1alpha1API group using the content created in exampleRestClient. An example might be as follows:

Result: = v1alpha1.ProjectList {} err: = exampleRestClient. Get (). Resource ("projects"). Do (). Into & result)

To use your API in a more type-safe manner, it's usually best to wrap these operations in your own client set. To do this, create a new subpackage clientset/v1alpha1. First, implement an interface that defines the API group type and move the configuration settings from your main method to the constructor of the clientset (NewForConfig in the following example):

Package v1alpha1import ("github.com/martin-helmich/kubernetes-crd-example/api/types/v1alpha1"k8s.io/apimachinery/pkg/runtime/schema"k8s.io/apimachinery/pkg/runtime/serializer"k8s.io/client-go/kubernetes/scheme"k8s.io/client-go/rest") type ExampleV1Alpha1Interface interface {Projects (namespace string) ProjectInterface} type ExampleV1Alpha1Client struct {restClient rest.Interface} func NewForConfig (c * rest.Config) (* ExampleV1Alpha1Client Error) {config: = * c config.ContentConfig.GroupVersion = & schema.GroupVersion {Group: v1alpha1.GroupName, Version: v1alpha1.GroupVersion} config.APIPath = "/ apis" config.NegotiatedSerializer = serializer.DirectCodecFactory {CodecFactory: scheme.Codecs} config.UserAgent = rest.DefaultKubernetesUserAgent () client, err: = rest.RESTClientFor (& config) if err! = nil {return nil, err} return & ExampleV1Alpha1Client {restClient: client} Nil} func (c * ExampleV1Alpha1Client) Projects (namespace string) ProjectInterface {return & projectClient {restClient: c.restClient, ns: namespace,}}

The above is the encapsulation of client

Next, you need to implement a specific set of Project clients to access custom resources (note that the above example already uses the ProjectInterface and projectClient types that we still need to provide). Projects.go creates a second file in the same package:

Package v1alpha1import ("github.com/martin-helmich/kubernetes-crd-example/api/types/v1alpha1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"k8s.io/apimachinery/pkg/watch"k8s.io/client-go/kubernetes/scheme"k8s.io/client-go/rest") type ProjectInterface interface {List (opts metav1.ListOptions) (* v1alpha1.ProjectList, error) Get (name string, options metav1.GetOptions) (* v1alpha1.Project Error) Create (* v1alpha1.Project) (* v1alpha1.Project, error) Watch (opts metav1.ListOptions) (watch.Interface, error) / /.} type projectClient struct {restClient rest.Interface ns string} func (c * projectClient) List (opts metav1.ListOptions) (* v1alpha1.ProjectList, error) {result: = v1alpha1.ProjectList {} err: = c.restClient. Get (). Namespace (c.ns). Resource ("projects"). VersionedParams (& opts, scheme.ParameterCodec) Do (). Into (& result) return & result, err} func (c * projectClient) Get (name string, opts metav1.GetOptions) (* v1alpha1.Project, error) {result: = v1alpha1.Project {} err: = c.restClient. Get (). Namespace (c.ns). Resource ("projects"). Name (name). VersionedParams (& opts, scheme.ParameterCodec) Do (). Into (& result) return & result, err} func (c * projectClient) Create (project * v1alpha1.Project) (* v1alpha1.Project, error) {result: = v1alpha1.Project {} err: = c.restClient. Post (). Namespace (c.ns). Resource ("projects"). Body (project). Do (). Into (& result) return & result, err} func (c * projectClient) Watch (opts metav1.ListOptions) (watch.Interface, error) {opts.Watch = true return c.restClient. Get (). Namespace (c.ns). Resource ("projects"). VersionedParams (& opts, scheme.ParameterCodec) Watch ()}

There are still some Delete Update methods missing above, just copy them, or refer to the implementation of pod

It's very easy to use it again:

Import clientV1alpha1 "github.com/martin-helmich/kubernetes-crd-example/clientset/v1alpha1" / /... func main () {/ /... ClientSet, err: = clientV1alpha1.NewForConfig (config) if err! = nil {panic (err)} projects, err: = clientSet.Projects ("default") .List (metav1.ListOptions {}) if err! = nil {panic (err)} fmt.Printf ("projects found:% + v\ n", projects)}

Create informer

When building a Kubernetes operator, you usually want to be able to listen for newly created or updated events. In theory, you can call the List () method periodically and check to see if new resources have been added.

In most cases, it is handled by initially loading all relevant instances of the resource using the initial List () and then subscribing to the relevant events using Watch (). The local cache is then built using the initial list of objects and updates received from informer, which allows quick access to any custom resources without having to access the API server every time.

This pattern is so common that the client-go library provides a cache package for this: Informerk8s.io/client-go/tools/cache from the package. You can build a new Informer for custom resources, as follows:

Package mainimport ("time"github.com/martin-helmich/kubernetes-crd-example/api/types/v1alpha1" client_v1alpha1 "github.com/martin-helmich/kubernetes-crd-example/clientset/v1alpha1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"k8s.io/apimachinery/pkg/runtime"k8s.io/apimachinery/pkg/util/wait"k8s.io/apimachinery/pkg/watch"k8s.io" / client-go/tools/cache ") func WatchResources (clientSet client_v1alpha1.ExampleV1Alpha1Interface) cache.Store {projectStore ProjectController: = cache.NewInformer (& cache.ListWatch {ListFunc: func (lo metav1.ListOptions) (result runtime.Object, err error) {return clientSet.Projects ("some-namespace") .list (lo)}, WatchFunc: func (lo metav1.ListOptions) (watch.Interface, error) {return clientSet.Projects ("some-namespace") .Watch (lo)} }, & v1alpha1.Project {}, 1*time.Minute, cache.ResourceEventHandlerFuncs {},) go projectController.Run (wait.NeverStop) return projectStore}

The NewInformer method returns two objects: the second return value, the controller controls the List () and Watch () calls and populates the first return value, and some information heard by the Store cache. You can now easily access CRD using store, listing all CRD or accessing them by name. The store function returns the interface {} type, so you must cast them back to the CRD type:

Store: = WatchResource (clientSet) project: = store.GetByKey ("some-namespace/some-project"). (* v1alpha1.Project)

In many cases, there is no need to call apiserver to relieve the pressure on apiserver.

Access in kubebuilder

By getting the reader in manager, but here you can only read but not write, you need mgr.GetClient () if you write, but this must run for a long time.

For more details, I would like to join the apiclient function PR.

Package mainimport ("context"fmt"os"k8s.io/apimachinery/pkg/types" v1 "github.com/fanux/sealvm/api/v1"github.com/prometheus/common/log"k8s.io/apimachinery/pkg/runtime" ctrl "sigs.k8s.io/controller-runtime" clientgoscheme "k8s.io/client-go/kubernetes/scheme ") var scheme = runtime.NewScheme () func init () {v1.AddToScheme (scheme) clientgoscheme.AddToScheme (scheme)} func main () {mgr Err: = ctrl.NewManager (ctrl.GetConfigOrDie (), ctrl.Options {Scheme: scheme,}) if err! = nil {os.Exit (1)} client: = mgr.GetAPIReader () / / for long running, use mgr.GetClient () ctx: = context.Background () name: = types.NamespacedName {Namespace: "default" Name: "virtulmachine-sample"} vm: = & v1.VirtulMachine {} if err: = client.Get (ctx, name, vm) Err! = nil {log.Error (err, "unable to fetch vm")} else {fmt.Println (vm.Spec.CPU, vm.Spec.Memory, vm)}}

It is recommended to call client directly:

Package mainimport ("context"fmt"sigs.k8s.io/controller-runtime/pkg/client"k8s.io/apimachinery/pkg/types" v1 "github.com/fanux/sealvm/api/v1"github.com/prometheus/common/log"k8s.io/apimachinery/pkg/runtime" ctrl "sigs.k8s.io/controller-runtime" clientgoscheme K8s.io/client-go/kubernetes/scheme) var scheme = runtime.NewScheme () func init () {v1.AddToScheme (scheme) clientgoscheme.AddToScheme (scheme)} func getClient () (client.Client Error) {config: = ctrl.GetConfigOrDie () if config = = nil {return nil, fmt.Errorf ("config is nil")} options: = ctrl.Options {Scheme:scheme} / / Set default values for options fields / / options = setOptionsDefaults (options) / / mapper Err: = options.MapperProvider (config) / / if err! = nil {/ / log.Error (err, "Failed to get API Group-Resources") / / return nil, err / /} client, err: = client.New (config, client.Options {Scheme: options.Scheme}) if err! = nil {return nil, err} return client Nil} func main () {client,err: = getClient () if err! = nil {fmt.Println ("client is nil", err) return} ctx: = context.Background () name: = types.NamespacedName {Namespace: "default", Name: "virtulmachine-sample"} vm: = & v1.VirtulMachine {} if err = client.Get (ctx, name) Vm) Err! = nil {log.Error (err, "unable to fetch vm")} else {fmt.Println (vm.Spec.CPU, vm.Spec.Memory, vm)}} the answer to the question on how to access Kubernetes CRD in client-go is shared here. I hope the above content can be of some help to you, if you still have a lot of questions to solve. You can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.