Shulou(Shulou.com)05/31 Report--

This article is about how Kubernetes programming in yaml is shared with you. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article.

Introduction

Performance testing is a routine requirement in daily development work, which is used to find out the performance of the service.

So how to do performance testing? It is either done by coding, writing a bunch of scripts and disposing of it, or it is based on the platform, in the process defined by the platform. For the latter, usually due to the complexity of target scenarios, such as deployment of specific workload, observation of specific performance items, network access problems, etc., often lead to high cost performance testing platform to meet the needs of changing development scenarios.

In the context of cloud native, can this problem be better solved?

First look at two yaml files:

Performance-test.yaml describes the operation flow in K8s:

Create a Namespace for testing

Initiate monitoring of Deployment creation efficiency and success rate

The following action is repeated N times: ① uses the workload template to create Deployment; ② and wait for Deployment to become Ready

Delete Namespace for testing

Basic-1-pod-deployment.yaml describes the workload template used

Performance-test.yaml:

ApiVersion: aliyun.com/v1alpha1kind: Beidoumetadata: name: performance namespace: beidouspec: steps:-name: "CreateNamespace If Not Exits" operations:-name: "create namespace" type: Task op: CreateNamespace args:-name: NSvalue: "Monitor DeploymentCreationEfficiency" operations:-name: "Begin To Monitor DeploymentCreationEfficiency" type: Task op: DeploymentCreationEfficiency args:-name: NSvalue: beidou-name "Repeat 1 Times" type: Task op: RepeatNTimes args:-name: TIMESvalue: "1"-name: ACTIONreference: id: deployment-operation-name: "Delete namespace" operations:-name: "delete namespace" type: Task op: DeleteNamespace args:-name: NSvalue: beidou-name: FORCEvalue: "false" references:-id: deployment-operationsteps:-name: "Prepare Deployment" operations:-name: "Prepare Deployment" type: Taskop: PrepareBatchDeploymentsargs:-name: NS value: beidou-name: NODE_TYPE value: ebm-name: BATCH_NUM value: "1"-name: TEMPLATE value: ". / templates/basic-1-pod-deployment.yaml"-name: DEPLOYMENT_REPLICAS value: "1"-name : DEPLOYMENT_PREFIX value: "ebm"-name: "WaitFor Deployments To Be Ready" type: Taskop: WaitForBatchDeploymentsReadyargs:-name: NS value: beidou-name: TIMEOUT value: "3m"-name: CHECK_INTERVAL value: "2s"

Basic-1-pod-deployment.yaml:

ApiVersion: apps/v1kind: Deploymentmetadata: labels:app: basic-1-podspec: selector:matchLabels: app: basic-1-pod template:metadata: labels:app: basic-1-podspec: containers:-name: nginximage: registry-vpc.cn-hangzhou.aliyuncs.com/xxx/nginx:1.17.9imagePullPolicy: Alwaysresources: limits:cpu: 2memory: 4Gi

Then execute performance-test.yaml through a command line tool:

$beidou server-c ~ / .kube/config services/performance-test.yaml

The execution effect is as follows (it takes time for each Deployment to create, all Deployment to create time-consuming TP95 values, and whether each Deployment is created successfully).

These metrics are output according to the Prometheus standard and can be collected by Prometheus server. Combined with Grafana, the performance test data can be displayed visually.

By expressing ideas in yaml, orchestrating the operation and monitoring of K8s resources, you no longer have to worry about the implementation of performance testing: d

Why are you programming in yaml?

Performance testing and regression testing are of great help for service quality assurance and need to be done, but the conventional implementation methods need to invest more time and energy in the initial stage, and the maintenance cost is relatively high after the new changes.

Usually this process implements atomic operations in code, such as creating Deployment, detecting Pod configuration, etc., and then combining atomic operations to meet the requirements, such as creating Deployment-> waiting for Deployment ready-> checking Pod configuration, etc.

Is there any way to achieve it at a low cost and reuse the existing experience?

Atomic operations can be encapsulated as primitives, such as CreateDeployment, CheckPod, and then express the process through the structure of yaml, then ideas can be described by yaml rather than code, and yaml files already written by others can be reused to solve the needs of certain scenarios.

That is, programming in yaml reduces repetitive code work, describes logic in a declarative way, and uses yaml files to meet scene-level reuse.

There are many types of declarative operation services in the industry, such as Ansible in the field of operation and maintenance, Argo Workflow in SaltStack,Kubernetes, and clusterloader2. Their ideas are similar as a whole, the operations used in high frequency are encapsulated as primitives, and users express the operation logic through primitives.

Through the declarative method, the operation oriented to K8s is abstracted into keywords in yaml, serial, parallel and other control logic is provided in yaml, then the work you want to do can be fully described through the yaml file.

This idea is similar to Argo Workflow, but with finer granularity than Argo, focusing on operation functions:

The design and implementation of the service are briefly described below.

Design and implementation 1. Service form

The user describes the operation logic declaratively in yaml.

Delivered as all-in-one binary tools or Operator

The service has a built-in implementation of common primitives, which are provided in yaml as keywords

Support the configuration of native K8s resources.

two。 Design

The core of this scheme lies in the design of configuration management, which configures the operation flow, and has the following concepts from top to bottom:

Choreography of Service:Modules or Tasks

Module: a task scenario that is a collection of operation units (including templates/ directories, a collection of template files that can be used to configure K8s native resources)

Task: an operation unit that uses plugin and parameters to perform operations

Plugin: operation instructions, similar to functions in a development language.

Abstract the common operations in the target scenario, which are primitives that can be used in yaml, corresponding to the above Plugin:

K8s correlation

CreateNamespace

DeleteNamespace

PrepareSecret

PrepareConfigMap

PrepareBatchDeployments

WaitForBatchDeploymentsReady

Etc.

Observational correlation

DeploymentCreationEfficiency

PodCreationEfficiency

Etc.

Test item correlation

CheckPodAnnotations

CheckPodObjectInfo

CheckPodInnerStates

Etc.

Control statement correlation

RepeatNTimes

Etc.

The relationship between the above four concepts is as follows:

Examples can be found in the yaml file at the beginning of the article, corresponding to form 2.

3. Core implementation

CRD Design:

Package v1alpha1import (corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1") / / BeidouType is the type related to Beidou execution.type BeidouType stringconst (/ / BeidouTask represents the Task execution type.BeidouTask BeidouType = "Task") / / + genclient// + k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object// Beidou represents a crd used to describe serices.type Beidou struct {metav1.TypeMeta `json: ", inline" `metav1.ObjectMeta `json: "metadata,omitempty" protobuf: "bytes 1protobuf spec,omitempty nameplate metadata "`Spec BeidouSpec `json:" spec,omitempty "protobuf:" bytes,2,opt,name=spec "`Status BeidouStatus `json:" status,omitempty "protobuf:" bytes,3,opt,name=status "`} / / BeidouSpec is the spec of a Beidou.type BeidouSpec struct {Steps [] BeidouStep `json:" steps "protobuf:" bytes,1,opt,name=steps "`References [] BeidouReference `json:" references "protobuf:" bytes,2,opt " Name=references "`} / / BeidouStep is the spec of step.type BeidouStep struct {Name string `json:" name "protobuf:" bytes,1,opt,name=name "`Operations [] BeidouOperation `json:" operations "protobuf:" bytes,2,opt,name=operations "`} / / BeidouOperation is the spec of operation.type BeidouOperation struct {Name string `json:" name "protobuf:" bytes,1,opt,name=name "`Type BeidouType `json:" type "protobuf:" bytes,2,opt Name=type "`Op string 'json:" op "protobuf:" bytes,3,opt,name=op "`Args [] BeidouArg `json:" args "protobuf:" bytes,4,opt,name=args "`} / / BeidouArg is the spec of arg.type BeidouArg struct {Name string `json:" name "protobuf:" bytes,1,opt,name=name "`Value string `json:" value,omitempty "protobuf:" bytes,2,opt Name=value "`Reference BeidouOperationReference 'json:" reference,omitempty "protobuf:" bytes,3,opt,name=reference "`Tolerations [] corev1.Toleration `json:" tolerations,omitempty "protobuf:" bytes,4,opt,name=tolerations "`Checking [] string `json:" checking,omitempty "protobuf:" bytes,5,opt,name=checking "`} / / BeidouOperationReference is the spec of operation reference.type BeidouOperationReference struct {ID string `json:" id "protobuf:" bytes,1,opt Name=id "`} / / BeidouReference is the spec of reference.type BeidouReference struct {ID string `json:" id "protobuf:" bytes,1,opt,name=id "`Steps [] BeidouStep `json:" steps "protobuf:" bytes,2,opt,name=steps "`} / / BeidouStatus represents the current state of a Beidou.type BeidouStatus struct {Message string `json:" message "protobuf:" bytes,1,opt Name=message "`} / / + k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object// BeidouList is a collection of Beidou.type BeidouList struct {metav1.TypeMeta `json:", inline "`metav1.ListMeta `json:" metadata "protobuf:" bytes,1,opt,name=metadata "`Items [] Beidou `json:" items "protobuf:" bytes,2,opt,name=items "`}

Core process:

/ / ExecSteps executes steps.func ExecSteps (ctx context.Context, steps [] v1alpha1.BeidouStep, references [] v1alpha1.BeidouReference) error {logger, _: = ctx.Value (CtxLogger). (* log.Entry) var hasMonitored boolfor i, step: = range steps {for j Op: = range step.Operations {switch op.Op {case "DeploymentCreationEfficiency": if! hasMonitored {defer func () {err: = monitor.Output () if err! = nil {logger.Errorf ("Failed to output:% s" Err)} ()} hasMonitored = true} err: = ExecOperation (ctx, op, references) if err! = nil {return fmt.Errorf ("failed to run operation% s:% s", op.Name, err)}} return nil} / / ExecOperation executes operation.func ExecOperation (ctx context.Context Op v1alpha1.BeidouOperation, references [] v1alpha1.BeidouReference) error {switch op.Type {case v1alpha1.BeidouTask:if! tasks.IsRegistered (op.Op) {return ErrNotRegistered} if! tasks.DoesSupportReference (op.Op) {return ExecTask (ctx, op.Op, op.Args)} return ExecTaskWithRefer (ctx, op.Op, op.Args, references)} return nil} / / ExecTask executes a task.func ExecTask (ctx context.Context, opname string) Args [] v1alpha1.BeidouArg) error {switch opname {case tasks.CreateNamespace:var ns stringfor _, arg: = range args {switch arg.Name {case "NS": ns = arg.Value}} return op.CreateNamespace (ctx, ns) / /.} / /} / ExecTaskWithRefer executes a task with reference.func ExecTaskWithRefer (ctx context.Context, opname string, args [] v1alpha1.BeidouArg References [] v1alpha1.BeidouReference) error {switch opname {case tasks.RepeatNTimes:var times intvar steps [] v1alpha1.BeidouStepvar err errorfor _, arg: = range args {switch arg.Name {case "TIMES": times, err = strconv.Atoi (arg.Value) if err! = nil {return ErrParseArgs} case "ACTION": for _ Refer: = range references {if refer.ID = = arg.Reference.ID {steps = refer.Stepsbreak}} return RepeatNTimes (ctx, times, steps)} return ErrNotImplemented}

An example of an implementation of an operation primitive:

/ / PodAnnotations is an operation used to check whether annotations of Pod are expected.func PodAnnotations (ctx context.Context, data PodAnnotationsData) error {kclient, ok: = ctx.Value (tasks.KubernetesClient). (kubernetes.Interface) if! ok {return tasks.ErrNoKubernetesClient} pods, err: = kclient.CoreV1 (). Pods (data.Namespace) .list (metav1.ListOptions {}) if err! = nil {return fmt.Errorf ("failed to list pods in ns% s:% s", data.Namespace, err)} for _ Pod: = range pods.Items {if pod.Annotations = = nil {return fmt.Errorf ("pod% s in ns% s has no annotations", pod.Name, data.Namespace)} for _, annotation: = range data.Exists {if _, exists: = pod.Annotations [annotation] ! exists {return fmt.Errorf ("annotation% s does not exist in pod% s in ns% s", annotation, pod.Name, data.Namespace)}} for k, v: = range data.Equal {if pod.Annotations [k]! = v {return fmt.Errorf ("value of annotation% s is not% s in pod% s in ns% s", k, v, pod.Name Data.Namespace)} return nil} this is how Kubernetes is programmed in yaml The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.