Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to test Kubernetes clusters". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's ideas to study and learn "how to test Kubernetes clusters".

Why test it?

There are many reasons why you should use automated cluster testing. Kubernetes is a very complex system, which is composed of several independent components. Just because they work in one configuration doesn't mean they will work perfectly in every configuration.

There are several ways to start a Kubernetes cluster by using an installer such as kubeadm or kops, or other ways to set up a cluster. For each approach, there are many configuration options that make the two configurations look different. Test the cluster before users use it to ensure that our settings work as expected, thus providing users with an available and valuable platform. However, cluster setup is only the first step, because updates to Kubernetes are released frequently. At this point, testing has its purpose, and basic clustering functionality is still available after the available verification update.

However, Kubernetes is not just a platform, it is also a framework that extends through plug-ins and add-ons. As a result, no Kubernetes cluster runs in the same way. Kubernetes's own tests do not include plug-ins, so the scope of testing depends only on the tests of the plug-in developer. To ensure that our plug-ins do not interfere with each other or affect Kubernetes, we should also test them in the setup. This is especially true for plug-ins that you write yourself.

Conformance testing

Kubernetes conformance testing is a subset of end-to-end test cases that test the core functions of Kubernetes. In the words of the developer team, "the current tests are only available, the necessary functions of API", and the clusters that pass these tests are qualified and can be certified by the CNCF K8s qualified working group.

Currently, functions that can be tested include creating API objects, launching containers and mounting basic volumes on nodes, and testing kubectl. Optional features such as role-based access control, NetworkPolicy and PodSecurityPolicy are not included. Plug-ins and add-ons are also mostly not subject to conformance testing, for example, testing DNS, but some tests implicitly require networking using plug-ins such as Weave or Calico. In the future, the plug-in may also be tested through a conformance test configuration file, but for now you need to test it separately.

However, their verification of basic cluster functions makes conformance testing an ideal starting point for testing clusters. To perform these tests, we can use tools such as kubetest or sonobuoy.

Kubetest

Kubetest is a CLI tool used in the Kubernetes pipeline to run end-to-end tests. Because conformance tests are a subset of end-to-end tests, you can use kubetest to run them on the cluster by filtering the tests to run.

# to install the latest version of kubetest go get-u k8s.io/test-infra/kubetest # test needs to match the cluster version K8SystVERSION version $(kubectl version-o json | jq-r '.serverversion.gitVersion') # set this variable to y export KUBERNETES_CONFORMANCE_TEST=y # display settings configuration export KUBECONFIG= "$HOME/.kube/config" # run the test Skeleton is the current cluster kubetest-provider=skeleton-test-test_args= "- ginkgo.focus=\ [Conformance\]"-extract ${K8S_VERSION}

After downloading and extracting the Kubernetes binaries required for our cluster version, all tests marked [Conformance] will be performed. You can also use parallel tests to reduce the time it takes to run tests.

Kubetest-- provider=skeleton-- test-- ginkgo.parallel-- test_args= "--ginkgo.focus=\ [Conformance\]-ginkgo.skip=\ [Serial\]"-- extract ${K8S_VERSION} kubetest-- provider=skeleton-- test-- test_args= "--ginkgo.focus=\ [Serial\]. *\ [Conformance\]"-extract ${K8S_VERSION}

You can also run kubetest extract only once, and then run the test from the Kubernetes directory to speed up subsequent execution. For debugging purposes, you can also tell kubetest not to delete the namespace where the test failed:

Kubetest-provider=skeleton-test-test_args= "- ginkgo.focus=\ [Conformance\]-delete-namespace-on-failure=false"-extract ${K8S_VERSION}

Although kubetest is highly customizable, it is not necessarily end-user-oriented, and its logos are rarely recorded and often confusing. To simply run conformance tests, Heptio has released Sonobuoy, which simplifies the process.

Sonobuoy

Sonobuoy is a diagnostic tool that runs Kubernetes conformance tests. It consists of a CLI that launches a pod to manage the test runs in the cluster and lets you then retrieve the results. It is a simple out-of-the-box solution and a standard tool for running conformance tests.

We can also choose to use kubetest and Sonobuoy to run other tests of the end-to-end test suite to test some of our plug-ins. For example, if we want to use network policies in a cluster, we might want to test whether they are enforced. You can use Sonobuoy to perform the following basic tests:

# execute sonobuoy, overriding the default skip and focus parameters sonobuoy run-- e2emurfocus = "\ [Feature:NetworkPolicy\]"-- e2emurskip= ""

These tests create basic network policies and Pod that are constrained by them, and verify that they are enforced in the cluster (note that they do not verify that the policies that exist in the cluster work as expected, so you can use tools such as netassert or illuminatio). Similar tests exist for other functions.

Write your own end-to-end tests

You can also write your own end-to-end tests for cluster settings. This is particularly useful when running locally developed attachments because unit tests can hardly mimic the behavior of a running Kubernetes cluster. To develop tests in Golang, you can use Kubernetes's own E2E framework.

If you use other programming languages, you can still use the kubernetes client library, but you must write some boilerplate code yourself, for example, to set up and demolish test namespaces. Unit test frameworks such as pytest are still useful for separating test cases and running tests from collecting results.

Whether you are new to Kubernetes or have been running clusters in a production environment for many years, you should start testing these clusters immediately. Run Sonobuoy to do conformance testing in the pipeline, start some E2E tests on the functionality you use, and develop your own tests for components that cause too many cluster failures. This will make it easier to operate and give you peace of mind.

Thank you for your reading, the above is the content of "how to test Kubernetes clusters", after the study of this article, I believe you have a deeper understanding of how to test Kubernetes clusters, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.