Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to build a distributed pressure measurement system based on kubernetes. The content of the article is of high quality. Therefore, Xiaobian shares it with you as a reference. I hope you have a certain understanding of relevant knowledge after reading this article.

Three techniques for distributed pressure measurement

In principle, three technologies are needed:

programming technology

Here we chose Python, which our team is familiar with. Different teams can have different choices.

pressure engine

We use Locust, because it is written in python scripts, in fact, can also be replaced by jmeter and other pressure testing engine,

kubernetes

It mainly uses its service orchestration technology to schedule a resource. After our test, if it is a normal cluster, it takes 90 seconds for all physical nodes to be ready when they need to be ejected. However, if elastic clusters are used, it can be compressed to 15~20 seconds. Therefore, elastic clusters are recommended. In principle, the whole technical framework is divided into three types: master node, slave node and monitor node:

master node

Responsible for task management and data collection aggregation, not for pressure testing tasks

from node

Responsible for pressure test task

monitoring node

transmitting the result from the master node to the web service processing end through the webhook;

In addition, the status and logs of these nodes will be collected through the api of k8s. According to the resources requested by the master and slave nodes in the pressure test task, the cluster will scale the nodes in advance and distribute the tasks to different nodes to achieve the purpose of dynamically improving the pressure test capability.

Pressure measurement process

The process perceived by the user is shown on the right. The pressure test set includes multiple pressure test scenarios. The pressure test task is generated by writing pressure test script and configuring pressure test parameters, and finally the pressure test report is generated.

On the left is the process of Python controlling the cluster to generate tasks. Specifically, after rendering the yaml files that generate different tasks, generate corresponding job pods, and then continuously feed back the pod status, logs, and pressure curve results on the page.

The technology used in the whole process is not very advanced, mainly an exploration in cluster applications.

implementation method

Use yaml to orchestrate job services. For example, slave node is mainly to declare a job-type workload, render the generated task from the node name and the namespace generated by the task, and then set up our pressure test base image and start command. Here we use several techniques of kubernetes. One is to perform internal parsing through hostAliases, so that some intranet agents can be pressure tested. The other is to declare the application resource CPU. In order to scale the resources provided by the physical node in advance before the task starts, there is also a way to mount the executable file through configmap, so that the startup command with changing parameters can be injected without rebuilding the image.

Then let's talk about our code framework, which is mainly divided into these modules:

k8s module, providing API functions such as creating and destroying namespaces or pods, viewing status, pulling logs, etc.

Basic image, relatively simple, mainly installed some basic general libraries, and then opened some internal use of ports;

Task orchestration declaration files, including the node services I mentioned above;

The task core method class mainly implements the above process code and provides some methods, which are not specifically expanded due to space constraints.

And then finally, let's look at the effect:

This is the management interface of our pressure measurement system. What we see now is the pressure measurement set, which is convenient for centralized management.

This is to create a pressure test scenario, and write python pressure test scripts based on this scenario, and set our task parameters.

This is the detail page of pressure test task. You can view pressure test parameters, status and node status and view logs.

This is a chart generated in real time during the pressure test process, which can be analyzed based on the chart situation.

Future directions for improvement

Engine type or version allows optional replacement;

Automatic pressure test plan with batch timing and stages;

Correlate all resource related graphs into a more detailed report;

Task resource restriction and use approval;

Report analysis conclusion archiving, related problem tracking processing results archiving;

Comparison and display of multiple pressure test results under the same conditions;

Manage the lifecycle of tasks in a more cloud-native way;

How to build a distributed pressure measurement system based on kubernetes is shared here. I hope the above content can be of some help to everyone and learn more knowledge. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.