Shulou(Shulou.com)06/03 Report--

The performance of Istio has always been the focus of relevant manufacturers at home and abroad, and more attention has been paid to the impact of Istio on the request delay of data applications. According to the performance test results of Istio officials and related manufacturers, four-digit qps is obviously far from meeting the requirements of production. Since its release, Istio officials have been optimizing and enhancing its performance. At the same time, the reliability of Istio control surface is another important consideration standard of Istio used in production, and automatic expansion is naturally an important means to ensure reliability. Let's start from the perspective of performance testing to understand the performance testing methods and benchmarks officially provided by Istio, which are mainly divided into the following four aspects.

First, function level testing

Istio provides performance test cases at the function level. Developers can use better targeted performance optimization. No expansion is done here, and interested friends can refer to:

Https://raw.githubusercontent.com/istio/istio/release-1.0/mixer/test/perf/singlecheck_test.go

II. End-to-end testing benchmark

In order to better track the performance problems of Istio, Istio provides a special testing tool for Isito performance testing-Fortio. You can easily deploy Forito through the kubernetes cluster. The installation links for the test tools are as follows:

Https://github.com/istio/istio/tree/release-1.0/tools#istio-load-testing-user-guide .

The following figure is the organization chart of the test tool, in which the upper part is the two services S1 and S2 managed by the Istio service grid, in which the S1 service turns off the mixer reporting function (the implementation scheme of the mixer telemetry function has always been controversial, it is generally believed that the sidecar reporting telemetry data to the mixer impairs the sidecar's request forwarding ability to some extent), and the request flows into the system through ingressgateway and then distributed to each service through sidecar. Is a typical way to access Istio services. The second half is the service access mode in the classic kubernetes cluster. The request is forwarded through the proxy of K8s and the load is balanced to each pod. The comparison between the two shows the performance loss of Istio access mode compared with classical mode. Here are some features of Fortio:

A) Fortio is a fast, compact, reusable, embeddable go library as well as command-line tools and server processes that include a simple Web UI and graphical representation of results

B) Fortio is also 100% open source, has no external dependencies except go and gRPC, and can easily reproduce the official Istio performance test scenarios and adapt to the variants or scenarios you want to explore

C) Fortio pressure tests the service with the specified qps per second, records the histogram that affects the delay, and calculates the statistical values such as percentage, average, tp99 and so on.

Steps for installing Fortio in kubernetes:

Kubectl-n fortio run fortio-image=istio/fortio:latest_release-port=8080

Kubectl-n fortio expose deployment fortio-target-port=8080-type=LoadBalancer

Third, test benchmark in real scene

Acmeair (aka BluePerf) is a customer-like micro-service application implemented in Java. This application runs on WebSphere Liberty and simulates the operation of a virtual airline.

Acmeair consists of the following microservices:

A) Flight Service retrieves flight route data The booking service invokes it to check the mileage of the reward operation (Acmeair customer loyalty Program).

B) customer service stores, updates and retrieves customer data It is used by Auth services to log in and subscribe to services for reward operations.

C) booking services store, update and retrieve booking data

D) if the user / password is valid, the Auth service generates a JWT.

E) the main service mainly includes the presentation layer (web page) that interacts with other services. This allows the user to interact directly with the application through the browser, but this action is not performed during the load test.

This simulates the airline's operating system demo, which can better simulate Istio applications in the actual production environment. Interested friends can use the following link to learn about it: https://github.com/blueperf

Build automated test results on a daily basis

Istio and IBM will test the performance of the daily build of Istio and publish the test results for your reference. Automated testing includes performance test results of end-to-end use case fortio and application-level bluePerf. Friends who are interested in Isito performance but don't have the time and energy to do performance testing can follow the official daily performance test results and keep track of the latest progress in Istio performance optimization.

● https://fortio-daily.istio.io/

● https://ibmcloud-perf.istio.io/regpatrol/

Here, let's take a look at the performance test results of IBM and analyze them.

The performance test comparison of IBM mainly includes three parts. The first part is the performance comparison between different Release versions, in which the performance tests of 0.6.0, 0.7.1, and 0.8.0 are listed. The indicator data here is qps. It can be seen that the data between the first three versions are very similar, and there is no significant improvement. And the comparison between Istio and IngressOnly shows that Istio has caused considerable performance loss. It can only achieve more than 30% qps without Istio, so you can see that Istio needs to be further optimized in terms of performance.

RowRelease (A) Istio Full (B) No Mixer (C) Ingress Only (A) / (C)% (B) / (C)% 10.6.013071987380434.452.220.7.112942050367135.830.8.013352222370836.059.9

The second part is the performance of the current daily build of release, which can be compared with the impact of daily changes on performance. Here we list some of them. For more detailed information, you can see that the recent daily build has improved compared to the 1.0 baseline version.

The last part is the daily build performance test results of the master branch. It can be seen that the performance test results of the latest master branch have been greatly improved compared with the baseline version, but the problem of serious loss of QPS still exists, at the same time, the thousand-level QPS can not really meet the production demand, we look forward to the development and progress of Istio.

5. Reliability and scalability of Isito

The function and fault impact of each component of the control plane are summarized, and the results are as follows:

The function and failure of the component affect the external traffic entrance of istio-ingressgateway. The failure of this component will cause the entire application service to be inaccessible from the outside. It is recommended to set up multiple instances to enhance the reliability of istio-telemetry Mixer related components, which are used to collect telemetry data reported by envoy. The failure of this component will cause various monitoring operation and maintenance plug-ins to be unable to collect data. At the same time, the component will bear a heavy load in high concurrency situations, so it is recommended to set it to multiple instances. Enhanced reliability istio-policy Mixer related components, used to interact with envoy, check needs to report data, determine cache content, hanging will affect check related functions istio-pilot controls the startup and parameter configuration of envoy in sidecar, that is, the actual issuer of traffic rules, hanging will cause the new policy configuration to fail istio-sidecar-injector is used to achieve automatic sidecar injection, hanging will cause sidecar not to be injected automatically, and at the same time If the injection policy is set to must be injected (policy is Fail), the pod of the new creating will not be able to start istio-citadel for security-related functions. Failure will lead to authentication and invalidation of security-related functions.

A) taking into account the reliability of the Istio control plane and the efficient use of resources, it is recommended that important components be set to multiple instances, including:

● ingressgateway: as an external traffic entry, the external traffic of all services managed by the service grid can only enter into the service grid through gateway, which is strongly related to business logic. It is recommended to configure it as multi-instance.

● mixer: the collection end of telemetry data, which is used to summarize the log and monitoring data reported by all services in the service grid. It handles a large amount of data. It is recommended to set it to multiple instances and allocate more resources.

The running pressure of other control plane components of ● is not high, and the number of instances can be adjusted as needed.

B) the container automatically expands and shrinks. Istio sets an automatic scaling policy for the main components gateway,pilot and mixer, and the policy can be configured during installation. Let's take pilot as an example to take a look at its automatic scaling configuration. The CPU occupancy rate of 55% is used as the threshold to scale up and reduce the number of pod instances:

C) High availability (HA), you can set the affinity policy for the Istio control plane component as needed, so that several different pod of the same control plane component run on different node to ensure the high availability state of the Istio control plane. Take pilot as an example, Istio adds a node anti-affinity strategy, so that pod does not run on nodes with the same architecture and operating system as much as possible. You can add affinity and anti-affinity strategy as needed.

D) Health check, Istio also configures a health check policy for the control plane component to ensure that when the control plane component is abnormal, K8s can pull it back up. Also take sidecarinjector as an example, set up the start health check readinessProbe and run-time health check livenessProbe to ensure the normal operation of the container:

VI. Summary

Again, performance and reliability is a vital part of Istio production availability. Istio has done a very powerful function, and is constantly improving and developing, but in terms of performance and reliability, Istio still has a long way to go. Among them, the impact of the routing and forwarding of microservice sidecar and mixer telemetry on request delay are two obstacles in front of the improvement of Istio performance. We work together and pay attention to the hope that Istio will mature, develop and set sail as soon as possible.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.