Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is Little's Law". In daily operation, I believe many people have doubts about what is Little's Law. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful for you to answer the doubts about "what is Little's Law?" Next, please follow the editor to study!

1. Preface

Before introducing today's protagonist Little's Law, it is necessary to unify our "basic language" for describing "performance". After all, it is impossible to communicate without a language.

2. The basic language of "performance"

Different service devices have different definitions of performance, for example, CPU mainly depends on the main frequency, and disk mainly depends on IOPS. This paper mainly discusses the back-end software service performance (such as api service, database service, etc.). Once you have defined the scope, you should give a definition of performance: performance is the ability of the service to process requests. There are three common metrics to measure performance: number of concurrent users, throughput, and response time.

2.1 number of concurrent users

Refers to the number of users who actually send requests to the service, which requires attention to the difference between the number of online users and the number of online users; for example, at one point, the number of online users is 1000, of which only 100 users trigger interaction with the remote service. In this case, for the remote service, the number of concurrent users is 1000, not 100.

2.2 Throughput

The number of requests processed per unit time.

2.3 response time

The corresponding English is response time, and in some places it is indicated by latency, that is, delay. It is necessary to count the response time in a period of time and find out the eigenvalues to represent the response time. Common characteristic values include average value, maximum value, minimum value and quantile value.

3. The definition of the protagonist Little's Law debut 3.1Little's Law

The number of users served at the same time in a stable system is equal to the speed at which the user arrives in the system multiplied by the time that each user resides in the system, which is suitable for all scenarios that need to be queued. The corresponding formula is expressed as follows:

N = X * R, where N represents the simultaneous activity of users in the system, and X represents the rate at which users arrive at the system one after another, in a stable (this word is very important!) The state (the speed at which the user arrives at the system is equal to the speed at which the user leaves the system) is the system throughput, and R represents the average residence time for each user in the system.

For example, if you are waiting in line to enter a physical examination center, you can know how much longer you have to wait by estimating the rate at which people enter.

The physical examination center can accommodate about 600 people, and the time for each person to complete the physical examination is 2 hours, that is, 2 hours and 600 people, according to the formula: X=N/R=600 / 2 hours = 300 people per hour, so enter at the rate of 300 people per hour. If there are 300 people in front of you now, you will have to wait about an hour before entering the physical examination center. 3.2 relationship between Little's Law and performance indicators

The relationship between the three metrics mentioned in the previous section can be expressed by Little's Law, because when user requests are constantly sent to the server for processing, it is a queuing scenario. The corresponding formula for Little's Law to this scenario is:

Number of concurrency = Throughput * response time 3.3.Sense the presence of Little's Law through the instance

The following is mainly from the interface service and mysql service two instances to observe the existence of Little's Law.

3.3.1 introduction to the preparation process of the instance 3.3.1.1 of the interface service

Exposing an interface based on springboot will sleep for a period of time according to the parameter values, so that the client's stress testing tool can control the response time of the interface.

@ RestControllerpublic class ApiLatency {@ RequestMapping ("/ test/latency/ {ms}") public String latency (@ PathVariable long ms) {try {Thread.sleep (ms);} catch (InterruptedException e) {e.printStackTrace ();} return "Hello World!";}}

Pressure test the above interface by setting different number of concurrency through jmeter.

3.3.1.2 result display and Analysis

In the initial stage, the response time is basically stable, and the throughput increases with the increase of the number of concurrency; with the increase of the number of concurrency, the system reaches the "inflection point", the throughput begins to decrease, and the response time begins to increase; continue to increase the number of concurrency, the system load is overloaded, thus entering the supersaturated zone, the response time increases sharply and the throughput decreases sharply. Before and just entering the inflection point, the system can be considered to be "stable", and the concurrency, throughput and average response time are in line with the Little's Law formula.

3.3.2 introduction to the preparation process for instance 3.3.2.1 of mysql Service

Prepare a Tencent Cloud mysql service (version 5.6, configured with 2-core 4G) and a CVM (Tencent Cloud private network test)

Test the mysql service with sysbench (version 0.5)

# # data preset sysbench-- mysql-host=192.168.0.10-- mysql-port=3306-- mysql-user=root-- mysql-password=test-- mysql-db=loadtest-- mysql-table-engine=innodb-- test=/usr/local/share/sysbench/tests/include/oltp_legacy/oltp.lua-- oltp_tables_count=8-- oltp-table-size=4000000-- rand-init=on prepare## execute shell script to test for i in 5 7 8 10 12 25 50 128 200 400 700 1000dosysbench-mysql -host=192.168.0.10-- mysql-port=3306-- mysql-user=root-- mysql-password=test-- mysql-db=loadtest-- test=/usr/local/share/sysbench/tests/include/oltp_legacy/oltp.lua-- oltp_tables_count=8-- oltp-table-size=4000000-- num-threads=$ {I}-- oltp-read-only=off-- rand-type=special-- max-time=180-- max-requests=0-- percentile=99-- oltp-point-selects=4-- report-interval=3-- forced-shutdown=1 run | tee -a sysbench.$ {I} .oltp.txtdone # # Clean up data sysbench-- mysql-host=192.168.0.10-- mysql-port=3306-- mysql-user=root-- mysql-password=test-- mysql-db=loadtest-- mysql-table-engine=innodb-- test=/usr/local/share/sysbench/tests/include/oltp_legacy/oltp.lua-- oltp_tables_count=8-- oltp-table-size=4000000-- meaning of the parameters in the rand-init=on cleanup## script-oltp_tables_count=8 Indicates that the number of tables used for testing is 8. -- oltp-table-size=4000000, indicating that the number of table rows used in this test is 4 million rows. -- num-threads=n, indicating that the number of concurrency of client connections in this test is n. -- oltp-read-only=off, off means that the test closes the read-only test model and uses the read-write mixed model. -- rand-type=special, indicating that the stochastic model is specific. -- max-time=180, indicating that the execution time of this test is 180 seconds. -- max-requests=0,0 means that the total number of requests is not limited, but is tested by max-time. -- percentile=99, which means to set the sampling ratio. The default is 95%, that is, 1% of long requests are discarded and the maximum value is taken in the remaining 99%. -- oltp-point-selects=4, which represents the sql test command in the oltp script. The number of select operations is 4, and the default value is 1. 3.3.2.2 result presentation and Analysis

In the initial stage, the response time is basically stable, and the throughput increases with the increase of the number of concurrency; with the increase of the number of concurrency, the system reaches the "inflection point", the throughput begins to decrease, and the response time begins to increase; continue to increase the number of concurrency, the system load is overloaded, thus entering the supersaturated zone, the response time increases sharply and the throughput decreases sharply. Before and just entering the inflection point, the system can be considered to be "stable", and the concurrency, throughput and average response time are in line with the Little's Law formula.

4. Summary 4.1 Little's Law formula system is "stable".

Based on the above two measured examples, we can use the above diagram to show the relationship among concurrency, throughput and response time.

At the beginning, it is a "linear growth area", when the response time is basically stable, and the throughput increases with the increase of the number of concurrent users.

When the resource utilization of the system is saturated, the system reaches the "inflection point". With the increase of the number of concurrent users, the throughput begins to decline and the response time begins to increase.

Continue to increase the number of concurrent users, the system load is overloaded, thus entering the supersaturated zone, when the response time increases sharply and the throughput decreases sharply.

Before and just entering the inflection point, the system can be considered to be "stable", and the concurrency, throughput and average response time are in line with the Little's Law formula.

4.2 concurrency is not the actual number of users

For example, if you set the number of threads in jmeter to 80, the measured result does not mean the performance of your service in the case of 80 users. In the actual scenario, the pressure caused by 1000 real users may not be as great as the 80 threads (or virtual users) here. I tend to think of concurrency as a measure of stress. 80% of concurrency must have greater pressure on services than 60% of concurrency on services.

4.3 low latency (response time) is not necessarily high throughput, and high latency (response time) is not necessarily low throughput.

If a program has only one thread and this thread can handle 10 events per second, then we say that the delay for this program to handle a single event is 100ms, and the throughput is 10 times per second.

If a program has four threads, each thread can handle five events per second, then we say that the delay for this program to handle a single event is 200ms, and the throughput is 20 times per second.

If a program has 1 thread, each thread can handle 20 events per second, then we say that the delay for this program to handle a single event is 50ms, and the throughput is 20 times per second.

As we know from Little's Law, concurrency = throughput * response time, so the relationship between latency and throughput is affected by concurrency, and it is irregular to put aside concurrency to find the relationship between the other two.

4.4 response time should be linked to throughput

It makes no sense for performance to look only at throughput without looking at response time. From the previous analysis, we know that with the gradual increase of the number of concurrency, the throughput has a process of first rising and then falling, that is, there is the same value of throughput, which corresponds to different response time under different concurrency. For example, if the throughput of an interface is 10000TPS and the response time is 5 seconds, then this 10000TPS is meaningless.

4.5 response time, throughput and success rate should be linked

For example, in the interface test, when the number of concurrency reaches 4000 and the error rate reaches 40%, then the 1270.8TPS is meaningless.

4.6 more, faster and better

I understand that the performance of the service is to process requests quickly and well. "many" means large throughput, "fast" means short response time, and "good" means low error rate as far as possible.

4.7 last but not least

In Little's Law, the response time we always use is "average response time", but in practice, we usually use the "quantile" of response time as the statistical value of response time to measure performance, and the average is only used as an auxiliary reference. The reason you should understand, for example, "average salary" usually does not have much reference value, it is possible that many people are averaged.

5. Expansion

In his "Method for Estimating the Number of Concurrent Users" paper published in 2004, Eric Man Wong introduced a formula for estimating the number of concurrent users in the system:

This formula is equivalent to Little's Law. It specifically discusses the equivalence between the estimation of the number of concurrent users with reference to Eric's and Little's law.

At this point, the study of "what is Little's Law" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.