Shulou(Shulou.com)06/02 Report--

This article focuses on "Java uses JMH for benchmark performance analysis". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "Java uses JMH for benchmark performance analysis".

I. Preface

In their daily development work, developers may have these puzzles: how does the method they write perform? If new business logic is added to the original interface implementation, how much impact does it have on the performance of the entire interface? There are many ways to implement (or open source class library), which one has better performance?

II. Overview of JMH

1. What is JMH

   JMH, or Java Microbenchmark Harness, is a suite of tools dedicated to code microbenchmarking. What is Micro Benchmark? To put it simply, it is a benchmark test based on the method level, and the accuracy can reach the level of microseconds. It was developed by Oracle/openjdk internal development JIT compiler bosses, as a java method-level performance testing tool can be said to be the root is popular. (official address: http://hg.openjdk.java.net/code-tools/jmh/)

2. Typical scenarios suitable for JMH

   a, optimize hot methods, accurately know the time-consuming execution of a method, and the relationship between different inputs participating in the final actual time-consuming, so as to optimize pertinently

   b, find the best solution, verify the actual throughput of different implementation methods of interface methods, so as to determine the best implementation method. For example, when choosing json conversion tool, choose fastjson or gson, string concatenation use StringBuilder or add directly

   c, analyze the performance loss and the impact on the performance of the whole business method when adding new business code to the business logic of the original interface method. For example, in the original business logic, adding an operation to insert the operation log can analyze the performance impact of the new operation on the entire business method.

   d, the time spent in the analysis percentage, that is, the time spent in the percentage range when the test method is called many times, for example, when a test calls a method, the time consumed by a test call within 50% is 8.2 Ms 10.2ms/op, which is less than 99.99% of the time, and so on. (mode is Mode.SampleTime)

3. Basic concepts of JMH

   an and Mode: represent the modes in JMH testing. By default, there are five modes: Throughput (throughput), AverageTime (average time consuming), SampleTime (random sampling), SingleShotTime (single execution), and All (all four above).

   b, Fork: indicates the number of processes that JMH will use to test

   c, Warmup: indicate preheating. In HotSpot, the JIT compiler of JVM will compile and optimize the hot code, so in order to be closest to the real situation, it is necessary to preheat the test code to make the JIT compiler complete the optimization that may be needed, so as to make the final test results of JMH more accurate.

   d, Iteration: indicates the minimum test iteration unit in JMH, that is, the number of tests. The default value is 1 second at a time.

   e, Benchmark: used to mark the method that JMH will test. (similar to the @ Test comment in Junit)

III. The use of JMH

1. Run quickly

The basic use of    JMH takes only 2 steps. The first step is to introduce the maven dependency package, and the second step is to write test classes according to the tool framework template. The following is explained in detail by a simple example:

Example 1: the average time spent testing a method

Step 1: introduce maven dependency: (the author uses jmh version 1.21)

Org.openjdk.jmh jmh-core ${jmh.version} org.openjdk.jmh jmh-generator-annprocess ${jmh.version} provided

Step 2: write the test method:

Package com.xiaojiang;import org.openjdk.jmh.annotations.Benchmark;import org.openjdk.jmh.annotations.BenchmarkMode;import org.openjdk.jmh.annotations.Mode;import org.openjdk.jmh.annotations.OutputTimeUnit;import org.openjdk.jmh.runner.Runner;import org.openjdk.jmh.runner.options.Options;import org.openjdk.jmh.runner.options.OptionsBuilder;import java.util.concurrent.TimeUnit / * * @ Description:jmh uses the first example * @ Author: xiaojiang * @ Date: Created in 2019-06-19 * @ Version: 0.0.1 * / @ OutputTimeUnit (TimeUnit.MILLISECONDS) @ BenchmarkMode (Mode.AverageTime) public class JmhDemoOne {public static void main (String [] args) throws Exception {Options options = new OptionsBuilder () .include (JmhDemoOne.class.getName ()) .build () New Runner (options). Run ();} / * it takes time to test the draw of sayHello * * @ throws Exception * / @ Benchmark public void sayHello () throws Exception {/ / TODO business method, where it takes 10 ms TimeUnit.MILLISECONDS.sleep (10) to simulate the business in a dormant way;}}

Code description:

From the above example, we can find that a basic JMH test implementation is not very complex and is very similar to unit testing with Junit. The details are as follows:

   a, @ OutputTimeUnit and @ BenchmarkMode on the class name JmhDemoOne indicate that this is a JMH test class. (for more information on the meaning of the notes and more notes, please refer to the following JMH common notes for details)

   b, the main method for the entry of the main function specifies some basic test parameter options; (basically fixed writing. In fact, there are more related parameter methods that can be added, but the author suggests that these parameters should be directly added to the class by annotations, which is more convenient.

   c, specific methods for tagging benchmark (benchmark testing) through @ Benchmark annotations

Run the test method directly, and the console outputs the test results as follows: (the author's JDK version is 1.8 and the IDE tool is IDEA2018)

# JMH version: 1.2 percent VM version: JDK 1.8.0mm 144, Java HotSpot (TM) 64-Bit Server VM, 25.144-b01# VM invoker: d:\ Java\ jdk1.8.0_144\ jre\ bin\ java.exe# VM options:-javaagent:D:\ Program Files\ JetBrains\ IntelliJ IDEA 2018.2.5\ lib\ idea_rt.jar=55987:D:\ Program Files\ JetBrains\ IntelliJ IDEA 2018.2.5\ bin-Dfile.encoding=UTF-8# Warmup: 5 iterations, 10 s each# Measurement: 5 iterations 10 s each# Timeout: 10 min per iteration# Threads: 1 thread, will synchronize iterations# Benchmark mode: Average time, time/op# Benchmark: com.xiaojiang.JmhDemoOne.sayHello# Run progress: 0.005% complete ETA 00ms/opIteration 08Warmup Iteration: 1 of Warmup Iteration 1: 10.716 ms/op# Warmup Iteration 2: 10.640 ms/op# Warmup Iteration 3: 10.737 ms/op# Warmup Iteration 4: 10.693 ms/op# Warmup Iteration 5: 10.723 ms/opIteration 1: 10.716 ms/opIteration 2: 10.724 ms/opIteration 3: 10.772 ms/opIteration 4: 10.758 ms/opIteration 5: 10.709 ms/op# Run progress: 20.00% complete ETA 00Warmup Iteration 06Warmup Iteration 4: 10.744 ms/op# Warmup Iteration 1: 10.732 ms/op# Warmup Iteration 3: 10.748 ms/op# Warmup Iteration 4: 10.728 ms/op# Warmup Iteration 5: 10.760 ms/opIteration 1: 10.701 ms/opIteration 2: 10.709 ms/opIteration 3: 10.719 ms/opIteration 4: 10.714 ms/opIteration 5: 10.703 ms/op# Run progress: 40.00% complete ETA 00Warmup Iteration 05Warmup Iteration: 3 of Warmup Iteration 1: 10.729 ms/op# Warmup Iteration 2: 10.731 ms/op# Warmup Iteration 3: 10.728 ms/op# Warmup Iteration 4: 10.700 ms/op# Warmup Iteration 5: 10.709 ms/opIteration 1: 10.708 ms/opIteration 2: 10.701 ms/opIteration 3: 10.708 ms/opIteration 4: 10.726 ms/opIteration 5: 10.698 ms/op# Run progress: 60.00% complete ETA 00ms/opIteration 03Warmup Iteration 1: 10.724 ms/op# Warmup Iteration 2: 10.688 ms/op# Warmup Iteration 3: 10.748 ms/op# Warmup Iteration 4: 10.732 ms/op# Warmup Iteration 5: 10.772 ms/opIteration 1: 10.729 ms/opIteration 2: 10.688 ms/opIteration 3: 10.705 ms/opIteration 4: 10.687 ms/opIteration 5: 10.709 ms/op# Run progress: 80.005 complete ETA 00Warmup Iteration 01Warmup Iteration 4: 10.688 ms/op# Warmup Iteration 1: 10.696 ms/op# Warmup Iteration 3: 10.692 ms/op# Warmup Iteration 4: 10.684 ms/op# Warmup Iteration 5: 10.683 ms/opIteration 1: 10.719 ms/opIteration 2: 10.720 ms/opIteration 3: 10.695 ms/opIteration 4: 10.710 ms/opIteration 5: 10.760 ms/opResult "com. Xiaojiang.JmhDemoOne.sayHello ": 10.716 ±(99.9%) 0.016 ms/op [Average] (min Avg, max) = (10.687, 10.716, 10.772), stdev = 0.021 CI (99.9%): [10.700, 10.731] (assumes normal distribution) # Run complete. Total time: 00:08:24REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up onwhy the numbers are the way they are. Use profilers (see-prof,-lprof), design factorialexperiments, perform baseline and negative tests that provide experimental control, make surethe benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.Do not assume the numbers tell you what you want them to tell.Benchmark Mode Cnt Score Error UnitsJmhDemoOne.sayHello avgt 25 10.716 ±0.016 ms/opProcess finished with exit code 0

The test results show that:

The whole test result is divided into three blocks, the basic test parameter information, the test process, and the test results. The meaning of each line is specified as follows:

Block description parameter information (1-10 lines) 1:jmh version

2:jvm version Information

3:jvm program (jdk installation path)

4:jvm parameter configuration

5: preheating parameters: preheating times and duration of each time

6: test parameters: number of tests, duration of each time

7: timeout for each test iteration

8: number of test threads per test process

9: the mode of testing

10: test method test process (12-75 lines) 12-23: the first fork test (fork can be understood as an independent process)

12: progress of test completion, it is estimated that it will take time for the rest

13: how many times is the current fork

14-18: warm-up execution, each warm-up execution takes time

19-23: formal test execution, each test execution takes time

25-36: 2nd fork Test

38-49: 3rd fork test

51-62: 4th fork test

64-75: test results of the fifth fork test (78-95 lines) 78-81: test results, including test method, average time [percentage of tie time], maximum and minimum time, data dispersion of test results (stdev), etc.

84: total test time

86-90: interpretation of test results

92-93: test conclusion {Test method, test type (Mode), total test times (Cnt), test result (Score), error (Error), unit (Units)}

95: end

Note:

   a, Measurement, Fork, Warmup and other parameters in the test results are the default configuration values for JMH. In practice, we can specify relevant parameters as needed.

   B. you can run this test class directly in IDEA, or you can run it on the server in a jar package.

   c. The test results are directly output to the console. If necessary, you can output the test results to a file by adding the output method in options to specify the test result output directory, as follows:

Options options = new OptionsBuilder () .include (JmhDemoOne.class.getName ()) .output ("D:/JmhDemoOne.log") / / output the test results to the specified directory file .build (); 2. Detailed introduction of JMH common notes

The annotations introduce the @ BenchmarkMode benchmark mode. There are 5 optional values: (actually 4)

Mode.Throughput: throughput mode, that is, the throughput of the method per unit time

Mode.AverageTime: average time-consuming mode, that is, the average time spent on method execution within a certain number of tests

Mode.SampleTime: random sampling mode, that is, the final result is the distribution proportion of the sampling result

Mode.SingleShotTime: single execution mode, that is, it will only be executed once (the above modes are usually prewarmed and iterated multiple times, and this mode can be used to test certain scenarios, such as performance at cold start)

Mode.All: that is, all the above modes are executed once.

-

Usage example: (benchmark mode is average time-consuming mode)

@ BenchmarkMode (Mode.AverageTime) @ OutputTimeUnit the time unit of the test result. The values are the values in the java.util.concurrent.TimeUnit enumeration, usually seconds, milliseconds, and subtleties. (note that in different test modes, you need to choose the appropriate unit of time to get more accurate test results. )

-

Usage example: (benchmark result time unit is millisecond)

@ OutputTimeUnit (TimeUnit.MILLISECONDS) @ Benchmark benchmark, method-level annotations (configured on method names). Methods used to mark the need for benchmark (benchmark testing)

-

Usage example: (method requires benchmark)

@ Benchmark@Warmup prefetch parameters. Configure parameters related to preheating. The parameters are: iterations (preheating times), time (preheating time), and timeUnit (time unit)

-

Usage example: (preheat 10 times, 20s each time)

@ Warmup (iterations = 10, time = 20, timeUnit = TimeUnit.SECONDS) @ Measurement metric, that is, the benchmark basic parameter. The parameters are: iterations (number of tests), time (time per test), timeUnit (unit of time)

-

Usage example: (test 5 times, 30s each time)

@ Measurement (iterations = 5, time = 30, timeUnit = TimeUnit.SECONDS) @ Fork fork, that is, the number of processes. Used to configure how many processes will be used for testing

-

Usage example: (using 3 processes)

@ Fork (3) @ Threads thread count. The number of threads in each Fork (process) can generally be set to the number of test machine cpu cores.

-

Usage example: (using 4 threads)

@ Threads (4) @ Param member parameter, attribute-level annotation. It is used to test the performance of the method under different input parameters.

-

Example of usage: (input values are 1, 10, 100 respectively)

@ Param ({"1", "10", "100"}) @ Setup setting, method-level comments. Used to mark operations before benchmark, usually used to initialize parameter resources before testing, such as initializing database connections, etc.

-

Usage example: (initialization method)

@ Setup@TearDown disassembly, method-level comments. It is used to mark the operation after benchmark, and is usually used to recycle resources after testing, such as closing database connections, etc.

-

Usage example: (recycling method)

The @ TearDown@State state, which represents the available scope of a class / method, has three values:

Scope.Thread: default state, each thread assigns an exclusive instance

Scope.Benchmark: all threads in the test share the instance; (in the case of multithreaded testing)

Scope.Group: threads of the same group share an instance

-

Usage example: (default value, one instance per thread)

@ State (Scope.Thread) @ Group test group, method-level annotations. Suitable for grouping testing, where the number of threads in each group is different.

-

Usage example: (a group named "group_name")

@ Group ("group_name") @ GroupThreads group threads, method-level comments. Usually used with @ Group

-

Usage example: (the number of threads is 10)

@ GroupThreads (10) @ Timeout timeout. Timeout for each test iteration

-

Usage example: (the timeout for each test is 20min)

@ Timeout (time = 20, timeUnit = TimeUnit.MINUTES)

The above annotations are commonly used in JMH testing. Of course, JMH also has some other annotations, such as @ CompilerControl, @ AuxCounters, and so on. These annotations can usually be used to meet the requirements of specific test scenarios. If necessary, please refer to the official example. Official demo is more detailed and easier to understand and learn.

3. More examples

In this section, the author will show the basic use of JMH through a few small examples.

Example 2:

When testing and verifying string concatenation processing, StringBuilder is better than direct addition.

Package com.xiaojiang;import org.openjdk.jmh.annotations.*;import org.openjdk.jmh.runner.Runner;import org.openjdk.jmh.runner.options.Options;import org.openjdk.jmh.runner.options.OptionsBuilder;import java.util.concurrent.TimeUnit / * * @ Description:jmh uses the second example * @ Author: xiaojiang * @ Date: Created in 2019-06-19 * @ Version: 0.0.1 * / @ OutputTimeUnit (TimeUnit.SECONDS) @ BenchmarkMode (Mode.Throughput) @ Warmup (iterations = 5, time = 1, timeUnit = TimeUnit.MILLISECONDS) @ Measurement (iterations = 10, time = 5 TimeUnit = TimeUnit.SECONDS) @ State (Scope.Thread) public class JmhDemoTwo {public static void main (String [] args) throws Exception {Options options = new OptionsBuilder () .include (JmhDemoTwo.class.getName ()) .build () New Runner (options). Run ();} / * number of strings * / @ Param ({"10", "100", "1000"}) private int number; / * * string direct addition * / @ Benchmark public void StringAddMode () {String str = "; for (int iTuno

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.