Shulou(Shulou.com)06/02 Report--

This article mainly explains "TestNG test case rerun analysis". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "TestNG test case rerun analysis".

Test case running stability is an important indicator of automation quality. Test case execution failures caused by non-bug need to be eliminated as much as possible. Rerunning for failed cases is one of the common strategies. One kind of rerun strategy is to rerun the failed use case after all the use cases are finished, and the other is to monitor the running status of the use case at run time and rerun in real time after failure.

Next, the practice and solution of how TestNG rerun the failed test case in real time and solve the problems encountered in the process of rerun is described in detail. There are several requirements for real-time rerunning of failed test cases:

The test case failed to run, and rerun immediately after listening to the failure.

The test case relies on other test cases through the dependsOnMethods/dependsOnGroups tag, and after the dependent test case reruns successfully, the test case can continue to run

For test cases that have been rerunned many times, only the success or failure of the last run is recorded.

Part one: test case reruns 1.1 retryAnalyzer annotations

This approach can be used for rerunning a small number of test cases that are prone to failure and unstable.

1.1.1 principle

The following is part of the code that TestNG handles the results of the test case run.

IRetryAnalyzer retryAnalyzer = testMethod.getRetryAnalyzer (); boolean willRetry = retryAnalyzer! = null & & status = = ITestResult.FAILURE & & failure.instances! = null & & retryAnalyzer.retry (testResult); if (willRetry) {resultsToRetry.add (testResult); failure.count++; failure.instances.add (testResult.getInstance ()); testResult.setStatus (ITestResult.SKIP);} else {testResult.setStatus (status); if (status = = ITestResult.FAILURE & &! handled) {handleException (ite, testMethod, testResult, failure.count++);}

Analyze the above code, where the return value of the interface IretryAnalyzer's method retry () is used as a condition for rerunning the failed test case. If the retry () result is true, the failed test case will be rerun, and the failed result will be modified to Skip;. If the result is false, the failed test case will keep the failed result and the run ends. Therefore, if you want to rerun the failed test case, you need to rewrite the retry () method of IretryAnalyzer, insert your own defined logic, and set the return value to true.

1.1.2 Code

Create the class RetryImpl, override the retry () method, and set the number of reruns for failed test cases. The code is as follows:

Public class RetryImpl implements IRetryAnalyzer {private int count = 1; private int max_count = 3; / / Failed test cases could be run 3 times at most @ Override public boolean retry (ITestResult result) {System.out.println ("Test case:" + result.getName () + ", retry time:" + count+ ""); if (count < max_count) {count++; return true;} return false } 1.1.3 instance public class TestNGReRunDemo {@ Test (retryAnalyzer=RetryImpl.class) public void test01 () {Assert.assertEquals ("success", "fail"); System.out.println ("test01");}}

The above test case test01 can be run repeatedly 3 times.

1.2 implement the interface IAnnotationTransformer method

If you want all failed test cases to rerun, it is troublesome to annotate each test case with retryAnalyzer annotations. By implementing the IAnnotationTransformer interface, you can set up the retry class for all test cases. This interface is a listener interface for modifying TestNG annotations. There is only one method for IAnnotationTransformer listener interface: transform (ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod). In the previous section, we customized the class RetryImpl implementation interface IRetryAnalyzer. TestNG modifies the retryAnalyzer annotation through the transfrom () method. The following code modifies the settings for the retryAnalyzer comment.

1.2.1 Code

Create the class RetryListener as follows.

Public class RetryListener implements IAnnotationTransformer {public void transform (ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {IRetryAnalyzer retry = annotation.getRetryAnalyzer (); if (retry = = null) {annotation.setRetryAnalyzer (RetryImpl.class);}} 1.2.2 configure Listener

TestNG can configure the Listener class in a configuration file or in a test class.

Method 1: make the following configuration in the configuration XML of TestNG

Method 2: configure through @ Listeners in the test class

Listeners ({RetryListener.class}) public class TestNGReRunDemo {@ Test public void test01 () {Assert.assertEquals ("success", "fail"); System.out.println ("test01");}}

After the configuration is complete, run the test case test01, and the results show that test01 will be re-run 3 times.

The second part is the rerun result processing of dependent test cases.

A further analysis of the running code of TestNG shows the logic shown in the following figure when rerunning the failed use case. For test cases that depend on other test cases through dependsOnMethods or dependsOnGroups annotations, test case execution is divided into two cases:

AlwaysRun=true, the test case will be executed regardless of the execution of the dependent test case, that is, the rerun of the dependent test case will not affect the execution of the test case.

AlwaysRun=false, or keep the default value (false), depends on the test results of other test cases or test case groups. At run time, TestNG gets the run results of the dependent test cases, checks whether all the dependent test cases are executed successfully, and sets the test case result to Skipped if not all of them are successful.

2.1 scenario Analysis: scenario 1

After the dependent test case failed, the rerun was carried out and the rerun succeeded. (note: in the RetryImpl class, the maximum number of reruns has been set max_count = 3)

Public static int number = 0 leading testpublic void test01 () {number++;System.out.println (String.valueOf (number)); Assert.assertEquals (number,2); System.out.println ("test01");} @ Test (dependsOnMethods = "test01") / / alwaysRun = false by defaultpublic void test02 () {System.out.println ("test02 is running only if test01 is passed.");} 1, TestNG test report

2. Test report on the number of runs of problem test cases: Test012 the first time: skipped; the second time: passed in the statistics of Skipped and Passed, test01 is recorded once Test020Skipped is recorded once Skipped

Test report: all the results of the test01 run are recorded, while the use case rerun only wants to record the final result.

Performance: the test case test02 depends on the test case test01 run results. After the test01 rerun, the test case test02 is not executed, which does not meet the requirements expectations.

2.2 scene analysis: scenario 2

A rerun was performed after the dependent test case failed, and the rerun failed. (note: in the RetryImpl class, the maximum number of reruns has been set max_count = 3)

Public static int number = 0 leading testpublic void test01 () {number++;System.out.println (String.valueOf (number)); Assert.assertEquals (number,10); System.out.println ("test01");} @ Test (dependsOnMethods = "test01") / / alwaysRun = false by defaultpublic void test02 () {System.out.println ("test02 is running only if test01 is passed.");} 1, TestNG test report

2. Problem test case run times run results test report Test013 first: skipped; second: skipped; third: failed in Skipped statistics, test01 is recorded twice in failed statistics, test01 is recorded once Test020Skipped records once Skipped

Performance: the test case test02 relies on the test case test01 run results. After the test01 rerun fails, the test case test02 is not executed, which is in line with the requirements.

Test report: in the same scenario 1, the test01 rerun fails and all the run results are recorded, while the use case rerun only wants to record the final result.

The third part optimizes the solution.

The following solution solves the problem that the subsequent test cases can not continue to run after the rerun test case is successful, and optimizes the test report.

3.1 TestListenerAdapter method rewriting

According to the TestNG logic analyzed above, when checking the results of dependent test cases, if you ignore the intermediate results of the rerun and only check the results of the last run, you can achieve the purpose of the requirement. For test reports, in the same way, ignore the run results of all intermediate test cases and record only the final results. The intermediate result of the test case is Skipped. The following code removes the intermediate result of the test case, skipped, by overriding the onTestSuccess () and onTestFailure () methods of TestListenerAdapter. The code is as follows:

Public class ResultListener extends TestListenerAdapter {@ Override public void onTestFailure (ITestResult tr) {if (tr.getMethod (). GetCurrentInvocationCount () = = 1) {super.onTestFailure (tr); return;} processSkipResult (tr); super.onTestFailure (tr) } @ Override public void onTestSuccess (ITestResult tr) {if (tr.getMethod (). GetCurrentInvocationCount () = = 1) {super.onTestSuccess (tr); return;} processSkipResult (tr); super.onTestSuccess (tr);} / / Remove all the dup Skipped results public void processSkipResult (ITestResult tr) {ITestContext iTestContext = tr.getTestContext () Iterator processResults = iTestContext.getSkippedTests (). GetAllResults (). Iterator (); while (processResults.hasNext ()) {ITestResult skippedTest = (ITestResult) processResults.next (); if (skippedTest.getMethod (). GetMethodName (). EqualsIgnoreCase (tr.getMethod (). GetMethodName ()) {processResults.remove ();} 3.2 configure the result to handle the Listener class

Make global settings in the configuration file or mark it in the test class.

Method 1: make the following configuration in the configuration XML of TestNG

Method 2: configure through @ Listeners in the test class

@ Listeners ({ResultListener.class}) public class TestNGReRunDemo {@ Test public void test01 () {Assert.assertEquals ("success", "fail"); System.out.println ("test01");} 3.3 scenario 1. Result verification

2, result analysis: test case run times run results test report Test012 first time: skipped; second time: passed is only recorded once in the statistical number of Passed test01 is recorded once Test021Passed is recorded once passed3.4 scenario 2, result verification

2. Result analysis: test case run times run results Test report Test013 first time: skipped; second time: skipped; third time: failedtest01 is recorded only once in failed statistics Test021Skipped dependent use case execution failed, test02 result is Skipped, only one result Skipped is recorded

Thank you for reading, the above is the content of "TestNG test case rerun analysis". After the study of this article, I believe you have a deeper understanding of the problem of TestNG test case rerun analysis, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.