Shulou(Shulou.com)06/03 Report--

Practical Analysis of Project cases of Precision testing

I. Preface

Testing is the key link to ensure product quality, whether from the developer's unit test, integration test, tester's system test, product requirement test, customer acceptance test, all is to ensure that the product can serve users more robust in the market, but the whole work and process of testing does not have the same output of a product as the work of development. Therefore, to a greater extent, it increases the assessment of the quality of the testing work, which makes it impossible to have a reliable basis to judge whether the product can run stably in the market after the completion of the product testing. there must be a variety of problems and difficulties in the testing process.

In traditional testing, the following problems often occur in the later stages of testing:

1. Insufficient test range and missed test

Often, the lack of knowledge of the development change test, or the insufficient evaluation of the test scope and the testers' lack of understanding of the product will lead to a high risk of test vulnerabilities and become the fuse of online accidents. and expect to be able to improve coverage through code coverage tools.

two。 The schedule and time are in a hurry, and I'm not sure when I go online.

Test: "time is too tight. I feel like I will be online without a test! it will be all right in a few days." Time is tight, it is impossible to plan their own test ideas and scope, feel that they have not tested completely, there is no foundation in the heart. It would be nice if there were tools to help do test screening and statistics, and determine whether the product can meet the online standards by code coverage.

3. The test regression range is wide and the cost is high.

Sometimes the scope of regression given by the developer is so large that the cost of testing regression testing is very high. Time and personnel need a lot of resource investment, or hope to be able to achieve accurate testing through code coverage tools, so as to reduce the investment of unnecessary resources and improve the efficiency of work.

4. Communication between testing and development

Testing and development often produce a variety of unpredictable situations due to human communication in the later stage of communication.

Such as:

Friendly:

After the developer modifies the code, the tester directly asks the developer whether it has been tested, and if the developer has tested it himself, the test is considered to be complete.

Contradiction:

After major problems arise, development and testing pass the buck to each other, resulting in the ossification of the relationship between team development and testing.

The following picture shows the scope of the online accident of a star Internet product company in 2015:

We can see that more than half of the whole accident is due to the failure caused by the communication between the developer and the test or the test's lack of understanding of the business. In order to avoid such accidents, we first need to solve the problem with a high proportion, which can improve the quality of the product to a great extent.

So how do we solve these problems? from the following 09:00 project case, we explain how to improve test coverage, test development communication, test omissions, test scope evaluation errors, etc., how to effectively use existing resources to solve.

II. 09:00 project introduction

09:00 Hotel Control Project is a hourly room booking APP for vertical subdivision of hotels, which allows users to book cooperative hotels in Shanghai through the application of mobile phone, which is convenient and quick, and selects nearby booking hotels through automatic positioning, and selects the right hotels according to price and distance, which is affordable, book in advance, and solve the problem of going out accommodation. In the future, it will cooperate with more travel channels and have a lot of room for development.

III. Purpose

This paper describes the Android APP of the 09:00 project, get the test report and related test data analysis through the test on the precision test platform (nebula test), control the quality of the whole application through manual black box test and logic test within the program, and reduce the problems after the product is put on the line.

Fourth, the design of test cases

Conventional boundary value analysis, orthogonal analysis, causality diagram, and equivalent class division are used to design test cases, such as:

1. For the limited character length of the input box, the boundary value of the limited number of characters is used to design the test case.

2. Design test cases by causality diagram method for search items.

3. Orthogonal analysis is used to design the test case for hotel list screening.

4. Carry out exploratory testing according to users' experience habits, etc.

V. Test environment and configuration

1. Local environment (cloud test version, the master needs to configure the local client environment, and the service does not need to be configured in the cloud)

2. The PC test network needs to be able to connect with the address of the nebula test server.

3. The mobile phone can use WIFI or USB to test the connection.

VI. Test methods and tools

Test method:

The black box manual testing method is adopted to carry out the test.

Test tools:

The nebula test Android cloud beta client version 2.1.4 is used for testing. Through manual black box test driver, the tool detects and records the execution logic within the program, quickly locates BUG, and automatically captures program crash points, test regression filtering, and test data integration to display cloud test accuracy reports. For test personnel, equipment and test task execution time, statistics are carried out on a daily basis to show the trend of the whole process in the form of charts. And the code quality repetition, complexity and maintainability of the whole project are analyzed and displayed.

Note:

The following analysis is expressed in the code. For example, if the tester does not have the permission to view the code, he or she can only use the test data to interact with the developer, and the developer can associate the code locally with the test data of the nebula platform.

Static analysis of the code

Many projects will be more difficult to maintain at a later stage, mainly because the increase in the amount of code and the changes of developers or code writing specifications make it difficult for their successors to understand the internal logic.

Through static monitoring and analysis, we can see that the 09:00 project lacks a large number of comments in the code, and references a large number of repetitive code and illegal code, which will cause repetition of defects and other problems, illegal code will make it difficult to read and maintain the code, and there will be hidden code logic BUG and resource closure and other errors in the code. Therefore, it is necessary to standardize the code according to a specific set of rules, although there are no problems in project generation and use, but there are a lot of risks considering the alternation of developers and the lack of maintenance in the later stage.

Through static metrics, you can view violations and duplicate code locations, and use this data to report and interact with developers and require code improvements.

VIII. Test case execution and analysis

1. Test case execution analysis

Total test cases: 136

Execute test case: 129

Number of defects: 60

Test case pass rate: 67%

According to the execution of the test case to understand the tester's workload and work efficiency, as well as the final result, pass rate and completion of the whole test, the manager can adjust the progress and work at any time according to the actual situation. better manage the team and supervise the testing progress of the entire product.

two。 Defect classification:

Judge the overall quality of the version according to the BUG classification and severity data lesson. If the proportion of serious BUG is too high, it may be retested according to the needs of the project, and then handed over to the test for testing to ensure that there will not be too many defects in the process, resulting in testing stagnation and affecting the actual time for release of the version.

3. Test cases and defects:

Through the monitoring trend chart of the nebula testing process, we can clearly see the BUG discovery of the project at 09:00 in the testing these days, and judge the module with a large number of BUG by the test case execution date, BUG submission date and description. For example, at 9 o'clock, a lot of test cases were done and a lot of BUG were submitted at 2015-12-7.

Through the intuitive curve drawing, we can see the discovery rate of BUG and the execution of test cases during the whole testing work so far, and the number of testers can be managed and adjusted according to the actual requirements.

Let's look at the description of the details

The detailed list of BUG shows all the BUG found, as well as the submitter and test environment corresponding to each BUG, as well as the corresponding test cases, which are helpful for us to determine the main factors that appear in BUG and the methods to fix them. It is easy for developers to fix and will also take into account compatibility in similar environments.

For example, we can see that the focus of our testing in 2015-15-7 is mainly on coupons, orders, payment and other modules. There are a lot of friendliness or operational problems in this module, and testers can interact with the development with this data. Total modification of these modules is required.

IX. Test coverage and vulnerability analysis

Design and run tests according to requirements documents and functional specifications, and check the daily coverage growth trend through nebula tests

In the past few days of testing, although the testers traversed the complete process, but the coverage rate has not been high, segment coverage is only 41.1 for this situation, we use vulnerability analysis to find out the reason.

First of all, we find out the complexity, high density and coverage 0 through the nebula report. These are all test loopholes and high-risk omissions. If they are not solved one by one, the impact of the problems caused by the late launch may be relatively serious. In order to avoid this phenomenon, we must conquer these toxins. At this time, if the tester does not understand the code, you can invite the developer to assist in checking it and view the code of the function through the visual interface.

Such as 1: function ID 1465 handleMessage through code visualization and development communication that this module is to deal with the list of drop-up events, but in the latest 09:00 project is no longer in use, which has caused the testers can not traverse the module of the reason, for these waste code, testers have the obligation to require the development of its comments, or delete processing, so that the later maintenance of the code has a guarantee.

For example, 2: function ID 1880 isrefreshview scroll learned through code visualization and development communication that this module is a discount volume pull load function, but this function will not appear until the discount volume exceeds a certain amount, but in the actual test, the tester only gets the account number of a discount volume, which naturally cannot be covered in the traversal.

For example, 3: function ID 1530 zoom learns through code visualization and development communication that this module is the main interface map function, and the reason for its low coverage is:

This function is mainly for the scale of the map for different scale value selection, map adjustment, but at 09:00 in the map is to call Baidu map, if you want to cover all, the need for the background to change its code to simulate. This situation is mainly aimed at the testing of core functions, testers need to pre-judge whether the module needs a variety of background state processing testing, and improve the compatibility coverage after interacting with the development.

Such as 4: function ID 1449 onClick through code visualization and development communication, this module is a hotel review function, according to visual analysis

Found that because there is no test in the hotel review, so it is not covered, the number of execution is 0, need to design use cases, hotel comments in the evaluation of the situation, improve test coverage, ensure that there are no serious test loopholes

Code-level testing BUG fast tracking

Eg: BUG ID: 1

BUG description: without network, lists and menus cannot be clicked, and need to be optimized and friendly.

Get the reason through BUG code-level traceability: the two-way traceability page traces to the code according to the BUG use case, showing that after clicking the menu without network, there is no prompt, and it can not be called and executed normally, so it needs to be optimized, and developers are advised to add relevant hints.

Testing is not only functional testing, but also user interaction testing and experience testing as users, which directly affect users' direct use. if users want to be sticky to the product, they must have a better user experience. so some suggestions are necessary, which is also needed within the testing responsibility.

11. Analysis of test team personnel

In previous tests, the evaluation of a functional test team and tester mainly depends on his ability to find BUG, but in practice, because of the quality of the test project, the tester's understanding of the business and the working years of the tester, it is not only evaluated by BUG.

The nebula report directly reflects the test status of testers by relating the test cases run by testers, the coverage of test cases, and the BUG of test cases, avoiding the communication problems between testing and development in article 4 of the preface, and providing targeted guidance to testers.

For example, the test case designed by a tester is very comprehensive, and after running the traversal, the report coverage of the nebula test is also very high, but we can never find BUG. At this time, we can judge that there are two situations: first, there is really no big problem with the project tested by the tester, and second, there may be deviations in the tester's understanding of the business, although most of the functions are run. But BUG also includes friendliness, logical output, and so on, which are all at the business understanding level, so you can provide business guidance to the tester in this case.

XII. Analysis of testing equipment

In many test scenarios, testers do not find any problems during testing, but customers do not make ordinary errors in the process of use, many of which are caused by compatibility. For example, for the 09:00 project, in the latest version of the developer released in 15-11-23, the testers showed that the click function of the main interface was invalid and flickered, but there was no problem with the developer. The developer used a mobile phone above android5.0, while the tester used an android4.2-4.4 mobile phone.

In view of the above problems, the general company or Party A will require the test team to be equipped with mainstream models and commonly used resolution phones to avoid the accident.

The nebula test report records the test equipment used by the tester during the testing process, and is associated with test cases, BUG, etc., which can effectively manage the use of the whole test equipment and the corresponding situation.

XIII. Traceability of test cases, code and modules

One of the major reasons why it is difficult to maintain the project when the developer changes. In the 09:00 project, we test the test case by running the platform and associate the test case with the function it runs. This makes the subsequent developers or testers understand the function through the correlation between the test case and the code. It greatly reduces the time spent by developers by developing documents, handing over documents, and reading code written by others themselves.

It is mentioned in the preface: incomplete consideration, development modification, test scope evaluation error, in traditional testing, after the developer changes a function, because the developer does not know how many other calling functions the function will affect, resulting in omissions when changing the function with the test, so that the test scope evaluation error, through the nebula test case, code, module traceability association Developers can clearly see the test cases corresponding to a piece of code, so that more consistency changes are taken into account in the process of modification.

14. Automatic selection of regression test cases

In the process of regression, due to the development of a large range of regression or to avoid missing the range of regression, the test is often required to conduct a full regression in the process of regression, but the test is incomplete due to time constraints and other factors, and there is no psychological bottom after the test is online.

In the 9 o'clock project, through the automatic selection of regression test cases, the Nebula platform extracts the test cases of the version that needs to be regressed and all versions of test cases before that version for query, obtains the last running version of each test case for data extraction, and compares it with the version to be regressed through the retroactive correlation technology of test cases, code, and modules. Analyze the regression test cases that are most affected by the development changes.

In the case of insufficient test time, this function can be used to evaluate the test cases together with developers to delineate the scope of test case regression, so as to reduce the cost of test regression.

The above analysis and explanation are the real data after the nebula test platform tested 09:00 app, and all the cases are completed by communication and cooperation between our testers and developers, so that developers and developers can communicate with each other, which can improve the work efficiency of the whole team, and exercise the ability of testing to find problems and the analysis ability to judge the source of the problems. Have a deeper contact and understanding of the logic of the internal procedures of the product, achieve accurate testing, reduce unnecessary workload, ensure product quality, no high-risk testing loopholes, more stable on-line.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.