Shulou(Shulou.com)06/03 Report--

Background

Due to the recent exposure of bug which was only found in the site testing phase due to missed testing, especially some problems related to identity verification, authentication, payment, transaction accounting and so on, the consequences are very serious. Therefore, the missing test of bug is considered and summarized.

Cause analysis

In fact, BUG is an unavoidable problem for any product. Not all bug can be found, including experienced testing, and there will be online defects more or less. No one can think about all the functional operation and application scenarios of the software. Although can not achieve completely zero defects, but each release of the product, we need to pursue fewer and fewer defects, fewer and fewer product complaints.

Why there is a defect missed detection, there are mainly the following points:

In the requirements review stage, there is no clear understanding of the details of business requirements, and there is no in-depth mining to expand the requirements.

Analysis of problems

In the actual product research and development process, the product requirements are actually in a refinement process, in the requirements prd document interactive document output for review, failed to reveal some product details, implicit requirements, and the test case writing is based on prd interactive documents.

Improvement measures

Before the requirements review, we should carefully read the prd and interactive documents, form our own thinking about the product, list the doubt points of the product design by the way of brain map, and find out the product design defects from the users or from the industry point of view.

Requirements review meeting, with the listed question points to the product, developers to communicate their own doubts and questions about the product, ask a few more why? How to achieve it? The source of data acquisition? How to deal with the data that exceed expectations? What is the cache processing mechanism? Where is the data saved? Is the logic handled by the front end or by the back-end service? Is the back-end service logic associated with a third party?

After the requirements review is completed, according to certain functions, the requirements are divided into several large modules, and the large modules are divided into small function points, and then the specific implementation process of the function points is considered.

The test case coverage is not comprehensive, and the scenario is omitted.

Analysis of problems

In the process of test case design, it is easy to have limited thinking or requirement blind spots, so it is impossible for us to completely cover all the scenarios used by users, and it is impossible for colleagues who write test cases to think through all the scenarios. It is also unrealistic to write all scenarios into test cases.

Improvement measures

List the mind map before the use case design begins

Through the mind map to list the business process, front and back interface logic. Then according to the prd and interactive documents, according to the ui interface, it is divided into large functional blocks, and then in the large functional blocks, and then cut into small functional blocks, and finally to the function point, each function point carries out the use case design map through the dimensions of ui, basic function, boundary, memory, interaction, network, exception, interface logic and so on, and lists the doubtful points that need to find the product and development confirmation.

Organize use case review after the completion of use case design

The main contents are as follows: (1) organize the development and product to review the test case, and throw out the questions in the use case design, and review and improve the use case through the perspective of product implementation, data storage and product experience.

(2) if there is plenty of time, it is also very necessary to organize the use case review within the test team, especially some experienced or business-familiar drivers who can quickly help point out the missing points of the use case in the use case review. it is helpful for testers to open up their ideas and cover as many user scenarios as possible. It is worth noting that if there is uncertainty in the use case review, it should be recorded immediately. After the end, check with the relevant personnel in time to avoid guessing.

Perfect use cases according to online user feedback

After the product test is released and launched, for the defects reported by users, if the defects are caused by incomplete scenario design, we first analyze whether the scenario in which the problem occurs is inevitable or occasional. If it does, we can communicate with the technical interface to confirm some specific reproduction steps of the scenario, confirm the reasons for the introduction, and solve the problem. Then improve the test cases: in addition to supplement the scenario case, consider some scenarios associated with the scenario, improve and review the test cases in a variety of scenarios, and add them to the use case base.

The test phase is not executed strictly according to the test case

Analysis of problems

Executing the test according to the test case allows us to avoid missing some test points as much as possible. But some of our students do not strictly follow the test cases to execute the test, so there are some omissions in BUG should not be.

Improvement measures

Test cases may not guarantee that all scenarios and function points can be covered, but strictly according to the test cases can ensure product quality to the maximum extent and avoid defects as far as possible.

In addition, form the habit of testing records: for testing blocking cases and testing fail cases, we should focus on and record them, and conduct accurate regression testing in the regression testing phase to ensure that the new bug introduced by the associated functions caused by repairing bug can also be found.

The test environment and test resources are limited, resulting in missing detection of defects.

Analysis of problems

The environment of Internet financial products is extremely complex, the business system does not exist in isolation, the related parties are interlinked, and the related systems are often unstable, in addition, the use of scarce resources such as × × × and bank cards is limited, and often a valid data is discarded after testing, so we can restore the customer's actual environment through mock and restore the customer's actual environment (such as network verification mock). Bank card authentication mock), but after all, it is not a real environment. Due to differences in the environment, many unexpected problems may occur. These problems may only be exposed under the characteristic environment and specific operation steps. They cannot be restored in our test environment, and can only be verified based on the production environment.

Improvement measures

Introduce grayscale release (pre-release) testing

The test group carries out regression testing in the pre-release environment, which can basically simulate the use cases that can not be tested in the real environment, and does not affect the normal use of online users.

Do well the case screening in the process of production verification

First of all, the production verification case is combed, and the production verification case should not only screen the p0+p1 level case for regression, but also include the test case of the test environment mock or bezel blocking, and the case of the back-end interface to the front end response. In the production regression phase, strictly follow the production verification case to cover the real online environment scenario.

Enhance the understanding of back-end and related party business logic

The front-end needs to understand not only the interactive business logic between the front-end interface and the back-end interface, but also the connection between the back-end interface and other related parties, so as to have overall control over the coverage of test cases in the test environment. To ensure comprehensive test case coverage in the production environment.

New BUG introduced by developers

Analysis of problems

There are some developers who will only solve the problem described in the BUG you submitted, instead of troubleshooting all the points that may be involved in the problem, it is possible to solve the problem and introduce a new problem.

In addition, a developer who is not familiar with functional modules fixes BUG because of business unfamiliarity and carelessness that leads to the unconscious introduction of new BUG.

Improvement measures

Code review

From the code management level: development repair a bug submission code self-test to prepare for testing, the development team submitted the code for code review, the introduction of a new BUG is less likely.

Accurate regression test

From the level of test self-cultivation: after the development and testing, through the way of diff code, understand the code change points, accurately analyze the impact of the change points on the associated function points, confirm and verify the BUG repaired by the developers, and traverse the related function points to regression test as much as possible.

Talk to developers about how developers fix this feature.

Talking to developers about implementation it is easy to grasp the points for attention of the test from the design of the development and record it in the use case. For example, A developers used to do a function in some way, there is a certain bug, and now the b function is implemented in the same way, then it is very likely that the previous bug will still appear in the b function.

Lack of ET testing (exploratory testing)

Analysis of problems

Many of the BUG we found are not found by test case execution, but are found by random testing during the testing process, but these steps are not reflected in the test cases, and our test cases cannot cover all scenarios.

Improvement measures

Conduct ET test after passing the admission test

After the completion of the test admission test, enter the SIT test phase: generally speaking, the ET test is the easiest to find bug, so when the test admission test is completed and enters the SIT test phase, first conduct a round of exploratory testing, so that most of the bug is exposed in the early stage of the test, so that the cumulative number of bug reaches a certain peak, and the bug is found as soon as possible, the higher the quality.

Perform an intra-group ET test before UAT test

The SIT test is coming to an end. Organize an intra-group ET test before the UAT test, and let the different tests in the group use different test methods, test thinking, test experience, and test habits to explore the test. You can find some missed bug, weird bug or unreasonable use due to the limitation of mindset.

Summary

Defect missed detection can not be put an end to, after the occurrence of defect missed detection, we need to learn to think, learn from experience and lessons, as far as possible to reduce the defect missing measurement.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.