Shulou(Shulou.com)06/03 Report--

Infoq has published an article (http://www.infoq.com/cn/articles/whole-software-testing-practice-requirements-to-operational), which is published here:

A previous article, "the Road to the Transformation of Software testing"

(http://www.infoq.com/cn/articles/transformation-way-software-testing/) has introduced some of the practices of the transformation, and the following describes the continuous improvement of software testing practices from March 2011 to the present.

1 diagram of software testing in the whole process

Traditional software testing can be simply described as shown in the following figure:

Figure-1-traditional delivery Test

After the developer completes the task, it is finally delivered to the tester. in this mode, the tester can not find the defects in the requirement phase early, at the same time, the testing work lags behind, and the product quality is not effectively controlled and analyzed. The overall progress may be delayed due to rework problems.

So what is full software testing, as shown in the following figure:

Figure-2-full software test chart

In the whole SDLC, there are three main lines of roles and four stages.

There are three main roles: development, QA and testing. This paper mainly explains testing.

There are four stages: demand, development, release and daily operation.

In a nutshell, it can be summarized as shown in the following figure:

Figure-3-Overview of full software testing

Testers carry out testing activities throughout these four stages, and the test practice is briefly described as shown in the following figure:

Figure-4-Test practice

Each phase also has activities for developers, as well as activities for QA personnel.

For the product, each version iteration will go through: requirements, development, release, and finally to the day-to-day operation, the release phase dotted line points to the requirements phase and the day-to-day operation phase, is not a termination phase, but a continuous iterative process.

So how do testers carry out full-process software testing activities?

2 demand phase testing

The main things developers, testers, and QA people do during the requirements phase are shown in the following table:

Stage

Developer

Tester

QA personnel

Demand stage

? User story analysis

? User story estimation time

? Participate in user story analysis and explore story ambiguity

? Query the time estimation of development with reference to the experience base

? Ensure that requirements activities are in line with the requirements management process

? Manage user story reviews

? Manage requirements change

The main practices as testers are as follows:

? Participate in user story analysis and explore story ambiguity

At the sprint meeting, analyze the user story to check that functional and non-functional requirements are clearly described, where non-functional requirements can be used as acceptance points, such as a user story:

"customers want to improve response time"

Testers should help developers eliminate the ambiguity of the story: improve response time and response time for what? It is recommended that it be changed to:

"the response time of the results returned by a general query for customer information is within 5s."

It shows that in the "customer information" module, the "ordinary query" operation is carried out, and the return time of the result is within 5 seconds. This statement has been clearly expressed and has achieved the effect of eliminating ambiguity. Similarly, testers can write user stories that improve query efficiency:

"the customer makes a general query in the information query module and can return the results within 5s."

"Note: 5s is a non-functional requirement and a key point of acceptance."

? Query the time estimation of development with reference to the experience base

At sprint meetings, developers estimate time based on experience (rules defined by the team, using playing cards), and testers should question the final result when it is given. Testers learn from the library of historical experience: what is the developer's skills in a certain area, what degree of defects the module has produced, how much time it takes to fix defects, and so on. Let the developer estimate the final time and take these factors into account as much as possible. Of course, one of the premises that testers can question is that testers have relevant development experience.

Summary: in the requirements phase, testers should play a role in reducing the introduction of ambiguous requirements into the development phase and assisting developers in time estimation.

3 testing in the development phase

The main things developers, testers, and QA people do during the development phase are shown in the following table:

Stage

Developer

Tester

QA personnel

Development stage

? Architecture review

? Confirmation of functional points

? Coding development

? Unit testing

? Development self-test

? Code review

? Bug Fix

? Confirmation of functional points

? Test case design

? Use case review

? Test and exploration

? Function test

? Bug Tracking

? regression testing

? System testing

? Acceptance test

? Management review activities

? Manage document products

The main practices as testers are as follows:

? Confirmation of functional points

Xmind is a very useful brain mapping tool. Usually before the developers code, the testers will confirm the user stories processed by the requirements with the developers, correct the understanding deviation and ensure the consistent understanding of the requirements.

Figure-5-brain map use case template

? Test case design

Testers mainly design test story points, using DSL (Domain Specific language) and infoq articles (DSL: http://www.infoq.com/cn/articles/leveraging-dsl-in-agile-test with the help of Agile testing) to describe test cases, including three basic elements:

Feature, Scenario, Example, supplementary elements: xmind, Requirement.

Feature: classify the tests into a module, and describe the business purpose of the feature itself, bring it into the business goal, and transfer business knowledge.

Scenario: to mark the test scenario of this Feature, you can use text to describe the steps, or use a xmind brain map

Description, the data in the scene uses the data listed in Examples.

Example: draw out a specific data table to show all the data used to avoid redundancy caused by repeating the same steps several times due to changes in the test data.

Xmind: brain map file showing test story points

Requirement: the requirements id associated with the requirements management system.

? Use case review

Mainly adhere to the principle of peer review, mainly carried out in the test team, responsible for the task of developers will also participate, simply the test case leak filling work.

? Test and exploration

After the "functional point confirmation" and "use case review", in order to ensure the coverage of the test scenario, it is necessary to carry out test exploration. After the developer has completed the prototype, the exploratory testing strategy is used to conduct a purposeful and rapid inspection of the basic functional process, mining functional uncertainties and supplementary test scenarios, so as to avoid uncertainties delaying until the later stage of the development stage. causing rework.

Among them: functional testing, Bug Tracking, regression testing, system testing, acceptance testing are the necessary links of daily testing work.

? Burnout diagram release

In addition, testers also have an important task to release burnout diagrams every day to keep the team informed of the current progress and summarize problems.

Look for solutions that take longer than expected.

Figure-6-burnout diagram

Graphic features:

1) the remaining working hours are above the planned benchmark, which means that the progress has been delayed, so we should keep pace with the progress.

When such problems are found, we need to analyze and summarize them. The principle is to ensure the delivery time, adjust the corresponding tasks, embrace the changes, and find that the granularity of the tasks is too large. For refactoring, you need to be careful not to go too deep into refactoring, which will bring additional workload to the test and affect the whole progress. For the entire version, only development and testing can be completed within the promised time. Delivery is not a success.

2) the remaining working hours are close to the planned benchmark, which means that the progress is good and will be maintained.

At this point, you also need to check whether the high-priority task is guaranteed by time, rather than making the burn-out image look good because you finish dealing with a simple task. Often some developers like to pick tasks and complete the easy-to-do, low-priority tasks first, because they can always be done as expected, so the trend of the early burnout diagram seems to be fine.

? Defect experience database

There are new developers / testers and old developers / testers in every team, and when testers confirm requirements with new developers, they also need to be reminded of defects and lessons to avoid detours.

Figure-7-defect summary

? Improve the quality of development self-testing

Testers can provide relevant checklist (reference: http://www.ibm.com/developerworks/cn/web/1303_sujg_webchecklist1/index.html, which can be modified to fit the team according to the original author) to help developers focus on the key points of development self-testing in the coding process, so as to improve the quality.

Figure-8-web Software Test checklist

? Continuous integration

The use of continuous integration (Jenkins) platform to achieve rapid construction of development code, automatic unit testing, to improve the efficiency and quality of development code.

The developer in charge of the unit test will receive an email from a failed build

The developer responsible for the integration test will receive an email from a failed build

The person in charge of automated testing (Selenium) will receive emails from failed builds

In this way, it ensures that unit testing, integration testing and automated testing are concerned and maintained by relevant personnel.

Figure 9-continuous Integration

? Sonar feedback

Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality .

Figure-results of 10-sonar analysis

The main feedback questions of the testers are as follows:

Code coverage: the team requires code coverage above 80%

Test success: the team requires a test success rate of 100%

Duplications: the team requires a code repetition rate of less than 10%

Violations: the team requires that the code rule defects of the Major category be less than 20

The development team must ensure the quality goal of each environment in order to ensure the overall quality goal.

Summary:

There is never a hostile relationship between testers and developers, but an assistance relationship, exactly on both sides of the quality calendar. If the work on either side is not done well, it will be out of balance.

4 release phase testing

The main things developers, testers, and QA people do during the release phase are shown in the following table:

Stage

Developer

Tester

QA personnel

Release phase

? Online application

? Online deployment

? Service monitoring

? Test report

? Online function check

? Management review activities

? Manage document products

The main practices as testers are as follows:

? Test report

Complete acceptance tests, provide test reports, and give test data metrics, such as:

1) the total number of defects found in the test: the number of defects with a removal state of "invalid" and "no need to change" generated during the test.

2) the number of serious defects found by the test: the total number of defects generated during the test and removed with a status of "invalid", "no need to change" and severity of "Major" and "Critical".

3) the number of defects fixed by the test: the number of defects with a status of "closed" generated during the test

4) number of unresolved defects: the total number of defects whose removal status is "invalid", "no need to change" and "closed".

5) defect repair rate: (number of defects found by test) / (total number of defects found by test) × 100%

6) serious defect rate: (number of serious defects found by test) / (total number of defects found by test) × 100%

7) repair rate of serious defects: (number of serious defects repaired) / (number of serious defects found by test) × 100%

8) Test requirement coverage: number of tested requirements / total requirements × 100%

? Defect statistical analysis report

In addition, testers have an important task to do statistical analysis of defects in the current version:

By defect level:

Critical

Major

Medium

Minor

Total

Home page

0

0

one

0

one

Module one

0

0

0

two

two

Module two

0

one

two

ten

thirteen

Module three

0

0

one

four

five

Module four

0

0

one

two

three

Module five

0

0

three

two

five

Module 6

0

one

0

one

two

Module 7

0

two

0

six

eight

Sonar

0

one

two

0

three

Total

0

five

ten

twenty-seven

Figure-11-defect Statistics

Statistics by defect source:

Development 1

Development 2

Development 3

Development 4

Development 5

Bequeath

Critical

0

0

0

0

0

0

Major

one

two

0

0

0

two

Medium

one

seven

0

one

0

one

Minor

one

seven

four

six

three

six

Total

three

sixteen

four

seven

three

nine

Statistics by defect status:

Total number of defects

Number of defects closed

Bequeath

Defect repair rate

Number of serious defects

Serious defect rate

Number of critical defects closed

Repair rate of serious defects

forty-two

forty

two

95%

five

12%

five

100%

Test progress and problem analysis:

1. According to the severity level distribution of BUG, BUG above Major level accounts for 12%, and the proportion is not high, indicating that most of the major functions have been implemented.

2. The defects at the sonar definition level are mainly focused on code specifications and unit test coverage, indicating that the code quality needs to be improved.

3. There is plenty of time in the early stage of version testing, and in the later stage, with the increase of the number of function points submitted by the development, the number of BUG increases, and the remaining testing time becomes tight.

4. During the version testing, it is found that the test environment is covered once and the test execution is affected twice by the developer's operation error.

Summary:

Testers should continuously feedback, improve, and summarize the problems in each version (whether they are defects or in the process), and analyze the defects to sum up some rules to help developers establish good habits and improve the quality of the code.

5 daily operation phase testing

The main things developers, testers, and QA personnel do during the day-to-day operation phase are shown in the following table:

Stage

Developer

Tester

QA personnel

Daily operation

? Production fault registration

? Feedback on version issues and improvement proposals

? Production fault analysis

? Manage daily operational activities

The day-to-day operation stage is not the termination stage, even if the activities are suspended in the demand, development and release phase, as long as the product provides services, the day-to-day operation exists.

The main practices as testers are as follows:

? Feedback on version issues and improvement proposals

Summarize the feedback on the problems occurred in the daily operation, make suggestions for improvement, and follow up the implementation.

? Production fault analysis

Assist developers to troubleshoot production failures and avoid omissions in test scenarios.

6 Summary

Software testing is not the last line of defense to ensure product quality, nor are testers. The work of testers can be done by more experienced developers, but the reality is always cruel. the current test-to-development ratio is: 1:3, this is the case in mature teams, and other teams that are still improving may go to 1:7 due to lack of resources. Developers can not completely replace testers for quite a long time, there is a key element: different ways of thinking, there is an old saying: rivers and mountains are easy to change. When developers change their way of thinking, they will become testers, so it is better to separate testers and cultivate certain testing literacy for developers, which is helpful to ensure product quality.

The whole software testing practice emphasizes the testing activities that run through each stage, whether it is development or testing, to understand the value of the activities of both parties, when to do what, and to what extent it is good to ensure the quality of each link in order to ensure the whole quality of the product. in addition, the product quality is not tested, but precipitated during the construction process. The quality of the product is determined by the literacy of the developers, the literacy of the testers, and the degree to which the team attaches importance to the development and testing process. Product quality is like a piece of cake, which should be divided into small pieces and implemented in everyone's hands, so that everyone can taste the benefits and bear the burden.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.