Shulou(Shulou.com)06/03 Report--

1. Brief introduction

1.1 purpose of writing

This document is used to record the testing process, summarize the testing situation of each round, analyze the test data, summarize the problems and remaining risks exposed in the testing process, and give the corresponding test suggestions for follow-up project reference.

1.2 Project background

Xx needs a community-based product with real users, which can improve user stickiness, increase the number of active members and bring long-term growth through the establishment of real and high-trust user relationships. In this context, the community based on real users came into being. It mainly has the following five meanings:

1. Increase the number of active members in the community

two。 Improve user viscosity

3. Establish real multi-dimensional user information (consistent with the user's community identity)

4. Establish a user relationship with high trust

5. Achieve the communication effect between users in a true and trusted user relationship.

1.3 Definitions, acronyms and acronyms

None

1.4 reference materials

Summary of each round system test phase

two。 Test summary

The testing of the whole xx project has gone through two stages: xx-1.0 and xx-1.1, including 1 round of integration test, 6 rounds of smoke test, 7 rounds of system test and 1 round of online tracking test. During the whole testing process, a total of 8100 use cases were executed and 1026 defects were found. By the end of the test of the fourth system of xx-1.1, the high weight problems found have been fixed and verified.

2.1 Test time

The testing time of the whole xx project begins on February 18, 2000 and ends on March 27, 2000. The work of each phase is as follows:

Working stage

Start time

End time

amount of work

(man-day)

Xx-1.0

Xx-1.0 requirements confirmation, review, test case writing & review

February 18, 2008

25 February 2008

thirty

Xx-1.0 integration testing

19:30 on 22 February 2008

1:00 on 23 February 2008

four

Smoke Test 1 of the first round of xx-1.0 system testing

10:30 on 26 February 2008

17:00 on 26 February 2008

five

Smoke Test II of the first round of xx-1.0 system testing

13:00 on 29 February 2008

19:00 on 29 February 2008

4.5

Smoke Test III of the first round of xx-1.0 system testing

10:00 on March 3, 2008

16:00 on March 3, 2008

4.5

The first round of xx-1.0 system testing

15:00 on March 5, 2008

16:30 on March 8, 2008

thirty-six

The second round of xx-1.0 system testing

10:30 on 10 March 2008

19:00 on 11 March 2008

twenty

The third round of xx-1.0 system testing

21:00 on 11 March 2008

22:00 on 11 March 2008

one

Xx-1.1

Xx-1.1 requirements Review, Test case Writing & Review

March 12, 2008

March 17, 2008

fifteen

Smoke Test of the first round of xx-1.1 system Test

10:00 on March 18, 2008

15:30 on March 18, 2008

four

The first round of xx-1.1 system testing

10:00 on March 19, 2008

18:00 on March 21, 2008

twenty

Smoke Test of the second Round of xx-1.1 system Test

16:00 on 22 March 2008

18:30 on 22 March 2008

1.5

The second round of xx-1.1 system testing

16:00 on 22 March 2008

16:00 on March 24th, 2008

eighteen

The third round of xx-1.1 system testing

10:00 on 25 March 2008

17:00 on 25 March 2008

6.25

The fourth round of xx-1.1 system testing

21:30 on 25 March 2008

1:30 on 26 March 2008

four

Xx-1.1 online tracking test

6:30 on 27 March 2008

12:00 on 27 March 2008

4.5

Total

one hundred and seventy eight

2.2 Test range

The scope of this test includes: functional testing, compatibility testing, interface testing, data migration testing, performance testing, security testing and quality monitoring. The following describes functional testing, compatibility testing, interface testing, data migration testing, performance testing and security testing.

Function test

The main features of xx-1.1 updates based on xx-1.0 are as follows:

No.

Module

Weight

one

Pass registration, login, and the launch of personal community products

A

two

System message

A

three

Subscription

A

four

Live chat

B

five

Business card

B

six

Update Tip

B

seven

Feed transformation

B

eight

UIC transformation

B

nine

Report the wrong page

B

ten

Defects left by xx-1.0 to xx-1.1

C

eleven

Modification of each product for xx-1.1

C

The main features of xx-1.0 are as follows:

No.

Module

Weight

one

Log in and register

C

two

Navigation

C

three

Profile

A

four

Account account Management

B

five

Privac Privacy Settings

B

six

Feed

A

seven

Personal data

B

eight

Space-q

B

nine

Space-bbs

B

ten

Space-bar

B

eleven

Firend friends

A

twelve

Vistor visitors

A

thirteen

Carton

A

fourteen

Leave a message

A

fifteen

UIC (avatar, nickname)

A

sixteen

Help

C

seventeen

Modification of each product for xx-1.0

C

Data migration test

No.

Concern item

Weight

one

Switching between blog podcast photo circle forums and Space user avatars

A

two

Blog regular user visitor data is provided by Space

A

three

Integration of friend data and Space of regular users in blog podcast photo album circle forum

A

four

Blog podcast photo album circle forum the integration of carton data and Space for regular users

A

five

The Integration of personal data and Space of regular users in Circle

A

six

Integration of regular users' message data and Space in blog podcast circle

A

Interface test

No.

Concern item

Weight

one

Interface test between each product and the feed function of Space

A

Compatibility testing

No.

Concern item

Weight

one

IE6

A

two

IE7

B

three

Firefox2.0

C

Performance testing

See the performance test report in SVN.

Security testing

In the whole process of xx testing, three rounds of security tests have been carried out, and two serious security problems have been found, all of which have been repaired and verified.

2.3 Test version

The following table shows the allocation of the test version and the corresponding test scope for each round:

2.4 Test cases

According to the requirements document, the tester wrote and audited the test cases, wrote a total of 3558 use cases for the xx project, and executed a total of 8100 use cases.

2.5 Test strategy

Xx-1.0 testing strategy

1. Test type: defined by phase as integration test and system test.

two。 A round of integration testing was carried out in the integration testing phase, which mainly aimed at requirements mining, analysis, confirmation and finding inconsistencies between requirements and requirements.

3. The testing phase of the system is divided into three rounds, and the basic strategies are as follows:

The first round is coverage testing, which covers all the areas described above and focuses on all levels of bug

In the second round, the modules with weights An and B are tested for function and compatibility, the modules with weights C are tested for smoke, and all repaired bug are tested by regression.

The third round carries on the function test and compatibility test to the module with weight A, smoke test to the module with weight B and C, regression test to all the bug to be solved, and the closed high priority bug.

A quick smoke test is carried out before the start of each round of testing, and when the smoke is sure that the system is testable, proceed to the next round of system testing.

Data migration testing and interface testing are only carried out in the first round.

4. Defect assessment: after each round of testing, development engineers, test engineers and product engineers are organized to jointly evaluate product defects, including defect solutions, whether requirements changes are involved, the start time of the next round and whether the tests can be completed.

Xx-1.1 testing strategy

1. Test type: defined as a system test by phase.

two。 The system test is divided into four rounds, and the basic strategies are as follows:

The first round is coverage testing, which covers all the areas described above and focuses on all levels of bug

In the second round, the modules with weights An and B are tested for function and compatibility, the modules with weights C are tested for smoke, and all repaired bug; are tested for system performance by regression test.

The third and fourth round of the weight A module for functional test, compatibility test, smoke test for the module with weight B and C, regression test for all bug to be solved, and closed high priority bug

A quick smoke test is carried out before the start of each round of testing, and when the smoke is sure that the system is testable, proceed to the next round of system testing.

Data migration testing and interface testing are only carried out in the first round.

3. Defect assessment: after each round of testing, development engineers, test engineers, product engineers and QA are organized to jointly evaluate product defects, including defect solutions, whether requirements changes are involved, the start time of the next round and whether the tests can be completed.

3. Result analysis

A total of 1026 effective defects were found in the whole xx testing process, including 3 grade A defects, 21 grade B defects, 800,169 grade D defects and 33 grade E defects. According to the evaluation of the project team, 51 defects were left by the release of xx-1.1, and the remaining 975 defects have been repaired and all passed the verification. Below, the defects are analyzed from different angles from the two stages of xx-1.0 and xx-1.1.

3.1 defect trend

Xx-1.0

A total of 734 defects were found in the whole testing process, and the distribution of defects in each round is shown in the table below.

The following figure shows the trend of defects during xx-1.0 testing:

Xx-1.1

A total of 292 defects were found in the whole testing process, and the distribution of defects in each round is shown in the table below.

The following figure shows the trend of defects during xx-1.1 testing:

From the defect trend diagram, we can see that in the two testing stages of xx-1.0 and xx-1.1, the defects converge with the progress of the testing process, which accords with the development law of testing defects, which proves that the test plan and strategy are reliable and effective.

3.2 defect priority distribution

Xx-1.0

The distribution of defects at each level of each round is as follows:

Xx-1.1

The distribution of defects at each level of each round is as follows:

There are 824 defects above C level (including C level) found in the whole xx project testing process, accounting for 80.31% of the total defects, which indicates that the system is in an unstable state during the testing process, and there are a large number of serious problems, but with the progress of the testing process, the high priority problems gradually reduce, and the whole system tends to be stable.

3.3 defects are distributed by module

The following table shows the distribution of defects found in each module during the xx test.

Module

Defect number

%

Demand 0221

two hundred and twenty three

21.73%

Friends

one hundred and one

9.84%

Personal data

seventy-five

7.31%

Feed

seventy-four

7.21%

Log in and register

sixty-one

5.95%

Space-blog

fifty-four

5.26%

Note

fifty-three

5.17%

Space-q

fifty-two

5.07%

Account management

thirty-eight

3.70%

Space-bbs

thirty-five

3.41%

Space-gallery

thirty-one

3.02%

UIC

thirty-one

3.02%

Leave a message

thirty-one

3.02%

Space-bar

twenty-five

2.44%

System message

twenty-five

2.44%

Other

twenty-one

2.05%

Space-vblog

twenty

1.95%

Business card

seventeen

1.66%

Subscription

seventeen

1.66%

Visitors

fifteen

1.46%

Navigation

eleven

1.07%

Privacy settings

ten

0.97%

Space Management backend

four

0.39%

Safety

two

0.19%

Total

1026

The distribution trend of defects in each module can be seen from the following figure:

From the above defect distribution, nearly 30% of all defects are related to product requirements, such as unclear requirement definition, ambiguous requirement description, undefined requirement, implementation and inconsistent requirements, and so on.

3.4 reopening defect condition

From the above table, we can see that during the whole Space testing process, the reopening rate of verification defects in each round is on the high side, which is what we need to pay attention to and improve in the follow-up project.

3.5 cases of legacy defects

By the time of the release of xx-1.1, there were 51 defects left over from the entire Space project, and these defects were assessed by the relevant members of PDT and confirmed that they could be left behind for subsequent version planning. Specific defect information is omitted here.

3.6 go online to track test results

After the launch of xx-1.1 at 7:00 on March 27th, we conducted a centralized follow-up test from 7:00 to 12:00 that day, and after that, we arranged for two test engineers to spend some time each day tracking the latest developments in the launch and customer service feedback issues. As of 11:00 on April 2nd, the tracking test results are as follows: a total of 40 defects, all of which are below Class C. Among them, there are 5 demand-related problems, 35 due to environmental differences after launch, 4 open defects, 16 closed defects, and 20 defects to be verified.

4. Conclusion & problem & suggestion

4.1 Test conclusion

1. After several rounds of testing before and after two stages, although some defects have not been solved, the function of the system has become stable, and the scope, strategy and plan determined by the project have been realized, the project testing can be completed and xx-1.1 can be online.

two。 Through the test, I feel that the user experience of the product needs to be further improved in the subsequent version, and the feeling of "dizziness" in the use of the product does not rule out.

4.2 problems presented

1. Demand issues. In particular, the requirements of the xx-1.0 project, although we have seen a lot of requirements documents one after another, they give people the impression that the requirements analysis is incomplete, the requirements description is not clear, the logic, readability, implementability and testability of the requirements documents are poor, and the requirements are ambiguous. Thus it feels like mining requirements, confirming requirements, changing requirements, and reviewing requirements throughout the xx-1.0 testing process. The project requirements of xx-1.1 have been greatly improved. The requirements of xx itself go through the process of collection, analysis, confirmation and review, but there is still no unified analysis, confirmation and review of the requirements of each interface product. This part of the requirements are ambiguous and change a lot, and the readability, testability, integrity and clarity of the whole requirements document are still poor.

two。 Change control issues. This is more obvious in the testing process of xx-1.0, such as the change of project requirements, the change of project responsibility, the change of project plan and so on. Xx-1.0 has been confirming and changing requirements throughout the testing process, and there is no specification for the mechanism of requirements change, which may be changed by a meeting, a mail, or an oral communication. This problem has been improved in the process of xx-1.1 testing, but the implementation of the change control protocol is not good. All the requirements changes are not updated to the requirements documents in a timely and good manner, and xx-1.0 is more obvious.

3. Version control issues. Version management was not carried out during xx-1.0 testing; the code of xx itself was versioned during xx-1.1 testing, and the code of each interface product was managed by various technical leaders. During this period, there were cases of code coverage, code forgetting to upload or omitting deployment. It is difficult to ensure the clarity of each test version and the consistency between the release version and the test version.

4. Test for environmental problems. During the xx-1.1 testing, the test environment and the development environment are not well separated, resulting in testing and development repair defects can not be parallel; during the testing period, development engineers directly repair defects and modify the test environment on the test environment; the test environment is unstable, such as incorrect hosts settings.

5. The project plan is not clear and the responsibilities of the personnel are not clear.

4.3 Test recommendations

1. Leave behind defects. It is recommended to solve the legacy defects in patch after the launch of xx-1.1 or in subsequent versions to improve the stability of the product and user experience.

two。 Demand advice. Whether it is xx itself or each interface product, it is suggested to further strengthen the process of requirements collection, analysis, confirmation and review to further improve the quality of requirements documents: reduce the ambiguity of requirements, and improve the integrity of requirements, clarity, consistency, readability, implementability and testability of requirements. At the same time, it is suggested that the design documents (such as UI/UE, etc.) can be reviewed in subsequent projects to enhance the usability of the product and enhance the user experience.

3. Change control. It is suggested that the formulation of change control strategy and protocol should be further strengthened in follow-up projects, and the implementation of change control protocol should be strengthened. Not afraid of change, the key is to control the timing and strategy of change.

4. Version control. Strengthen the version control strategy of xx itself, especially each interface product, to ensure the clarity of the test version, the consistency of the release / launch version and the final test version.

5. Test environment. It is expected that in the follow-up project, the test environment and development environment of xx and each interface product are completely separate, or phased completely independent, and each part of the environment has a special interface person responsible for, during the testing period, it is strictly forbidden to repair defects or change the environment configuration on the test environment (if you really need to change the configuration, please notify the test and other relevant responsible persons in advance). In order to reduce the cost of communication and repeated detection.

6. Project management. It is mainly suggested to strengthen the planning of the project, such as schedule plan, human resource plan, risk prevention mechanism, etc., which will also be more conducive to efficient cooperation among project members: they can make their own work plans more timely and reasonably. and have a better understanding of when I will output and what I will do with others. Reduce the tension and panic in the process of the project, and the project becomes more controllable and controllable.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.