Shulou(Shulou.com)06/03 Report--

2.3 estimation of test workload

After determining the test requirements and defining the test scope, it is necessary to define the test tasks and estimate the test workload. Based on the quality requirements and test workload, test environment, product release time and other requirements, we can determine the test progress and required test resources, or based on the existing test resources to determine the test schedule.

In the traditional development model, test workload estimation is one of the basic tasks of test planning, but in agile development, although it is also strongly recommended to have a test plan, the test plan is concise and to the point, mainly listing test objectives, test boundaries, test points, major test risks and matters needing attention. Its testing tasks are considered in iterative planning (such as Scrum's SprintPlanning) meetings and development tasks, and can be estimated by using Scrum to estimate playing cards, so the estimation of test workload mainly depends on personal experience, team communication, and so on. Even in this way, after understanding the following, there is a basis for scientific estimation that will still play a role in agile development.

2.3.1 estimation of workload

The workload of testing is determined according to the test scope, test task, and development phase. Test scope and test tasks are the main basis for the estimation of test workload. How to determine the test scope has been fully discussed in the previous section and can be decided according to the product requirements specification. The testing task is determined by the quality requirements and test objectives. The higher the quality requirements are, the deeper and more adequate the testing should be, and the frequency and frequency of regression testing should also be increased. Naturally, the workload of testing will increase. At different stages of development, the test workload varies greatly. The development process of the first version of a new product requires a greater amount of testing than the later version. But it is not absolute, for example, the function of the first version is less, in the second and third version, more new functions have been added, although the newly added function is not as much as that of the first version, but in the test of the second and third version, it is necessary to complete not only the test of the new function, but also the functional regression test of the first version to ensure that the original function is normal.

In general, a project needs 2 or 3 regression tests. Therefore, assuming that a round of (Round) functional tests requires 100man-days (man-day), then completing all the functional tests of a project must take more than 100man-days, often more than 200,300 person-days. The following formula can be used to calculate:

W = Wo+ Wo? R1 + Wo? R2 + Wo? R3

W is the total workload, and Wo is the workload of a round of tests.

? r _ 1, R _ 2, and R _ 3 is the decreasing coefficient of each round. Affected by different code quality, development process and test cycle, the values of R1, R2 and R3 are different. For every company, the experience value can be obtained from the data accumulated in history.

The workload of testing is also affected by many factors, such as the degree of automated testing, programming quality, development mode and so on. Among these factors, programming quality is the main one. The lower the programming quality, the more repeated the test (regression test). The scope of regression testing may vary in these three times, depending on the test results, that is, the distribution of test defects. If there are many defects and are widely distributed, all test cases will be executed again. There are few defects and the distribution is relatively concentrated, so we can select some or a small number of test cases as the scope of regression testing.

In the case of relatively low code quality, assuming that the values of R1, R2 and R3 are 80%, 60% and 40% respectively, if the workload of a round of functional testing is 100 person-days, the total test workload is 280 person-days. If the code quality is high, generally only two rounds of regression tests are needed, and the values of R1 and R2 are also reduced to 60% and 30%, then the total testing workload is 190 person-days, and the workload is reduced by more than 32%.

The higher the degree of automation, the lower the testing workload. Automated scripts run by computers are very efficient and can greatly reduce the workload of executing actual tests. However, in many cases, test automation does not significantly reduce the workload because of the heavy workload of test script development. In other words, the overall test workload has been moved forward, from the test execution phase to the test script design and development stage, the overall workload has not been significantly reduced. At the same time, because automated scripts can be reused and machines can run day and night, regression testing can be carried out frequently, such as once a day, so that any regression defects can be found instantly and improve the quality of software products.

The estimation of workload is complex, and the estimation methods are different for different application fields, programming techniques, programming languages and so on. Its estimates may be based on some assumptions or definitions.

(1) efficiency hypothesis, that is, testing the efficiency of the team. For functional testing, it mainly depends on the complexity of the application, the number of windows, and the number of actions in each window. For capacity testing, it mainly depends on the amount of data needed to build the test.

(2) the test hypothesis is designed to verify the number of actions to be tested for a test requirement, including the estimated time taken for each test case.

(3) Phase assumption refers to the division of different stages of the test cycle (test design, script development, test execution, etc.), including the length of time.

(4) complex vacation decision, the complexity index of the application and the influence degree of the demand change determine the dimension of the test demand. The more dimensions of the test requirements, the greater the workload.

(5) risk hypothesis. The risks under the influence of various factors are generally considered, and the workload caused by these risks is set at 10% to 20% of the estimated workload.

2.3.2 work breakdown structure table method

In order to estimate the test workload well, it is necessary to refine the test tasks, decompose each test task, and then estimate it according to the decomposed sub-tasks. Generally speaking, the smaller the particle size of decomposition is, the higher the estimation accuracy is. A floating range of 10% to 15% can be added to determine the actual test workload required. A more professional approach is the work breakdown structure table (WBS), which is done in the following three steps.

(1) list the tasks that need to be completed in this project, such as test plan, requirements and design review, test design, script development, test execution, etc.

(2) each task is further subdivided and can be subdivided into multiple levels until it can not be subdivided. For a test plan, it can first be subdivided into:

? Determine the test target

? Determine the test scope

? Determine test resources and progress

? Test plan writing

? Test plan review.

"determine test scope" can also be subdivided into functional test scope and non-functional test scope analysis. "Test Plan Review" can be subdivided into test team review, project team review, company quality assurance team review and final approval.

(3) after listing all the tasks that need to be completed, the tasks are numbered according to the level of the task to form a complete work breakdown structure table (as shown in Table 2-5).

In addition to being expressed as a table, WBS can also be expressed as a structure diagram, which will be more intuitive and convenient, as shown in figure 2-5.

When the WBS is completed, you will have the basic information for scheduling, resource allocation and budgeting, so that you can obtain not only the overall test workload, but also the workload of each phase or task, which is conducive to resource allocation and scheduling. Therefore, the WBS method is not only suitable for workload estimation, but also suitable for scheduling, resource allocation and other planning work.

2.3.3 examples of workload estimation

Combined with the function points of the Google calendar, we can see that the test workload is proportional to the number of test cases. According to the comprehensive and detailed test cases, the timing of each phase of the test cycle can be estimated more accurately. According to the functional calculation of the Google calendar, the number of test cases is 6 × 60 = 360 (based on an average of 60 use cases per large module). In addition to the number of test cases, consider the following factors.

? According to the specific circumstances of the test team and the project, such as the assumptions in Section 2.3.3: efficiency assumptions, test assumptions and application dimensions.

? Different combinations of test platform and environment, including operating system, browser, communication protocol, firewall, proxy server and so on.

? Regression test frequency and repetition times.

? The level of automated testing.

? For other specific factors, increase the margin by 10% to 20%.

In the test of the Google calendar, make the following assumptions and analysis.

? All personnel are intermediate software test engineers.

? The design time for each test case is 20 minutes, including the time it takes to review and input into the use case management database. So the test case design time is 120 hours, that is, 15 person-days.

70% of the test cases can be automated and 30% manual. That is, the number of automatic test cases is 252, and the number of manual test cases is 108.

? Each engineer can develop test scripts for 10 test cases per day, including debugging. So the workload of test script development is 26 man-days.

? To carry out two regression tests, if the values of R1 and R2 are 70% and 40%, the number of test cases manually run on a single platform is 108 × (1.0 70% 40%) = 227.

? There is no impact on the operating system, and SSL support is not considered, only the impact of browsers IE6.0, IE7.0, Firefox1.5, Firefox2.0 and proxy server is considered. As a cross combination, there are 4 kinds of them.

? It is also not necessary to run all the test cases on the four combinations, the two main combinations run 100% of the manual test cases, and the other two combinations run 50% of the manual test cases, that is, the number of test cases is three times the original, so the number of test cases run manually is 227 × 3 = 681.

? Assume that each test engineer can run 60 test cases per day, that is, it takes 5 minutes for each test case to execute, 5 hours to run the test case, and another 3 hours for processing defect reports and emails, communicating with developers, and so on. So the manual test case execution time is 12 person-days.

? Automated tests are run at night, and engineers need time to analyze test results, modify scripts to adapt to new changes, make defect reports, etc., which is estimated to take 5 man-days.

In this way, the basic workload of functional testing is estimated, that is, 15,26,12,558 person-days.

The workload of system testing can be carried out according to the same method, but the difference is that the system testing is almost completed by testing tools, and the workload is mainly concentrated in environment construction, test data preparation and result analysis. Table 2-6 shows the testing effort required for the Google calendar.

2.4 Test resource requirements

After analyzing the test scope, the test resources required are clearer. Resource requirements for testing, including human resources and software and hardware resources. Human resources focus on how to set up a test team-project test group, while software and hardware resources vary greatly from project to project. Only the general operation methods and the establishment of the design test environment are discussed here, which will be introduced in detail in chapters 7 and 9.

If the test resources are classified in more detail, they can be summarized as shown in figure 2-6.

1. Human resource demand

After the estimation of the test workload is completed, the number of personnel required for the software test project can be basically determined. The personnel and requirements required for a software testing project vary from phase to stage.

(1) in the initial stage, the test team leader should first intervene, participate in the requirements review, determine the test requirements and test scope, and formulate the test strategy and test plan.

(2) in the early stage of testing, some senior test designers, test scripts or test tool developers are required to participate in or be responsible for the formulation and decomposition of software test requirements, the design of test cases, the development of test scripts and so on.

(3) in the middle of the test, it is mainly the execution of the test, and the number of test requirements depends on the degree of test automation. If the degree of test automation is high, the manpower investment does not need to be significantly increased; if the degree of test automation is low, there will be more requirements for the personnel who execute the test.

(4) in the later stage of testing, senior testers can take part of the time to prepare for the new project.

2. Test environment resources

The computer software resources and hardware resources needed to establish all the necessary test environment are collectively called test environment resources. Hardware provides a basic platform to support the operation of operating systems, application systems and testing tools, and software resources include operating systems, third-party software products, testing tools software and so on.

? Hardware: switches, routers, load balancers (Load balance), servers, client PC, cameras, special display and sound cards, headphones, microphones, etc.

? Supporting system software: Linux operating system, Web server (such as Apache), middleware (such as Tomcat, WebLogic), database system software MySQL/Oracle and so on.

? Testing tools: JUnit, JMeter, Selenium, IBM-Rational Robot and so on.

2.5 Test milestones and schedule

Software testing runs through the whole life cycle of software product development, from product requirements analysis and review to final acceptance testing to software release. From the actual process before and after testing, the process of software testing is composed of a series of different testing stages. these stages are: requirements analysis review, design review, unit testing, integration testing (assembly testing), functional testing, system testing, acceptance testing, regression testing (maintenance) and so on. In the plan of a software test project, it is necessary to set a clear start and end time for each phase, which is commonly known as the schedule. The schedule of the project actually depends on the test workload and existing human resources. When the human resources are sufficient, the test cycle is short; when the human resources are less, the test cycle is longer.

A milestone is generally a sign of the completion of phased work in a project, that is, a concluding sign is used to describe a clear starting and ending point of a procedural task, and the schedule is to determine the starting and ending point of the milestone. A milestone marks the end of the previous phase and the beginning of the next phase, that is, defining the criteria for the completion of the current phase (Entry Criteria) and the conditions or prerequisites for the start of the next new phase (Entry Criteria). Milestones are highly sequential and have the following characteristics.

? Milestones are also hierarchical, defining child milestones in the next level of a parent milestone.

? Milestones may be different for different types of projects.

? The number of milestones of projects of different sizes varies, and milestones can be merged or broken down.

2.5.1 traditional testing

In the software test cycle, it is recommended that you define the following six parent milestones.

(1) M1: requirements analysis and design review.

(2) M2: test plan and design.

(3) M3: the code (including unit testing) is complete.

(4) M4: test execution.

(5) M5: code freeze.

(6) M6: the test is over.

Each milestone is subdivided into sub-milestones, and if the project cycle is long, each sub-milestone can be further divided into smaller milestones for more effective control, as shown in Table 2-7.

2.5.2 Agile testing

In the introduction at the beginning of this book, we take Scrum as an example to briefly describe the agile testing process. In agile testing projects, how do you identify test milestones? Agile testing also needs to go from test plan to test design to execution, but the boundary between test design and execution is not so clear, and test design and execution are often carried out alternately or side by side. In agile testing, you can even directly verify against Use Case or User Story and conduct exploratory testing without the need for test cases. The time saved is used to develop automated test scripts with relatively stable functions to serve the later regression testing. Automated test scripts will replace test cases and become the wealth of software organizations. The original test specification also requires two rounds of regression testing, but only one round of regression testing can be carried out in agile testing. Taking the above considerations into consideration, the actual flow of agile testing is shown in figure 2-7, which is simple and effective.

In such a process, the big framework is no different, and the review of various test pieces (test plans, requirements, automated scripts, etc.) is still needed, but there is no clear review phase, and testing is a continuous quality feedback process. The phase is not so prominent, but you can still set some control points, that is, milestones:

(1) Test task definition

(2) Test plan formulation, review and approval

(3) the list of test requirements or test points (or test scenarios) is clear and approved.

(4) the acceptance test is over.

-this article is excerpted from "whole process Software testing (2nd Edition)"

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.