Shulou(Shulou.com)06/03 Report--

At the beginning of this book, it is necessary to clarify the understanding of software testing, including the core value and role of software testing, and the relationship between software testing and software development, so as to help readers establish the correct concept of software testing. this will be of great help to read and understand the contents of the following chapters.

How to understand software testing and establish the correct concept of software testing? Or from the basic concept of software testing, answer the following questions, gradually reveal the connotation of software testing, and go deep into the core values of software testing. This process is accomplished by answering the following series of questions.

What on earth is software testing?

What on earth is agile testing?

What is the purpose of software testing?

What is the position of software testing in the software development life cycle (SDLC)?

What is the traditional software testing process?

How is the process of agile testing different?

Let's begin to answer these questions. Don't worry, even if you don't fully understand, the following chapters will slowly help you understand these contents and unveil the mystery of software testing. But one thing is clear: after watching this introduction, you will have an overall understanding and correct understanding of software testing, so as not to fall into the predicament of "blind people touching the elephant" or confusion.

0.1 what on earth is software testing?

What is software testing? People often answer: software testing is to find Bug (defects) in software products. Others say, no, software testing is to verify whether the features of software products meet the needs of users. In fact, the above answers are correct and explain both the pros and cons of software testing.

(1) Software testing is to discover Bug in software products, emphasizing that testers constantly think about developers' possible misunderstandings, bad habits, system boundary conditions, abnormal input and operation, system weaknesses and vulnerabilities, in order to find software system problems more quickly. After all, developers strive to construct software, mainly in a positive way of thinking, so testers can achieve higher testing efficiency from reverse thinking.

(2) Software testing is to verify whether the characteristics of software products meet the needs of users, and to verify the correctness of all the functional points of the software system in a positive way of thinking, which is the natural extension of the quality inspection work of traditional industry in the software industry.

But only this understanding of software testing is not enough, we need a more comprehensive understanding of software testing, we can do a better job, and can also adapt to the challenges brought by different software development processes, including the great challenges brought by agile methods to software testing.

Nearly 30 years ago, G.J.Myers gave the definition of testing in his classic book, the Art of Software testing (The Art of Software Testing): "Program testing is the process of executing a program to find errors." At that time, the understanding of software testing is still very limited, which is also affected by the software development waterfall model, that software testing is a stage after programming. Only after waiting for the code to be developed, the problem is found by executing the program and operating the software like the user, which is called "dynamic testing".

If you find that the functional design is unreasonable or the performance is not good at this time, you need to modify the requirements or modify the design, then you have to rework to the requirements definition or system design stage, resulting in a high cost. Therefore, it is necessary to extend software testing to the requirement and design stage, that is, to verify the phased results-requirements definition documents and design technical documents, so as to extend dynamic testing to static testing and find problems as soon as possible. eliminate the problems in the bud, remove the defects in each stage in time, greatly improve the product quality and effectively reduce the cost of the enterprise. Static testing is to review the software or phased results when the software system is not running, including requirements review, design review, code scanning, code review and so on.

Software testing extends from "dynamic testing" to "static testing". It develops from software testing in a narrow sense to software testing in a broad sense, and makes "software testing" no longer stay at a certain stage after programming, but a quality assurance activity throughout the whole software development life cycle (Software Development Life Cycle,SDLC). With the concept of generalized software testing, in agile development, software testing can be interpreted as a continuous evaluation of software product quality. In agile methods, continuous integration is not only advocated, but also continuous testing. Continuous integration is actually for continuous testing.

According to the definition of software testing in international standards, software testing is regarded as the whole of "Verification" and "Validation", both of which are indispensable. If only one of them is done, the test is incomplete.

(1) "verification" is to verify that the software has correctly implemented the system functions and features defined in the product specification. The verification process provides evidence that software-related products are consistent with the requirements of all lifecycle activities, that is, to verify that the software implementation (that is, the product delivered to the customer) meets the software requirements definition and design goals.

(2) "validation" is an activity to confirm whether the developed software meets the actual needs of users. Because the definition and design of software requirements may be incorrect, the above consistency does not guarantee that software products meet the actual needs of customers, and customer requirements are also changing, when the definition of requirements is determined half a year ago. The possibility of such a change is greater.

Different from hardware, software is generally an application system, which is often closely related to people's entertainment, transaction processing, business activities, community communication and so on, so software has a strong sociality, so some testing experts associate testing with sociality and anthropology, and think that software testing is an interdisciplinary (interdisciplinary) activity and takes the system as the focus (systems-focused). Complete software quality assessment through continuous investigation (investigative) and storytelling (storytelling). Through its sociality, it emphasizes the tester's thinking ability and exploration ability, emphasizes the validity and reliability of the test, understands the user's behavior, the background and purpose of people's activities (context), observes and learns constantly, discovers the information related to quality (difference or query), and protects the value of the product from the customer's benefit.

In a nutshell, what exactly is software testing? It can be said that software testing is not only an activity process of verifying and confirming software products (including phased products) throughout the whole software development life cycle, but also a continuous process of evaluating the quality of software products. its purpose is to find all kinds of problems in software products (including phased products) as soon as possible and eliminate product quality risks in the process of software development as much as possible.

0.2 what on earth is agile testing?

First discuss what agile testing is from the methodological level of agile development, that is, what are the specific characteristics of agile testing, or what are the main practices of agile testing, and then discuss agile testing (or Scrum Testing) in Scrum based on the current very hot agile concrete framework Scrum. First study the following 12 principles behind the Agile Manifesto [43].

(1) our most important goal is to satisfy our customers through the continuous and early delivery of valuable software.

(2) be happy to face the changes in requirements, even in the later stage of development. Agile processes control change for the sake of customers' competitive advantage.

(3) deliver workable software regularly, at intervals of a few weeks or a month or two, and tend to take shorter cycles.

(4) Business people and developers must cooperate with each other, and every day in the project is no exception.

(5) stimulate the fighting spirit of individuals and build projects with them as the core. Provide the necessary environment and support, supplemented by trust, in order to achieve the goal.

(6) whether inside or outside the team, the most effective and efficient way to transmit information is face-to-face conversation.

(7) working software is the primary measure of progress.

(8) Agile process advocates sustainable development. Responsible people, developers, and users should be able to work together to maintain a stable pace.

(9) Agility is enhanced by the unremitting pursuit of technical excellence and good design.

(10) based on simplicity, it is the art of trying to reduce unnecessary workload.

(11) the best architectures, requirements, and designs come from self-organizing teams.

(12) the team regularly reflects on how to improve effectiveness and adjusts its behavior accordingly.

None of these 12 principles directly talk about testing, does that mean there is no agile testing? Where there is development, there is testing, but the 17 people who originally participated in the Agile Manifesto are basically all programmers and do not elaborate on the principles of testing separately in the principles. But some of the following principles are highly relevant to testing.

(1) how can software testing support or assist in the "continuous and early delivery of valuable software"? How to conduct adequate testing in a very limited time? This is what we often emphasize in agile testing as "automated testing". If there is no automated testing, there will be no agile, there will be no continuous early delivery of valuable software, and "customer satisfaction".

(2) "gladly face the demand change, even in the later stage of development" is different from the traditional development principle, the traditional development wants to have strict requirement change control, and the later stage control becomes stricter. Agile development embraces change, so how does testing adapt to this change? How to quickly complete regression testing? This may depend on the development of unit testing, or full participation in testing, and full support for automated test execution of system-level, end-to-end regression testing.

(3) traditional development also requires that "business personnel and developers must cooperate with each other", but there are certain stages, such as pre-requirements review, during product inspection (product walk-through), late acceptance testing and other requirements for close communication and cooperation. But agile development emphasizes that "every day in the project is no exception". Under this principle, how to do agile testing? This reduces the number of test documents, and there is no need to write the test plan in detail at the beginning, but to write an one-page test plan that can be continuously refined and adjusted in the future.

(4) "workable software is the primary measure of progress", which is no longer the completion of the test plan, the number of test cases completed, the number of test scripts, etc., but how to verify the functional features completed every day in time. The workload of development can not be measured by lines of code, but by how many specific user stories (features) have been implemented (done). A developer says that he has completed a user story, either through his own verification or by the tester. It doesn't matter who did the test. The key is to have a ready test and verify what has been done at any time.

(5) "unremitting pursuit of technical excellence and good design" requires, on the one hand, that the technology of testing should be continuously improved, and that the most effective way should be found when dealing with each test task; on the other hand, we should participate more in the design review in the early stage and find the design problems in a timely manner. Only with good design can the function expansion and continuous reconstruction of the system be better supported.

Based on these principles, we can summarize some of the following characteristics of agile testing.

(1) Agile testing must be part of the agile development methodology and should be in line with the ideas of the Agile testing Manifesto and the principles of agile development listed above, emphasizing the personal skills of testers. Always work closely with customers / users and other members (especially business people, product designers, etc.) Establish a good testing framework (especially the infrastructure of continuous integration testing and automated regression testing) to adapt to the changes in requirements and pay more attention to the system under test rather than test documents (such as test plans, test cases, etc.).

(2) Agile testing has distinct characteristics of agile development, such as test-driven development (TDD) and acceptance test-driven development (ATDD), as discussed in my other article, "thinking and New Development of Agile testing". The idea of test-driven development is the core of agile testing, that is to say, unit testing is the foundation of agile testing. If there are not enough unit tests, we can not cope with the rapid changes in future requirements and achieve continuous delivery. This also shows that in agile testing, developers take on more testing, which is what we call the joint efforts of the whole team in agile testing. In agile testing, there can be no full-time testers, everyone can take the initiative to take the design task and code task to do, can also take the test task to do. In agile testing, you can also practice pair testing, as developers do in pair programming-one tester corresponds to one developer, or one tester corresponds to another tester.

(3) Agile testing is everywhere and all the time. Early testing, including requirements and design review, is also advocated in traditional testing, and full-process testing is also advocated in traditional testing. However, in traditional testing, the phased characteristics are relatively prominent, for example, requirements review, which means that the product staff is asked to write the requirements first, but after the requirements document is written, the testers participate in the review. In agile testing, teams work together every day, discussing requirements and reviewing requirements together. In agile testing, this persistence is more significant.

(4) Agile testing is based on automated testing, and automated testing plays an absolutely dominant role in agile testing. Automated testing is also advocated in traditional testing, but due to the long cycle of traditional development (months to years), it is possible to cope even without automated testing. generally speaking, regression testing can take several weeks, or even 1-2 months. The persistence of agile testing urgently requires a high degree of automation, and the entire acceptance test (including regression testing) can be completed within 1-3 days. Without automation, there can be no agility.

Agile testing is a series of testing practices that conform to the idea of Agile Manifesto, abide by the principles of agile development, and can be well integrated with its overall development process in the agile development environment. These practices have distinct characteristics of agile development, such as TDD, ATDD, pair programming, continuous testing and so on. The distinction between agile testing and traditional testing can be summarized as follows.

(1) traditional testing puts more emphasis on the independence of testing, distinguishing the roles of "developer" and "tester" more clearly. Agile testing can have full-time testers or national testing, that is, in agile testing, there can be no "tester" role, emphasizing that the whole team is responsible for the test.

(2) traditional testing is more phased, from requirements review, design review, unit testing to integration testing, system testing, etc., from test plan, test design to test execution, test report, etc., but agile testing emphasizes continuous testing and continuous quality feedback, and the stages are vague.

(3) the traditional testing emphasizes the planning of the test, thinking that without a good test plan and execution according to the plan, the test is difficult to control and manage, while agile testing emphasizes the speed and adaptability of the test. focus on the continuous adjustment of the plan to adapt to the changing requirements.

(4) traditional testing emphasizes that testing is composed of "verification" and "confirmation" activities, while agile testing does not have this distinction, always focuses on user needs, and unifies verification and validation all the time.

(5) traditional testing emphasizes that any defects found should be recorded in order to analyze the root causes of defects, achieve the purpose of defect prevention, and emphasize the process of defect tracking and handling, distinguishing the different responsibilities of testers and developers. Agile testing emphasizes face-to-face communication and collaboration, emphasizes the responsibility of the team, and pays less attention to the recording and tracking of defects.

(6) traditional testing pays more attention to defects and carries out a series of activities around defects, such as defect tracking, defect measurement, defect analysis, defect report quality inspection, etc., while agile testing pays more attention to the product itself and the customer value that can be delivered. In the agile development model of rapid delivery, the cost of defect repair is very low.

(7) traditional testing encourages automated testing, but the success of automated testing does not have a fatal impact on testing, but the foundation of agile testing is automated testing. Agile testing is a fast test supported by a good automated testing framework.

0.3 the role of software testing

When you buy a product, you will find that there is a "QC" label on it, which is the sign that the product has been Quality Control. Software testing is like the quality inspection work of a manufacturing factory, which is to check the quality of software products and phased work results, not only to verify whether the products meet the prior requirements definition, design requirements and code specifications, etc., to complete the consistency check. and to confirm whether the functional features of the products meet the needs of users, each functional feature is really needed by users. Due to time and budget constraints, we can not prove that the general application system software is without problems, but can only find and eliminate these problems to reduce product quality risk and improve product quality. Therefore, software testing is one of the important means for software companies to measure product quality and ensure product quality.

Some people retort that quality is built, not tested. Yes, from the perspective of "quality is built", developers contribute more to product quality, testing contributes less to quality than development work, and testers contribute less to quality. But this can not deny the role of testing, testers help the whole team to find various defects in the product, and then urge developers to eliminate these defects, the quality of software products has been significantly improved. From the perspective of product quality and quality responsibility, whether comparing testers as "product quality gatekeepers" or "supervisors of product quality process", it shows that testers have greater responsibility for product quality, which is determined by the role of "software testers". Software testing is one of the important means of quality assurance, and many companies also put testers in Quality Assurance departments. Some companies even call testers QA personnel.

To sum up, software testing has the following four functions.

(1) Product quality assessment: comprehensively evaluate the quality of software products, provide all kinds of information necessary for product quality for software product release (acceptance testing), software system deployment (performance planning testing), software product identification (third-party independent testing) dispute arbitration (third-party independent testing) and other decisions, that is, to provide accurate, objective and complete software product quality reports.

(2) continuous quality feedback: continuous testing (including requirements review, design review, code review, etc.) can provide continuous and rapid feedback on product quality, so as to solve the existing quality problems in the whole development process in a continuous and timely manner, constantly improve product quality, and reduce all kinds of rework, so as to minimize the inferior cost of software development.

(3) improvement of customer satisfaction: find defects in the products to be delivered through testing, especially find all kinds of serious defects as far as possible, reduce or eliminate product quality risks, improve customer satisfaction, expand market share, and improve customer loyalty.

(4) defect prevention: through the analysis of defects, find out the root causes of defects (process defects, non-compliance, wrong behavior, bad habits, etc.) or summarize software defect patterns, take measures to correct deep-seated problems, avoid making the same mistakes in the future, achieve the effect of defect prevention, and effectively reduce problems in development. Improve the efficiency of development.

0.4 position of software testing in SDLC

In the famous software waterfall model, software testing is in the downstream of "programming" and upstream of "software maintenance". The location of testing is very clear, but the waterfall model does not reflect the essence of SDLC and does not accurately reflect the position of testing in SDLC. Software testing in waterfall model is a narrow test and backward testing concept.

As mentioned earlier, software testing runs through the whole SDLC, and is involved in the development activities of software products or the implementation of software projects from the beginning of requirements review and design review. Testers participate in requirements analysis and requirements review. by actively participating in requirements activities, testers can not only find the problems in requirement definition, but also have a deeper understanding of business requirements, specific user requirements and product functional features. to lay a solid foundation for test requirements analysis and design. Furthermore, the acceptance criteria for product testing can be determined at this stage, the acceptance test plan can be made and the acceptance test case (test case) can be designed. Similarly, in the software design stage, testers should have a clear understanding of all kinds of problems such as how the system is implemented, which development technologies are adopted, and what kind of application platform they are built on, so that they can prepare the test environment of the system in advance, including the purchase of hardware and third-party software. What's more, it is necessary to check the rationality and effectiveness of the system architecture design according to some non-functional features (such as performance, security, compatibility, reliability, etc.), find the problems existing in the design, and start to study how to test the current software system. complete the design of system test cases, the selection of test tools or the development of startup testing tools, and further improve the test plan. All these preparations will take a lot of time and should be carried out as soon as possible.

When the designer is doing the detailed design, the tester can directly participate in the specific design, participate in the design review, and find out the defects of the design. At the same time, complete the use cases of functional feature testing, and develop test scripts based on these test cases.

Unit testing in the programming phase is very effective and has been paid more and more attention and implemented by the industry. Some data show that unit testing can find 60% and 70% of the problems in the code, and adequate unit testing can greatly improve the quality of the program. Secondly, unit testing and programming are carried out simultaneously, which is extremely natural, and program problems can be found as soon as possible.

The position of software testing in SDLC can be fully reflected in figure 0-1 (also refer to the W model). Software testing and software development constitute an interactive and collaborative relationship in the whole process. They work together from beginning to end and work together to achieve the same goal-to complete the project on time and with high quality.

From the perspective of the team, testers are also the main force in the software development team. Although there are many roles in the software R & D team, including project manager, product manager, user interface (User Interface, UI) designer, developer, tester, software configuration manager, etc., but in terms of the proportion of the number of people, they are mainly developers and testers. In the traditional software development, especially some key systems (such as operating system, banking system, traffic signal control system, etc.), the proportion of testers is relatively high. Take Microsoft as a typical example, in its development team of Windows operating system and Office products in the past 10 ~ 20 years, the ratio of developers to testers is 1:2. The opposite example is Internet software such as Google and facebook. For example, in the development team of Google, the ratio of developers to testers is 10:1, while facebook does not have full-time testers. The ratio of developers to testers is also a hot topic that has been discussed by people. To this end, I have written two articles and published them on my blog [28]. But the basic point is that because each company, each product, product project or stage is different, it is impossible to measure different test teams with a single ratio. Because this ratio is affected by many factors, such as "what work does the developer do (whether it includes more testing work), the quality of development delivery, product quality requirements (different product quality requirements vary), development patterns" and so on. For example:

Developers have done enough unit testing, and testers can greatly reduce

Non-mission-critical systems, non-life-critical systems, software quality requirements can be reduced, testers can continue to reduce

If it is an online service system, new versions can be delivered at any time, the cost of repairing defects is low and fast, and testers can continue to reduce

In short, specific cases should be analyzed concretely.

0.5 traditional software testing process

When discussing "the position of software testing in SDLC" in the previous section, we have touched on the software testing process, and the improved V model (figure 0-1) reflects the testing process. However, in order to clarify the testing process, the basic process of software testing is shown from two lines, as shown in figure 0-2.

(1) A line is from the perspective of software engineering process, which goes through the stages of requirements review, design review, code review and unit testing, integration testing, system testing and acceptance testing.

(2) from the point of view of project management, it goes through the stages of test planning, test design, test execution and monitoring, test result analysis and evaluation (report) and project summary.

The description of the process is as simple as possible, so that the reader can understand at a glance and basically know the main work of each link, but in fact a lot of work is carried out alternately or at the same time, for example, as described in figure 0-1, the design work of system testing and acceptance testing started very early. As shown in figure 0-1, the design work of system testing and acceptance testing can be carried out in the requirements review and design review stages, respectively. Most of the content can be completed and can be improved at a later time.

Because daily build or continuous integration is advocated, unit testing and integration testing take place at the same time from a software code perspective, and there is no separate integration testing phase. However, if we consider the integration with other subsystems, third-party system integration, and hardware integration, the stage of integration testing still exists.

In practice, you can also define your own required phases, or milestones. Here is an example of the testing process that was defined in the Webex R & D process to better understand the traditional software testing process, as shown in figure 0-3 and Table 0-1. In this example, the main milestones are:

Review and issue of product requirements document (PRD) or market requirements document (MRD)

Review and issue of product specification (Spec)

Review and issuance of test plan and test plan

Design, review and issuance of test cases

Function test

System testing

Acceptance test.

0.6 Agile testing process

The traditional software testing process is discussed above, and the agile testing process is briefly introduced below, which will be discussed more in the text. In the agile testing process, participate in unit testing, pay attention to the new functions of continuous iteration, and conduct sufficient acceptance testing for these new functions, while the regression testing of the original functions depends on automated testing. Because of the short iteration cycle in agile methods, testers start testing as soon as possible, including timely review of requirements, development and design, and more importantly, timely and continuous feedback on software product quality. To put it simply, in the agile development process, the phases are not obvious enough, and the characteristics of continuous testing and continuous quality feedback are obvious, which can be described in figure 0-4.

If we are more specific and make the process more operable, we need to take Agile Scrum as an example to introduce the process of agile testing. Looking at the Scrum process first, you can see from figure 0-5 that except for the final "acceptance testing" phase, other processes do not seem to have significant test characteristics, but implicit test requirements and characteristics still exist.

(1) ProductBacklog (requirements definition phase), what does testing do when defining user requirements? In addition to the customer value (priority) and basic workload estimation, the testing needs to carefully study the user behavior patterns related to the product (such as Behavior Driven Development,BDD), product quality requirements, and which quality characteristics we need to consider? What are the competitive products? What are the characteristics (advantages, disadvantages, etc.) of these competitive products?

(2) SprintBacklog (phased task division and scheduling). At this time, you need to specify the specific functional features and tasks to be implemented. As a test, you should pay special attention to "Definition of Done", that is, the requirements for the end of each task, that is, the acceptance criteria for the completion of tasks, especially the acceptance criteria for the design of functional features and code implementation. A key step in ATDD (using acceptance test-driven development) is also reflected here, where acceptance criteria are determined before designing and writing code. On the one hand, in line with the idea of test-driven development, it is necessary to do things right at first to prevent defects; on the other hand, the basis of continuous testing and acceptance testing is also clear, and you can quickly judge whether the test is passed or not.

(3) in each Sprint implementation phase, the tasks defined by Sprint Backlog are mainly completed, at this time, in addition to test-driven development (TDD) or unit testing, continuous integration testing or commonly known as BVT (Build Verification Test) should be carried out. And developers will carefully consider the testability of each component or code block when designing and writing code, because the testing task may be done by themselves. If there is a full-time tester role, on the one hand, we can improve the unit testing and integration testing framework to assist developers in unit testing; on the other hand, we can conduct more exploratory tests according to the newly implemented functional features. At the same time, develop acceptance testing scripts. If there is no full-time tester role, these things need to be done, but by the whole team. Although there is no division of work, there is also a division of tasks.

(4) acceptance testing can be done by automated testing tools, but in general, it is impossible to achieve 100% automated testing. For example, ease-of-use testing is difficult to be done by tools. Even if performance testing is done by tools, people are still needed to design test scenarios, including key business choices, load patterns, and so on. Agile acceptance testing is different from traditional acceptance testing, which focuses on the verification of "Definition of Done", but the basic idea is consistent with traditional development, and any unverified product features can not be released directly.

-this article is excerpted from "whole process Software testing (2nd Edition)"

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.