Shulou(Shulou.com)06/03 Report--

Advantages and disadvantages of automated testing

Why should automated testing be carried out? Which is better or worse than manual testing or automated testing? This section discusses and explains this problem.

In the testing work, we often have to face a lot of problems. Complaints from test engineers are often heard, and their complaints are roughly as follows:

V too much repetitive and frequent work

V testing has the advantages of heavy task, short testing cycle and high working pressure.

V innovative work is too few to reflect one's own ability and value of life.

V test results sometimes need to be accurate to seconds, manual testing is too difficult to do.

So how to free the test engineer from the complicated testing work? How to reflect the value of the test engineer and give full play to the specialty of the test engineer? How can the test results be accurate to seconds?

The above problems can be solved by automated testing, on the one hand, it can improve the efficiency of testing, on the other hand, it can reflect the ability of test engineers through automated testing, and change people's views on testing work.

1.1 automated test definition

Since automated testing can make up for some of the shortcomings of manual testing and reflect the value of test engineers, what is automated testing? What are the characteristics of it?

Automated testing is a process of partially or completely replacing manual testing and improving testing efficiency with the help of testing tools and according to test specifications. It has the following main characteristics:

V automatic testing process is a process of simulating manual operation to complete the input to the tested system and checking the output.

V automatic testing is performed by software instead of manual operation, issuing instructions to the GUI of the system being tested, simulating the operation, and completing the automatic testing process.

Compared with manual testing, automated testing can be summarized as follows:

V optimization cost: reduce the amount of labor and test cost

V reliable: improve the comprehensiveness and accuracy of testing

V Fast: speed up testing

V normalization: provide a standardized testing process

V reusable: improve the reusability of testing;

1.2 the relationship between automated testing and manual testing

Since automated testing has such a great advantage, can manual testing be withdrawn from the testing stage? First of all, let's think about a few questions. Answer the reader's questions through the discussion of the following questions.

1. Who finds more defects than manual testing or automated testing?

The main purpose of testing is to improve software quality by finding and solving defects. In general, the execution of tests depends on test cases. The common way to execute test cases in testing work is to run test cases manually. Imagine that if a test case is automated, the correctness of the script after automation should be tested first.

According to statistics, manual testing can find more than 80% of defects, while automatic testing can only find about 20% of defects. This precisely reflects that automated testing originates from manual testing and is only a substitute for manual repetitive labor.

two。 Which test quality is higher or lower?

Automated testing (tools) can only judge the difference between the actual result and the expected result. Therefore, in the process of automated testing, the testing task evolves to verify the consistency between the actual results and the expected results. The purpose of testing is to improve the quality of testing. Therefore, through manual testing, the test quality is higher.

3. What is the relationship between automated testing and software development?

Automated testing is more "fragile" than manual testing. In the process of software development, some functional changes may also cause automated testing to collapse. Because automated testing is more expensive than manual testing and requires constant maintenance, it also limits the development of automated testing.

4. Are there any limitations of automated testing tools?

After all, automated testing tools are software, and tools can only be executed according to booking instructions. Both automated testing tools and testers can perform a set of tests according to instructions, but people have ideas and can accomplish the same task in different ways and thinking.

For example, test engineers run test cases (or test scenarios) and often need to check whether the actual output is correct during test execution. At this time, even if the actual output of the software is consistent with the expected output, the software may have defects, which can be judged by the tester, but not by the testing tool. Testers can use their imagination and creativity to improve test cases (or test scenarios), while test tools are only rigidly executed.

Through the discussion of the above four issues, we can draw a conclusion: manual testing will not withdraw from the stage of history, it has the value of existence. At the same time, manual testing has many advantages over testing tools, manual testing can flexibly deal with unexpected events. For example, when a network connection is lost, manual testing can solve the problem as quickly as possible, but such an accident can terminate the execution of the automated test.

Sections 1.2 and 1.3 above explain the necessity of automated testing and manual testing, respectively, so when will automated testing be introduced into testing? What are the advantages of automated testing? The following sections will reveal one by one.

1.3 when to start automated testing

When is the right time to carry out automated testing?

In the testing work, the software under test (or system) requires multiple versions of iterations. The timing of release varies from company to company (or project).

Before carrying out automated testing, it is necessary to manually test the functional modules under test according to the test cases. When the manual test passes, use the automated test tool to record the manual test operation and save the correct results (called expected results). Since the automated testing tool records a key × × module, the module should still be tested after the next version of the software (system) under test is released.

In the testing of the new version of the software, start the automated test tool and run the previously recorded automated test script. Comparing the actual running results with the expected results, if it is inconsistent, it will be determined as a defect; if consistent, it is considered that the functional module has passed the test in the new version.

At this point, we can draw a conclusion: automated testing is suitable to be carried out after the version of the software (or system) under test is relatively stable. If the software version is relatively unstable, it will cause scripts recorded with automated testing tools to fail to play back in the new version, which is a waste of manpower and time.

1.4 advantages of automated testing

What improvements, breakthroughs and delight can the development of automated testing bring to testing work? In fact, the characteristics of automated testing can well summarize the advantages of automated testing. Its main features are as follows:

1. Repeatable

There is no denying that software testing is sometimes a complicated and repetitive task. The key × × modules have to be tested repeatedly in the iterative test version, and this work will continue with the iteration of the software test version. When automated testing is carried out, these repetitive tasks can be replaced by automated tools. It can greatly reduce the workload and pressure of regression testing, effectively improve work efficiency and shorten the time of regression testing.

two。 Programmable

After the in-depth development of automated testing, it will no longer be a simple recording and playback, automated testing will optimize the recorded test script and greatly improve the flexibility and interaction of the script. A script is generated after automated test recording, which contains the actions and data generated during recording. In the testing work, different test paths need to be covered with different test data to meet different test scenarios.

Therefore, data maintenance will be a complicated work. You can separate the data from the recorded automation scripts and manage the test data with external data sources, while the test scripts are only responsible for the organization of the test process. In this way, the flexibility and sustainability of automated testing can be greatly improved.

In the follow-up automated testing work, we can also develop UI interface and use UI to drive automated test scripts. Increase the interactivity of automated testing.

3. Reliable, improve test accuracy

Automated tests can perform tests that manual tests are difficult to achieve or impossible to implement. For example, the execution of the test should be accurate to seconds; simulate a large number of users to test a certain function point at the same time. These tasks are impossible or difficult to achieve by manual testing. After automated testing, it is easy to achieve these testing requirements.

4. Effective use of resources

Automating the complicated and repetitive testing tasks can improve the accuracy and work efficiency, and improve the enthusiasm of the test engineers. Free the test engineer from the complicated and repetitive work and devote more energy to other testing work (such as test quality), which is more conducive to the improvement of test quality. In practice, some test scenarios are only suitable for manual testing, and test engineers can focus on manual testing to improve the efficiency of manual testing.

1.5 scenarios for automated test implementation

As mentioned in section 1.4, not any manual test is suitable to be replaced by automated testing. Which occasions are suitable for automated testing?

V regression test

V more and more frequent testing

V work that cannot be done by manual testing

V testing of cross-platform products

V highly repetitive operation.

Which occasions are not suitable for automated testing?

V software version is unstable

V involves testing of interaction with physical devices

V test results are easier to pass the manual judgment test.

1.6 cost of implementing automated testing

At present, many enterprises or managers expect to use software test automation as a catalyst to improve efficiency and quality, and at the same time save money. Some enterprises even hope that after the implementation of automated testing, it can bring benefits to the enterprise.

Here, it is certain that automated testing has been successfully implemented in many fields, and there are many successful cases that give enterprises hope, and these enterprises have indeed saved considerable costs after the successful implementation of automated testing. Unfortunately, there are also many cases of failure, even behind the glamorous success cases, there is also a process of disappointment and failure. There are also many enterprises around them that have tasted the taste of automated test failure more than once. Spent a lot of money on automation software, was shelved, efforts came to naught. The result of failure not only caused the loss of manpower and material resources, but also directly brought economic losses to the company.

Many factors need to be considered in the early stage of the implementation of test automation, and human, material and financial resources need to be carefully considered and planned. When automated testing is carried out throughout the test cycle and which testing can be replaced by automated testing need to be carefully considered before you begin to deploy automated testing. The factors that affect the efficiency of automated testing are not only the quantifiable testing work, but also many intangible factors that affect automated testing, such as the deployment of testing organizations. Therefore, before the real implementation, we must carefully plan and consider the automated test implementation plan.

The implementation process of automated testing is completely different from manual testing. Automated test cases are also very different from manual test cases. In the process of automated test implementation, it is necessary to constantly develop and maintain scripts, so there are high requirements for testers' ability.

Before the implementation of automated testing, it is necessary to carefully analyze and plan the test scheme and calculate the cost of automated testing. The cost of automated testing includes the following:

V software cost: the cost of purchasing automation software products

V training costs: training fees for automated test engineers

V labor cost: the labor cost of automating test cases and test script writing.

Only by reasonably planning the cost of automation and considering the costs and benefits of automation from a global point of view can we increase the possibility of successful implementation of the automation scheme.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.