Shulou(Shulou.com)06/03 Report--

Write your own essays, anyway, it doesn't matter whether the writing is good or not, the important thing is to find a place to put it, and friends can see it, like it, have fun, and if you don't like it, you can beat the brick ~ 1, about the insecticide effect of the test.

The word I have seen recently may be a little out of date, but I think it makes a lot of sense. When a test works with a development team for a long time, just like the effect of the same pesticide on pests on crops, at first it is sharp and effective, and slowly the effect will get worse and worse, that is, the pests know where the weakness of the insecticide is, where the blind spot is, and thus produce tolerance and immunity to pesticides. Testing is like pesticides. We want to remove bug from the developed software, but we have been working together for a long time. Students' techniques and ways of thinking will be confined in a certain range, so that the bug outside this range is not easy to see. In fact, this problem is not insurmountable. Learn from the example of pesticides, maybe you can think about it:

Changing other brands of pesticides, that is, the rotation of test posts, everyone's way of thinking is different, and everyone's blind area is also different. at this time, rotation can reinforce the overlapping parts, and the blind areas can be found little by little and complement each other and communicate with each other. can become living water.

Pesticide upgrading, formula modification, that is, testing students through a variety of ways to expand their horizons and ideas, try to eliminate their own blind spots, this expansion and improvement is not limited to technology, but also includes the way of thinking and the understanding of the core of the business.

Multi-use of different pesticides, that is, cross-testing, is about the same as changing brand pesticides. A variety of different coverage will make the blind area smaller, but it may also lead to side effects, such as longer time-consuming and low ROI.

Increasing the concentration of pesticides, that is, increasing the density and intensity of testing, can also solve some blind areas, and some intolerant bugs will be removed, but there are also drawbacks, such as low ROI, high density and intensity will make stubborn bug more stubborn, but also reduce work enthusiasm and sense of achievement.

two。 Reverse thinking about testing

The core idea that we've been doing automated coverage is: "Let's see if the system has bug and it's not the same as before." it's like there's a road where a robot follows a given route to see if there's a hole in the flat area.

According to this idea, another core idea may be "there is now a bug in the system, can you find it?" just like there is a way, it is now clear that there is a hole, and the robot follows its established route and can find this hole.

Then think about it according to this train of thought, and find that it may be possible to think the same way.

There are 1-10 bug in the system now. Can you find them all?

There are at least 10 bug in the system now. How many can you find?

There is a bug about digital formatting in the system. Can you find it?

There is a bug about digital formatting and a null pointer bug in the system. Can you find it?

There are three types of bug in the system, two each. Can you find all of them?

...

After thinking about it, someone at the Cloud Congress this year shared an idea to modify the code under test, inject faults or modify runtime data by means of AOP, so as to check the validity of test cases, that is to say, instead of finding holes in the darkness of the unknown, create a clear one or more holes to see if our use cases can be found.

There are actually a lot of things that can be done in this way, and it is also a higher requirement for testing students' understanding of business codes. They need to be able to understand business codes and have the ability to simulate fault simulation bug, and then use this to test their own automation coverage ability. Perhaps this is the question that some students have asked, but do not know how to answer:

"you use automation to test business systems, so what do you use to test your automation?"

Now this question may be answered like this: "using automated assertions to test business systems and bug testing automation of business systems to open closed loops and complement each other."

3. Write automated test cases with the idea of developing business systems

When we test the business system, we consider ui from unit to interface, and then consider large qps to consider multiple concurrency, fault tolerance to consider exception handling, and monitoring to consider.

When designing use cases for automated tests, you might also want to see if you can do this, such as:

Unit capability encapsulation, which may be an interface or data preparation, each unit can be run separately and can be reused

Unit capabilities are organized into processes, which may be interfaces-interfaces-data may be data-data-interfaces. Each choreography can be run separately and can be reused, such as choreography into a submission process, initiation of approval process, and approval process.

The process is organized into business, which may be submitted-approved-approved may be submitted-approved-rejected

Considering the concurrency problem, multiple triggers in a single use case are triggered to run at the same time.

Unit use cases are also input-run-output-assertions that need to be tested, just like a method in business code.

Encapsulate the automation use case of one's own business line into a set of hierarchical framework with verification emphasis composed of unit use case, unit process and unit business. for example, the unit use case mainly verifies the robustness, reliability and correctness of a single interface or data preparation, etc., and the unit process use case mainly carries out business process cascading through the data output from each unit use case to verify a single process [or a page. The rendering of a page requires many APIs to take effect at the same time] correctness. Business use cases complete business mainly through the capabilities of each process in series to verify the correctness of the business.

Take the interface as the unit to encapsulate the unit use case, the page as the unit to encapsulate the process use case, and the page operation step as the unit to encapsulate the business use case.

4. About automated testing

The core purpose of data construction automation is to construct test data. if the expected data can be constructed, it represents that each link and interface is ok, which verifies the function by the way, and the function can be replaced by manual testing.

Business verification automation, the core purpose is to verify business functions, about three basic blocks are needed, and the role can be replaced by manual testing. This kind of automation is seriously interfered with its stability and result credibility by data and environmental problems.

Data construction, which is used to generate data to be validated for subsequent processes

Perform verification, carry out relevant business logic operations through established data, and verify correctness

Data destruction, the constructed data, temporary data generated after execution, garbage data are destroyed to ensure automatic cleaning.

Necessary automation, the core purpose is to complete some necessary enumerations and permutations related to verification, its role can not be replaced by manual testing.

Business verification automation is mostly based on verifying business integrity. The core logic of this kind of automation is "it's supposed to be like this. Next time I run, I'll see if it's still like this." for bug discovery ability is actually relatively weak, necessary automation can find bug in various limit scenarios of permutation and combination to a great extent, but there are few scenarios that need this kind of automation, and data type automation Its ability to find problems and bug is the weakest, but the improvement of manual testing is the most obvious.

UI Automation

The front end is closest to the user's habits, and the user's intuitive perception of the system's ability directly determines the user's evaluation of the system.

The stability, reliability and ROI of UI automation have always been the core reasons that hinder this should be at the forefront of the business.

Interface automation

The correctness and robustness of the business logic on the server side is the foundation for the smooth operation of the business and the foundation for the business to win. Directly determine the ability to support and expand the business.

Compared with UI, interface automation is more stable, can verify more points, is more in-depth, and is easier to find problems with server-side business logic, ensuring the correctness and integrity of business core logic.

Unit testing

The correctness and robustness of the unit logic directly determines the interface and business ability assembled by the unit, and is the cornerstone of the robustness, stability and expansibility of the whole project.

Compared with interface automation, unit testing is extremely stable, generally in the same project as business, verification granularity is finer, verification is more comprehensive and in-depth, and it can better find the problems of each logical unit. to ensure that the foundation of each logical unit is correct, complete and robust, but at present, unit testing is difficult to advance, and it is quite difficult to implement, which increases the development time, the project time and the workload. Therefore, most applications today do not have unit tests installed.

Safety testing

Try to break the system through technical means, similar to security drills, mining deeper framework, or deployment architecture, or network, or middleware and other related bug.

There is a high demand for professional skills, not only an overall understanding of the network, operating system, middleware and open source framework, but also a deep understanding and analysis of the business of the current business system, and even a better understanding of the business system code than the developer classmates. comprehensive professional technical testing of the system from all aspects to ensure the security of the system

Performance testing

Through a great deal of destruction of the system in all aspects, to see the supporting capacity and reaction speed of the system, such as whether there will be a problem when 10000 people are using it at the same time, how long can 10000 people use it at the same time, how many people can support at the same time, and so on. In fact, in order to ensure the availability of the system, when a system has full marks in business capabilities and UI design interactions, if it is always unavailable or slow to use, such good things as full marks in business capabilities and full marks in UI design interactions will disappear.

There is a high requirement for professional skills, which requires not only a good understanding of the overall deployment interface of the system, but also the evaluation of the framework, implementation scheme, database and network structure adopted by the system, and the analysis of business characteristics. thus, through the simulation of various scenarios of the system, a comprehensive evaluation of the support capacity of the system, roughly, stability testing (large parallel support for a long time without a problem) Capacity testing (how much concurrency the system can withstand), concurrency testing (business logic processing when a business is operated at the same time, increasing or decreasing inventory problems).

In fact, no matter which kind of test, when the accumulation is abstracted to one level, it can be automated, performance test automation, security test automation, UI, interface and other automation, and even with the help of AI can automatically generate automatic case, automatic correction and so on, but the problem is that when these things are automated, then there needs to be a means to verify automation. Otherwise, automation becomes an open-loop thing, which often becomes "in order to achieve the goal". In the end, automation does not work, manual return coverage is not done, and thus abandoned.

5. Test [automated test]

Dare to say the next words:

Automated testing is used as a means to measure the capability of business systems, but there are few measures to measure the ability of automated testing.

Code coverage, branch coverage, core business coverage, these can prove that the test has covered, or covered, but does not reflect the ability of automation, for example, I added case, but did not make an assertion, coverage will go up, but in fact it is invalid case

When measuring the proportion or duration of automation efficiency, it is rare to take into account the proportion and duration of maintenance automation. If one hour of maintenance is needed for every 5 minutes of efficiency improvement, then it is necessary to consider whether automation is unnecessary or whether there are problems to be solved.

Single, non-closed-loop metrics often lead to "for metrics"! Automated testing has checks and balances on business systems, but automated testing has no checks and balances.

Try to think about the closed-loop measurement of centralized automation and business systems to measure business systems: use automation to measure business systems

Measure business system: use automation to confirm whether a function or interface or front end of a business system meets expectations.

Measure automation: confirm whether automation will be found by using a function or interface or front end returned by the interface that does not meet expectations.

Maybe a few things need to be done.

Starting from the systematic definition of the business scenario: look at what the automation core verifies and what is guaranteed. If it is the guarantee business, then select the business defined by the core business and fault scenario, and randomly select a point for bug injection.

Starting from the intuitive perception of the user's use: you can randomly select yourself as a user, or find a classmate next to him to see what functions and places he will care about on the system, and then inject them randomly in these scenarios.

From a competitive point of view: if I am a user of your system and you are a user of my system, then we transfer over and you inject yourself into the place where you use my system most, and I do the same thing to your system. Let's all see if each other's automation can have problems and have the ability to perceive each other's business concerns.

From the point of view of red and blue: when the developer is released, he or she will randomly write a bug for the most important place to check whether the automated test can be found after the release, or provide the injection ability to the developer, PD, and business side. this behavior can greatly improve the developer's recognition of testing professionalism, automation, and the credibility of automation results.

A big question is, where is the bug and fault injected?

We still need to use data mock, environment probing, data preparation and other means to ensure that the business logic that needs to be verified has all the necessary conditions when the automation is running, and then we need to inject bug into the system by technical means or directly changing the code (not recommended), and then execute automation. Try to simulate the situation of bug in the system when all the necessary conditions are complete. Verify the ability of automation with this known bug.

Most of the time, the bug on the server is difficult to be perceived by users, but the front end is easily perceived by users. Therefore, the bug injection verification automation capability of the front end becomes more valuable, and the data is returned to the right pair, but the front end is not loaded, the data return is incorrect, but the front end shows, the back end reports that the front end does not have a pop-up window, the front end icon is not positioned correctly, and so on. Try to make the front end the most obvious place where the user perceives it as much as possible, and magnify the ability of automatic verification to look like a real user is operating.

In reverse way, "my system told you today that I have bug, and I'd like to see if you can find it automatically."

By means of data mock, environment probing, data preparation and so on, to ensure that the business logic that needs to be verified has all the necessary conditions when the automation is running, try to verify the business logic as pure as possible. Use automation to verify the business capability of the system.

Through data cleaning and data recovery, we will clean up the intermediate data and temporary data generated by automated operation, and try to automate what I have done as much as possible. I have seen it before, but it does not take away a cloud or leave a piece of garbage.

Positive mode "I want to automate today to see if you have bug in this system."

6. I am still thinking. I will put everything I think about here. Of course, things that have relevant implementation and effectiveness will also be updated.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.