Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what are the problems that test engineers must know?" in the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

First, "developer testing" is "developer testing".

Developer testing is a very important part of modern software engineering. Agile development, backbone development and other advanced project management methods and processes are based on perfect developer testing. When a version is delivered every month or even every week, it is impossible to invest a large number of test engineers to conduct large-scale system-level testing, so it is necessary to automate most of the tests in the whole test pyramid.

Today we talk about developer testing. What is "developer testing"? We have a clear distinction between development and testing. Writing the code belongs to the development engineer, the test belongs to the test engineer, and in most cases the two sides are in a state of "red and blue confrontation". This is very similar to the situation of my R & D team more than 10 years ago. In today's software engineering, there are very few full-time "test engineers". The proportion of development and testing in many companies is greater than 10:1, and even some departments do not have test engineers. The role of the test engineer is no longer to run the "coolies" of test cases manually, but to manage the test system of the product. planning product tests, analyzing and summarizing functional test mind maps, designing test cases and leading the R & D team to carry out testing work is more like a "test expert / test coach". To take a simple example, my previous product was an online video conferencing collaboration product. Our daily online meetings are made with our own products, and we will use the new functional test site developed by ourselves to hold "station meetings". In addition to spending a small amount of time doing dialy update, then the test expert leads the team (including PO, architect, SM, Dev) to conduct centralized (half-hour) testing as planned. So "developer testing" is "developer testing", and many of our traditional test engineers face three ways: growth, transformation, and elimination. "Test expert" also has a high voice in the project. My previous company used backbone development and had a "one in, one out" review, and this type of "test expert" of the team had a veto. There are even PE-level testing experts in the company (equivalent to 20-21 technical experts in our company).

One forward: for whether a feature can enter release branch, open feature toggle in release branch for release-level testing.

One out: in engineer release, the quality of this function is qualified, allowing feature toggle to enter the production line.

Second, there is no test that cannot be tested by "automated testing"

Going back to the testing pyramid, white-box testing based on code level is very important from the four dimensions of "development cost", "execution cost", "test coverage" and "problem location".

Development cost: the cost of implementing test cases.

Execution cost: the cost of running a test case.

Test coverage: what we usually call line coverage and branch coverage

Problem location: test problems, locate the efficiency of the problem

Through the evaluation of the test pyramid and its four test dimensions, we can draw a conclusion:

Do as much Low Level Test as possible: because their execution speed is much faster and relatively stable than several test types at the top, they can be executed multiple times a day. In general, LLT gray does continuous integration build tasks, and even executes in MR, to ensure the quality of code entering the code repository.

In the case of automation guarantee, execute a certain scale of IT, ST, UI Test: because their execution speed is slow, the environment is more dependent, and the test is unstable first. It is usually performed once at night to periodically check the code quality and report back code problems.

Do as few large-scale manual tests as possible: because they are faster and less stable than LLT, have high labor costs, and cannot be executed multiple times a day, they have to wait a long time to get feedback results. However, they are closer to real user scenarios, so make sure that the following tests are performed within a certain cycle or critical node time to ensure software quality.

Now many companies have iterated release cycle is getting shorter and shorter, even achieved 2 weeks. Manual testing is obviously not adapted to this development model, and automating the use cases of manual testing through various technical solutions is the only way. At the code level, from the underlying business code to UI code, as long as the architecture design is reasonable, you can do UT. For the top-level UI interaction testing, test cases can also be run automatically (most UI frameworks can automate UI testing through the interface of accessibility). Even our mobile hardware can automate the extreme test of "dropping the phone". What can't the software do? At least some of the products of Tech Daniel in the industry, from the code submission Merge Request to the production line of the product can be calculated on a daily basis. The testing of this product cannot and cannot be done by manual testing.

Third, developers test "benefit in the present" and "win the future"

Many people think that the underlying developer testing, spend a lot of time, write a lot of code, and then to ensure the correctness of the function, but every time the code function or structure changes have to modify the test code. I am more efficient to debug and verify manually. It is true that debugging code through UT,API tests is not much different from running debugging manually, but debugging code through developer tests ensures the quality of current project iterations; but its more important role is not this.

We have some nouns in the bug category: Build Regression Bug, Release Regression Bug.

Build Regression Bug: the same feature in development has a bug in the new version, but there is no such problem in the previous version, we call it Build Regression Bug.

Release Regression Bug: the same feature on the production line has a bug in the new version, but there is no such problem in the previous version, we call it Release Regression Bug.

Every time we commit the code in the product, no one can guarantee that there will be no problems. Under the rapid iteration of agile development, it is impossible to carry out full-function manual testing, so developer testing, especially the underlying UT, API testing and integration testing, can easily identify and find such problems. So developers test "benefit in the present" and "win the future".

4. TDD doesn't have to write test code first.

For TDD, people's perception is to write test code first, and then write real code, which is both true and wrong. Conceptually correct, but if you do so strictly, it may not be the most efficient, which is one of the reasons why TDD is so difficult to promote. We divide the coding implementation into three parts: implementation code, test code, and debugging code. According to the concept of TDD, first write the test code, then code it, and finally debug it. Usually, when we implement the code, we are unlikely to consider it very clearly at first, and the definition of the interface is completely accurate. if we strictly follow the testing, coding and debugging, the test code will be modified frequently with the coding. Of course, this is not a big problem in itself. in the actual implementation process, many people are used to building a code framework and testing framework first, and then coding and testing. Debugging is carried out after the test is completed. So from the perspective of Huawei grayscale management, as long as the unit test before debugging, it can be called TDD development mode. BTW, of course, BDD is becoming popular now. What I want to say here is that if a team that can't even do the TDD I'm talking about, don't think about BDD.

Behavior-Driven Development:BDD combines the general techniques and principles of TDD with the idea of domain-driven design (DDD). BDD is a design activity where you can build functional blocks step by step based on the expected behavior. BDD focuses on the languages and interactions used in the software development process. Behavior-driven developers use their native language combined with the language of domain-driven design to describe the purpose and benefits of their code. Teams using BDD should be able to provide a large number of "functional documents" in the form of user stories and add executable scenarios or examples. BDD usually helps domain experts understand the implementation rather than exposing code-level testing. It is usually defined in GWT format: GIVEN WHEN&THEN. )

UT coverage of 100% is really bad.

In unit testing, we all focus on the "coverage" of an indicator. Regardless of module, function, line and branch coverage, there must be a certain percentage of coverage. But if you have achieved 100% of each item, you will be given a "bad review". It's not that you didn't do well (let's not talk about whether you did it in the right way here), but it's a matter of cost and cost performance. With the most difficult branch coverage (branch coverage), if you want to achieve 100% coverage, some memory allocation or fault-tolerant branches must be tested, then your test case consideration should be doubled, but does not bring the corresponding value. Even some code conditional branches are not executed during the life cycle of the program.

Module coverage: business module code through UT, architecture module code through IT; from the perspective of UT coverage, there is no need to test architecture code.

Function coverage: don't write UT for code that has no logic. For example, some of our functions are an attribute of get/set, and the internal implementation is saved with a variable assignment. This kind of function writes UT for coverage, and it doesn't really make any sense.

Line coverage: generally speaking, a line coverage of about 80% in a draw is a reasonable indicator. Some can be 0%, while others need 100%. If all the codes are more than 90%, the cost is high and the efficiency is low. This is not recommended.

Branch coverage: the more complex the business logic, the more test cases to write to cover, while some memory allocation error logic judgments do not need to be tested.

Use testing to drive architecture and code quality

Here we talk about test-driven architecture and code quality, mainly about making the code have perfect testability. What is the testability of the code? To put it simply, between classes, modules and modules are decoupled, classes and classes, modules and modules are programmed through interfaces. Dependent interfaces are passed in through passive injection rather than active acquisition. When the program is running normally, the interface parameters passed in are the real business objects, but when testing, you can pass in the simulation implementation of fake. Of course, not all dependent modules do this, and some business-independent Utility Library, or some specific data object implementations, can be called directly.

Here we talk about fake and mock, about Test Doubles, the basic concepts are as follows, what specific meaning each represents, you can search the Internet on your own.

Dummy object (dummy)

Stub (stub)

Spy (spy)

Mock object (mock)

Pseudo object (fake)

At present, when we are doing developer testing, we are basically using Mock Object (in fact, many of them are using Stub controlled by parameter return values). Conceptual problems aside, although the code can also be tested through Mock, in fact, using Mock basically means that our code is more relevant, module display dependence is heavy, and module portability is poor, especially C language programming. So much so that now many modules are unable to carry out unit testing at all, and more are doing integration testing.

Why did this happen? Our high-level architects are more concerned with system-level architectural design, combing the relationship between system modules and applications very clearly, usually, high-level architects can design the relationship between system modules or applications more reasonably. However, when it comes to the internal design and implementation of the specific application business, it is handed over to the low-level architect to complete. In fact, the amount of code inside these modules is not small, many of which are hundreds of thousands or even millions of lines of code. At this point, the level of the architect determines the Clean Code quality of the code. Many of the problems in our current code are not the problems of the system architecture, but the lack of strict requirements and reasonable architecture design in the specific business implementation. If there is a set of architecture scheme to standardize at the application level, then at least the interface of the module and the interaction between the module and the module can be as clear and reasonable as the system design. Then the uncertain part is the tens of thousands of lines of code within each sub-module.

The reason why the use of test-driven architecture and code quality is proposed, when a high standard is proposed for testing, we have to solve the problem of testing from the perspective of architecture. when the problem of testing is solved, Clean Code L3 is naturally achieved.

From "I want to write test dependent code" to "I want to write test dependent code"

This sentence seems strange, but it is actually the fundamental solution to unit testing. There are dependencies between modules, whether through Mock or Fake methods, no matter how reasonable the architecture, this dependency can not be eliminated, we do more reasonable design to decouple the dependency from the module. The first "I'm going to write test-dependent code" means that when I implement my module, I write test code to test. But what I want to test is how to write my test dependence. And the second "I'm going to write test dependency code" means that when I implement my code, I have to think about relying on my module in testing, how to solve my dependency, "I'm going to write test dependency code" (what I call fake object and implementation) to help modules that rely on me solve test dependency problems.

Thinking change, test-driven: to develop a module, don't think about how to test yourself first, think about how I can make it easier for others to test if they rely on me. The provider of the module provides not only the module code, but also a reusable Faked object (call validation; return value; parameter validation; parameter handling; functional simulation, etc.).

Module code writers implement their own Fake implementations, basically most of the code is done by module writers, and this is a reusable Fake implementation. The module relying party adds its own code according to its own special business needs. Basically follow the principle of 80 + 20.

Architecturally dependent decoupling, interface programming is carried out by injecting dependencies. Developer testing uses Fake to implement dependencies.

When writing test code, all dependent interfaces and dependent implementations are basically completed, focusing more on test cases rather than test dependencies.

This is the end of the content of "what are the problems that test engineers must know". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.