Shulou(Shulou.com)06/03 Report--

The original translation of "What Test Engineers do at Google" is available online, as well as the related Chinese book "how to Test google Software". I will not move the content here today, but write some reading notes and insights.

Test organizational structure

Organizational structure is also one of the factors affecting the success or failure of the test team. Google's organizational reporting relationship is divided into different areas of focus, including client, geography, advertising, Apps, mobile, etc., and all engineers report to managers, directors, or vice presidents in these areas of focus. Testing is an independent department, and testing is based on various product areas, which is called engineering productivity team.

Cdn.xitu.io/2018/9/5/165a8d64ddf65e75?w=1320&h=328&f=png&s=35126 ">

Software test development engineer

Responsibilities: responsible for testability and long-term effectiveness of the test automation system.

Play the role of quality consultant

Provide support to developers in unit testing

Provide a testing framework for developers to facilitate developers to improve testing efficiency

Participate in design review, refactoring code to increase testability, write unit test framework and automated test framework

Focusing more on quality improvement and increased test coverage, SET writes code to allow SWE to test its own functionality

Test engineer

Responsibilities: assess the impact on users and risks in the overall objectives of the software product

Think about all kinds of quality problems from the point of view of users

From a development perspective, they write automated use case code for user usage scenarios

From a product perspective, they evaluate overall test coverage and verify the effectiveness of other engineer roles in cooperating on testing

Product experts, quality consultants and risk analysts

Among them, several important messages:

Developers can do tests, tests can write code, but Google hasn't fully done this yet.

SET needs to be coded, familiar with system design, and personally feels more like the role of a test architect

There is no proportion of test development, and development also takes into account testing, and full-time testing makes developers do tests more effectively and efficiently.

Equal pay for equal work in test and development

There are outsourced testers

It has been introduced that traditional software testers focus on black box testing. in the team transformation, we have made changes, giving priority to single-end full-stack testing, and taking unit testing as a watershed of concern.

Test quality concept

Quality is not tested, but this seemingly clich é contains some truth.

From the automotive industry to the software industry, if it was wrong at the beginning of the design, it will never be right. Ask companies in the auto industry how expensive it is to recall a large number of products that actually have quality problems. Therefore, it is necessary to be correct from the initial stage of creation, otherwise even if the quality inspection finds quality problems, it will fall into the abyss of chaos.

However, this sentence is not as simple and accurate as it sounds. Although the quality is not measured, there is also evidence that it is impossible to develop quality software without testing. If you don't even test it, how can you make sure your software is of high quality?

A simple solution to this problem is to stop the isolation of development and testing. Development and testing should go hand in hand. Test each piece of code as soon as you finish it, and do more tests as you complete more code. Testing is not an isolated activity, it is itself part of the development process. Quality is not equal to testing, when you put the development process and testing together, just like mixing in a blender, until you can't tell them apart, you get quality.

Quality is not equal to testing, testing can not guarantee quality, quality is built-in, not additional

Quality is a development process problem, not a testing problem.

The developer is responsible for quality

Integration of development and testing

Write a piece of code and test it immediately. If you finish more code, you will do more testing and development.

Simple and unified

Test type

The classification of test types: small tests (70%), medium tests (20%), and large tests (10%) are actually the concept of layering. Discard confusing test type terms: unit testing, code-level testing, white-box testing, integration testing, system testing, end-to-end testing.

One of the highlights, Google, was in 2007, with 15 pilot teams running at different levels: test certification. Developers follow some specific testing practices and get the desired results, then they are certified.

Test maturity level

Test maturity is what I just mentioned: test certification. Personal view: in agile teams, if the R & D team is certified for maturity, you can distinguish between different levels of testing resource input.

Level 1

Set up test coverage bundles.

Set up a continuous build.

Classify your tests as Small, Medium, and Large.

Identify nondeterministic tests.

Create a smoke test suite.

Level 2

No releases with red tests.

Require a smoke test suite to pass before a submit.

Incremental coverage by all tests > = 50%.

Incremental coverage by small tests > = 10%.

At least one feature tested by an integration test.

Level 3

Require tests for all nontrivial changes.

Incremental coverage by small tests > = 50%.

New significant features are tested by integration tests.

Level 4

Automate running of smoke tests before submitting new code.

Smoke tests should take less than 30 minutes to run.

No nondeterministic tests.

Total test coverage should be at least 40%.

Test coverage from small tests alone should be at least 25%.

All significant features are tested by integration tests.

Level 5

Add a test for each nontrivial bug fix.

Actively use available analysis tools.

Total test coverage should be at least 60%.

Test flow

Test as early as possible to participate in all aspects of participation, multiple Review documents, code, architecture. Code Review has a special Submit process.

High degree of automation with emphasis on continuous integration

Tests are divided into large, small and medium-sized tests, with different scope, executor, time and requirements.

Participate in the test as early as possible, after all, the quality is not tested, the first line of code in the whole R & D process has determined the quality, feedback risks in the process, and use effective testing strategies to eliminate quality obstacles to ensure that there are problems in the inspection office to correct in time to avoid omission.

Version division

First can climb, then walk, and finally run, the version divides the short frequency × × × key points.

The Canary version (Canary Channel), a less reliable version, is not suitable for release. Just like a canary in a coal pile, if it dies unfortunately, it means there is still work to be done. Only super tolerant users will be able to use the Canary version for trial run, and you can't rely on such an application to get the actual work done.

The development version (Dev Channel) is the version used by development engineers in their daily work. All engineers in the same product group need to install this version and use them in real work.

The test version (Test Channel), which is for internal dog food, may also be a candidate for the beta version if it has sustained good performance. [translator's note, dog food, generally refers to the product of one's own team, an intermediate product that one or someone within the company tries to use]

The Beta version or release version (The Beta Channel or Release Channel) is the first version for external users. It will become the beta version only after it has passed testing and a hail of verification from real users in the course of previous versions.

The above climb-to-walk, walk-to-run model allows us to run some tests while making some experimental adjustments to our application system, and get timely feedback from real users and each version of the daily automated test.

There are also some benefits from an analytical perspective for such a process. For example, when a bug is found, testers can create a test case based on the bug and run the test case against all versions, thus verifying that the bug fix has actually been fixed in all versions.

Fatal defects in Google process

Testing has become a crutch for developers.

Quality requires the contribution of everyone, not exclusively to the "test" engineer. The less we let developers think about testing, and the easier it is to make testing, the less developers will do testing. Testing is an independent department in Google, making the problem even more serious. Ensuring quality is not only someone else's problem, it even belongs to another department. It is easy to identify the responsible party, and it is easy to pass the buck to the quality department when something goes wrong.

Related to the organizational structure separation of development and testing

Testers focus more on their own roles than on their products. Each engineer's role serves the overall product, while the role itself is secondary. If the product goes unnoticed, it won't get better. After all, the ultimate goal of software development is not coding, testing, documentation, but the completion of a product. Each engineer's role serves the overall product, while the role itself is secondary. One sign of a healthy organization is that people say "I'm working for Chrome" instead of "I'm testing".

No role should be overemphasized. Everyone on the team is working on the product, not for some part of the development process. The development process itself serves the product. Users fall in love with the product, not the process of developing the product.

In Google, the separation of development and testing results in role-based associations that hinder testers' attention to the product.

Testers tend to worship the test product more than the software itself

The value of testing lies in the action of the test, not the test product. The test product generated by the test engineer is secondary to the code under test: the test case is secondary; the test plan is secondary; and the bug report is secondary. These products need to be tested in order to reflect their value. The value of all test products lies in their impact on the code, which is reflected through the product.

Independent test teams tend to focus on building and maintaining test products. If the test is targeted at the source code of the product, the whole product will benefit. Therefore, testers must put the product side first.

After the product has been released after the most stringent testing, users are almost bound to find the problems left out in the test.

After the product has been released after the most rigorous testing, how likely is it that users will still be able to find the omissions in the test? The answer is: almost certainly. It doesn't matter who is doing the test, the key is to do the test. Internal testers, trusted testers, crowdsourced testers, and early users may find bug more easily than test engineers. In fact, the fewer tests TE does and the more tests it supports others to do, the better it will be.

The growth process of Google Software testing

The development of software testing team also grows around the quality closed-loop activities, and different quality activities require different people. At the beginning of starting a business, one person may have multiple functions, but in the back it may be full-time and like the division of labor, no matter how it is divided, it is inseparable from the quality closed-loop activities.

Mobile Internet APP team Test Technology Stack

As the team grows and the skill set expands, the following figure shows the test technology stack, which shows the coverage strategies and tools of each aspect through layering, on which the echelon capability can be established.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.