Shulou(Shulou.com)05/31 Report--

This article mainly explains "how the quality of SAP software ensures the change of work". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how the quality of SAP software ensures the change of work".

When we talk about the quality assurance of software products, it must be based on a certain software development model. If the skin does not exist, how can the hair be attached? Without the software development model, quality assurance work is a castle in the air. I believe you all feel that in the past decade, new concepts and words have been emerging in the software development model, such as Agile, Continuous Integration, Continuous Delivery, DevOps. We must first understand the changes of software development patterns, and then we can carry out quality assurance activities that match the development patterns.

1. Waterfall development

The traditional waterfall model is shown below:

In this mode, the test activity is only the later activity of the linear development activity. Quality assurance depends strictly on individual documents (requirements documents, design documents, test plans and test reports) and review meetings, and automated testing is optional.

two。 Incremental development

The team completes the requirements, design, implementation and testing of the product in several iterative cycles, and the deliverables at the end of each iteration are regarded as increments of the product, which are not required to meet the deliverable requirements, but need to be able to work basically. Product delivery still occurs at the end, as shown in the following figure:

The core of incremental development is continuous testing and continuous integration. For quality assurance work, it is divided into two types of activities. One is the quality assurance of increment in iteration, and the other is the quality assurance of the whole product before release. Because the requirements of increment and final delivery of the product are different, the team usually stops functional development and carries out comprehensive regression testing and defect repair before the software release, so as to ensure that the product quality meets the delivery requirements. The advantages of incremental development are obvious:

The planning, execution, and evaluation of tests are not only based on each release, but are refined into each iteration. The quality of the product is checked frequently in the development process, the visibility of the quality is higher and the feedback is more timely.

The quality of the process is more considered in the category of quality management. In the process of the project, quality managers can observe the overall operation of the team and feedback the team's problems from some actual quality phenomena and data, so as to help the team identify risks and adjust the development and testing strategy accordingly.

3. Agile development

In fact, well-run incremental development already has the prototype of agile development, all of which have the following characteristics:

Emphasize short iterations

Continuous testing and continuous integration must be achieved

Be able to respond to frequent demand changes.

So what is agile development? What is its core? As shown in the following figure, compared to "non-agile", agile development emphasizes Continuous Delivery (CD) on the basis of Continues Integration (CI). The core of each iteration is to include release (to the customer's production environment) in a short iteration to meet deliverable quality requirements.

Chengdu Revenue Cloud team has clearly defined this direction from the beginning of the project in 2016, and we need to achieve real Continuous Delivery step by step. German colleagues in charge of Infrastructure have done a lot of work to build a complete framework to support continuous delivery, including continuous integration, construction management, configuration management, release management, we call it DWC (Dev With Confidence). Interested colleagues can consult our group of Andy Ma and Vicky Chen students.

So in this development mode, how can we carry out quality assurance work? The following is my superficial opinion:

First, the goal of the team is to deliver.

Anytime, anywhere, in all forms, in all ways, use it to emphasize that our goal is to deliver. When we say whether a function is completed, it must mean whether the feature works well in the production environment (rather than the local or test environment) and meets defined quality requirements (functionality, performance, security, etc.).

Second, all staff are responsible for quality, and quality assurance activities are part of daily development activities.

When the product only has a long cycle, large version delivery, in our daily work, we tend to put some tasks, especially the quality assurance task to the later stage, and take advantage of the quality debt. If the increments are to be delivered quickly, we have to integrate quality assurance tasks into our daily development activities. Developers, QE, product managers, and all members of the team should carry out corresponding quality assurance activities so that defects have nowhere to hide.

How can it be implemented? That is to define our Quality Strategy to make sure that each who knows exactly when (when), under what environment (where), how to how and what kind of (what) quality assurance activities. It is suggested that the team can have a picture to guide you. This is the Overview Picture of the quality assurance activity of Revenue Cloud Chengdu team (landscape was mosaic by me for security reasons).

Quality Strategy is definitely not immutable, requirements are changing, products are changing, teams are changing, and quality assurance activities should change accordingly. Every time we run, we collect feedback, whether external quality feedback (such as feedback from the product team, defects or requirements reported by customers) or internal quality feedback, such as whether the requirements are clear, whether the test cases are valuable, whether the code quality is good enough, whether the automation ROI (Return on Investment) is acceptable, and so on. Based on these feedback, let's improve the quality strategy.

Third, prevent defects

Testing is a posteriori-based quality assurance method. Another more important a priori method is defect prevention. That is, to prevent defects before the developer submits the test, including:

Before developers implement the code, try to make sure that the requirements are clear, Accept Criteria and self-test points are clear.

In the process of product function implementation, developers, product managers, QE,UX and UA communicate closely to ensure the correctness and comprehensiveness of requirements, implementation and test points. Everyone is sitting in the same office, whether it is Daily Meeting or face-to-face, communication is very easy, the key lies in whether we have this awareness and habit.

In addition to passing all automated regression tests, developers need to press the self-test point to verify the new functionality before the developer code is submitted (from their own branch to the mainline). At this point, we need to think about how to help developers do self-testing better and more effectively. For example, is the self-test site Scope appropriate? Are there some important scenes that are not covered or too many scenes are defined? Do developers need to develop testing thinking or methods? Is there no estimated self-test time in Planning? Did the developer get timely and correct feedback from the product manager / QE?

Fourth, implement strategic automated testing.

When our release cycle is very long, we may feel that automated testing is optional and the role is not so obvious, but as the release cycle becomes shorter and shorter, the importance of automated testing becomes more and more obvious. At Revenue Cloud, in addition to quarterly large releases, we also have shorter cycle feature releases and daily patch releases. It can be said that automated testing is an unshakable foundation. However, there must be many factors to consider in order to realize automated testing. Who will do it? What tool do you choose? Which tests are automated? How to combine automation at all levels? This strategy needs to be decided, tried and improved by the team. After all, what is suitable is the best. But I think there are several principles that have something in common:

Automated testing is by no means a matter for QE alone. Automated testing, like functional implementation, should be the task of the entire team, and like functional backlog, all team members, including QE and developers, can receive the task of automated testing. Test code should also be treated in the same way as functional code, with code review and code maintenance. Do not be reluctant to let experienced personnel participate in automated testing, good and reliable automated testing will eventually benefit the team.

The effectiveness of automated testing is more important than completeness. If the "false failure" and "false pass" of automated testing are too high, it is not only not helpful to the team, but also a kind of interference. To ensure the effectiveness of testing, in addition to ensuring the quality of test scripts, there is also a very important point, do not let go of every fail of automated testing, to analyze clearly the reasons for fail, is to change the implementation of defects at the product implementation level, to change the script if it is a problem of the test script, and to optimize the environment if it is an environmental problem. If the automated test is not analyzed in depth on the grounds that it is unstable, it will never be stable, and the automated test results will never be trusted.

When our team started to do the E2E (End-to-End) automated test, the test was always unstable, but after a period of result monitoring, we gradually summarized and optimized some common problems encountered: for example, there are dependencies or conflicts between test data, the ID of identify UI elements is not unique, assertions are not accurate, and test preconditions are broken by other automatic or manual tests. New adjustments or implementations of UI lead to test failure, and so on. After a period of team efforts, the effectiveness of E2E testing has been greatly improved, and all team members agree with the feedback of automated testing. The process of analysis and optimization can be painful and even make you wonder if the investment is worth it. But persevere, when the effectiveness of automated testing is guaranteed, you will feel the sense of security it brings to you.

Multi-level automated testing should be considered comprehensively. Automated testing is multi-level. In Revenue Cloud, taking functional testing as an example, testing can be divided into Unit Test, Integration Test, Contract Test, and E2E Test. As shown in the following figure:

We should not only avoid weak testing at one level, but also avoid repeated automated testing at multiple levels. Take Chengdu team as an example. In the first one or two release, our requirement is Service Unit Test coverage > 80%, Service Integration Test roughly covers 60% of the API test cases, then E2E GUI Test covers the core business scenarios, and UI's Integration Test is not introduced. Later, with the progress of the project, we found that API Integration Test has the highest input-output ratio. It is closer to the real behavior of service than Unit Test, it is earlier, faster and easier to implement than E2E GUI Test feedback. We have gradually adjusted our strategy to reduce the proportion of Unit Test and increase the coverage of Integration Test. At present, our Integration Test of API covers more than 80% of the test cases.

Later, with the increase of the function of the product, we found that the E2E GUI test was running more and more slowly, so we adjusted the strategy again. One is to introduce the UI Integration Test of OPA5, which completely moved the logic of the pure UI in the original E2E GUI test to the OPA5 test, which greatly shortened the running time of the automated test. Second, it reduces the repeated testing of part and Service Integration Test, so that E2E GUI testing focuses more on the complete end-to-end business scenario, not just on a specific function. Through these two adjustments, multi-level automated testing can be more efficient division of labor and cooperation for product quality protection.

The above three points are important principles for defining automated testing strategies. In addition, I am often asked a question: what automated testing framework / tools do you use in your project? When it comes to multifaceted automated testing, I list the automated testing tools used by Revenue Cloud. For Unit Test, Contract Test, Integration test and other technical platform / language-related tests, the testing tools we have adopted are no "surprises". Junit,Spring Contract Cloud, OPA5 and Rest-Assured are all familiar testing frameworks, which are widely used in SAP projects with similar technical background. Let me focus on the E2E test scheme of Nightwatch + SauceLabs, which may be unfamiliar to all of you.

SauceLabs is a cloud test service platform, which provides VMs to run multiple tests on the cloud, and provides video recording, screenshot and logging functions, which well solves the device problem of multiple automated tests running in parallel. And it supports different browsers and different screen resolutions, which can be applied to browser compatibility testing. Of course, this is a commercial service. The more VM you apply for, the more expensive it will be.

Nightwatch (Night Watch), an open source E2E testing framework implemented using Selenium 2 (webdriver), encapsulates Selenium API to make it easier and simpler to implement test scripts, but it does not support recording UI operations. In fact, in essence, it is no different from Selenium, Ranorex, Start and other tools. Just like quack masters will choose weapons according to their own preferences and kung fu characteristics, we can also choose tools according to the team's technical characteristics and preferences, as well as the budget. However, tools are only tools, just as the decisive factor that determines the outcome of the competition is not weapons, the key factor that determines the success of automation is never the tool, but our own kung fu.

Fifth, the role orientation of QE. The Revenue Cloud Chengdu team, which was founded in 2016, has returned to being full of defects, and there has been a function to submit tests that can't even run Smoke Test (smoke test). At that time, QE was actually very busy, with all kinds of tests to do, all kinds of defects to return to testing, and the product was very nervous before it was released. But up to now, the team is more and more mature, the quality awareness is getting better and better, and the one-time pass rate of backlog submitted by developers is basically maintained at around 80%. During the cross-testing of the entire project, there were fewer and fewer defects given to us by other groups, and the delivery of the team became more and more smoothly, while as QE, I was no longer immersed in basic testing and could have more time to do more valuable things. I also studied the automated testing framework and studied the Performance, Accessibility, GDPR and Fiori Guideline of SAP product standards with the help of the team, and expanded my own technical field.

Therefore, the last thing I particularly want to share with you is the role positioning of QE. QE is not playing the role of a policeman, picking on the opposite of everyone. QE is not the last line of quality and safety defense, standing behind everyone to fill the hole to put out the fire. QE is a comrade-in-arms who fought side by side with everyone. On the one hand, QE acts as a quality coach, guiding and helping the team to improve quality and establish a mature quality culture. On the other hand, like every member of the Agile team, QE needs to continue to learn and grow in the team, not only to strengthen QE skills, but also to enhance the understanding of the business, user behavior, and even specific implementation technologies.

At this point, I believe you have a deeper understanding of "SAP software quality how to ensure the change of work". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.