Shulou(Shulou.com)06/03 Report--

Microservice architecture-full link design for automated testing

Tag: microServices autoTest mock unitTest testTrace

Background neglected software engineering link-testing complexity and efficiency under DEVTESTOPS microservice architecture development phase unitTest mock external dependency continuous modulation phase mock external dependence automation test phase mock requirements autoTest Mock Gateway surfaced lightweight version implementation of overall logical architecture including mock parameter into service framework standard request contract preliminary implementation of Mock summary background using AOP + RestEasy HttpClientRequest SPI

From the SOA architecture to the popular micro-service architecture, the system is getting smaller and smaller, and the complexity of the overall architecture is also rising linearly. The technical difficulties and solutions under the micro-service architecture that we have been talking about are becoming more and more mature (including typical data consistency, consistency problems caused by system calls, or consistency problems caused by cross-node and cross-computer room replication.) But there is one link that we obviously overlooked.

Under the current trend of micro-service architecture, micro-services are basically perfect in terms of operation and maintenance and automatic deployment. From my personal experience, the great changes brought about by upper-level development and testing to the micro-service architecture are still responding and learning.

At the development level, the discussion of micro-services is more about framework, governance, performance and so on, but from the perspective of complete software engineering, we seriously lack the knowledge of analysis and design, which is also the technology that our engineers generally lack.

We often find that once you want to reconstruct something, it is because there is a serious lack of overall analysis and design in the initial construction of the building, which eventually leads to the gradual ossification of the building and the fear of everyone. Because he gradually became a monster. (for example, developers rarely write unitTest, and we always ignore the value of software engineering behind unit testing. )

Neglected Software Engineering Link-DEVTESTOPS

Have we found a phenomenon that testing is easy to be ignored in the whole software process? Any software engineering model has a QA link, but this link seems to be very thin and weak. at present, most of our engineers and architects seriously underestimate the power and value of this link, and remain in the impression of no technical content and low efficiency of manual functional testing.

This is mainly to test the role of the entire technical system, the engineering ability is weak, some are objective environmental problems, and some of their own problems, do not let themselves go out, to learn more about the entire engineering technology, to learn more about the development technology, the physical structure of production, which will help to test and amplify their own voice.

Another main reason leading to the weak design innovation of the testing link in China is that the development engineers generally do not have a complete engineering foundation. In foreign IT developed countries, Japan, the United States and so on, a qualified development engineer and test engineer all have fuzzy boundaries and develop their own products to test themselves, which requires switching the mode of thinking and having these two abilities at the same time, but this is the complete process of the whole software engineering.

Have we ever thought about a question: why are we talking about DevOps instead of DevTestOps? why do we skip testing? do we not need testability if the developed system needs to have good operation and maintainability? development needs to have operation and maintenance capabilities, operation and maintenance needs to have development capabilities, why the testing link is ignored.

Our contempt for the QA link and our lack of attention to the testing role actually have a lot of side effects.

Test complexity and efficiency in Micro-Service Architecture

The split granularity of micro-service is much smaller than that of SOA. It is very convenient to measure it with automatic deployment of portable images, but it will add a lot of complexity and efficiency to the whole development and testing process.

In the era of SOA, the principle of contract-driven is also applicable in micro-services, and you can develop and go online after defining contracts across departmental requirements. But the biggest problem in this is the partial tuning problem and automatic regression problem of the current system. If the new system is launched, it still needs to do performance stress testing, how to solve this external dependence.

Maybe we will say, shouldn't we rely on Fang to ready first, and then we will test and release it immediately? If the business is reasonable and the structure is reasonable, the biggest problem in this scenario is that our project is easily constrained by relying parties, which will bring a lot of problems. For example, R & D personnel need to switch out to do other things, and branch has been hanging all the time. I don't know when I suddenly come to you and say that it can be docked. Maybe it has been a month or more since this approach has formed a habitual R & D process, it is easy to generate online BUG.

Another reasonable situation is that the platform provider needs to call the business side's interface, which includes the callback interface that is generally called, the marketing interface on the transaction link, the routing interface for distribution, and so on.

Here to share our ongoing marketing-cloud (Marketing Cloud) rules engine project.

Marketing-cloud provides some marketing services, such as group buying, coupons, promotions, etc., but our business side needs to have its own personalized marketing activities, we need to abstract the return information of business side marketing activities in the marketing-cloud rules engine, and at the same time open up personalized marketing activities and public transactions, settlement links to form a complete business flow.

This is a marketing-cloud logical architecture diagram, and related to our theme is the marketing rules engine, which is what we call a reasonable business scenario here.

In the whole process of issuing an order, the marketing rules engine has to shoulder all kinds of marketing games that not only provide common marketing activities within the marketing-cloud, but also bridge external marketing centers. There will be many external marketing centers, and at present we have two main ones.

Since this article is not about how to design a marketing platform, I don't intend to expand the topic here. Mainly for the purpose of throwing a brick to attract jade, platform-based business will have a variety of external system-dependent business scenarios. The next part of the article will expand the practice of the marketing-cloud rule engine in getting through the test link.

Development phase unitTest mock external dependencies

In the development phase, we will often write unit tests to test our logic. When writing unitTest, we all need dependencies around mock. The objects from mock are divided into two types, one is the stub pile object without Assert logic, and the other is the mocker simulation object that needs to support Assert.

But we do not need to distinguish them obviously, the difference between the two is not so obvious, it may need to be distinguished within the coding specification.

What we care about is how to solve the problem of dependency between objects. Various mock frameworks actually provide a lot of very useful tools, and we can easily mock peripheral dependencies.

Given (marketingService.mixMarketingActivity (anyObject () .willReturn (stubResponse); RuleCalculateResponse response = this.ruleCalculatorBiz.ruleCalculate (request)

Here we mock the marketingService.mixMarketingActivity () method.

There are many useful mock frameworks in the Java world, and one of the more popular and useful frameworks, mockito, can easily rely on the mock Service layer. Of course, in addition to mockito, there are many excellent mock frameworks.

These frameworks are more or less the same, and the biggest problem in writing unitTest is how to ReFactor the logic to make it easier to test, that is, whether the code is well testable, whether it has eliminated the vast majority of private methods, and whether private methods have some accusations that we did not capture business concepts.

External dependence of mock in continuous modulation phase

After we have completed all the development, the sound unit testing ensures that there is no problem with our internal logic (of course, the design of unitTest's case is not discussed here).

Now we need to continuously tune the development of the peripheral system, which still belongs to other supporting systems such as this platform. For example, the relationship between our marketing-cloud rule engine system and the order issuing system. At the time of development, we successfully completed the verification of the development solution when we wrote unitTest, but now we are faced with the problem of continuous tuning.

The system needs to run formally, but we lack dependence on external marketing centers. What should we do? In fact, we also need to rely on mock externally in the tuning phase, but the technology and methods of this mock are not supported by the unitTest framework, but we need to design the development architecture of our entire service.

First of all, to be able to recognize that this request requires mock, then you need some kind of mock parameter parameter to provide recognition capability.

Let's take a look at a preliminary attempt of the marketing-cloud marketing rules engine in this area.

Public interface CCMarketingCentralFacade {CallResponse callMarketingCentral (CallRequest request);} public interface ClassMarketingCentralFacade {CallResponse callMarketingCentral (CallRequest request);}

The marketing rules engine uses RestEasy client api as the rest invocation framework. These two Facade are the Facade that the marketing platform invokes to the marketing center of the two subsidiaries of CCTalk and Hujiang Network School.

In order to restore the practical information of our engineering practice while eliminating some sensitive information, I deleted some code that did not affect reading and had nothing to do with this article for all the code examples in the whole article. At the same time, I did some pseudo-coding and omission to make the code more concise and easier to read.)

Under normal logic, we will decide which company's marketing center interface to call according to the marketing route key, but since the business side does not have an existing address for us to connect with when we develop this project, we have done mock facade to solve the problem of continuous tuning.

Public class CCMarketingCentralFacadeMocker implements CCMarketingCentralFacade {@ Override public CallResponse callMarketingCentral (CallRequest request) {CallResponse response =... MarketingResultDto marketingResultDto =... MarketingResultDto.setTotalDiscount (new BigDecimal ("90.19"); marketingResultDto.setUseTotalDiscount (true); response.getData () .setMarketingResult (marketingResultDto); return response;}} public class ClassMarketingCentralFacadeMocker implements ClassMarketingCentralFacade {@ Override public CallResponse callMarketingCentral (CallRequest request) {CallResponse response =.. MarketingResultDto marketingResultDto =... MarketingResultDto.setUseCoupon (true); marketingResultDto.setTotalDiscount (null); marketingResultDto.setUseTotalDiscount (false); List discountDtos =. Request.getMarketingProductTagsParameter (). GetMarketingTags (). ForEach (item-> {MarketingProductDiscountDto discountDto =. DiscountDto.setProductId (item.getProductID ());... DiscountDtos.add (discountDto);}; Return response;}}

We define two mock classes, both of which are test data, to solve the problem in the continuous debugging phase, that is, the dependency problem in the DEV environment.

Once you have mock facade, you need request to define the mock parameter parameter.

Public abstract class BaseRequest implements Serializable {public MockParameter mockParameter;} public class MockParameter {/ * mock cc Marketing call API * / public Boolean mockCCMarketingInterface; / * mock class Marketing call API * / public Boolean mockClassMarketingInterface; / * * whether to automatically test mock * / public Boolean useAutoTestMock; / * Test mock parameters * / public String testMockParam;}

Let's ignore the general-purpose design for the time being, this is just an iterative attempt to catch up with the project, and then consider refactoring the extraction framework when we have run through the whole process.

With the input parameters, we can dynamically inject mock facade based on the parameter judgment.

Mock requirements in automated testing phase

We moved on and entered the testing phase immediately after the continuous tuning phase, and now most Internet companies are basically automated testing, rarely with manual, especially back-end systems.

So one of the problems we face in the autoTest phase is that we need a public autoTest address, and this test address is the same. Under our automation test, the facade bean address of mock is this address, and the output value of this address needs to be able to correspond to the context of each execution of the automation script.

We have many micro-service systems to form a platform, and each service has a dependent third-party interface. Originally, when automating testing these services, you need to understand the business side system interface, DB, foreground entry, etc., because when writing automation scripts, you need to create test data synchronously before you can Assert.

This inter-departmental communication and collaboration is seriously inefficient, and personnel changes, system changes will directly affect the launch cycle, here is definitely worthy of innovation to solve this problem of serious efficiency congestion.

@ Value ("${marketing.cloud.business.access.url.mock}") private String mockUrl;/** * Automated Test mocker bean * / @ Bean ("CCMarketingCentralFacadeTestMock") public CCMarketingCentralFacade CCMarketingCentralFacadeTestMock () {RestClientProxyFactoryBean restClientProxyFactoryBean.. RestClientProxyFactoryBean.setBaseUri (this.mockUrl);.} / * Automated Test mocker bean * / @ Bean ("ClassMarketingCentralFacadeTestMock") public ClassMarketingCentralFacade ClassMarketingCentralFacadeTestMock () {RestClientProxyFactoryBean restClientProxyFactoryBean. RestClientProxyFactoryBean.setBaseUri (this.mockUrl);...}

The mockUrl here is the unified autoTest address we abstracted. There is a parameter of type useAutoTestMock Boolean in the previous mock parameter. If the current request parameter is true, we will dynamically inject the automated test mock bean, and all subsequent calls will go to the place specified by mockUrl.

AutoTest Mock Gateway surfaced

So far, we have encountered the need for automated testing of a unified mock address to close all microservices in this area. The biggest problem now is that all microservices depend on different response, and the pre-created response for automated scripts to be executed should be adapted to the context of the current test.

For example, the marketing rules engine, our automated script needs to pre-construct the current product (for example, productID:101010) when creating an order, and obtain the response of the activity information and deduction information provided by the external marketing center. Finally, we can go to Assert whether the order amount and activity information record is correct. This is an autoTest context.

There are two ways to identify the current autoTest context, one is to determine the commodity ID when the case is executed, and finally to obtain the response of the mock through the commodity ID. Another option is to support passing the autoTest mock parameter to the service specified by mockUrl, which can be used to identify the current test context.

A test case may go through many microservices, and all of these dependent services may require a preset mock response, which is basically once and for all.

Therefore, we abstract autoTest Mock Gateway (Automated Test mock Gateway Service), and there is still a lot of work to be supported in the whole automated testing process, such as authentication between services, mock of authentication key, encryption and decryption, mock of encryption and decryption key, alternate parallel execution of automated test case, and so on.

As engineers, we all want to solve the overall problem in a systematic and engineering way, rather than individual point problems. With this mock gateway, we can do a lot of things, and we can benefit all other departments that need it.

Construct mock response in an autoTest context, and then dynamically identify specific source services through mock parameter for routing, authentication, encryption and decryption and other operations.

MockGateway is a fulcrum, I believe this fulcrum can pry a lot of test space and innovation ability.

Lightweight version implementation

Next we will show the preliminary attempt in the marketing-cloud marketing rules engine.

Overall logical architecture

Each time you run a case, the automation script creates an autoTestContext corresponding to the current case, which is full of meta data, which is used to indicate which of all the microservice systems involved in this case need to go to mock gateway.

All configurations in mockGateway have an autoTestContext, if there is no autoTestContext description, it is shared by all case.

Incorporate mock parameter into the service framework standard request contract

To get through all the channels in the entire microservice architecture, you need to define mockParameter in the standard request contract, which is the premise of all this.

The calls between services and services follow the standard micro-service request contract, and the dependence between services and external systems can choose HTTP Header or standard request, depending on whether our entire service framework has covered all production lines and some legacy system problems.

Public abstract class BaseRequest implements Serializable {public MockParameter mockParameter;}

BaseRequest is the base class for all request, so that all requests can be delivered properly.

Preliminary implementation of Mock using AOP + RestEasy HttpClientRequest SPI

The hierarchical dependence of the development architecture of the whole system is: facade- > biz- > service. All the basic core logic is in service, and the requested request dto cannot cross the boundary to the service layer at most. According to the specification, request dto is stuck in the biz layer at most, but in the Internet world, some of them can be iterated quickly, not so rigidly, and timely reconstruction is the main method to repay the technical debt.

As we mentioned earlier, the RPC framework we adopted is RestEasy + RestEasy client. Let's take a look at the entrance first.

@ Component@Path ("v1/calculator/") public class RuleCalculatorFacadeImpl extends BaseFacade implements RuleCalculatorFacade {@ MockFacade (Setting = MockFacade.SETTING_REQUEST_MOCK_PARAMETER) public RuleCalculateResponse ruleCalculate (RuleCalculateRequest request) {...}}

Take a look at the service object again.

Componentpublic class MarketingServiceImpl extends MarketingBaseService implements MarketingService {@ MockFacade (Setting = MockFacade.SETTING_FACADE_MOCK_BEAN) public MarketingResult onlyExtendMarketingActivity (Marketing..Parameter tagsParameter) {.}

Let's focus on the @ MockFacade annotation statement.

@ Target ({ElementType.METHOD}) @ Retention (RetentionPolicy.RUNTIME) public @ interface MockFacade {String SETTING_REQUEST_MOCK_PARAMETER = "setting_request_mock_parameter"; String SETTING_FACADE_MOCK_BEAN = "setting_facade_mock_bean"; String Setting ();}

Our main purpose with this annotation is to put the mockParameter into the ThreadLocal and clean up when the request is processed. Another function is mock bean processing in the service layer.

@ Aspect@Component@Slf4jpublic class MockMarketingFacadeInterceptor {@ Before ("@ annotation (mockFacade)") public void beforeMethod (JoinPoint joinPoint, MockFacade mockFacade) {String settingName = mockFacade.Setting (); if (MockFacade.SETTING_REQUEST_MOCK_PARAMETER.equals (settingName)) {Object [] args = joinPoint.getArgs (); if (args = = null) return; List argList = Arrays.asList (args) ArgList.forEach (item-> {if (item instanceof BaseRequest) {BaseRequest request = (BaseRequest) item; if (request.getMockParameter ()! = null) {MarketingBaseService.mockParameterThreadLocal.set (request.getMockParameter () Log.info ("- setting mock parameter: {}", JSON.toJSONString (request.getMockParameter ());});} else if (MockFacade.SETTING_FACADE_MOCK_BEAN.equals (settingName)) {MarketingBaseService marketingBaseService = (MarketingBaseService) joinPoint.getThis (); marketingBaseService.mockBean () Log.info ("- setting mock bean.");} @ After ("@ annotation (mockFacade)") public void afterMethod (JoinPoint joinpoint, MockFacade mockFacade) {if (MockFacade.SETTING_FACADE_MOCK_BEAN.equals (mockFacade.Setting () {MarketingBaseService marketingBaseService = (MarketingBaseService) joinpoint.getThis (); marketingBaseService.mockRemove () Log.info ("- remove mock bean.");} if (MockFacade.SETTING_REQUEST_MOCK_PARAMETER.equals (mockFacade.Setting () {MarketingBaseService.mockParameterThreadLocal.remove (); log.info ("- remove ThreadLocal. ThreadLocal get {} ", MarketingBaseService.mockParameterThreadLocal.get ();}

This logic is based entirely on a convention, that is, MarketingBaseService, which is not generic, but will eventually be a plugin framework in step-by-step refactoring and extraction.

Public abstract class MarketingBaseService extends BaseService {protected ClassMarketingCentralFacade classMarketingCentralFacade; protected CCMarketingCentralFacade ccMarketingCentralFacade; public static ThreadLocal mockParameterThreadLocal = new ThreadLocal (); public void mockBean () {MockParameter mockParameter = mockParameterThreadLocal.get (); if (mockParameter! = null & & mockParameter.mockClassMarketingInterface) {if (mockParameter.useAutoTestingMock) {this.setClassMarketingCentralFacade (SpringContextHolder.getBean ("ClassMarketingCentralFacadeTestMock", ClassMarketingCentralFacade.class)) } else {this.setClassMarketingCentralFacade (SpringContextHolder.getBean ("ClassMarketingCentralFacadeMocker", ClassMarketingCentralFacadeMocker.class);}} else {this.setClassMarketingCentralFacade (SpringContextHolder.getBean ("ClassMarketingCentralFacade", ClassMarketingCentralFacade.class)) } if (mockParameter! = null & & mockParameter.mockCCMarketingInterface) {if (mockParameter.useAutoTestingMock) {this.setCcMarketingCentralFacade (SpringContextHolder.getBean ("CCMarketingCentralFacadeTestMock", CCMarketingCentralFacade.class);} else {this.setCcMarketingCentralFacade (SpringContextHolder.getBean ("CCMarketingCentralFacadeMocker", CCMarketingCentralFacadeMocker.class)) } else {this.setCcMarketingCentralFacade (SpringContextHolder.getBean ("CCMarketingCentralFacade", CCMarketingCentralFacade.class));}} public void mockRemove () {mockParameterThreadLocal.remove ();}}

We can smoothly put the mockParameter in request into ThreadLocal, and we can dynamically inject the corresponding mockerBean through AOP.

Now all we have to deal with is the call to mockGateway to put the tag string in autoContext in _ mockParameter into HTTP Header.

Componentpublic class MockHttpHeadSetting implements ClientRequestFilter {@ Override public void filter (ClientRequestContext requestContext) throws IOException {MultivaluedMap header = requestContext.getHeaders (); MockParameter mockParameter = MarketingBaseService.mockParameterThreadLocal.get (); if (mockParameter! = null & & StringUtils.isNotBlank (mockParameter.getTestingMockParam () {header.add ("Mock-parameter", mockParameter.getTestingMockParam ());}

Then configure it in the SPI (javax.ws.rs.ext.Providers) file

Com.hujiang.marketingcloud.ruleengine.service.MockHttpHeadSetting summary

In the whole practice of micro-service architecture, what has not been discussed in the engineering field is the testing of micro-service architecture, and what is closer to us is automated testing, because automated testing is basically needed by all systems.

However, there is one piece that we have not paid attention to, which is the full-link stress testing. The real stress testing of the full-link in production needs to solve a lot of problems, and the more important one is DB. All the transaction data generated during the stress testing can not participate in the settlement and financial process, which needs to be solved with the help of shadow tables, and all the data will not be written into the final real transaction data. Of course, there are other places that need to be solved. Once you turn on the full-link voltage test switch, you should need to deal with all the places where the data is generated. This is a huge project, but it will also be very interesting.

This article is only a preliminary attempt in this area, we will continue to expand, in the next production line full-link test, we can use the current practical architecture to expand.

Author: Wang Qingpei (Senior JAVA architect, Hujiang Group)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.