Shulou(Shulou.com)06/01 Report--

This article is based on Dong Yiquan's speech at Dubbo Community developer Day in Shanghai. Why did Ctrip introduce Dubbo in the first place? In fact, since the end of 2013, Ctrip has mainly used the SOA micro-service framework based on the HTTP protocol. This framework is self-developed within Ctrip, and the overall structure has not been greatly reconstructed in the past 6 years. Due to the limitations of the original design, the expansibility of the framework itself is not very good, which makes it difficult for users to expand some functions themselves. In addition, because of the HTTP protocol, only one request can be processed at a time for a connection. In the case of high concurrency, resources such as the number of connections and thread pools on the server side will be tight, affecting the performance of request processing. As a high-performance RPC framework, Dubbo is not only a well-known open source product in the industry, but also can solve these problems mentioned above because of its excellent architecture design and data transmission. Just in the second half of 2017, Ali announced that it would restart maintenance of Dubbo. For these reasons, our team decided to introduce Dubbo into Ctrip. The first step of Dubbo landing is to implement the new service framework of Dubbo in the company, and the first step is to solve the problems of service governance and monitoring. Service governance in the aspect of service governance, Ctrip's existing SOA framework already has a complete set of service registry and service governance system. For the service registry, you may be more commonly used is Apache Zookeeper. We are using Artemis, a registry developed by ourselves with reference to Netflix's open source Eureka. The architecture of Artemis is a decentralized peer-to-peer cluster. Each node has the same status, and there is no distinction between master and subordinate. The service instance maintains a long connection with any node in the cluster and sends registration and heartbeat information. The node that receives the information will distribute the information to other nodes to ensure the consistency of data between clusters. The client also accepts the list of service instances pushed by the registry through a long connection.

In terms of service data model, we directly reuse the data model of existing SOA services. As shown in the figure, the core service model corresponds to an interface in Dubbo. An application can contain multiple services, and a service can be deployed on multiple servers. We refer to the service application running on each server as the service instance.

All services need to be registered in the governance system before going online. After registration, the system will assign it a unique identity, that is, ServiceID. The ServiceID will be sent to the registry to identify the ownership of the instance when the service instance is registered, and the client will also need this ID to obtain the instance list of the specified service.

Since Dubbo itself does not have the design of ServiceID, the problem here is how to pass the ServiceID information corresponding to an interface to the registry. Our approach is to add a serviceId parameter to the Service and Reference configurations. The implementation of ArtemisServiceRegistry reads this parameter and passes it to the registry. This allows you to interact with the registry normally.

Service monitoring in terms of service monitoring, we have mainly done two parts of work: the monitoring at the statistical data level and the monitoring at the call chain level. Statistics refer to a periodic summary of various service invocation data, such as call volume, response time, request body and response body size, request exception, and so on. This part of the data is summarized at a minute granularity on the client side and the server side respectively, and then output to Dashboard Kanban. At the same time, we also add some tags to the data, such as: Service ID, server-side IP, calling methods, and so on. Users can easily query the monitoring data they need.

On the monitoring service invocation chain, we are using CAT. CAT is a real-time application monitoring platform open source by Meituan Dianping. It can record the whole process of the request through the tree-shaped Transaction and Event nodes. We add Transaction and Event burial sites of CAT on both the client side and the server side of Dubbo, and record the called service, the version of SDK, the time consuming of the service, the identification of the caller and other information, and transmit the context information of the CAT service call to the server side through the Attachment of Dubbo, so that the monitoring data of the client side and the server side can be connected. When troubleshooting, it can be very convenient to inquire. On the figure, what we see on the outside layer is the monitoring data recorded by the client. After expanding at the originator of the call, we can see the corresponding monitoring data on the server side. After the initial release has solved the problems of service governance and monitoring docking, we have completed a preliminary localization of Dubbo in Ctrip, and in March 2018, we released the first available version of the customized version of Dubbo Ctrip. We need to give this product a new name before it is officially released. Since it is Ctrip plus Dubbo, we will call this customized version CDubbo. CDubbo functional extension in addition to the basic system docking, we have also carried out a series of functional extensions to CDubbo, including the following five points: Callback enhancement, serialization extension, circuit breaker and request testing tool. Next, let me introduce you one by one. Callback enhancements first, let's take a look at this code. Is there anything wrong with the code?

There is a DemoService in this code. The parameter of the callbackDemo method is an interface. The callbackDemo method is called in the foo and bar methods in the following Demo class. I believe friends who have used Callback should know that the method foo is called correctly, and the method bar will report an error when it is called repeatedly. Because the client can only create one instance for the same Callback interface. But what's wrong with that? Let's take a look at such a scene.

A user initiates a request for a ticket on the page. After receiving the request, the site server invokes the back-end ticket query service. Considering that this call may take a long time, callback is used on the interface to pass back the actual query results. It is then pushed to the client by the site server through a technology similar to WebSocket. So here comes the question. When the site server receives the callback data, it needs to know which call request of which user it corresponds to, so that it can push the data to the user correctly. However, for globally unique callback interface instances, it is more difficult to get this request context information. You need to be prepared in advance on the interface definition and implementation. You may need to introduce some additional global objects to hold this part of the context information. To solve this problem, we have added Stream function to CDubbo. As before, let's look at the code first.

How is this code different from the previous code? First, the parameters of the callback interface are replaced with a StreamContext. There is also a place to accept the callback is not the previous global unique instance, but an anonymous class, and no longer a single method, but there are three methods, onNext, onError and onCompleted. In this way, the caller can obtain the originally requested context information through the closure in the anonymous class. Isn't it better to experience it? So how exactly is Stream implemented? Let's take a look at this picture.

On the client side, when the client initiates a call with Stream, you need to create a StreamContext through the StreamContext.create method. Although it is created, it is actually a unique StreamID in a global StreamContext and the actual processing logic corresponding to the callback. This StreamID is sent to the server when the request is sent. The server will also bring this StreamID when initiating a callback. In this way, the client can know which StreamContext this callback corresponds to. Some business units of serialization extension Ctrip used the request data model written by Google Protocol Buffer's contract when developing SOA services. The requirement of Google PB is that the data model generated by the contract must be serialized using PB's serializer. To make it easier for them to migrate SOA services to Dubbo, we have also added support for GooglePB serialization in Dubbo. In order to facilitate users to expand themselves, we add an extension interface to the implementation of PB serializer, which allows users to continue to increase the function of data compression on the periphery. The implementation of the overall serializer is not very difficult, but it is important to note that since Dubbo services can only expose one serialization method, this serialization method should be compatible with all Java data types. And PB happens to be the kind of serializer that can only serialize the data types generated by its own contract. So when we encounter unsupported data types, we still fallback to use the default hessian for serialization. Ask for a circuit breaker. I believe you are no stranger to circuit breakers. When a wide range of request errors or timeouts occur on the client or server, the system will automatically execute fail-fast logic and no longer continue to send and accept requests, but directly return error messages. Here we are using a more mature solution in the industry: Netflix's open source Hystrix. It not only includes the function of circuit breaker, but also supports concurrency control, isolation between different calls and other functions. An error in a single call does not affect other calls. All features support custom configuration on demand. The server and client of CDubbo integrate Hystrix to handle the abnormal cases of the request to avoid avalanche effect. Dubbo, a service testing tool, is a RPC protocol that uses binary data streams for transmission, so service testing is a difficult problem. To get testers to test a Dubbo service without writing code, we have to solve three problems: how to write test requests, how to send test requests, and how to view response data. The first is how to construct the request. The problem is actually divided into two parts. One is what format the user uses to construct the request without writing code. Considering that many testers are familiar with Restful Service testing, we finally decided to use JSON format to represent the request data. So isn't it a little difficult for a tester to construct a request from a blank JSON? So we still want to let the user know the requested data model. Although we are using Dubbo 2.5.10, this part of the functionality is already available in Dubbo 2.7.3. So we copied this part of the code over and extended it to save the service's metadata information in a global context. And we added an internal operation, $serviceMeta, in CDubbo through Filter to expose the service's metadata information. This metadata information includes the method list, the parameter list of each method, the data model of the parameters, and so on. In this way, after the user gets the data model by calling the internal operation, a basic JSON structure can be generated. Then users only need to fill the structure with the actual test data to easily construct a test request.

Then, how do you send the edited request to the server? Because there is no model code, the call cannot be made directly. And Dubbo provides a good tool, which is to generalize calls, GenericService. We send the request body to the server through the generalization call, and then serialize the Map returned by the server into JSON and display it to the tester. The whole testing process is complete. By the way, it also solves the problem of how to view the response data.

In order to facilitate users to use, we have developed a service testing platform. Users can directly select services and instances above, write and send test requests. In addition, in order to facilitate users to conduct automated testing, we have also packaged this part of the function as a jar package and released it.

In fact, in the process of doing testing tools, I also encountered a small problem. It is possible to convert Map from JSON to POJO. But as mentioned earlier, there are some objects that are generated through contracts like Google Protobuf. They are not simple POJO and cannot be converted directly. Therefore, we extend the generalization call. First of all, for this custom serializer, we allow users to define their own implementation of format conversion from data objects to JSON. Second, when the server handles generalization calls, we add the ability to convert JSON and Google PB objects to and from Dubbo. Now these two extensions have been merged into Dubbo's code base and released with version 2.7.3. The Fortress Test Gateway has finished testing the service, and sometimes we want to test the service in a production environment, especially when the application is released. In Ctrip, there is a test method called Fortress Test, which means that during the application release process, the release system will first pick a server as the fortress machine and release the new version of the application to the fortress machine. Then the user sends the request to the fortress machine through a specific test method to verify that the function of the new version of the application is working properly. Since the fortress machine has not yet been pulled into the cluster during the fortress test, it is necessary for the client to identify a fortress test request and forward the request to the specified fortress service instance. Although we can achieve this through routing, but this requires the client to know a lot of forwarding details, and the integration of SDK functions will cause some trouble for subsequent upgrades and maintenance. So we developed a service gateway specifically for fortress testing. When a client recognizes that the context of the current request contains the fortress request identity, it forwards the Dubbo request to the preconfigured test gateway. The gateway parses the service request, determines which service it corresponds to, and then finds the fortress machine for the service and forwards the request. After the service completes the request processing, the gateway also forwards the response data back to the caller.

Unlike ordinary HTTP gateways, the service gateway of Dubbo needs to consider an additional request method, which is the callback mentioned earlier. Because the callback is a request initiated from the server, the whole processing process is different from the normal request from the client. The gateway binds the connection initiated by the client and the connection between the gateway and the server, and records the most recent request ID to be returned. In this way, it can be routed accurately when the request and response from callback are received. As of today, a total of 27 versions of CDubbo have been released. Many of Ctrip's business units have been connected to Dubbo. In the future, CDubbo will extend more functions, such as request throttling and authentication authorization, etc. We hope to contribute more new features to the open source community in the future. Author: Dong Yiquan original link this article is the original content of Yunqi community and may not be reproduced without permission.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.