Shulou(Shulou.com)05/31 Report--

This article is about using tunnel protocol to achieve transparent communication between different dubbo clusters. I think it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it with the editor.

Using tunneling protocol to realize transparent communication between different dubbo clusters preface

The author recently completed a very interesting tunneling mechanism (which is already running on the production line), which allows normal communication between dubbo clusters registered to different zookeeper. As shown in the following figure:

For example, in the figure, there are two network-isolated clusters of Ajump B, and the two can only communicate through dedicated lines. But for applications inside, invoking dubbo services from another cluster (such as app1 calling app3) remains exactly the same without any changes. This feature is very useful for scenarios such as new units (computer rooms) and business network isolation.

Scene

This dubbo cluster communication mechanism can be used in the following scenarios.

Build a new computer room

In the process of building a new computer room. Normally, all applications and related facilities of an entire link need to be deployed to the new computer room. As shown in the following figure:

In the author's new mechanism, if the cluster does not have a corresponding interface, it will look for a cluster with a corresponding interface. Even if some systems are missing, the whole computer room can still be work, turning the new computer room into an iterative one. The complexity of the new computer room is greatly reduced.

New business unit

Due to the limitations of the stand-alone machine or some other reasons, some businesses want to be spun off into a separate unit (inside the computer room). But the business does need a lot of basic services from the original unit. And the network between different units can not get through (security requirements).

If you follow the traditional mode, it is necessary to transform the business system, such as establishing a service gateway to communicate with the basic system. This gateway is obviously time-consuming and unskilled, such as forcibly converting the dubbo call into the http call to the business gateway in the business code, as shown below:

Moreover, each additional interface call has to be converted in the service gateway, the corresponding interface package is added, and then released. Such a gateway must be a sinkhole to maintain! With the increasingly stringent security requirements, the requirements of network isolation between different services will increase day by day.

The author is engaged in middleware and firmly believes that the basic service can be transparent to the business and make it imperceptible is a good design. Once you need a lot of business to cooperate with this transformation caused by infrastructure changes, it is undoubtedly very unfriendly, or even a failed design.

Fault isolation

In fact, the original intention of the author to engage in this set of tunnel mechanism is fault isolation. For example, the author encountered the risk of writing a lot of data to zookeeper several times because the business system used zookeeper improperly, thus making the whole cluster unavailable. The new mechanism allows different businesses to register with different zookeeper,zookeeper to hang up, but this business is down, and other businesses are not affected.

In fact, not only for zookeeper, but also because the author has done a similar tunneling mechanism for messages (such as activemq). So that our whole business can be better fault isolation!

Tunnel mechanism

The biggest convenience of the author's mechanism is that it is less intrusive to the business. The application of the basic cluster does not even need to be modified at all. In order to meet this requirement, the author introduces the concept of tunnel (Tunnel), which is very commonly used on the network, which we may have come into contact with, VPN/Vxlan, these network protocols all use tunneling.

Tunnel penetration

Let's first take a look at the most basic principle: when system A calls system B through Dubbo, the dubbo protocol is used in the same cluster. When crossing the cluster, the author carries the dubbo raw bitstream on the http protocol and sends it out on the dedicated line.

Since from the point of view of system B, all the received byte streams are the same, it is impossible (and needless) to tell whether it is a dedicated line or a direct call. So system B does not need to change any code.

Tunnel implementation

So, how exactly is this tunnel implemented, and how does system A know that there is no corresponding interface for this cluster and that it needs to be called to another cluster through the http tunnel? This introduces our tunnel gateway.

The concept here is also similar to the default gateway on the network. If the corresponding recipient cannot be found in this cluster, it will be delivered to a default gateway, and this tunnel gateway will deliver the call for us.

How to discover this gateway

In order to make full use of the registration discovery mechanism of the dubbo interface, the tunnel gateway is also exposed as a dubbo interface, and its inputs and outputs are as follows:

/ / Tunnel gateway interface request body class TunnelInterfaceReq {/ / dubbo meta-information, such as specific call interface information MetaData dubbo / / original request A calls serialized bitstream byte [] body } / / the tunnel gateway interface return body class TunnelInterfaceResp {/ / dubbo meta-information MetaData dubbo / / the returned value calls the serialized bitstream, and the corresponding system of another cluster returns byte [] resp;}

With this dubbo interface, we can easily transfer data to the default gateway.

Note that there is actually a layer of tunnel protocol here, that is, the dubbo protocol is used to carry the dubbo protocol, which is similar to the nesting method to effectively make use of the registration discovery mechanism of dubbo itself.

Gateways communicate with each other through http

Because of the communication between different clusters through dedicated lines, the author uses http communication. After the App1 request reaches the tunnel gateway, the gateway takes the original body bitstream out of the TunnelInterfaceRequest. It is then passed on to a request from http. As shown in the following figure:

It is worth noting that because it is a byte stream and does not carry any business information (such as type information, etc.), our tunnel gateway can tunnel any dubbo request, unlike traditional gateways that need to add jar packets corresponding to various services and keep publishing-_ -!

In the figure, after it is delivered to the tunnel gateway on the other end, it takes the call meta-information and the original call byte flow from the http protocol, and finds the App2 through the call meta-information. The byte stream is then replayed to App2 so that the dubbo call can be made. In fact, the byte flow that App2 sees from the tunnel gateway is exactly the same as the byte flow called from other machines in the cluster. As shown in the following figure:

The return value is also through the tunneling mechanism.

Obviously, our return value also needs to go through the tunneling mechanism. Like Request, it uses the tunnel protocol twice, as shown in the following figure:

So what App1 really receives is Tunnel Response, how to make it transparently receive the original response bitstream? This requires the caller to access the lightweight jar package developed by the author (in fact, this jar package is also needed for the tunnel of request at the beginning)

Extend dubbo

Because dubbo has an excellent filter mechanism, it can be extended in a variety of places. For this tunnel mechanism, the author extends the invoke calling logic. As shown in the following figure:

As long as the introduction of the author's jar package, it is very easy to automatically expand, except for pom.xml plus two lines, other business code does not need to be modified at all.

Interface Discovery of Tunnel Gateway

So how does tunnel gateway A know that the interface is in cluster B and deliver it to tunnel gateway B? Obviously, we need a trunking communication mechanism between tunnel gateways.

For example, the tunnel gateway asks other different tunnel gateways whether there is this interface and caches it according to a certain policy.

Discovery of dubbo Cluster

The final question is how the tunnel gateway knows about other dubbo clusters, because the number of clusters is small and does not change often relative to the number of dubbo interfaces. We just need to find a place to simply record it, such as putting it in the database. Then, because it is called by http, you can do load balancing by resolving the domain name directly through DNS.

Performance

Because the serialization and deserialization of this mechanism are entirely on the Provider/ consumers side, there is no pressure on the gateway at all, so the CPU consumption of the gateway is very low. In a single call delay, due to two more hops, there will inevitably be some loss, about more 2ms per interface.

The above is then using the tunnel protocol to achieve transparent communication between different dubbo clusters. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.