Shulou(Shulou.com)06/01 Report--

Dubbo 3.0 commonly used protocol comparison and what is the new form of RPC protocol, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

The protocol is the foundation of RPC. In what format the data is transmitted on the connection, how does the server determine the size of the request received, whether there can be multiple requests on the same connection at the same time, and how to respond if something goes wrong. These are all problems that need to be solved by agreement.

By definition, protocols define how data is transmitted between networks by defining rules, formats, and semantics. RPC requires that both sides of the communication be able to recognize the same protocol. Data is transmitted in the way of bit stream on the network, if the peer of the local protocol is not identified, the peer can not get useful information from the request, and the upper business requirements can not be realized.

A simple protocol needs to define the data exchange format, protocol format and request method.

The data interchange format is also called a serialization format in RPC. Commonly used serialization includes JSON/Protobuf/Hessian, etc. Serialization is generally evaluated from three dimensions:

Serialized byte array size

Speed of serialization and deserialization

Readability after serialization

When choosing the serialization mode, the protocol chooses each other among the three dimensions according to the specific requirements. The smaller the serialized array, the more network traffic is saved, but the serialization process may be more time-consuming. Text-based serialization methods such as JSON / XML are often easier for developers to accept because text is easier to understand and easier to identify in all layers of devices than a continuous byte array, but the result of improved readability is significant performance degradation.

The protocol format is closely related to the RPC framework, and there are two kinds of protocols according to their functions. One is a compact protocol, which only provides simple unit data and data content for invocation. The other is composite protocols, which carry framework-level metadata to provide functional enhancements. One representative of this type of protocol is RSocket.

The request method is closely related to the protocol format. The common request formats are synchronous Request/Response and asynchronous Request/Response. The difference is whether the client needs to synchronously wait for the response to return after sending a request. If you do not need to wait for a response, there can be multiple outstanding requests on a link at the same time, which is also called multiplexing. Another request model is Streaming. There are multiple RPC in a complete business call, and a part of the data is transferred each time, which is suitable for streaming data transmission.

With these three basic conventions, a simple RPC protocol can be implemented.

One of the core contents of Dubbo3 is to define the next generation RPC protocol. In addition to the basic communication functions, the new protocol should also have the following characteristics:

Unified cross-language binary format

Support Streaming and application layer full duplex invocation model

Easy to expand

Can be identified by devices on all layers

Here we compare some commonly used protocols to explore the form of new protocols.

HTTP/1.1

HTTP/1.1 should be the most widely used protocol, and its simple and clear syntax, cross-language and support for native mobile make it in fact the most widely accepted RPC solution.

However, from the demand of RPC protocol, HTTP1.1 mainly has the following problems

Head-of-line blocking (HOL) results in poor performance on a single connection. Although pipeline is supported, the response is returned sequentially.

Text-based protocols repeatedly carry a lot of complicated and useless header information in each request, wasting bandwidth and affecting performance.

The pure Request/Response request model cannot implement Server Push and can only rely on client polling. Similarly, the full duplex of Streaming is also insecure.

RESP

RESP is the communication protocol used by Redis, and its concise and easy-to-understand format also contributes to the rapid development of Redis language clients. But this HTTP/1.1-like protocol also has the same performance problems.

The serialization expression ability is weak, and it usually needs to be assisted by other serialization methods. however, the protocol does not support setting a specific serialization mode, so we can only rely on the client convention.

There is also the problem of head-of-line blocking, and pipeline cannot fundamentally solve the performance problem of single connection.

Pub/Sub also has quantitative bottlenecks in the case of a single connection.

Dubbo2.0

Dubbo2.0 protocol is defined directly on the TCP transport layer protocol, which provides the greatest flexibility for the definition of protocol functions. But at the same time, because of this obvious flexibility advantage, RPC protocols are generally customized private protocols.

The Dubbo protocol body Body has an extensible attachments section, which makes it possible to pass additional properties in addition to the RPC method, which is a good design. However, similar Header parts lack similar extensible attachments, which can be divided into Body Attachments and Header Attachments responsibilities by referring to the Ascii Header design defined by HTTP.

Some RPC request locators in the Body protocol body, such as Service Name, Method Name, Version, etc., can be mentioned in Header and decoupled from specific serialization protocols to be better identified by the network infrastructure or used for traffic control

Scalability is not good enough, lack of protocol upgrade design, such as no reserved status identification bits in the Header header, or special packet designed for protocol upgrade or negotiation in HTTP.

In the Java version of the code implementation, it is not concise and general-purpose. For example, in the link transmission, there are some language-bound content; there is redundant content in the message body, such as Service Name in both Body and Attachments

HTTP/2.0

HTTP/2.0 retains all the semantics of HTTP/1, while maintaining compatibility, it has made great improvements in communication model and transmission efficiency, mainly to solve the problems in HTTP/1.

Multiplexing on a single link is supported. Compared with Request-Response exclusive links, link utilization based on Frame is more efficient. StreamId provides context state, and client can support out-of-order Response returns according to StreamId.

Header compression HPACK, based on static table and dynamic table to achieve Header cache, reduce the amount of data transmission

Request-Stream semantics, native support for Server Push and Stream data transfer

Binary Frame, binary framing, can handle Header and Data separately

Although HTTP/2.0 overcomes the above problems, there are also some controversial points, such as the necessity of flow control in the upper layer of TCP and whether it is too complicated to be compatible with HTTP semantics through HPACK.

GRPC

Compared with some frameworks that build the application layer protocol on the bare TCP, gRPC chooses HTTP/2.0 as the transport layer protocol. The upper layer protocol function is realized by defining the Header content and Payload format. Here are some of gRPC's design ideas.

Coverage & Simplicity, protocol design and framework implementation should be universal and simple enough to run on any device, even some resources first, such as IoT, Mobile, etc.

Interoperability & Reach, to be built on a more generic protocol, the protocol itself must be supported by almost all the infrastructure on the network

General Purpose & Performant, in order to strike a good balance between scenarios and performance, first of all, the protocol itself should be suitable for various scenarios, and at the same time, it should also have high performance as far as possible.

Payload Agnostic, the load transferred on the protocol should remain language and platform neutral.

Streaming, to support Request-Response, Request-Stream, Bi-Steam and other communication models

Flow Control, the protocol itself has the ability to perceive and restrict traffic.

Metadata Exchange, which provides additional data transfer capabilities beyond the RPC service definition

Under the guidance of this design concept, gRPC is finally designed as a cross-language, cross-platform, general protocol. The functions are basically fully available or can be easily extended to the required new functions. However, we know that there is no silver bullet in software engineering, and the ultimate performance of gRPC is definitely worse than that of naked TCP proprietary protocols. But for most applications, compared with the HTTP/1.1 protocol, gRPC/HTTP2 has made great progress in performance while taking into account the readability.

In serialization, gRPC is designed to keep payload neutral, but actual cross-language scenarios require a strongly standardized interface definition language to ensure consistent serialization results. In the official implementation of gRPC, protobuf and json are used to support performance scenarios and development efficiency scenarios, respectively. From the choice of serialization mode to the comparison of various dimensions of the protocol, extending a new protocol based on gRPC is the best choice.

Dubbo3.0

Dubbo3.0 's protocol is based on gRPC and provides extensions in the application layer, exception handling, protocol layer load balancing support and Reactive support. There are three main goals:

In the distributed large-scale cluster scenario, provide better load balancing to achieve higher performance and ensure stability.

Support tracing/monitoring and other distributed standard extensions, support micro-service standardization and smooth migration

Reactive semantics are enhanced at the protocol layer to provide distributed back-pressure capabilities and better Streaming support.

In addition to protocol layer support, the new Dubbo3.0 protocol also includes ease of use support, including support for both IDL compiler and Annotation Compiler. The client will more fully support native asynchronous callbacks, Future asynchronous and synchronous calls. The server will use a non-reflective call. This will significantly improve client and server performance. From the perspective of user migration, the Dubbo framework will provide smooth protocol upgrade support, and strive to double the performance with as little modification of the code or configuration as possible.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.