Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to use Dubbo to develop gRPC services". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's ideas to study and learn "how to use Dubbo to develop gRPC services".

Basic introduction

Dubbo protocol

At the protocol level, the following are the Dubbo protocols currently supported in version 2.7

As we all know, Dubbo protocol is directly defined on top of TCP transport layer protocol. Because of the high reliability and full duplex of TCP, it provides the greatest flexibility for the definition of Dubbo protocol. But at the same time, because of this flexibility, RPC protocol is generally a customized private protocol, and Dubbo is also faced with this problem. Here we focus on the areas where Dubbo should be improved in terms of the generality of the agreement. For detailed analysis of the protocol, please refer to the official website blog.

The Dubbo protocol body Body has an extensible attachments section, which makes it possible to pass additional properties in addition to the RPC method, which is a good design. However, similar Header parts lack similar extensible attachments, which can be divided into Body Attachments and Header Attachments responsibilities by referring to the Ascii Header design defined by HTTP.

Some RPC request locators in the Body protocol, such as Service Name, Method Name, Version, etc., can be mentioned in Header and decoupled from specific serialization protocols to be better identified by the network infrastructure or used for traffic control.

Scalability is not good enough, lack of protocol upgrade design, such as there is no reserved status identification bit in the Header header, or HTTP has a special packet designed for protocol upgrade or negotiation.

In the Java version of the code implementation, it is not concise and general-purpose. For example, in the link transmission, there are some language-bound content; there is redundant content in the message body, such as Service Name in both Body and Attachments.

HTTP/1

Compared with directly building the private RPC protocol with the TPC transport layer, the remote call solution built on HTTP will have better versatility, such as WebServices or REST architecture, using HTTP + JSON can be said to be a de facto standard solution.

All the options are built on top of HTTP, and I think there are two biggest advantages:

The semantic and extensible performance of HTTP meets the requirements of RPC calls.

Versatility, HTTP protocol is supported by almost all devices on the network, and has a good protocol penetration.

Specifically, the advantages and limitations of HTTP/1 are:

In a typical Request-Response model, there can be only one waiting Request request on a link at a time.

HTTP/1 supports Keep-Alive links, avoiding the overhead of repeated link creation.

Human Readable Headers, using a header transport format that is more general and easier for humans to read

Without direct Server Push support, flexible modes such as Polling Long-Polling need to be used.

HTTP/2

HTTP/2 retains all the semantics of HTTP/1 and makes great improvements in communication model and transmission efficiency while maintaining compatibility.

Support Multiplexing on a single link, and use the link more efficiently based on Frame than the Request-Response exclusive link.

Request-Stream semantics, native support for Server Push and Stream data transfer

Flow Control, single Stream granularity and whole link granularity flow control

Head compression HPACK

Binary Frame

Native TLS support

GRPC

The advantages and disadvantages of building RPC protocol based on HTTP and TCP protocols are mentioned above. Compared with Dubbo based on TPC transport layer, Google chooses to define gRPC directly on HTTP/2 protocol. For the basic introduction and design vision of gRPC, please refer to the above two articles. Here, I will only extract a few features in the design vision that can reflect the design purpose of gRPC.

Coverage & Simplicity, protocol design and framework implementation should be universal and simple enough to run on any device, even some resources first, such as IoT, Mobile and other devices.

Interoperability & Reach, to be built on a more general protocol, the protocol itself needs to be supported by almost all the infrastructure on the network.

General Purpose & Performant, to strike a good balance between scenarios and performance, first of all, the protocol itself should be suitable for various scenarios, and at the same time, it should also have high performance as far as possible.

Payload Agnostic, the load transferred on the protocol should remain language and platform neutral.

Streaming, to support Request-Response, Request-Stream, Bi-Steam and other communication models.

Flow Control, the protocol itself has the ability to perceive and limit traffic.

Metadata Exchange, in addition to the RPC service definition, provides additional data transfer capabilities.

Generally speaking, under the guidance of this design concept, gRPC is finally designed as a cross-language, cross-platform, general-purpose, high-performance, HTTP/2-based RPC protocol and framework.

Protobuf

Protocol buffers (Protobuf) is a cross-platform, language-neutral structured data description and serialization product developed by Google. It not only defines a set of structured data definition protocols, but also provides corresponding Compiler tools to transform language-neutral descriptions into specific descriptions of corresponding languages.

Some of its features include:

Cross-language, cross-platform, language-neutral data description format, which provides Compiler tools for generating multiple languages by default.

Security, because the scope of deserialization and the format of the output content are pre-generated by Compiler at compile time, thus bypassing problems similar to Java Deserialization Vulnarability.

Binary high performance

Strong type

Backward compatibility of field changes

Message Person {required string name = 1; required int32 id = 2; optional string email = 3; enum PhoneType {MOBILE = 0; HOME = 1; WORK = 2;} message PhoneNumber {required string number = 1; optional PhoneType type = 2 [default = HOME];} repeated PhoneNumber phone = 4;}

In addition to structured data description, Protobuf also supports the definition of RPC services, which allows us to define a .proto service description file, and then use Protobuf Compiler tools to generate interfaces and stub for specific languages and RPC frameworks. GRPC + Protobuf, Dubbo-gRPC + Protobuf, and Dubbo + Protobuf, which will be discussed in detail later, are all implemented through custom Compiler classes.

Service SearchService {rpc Search (SearchRequest) returns (SearchResponse);}

Support made by Dubbo

Cross-language service development involves many aspects, from service definition, RPC protocol to serialization protocol, and there is also a corresponding SDK implementation for each language. Although thanks to the contribution of the community, Dubbo has gradually made improvements in the implementation of multilingual SDK, including Java, Go, PHP, client, Python, NodeJs, C, Python, NodeJs, C, etc., but there are still many improvements in the above three points of cross-language friendliness mentioned above.

Protocol, above we have analyzed the shortcomings of Dubbo protocol. If we can build an application layer protocol on top of HTTP/2, we can undoubtedly avoid these disadvantages. At the same time, we can improve the penetration of the protocol as much as possible, avoid the existence of protocol conversion components such as gateways, and be more conducive to traffic control on the link. Considering that gRPC is built on HTTP/2 and is already the recommended communication protocol in the cloud native domain, Dubbo chose to directly support gRPC protocol as the current HTTP/2 solution in the first phase. We also know that the disadvantages of the gRPC framework lie in the lack of ease of use and the lack of service governance capabilities (which is why most vendors will not directly use the gRPC framework). By integrating it into the Dubbo framework, users can easily use the combination of Dubbo programming model + Dubbo service governance + gRPC protocol communication.

Service definition, current Dubbo service definition and specific programming language bindings, do not provide a language-neutral service description format. For example, Java defines the Interface interface, which has to be redefined in another format in other languages. So Dubbo implements language-neutral service definitions by supporting Protobuf.

Serialization, the serialization currently supported by Dubbo includes Json, Hessian2, Kryo, FST, Java, etc., among which only Json and Hessian2 are supported across languages. General Json has inherent performance problems, while Hessian2 lacks both efficiency and multilingual SDK. To this end, Dubbo provides a more efficient and easy-to-use cross-language serialization solution by supporting Protobuf serialization.

Example

Example 1, using Dubbo to develop gRPC services

GRPC is Google's open source PRC communication protocol built on top of HTTP/2. Dubbo relies on its flexible protocol extension mechanism to increase the support for gRPC (HTTP/2) protocol.

Current support is limited to the Dubbo Java language version, and subsequent Go or other language versions will provide support in a similar manner. Next, use a simple example to demonstrate how to communicate using the gRPC protocol in Dubbo.

1. Define service IDL

First of all, the service is defined through the standard Protobuf protocol as follows:

Syntax = "proto3"; option java_multiple_files = true; option java_package = "io.grpc.examples.helloworld"; option java_outer_classname = "HelloWorldProto"; option objc_class_prefix = "HLW"; package helloworld; / / The greeting service definition. Service Greeter {/ / Sends a greeting rpc SayHello (HelloRequest) returns (HelloReply) {}} / / The request message containing the user's name. Message HelloRequest {string name = 1;} / / The response message containing the greetings message HelloReply {string message = 1;}

Here, we define a Greeter service with only one method sayHello, and define the input and output parameters of the method

2. Protobuf Compiler generates Stub

Define the Maven Protobuf Compiler plug-in tool. Here we extend Protobuf's Compiler tool to generate Dubbo-specific RPC stub, which is currently released as a Maven plug-in.

Org.xolstice.maven.plugins protobuf-maven-plugin 0.5.1 com.google.protobuf:protoc:3.7.1:exe:$ {os.detected.classifier} dubbo-grpc-java org.apache.dubbo:protoc-gen-dubbo-java:1.19.0-SNAPSHOT:exe:$ {os.detected.classifier} build/generated/source/proto/main/java false grpc compile compile-custom

Among them

PluginArtifact specifies a custom version of Dubbo's Java Protobuf Compiler plug-in, which is used to generate a custom version of Dubbo's gRPC stub during compilation.

Org.apache.dubbo:protoc-gen-dubbo-java:1.19.0-SNAPSHOT:exe:$ {os.detected.classifier}

Since protoc-gen-dubbo-java supports both gRPC and Dubbo protocols, the type of stub that can be generated. The default value is gRPC. For more information on the use of dubbo protocol, please see developing Dubbo Services using Protobuf.

Grpc

two。 Generate Java Bean and Dubbo-gRPC stub

# run the following maven command $mvn clean compile

The generated Stub and message classes are as follows:

Focus on GreeterGrpc, including all gRPC standard stub classes / methods, while adding Dubbo-specific interfaces, which will be relied on for service exposures on the Provider side and service calls on the Consumer side.

/ * * Code generated for Dubbo*/public interface IGreeter {default public io.grpc.examples.helloworld.HelloReply sayHello (io.grpc.examples.helloworld.HelloRequest request) {throw new UnsupportedOperationException ("No need to override this method, extend XxxImplBase and override all methods it allows.");} default public com.google.common.util.concurrent.ListenableFuture sayHelloAsync (io.grpc.examples.helloworld.HelloRequest request) {throw new UnsupportedOperationException ("No need to override this method, extend XxxImplBase and override all methods it allows.") } public void sayHello (io.grpc.examples.helloworld.HelloRequest request, io.grpc.stub.StreamObserver responseObserver);}

3. Business logic development

Inherit GreeterGrpc.GreeterImplBase (from step 2) and write business logic, which is consistent with native gRPC.

Package org.apache.dubbo.samples.basic.impl;import io.grpc.examples.helloworld.GreeterGrpc;import io.grpc.examples.helloworld.HelloReply;import io.grpc.examples.helloworld.HelloRequest;import io.grpc.stub.StreamObserver;public class GrpcGreeterImpl extends GreeterGrpc.GreeterImplBase {@ Override public void sayHello (HelloRequest request, StreamObserver responseObserver) {System.out.println ("Received request from client."); System.out.println ("Executing thread is" + Thread.currentThread () .getName ()) HelloReply reply = HelloReply.newBuilder () .setMessage ("Hello" + request.getName ()) .build (); responseObserver.onNext (reply); responseObserver.onCompleted ();}}

Provider side exposes Dubbo service

Take Spring XML as an example

Public static void main (String [] args) throws Exception {ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext ("spring/dubbo-demo-provider.xml"); context.start (); System.out.println ("dubbo service started"); new CountDownLatch (1). Await ();}

Reference Dubbo service

Public static void main (String [] args) throws IOException {ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext ("spring/dubbo-demo-consumer.xml"); context.start (); GreeterGrpc.IGreeter greeter = (GreeterGrpc.IGreeter) context.getBean ("greeter"); HelloReply reply = greeter.sayHello (HelloRequest.newBuilder (). SetName ("world!"). Build (); System.out.println ("Result:" + reply.getMessage ()); System.in.read ();}

Example 1 attached: advanced usage

I. Asynchronous call

Take a look at the interface generated by protoc-gen-dubbo-java again:

/ * Code generated for Dubbo*/public interface IGreeter {default public HelloReply sayHello (HelloRequest request) {/ /. } default public ListenableFuture sayHelloAsync (HelloRequest request) {/ / } public void sayHello (HelloRequest request, StreamObserver responseObserver);}

Here, three types of overloaded methods are generated for the sayHello method, which are used for synchronous invocation, asynchronous invocation and streaming invocation respectively. If the consumer wants to make an asynchronous call, simply call sayHelloAsync ():

Public static void main (String [] args) throws IOException {/ /... GreeterGrpc.IGreeter greeter = (GreeterGrpc.IGreeter) context.getBean ("greeter"); ListenableFuture future = greeter.sayHAsyncello (HelloRequest.newBuilder (). SetName ("world!"). Build (); / /.}

II. Advanced configuration

Because the current implementation is directly integrated with gRPC-java SDK, many configurations have not been aligned with the Dubbo side, or have not been opened in the form of Dubbo configuration. Therefore, in order to provide maximum flexibility, we directly expose the configuration interface of gRPC-java.

In most scenarios, you may not use the following extensions because they are more about intercepting the gRPC protocol or configuring at the HTTP/2 level. Using these extension points at the same time may require basic knowledge of HTTP/2 or gRPC.

Extension point

Currently supported extension points are as follows:

Org.apache.dubbo.rpc.protocol.grpc.interceptors.ClientInterceptor

Org.apache.dubbo.rpc.protocol.grpc.interceptors.GrpcConfigurator

Org.apache.dubbo.rpc.protocol.grpc.interceptors.ServerInterceptor

Org.apache.dubbo.rpc.protocol.grpc.interceptors.ServerTransportFilter

GrpcConfigurator is the most general extension point. Let's take this as an example. The basic definition is as follows:

Public interface GrpcConfigurator {/ / to customize gRPC NettyServerBuilder default NettyServerBuilder configureServerBuilder (NettyServerBuilder builder, URL url) {return builder;} / / to customize gRPC NettyChannelBuilder default NettyChannelBuilder configureChannelBuilder (NettyChannelBuilder builder, URL url) {return builder;} / / to customize gRPC CallOptions, to define a service to pass data default CallOptions configureCallOptions (CallOptions options, URL url) {return options;}} between each request

The following is an example extension implementation:

Public class MyGrpcConfigurator implements GrpcConfigurator {private final ExecutorService executor = Executors .newFixedThreadPool (200, new NamedThreadFactory ("Customized-grpc", true); @ Override public NettyServerBuilder configureServerBuilder (NettyServerBuilder builder, URL url) {return builder.executor (executor);} @ Override public NettyChannelBuilder configureChannelBuilder (NettyChannelBuilder builder, URL url) {return builder.flowControlWindow (10);} @ Override public CallOptions configureCallOptions (CallOptions options, URL url) {return options.withOption (CallOptions.Key.create ("key"), "value");}}

Configure to Dubbo SPI and add configuration files to `resources/META-INF/services

Default=org.apache.dubbo.samples.basic.comtomize.MyGrpcConfigurator

Specifies that the thread pool on the Provider side defaults to Dubbo's thread pool, such as fixed (default), cached, direct, and so on. The following demonstrates switching to a business custom thread pool.

Private final ExecutorService executor = Executors .newFixedThreadPool (200,200, new NamedThreadFactory ("Customized-grpc", true); public NettyServerBuilder configureServerBuilder (NettyServerBuilder builder, URL url) {return builder.executor (executor);}

Specify the Consumer end current limit value to set the Consumer current limit value to 10

@ Overridepublic NettyChannelBuilder configureChannelBuilder (NettyChannelBuilder builder, URL url) {return builder.flowControlWindow (10);}

Pass additional parameters DemoService service invocation pass key

@ Overridepublic CallOptions configureCallOptions (CallOptions options, URL url) {if (url.getServiceInterface () .equals ("xxx.DemoService")) {return options.withOption (CallOptions.Key.create ("key"), "value");} else {return options;}}

Third, the two-way streaming communication code also provides an example of supporting two-way streaming communication, as well as an example implementation of Interceptor extension to intercept streaming calls.

* MyClientStreamInterceptor, which works on the client side, intercepts the sent request flow and the received response flow * MyServerStreamInterceptor, works on the server side, intercepts the received request flow and the sent response flow.

IV. TLS configuration

The configuration is consistent with the general TLS support provided by Dubbo. For more information, please see the official Dubbo documentation.

Example 2, using Protobuf to develop Dubbo services

Next, let's look at the Protobuf-based Dubbo service development process with a concrete example.

1. Define services

Define services through standard Protobuf

Syntax = "proto3"; option java_multiple_files = true; option java_package = "org.apache.dubbo.demo"; option java_outer_classname = "DemoServiceProto"; option objc_class_prefix = "DEMOSRV"; package demoservice; / / The demoservice definition. Service DemoService {rpc SayHello (HelloRequest) returns (HelloReply) {}} / / The request message containing the user's name. Message HelloRequest {string name = 1;} / / The response message containing the greetings message HelloReply {string message = 1;}

A DemoService service is defined here, which contains only one sayHello method, and defines the input and output parameters of the method.

2. Compiler compilation service

Introduce the Protobuf Compiler Maven plug-in and specify the protoc-gen-dubbo-java RPC extension

Org.xolstice.maven.plugins protobuf-maven-plugin 0.5.1 com.google.protobuf:protoc:3.7.1:exe:$ {os.detected.classifier} dubbo-grpc-java org.apache.dubbo:protoc-gen-dubbo-java:1.19.0-SNAPSHOT:exe:$ {os.detected.classifier} build/generated/source/proto/main/java false dubbo compile compile-custom

Note that the difference between this and the gRPC support section of Dubbo is: dubbo

Generate RPC stub

# run the following maven command $mvn clean compile

The generated Java class is as follows:

DemoServiceDubbo customized stub for Dubbo

Public final class DemoServiceDubbo {private static final AtomicBoolean registered = new AtomicBoolean (); private static Class init () {Class clazz = null; try {clazz = Class.forName (DemoServiceDubbo.class.getName ()); if (registered.compareAndSet (false, true)) {org.apache.dubbo.common.serialize.protobuf.support.ProtobufUtils.marshaller (org.apache.dubbo.demo.HelloRequest.getDefaultInstance ()) Org.apache.dubbo.common.serialize.protobuf.support.ProtobufUtils.marshaller (org.apache.dubbo.demo.HelloReply.getDefaultInstance ());}} catch (ClassNotFoundException e) {/ / ignore} return clazz;} private DemoServiceDubbo () {} public static final String SERVICE_NAME = "demoservice.DemoService"; / * Code generated for Dubbo * / public interface IDemoService {static Class clazz = init () Org.apache.dubbo.demo.HelloReply sayHello (org.apache.dubbo.demo.HelloRequest request); java.util.concurrent.CompletableFuture sayHelloAsync (org.apache.dubbo.demo.HelloRequest request);}}

The most noteworthy is the IDemoService interface, which defines the underlying interface as a Dubbo service.

3. Develop business logic

From this step on, all the development processes are the same as defining the Java interface directly. The implementation interface defines the business logic.

Public class DemoServiceImpl implements DemoServiceDubbo.IDemoService {private static final Logger logger = LoggerFactory.getLogger (DemoServiceImpl.class); @ Override public HelloReply sayHello (HelloRequest request) {logger.info ("Hello" + request.getName () + ", request from consumer:" + RpcContext.getContext () .getRemoteAddress ()); return HelloReply.newBuilder () .setMessage ("Hello" + request.getName () + ", response from provider:" + RpcContext.getContext () .getLocalAddress ()) .build () @ Override public CompletableFuture sayHelloAsync (HelloRequest request) {return CompletableFuture.completedFuture (sayHello (request));}}

4. Configure Provider

Expose Dubbo services

Public static void main (String [] args) throws Exception {ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext ("spring/dubbo-provider.xml"); context.start (); System.in.read ();}

5. Configure Consumer

Reference Dubbo service

Public static void main (String [] args) throws Exception {ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext ("spring/dubbo-consumer.xml"); context.start (); IDemoService demoService = context.getBean ("demoService", IDemoService.class); HelloRequest request = HelloRequest.newBuilder (). SetName ("Hello"). Build (); HelloReply reply = demoService.sayHello (request); System.out.println ("result:" + reply.getMessage ()); System.in.read () } Thank you for reading, the above is the content of "how to use Dubbo to develop gRPC services". After the study of this article, I believe you have a deeper understanding of how to use Dubbo to develop gRPC services, and the specific usage needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.