Shulou(Shulou.com)05/31 Report--

This article shows you how to analyze the principle and application, the content is concise and easy to understand, it can definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

01 background

For an e-commerce platform, it often involves multiple modules such as shopping cart, order, payment, merchandise, etc., which requires multiple personnel to maintain. If a stand-alone system is adopted, it will lead to such problems:

1. Someone needs to compile and package all the code when maintaining their own module code, and the rest of the modules may need regression testing

2. There is something wrong with one of the modules, which may lead to the failure of the whole system.

This leads to the remote procedure call RPC, which allows us to call the functions of the remote server as if we were calling local functions.

RPC is the cornerstone of distribution.

From stand-alone to distributed, there are many distributed ways of communication:

1. TCP/UDP binary transmission: the oldest and most efficient, in fact, all modes of communication are TCP/UDP in the final analysis.

2. CORBA (Common Object Request Broker Architecture): ancient and complex, supporting object-oriented communication protocols.

3. Web Service (SOA SOAP RDDI WSDL): standardized Web API based on http+xml

4. RestFul (Reprensentational State Transfer): http+json

5. RMI (Remote Method Invocation): a distributed communication protocol within Java

6. JMS (Java Message Service): JavaEE's message framework standard, which is supported by many MQ

7. RPC (Remote Procedure Call): remote procedure call, which is just a general term, focusing on method calls, the specific implementation can even be implemented with RMI RestFul, etc., but generally not, because RMI can not cross languages, and ResuFul is too inefficient.

It is mostly used for communication between server clusters (distributed or micro-service architecture), so more efficient and short transmission modes are often used to improve efficiency.

02 Overview

RPC (Remote Procedure Call), that is, remote procedure call, is a common communication method in distributed systems. In addition to RPC, common multi-system data interaction schemes include distributed message queues, HTTP request calls, databases and distributed caches.

RPC is a concept, and HTTP is also a way of implementing RPC.

Services can be decoupled through RPC, which is the real purpose of using RPC. The principle of RPC mainly uses the dynamic proxy mode, but as for the HTTP protocol, it is only a transport protocol.

2.1 RPC and HTTP

Compared with general Message-based distributed frameworks such as MPI, RPC is more abstract and easier to understand. However, at the bottom, whether it is RPC or MPI, it will eventually be transformed into TCP/UDP packets and sent back and forth.

Contrast:

RPC is suitable for internal use, with low performance consumption, high transmission efficiency and convenient service governance, but it is not recommended to transmit large text, video, etc.

HTTP is suitable for external technology, and MQ can be used for upstream and downstream calls of the company. Mainly used for external heterogeneous environment, browser interface call, APP interface call, third-party interface call and so on.

2.2 RPC and IPC

What we usually call pipeline, FIFO, etc., is the inter-process communication mode (IPC) of the same machine, and RPC is the process communication mode between different machines.

2.3 RPC and Socket

RPC (Remote Procedure Call) is built on top of Socket. The main program running on one machine can call subroutines prepared on another machine, just like LPC (local procedure call). The RPC protocol assumes the existence of certain transport protocols, such as TCP or UDP, to carry information data between communication programs. In the OSI network communication model, RPC spans the transport layer and the application layer. RPC makes it easier to develop applications, including web-based distributed multiprograms.

As a common development method of Cramp S, RPC is efficient and reliable. But the basic principle of RPC method is that the details of communication are ignored by the simplicity of module call, so that programmers do not have to care about the communication protocol between Candros and concentrate on the implementation process. This determines that the communication packet generated by RPC can not have the most appropriate way to deal with each application. Compared with the Socket method, RPC takes up more network bandwidth to transmit the same effective data.

2.4 RPC and MQ

Difference:

1. Architecturally, MQ has an intermediate node that can store messages.

2. Synchronous call: RPC is the first choice for scenarios where you want to wait for the processing result to be returned immediately.

3. The use of MQ, on the one hand, is based on performance considerations, such as the server cannot respond quickly to the client (or the client does not require real-time response) and needs to be cached in the queue. On the other hand, it focuses more on data transmission, so it has more diversified ways, in addition to peer-to-peer, there are subscriptions to publications and other functions.

4. With the growth of business, there is a performance bottleneck at the processing end, and synchronous calls will be transformed into asynchronous calls. You can consider using MQ.

From this, the biggest difference between the two is that RPC has no broker, while message queuing needs to manage the storage of messages, and RPC has no storage, only communication.

Summary:

Asynchronous MQ, synchronous RPC.

03 classification

Common RPC technologies are Cobra, RMI, .NET Remoting, WebService, JSON-RPC, XML-RPC, Hessian, Thrift, Protocol Buffer, gRPC, and so on.

According to the characteristics of serialization mechanism, RPC technology can be divided into textual (WebService, JSON-RPC, XML-RPC, etc.) and binary (RMI, Hessian, Thrift, Protocol Buffer, etc.).

According to the common communication protocols, we can be divided into HTTP-based (WebService, Hessian, etc.) and TCP-based (RMI, .NET Remoting, etc.).

According to whether it can be used on multiple different platforms, it can be divided into platform-specific (RMI is Java platform-specific, .net platform-specific) and platform-independent (such as WebService, JSON-RPC, Hessian, etc.).

04 mode

According to the way it is called, RPC has four modes:

The RR (Request-Response) pattern, also known as the request response pattern, means that each call must have a specific return result information.

Oneway mode, also known as one-way invocation mode, is called and returns, and there is no response information.

Future mode, also known as asynchronous mode, returns to get a Future object, and then executes to get the result information.

Callback mode, also known as callback mode, after processing the request, the processing result information is passed as a parameter to the callback function for processing.

Of the four call modes, RR and Oneway are the most common.

In essence, RPC is generally a synchronous remote service invocation technology for the client. In contrast, MQ is generally an asynchronous invocation technology.

For MQ, we can summarize two message modes according to the characteristics of message processing:

Peer-to-peer mode (Point to Point,PTP), where every message sent by a producer can only be consumed by one consumer, it looks as if the message is passed from one point to another.

The publish-subscribe model, (Publish-Subscribe,PubSub), where every message sent by a producer is sent to all consumers who subscribe to this queue.

05 principle

RPC generally needs to go through four steps:

1. Establish communication

If machine A wants to call machine B, it must first establish a TCP connection between the client and the server.

2. Service addressing

To solve the addressing problem, how the A server connects to the B server (such as a host or IP address) and a specific port, and what is the name of the method.

3. Network transmission

1) Serialization

When the application on the A server initiates a RPC call, the calling method and parameter data need to be serialized first.

2) deserialization

When the B server receives the request from the A server, it needs to deserialize the received parameters and other information.

4. Service call

Server B gets the return value after making a local call (through the proxy Proxy). At this time, the return value needs to be sent back to server A, which also needs to be serialized, and then the binary data is sent back to server A through network transmission.

Typically, a complete PRC call needs to go through these four steps.

5.1 component

An RPC framework generally consists of the following components:

RPC Server: responsible for exporting (export) remote interface

RPC Client: agent implementation responsible for importing (import) remote interface

RPC Proxy: proxy implementation of remote interface

RPC Invoker:

Client implementation: responsible for encoding the call information and sending the call request to the server and waiting for the call result to return

Server implementation: responsible for calling the specific implementation of the server-side interface and returning the call result

RPC Protocol: responsible for protocol encoding / decoding

RPC Connector: responsible for maintaining the connection channel between the client and the server and sending data to the server

RPC Acceptor: responsible for receiving the client's request and returning the request result

RPC Processor: responsible for controlling the invocation process on the server side, including managing the calling thread pool, timeout, etc.

RPC Channel: data transmission channel

5.2 Serialization

RPC implements call communication between another machine, essentially through serialization and deserialization with the help of the underlying TCP/IP protocol.

Serialization converts an object to a transportable binary. You can use JDK (for Java only), JSON (cross-language, text, larger than binary packages, slightly poor performance), Hessian,Protobuf (using IDL files, data types are agreed, cross-language ability is strong).

For deserialization security issues, use whitelist:

1. Scan the type declared by the interface class

2. System built-in whitelist

3. User-defined whitelist

5.3 Service discovery

Service discovery:

The caller and the provider publish and subscribe the function (address) through the registry, the provider registers its own function with the registry, and the caller subscribes to the corresponding function from the registry.

Health check:

The transfer of node fake death is realized by state machine.

The sub-healthy node is transformed into a dead state through the heartbeat mechanism, and then the node state can be checked through the probing mechanism.

5.4 load mode

Load balancing:

Load mode:

1. Random load

2. Consistent hash load

3. Adaptive load (calculate TPS, utilization, etc.)

Cluster strategy:

Cluster strategy: quick failure, failed retry, fixed-point call

5.5 flow control

Server flow control:

Flow control scheme:

1. Identify the source of the call

2. Single-node current limiting: smooth current limiting algorithm

3. Cluster current restriction

5.6 packet deployment 5.7 Security Authentication

06 characteristics

Advantages:

1. With long links, you don't have to shake hands 3 times like http for every communication, which reduces the network overhead.

2. RPC frameworks generally have registration centers and rich monitoring and management; publishing, offline interfaces, dynamic extensions, etc., are imperceptible and unified operations for callers.

3. Improve the scalability of the system

4. Improve system maintainability and continuous delivery ability

5. Realize the high availability of the system.

Disadvantages:

1. It is difficult to develop a perfect RPC framework.

2. The success rate of calling RPC framework is limited by the network condition.

3. It is difficult for beginners to call remote methods.

07 framework

Dubbo

Dubbo is a Java high-performance and excellent service framework developed by Alibaba, which can be seamlessly integrated with the Spring framework.

Montan

Montana is an open source Java framework for Sina Weibo.

Spring Cloud

Rpcx

Rpcx is the Dubbo of the go language ecosystem.

GRPC

GRPC is a high-performance, general open source RPC framework developed by Google.

Gsaop

Gsoap is more suitable for Cmax Candle + programs and heavyweight applications.

Thrift

Suitable for Java programs, middleweight applications.

Rest

Suitable for scripting languages, lightweight applications.

Contrast

08 application

RPC can be found in all kinds of middleware that we are familiar with. The RPC we are talking about here refers to the generalized RPC, that is, the communication technology of distributed systems.

8.1 Nginx and RPC

The interaction between Nginx and back-end services can also be understood as RPC data interaction in essence.

8.2 Hadoop and RPC

The Hadoop file system HDFS, which generally includes a NameNode and multiple DataNode,NameNode and DataNode, communicates through a binary protocol called Hadoop RPC.

8.3 TensorFlow and RPC

Tensorflow Cluster's RPC communication framework uses the gRPC framework developed by Google.

The above content is how to analyze the principle and application. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.