Shulou(Shulou.com)06/03 Report--

1. On the concept of RPC (1) RPC

    RPC (Remote Procedure Call)-remote procedure call, which is a protocol that requests services from remote computer programs over a network without knowing the underlying network technology. The RPC protocol assumes the existence of some transport protocols, such as TCP or UDP, to carry information data between communication programs. In the OSI network communication model, RPC spans the transport layer and the application layer. RPC makes it easier to develop applications, including web-based distributed programs.

(2) the seven-layer model of OSI network

The first layer: physical layer. This layer is mainly used to transmit these binary data.

The second layer: link layer. Encapsulate the packets of the above network layer into data frames to facilitate physical layer transmission

The third layer: the network layer. Define how data is transferred between network devices

The fourth layer: the transport layer. Manages end-to-end data transmission in the network

The fifth layer: the conversation layer. Manage user sessions and control the establishment and interruption of logical connections between users

The sixth layer: the presentation layer. Define the transmission format, encoding and decoding specifications of data in different systems

The seventh layer: application layer. Defines the interface for communicating and transmitting data in the network

(3) the working mechanism of RPC

    RPC follows the client / server model: the requester is a client and the service provider is a server. First, the client invocation process sends a call message with process parameters to the service process, and then waits for the reply message. On the server side, the process stays asleep until the call message arrives. When a call message arrives, the server gets the process parameters, calculates the result, sends the reply message, and then waits for the next call message. Finally, the client invokes the process to receive the reply message, obtains the process result, and then calls execution to continue.

    popularly said: RPC refers to remote procedure calls, that is, two servers A Magi B, an application deployed in server A, want to call the functions and methods provided by server B, because it is not in a memory space, can not be called directly, need to express the semantics of the call and convey the data of the call through the network. It can also be understood as a service call between different processes.

Specific steps for implementation:

   first (communication problem): mainly by establishing a TCP connection between the client and the server, all the exchanged data of the remote procedure call is transmitted in this connection. The connection can be an on-demand connection, which is broken at the end of the call, or a long connection, where multiple remote procedure calls share the same connection.

   second (addressing problem): how does the application on server A tell the underlying RPC framework, how to connect to server B (such as host or IP address) and the specific port, the name of the method and what the name is, so that the call can be completed.

   third (network passing of method parameters): when an application on server An initiates a remote procedure call, the parameters of the method need to be passed to server B through the underlying network protocol such as TCP. Because the network protocol is based on binary, the values of parameters in memory should be serialized into binary form, that is, Serialize or marshal. The serialized binary is sent to the B server through addressing and transmission.

   fourth (deserialization): after receiving the request, the B server needs to deserialize the parameters (the reverse operation of serialization), revert to the in-memory expression, and then find the corresponding method (part of the addressing) to call locally, and then get the return value.

   fifth (return value): the return value should also be sent back to the application on server An and serialized. Server A receives it, then deserializes it, returns it to the in-memory expression, and gives it to the application on server A.

(4) the function of RPC

  -makes callers feel like calling remote functions as if they were calling local functions, makes service providers feel like they are implementing a local function, and shields programming language differences.

  -makes it easier to build distributed computing (applications) without losing the semantic simplicity of local calls while providing powerful remote invocation capabilities

  -on client A, a proxy for server B is generated. (summary)

(5) the popular RPC framework in JAVA

   RMI (remote method invocation): JAVA's own remote method invocation tool, but it has some limitations. After all, it was designed at the beginning of the JAVA language, and later many frameworks are based on RMI.

   Hessian (remote method call based on HTTP): based on HTTP protocol, the performance is not perfect. Load balancing and failure transfer depend on the application's load balancer. The use of Hessian is similar to RMI, except that the role of Registry is watered down. Through the address call displayed, HessianProxyFactory is used to create a proxy object according to the configured address. In addition, the Jar package of Hessian is introduced.

   Dubbo (Ali's open source TCP-based RPC framework): a high-performance RPC framework based on Netty.

2. Introduction to related concepts of RPC (1) of Hadoop

Like other RPC frameworks,    is divided into four parts:

   -serialization layer: Client communicates with Server using serialization classes or custom Writable types provided in Hadoop

   -function call layer: Hadoop RPC implements function calls through dynamic proxy and Java reflection

   -Network Transport layer: Hadoop RPC adopts socket mechanism based on TCP/IP

   -server-side framework layer: RPC Server makes use of Java NIO and uses the event-driven Istroke O model to improve the concurrency processing ability of RPC Server

   Hadoop RPC is widely used in the whole Hadoop, and the communication among Client, DataNode and NameNode depends on it. For example, when we operate HDFS, we use the FileSystem class, which has a DFSClient object inside it, which is responsible for dealing with NameNode. At run time, DFSClient creates a NameNode proxy locally and then manipulates the proxy, which remotely invokes the NameNode method over the network and returns a value.

(2) the relevant technical points involved

  -proxy: a dynamic proxy can provide access to another object while hiding the specific facts of the actual object. The proxy object hides the actual object from the client. Support for dynamic proxies is currently provided in the Java development package, but now only the implementation of the interface is supported.

  -reflection: dynamically loading classes

  -serialization: used to transfer objects over the network

  -NIO: non-blocking async

(3) the construction steps of HadoopRPC

  -defines the PRC protocol: the RPC protocol is the communication interface between the client and the server, which defines the service interface provided by the server.

  -implement the PRC protocol: the Hadoop RPC protocol is usually a Java interface that users need to implement.

  -construct and start RPC server: directly use the static class Builder to construct a RPC Server and call the function start () to start the Server.

  -build the RPC client and send the request: construct the client-side proxy object using the static method getProxy, and invoke the remote-side method directly through the proxy object.

(3) cases of getting started with HadoopRPC

1. Define the protocol

/ / define PRC protocol

Public interface BussinessProtocol {/ / requires a consistent version of long versionID=11511056056L; void mkdir (String name); void hello (String message);} when communicating between client and server

two。 Implementation protocol

/ / implement RPC protocol public class BussinessProtocolImpl implements BussinessProtocol {@ Override public void mkdir (String name) {System.out.println ("created:" + name);} @ Override public void hello (String message) {System.out.println ("Hello" + message);}}

3. Build the server

/ / build RPCserver and start server public class MyRPCServer {public static void main (String [] args) {RPC.Server server = null Try {/ * in new RPC.Builder: input Configuration object * setProtocol (BussinessProtocol.class): set protocol class * setInstance (new BussinessProtocolImpl ()): protocol implementation class object * setBindAddress ("localhost"): bound host * setPort (6789): set port * / server = new RPC.Builder (new Configuration ()) .setProtocol (BussinessProtocol.class) .setInstance (new BussinessProtocolImpl ()) .setBindAddress ("localhost") .setPort (6666) .build () } catch (IOException e) {e.printStackTrace ();} / / start the service server.start ();}}

4. Build client

/ / build the RPC Client and issue the request

Public class MyRPCClient {public static void main (String [] args) {try {/ * get proxy class object Through this proxy, the client can call the server method for logical processing: * class object of Class protocol protocol class * long clientVersion: version * InetSocketAddress addr: host port * Configuration conf: profile object * / BussinessProtocol proxy = RPC.getProxy (BussinessProtocol.class, BussinessProtocol.versionID) New InetSocketAddress ("localhost", 6666), new Configuration () / / the method of the proxy object on the server side is called on the client side / / the real code runs proxy.hello ("zs"); proxy.mkdir ("/ root/zy");} catch (IOException e) {e.printStackTrace ();} on the server side

5. Start the test

Start the server: run:

Start the client to run:

Observe the server console:

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.