Shulou(Shulou.com)06/03 Report--

[TOC]

1. Basic concepts of RPC 1. What is RPC?

​ 's so-called RPC is actually generated for the communication between two processes of different hosts, usually the process communication between different hosts, then the programming needs to take into account the function of network communication, so the programming will become complex. RPC is a mechanism of message exchange. When a process on one host initiates a request to the process of another host, the kernel will transfer the request to the RPC client,RPC client through the encapsulation of the message to the RPC server,RPC server of the target host, and then parse the message, restore it to a normal request, and transfer it to the target process on the target host. So from the point of view of the two processes of the two hosts, it is like two processes communicating on the same host, completely unaware that they are on different hosts. So RPC is actually a protocol or programming framework that simplifies the writing of distributed programs.

​ while hadoop is a typical distributed project, it is common to invoke services across hosts (not http services). To be clear, the caller clearly knows that the callee is on a different host through the http of these restfulapi-exposed services. When calling through RPC, from the caller's point of view, he thinks that the callee is on the same host, but in fact it is on a different host.

2. The basic process of RPC

1. First of all, the caller needs to have a RPCclient and the callee needs to have a RCPServer. These two services are used for RPC communication.

2. The caller calls the specified method through RPCClient, and RPCClient encapsulates the request (binary serialization of the method parameter value) and passes it to server

3. After receiving the request, server deserializes the parameters, restores them to the original form of the request, and then finds the corresponding method to make local calls. Return the return value of the method to client through RPC

4. After client receives the result, it returns the result to the caller as the return value.

2. The basic flow of RPC1 of hadoop and RPC of hdfs

The rpc process for hdfs is basically the same, and the key is to get the NameNode proxy object.

2. Agent mode in java

(1) static agent

First define a public interface that includes a list of methods that can be called through RPC. And both the proxied object and the object itself need to implement the interface.

(2) dynamic agent

First define a public interface that includes a list of methods that can be called through RPC. Neither the proxied object nor the object itself needs to implement the interface. Instead, it is implemented through the mechanism of anonymous inner class + reflection. This is how hadoop uses it.

3. An example of the hadoop rpc framework

Example structure:

Specific implementation code of methods executed locally by Server MyImpl.java server MyInterface.java can be executed by rpc list of methods on the MyRpcServer.java rpc server side and the Client MyRpcClient.java rpc client side

Server:

/ * MyInterface.java * / package Server;import org.apache.hadoop.ipc.VersionedProtocol;public interface MyInterface extends VersionedProtocol {public static long versionID = 1001; / / this is the tag public String helloWorld (String name) corresponding to the client and server of the tag RPC;} / * MyImpl.java*/package Server;import org.apache.hadoop.ipc.ProtocolSignature;import java.io.IOException Public class MyImpl implements MyInterface {/ * this is the actual target * / override the method @ Override public String helloWorld (String name) {return "hello," + name;} / / return version number @ Override public long getProtocolVersion (String s, long l) throws IOException {return MyInterface.versionID } / / return signature information @ Override public ProtocolSignature getProtocolSignature (String s, long l, int I) throws IOException {return new ProtocolSignature (MyInterface.versionID, null);}} / * MyRpcServer.java*/package Server;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.ipc.RPC;import java.io.IOException Public class MyRpcServer {public static void main (String [] args) {/ / establish rpc channel object RPC.Builder builder = new RPC.Builder (new Configuration ()); / / set RPC server parameter builder.setBindAddress ("localhost"); builder.setPort (7788) / / deployment program, pass in the interface definition that implements the server business code, which includes the methods that the rpcserver can provide, that is, the list of methods called to client, and introduces the class object builder.setProtocol (MyInterface.class) by reflection; / / deploy the implementation class object builder.setInstance (new MyImpl ()) of the interface / / enable server try {RPC.Server server = builder.build (); server.start ();} catch (IOException e) {e.printStackTrace ();}

Client:

/ * MyRpcClient.java*/package Client;import Server.MyInterface;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.ipc.RPC;import java.io.IOException;import java.net.InetSocketAddress Public class MyRpcClient {public static void main (String [] args) {try {/ / get the proxy object, set the interface class object and the versionID of RPC communication Rpcserver address, configuration object MyInterface proxy = RPC.getProxy (MyInterface.class, MyInterface.versionID, new InetSocketAddress ("localhost", 7788), new Configuration ()) / / after obtaining the proxy object, you can call the method in the interface class through proxy, where the helloWorld object System.out.println (proxy.helloWorld ("king")) defined above is called;} catch (IOException e) {e.printStackTrace ();}

The server side and client side are started below, and the execution result is:

/ / server: you can see the listener port 7788 [main] INFO org.apache.hadoop.ipc.CallQueueManager-Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 100scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler [Socket Reader # 1 for port 7788] INFO org.apache.hadoop.ipc.Server-Starting Socket Reader # 1 for port 7788 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server-IPC Server Responder: starting [IPC Server listener on 7788] INFO org.apache.hadoop.ipc. Server-IPC Server listener on 7788: starting//client: we passed in "King" as the parameter Be able to strive to implement hello,king

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.