Shulou(Shulou.com)06/03 Report--

Recently, I have been reading the part about RPC in the second chapter of "hadoop 2.x HDFS Source Code Analysis" written by Xu Peng, saying that the programming foundation of java is really too much, and the dynamic agent can barely understand that the annotation of proto buffer, nio and java is really too much, and I don't understand it very well in many places. I decided to put this aside for a while and finish reading the whole book and write a few more articles about hadoop RPC. But write some recent reading notes.

RPC's full name is remote procedure call, or remote invocation, and just like dubbo is often used in production, a local process can invoke a remote service through RPC as if it were a local method.

The following is to talk about your understanding of RPC by introducing a complete hadoop RPC process:

1. First of all, define the protocol (protocol) on both sides of the communication, in fact, define the interface to be called, so that the caller (client) can know what functions and parameters should be passed to initiate a RPC request. Since it is transmitted to another jvm over the network, it needs to be serialized once. Here, the implementation of hadoop RPC supports multiple serializations. There are self-provided serialization methods and proto buffer serialization methods. I heard that other serialization methods can be supported, such as ClientProtocol, which namenode communicates with the client.

There is a ClientNameNodeProtocolTranslatorPB instance called server on the 2.Client side. This class "implements" ClientProtocol. In fact, it transforms ClientProtocol that does not support proto buffer into a client NamenodeProtocolPB protocol that supports this serialization mode. Of course, a lot of dynamic agents are involved, and the process is very complicated, and now I don't understand it very well.

3. Requests will not be sent so easily. Namenode has supported high availability since hadoop2.X, so when server objects are instantiated, they should consider whether to support high availability according to the configuration file. In fact, when active namenode fails, they can actively failover to the namenode of standby and send RPC requests to the backup namenode.

4. Since the request has been serialized, when it is transmitted through socket to the server, there must be a process of deserialization, that is, the ClientNamenodeProtocolPB protocol is transformed into the corresponding ClientProtocol protocol, and then the required operations are performed by calling the corresponding method of NameNodeRPCServer that really implements the ClientProtocol interface. Here the Server side uses the nio programming method to handle the RPC request. The so-called nio means that there are some listening processes listening for connection events, and then putting PRC requests into a queue, and then there are many handler processing RPC requests in the queue. Of course, for network transmission, all handler execution results are completed by a Responder process.

This is the most detailed analysis of what RPC can do so far, with a picture of your own painting below:

The foundation is too poor, the writing is very bad, hope to be able to hit their own face in the future!

2017.2.13

Today's second forced Day, tomorrow's Dog abuse Festival, my study is very bad, and I'm not happy.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.