Shulou(Shulou.com)05/31 Report--

This article mainly introduces "Server side of RPC mechanism analysis in Hadoop". In daily operation, I believe that many people have doubts about RPC mechanism analysis in Hadoop. The editor has consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "RPC mechanism analysis Server side in Hadoop". Next, please follow the editor to study!

1. Server.Listener

After the RPC request on the RPC Client side is sent to the Server side, it is first received by the Server.Listener

The Server.Listener class inherits from the Thread class and listens for OP_READ and OP_ACCEPT events

Server.Listener receives the RPC request, reads the data in the Server.Listener.doRead () method, and calls the Server.Connection.readAndProcess () method in the doRead () method

Finally, the Server.Connection.processRpcRequest () method is called. The source code is as follows:

Private void processRpcRequest (RpcRequestHeaderProto header, DataInputStream dis) throws WrappedRpcServerException, InterruptedException {... Writable rpcRequest; / / deserializes the RPC request (WritableRpcEngine.Invocation object) sent by the client from the member variable dis: try {/ / Read the rpc request rpcRequest = ReflectionUtils.newInstance (rpcRequestClass, conf); rpcRequest.readFields (dis) } catch (Throwable t) {/ / includes runtime exception from newInstance...} / / construct Server.Call instance object Call call = new Call (header.getCallId (), header.getRetryCount (), rpcRequest, this, ProtoUtil.convert (header.getRpcKind ()), header .getClientId (). ToByteArray ()) / / put the Server.Call instance object in the call queue callQueue.put (call); / / queue the call; maybe blocked here incRpcCount (); / / Increment the rpc count}

Call queue callQueue is a member variable of Server. Server.Listener and Server.Handler are typical producer and consumer models.

The doRead () method of Server.Listener (producer) finally calls the Server.Connection.processRpcRequest () method

While Server.Handler (consumer) handles RPC requests

2. Server.Handler inherits the Thread class, and its main work is to handle the calls in callQueue, all of which are completed in the run () method. In the main loop of run (), each time a request for dequeue from callQueue is processed, Server.call () is an abstract method, which actually calls the RPC.Server.call () method, and finally completes the method call on the Server side through the WritableRPCEngine.call () method.

/ * Handles queued calls. * / private class Handler extends Thread {... @ Override public void run () {... ByteArrayOutputStream buf = new ByteArrayOutputStream (INITIAL_RESP_BUF_SIZE); while (running) {. Final Call call = callQueue.take (); / / get a RPC call request... Writable value = null Value = call.connection.user.doAs (new PrivilegedExceptionAction () {@ Override public Writable run () throws Exception {/ / call RPC.Server.call () method / / call.rpcKind: type of RPC call request Generally, it is the class name of Writable / / call.connection.protocolName: RPC protocol interface / / call.rpcRequest: Invocation instance object, including method name, parameter list, Class object array of parameter list / / call.timestamp: call timestamp return call (call.rpcKind, call.connection.protocolName) Call.rpcRequest, call.timestamp) }});}.}}

The RPC.Server.call () method is as follows:

@ Overridepublic Writable call (RPC.RpcKind rpcKind, String protocol, Writable rpcRequest, long receiveTime) throws Exception {return getRpcInvoker (rpcKind) .call (this, protocol, rpcRequest, receiveTime);}

Finally, the method call on the Server side is completed through the WritableRPCEngine.call () method, with the following code:

@ Overridepublic Writable call (org.apache.hadoop.ipc.RPC.Server server, String protocolName, Writable rpcRequest, long receivedTime) throws IOException, RPC.VersionMismatch {Invocation call = (Invocation) rpcRequest; / / forcibly converts the RPC request to a WritableRpcEngine.Invocation object. Long clientVersion = call.getProtocolVersion (); final String protoName; ProtoClassProtoImpl protocolImpl / / instance object of the implementation class of the RPC protocol interface on the Server side. / / Invoke the protocol method try {. / / get the method object called in the RPC request Method Method method = protocolImpl.protocolClass.getMethod (call.getMethodName (), call.getParameterClasses ()); method.setAccessible (true) . / / the instance object protocolImpl of the implementation class of the RPC protocol interface on the server side calls the specific method Object value = method.invoke (protocolImpl.protocolImpl, call.getParameters ());. / / the call ends normally and returns the call result return new ObjectWritable (method.getReturnType (), value). } catch (InvocationTargetException e) {/ / call has an exception, wraps the exception with IOException, and finally throws the exception Throwable target = e.getTargetException (); if (target instanceof IOException) {throw (IOException) target;} else {IOException ioe = new IOException (target.toString ()); ioe.setStackTrace (target.getStackTrace ()) Throw ioe;}} catch (Throwable e) {...}

In the WritableRpcEngine.call () method, the incoming rpcRequest is forcibly converted to an object call of type WritableRpcEngine.Invocation, and the Method object is obtained through the method name (getMethodName () method) contained in the object call and the Class object array (getParameterClasses ()) of the parameter list. Finally, the method on the instance object protocolImpl of the implementation class is called through the invoke () method of the Method object, and the remote procedure call of Hadoop is completed.

OK, now the specific method on the Server side has been called, and the result of the call can be divided into two cases:

1) if the call ends normally, the return value of the method and the result of the call will be encapsulated into an object of type ObjectWritable and return

2) an exception occurs in the call, and an exception of type IOException is thrown

3. Server.Responder

The function of this class: send the reply of the Hadoop remote procedure call to the Client side. The Server.Responder class inherits from the Thread class and listens for OP_WRITE events, that is, the channel is writable. I can't write down the details.

At this point, the study on "the Server side of RPC mechanism analysis in Hadoop" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.