Shulou(Shulou.com)05/31 Report--

This article mainly introduces "Hadoop RPC-Client source code analysis". In daily operation, I believe many people have doubts about Hadoop RPC-Client source code analysis. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts of "Hadoop RPC-Client source code analysis". Next, please follow the editor to study!

Since it is RPC, there are naturally clients and servers, and of course, org.apache.hadoop.rpc also has class Client and class Server. Here, let's take a closer look.

Check org.apache.hadoop.rpc.Client. The following figure contains the key classes and methods in org.apache.hadoop.rpc.Client.

Because Client may communicate with multiple Server, a typical HDFS read requires dealing with NameNode and communicating with one / some DataNode. this

It means that a Client needs to maintain multiple connections. At the same time, in order to reduce unnecessary connections, what Client does now is to take ConnectionId (the far right in the figure)

Side) to act as the ID of Connection. ConnectionId includes an InetSocketAddress (IP address + port number or hostname + port number) object and a

User information object. That is, communications from the same user to the same InetSocketAddress will share the same connection.

The connection is encapsulated in the class Client.Connection, and all RPC calls communicate through Connection. A RPC call, naturally with input parameters

Output parameters and possible exceptions, and each call has a unique id in order to distinguish between different calls on the same Connection. Whether the call ends or not is also required.

A tag, all of which is reflected in the object Client.Call. The Connection object maintains all Call on this connection through a Hash table:

Java code

1. Private Hashtable calls = new Hashtable ()

A RPC call adds the request to the Connection through addCall. In order to be able to transfer the basic types of Java over this framework, String and Writable connect

The implementation class of the port, as well as the array whose elements are of the above types, we generally package the parameters needed by Call into ObjectWritable objects.

Client.Connection will connect to the server through socket, and verify the version number (Client.ConnectionwriteHeader ()) of the client / server after the connection is successful.

Method, after the verification is successful, you can send / reply the request through the Writable object. Note that each Client.Connection will have a thread, continuously

Read the socket, unpack the received result, find the corresponding Call, set the Call, and notify that the result has been obtained.

Call uses Obejct's wait and notify to convert asynchronous message interactions on RPC into synchronous calls.

It is also important to note that a Client will have multiple Client.Connection, which is a natural result.

At this point, the study of "Hadoop RPC-Client source code analysis" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.