Hadoop3.2.1 (HDFS) source code analysis: RPC client implementation and source code

By | February 12, 2020
0 Comment

Client-side architecture

The Client class has only one entry, which is the call() method. The proxy class will call the Client.call() method to send the RPC request to the remote server, and then wait for the response from the remote server. If an exception occurs when the remote server responds to the request, an exception is thrown in the call() method.

In the call method, the remote call information is first encapsulated into a Client.Call object (save the completion flag, return information, exception information, etc.), and then the connection object is used to manage the Socket connection between Client and Server.

In the getConnection method, the socket connection to the server is established through setupIOstreams, the Connection thread is started, and the socket is monitored to read the server response.

The call() method sends an RCP request.

The call() method calls Call.wait() to wait for the Server response information on the Call object.

The Connection thread receives the response information and sets the return information field of the Call object, and calls Call.notify() to wake up the call() method thread to read the return value of the Call object.

Client creation process

The following is the code to create the client. The protocol uses proto, so the RpcEngine produced is ProtobufRpcEngine. So the next article is based on ProtobufRpcEngine for source code analysis.

     public static void main(String[] args) throws Exception {
        //1. configure object
        Configuration conf = new Configuration();
 
        //2. configure protocol
        RPC.setProtocolEngine(conf, Server.MetaInfoProtocol.class,
                ProtobufRpcEngine.class);
 
 
        //3. get request
        Server.MetaInfoProtocol proxy = RPC.getProxy(Server.MetaInfoProtocol.class, 1L,
                new InetSocketAddress("localhost", 7777), conf);
 
        //4. object initialize
        CustomProtos.GetMetaInfoRequestProto obj =  CustomProtos.GetMetaInfoRequestProto.newBuilder().setPath("/meta").build();
 
        //5. receive request
        CustomProtos.GetMetaInfoResponseProto metaData = proxy.getMetaInfo(null, obj);
 
        //6. output
        System.out.println(metaData.getInfo());
    }

The above code can be divided into 4 parts.

  • Build configuration objects & set RpcEngine engine
  • Get the proxy object
  • Set request parameters & request through the proxy object.
  • Processing results

In the getProxy method, there are two main steps:

  1. Construct an invoker object that implements the InvocationHandler interface (The InvocationHandler object in the dynamic proxy mechanism will proxy all calls on the target interface in the invoke() method, and users can add proxy operations in the invoke() method
  2. Call Proxy.newProxylnstance() to get the dynamic proxy object and return it through ProtocolProxy

Related Posts