Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of "what is the Ping mechanism of elasticsearch cluster zendiscovery". The editor shows you the operation process through an actual case, and the operation method is simple, fast and practical. I hope this article "what is the Ping mechanism of elasticsearch cluster zendiscovery" can help you solve the problem.

ZenDiscovery implementation mechanism

Ping is the basic means of cluster discovery. By broadcasting on the network or specifying some nodes of ping to obtain cluster information, we can find the master of the cluster to join the cluster. ZenDiscovery implements two ping mechanisms: broadcast and unicast. This article will analyze in detail some of the implementation of this MulticastZenPing mechanism to pave the way for subsequent cluster discovery and master elections.

The process of broadcasting

First of all, let's take a look at broadcast (MulticastZenPing). The principle of broadcasting is very simple. After the node starts, it sends broadcast information to the network. Any node that receives the same cluster name should respond to this broadcast message. In this way, the node gets the relevant information about the cluster. It defines an action: "internal:discovery/zen/multicast" and the broadcast header: INTERNAL_HEADER. It has been said before that NettyTransport is the foundation of cluster communication, but broadcasting does not make it. It uses java's MulticastSocket. Here is a brief introduction to the use of MulticastSocket. It is a socket of UDP mechanism, which is used to broadcast multiple packets. It can help an ip to form a group, any MulticastSocket can join in, and the messages sent by the socket in the group will be received by all machines subscribed to the group. Elasticsearch encapsulated it to form a MulticastChannel, if you are interested, you can refer to the relevant source code.

First, take a look at several auxiliary inner classes of MulticastZenPing:

It defines a total of four inner classes, which work with it to complete the broadcast function. FinalizingPingCollection is a pingresponse container that is used to store all responses. MulticastPingResponseRequestHandler is a response processing class, similar to the nettytransportHandler mentioned earlier. Although it does not use netty, it also defines a messageReceived method that returns a response directly when a request is received.

MulticastPingResponse needn't go into details, it is just a response class. Finally, I would like to focus on the Receiver class, because broadcasts do not use NettyTransport, so the message processing logic is in Receiver. Receiver is registered when MulticastZenPing is initialized.

Protected void doStart () throws ElasticsearchException {try {.... MulticastChannel = MulticastChannel.getChannel (nodeName (), shared, new MulticastChannel.Config (port, group, bufferSize, ttl, networkService.resolvePublishHostAddress (address)), new Receiver (); / / register receiver to channel} catch (Throwable t) {.... }}

The Receiver class is based on Listener and implements three methods. The message is distinguished by the onMessage method. If it is an internal ping, it is processed by the handleNodePingRequest method, otherwise it is processed by handleExternalPingRequest. The distinguishing method is very simple, that is, all the information is read to see whether it conforms to the defined INTERNAL_HEADER header.

Nodeping processing code private void handleNodePingRequest (int id, DiscoveryNode requestingNodeX, ClusterName requestClusterName) {.... Final DiscoveryNodes discoveryNodes = contextProvider.nodes (); final DiscoveryNode requestingNode = requestingNodeX; if (requestingNode.id (). Equals (discoveryNodes.localNodeId () {/ / self-issued ping, ignore return;} / / only accept this cluster ping if (! requestClusterName.equals (clusterName)) {. Return } / / ping if (! discoveryNodes.localNode (). ShouldConnectTo (requestingNode)) {return;} / / create a new response final MulticastPingResponse multicastPingResponse = new MulticastPingResponse (); multicastPingResponse.id = id; multicastPingResponse.pingResponse = new PingResponse (discoveryNodes.localNode (), discoveryNodes.masterNode (), clusterName, contextProvider.nodeHasJoinedClusterOnce ()) between two client / / unable to connect if (! transportService.nodeConnected (requestingNode)) {/ / do the connect and send on a thread pool threadPool.generic () .execute (new Runnable () {@ Override public void run () {/ / connect to the) Node if possible try {transportService.connectToNode (requestingNode) TransportService.sendRequest (requestingNode, ACTION_NAME, multicastPingResponse, new EmptyTransportResponseHandler (ThreadPool.Names.SAME) {@ Override public void handleException (TransportException exp) {logger.warn ("failed to receive confirmation on sent ping response to [{}]", exp, requestingNode) }});} catch (Exception e) {if (lifecycle.started ()) {logger.warn ("failed to connect to requesting node {}", e, requestingNode) }) } else {transportService.sendRequest (requestingNode, ACTION_NAME, multicastPingResponse, new EmptyTransportResponseHandler (ThreadPool.Names.SAME) {@ Override public void handleException (TransportException exp) {if (lifecycle.started ()) {logger.warn ("failed to receive confirmation on sent ping response to [{}]", exp, requestingNode) );}}

Another method is to deal with external ping information, which is to return information about cluster (the exact role of this external ping is not well understood). The above is the process of responding to MulticastZenPing. After receiving the response information from other nodes, it will return the relevant information of the master node of this node and the cluster to the broadcast node. In this way, the broadcast node gets the relevant information about the cluster. There is also a class MulticastPingResponseRequestHandler in the MulticastZenPing class, which is the broadcast node's response to other nodes' response to broadcast information, and the process of broadcasting node's second transmission of information. Like other TransportRequestHandler, it has a messageReceived method that registers with transportserver at startup and deals with only one type of action: "internal:discovery/zen/multicast".

Sending policy of ping request

The code is as follows:

Public void ping (final PingListener listener, final TimeValue timeout) {.... / / generates an id final int id = pingIdGenerator.incrementAndGet (); try {receivedResponses.put (id, new PingCollection ()); sendPingRequest (id) / / send ping request for the first time / / send a request threadPool.schedule (TimeValue.timeValueMillis (timeout.millis () / 2), ThreadPool.Names.GENERIC, new AbstractRunnable () {@ Override public void onFailure (Throwable t) {logger.warn ("[{}] failed to send second ping request", t, id) again after sending 1 / 2 of the waiting time. FinalizePingCycle (id, listener);} @ Override public void doRun () {sendPingRequest (id) / / send a request threadPool.schedule (TimeValue.timeValueMillis (timeout.millis () / 2), ThreadPool.Names.GENERIC again in 1 / 2 time. New AbstractRunnable () {@ Override public void onFailure (Throwable t) {logger.warn ("[{}] failed to send third ping request", t, id) FinalizePingCycle (id, listener);} @ Override public void doRun () {/ / make one last ping, but finalize as soon as all nodes have responded or a timeout has past PingCollection collection = receivedResponses.get (id) FinalizingPingCollection finalizingPingCollection = new FinalizingPingCollection (id, collection, collection.size (), listener); receivedResponses.put (id, finalizingPingCollection); logger.trace ("[{}] sending last pings", id); sendPingRequest (id) / / send the request for the last time ThreadPool.schedule (TimeValue.timeValueMillis (timeout.millis () / 4), ThreadPool.Names.GENERIC, new AbstractRunnable () {@ Override public void onFailure (Throwable t) {logger.warn ("[{}] failed to finalize ping", t, id) after 1 / 4 timeout @ Override protected void doRun () throws Exception {finalizePingCycle (id, listener);}}) );}});} catch (Exception e) {logger.warn ("failed to ping", e); finalizePingCycle (id, listener);}}

The sending process is mainly to call the sendPingRequest (id) method, in which the id, information header, version, and local node information will be written into the BytesStreamOutput and then broadcast. This broadcast information will be received and processed by the Receiver on other machines, and respond to the ping request. Another thing to pay attention to is the part mentioned above. It sends requests periodically during the chain, and may send 4 requests within the waiting time. This transmission method will cause a large number of ping requests to be repeated. Fortunately, the resource consumption of ping is small, but the advantage is that all the new nodes in the cluster can receive this ping message during the timeout period as far as possible. This strategy is also used in unicast.

This is the end of the introduction on "what is the Ping mechanism of elasticsearch cluster zendiscovery". Thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.