Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how Dubbo realizes the dynamic discovery of services". In the daily operation, I believe that many people have doubts about how to realize the dynamic discovery of services by Dubbo. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts of "how to achieve dynamic discovery of services by Dubbo". Next, please follow the editor to study!

Ps: the following abbreviates ZooKeeper to zk.

1. Dubbo zk data structure

In the article on basic concept sharing of ZooKeeper, the interior of ZK is a tree hierarchy, and there are many types of nodes. Dubbo, on the other hand, only creates persistent and temporary nodes.

If the service provider service interface is com.service.FooService, the following path / dubbo/com.service.FooService/providers/providerURL will be created in ZK.

The service path is divided into four layers, and the root node defaults to dubbo, which can be changed by setting the group attribute in dubbo-registry.

Ps: if there is no registry isolation requirement, do not modify it casually.

The second layer node is the full name of the service node, such as com.service.FooService.

The third layer node is the service directory, such as providers. There are also other directory nodes, namely consumers (consumer directory), configurators (configuration directory), and routers (routing directory). The following service subscriptions are mainly targeted at this layer of nodes.

The fourth node is a specific service node, and the node name is a specific URL string, such as dubbo://2.0.1.13:12345/com.dubbo.example.DemoService?xx=xx, which is temporary by default. An example of the dubbo ZK tree internal structure is:

Specific examples of ZK internal services are as follows:

2. RegistryFactory implementation

Dubbo can specify the use of the registry in the configuration file, you can use dubbo.registry.protocol to specify a specific registry type, or you can set the dubbo.registry.address assignment. The registry-related implementation will be created using the RegistryFactory factory class.

The source codes of RegistryFactory API are as follows:

@ SPI ("dubbo") public interface RegistryFactory {@ Adaptive ({"protocol"}) Registry getRegistry (URL url);}

The RegistryFactory interface method uses the @ Adaptive annotation, and here the Dubbo SPI mechanism will be used to automatically generate some implementation logic of the code. Here you will call the final implementation subclass based on the protocol attribute in URL.

The RegistryFactory implementation subclass is shown in the figure:

AbstractRegistryFactory will implement the getRegistry method of the interface, mainly complete locking, and call the abstract template method createRegistry to create a concrete registry implementation class and cache it in memory.

The AbstractRegistryFactory#getRegistry source code is as follows:

Public Registry getRegistry (URL url) {url = URLBuilder.from (url) .setPath (RegistryService.class.getName ()) .addParameter (Constants.INTERFACE_KEY, RegistryService.class.getName ()) .removeParameters (Constants.EXPORT_KEY, Constants.REFER_KEY) .build (); String key = url.toServiceStringWithoutResolving () / / Lock to prevent concurrency of LOCK.lock (); try {/ / first fetch Registry registry = REGISTRIES.get (key) from cache; if (registry! = null) {return registry;} / / create registry = createRegistry (url) using Dubbo SPI base If (registry = = null) {throw new IllegalStateException ("Can not create registry" + url);} / / put into cache REGISTRIES.put (key, registry); return registry;} finally {/ / Release the lock LOCK.unlock ();}}

The registry instance will be created through a specific factory class. Here we take a look at the ZookeeperRegistryFactory source code:

Public class ZookeeperRegistryFactory extends AbstractRegistryFactory {private ZookeeperTransporter zookeeperTransporter; / * through Dubbo SPI binary injection * @ param zookeeperTransporter * / public void setZookeeperTransporter (ZookeeperTransporter zookeeperTransporter) {this.zookeeperTransporter = zookeeperTransporter;} @ Override public Registry createRegistry (URL url) {return new ZookeeperRegistry (url, zookeeperTransporter);}}

The ps:Dubbo SPI mechanism also has IOC features. For the ZookeeperTransporter injection here, please refer to: Dubbo extension point loading

III. Source code analysis of zk module

After talking about the process of creating a registry instance, let's go deep into the ZookeeperRegistry source code.

ZookeeperRegistry inherits the FailbackRegistry abstract class, so it needs to implement its parent abstract template method. The following is mainly about doRegister and doSubscribe source code.

3.1 doRegister

The service provider needs to register the service with the registry, the purpose of registration is to make consumers aware of the existence of the service so as to initiate remote invocation, on the other hand, it also makes the service governance center aware of the new service provider online. The zk module service registration code is relatively simple, using the zk client directly to create a node in the registry.

The source code for ZookeeperRegistry#doRegister implementation is as follows:

Public void doRegister (URL url) {try {zkClient.create (toUrlPath (url), url.getParameter (Constants.DYNAMIC_KEY, true));} catch (Throwable e) {throw new RpcException ("Failed to register" + url + "to zookeeper" + getUrl () + ", cause:" + e.getMessage (), e);}}

The zkClient.create method needs to pass in two parameters.

Void create (String path, boolean ephemeral)

The first parameter is the node path, and the URL instance will be converted to the path format in ZK through toUrlPath. The conversion result is as follows:

# # URL before conversion is as follows: dubbo://10.20.82.31:12345/com.dubbo.example.DemoService## calls `toUrlPath` after conversion / dubbo/com.dubbo.example.DemoService/providers/dubbo%3A%2F%2F10.20.82.31%3A12345%2Fcom.dubbo.example.DemoService

The second parameter mainly determines that the ZK node type is mainly taken from the dynamic parameter value in the URL instance object. If it does not exist, it defaults to true, that is, a temporary node will be created by default.

The zkClient.create method will be called recursively. First, if the parent node exists or not, it will be created until the last node jumps out of the recursive method.

Public void create (String path, boolean ephemeral) {/ / before creating a permanent node, you need to determine whether if (! ephemeral) {if (checkExists (path)) {return;}} / / determines whether there is a parent node int I = path.lastIndexOf ('/') If (I > 0) {/ / Recursively create parent node create (path.substring (0, I), false);} if (ephemeral) {/ / create temporary node createEphemeral (path);} else {/ / create permanent node createPersistent (path);}}

Finally, the actual node creation operation of createEphemeral and createPersistent will be handed over to the ZK client class. The implementation here is relatively simple, and you can refer to the source code.

Ps: dubbo defaults the zk client to Curator since 2.6.1, while previous versions used zkclient. Dubbo 2.7.1 begins to remove the zkclient implementation, which means that only Curator can be used.

3.2 Why dubbo service provider nodes use zk temporary nodes

The zk temporary node will be automatically deleted after the zk client is disconnected. The dubbo service provider goes offline normally, and it will actively delete the zk service node.

If the service is unusually down, the zk service node cannot be deleted normally, which causes the failed service to exist on the ZK all the time, and the consumer will call the failed node, causing the consumer to report an error. Through the zk temporary node characteristics, let the zk server actively delete the failure node, thus offline the failure service.

4. DoSubscribe: the principle of service dynamic discovery 4.1 the basic principle of subscription

There are usually two ways to subscribe to services: pull and push. The pull mode requires the client to regularly pull the configuration from the registry, while the push mode uses the registry to actively push data to the client.

The dubbo zk registry uses event notification and client pull. When the service subscribes for the first time, it will pull all the data under the corresponding directory, and then register a watcher with the subscribed node. Once any data changes occur under the directory node, zk will notify the client through watcher. The client is notified that it will re-pull all the data in the directory and re-register watcher. Using this pattern, dubbo services can achieve dynamic service discovery.

4.2 Source Code parsing

Finish talking about the basic principles of subscription, and then go deep into the source code.

The doSubscribe method needs to pass in two parameters, one for the URL instance and the other for NotifyListener, to change the listener for the event. The method is divided into two parts of logic according to the type of URL interface, full subscription service and partial category subscription service.

The overall source logic of doSubscribe method:

Public void doSubscribe (final URL url, final NotifyListener listener) {if (Constants.ANY_VALUE.equals (url.getServiceInterface () {/ / full subscription logic} else {/ / partial category subscription logic}

The Service Governance Center (dubbo-admin) needs to subscribe to all service APIs to sense the status of each service, so before subscribing, service will be set to * to handle all service.

Service consumers or service providers will subscribe to services in some categories, so let's go deep into the subsequent source code from the consumer's point of view.

At the beginning of the article, we talked about the four types of zk directory nodes. Here, the path of the subscription node will be determined according to the value of subscription in URL.

The category value in the service provider URL defaults to configurators and the category value in the consumer URL defaults to providers,configurators,routers. If the category category value is *, four category paths will be subscribed, otherwise only paths of type providers will be subscribed.

The toCategoriesPath source code is as follows:

Private String [] toCategoriesPath (URL url) {String [] categories; / / if category is *, subscribe to four types of full data if (Constants.ANY_VALUE.equals (url.getParameter (Constants.CATEGORY_KEY) {categories = new String [] {Constants.PROVIDERS_CATEGORY, Constants.CONSUMERS_CATEGORY, Constants.ROUTERS_CATEGORY, Constants.CONFIGURATORS_CATEGORY} } else {categories = url.getParameter (Constants.CATEGORY_KEY, new String [] {Constants.DEFAULT_CATEGORY});} / / return path array String [] paths = new String [categories.length]; for (int I = 0; I

< categories.length; i++) { paths[i] = toServicePath(url) + Constants.PATH_SEPARATOR + categories[i]; } return paths; } 接着循环路径数组，循环内将会缓存节点监听器，用以提高性能。 // 循环路径数组 for (String path : toCategoriesPath(url)) { ConcurrentMap listeners = zkListeners.get(url); // listeners 缓存为空，创建缓存 if (listeners == null) { zkListeners.putIfAbsent(url, new ConcurrentHashMap()); listeners = zkListeners.get(url); } ChildListener zkListener = listeners.get(listener); // zkListener 缓存为空则创建缓存 if (zkListener == null) { listeners.putIfAbsent(listener, (parentPath, currentChilds) ->

ZookeeperRegistry.this.notify (url, listener, toUrlsWithEmpty (url, parentPath, currentChilds)); zkListener = listeners.get (listener);} / create subscription node zkClient.create (path, false); / / use ZK client subscription node List children = zkClient.addChildListener (path, zkListener) If (children! = null) {/ / stores full URL urls.addAll (toUrlsWithEmpty (url, path, children));}} / / callback NotifyListener notify (url, listener, urls)

Eventually, you will use CuratorClient.getChildren () .usingWatcher (listener) .forPath (path) to register the watcher with the ZK node and get all the child node data under the directory node.

Here watcher uses the Curator interface CuratorWatcher. Once the ZK node changes, the CuratorWatcher#process method will be called back.

The source code of CuratorWatcher#process method is as follows:

Public void process (WatchedEvent event) throws Exception {if (childListener! = null) {String path = event.getPath () = = null? ": event.getPath () ChildListener.childChanged (path, / / reset watcher and get all child nodes StringUtils.isNotEmpty (path) under the node? Client.getChildren () .usingWatcher (this) .forPath (path): Collections.emptyList ();}}

The sequence chart of consumer subscription is as follows:

4.3 listener diagram

We have come across several listener classes in the subscription method, which may be a bit messy at first. You can refer to the following diagram to clarify the relationship.

The listener diagram is as follows:

The callback relationship is shown in the figure:

4.4 there are problems with ZK module subscription

ZK will get all the child nodes under the directory node for the first subscription, and any subsequent child node changes will be notified through watcher callback. The callback notification will pull all the child nodes under the node directory again. In this way, full pull will have a limitation, when there are more service nodes, it will cause a lot of pressure on the network.

After Dubbo 2.7, metadata center is introduced to solve this problem. For more information, Ali technical experts explain the practice, evolution and future planning of Dubbo.

A solution cited in the article is shown in the following figure:

At this point, the study on "how Dubbo implements dynamic service discovery" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.