Shulou(Shulou.com)06/01 Report--

What is the SPI application and principle of dubbo? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

DubboSPI (Service Provider Interface)

The essence is to configure the fully qualified name of the interface implementation class in the file, and the service loader reads the configuration file and loads the implementation class. This allows you to dynamically replace the implementation class for the interface at run time.

In Java, SPI is designed to be used as plug-ins for service providers. The mechanism of dynamic loading based on policy pattern. We define only one interface in the program, and the specific implementation is handed over to a different service provider; when the program starts, the configuration file is read, and the configuration determines which implementation to call.

Through the SPI mechanism to provide extension function for our program, in dubbo, based on SPI, we can easily expand the Dubbo. For example, protocol,LoadBalance in dubbo is extended through the SPI mechanism.

If you want to learn the source code of Dubbo, you must understand the SPI mechanism. Next, we understand the use of JAVA SPI and dubbo SPI, and then analyze the source code of DUBBO SPI. The dubbo source code of this article is based on version 2.7.5.

JAVA native SPI exampl

First, the application of JAVA SPI is briefly introduced. First, we define a Car interface

Public interface Car {String getBrand ();}

Define two implementation classes for the interface.

Public class BM implements Car {public String getBrand () {System.out.println ("BM car"); return "BM";}} public class Benz implements Car {public String getBrand () {System.out.println ("benz car"); return "Benz";}}

Then create the META-INF/services folder under resources and create a file with the fully qualified name com.dubbo.dp.spi.Car of the Car interface. The content is the fully qualified class name of the interface implementation class.

Com.dubbo.dp.spi.Benzcom.dubbo.dp.spi.BM

Use the following to call all the implementation classes of the Car interface in the configuration file.

Public class JavaSPITest {@ Test public void sayHello () throws Exception {ServiceLoader serviceLoader = ServiceLoader.load (Car.class); serviceLoader.forEach (car-> System.out.println (car.getBrand ());}}

JAVA SPI decouples the definition of interface from the implementation of specific business, and the application process can enable or replace specific components according to the actual business situation.

For example: an interface Driver is defined in the java.sql package of JAVA, and each service provider implements this interface. When we need to use a database, we import the corresponding jar package.

Shortcoming

Cannot be loaded on demand. When Java SPI loads extension points, it will load all available extension points at once, many of which are unnecessary and will waste system resources.

The way to get an implementation class is not flexible enough, it can only be obtained in the form of Iterator, and the corresponding implementation class cannot be obtained according to a parameter.

AOP and dependency injection are not supported.

JAVA SPI may lose the exception information of loading extension points, making it difficult to trace the problem.

Dubbo SPI example

Dubbo reimplements a set of more powerful SPI mechanism, which supports AOP and dependency injection, and uses cache to improve the performance of loading implementation classes, while supporting flexible acquisition of implementation classes. Next, we will talk about the application and principle of SPI.

Dubbo's SPI interface is marked with the @ SPI annotation, and the main purpose of this annotation is to mark that the interface is a SPI interface. The source code is as follows

@ Documented@Retention (RetentionPolicy.RUNTIME) @ Target ({ElementType.TYPE}) public @ interface SPI {/ * default extension name * set the default extension class * / String value () default ";}

This annotation works only on the interface, and value is used to set the default extension class

First of all, I will explain the use of dubbo SPI. Add @ SPI annotation to the above Car interface, and its implementation class remains unchanged, the path and file name of the configuration file remain unchanged, and the file content is adjusted as follows:

@ SPIpublic interface Car {String getBrand ();} benz=com.dubbo.dp.spi.Benzbm=com.dubbo.dp.spi.BM

The configuration file is configured as a key-value pair so that we can load the specified implementation class as needed. The use is as follows:

Public class JavaSPITest {@ Test public void sayHello () throws Exception {ExtensionLoader carExtensionLoader = ExtensionLoader.getExtensionLoader (Car.class); / / get the implementation class object Car car = carExtensionLoader.getExtension ("benz"); System.out.println (car.getBrand ());}}

The output is

Benz carBenzDubbo SPI source code analysis

In the dubbo SPI example method, we first obtain an ExtensionLoader instance of the interface through the getExtensionLoader method of ExtensionLoader, and then obtain the extended class object through the getExtension method of ExtensionLoader. The source code is as follows, first of all, the getExtensionLoader method:

/ * extend the classloader cache, that is, the extension point ExtendsLoader instance cache Key= extension interface value= extension class loader * / private static final ConcurrentMap > EXTENSION_LOADERS = new ConcurrentHashMap (); public static ExtensionLoader getExtensionLoader (Class type) {/ / verify whether the incoming type class is empty if (type = = null) {throw new IllegalArgumentException ("Extension type = = null") } / / verify whether the passed type class is interface if (! type.isInterface ()) {throw new IllegalArgumentException ("Extension type (" + type + ") is not an interface!") } / / verify whether the passed type class has @ SPI annotation if (! withExtensionAnnotation (type)) {throw new IllegalArgumentException ("Extension type (" + type + ") is not an extension, because it is NOT annotated with @" + SPI.class.getSimpleName () + "!") } / / query whether an ExtensionLoader instance of the corresponding type already exists from the ExtensionLoader cache ExtensionLoader loader = (ExtensionLoader) EXTENSION_LOADERS.get (type); if (loader = = null) {/ / does not new an ExtensionLoader instance and stores it in the local cache EXTENSION_LOADERS.putIfAbsent (type, new ExtensionLoader (type)); loader = (ExtensionLoader) EXTENSION_LOADERS.get (type) } return loader;}

GetExtensionLoader verifies the incoming interface, including whether there is @ SPI annotation, which is why @ SPI needs to be added to the interface. Then get the ExtensionLoader of this interface type from the EXTENSION_LOADERS cache, if not, create an ExtensionLoader of this interface type and put it in the cache, and return the ExtensionLoader.

Note that the construction method for creating the ExtensionLoader object here is as follows: ExtensionLoader.getExtensionLoader gets the extension class of the ExtensionFactory interface, and then obtains the target extension class from the extension class through getAdaptiveExtension. It sets the objectFactory constant for this interface to AdaptiveExtensionFactory. Because the @ Adaptive annotation is added to the AdaptiveExtensionFactory class, the reason for AdaptiveExtensionFactory will be explained in a later article, and objectFactory will be used later.

Private ExtensionLoader (Class type) {this.type = type; / / type is not usually an ExtensionFactory class, then objectFactory is the default extension class of the ExtensionFactory interface AdaptiveExtensionFactory objectFactory = (type = = ExtensionFactory.class? Null: ExtensionLoader.getExtensionLoader (ExtensionFactory.class). GetAdaptiveExtension ();}

When the loader Loader of the interface is fetched through ExtensionLoader.getExtensionLoader, the class object that needs to be extended is obtained through the getExtension method. The entire execution process of the method is shown in the following figure

According to the execution flow chart, the source code of the extended class object is as follows:

/ * extension point instance cache key= extension point name: Holder instance of value= extension instance * / private final ConcurrentMap cachedInstances = new ConcurrentHashMap (); / * * obtain API extension class instance * 1. Check to see if * 2 exists in the cache. Create and return the key * @ return * / public T getExtension (String name) {if (StringUtils.isEmpty (name)) {throw new IllegalArgumentException ("Extension name = = null") of the extension class in the configuration file to be obtained by creating and returning the instance of the extension class * @ throw new IllegalArgumentException } if ("true" .equals (name)) {/ / get the default implementation class on the @ SPI annotation, such as @ SPI ("benz") return getDefaultExtension ();} / / Holder, as the name implies, is used to hold the target object, take it from the cache, and create final Holder holder = getOrCreateHolder (name) if not. Object instance = holder.get (); / / double check if (instance = = null) {synchronized (holder) {instance = holder.get (); if (instance = = null) {/ / create an extension instance instance = createExtension (name) / / set the instance to holder holder.set (instance);} return (T) instance } / * get or create a Holder object * / private Holder getOrCreateHolder (String name) {/ / first get the Holder object Holder holder = cachedInstances.get (name) from the extended instance cache through the extension If (holder = = null) {/ / if not obtained, an empty Holder instance of new is stored in the cache cachedInstances.putIfAbsent (name, new Holder (); holder = cachedInstances.get (name);} return holder;}

The logic of the above code is relatively simple, first check the cache, and if the cache misses, an extension object is created. Dubbo contains a large number of extension point caches. This is a typical practice of using space for time. It is also one of the reasons for the strong performance of Dubbo, including

Extension point Class cache. When getting the extension point, Dubbo SPI will first read from the cache, load the configuration file if it does not exist in the cache, cache the Class into memory according to the configuration, and will not initialize it directly.

Extension point instance cache, Dubbo caches not only Class, but also instances of Class. Each time the instance is fetched, it is preferentially fetched from the cache. If it is not available, it is loaded from the configuration, instantiated and cached in memory.

Let's take a look at the process of creating an extended object.

/ * extended instances are cached in memory; key= extension class Value= extension class instance * / private static final ConcurrentMap clazz = getExtensionClasses (). Get (name); if (clazz = = null) {throw findException (name);} try {/ / get the corresponding instance object T instance = (T) EXTENSION_INSTANCES.get (clazz) from the extension point cache If (instance = = null) {/ if there is no such extension point in the cache, create an instance through reflection and store it in the cache EXTENSION_INSTANCES.putIfAbsent (clazz, clazz.newInstance ()); / / then get the corresponding instance instance = (T) EXTENSION_INSTANCES.get (clazz) from the cache } / / injecting dependencies into the instance, automatically injecting the corresponding attribute instance injectExtension (instance) through the setter method; / / fetching all the wrapper classes from the cache to form the wrapper chain Set > > cachedClasses = new Holder () / * parse the extension name of the interface in the configuration file and the mapping table of the extension class map * @ return * / private Map > classes = cachedClasses.get (); / / double check if (classes = = null) {synchronized (cachedClasses) {classes = cachedClasses.get () If (classes = = null) {/ / load the extension class classes = loadExtensionClasses (); cachedClasses.set (classes);} return classes;}

Here, the cache is also checked first. If the cache misses, the extension class is loaded through loadExtensionClasses, which avoids the time-consuming of reading the configuration file multiple times. The logic of loading the configuration file with the loadExtensionClasses method is analyzed below

/ * * three default scan paths for dubbo SPI * / private static final String SERVICES_DIRECTORY = "META-INF/services/"; private static final String DUBBO_DIRECTORY = "META-INF/dubbo/"; private static final String DUBBO_INTERNAL_DIRECTORY = DUBBO_DIRECTORY + "internal/"; private Map > extensionClasses = new HashMap () / / load the configuration file under the specified folder. Constant contains three folders of META-INF/dubbo/internal/,META-INF/dubbo/,META-INF/services/: loadDirectory (extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName (), true); / / compatible historical version loadDirectory (extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName (). Replace ("org.apache", "com.alibaba"), true) LoadDirectory (extensionClasses, DUBBO_DIRECTORY, type.getName ()); loadDirectory (extensionClasses, DUBBO_DIRECTORY, type.getName (). Replace ("org.apache", "com.alibaba")); loadDirectory (extensionClasses, SERVICES_DIRECTORY, type.getName ()); loadDirectory (extensionClasses, SERVICES_DIRECTORY, type.getName (). Replace ("org.apache", "com.alibaba"); return extensionClasses;}

The loadExtensionClasses method does a total of two things. First, the method calls cacheDefaultExtensionName to parse the SPI annotation, getting and caching the default extension class on the @ SPI annotation of the interface in cachedDefaultName. Then call the loadDirectory method to load the specified folder configuration file.

SPI annotation parsing process is relatively simple, the source code is as follows. Only one default extension class is allowed.

Private void cacheDefaultExtensionName () {/ / get SPI comments, where the type variable is passed in when the getExtensionLoader method is called, representing the interface class final SPI defaultAnnotation = type.getAnnotation (SPI.class); if (defaultAnnotation = = null) {return;} String value = defaultAnnotation.value () If ((value = value.trim ()). Length () > 0) {String [] names = NAME_SEPARATOR.split (value); / / check whether the SPI annotation content is legal (at most one default implementation class), and throw an exception if (names.length > 1) {throw new IllegalStateException ("...") } / / set the default extension class name if (names.length = = 1) {cachedDefaultName = names [0];}

From the source code, you can see that there are three paths for the loadExtensionClasses method to load the configuration file, which are three folders of META-INF/dubbo/internal/,META-INF/dubbo/,META-INF/services/. The source code of the method is as follows:

Private void loadDirectory (Map > extensionClasses, String dir, String type, boolean extensionLoaderClassLoaderFirst) {/ / fileName = folder path + type fully qualified name String fileName = dir + type; try {Enumeration urls = null; / / get the current thread's classloader ClassLoader classLoader = findClassLoader () / / try to load from ExtensionLoader's ClassLoader first if (extensionLoaderClassLoaderFirst) {/ / get the classloader ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader () that loads the class ExtensionLoader.class / / if extensionLoaderClassLoaderFirst=true and the two classloaders are different, extensionLoaderClassLoader if (ClassLoader.getSystemClassLoader ()! = extensionLoaderClassLoader) {urls = extensionLoaderClassLoader.getResources (fileName) is preferred. }} / / load all files with the same name if (urls = = null | |! urls.hasMoreElements ()) {if (classLoader! = null) {urls = classLoader.getResources (fileName);} else {urls = ClassLoader.getSystemResources (fileName) based on the file name }} if (urls! = null) {while (urls.hasMoreElements ()) {java.net.URL resourceURL = urls.nextElement (); / / parse and load the implementation classes configured in the configuration file to extensionClasses and loadResource (extensionClasses, classLoader, resourceURL) } catch (Throwable t) {logger.error (""). T);}}

First of all, find the configuration file under the folder. The file name needs to be fully qualified for the interface. Use the class loader to get the file resource link, and then parse the implementation class configured in the configuration file and add it to the extensionClasses. Let's move on to see how loadResource loads resources.

Private void loadResource (Map > extensionClasses, java.net.URL resourceURL, Class clazz, String name) throws NoSuchMethodException {/ / determines whether the configured implementation class implements the type interface if (! type.isAssignableFrom (clazz)) {throw new IllegalStateException ("...") } / / classify and cache according to the type of implementation class in the configuration / / detect whether there is an Adaptive comment on the target class, indicating that the class is an adaptive implementation class, cached to cachedAdaptiveClass if (clazz.isAnnotationPresent (Adaptive.class)) {cacheAdaptiveClass (clazz) / / check whether clazz is of Wrapper type, based on whether any parameter is the constructor of the API class, cache it to cachedWrapperClasses} else if (isWrapperClass (clazz)) {cacheWrapperClass (clazz);} else {/ / detect whether clazz has a default constructor, and if not, throw an exception clazz.getConstructor () / / if the name of key in the configuration file is empty, try to get the name from the Extension comment, or use a lowercase class name as the name. If (StringUtils.isEmpty (name)) {name = findAnnotationName (clazz); if (name.length () = = 0) {throw new IllegalStateException ("...") is not discussed if it has been deprecated. }} / / use commas to split name into string arrays String [] names = NAME_SEPARATOR.split (name) If (ArrayUtils.isNotEmpty (names)) {/ / if the implementation class of the extension point configuration uses the @ Activate annotation, cache the corresponding annotation information to cacheActivateClass (clazz, names [0]) For (String n: names) {/ / cache extension point to implement the correspondence between class class and extension point name cacheName (clazz, n); / / finally store class in extensionClasses saveInExtensionClass (extensionClasses, clazz, n);}}

LoadClass method implements the classification cache function of extension point, such as wrapper class, adaptive extension point implementation class, general extension point implementation class and so on. It should be noted that the adaptive extension point implementation class @ Adaptive annotation, the source code of which is as follows

* For example, given String [] {"key1", "key2"}: * * find parameter 'key1' in URL, use its value as the extension's name * try' key2' for extension's name if 'key1' is not found (or its value is empty) in URL * use default extension if' key2' doesn't exist either * otherwise Throw {@ link IllegalStateException} * @ return * / @ Documented @ Retention (RetentionPolicy.RUNTIME) @ Target ({ElementType.TYPE, ElementType.METHOD}) public @ interface Adaptive {String [] value () default {} }

The purpose of this annotation is to determine which adaptive extension class is injected. The target extension class is determined by the parameter in URL. The parameter key in URL is given by the value of the annotation, and the value of the key is used as the name of the target extension class.

If there are multiple values in the annotation, go to the URL from small to large according to the subscript to find out if there is a corresponding key. Once found, use the value of the key as the target extension class name.

If none of these values have a corresponding key in url, use the default values on spi.

@ Adaptive annotation can work on classes and methods. In most cases, the annotation acts on methods. When Adaptive annotations are on a class, Dubbo will not generate a proxy class for that class. When annotated on a method (interface method), Dubbo generates a proxy class for that method. Adaptive notes on interface methods, indicating that the extended loading logic needs to be automatically generated by the framework. Note on the class, indicating that the loading logic of the extension is manually coded.

The above loadClass scans for acting on classes. In Dubbo, only two classes are annotated with @ Adaptive, AdaptiveCompiler and AdaptiveExtensionFactory.

Setting the cache cacheAdaptiveClass by the loadClass method will cause the cacheAdaptiveClass of the interface to be not empty, and this extension class will be used by default, with the highest priority.

Back to the mainline, when the loadClass method is executed, all the extension classes in the configuration file have been loaded into map, and the cache class loading process is analyzed.

Dubbo IOC

When the getExtensionClasses () method execution process is complete, the extension class Class can be obtained by fetching the corresponding extension class from the map according to the extension name, and the instance is created through reflection and through injectExtension (instance); the method injects dependency into the instance, which will be introduced in the next article.

DUBBO AOP

When the injectExtension (T instance) method is executed, the wrapper of wrapper is executed in createExtension (String name), similar to the decorator pattern used in AOP,dubbo in spring.

Set

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.