Shulou(Shulou.com)06/01 Report--

How to use Stevedore to realize the dynamic management of plug-ins in your own application? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Demonstrate step by step how to define plug-ins, and then use them to load and use them in your application.

Naming plug-in Guide

Stevedore uses the setuptools entry point to define and load plug-ins. Entry point is a standard way to name objects defined in a Python module or package. The name can be a reference to any class, function, or instance, as long as it is created when the containing module is imported. (that is, it requires a module-level global)

Name and namespace

Entry point registers with the name in the namespace.

The entry point name is generally considered to be visible to the user. For example, they often appear in the configuration file, which is where the driver is started. Because they are public, the names are usually as short as possible while remaining descriptive. For example, the name of the database driver plug-in may be mysql, postgresql, sqlite, etc.

On the other hand, namespaces are an implementation detail, and although they are known to developers, they are usually not exposed to users. The naming syntax of namespaces looks a lot like Python's package syntax (a.b.c), but namespaces do not correspond to Python packages. Using Python package names for entry point namespaces is an easy way to ensure unique names, but this is not necessary at all. The main feature of entry point is that they can be found across packages. This means that a plug-in can be developed and installed completely separate from the application, as long as they agree to namespaces and API.

Each namespace is owned by the code that uses the plug-in and is used to search for entry points. Entry point names are usually owned by plug-ins, but they can also be defined by code that names hooks (see HookManager). The name of the entry point must be unique in a given distribution, but not necessarily in the namespace.

Create a plug-in

After trial and error, I found that the easiest way to define API is to follow these steps:

Use the abc module to create a basic abstract class to define the behavior required by the API plug-in. Developers do not need to subclass from the base class, but it provides a convenient way to record API and uses abstract base classes to keep the code reliable.

Create a plug-in by subclassing the base class and implementing the required methods.

Define a unique namespace for each API by combining the name of the application (or library) with the name of the API. Keep it simple, for example: "cliff.formatters" or "ceilometer.pollsters.compute".

Sample plug-in set

The sample program in this tutorial will create a set of plug-ins with several data formatters, such as what command-line programs can use to prepare to print data to the console. Each formatter will be used as input, using string keys and built-in data types as values. It returns an iterator as output that generates the data structure of the data structure according to the rules of the specific formatter used. The constructor of formatter allows the caller to specify the maximum width that the output should have.

The base class of a plug-in

Step 1 above defines an abstract base class for the API that needs to be implemented by each plug-in.

# stevedore/example/base.pyimport abcimport six@six.add_metaclass (abc.ABCMeta) class FormatterBase (object): "Base class for example plugin used in the tutorial." Def _ init__ (self, max_width=60): self.max_width = max_width @ abc.abstractmethod def format (self, data): "Format the data and return unicode text.: param data: A dictionary with string keys and simple types as values.: type data: dict (str:?): returns: Iterable producing the formatted text."

The constructor is a concrete method because the subclass does not need to override it, but the format () method does nothing useful because there is no default implementation available.

Inherit and implement the plug-in base class

The next step is to create several plug-in classes with a concrete implementation of format (). A simple example formatter uses each variable name and value in one line to generate output.

# stevedore/example/simple.pyfrom stevedore.example import baseclass Simple (base.FormatterBase): "A very basic formatter. Def format (self, data): "" Format the data and return unicode text. : param data: A dictionary with string keys and simple types as values. : type data: dict (str:?) "" for name, value in sorted (data.items ()): line ='{name} = {value}\ n'.format (name=name, value=value,) yield line

There are many other formatting options, but this example will give us enough work to demonstrate registering and using plug-ins.

Register plug-in

To use setuptools entry point, you must package your application or library with setuptools. The build and packaging process generates metadata, which can be found after installation to find the plug-ins provided by each python distribution.

Entry point must be declared to belong to a specific namespace, so we need to select one before moving on to the next step. These plug-ins are examples from stevedore, so I'll use the "stevedore.example.formatter" namespace. You can now provide all the necessary information in the packaging instructions:

# stevedore/example/setup.pyfrom setuptools import setup, find_packagessetup (name='stevedore-examples', version='1.0', description='Demonstration package for stevedore', author='Doug Hellmann', author_email='doug@doughellmann.com', url=' http://git.openstack.org/cgit/openstack/stevedore', classifiers= ['Development Status:: 3-Alpha',' License:: OSI Approved:: Apache Software License' 'Programming Language:: Python',' Programming Language:: Python:: 2percent, 'Programming Language:: Python:: 2.7percent,' Programming Language:: Python:: 3percent, 'Programming Language:: Python:: 3.4 percent,' Intended Audience:: Developers' 'Environment:: Console',], platforms= [' Any'], scripts= [], provides= ['stevedore.examples',], packages=find_packages (), include_package_data=True, entry_points= {' stevedore.example.formatter': ['simple = stevedore.example.simple:Simple' 'plain = stevedore.example.simple:Simple',],}, zip_safe=False,)

The most important thing at the bottom is to set entry point for setup (). This value is a dictionary that maps the namespaces of plug-ins to their list of definitions. Each item in the list should be a form with name=module:importable, this name is visible to the user, module is the Python import reference of the module, and importable is the name that can be imported from within the module, as follows:

'simple = stevedore.example.simple:Simple',' plain = stevedore.example.simple:Simple',],}, zip_safe=False,)

In this example, two plug-ins are registered. The simple plug-in defined above and a normal plug-in are just aliases for simple plug-ins.

Setuptools metadata

During the build, setuptools copies the entry point definition for the package to a file in the ".egg-info" directory. For example, the entry point of the stevedore example is located in stevedore.egg-info/entry_points.txt and is as follows:

[stevedore.example.formatter] simple = stevedore.example.simple:Simpleplain = stevedore.example.simple: simple [stevedore.test.extension] T2 = stevedore.tests.test_extension:FauxExtensiont1 = stevedore.tests.test_extension:FauxExtension

Pkg_resources uses entry_points.txt to find plug-ins from all software installed into the environment. You should not modify these files, except to change the entry point list in setup.py.

Add plug-ins to other packages

The attraction of plug-ins in addition to entry points is that they can be distributed independently of the application. The setuptools namespace is used to distinguish between plug-ins and Python source code namespaces. You typically use a plug-in namespace prefixed with the name of the application or library that loads the plug-in to ensure that it is unique, but this name has no effect on how the code of the Python package should survive.

For example, we can add another implementation of a formatter plug-in that generates a reStructuredText field list

# stevedore/example2/fields.pyimport textwrapfrom stevedore.example import baseclass FieldList (base.FormatterBase): "Format values as a reStructuredText field list. For example::: name1: value: name2: value: name3: a long value will be wrapped with a hanging indent "" def format (self, data): "Format the data and return unicode text. : param data: A dictionary with string keys and simple types as values. : type data: dict (str:?) "" for name, value in sorted (data.items ()): full_text =': {name}: {value} '.format (name=name, value=value,) wrapped_text = textwrap.fill (full_text, initial_indent='', subsequent_indent='') Width=self.max_width,) yield wrapped_text +'\ n'

New plug-ins can be packaged using setup.py

# stevedore/example2/setup.pyfrom setuptools import setup, find_packagessetup (name='stevedore-examples2', version='1.0', description='Demonstration package for stevedore', author='Doug Hellmann', author_email='doug@doughellmann.com', url=' http://git.openstack.org/cgit/openstack/stevedore', classifiers= ['Development Status:: 3-Alpha',' License:: OSI Approved:: Apache Software License' 'Programming Language:: Python',' Programming Language:: Python:: 2percent, 'Programming Language:: Python:: 2.7percent,' Programming Language:: Python:: 3percent, 'Programming Language:: Python:: 3.4 percent,' Intended Audience:: Developers' 'Environment:: Console',], platforms= [' Any'], scripts= [], provides= ['stevedore.examples2',], packages=find_packages (), include_package_data=True, entry_points= {' stevedore.example.formatter': ['field = stevedore.example2.fields:FieldList',],}, zip_safe=False,)

The new plug-in is in a separate package called stevedore-examples2

Setup (name='stevedore-examples2'

This plug-in is also registered in the stevedore.example.formatter namespace

'stevedore.example.formatter': [' field = stevedore.example2.fields:FieldList',],}

When the plug-in namespace is scanned, all packages in the current PYTHONPATH are detected, the entry point from the second package can be found and loaded, and the application does not need to know exactly where the plug-in is installed.

Load plug-in

There are several different ways to enable calls when using plug-ins, depending on your needs.

Load driver

The most common way is for plug-ins to act as separate drivers. In this case, there are usually multiple plug-ins to choose from, but only one needs to be loaded and invoked. DriverManager can support this approach. The following example is implemented using DriverManager

# stevedore/example/load_as_driver.pyfrom _ _ future__ import print_functionimport argparsefrom stevedore import driverif _ _ name__ ='_ _ main__': parser = argparse.ArgumentParser () parser.add_argument ('format', nargs='?', default='simple', help='the output format',) parser.add_argument ('-- width', default=60, type=int Help='maximum output width for text',) parsed_args = parser.parse_args () data = {'await:' Agar, 'baked:' baked, 'long':' word'* 80,} mgr = driver.DriverManager (namespace='stevedore.example.formatter', name=parsed_args.format, invoke_on_load=True Invoke_args= (parsed_args.width,),) for chunk in mgr.driver.format (data): print (chunk, end='')

Manager takes the namespace and name of the plug-in as parameters and uses them to find the plug-in. Because invoke_on_load calls true, it invokes object loading. In this case, the object is the plug-in class that is registered as formatter. Invoke_args is an optional parameter that is passed into the class constructor to set the maximum width parameter.

Mgr = driver.DriverManager (namespace='stevedore.example.formatter', name=parsed_args.format, invoke_on_load=True, invoke_args= (parsed_args.width,),)

After manager is created, it returns an object by calling the code registered as a plug-in. This object is the actual driver, in this case an instance of the formatter class from the plug-in. You can access a single driver through the manager's driver, and then you can call its methods directly.

For chunk in mgr.driver.format (data): print (chunk, end='')

Running the sample program produces this output

$python-m stevedore.example.load_as_driver a = Ab = Blong = word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word$ python-m stevedore.example.load_as_driver field: a: a: B: B : long: word word word word$ python-m stevedore.example.load_as_driver field-- width 30: a: a: B: B: long: word word Word word word load extension

Another common use case is to load multiple extensions at a time and then call them. Several other manager classes support this invocation mode, including: ExtensionManager, NamedExtensionManager, and EnabledExtensionManager.

# stevedore/example/load_as_extension.pyfrom _ _ future__ import print_functionimport argparsefrom stevedore import extensionif _ _ name__ ='_ _ main__': parser = argparse.ArgumentParser () parser.add_argument ('--width', default=60, type=int, help='maximum output width for text',) parsed_args = parser.parse_args () data = {'a':'A' Mgr = extension.ExtensionManager (namespace='stevedore.example.formatter', invoke_on_load=True, invoke_args= (parsed_args.width,),) def format_data (ext, data): return (ext.name, ext.obj.format (data)) results = mgr.map (format_data, data) for name Result in results: print ('Formatter: {0}' .format (name)) for chunk in result: print (chunk, end='') print ('')

The creation of ExtensionManager is slightly different from that of DriverManager because it does not need to know which plug-in to load in advance. It loads all the plug-ins it finds.

Mgr = extension.ExtensionManager (namespace='stevedore.example.formatter', invoke_on_load=True, invoke_args= (parsed_args.width,))

Call the plug-in, using the map () method, passing a callback method to invoke each plug-in. Format_data () takes two parameters.

Def format_data (ext, data): return (ext.name, ext.obj.format (data)) results = mgr.map (format_data, data)

A class defined by stevedore as the Extension parameter passed into format_data (). It contains the name of the plug-in, the EntryPoint returned by pkg_resources, and the plug-in itself. When invoke_on_load is true, Extension will have an object property that contains the value returned when the plug-in is called. Map () returns the sequence of values returned by the callback function. In this case, format_data () returns a tuple that contains a plug-in name and an iterable. When processing the results, the name of each plug-in is printed, followed by formatted data.

For name, result in results: print ('Formatter: {0}' .format (name)) for chunk in result: print (chunk, end='') print ('')

The order in which the plug-ins are loaded is undefined and depends on the order packages found on the import path and how the metadata files are read. If you use the order extension, try using NamedExtensionManager.

$python-m stevedore.example.load_as_extension-- width 30Formatter: simplea = Ab = Blong = word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word wordFormatter: field: a: a: B: B: long: word word word Word word word word word word word word word word word word word word word word wordFormatter: plaina = Ab = Blong = word word word word word word word word word word word word word word word word word word word word Why doesn't word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word call the plug-in directly?

Using a separate callable parameter map () instead of calling the plug-in directly results in a separation between the application code and the plug-in. The benefits of this separation are reflected in application code design and plug-in API design.

If map () calls the plug-in directly, each plug-in must be callable. This will mean a separate namespace, which is really just a plug-in method. By using a separate callable parameter, the plug-in API does not need to match any specific use cases in the application. This allows you to create a finer-grained API that can be called in different ways to achieve different goals using more methods.

This is the answer to the question about how to use Stevedore to implement the dynamic management of plug-ins in your own application. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.