Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to automatically import missing libraries in Python". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to import missing libraries automatically in Python.

When writing Python projects, we may often encounter errors that import modules fail: ImportError: No module named 'xxx' or ModuleNotFoundError: No module named' xxx'.

The problem of import failure is usually divided into two types: one is to import the module written by yourself (that is, the file with the suffix .py), and the other is to import the third-party library. This article mainly discusses the second situation. When we have the opportunity in the future, we will discuss other related topics in detail.

To solve the problem of failure to import Python libraries, the key is to install the missing libraries in the runtime environment (note whether it is a virtual environment), or to use the appropriate alternative. This problem can be divided into three situations:

I. missing libraries in a single module

When writing code, if we need to use a tripartite library (such as requests), but are not sure whether the actual running environment has it installed, we can write like this:

The effect of writing like this is that if you can't find the requests library, install it first and then import it.

In some open source projects, we may also see the following (take json as an example):

The effect of writing like this is to import the tripartite library simplejson first, and if you can't find it, use the built-in standard library json.

The advantage of this method of writing is that there is no need to import additional libraries, but it has the disadvantage of ensuring that the two libraries are compatible in use, and it is not feasible if an alternative library is not found in the standard library.

If you really can't find a compatible standard library, you can write your own module (such as my_json.py), implement what you want, and then import it in the except statement.

Second, the missing libraries in the whole project

The above idea is aimed at the project under development, but it has several shortcomings: 1, it is not advisable to pip install every possible missing tripartite library in the code; 2, a tripartite library cannot be replaced by a standard library or your own handwritten library, what should I do? 3. What if these changes are not allowed for projects that have already been formed?

So the question here is: what if there is a project that wants to deploy to a new machine, which involves a lot of three-party libraries, but there is no preinstalled on the machine?

For a compliant project, by convention, it usually contains a "requirements.txt" file that records all dependent libraries for the project and their required version numbers. This is generated using the command pip freeze > requirements.txt before the project is released.

Using the command pip install-r requirements.txt (execute in the same directory as the file, or write the path to the full file in the command), you can automatically install all dependent libraries.

But what if the project is not compliant, or due to other unfortunate reasons, we do not have such a document?

A stupid way is to run the project, wait for it to go wrong, and install one manually when a guide fails, then run the project again, install it if you fail to guide the library, and so on. (ten thousand dirty words are omitted here).

3. Automatically import any missing libraries

Is there a better way to import missing libraries automatically?

Is there any way to import the required libraries automatically without modifying the original code and without the need for a "requirements.txt" file?

Of course there is! Let's see the effect first:

We take tornado as an example. In the first step, we can see that we have not installed tornado. After the second step, when we import tornado again, the program will help us download and install tornado automatically, so we will no longer report an error.

Autoinstall is our handwritten module with the following code:

Sys.meta_path is used in this code. Let's print it first and see what it is.

The approximate order of the import mechanism of Python 3 in the lookup process is as follows:

Look in sys.modules, which caches all imported modules

Look in sys.meta_path, which supports custom loaders

Look in sys.path, which records the directory names where some libraries are located

If it is not found, an ImportError exception is thrown

Note that sys.meta_path varies from Python to Python, such as between Python 2 and Python 3; in the newer Python 3 version (3.4 +), the custom loader needs to implement the find_spec method, while earlier versions use find_module.

The above code is a custom class library loader AutoInstall, which can be automatically imported into the third-party library. It is important to note that this method will "hijack" all newly imported libraries and destroy the original import method, so there may also be some strange problems, please pay attention.

Sys.meta_path is a kind of application of Python probe. Probe, or import hook, is the mechanism of Python that attracts little attention, but it can do many things, such as loading libraries on the network, modifying modules when importing modules, automatically installing missing libraries, uploading audit information, delaying loading, and so on.

At this point, I believe you have a deeper understanding of "how to automatically import missing libraries in Python". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.