Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to achieve operating system compatibility packaging in Python, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

There is such a problem: now you want to package a project into a whl file with setuptools, and then pip install is on both Windows/Linux operating systems, but there are some dependent libraries in this project that are only available on Windows (for example, pywinauto, pywingui, pywinrm), so the question is, how to achieve compatible installation of packaged files?

From a packaging point of view, the crux of the problem is to look at setup.py and requirements.txt files.

Here is a well-written article about the package construction and distribution of Python and the use of setup.py. It is recommended to read it. In addition, with regard to the management of Python dependent libraries (requirements.txt), this article makes a detailed comparison of pip, pipreqs, pigar, pip-tools, and pipdeptree and other tools, and is also recommended.

A clumsy implementation is to maintain two requirements.txt files, which are packaged and distributed to different operating systems.

But there will be trouble: it is very difficult to maintain two dependent files and two kinds of package files, and during the generation process, you have to rename them each time to distinguish them (note that there are certain normative constraints on the package name, if you change it indiscriminately, pip may not recognize it), the maintenance cost is very high.

In fact, maintenance packages are not uncommon in different operating system versions. If you have ever noticed different versions of Python library files, you will notice that many libraries distribute different versions according to different operating systems. For example, the following is the https://pypi.org/simple/numpy/ of Numpy with the same version number on different operating systems:

It can be seen that it is divided into five versions according to macos, linux and win operating systems and their digits. Maintaining so many versions must be a hassle, but the result means that Numpy officials believe that the advantages of distributing different system versions outweigh the disadvantages, and there is a way to achieve it.

Back to our question, is it necessary to try to package it into multiple operating system customized packages like Numpy?

The answer is no. The main reasons:

Numpy does this because it does scientific computing, and to improve efficiency, it packages compiled C extension files so that it does not have to rely on libraries such as libxxx-devel on the environment. If you have compiled and installed Python, you should have the impression that you need to install system dependencies such as zlib-devel, openssl-devel, and libffi-devel. But our previous problem is relatively simple, not that there are different compilation dependencies (system-level), but that the three-party libraries depend on different ones (project-level).

Another major reason is that the different versions of the system packaged by Numpy are not simply packaged with Python libraries such as setuptools, but are built with standard images. For example, for the packaging of the manylinux version, see Github (https://github.com/pypa/manylinux), you need to use the official Docker image. Obviously, we don't want to be so troublesome about our problems.

In short, according to the previous analysis, if you want to achieve operating system-compatible packaging, maintain multiple dependency files, use different methods of building packages, and maintain multi-system-specific packages, the method is feasible, but not very applicable.

If there is no new way, this is a consideration, but is there any other way?

I was troubled by this problem, but I didn't delve into it until I accidentally saw it in the setup.py of loguru, a library used to log:

If you look at the famous requests library file, you can write something like this:

Both examples are written in the setup.py file, but if we use the requirements.txt file, we can also write in this format and then read it in.

What is this magical way of writing?

It is based on PEP-508 created in November 2015 (and related but revoked or rejected PEP-390, PEP-426, PEP-459, PEP-496). The main purpose of this PEP is to enhance the ability of tools such as pip to find software packages.

The more important part is related to our problem, that is, the identification of the operating system, which is related to:

With such extension support, compatibility issues can be solved when packaging dependencies.

For example, the colorama library, if we only need to rely on the win32 system, we can specify "colorama > = 0.3.4; sys_platform=='win32'" when packaging; if you don't need to qualify the win32 system, but install it in the windows environment, you can write "colorama > = 0.3.4; platform_system=='Windows'".

After reading the above, do you have any further understanding of how to achieve operating system compatibility packaging in Python? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.