Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "the principle and use of Python virtual environment". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. Why use a virtual environment?

Virtual environments provide simple solutions to a range of potential problems, especially in the following areas:

Different projects are allowed to use different versions of packages, thus resolving dependency issues. For example, you can use Project A v2.7 for Project X and Package A v1.3 for Project Y.

Make the project self-contained and reproducible by capturing all package dependencies in the requirements file.

Install the package on a host that does not have administrator privileges.

With only one project and no system-wide installation of software packages, you can keep the global site-packages / directory clean.

Sounds convenient, doesn't it? The importance of virtual environments is highlighted when you start to build more complex projects and collaborate with others. Many data scientists also need to be familiar with the multilingual Conda environment in virtual environments.

Can be used in order!

two。 What is a virtual environment?

What exactly is a virtual environment?

Virtual environments are Python tools for dependency management and project isolation, allowing Python site packages (third-party libraries) to be installed in an isolated directory of a local specific project, rather than globally (that is, as part of a system-wide Python).

That sounds good, but what exactly is a virtual environment? A virtual environment is just a directory that contains three important components:

Site-packages / folder with third-party libraries installed.

The symlink symbolic link to the Python executable file installed on the system.

Ensure that the script that executes the Python code uses the Python interpreter and site package installed in a given virtual environment.

The point is that some unexpected errors can occur, which I'll talk about later, but let's take a look at how to actually use a virtual environment in practice.

3. Using a virtual environment

(1) create a virtual environment

Suppose you want to create a virtual environment called test-project/ for the project you are working on with the following directory tree:

Test-project/ ├── data ├── deliver # Final analysis, code, & presentations ├── develop # Notebooks for exploratory analysis ├── src # Scripts & local project modules └── tests

The venv module needs to be executed, which is part of the Python standard library.

Cd test-project/ python3-m venv venv/ # Creates an environment called venv/

Note: "venv/" can be replaced with a different environment name.

Look at that! The virtual environment was born. Now the project becomes:

Test-project/ ├── data ├── deliver ├── develop ├── src ├── tests └── venv # There it is!

Reminder: the virtual environment itself is a directory.

The only thing to do is to "activate" the environment by running the script mentioned earlier.

% source venv/bin/activate (venv)% # Fancy new command prompt

Now we are in the active virtual environment (indicated by the command prompt, prefixed with the name of the active environment).

We will deal with the project as usual to ensure that the project is completely isolated from the rest of the system. In a virtual environment, we cannot access the system-wide site package, and we cannot access the installation package outside the virtual environment.

When you complete the project work, you can exit the environment with the following code:

(venv) deactivate # Old familiar command prompt

(2) install the package

By default, only pip and setuptools are installed in the new environment.

(venv)% pip list # Inside an active environmentPackage Version-- pip 19.1.1 setuptools 40.8.0

If you want to install a specific version of a third-party library, such as numpyv1.15.3, use pip as usual.

(venv)% pip install numpy==1.15.3 (venv)% pip listPackage Version-- numpy 1.15.3 pip 19.1.1 setuptools 40.8.0

You can now import numpy in a script or active Python shell. For example, suppose the project contains the following lines of script tests / imports-test.py.

#! / usr/bin/env python3 import numpy as np

When you run this script directly from the command line, you get:

(venv)% tests/imports-test.py (venv)% # Look, Ma, no errors!

Success. There is no failure in importing the script into numpy.

Dante and Virgil cross the Styx River-chapter 8 of Dante's Divine Comedy Hell

Illustration: Gustave Dor é

4. Management environment

(1) requirements document

The easiest way to make our work reusable by others is to add a requirements file to the root (top-level directory) of the project. To do this, you need to run pip freeze, and the installed third-party software packages and their version numbers are listed below:

(venv)% pip freeze numpy==1.15.3

And write the output to a file, which we call requirements.txt.

(venv) pip freeze > requirements.txt

When you update a package or install a new package, you can use the same command to rewrite the requirements file.

Anyone who shares a project can now use the requirements.txt file to run the project on the system by copying the environment.

(2) replication environment

Wait-- how on earth did you do that?

Imagine that our teammate Sara removed the test project from the team's GitHub repository. On her system, the directory tree of the project is as follows:

Test-project/ ├── data ├── deliver ├── develop ├── requirements.txt ├── src └── tests

Did you notice anything unusual? Yes, that's right! There is no venv / folder.

We have removed it from the team's GitHub repository because its existence can cause trouble.

This is one reason why using requirements.txt files is critical to copying project code.

To run the test project on the machine, all Sara needs to do is create a virtual environment in the root directory of the project:

Sara% cd test-project/ Sara% python3-m venv venv/

And use pip install-r requirements.txt to install project dependencies in the active virtual environment.

Sara% source venv/bin/activate (venv) Sara% pip install-r requirements.txt Collecting numpy==1.15.3 (from-r I (line 1)) Installing collected packages: numpy Successfully installed numpy-1.15.3

Now, the project environment on the Sara system is exactly the same as ours. It's neat, isn't it?

(3) troubleshooting

Unfortunately, things don't always go according to plan, and there are always some problems. You may have mistakenly updated a specific site package and found yourself at level 9 of Dependency Hell, unable to run a single line of project code. Maybe it's not that bad, maybe you'll find yourself at level 7.

No matter to what extent you find yourself, the easiest way to solve the problem and see hope again is to recreate the virtual environment of the project.

% rm-r venv/ # Nukes the old environment% python3-m venv venv/ # Makes a blank new one% pip install-r requirements.txt # Re-installs

It's done, and thanks to the requirements.txt file, it's back to normal. Another reason, however, is that requirements documents are always included in the project.

5. How does a virtual environment do this?

Want to know more about virtual environments? For example, how does the active environment use the correct Python interpreter and find the right third-party library?

(1) echo $PATH

It all boils down to the value of PATH, which tells shell what Python instance to use and where to find the site package. In the base shell, PATH seems to behave more or less this way.

Echo $PATH / usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin

When calling the Python interpreter or running the .py script, shell searches the directories listed in PATH sequentially until it encounters an Python instance. To see the Python instance that PATH first found, run which python3.

% which python3 / usr/local/bin/python3 # Your output may differ

It is also easy to find the location of the site package for this Python instance through the site module, which is part of the Python standard library.

% python3 # Activates a Python shell > > import site > site.getsitepackages () # Points to site-packages folder ['/ usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages']

Run the script venv / bin / activate to modify PATH so that shell searches for the project's local binaries before searching for the system's global binaries.

Cd ~ / test-project/ source venv/bin/activate (ven) echo $PATH~/test-project/venv/bin:/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin

Now shell knows how to use the local Python instance of the project:

(venv)% which python3 ~ / test-project/venv/bin/python3

Where can I find the local site package for the project?

(venv)% python3 > import site > site.getsitepackages () ['~ / test-project/venv/lib/python3.7/site-packages'] # Ka-ching

(2) rational examination

Remember the old tests / imports-test.py script? It looks like this:

#! / usr/bin/env python3 import numpy as np

We were able to run this script in the active environment without any problems because the Python instance in the environment had access to the project's local site package.

What happens if you run the same script from outside the project's virtual environment?

% tests/imports-test.py # Look, no active environmentTraceback (most recent call last): File "tests/imports-test.py", line 3, in import numpy as npModuleNotFoundError: No module named 'numpy'

Yes, there was a mistake, but we should do so. If we don't, it means that we can access the project's local site package from outside the project, thus undermining the entire purpose of having a virtual environment. The fact that something went wrong proves that our project is completely isolated from the rest of the system.

(3) Catalog tree of environment

One thing that can help organize all this information is a clear understanding of what the environmental directory tree looks like.

Test-project/venv/ # Our environment's root directory ├── bin │ ├── activate # Scripts to activate │ ├── activate.csh # our project's │ ├── activate.fish # virtual environment. │ ├── easy_install │ ├── easy_install-3.7 │ ├── pip │ ├── pip3 │ ├── pip3.7 │ ├── python-> usr/local/bin/python # Symlinks to system-wide │ └── python3-> python3.7 # Python instances. This is the end of the introduction of ├── include ├── lib │ └── python3.7 │ └── site-packages # Stores local site packages └── pyvenv.cfg "principles and uses of Python Virtual Environment". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.