Shulou(Shulou.com)06/02 Report--

Editor to share with you what Python compilation-related knowledge, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Python is an interpretive language that can be developed rapidly.

Comparison of interpretation methods between Python and Java

Java: source code-> compile to class-> Jvm interpretation run

Python: source code-> Python interpreter interpretation run

In fact, Python and Java are interpreted in the same way, except that on the surface, Python interprets the source code directly, but in fact the python interpreter only loads bytecode. Careful friends must have noticed this. When we import a module, we always create a _ _ pycache__ directory in the same directory as the module, in which the bytecode file of the loaded module is stored.

Compiling the source code has the following functions:

Source code protection (algorithm protection)

Prevent users from tampering with the source code interpreter to load code faster

Several file types of Python

Py:Python console program source code file

Pyw:Python source code file with user interface

Pyx:Python package source file

Pyc:Python bytecode file

Bytecode file optimized by pyo:Python

The library file of pyd:Python (Python version of DLL), which is a so file on Linux

The generating method of pyc and pyo

The role of pyc is for cross-platform use, similar to the Class file in Java. Pyc file is a kind of bytecode file, which can speed up the loading speed of the Python interpreter. Of course, it can also be used for simple source code disclosure protection.

Pyo is an optimized bytecode file, but pyo is more like an intermediate file in a compiled language.

We can compile the source code through the py_compile module provided by Python.

The py_compile module only provides three methods, namely, the compilation exception PyCompileError, the compilation compile, and the program entry main

What we are going to use is the compile method, and the compile prototype is as follows:

Compile (file, cfile=None, dfile=None, doraise=False, optimize=-1)

There are five parameters:

File: required parameter, source file to compile

Cfile: the compiled file defaults to the _ _ pycache__/ source file name under the source file directory. Interpreter type-python version. Bytecode type

# for example: _ _ pycache__/abc.cpython-34.pyo

Dfile: error message file, which is the same as cfile by default

Doraise: whether to enable exception handling. Default is False.

Optimize: optimize bytecode level

When optimize is 1, the optimization bytecode level is the highest-1 and 0: set pyc optimization levels 1 and 2: the smaller the number of pyo optimization levels, the higher the optimization level

Prepare the source files a.py and b.py, with the same content, just a sentence of print ("python") code

Write a compilation script:

Import py_compilepy_compile.compile (file = "a.py", cfile = "a.pyc", optimize=-1) py_compile.compile (file = "b.py", cfile = "b.pyo", optimize=1)

After running, you can see that it has been successfully compiled into a bytecode file, a.pyc and b.pyo, respectively.

Try to run the two bytecode files:

Python a.pycpython a.pyo

The bytecode file ran successfully.

You can also run it directly through the Python load module:

# compiled to pyc

Python-m py_compile source code

# compiled to pyo

Python-O-m py_compile source code

This does simply protect our code, and it seems to be the effect of encryption, but note that this is not encryption, it just turns the source code into optimized bytecode, if you want to get the source code, we can also reverse compile to get the source code, there are special tools to reverse Python bytecode.

If you need to compile all the source code in the entire directory, please refer to Python compileall

Pyd can make our code more secure

If you really want to protect your code, why not consider turning it into an python extension module? (there is no message that pyd has been decompiled.)

Pyd is an extension module in Python, which is equivalent to the dll of windows, except that pyd is only for python calls.

In fact, most packages and modules are released in the form of pyd.

If you are particularly interested, you can take a closer look at setuptools and distutils.

Before we can successfully pyd the source code, we need to use the Cython package.

Pip list | findstr "Cython"

Check if Cython is installed. Do not ask pip install Cython to install it.

Compile pyd step 1: generate C code

Import Cython.Build

# Import Build module

Cython.Build.cythonize ("a.py")

# convert a.py to C code

After the cythonize is run, an A.C file is created in the a.py directory without exception, and a list of distutils.extension.Extension objects is returned.

It is important to note that if you are testing in Python Shell, be sure to use an absolute path, otherwise it will ValueError exception and cythonize will not read the path from sys.path.

Step 2 of compiling pyd: use distutils to generate pyd expansion module

At this point, we can use the distutils package to compile the pyd module we want.

Compile a.py into pyd

Import Cython.Build

Import distutils.core

A = Cython.Build.cythonize ("a.py")

# return the list of distutils.extension.Extension objects

Distutils.core.setup (

Name = 'compilation of pyd', # package name

Version = "1.0", # package version number

Ext_modules= aMar # extension module

Author = "Happyran", # author

Author_email='happyran163@163.com' # author's mailbox

)

Python executes compiled script build

Or

Python executes compiled script build_ext

At this point, a build directory is generated in the directory where the compilation script is located, in which the .def and .o files of the C language are stored, as well as the pyd file we want.

Misunderstandings of batch compilation of pyd files

At this point, we have generated a pyd file, if we are the developer of the extension pack / module, how to compile in batch?

There are always people who make mistakes, such as the following two examples:

A = Cython.Build.cythonize ("a.py")

B = Cython.Build.cythonize ("b.py")

Distutils.core.setup (

...

Ext_modules= [a,b]

)

Do you do this? NO.

A = Cython.Build.cythonize ("a.py")

A.append (Cython.Build.cythonize ("b.py"))

Distutils.core.setup (

...

Ext_modules= a

)

Or like this?

The reason for making such a mistake is because:

A = Cython.Build.cythonize ("a.py")

Type (a)

Prompt

Yes, Cython.Build.cythonize returns a list with only one distutils.extension.Extension object in it

Will report an error. 1 Extension or 2 tuples are required.

Batch compilation of pyd

Method 1: extract the Extension object we want

Import Cython.Build

Import distutils.core

A = Cython.Build.cythonize ("a.py") [0] # extract Extension object

B = Cython.Build.cythonize ("b.py") [0]

Distutils.core.setup (

Name = 'compilation of pyd', # package name

Version = "1.0", # package version number

Ext_modules= [aformab], # extended module

Author = "Happyran", # author

Author_email='happyran163@163.com' # author's mailbox

)

Method 2: convert to C code and then instantiate the Extension object

Import Cython.Build

Import distutils.core

Cython.Build.cythonize ("a.py")

Cython.Build.cythonize ("b.py")

Distutils.core.setup (

Name = 'compilation of pyd', # package name

Version = "1.0", # package version number

Ext_modules= [distutils.core.Extension ('averse, ["a.c"]), distutils.core.Extension (' baked, ['b.c'])], # extended module

# [

# distutils.core.Extension ('asides, ["a.c"])

# distutils.core.Extension ('baked, [' b.c'])

#]

Author = "Happyran", # author

Author_email='happyran163@163.com' # author's mailbox

)

Pyc and pyo have relatively low security, and pyd is the best solution to solve the problem of code security in Python development.

But there is one thing to note: both pyc, pyo and pyd follow the Python version, and don't expect Python2.7 things to run perfectly on Python3.

PS: if you encounter running build... Hint, delete the build directory and recompile.

The above is all the content of this article "what is the knowledge of Python compilation?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.