Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to improve the computing speed of python. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

I. Preface

The Python language has become very popular in recent years. It is widely used in data science, artificial intelligence, and network security problems. Because of the strong readability of the code and high learning efficiency, it attracts many non-subject students to study. However, after using Python for a period of time, it is found that it has no advantage in speed at all, especially in computing-intensive tasks, performance problems have always been the weakness of Python. This paper mainly introduces Python's JIT compiler Numba, which can greatly accelerate the running speed of core functions with the least code intrusion, which is suitable for students related to data analysis business.

The first question to answer is: why is Python 2 to 10 times slower than other languages when running the same program? Why can't we make it faster?

The following are the main reasons:

"it is a GIL (Global Interpreter Lock global interpreter lock)"

"it's an interpretive language, not a compiled language."

"it's a dynamically typed language.

As the focus of this article is not to explain the reasons for the slow speed of Python and the logic behind it, this part will not be discussed in depth. Students who are interested are welcome to search by themselves.

2. JIT compiler of Python

In order to have both portability and performance, smart engineers invented JIT. The so-called JIT means that in an interpreted language, the code that is often used or has a great performance improvement is compiled into machine code when it is interpreted, and other code that is one-time or without much performance improvement is executed in bytecode. In this way, the portability can be guaranteed and the performance can be greatly improved at the same time.

JIT compilation dynamically compiles Python code to machine code execution when the code is running, which greatly improves the running speed by avoiding the built-in interpreter of Python. The popular JIT schemes are Numba and Pypy, but there is no perfect solution because of the historical burden and syntax changes of Python. Each scheme has different advantages and disadvantages, so it is necessary to choose the appropriate scheme according to the field of use.

Pypy supports global acceleration, but it does not support C library well, so it is more suitable for transactional tasks such as Web services.

Numba can accelerate some functions and libraries with high performance while maintaining Python compatibility, but the scope of use will be limited.

III. Numba Fast Learning

We mainly introduce the basic usage of Numba, which can greatly accelerate the running speed of core functions with the least intrusions into the code, which is suitable for students related to data analysis business.

By using LLVM technology, Numba compiles Python code to generate optimized machine code, which can greatly improve the efficiency of code execution.

About installation

The first step is to install numba and run different installation commands depending on the python environment:

Conda install numbapip install numba IV. About using

To sum up: the easiest way to use Numba is to prefix the function definition with @ jit or @ njit.

Numba declares whether or not to accelerate by adding a decorator (modifier) before the function definition. As mentioned above, the easiest way to use it is @ jit. There are two compilation modes for Numba's @ jit: nopython and object.

Nopython mode will fully compile this decorated function, the function will run completely independent of the Python interpreter, and will not call Python's C language API. This mode is recommended if you want the best performance. At the same time, because @ jit (nopython=True) is too common, Numba provides the @ njit modifier, which is equivalent to this sentence and is easy to use. However, this mode requires that the types of all variables in the function can be deduced by the compiler (some basic types, such as libraries or self-defined data types, etc.), otherwise an error will be reported.

In object mode, the compiler will automatically identify the code parts that can be compiled and accelerated, such as loop statements in the function, and compile them into machine code, and leave the rest of the unrecognized parts to the Python interpreter to run. If you want the best performance, avoid using this approach (For best performance avoid using this mode!).

If the parameter nopython=True,Numba is not set, it will first try to use nopython mode, and if it cannot be used for some reason, it will use object mode. The addition of nopython forces the compiler to use nopython mode, but there is a risk of an error if the code has a type that cannot be deduced automatically.

Fifth, experiment to improve from numba import jitimport random Timedef monte_carlo_pi (sam): account = 0 for i in range (sam): X = random.random () y = random.random () if (x * * 2 + y * * 2) < 1.0: account + = 1 return 4.0 * account / sam@jitdef jit_monte_carlo_pi (sam): account = 0 for i in range (sam): X = random. Random () y = random.random () if (x * * 2 + y * * 2) < 1.0: account + = 1 return 4.0 * account / samloops = [100000 1000000, 10000000, 100000000, 1000000000] for loop in loops: startTime = time.time () monte_carlo_pi (loop) t = time.time ()-startTime print ('python {} loop: {}' .format (loop, t)) startTime = time.time () jit_monte_carlo_pi (loop) t = time.time ()-startTime print ('numba {} loop: {}' .format (loop) T))

For the above code, the result of running is:

Python 100000 loop: 0.0469999313354

Numba 100000 loop: 0.213999986649

Python 1000000 loop: 0.478999853134

Numba 1000000 loop: 0.0110001564026

Python 10000000 loop: 4.82499980927

Numba 10000000 loop: 0.107000112534

Python 100000000 loop: 48.728000164

Numba 100000000 loop: 1.05900001526

Python 1000000000 loop: 489.142100134

Numba 1000000000 loop: 11.01402001452

As you can see, jit is about 47 times higher after compilation. The more the number of cycles, the more obvious the acceleration effect of numba. For more complex computational functions, numba may have better results.

This is the end of the article on "how to improve the computing speed of python". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.