Shulou(Shulou.com)06/03 Report--

This article will explain in detail how to optimize the performance of Python, the content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Preface

The most criticized aspect of Python as a program language is its performance. Python as a dynamic language, performance is not its advantage, but in many cases, it will not face too many performance problems, so Python is widely used. However, there will always be a performance problem, in which case the first thing to do is to optimize the performance within the framework of Python technology.

Decide whether to optimize or not

Performance optimization is a means to solve other problems. Whether or not to make performance optimization a formal development task, the first thing is to reach a consensus. The project during the development period is naturally easy to say, and the project after launch needs to estimate the workload of performance optimization, the corresponding risks, and assess the general scope of impact. Make the product, development, and testing aware of the need for this task before it is appropriate to formally implement it.

Performance analysis.

The necessary step in performance optimization is to analyze the bottleneck. First of all, you have to find out where it is slow, and then fix it later. For Python performance analysis, you can directly use the built-in cProfile for processing. The usage is also simple, it is enough to read the The Python Profilers section in the official documentation.

CProfile results have a lot of visual chemical industry, but with the coding together, it is most convenient to use PyCharm directly to open it. You can quickly jump to the code corresponding to each time-consuming function.

Pay attention to those that have a large number of calls and poor performance in a single call, and those high probabilities are the logic that needs to be optimized.

Look at the profile.

Optimization inspection

After locating the problem, you can modify it. Control the scope of each optimization change, one change at a time, and then Profile the comparison again. Each improvement of the optimization needs to be supported by data, so as to ensure that it is moving in the right direction and has some control over the time.

Some code optimization is not necessarily easy to implement, so this gradual optimization way is easy to reduce the overall development risk of the project, can ensure that a version is available at any time, and has been optimized.

Continuous monitoring

After performance optimization, you need to pay attention to monitoring the performance metrics. In this way, we can get a full picture of the performance in time, and once we need to optimize it again, we can also know the decision as soon as possible.

Common directions of Python code optimization to reduce unnecessary function calls

In many cases, there is no need to optimize a function implementation itself, giving priority to whether you can reduce the number of calls to the function, or delete the function directly.

This is a very common situation, and some legacy code that is no longer needed at the functional level takes up a lot of performance. If a piece of code is handled by many people, the requirements are changeable and iterated over and over again, then this kind of code is likely to exist.

Figure it out and delete them.

Static dynamic computation at run time

Some of the running logic can be generated offline to get static results, and the static results can be loaded directly at runtime, and this part of the performance should also be recycled.

Apply caching appropriately

Calculations that cannot be directly static may also be cached. The creation and destruction of resources can also determine whether to apply implementation methods such as caching or object pool resource pool according to the situation, so as to avoid repeated operations.

It is important to avoid memory leaks caused by caching.

Reduce gc

Python manages object memory with gc through reference counting. For cases where reference counting cannot be handled, Python will evoke gc to handle it, and gc has a certain impact on performance. Therefore, when the code is implemented, it is best to be gc-friendly and reduce the reference ring in the code.

In addition, you can also set gc parameters to control the probability of gc trigger, reduce the gc trigger in high frequency calculation, and manually trigger gc in idle time.

Modify the code implementation with obvious errors

Many performance problems are caused by non-critical logic, which often lacks performance considerations at first glance, such as

Keep searching through the for loop

Wrong selection of containers, resulting in constant traversal lookups

There is complex time-consuming logic in the property implementation, which makes the caller ignore this point.

Added a lot of function calls to the loop that can be moved outside the loop and run only once

The logic that can be triggered by an event becomes polling.

There are a large number of repeated calls in the function

This kind of code should not exist in the first place, find out and change it, and try your best to ensure that such code will not enter the code base in the future.

Improve some basic usage in high frequency call

For example, in the performance comparison of some common uses listed here, you can see the performance difference when making high-frequency calls, and choose a more efficient implementation.

Optimization algorithm

Part of the logic may be optimized from the implementation method level to optimize the complexity of the algorithm itself.

Change part of the logic to native code implementation

Replacing part of the logic with CramCure + is also a common means of Python optimization. There are several options.

Nude writing C _ blank + extension

Using Boost.Python to implement C++ extension

Use Cython

Use PyPy

Nude writing C _ Boost.Python + needs to deal with tedious object conversion and reference handling, and generally chooses to use it. It is much easier to use.

With Cython, you don't have to write C++ code, but you have to write Cython scripts. Depending on the situation, there is a choice of use.

PyPy is equivalent to replacing the runtime and cannot be used when the runtime is out of control. If the server side, you can consider partial use, through process isolation to solve some dependency problems.

You need to be careful to keep the necessary Python implementation so that you can quickly switch back if you encounter a problem.

On the Python performance optimization method path how to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.