Shulou(Shulou.com)06/02 Report--

This article mainly introduces "what are the optimization methods of Python code". In the daily operation, I believe that many people have doubts about the optimization methods of Python code. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "what are the optimization methods of Python code?" Next, please follow the editor to study!

1. Optimize code and algorithm

First of all, read your code and algorithm carefully. Many speed problems can be solved by implementing better algorithms or adding caches. Specific guidelines can be used to write an entire book, but some general guidelines to follow are:

Test, don't guess. To test which parts of the code take the longest time to run, focus on those parts first.

Reduce memory usage. In general, try to reduce memory usage. For example, analyze a large file line by line without first storing it in memory.

Perform caching. If you perform many repetitive searches from disk, network, and database, then performing caching can be a big optimization.

Minimize the number of iterations in your code, especially the number of internal iterations.

Avoid (depth) recursion. For the Python interpreter, this requires a lot of memory and housekeeping, switching to generators, iterations, and so on.

Reuse objects instead of creating new objects in each iteration. Python must clean up each object that has been created to free memory. This is called garbage collection. The garbage collection process for many unused objects can greatly slow down the software.

Don't do this. Do you really need to do this? Can it be done later? Or can the operation be done at once and its results can be stored without calculating over and over again?

two。 Use PyPy

You may be using the reference implementation of Python, CPython, which is called CPython because it is written in C. A lot of people use it.

But if you decide that your code is computationally intensive, PyPy is a better choice. It can be a quick solution without changing a single line of code.

PyPy claims that it is on average 4. 4 times faster than CPython. It accelerates code execution by using a technique called just-in-time compilation (JIT). Other JIT include Java and the .NET Framework. CPython, on the other hand, uses an interpreter to execute code, which, while providing a great degree of flexibility, is very slow.

With JIT, you can compile code while running the program. It combines the speed advantages of pre-compilation (used by languages such as C and C +) with the flexibility of the interpreter. In addition, the JIT compiler can continue to optimize code while it is running. The longer the code runs, the more optimized it is.

PyPy has developed rapidly in recent years and can usually be used as a direct substitute for Python 2 and 3. It also works perfectly with tools such as Pipenv.

3. Use Asyncio

Asyncio is a relatively new core library in Python. It solves the same problem as threading: it accelerates the Imax O-intensive software system, but the Asyncio implementation is different.

I don't really like using asyncio in Python. Asyncio is quite complex, especially for beginners. And because the asyncio library has evolved a lot over the past few years, online tutorials and sample code are often out of date.

But that doesn't mean it's useless. This is a powerful example that can be used in many high-performance applications.

4. Use multithreading

Most software systems are Imax O-intensive rather than compute-intensive:

I am O-intensive-software systems are usually waiting for input / output operations to be completed. This is usually the case when getting data from a network or slow memory.

Compute intensive-software systems push CPU (central processing unit) to its limits. It uses all the features of CPU to produce the desired results.

While waiting for an answer from the network or disk, you can use multiple threads to keep other parts running.

Threads are independent execution sequences. By default, the Python program has a main thread. But you can create more threads and have Python convert between different threads. This transition occurs so fast that it seems to be running side by side at the same time.

Threads are independent execution sequences that share the same storage

Unlike other languages, Python multithreading does not run at the same time, but takes turns, mainly because of its global interpreter lock (GIL) mechanism.

It can be seen that multithreading will have a great impact on I / O-intensive software systems, but it is of little use to computing-intensive software systems.

Why is this happening? It's simple. While one thread waits for an answer from the network, other threads can continue to run. Using multithreading can make a big difference if you perform a large number of network requests. But if multithreads are performing a large number of calculations, they are just waiting for their own turn to continue. Threads only introduce more overhead.

5. Use more processors at the same time

If the software system is computationally intensive, you can usually rewrite code by using more processors at the same time. In this way, the execution speed can be extended linearly.

This is called parallelism. Not all algorithms can run in parallel. For example, it is impossible to simply parallelize recursive algorithms. But there is almost always an alternative algorithm that works well in parallel.

There are two ways to use more processors:

Use multiple processors and / or cores on the same computer. In Python, this can be done by using a multiprocessing library.

Use computer networks to use processors distributed across multiple machines. We call it distributed computing.

Unlike thread libraries, multiprocessing libraries bypass Python global interpreter locks. It is actually achieved by generating multiple Python instances. Therefore, by using multithreading to take turns in a Python process, you will have multiple Python processors to run the code at the same time.

Visual multiprocessing

The multiprocessing library is very similar to the thread library. Then why consider threading? Yes, threading is "lighter". All it needs is a running Python interpreter, which requires less memory, and generating new processes has its overhead. Therefore, if the code is Imax O-intensive, the use of threads may be sufficient.

After making the software system work in parallel, it is only a small step to combine distributed computing with functions such as Hadoop. By taking advantage of the cloud computing platform, it is now relatively easy to speed up. For example, you can process large datasets in the cloud and use the results locally. Using a hybrid mode of operation, you can save some cash, you know, the computing function of the cloud platform is very expensive.

At this point, the study of "what are the optimization methods of Python code" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.