Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "how to make Python code run faster". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

0. Code optimization principle

This article will introduce a number of techniques for speeding up Python code. Before delving into the details of code optimization, you need to understand some basic principles of code optimization.

The first basic principle is not to optimize prematurely. Many people start writing code with the goal of performance optimization. "it's much easier to make the right program faster than to make the fast program right." Therefore, the premise of optimization is that the code works properly. Optimizing prematurely may ignore the grasp of the overall performance indicators and do not reverse the primary and secondary until you get the overall results.

The second basic principle is to weigh the cost of optimization. Optimization comes at a cost, and it is almost impossible to solve all performance problems. The usual choice is to change time for space or space for time. In addition, the development cost also needs to be considered.

The third principle is not to optimize the unimportant parts. If every part of the code is optimized, these changes can make the code difficult to read and understand. If your code is slow, first find where the code is slow, usually an internal loop, and focus on where it is slow to optimize. Elsewhere, a little loss of time makes no difference.

1. Avoid global variables

# it is not recommended. Code time: 26.8s import math size = 10000 for x in range (size): for y in range (size): Z = math.sqrt (x) + math.sqrt (y)

Many programmers initially write simple scripts in the Python language, and when writing scripts, they are usually used to writing them directly as global variables, such as the code above. However, due to the different implementation of global variables and local variables, the code defined in the global scope runs much slower than that defined in the function. By putting script statements into a function, you can usually bring a 15%-30% speed increase.

# recommended writing method. Code time: 20.6 seconds import math def main (): # defined into the function to reduce the use of all variables size = 10000 for x in range (size): for y in range (size): Z = math.sqrt (x) + math.sqrt (y) main ()

two。 Avoid. (attribute access operator)

2.1 avoid access to module and function properties

# it is not recommended. Code time: 14.5s import math def computeSqrt (size: int): result = [] for i in range (size): result.append (math.sqrt (I)) return result def main (): size = 10000 for _ in range (size): result = computeSqrt (size) main ()

Each time you use. (when the attribute accesses the operator), specific methods, such as _ _ getattribute__ () and _ _ getattr__ (), are triggered, which perform dictionary operations and therefore incur additional time overhead. Property access can be eliminated through the from import statement.

# optimize the writing method for the first time. Code time: 10.9 seconds from math import sqrt def computeSqrt (size: int): result = [] for i in range (size): result.append (sqrt (I)) # avoid using math.sqrt return result def main (): size = 10000 for _ in range (size): result = computeSqrt (size) main ()

As we mentioned in Section 1, local variables can be found faster than global variables, so the frequently accessed variable sqrt can be accelerated by changing it to a local variable.

# optimize the writing for the second time. Code time: 9.9 seconds import math def computeSqrt (size: int): result = [] sqrt = math.sqrt # assign to local variable for i in range (size): result.append (sqrt (I)) # avoid using math.sqrt return result def main (): size = 10000 for _ in range (size): result = computeSqrt (size) main ()

In addition to math.sqrt, there are other functions in computeSqrt. That is to call the append method of list. By assigning this method to a local variable, the internal for loop in the computeSqrt function can be completely eliminated. Use.

# recommended writing method. Code time: 7.9s import math def computeSqrt (size: int): result = [] append = result.append sqrt = math.sqrt # assign to local variable for i in range (size): append (sqrt (I)) # avoid the use of result.append and math.sqrt return result def main (): size = 10000 for _ in range (size): Result = computeSqrt (size) main ()

2.2 avoid in-class attribute access

# it is not recommended. Code time: 10.4 seconds import math from typing import List class DemoClass: def _ _ init__ (self, value: int): self._value = value def computeSqrt (self Size: int)-> List [float]: result = [append = result.append sqrt = math.sqrt for _ in range (size): append (sqrt (self._value)) return result def main (): size = 10000 for _ in range (size): demo_instance = DemoClass (size) result = demo_instance.computeSqrt (size) main ()

Avoid. The principle also applies to in-class properties, where accessing self._value is slower than accessing a local variable. Code can be run faster by assigning frequently accessed in-class attributes to a local variable.

# recommended writing method. Code time: 8.0 seconds import math from typing import List class DemoClass: def _ _ init__ (self, value: int): self._value = value def computeSqrt (self) Size: int)-> List [float]: result = [] append = result.append sqrt = math.sqrt value = self._value for _ in range (size): append (sqrt (value)) # avoid using self._value return result def main (): size = 10000 for _ in range (size): Demo_instance = DemoClass (size) demo_instance.computeSqrt (size) main ()

3. Avoid unnecessary abstractions

# Writing is not recommended Code time: 0.55 seconds class DemoClass: def _ _ init__ (self, value: int): self.value = value @ property def value (self)-> int: return self._value @ value.setter def value (self X: int): self._value = x def main (): size = 1000000 for i in range (size): demo_instance = DemoClass (size) value = demo_instance.value demo_instance.value = I main ()

Any time you use additional processing layers (such as decorators, property accesses, descriptors) to wrap the code, it slows down the code. In most cases, you need to re-examine whether it is necessary to use the definition of a property accessor, and using the getter/setter function to access properties is usually a code style left over by CAccord + programmers. If it is really not necessary, use simple attributes.

# recommended writing method, code time: 0.33 seconds class DemoClass: def _ init__ (self, value: int): self.value = value # avoid unnecessary attribute accessors def main (): size = 1000000 for i in range (size): demo_instance = DemoClass (size) value = demo_instance.value demo_instance.value = I main ()

4. Avoid data replication

4.1 avoid meaningless data replication

# not recommended. Code time: 6.5s def main (): size = 10000 for _ in range (size): value = range (size) value_list = [x for x in value] square_list = [x * x for x in value_list] main ()

There is no need for value_list in the above code, which creates unnecessary data structures or replication.

# recommended writing method, code time: 4.8s def main (): size = 10000 for _ in range (size): value = range (size) square_list = [x * x for x in value] # avoid meaningless copying of main ()

Another situation is that you are too paranoid about Python's data sharing mechanism, do not understand or trust Python's memory model well, and abuse functions such as copy.deepcopy (). Replication operations can usually be removed from this code.

4.2 No intermediate variables are used when exchanging values

# not recommended. Code time: 0.07s def main (): size = 1000000 for _ in range (size): a = 3b = 5 temp = an a = b b = temp main ()

The above code creates a temporary variable temp when exchanging values, and without intermediate variables, the code is cleaner and runs faster.

# recommended writing method, code time: 0.06s def main (): size = 1000000 for _ in range (size): a = 3b = 5a, bb = b, a # without intermediate variable main ()

4.3 string concatenation with join instead of +

# not recommended. Code time: 2.6s import string from typing import List def concatString (string_list: List [str])-> str: result =''for str_i in string_list: result + = str_i return result def main (): string_list = list (string.ascii_letters * 100) for _ in range (10000): result = concatString (string_list) main ()

When using a + b concatenation string, because the string in Python is an immutable object, it requests a piece of memory space and copies an and b to the newly requested memory space, respectively. Therefore, if you want to concatenate a string, it will produce an intermediate result, and each intermediate result needs to be applied for and copied once, which seriously affects the running efficiency. When you use join () to concatenate strings, you first calculate the total memory space that needs to be applied for, then request the required memory at once, and copy each string element to that memory.

# recommended writing method, code time: 0.3 seconds import string from typing import List def concatString (string_list: List [str])-> str: return''.join (string_list) # use join instead of + def main (): string_list = list (string.ascii_letters * 100) for _ in range (10000): result = concatString (string_list) main ()

5. Using the short-circuit characteristic of if condition

# Writing is not recommended Code time: 0.05s from typing import List def concatString (string_list: List [str])-> str: abbreviations = {'cf.',' e.g.codes, 'ex.',' etc.', 'flg.',' i.e.codes, 'Mr.' 'vs.'} abbr_count = 0 result =' 'for str_i in string_list: if str_i in abbreviations: result + = str_i return result def main (): for _ in range (10000): string_list = [' Mr.', 'Hat',' is', 'Chasing',' the', 'black',' cat','.] Result = concatString (string_list) main ()

The short-circuit feature of the if condition means that statements such as if an and b will be returned directly when an is False, and b will no longer be calculated; for statements such as if an or b, it will be returned directly when an is True, and b will no longer be calculated. Therefore, in order to save running time, for or statements, variables with a higher probability of True should be written before or, while and should be postponed.

# recommended writing Code time: 0.03s from typing import List def concatString (string_list: List [str])-> str: abbreviations = {'cf.',' e.g.codes, 'ex.',' etc.', 'flg.',' i.e.codes, 'Mr.',' vs.'} abbr_count = 0 result =''for str_i in string_list: if str_i [- 1] = ='. And str_i in abbreviations: # using the short-circuit characteristic of if condition result + = str_i return result def main (): for _ in range (10000): string_list = ['Mr.',' Hat', 'is',' Chasing', 'the',' black', 'cat','.] Result = concatString (string_list) main ()

6. Cycle optimization

6.1 use for loop instead of while loop

# it is not recommended. Code time: 6.7 seconds def computeSum (size: int)-> int: sum_ = 0 I = 0 while I

< size: sum_ += i i += 1 return sum_ def main(): size = 10000 for _ in range(size): sum_ = computeSum(size) main() Python 的for循环比while循环快不少。 # 推荐写法。代码耗时：4.3秒 def computeSum(size: int) ->

Int: sum_ = 0 for i in range (size): # for loop instead of while loop sum_ + = I return sum_ def main (): size = 10000 for _ in range (size): sum_ = computeSum (size) main ()

6.2 use implicit for loops instead of explicit for loops

For the above example, an implicit for loop can be used instead of an explicit for loop

# recommended writing method. Code time: 1.7s def computeSum (size: int)-> int: return sum (range (size)) # implicit for loop instead of explicit for loop def main (): size = 10000 for _ in range (size): sum = computeSum (size) main ()

6.3 reduce the calculation of the inner for cycle

# it is not recommended. Code time: 12.8s import math def main (): size = 10000 sqrt = math.sqrt for x in range (size): for y in range (size): z = sqrt (x) + sqrt (y) main ()

The sqrt (x) in the above code is located in the inner for loop and is recalculated during each training, adding time overhead.

# recommended writing method. Code time: 7.0s import math def main (): size = 10000 sqrt = math.sqrt for x in range (size): sqrtsqrt_x = sqrt (x) # reduce the calculation of the inner for loop for y in range (size): Z = sqrt_x + sqrt (y) main ()

7. Use numba.jit

We follow the example described above and use numba.jit on this basis. Numba can compile the Python function JIT to machine code execution, which greatly improves the running speed of the code. For more information about numba, see the following home page:

Http://numba.pydata.org/numba.pydata.org

# recommended writing method. Code time: 0.62 seconds import numba @ numba.jit def computeSum (size: float)-> int: sum = 0 for i in range (size): sum + = i return sum def main (): size = 10000 for _ in range (size): sum = computeSum (size) main ()

8. Select the appropriate data structure

Python built-in data structures such as str, tuple, list, set, dict are all implemented in C, and the speed is very fast. It is almost impossible to achieve the built-in speed in the performance of the new data structure.

List, similar to std::vector in C++, is a dynamic array. It pre-allocates a certain amount of memory space, and when the pre-allocated memory space is used up and continues to add elements to it, it will apply for a larger piece of memory space, then copy all the original elements, and then destroy the previous memory space. Insert a new element. When deleting an element, the operation is similar. When the used memory space is less than half of the pre-allocated memory space, another small piece of memory is applied, an element copy is made, and then the original large memory space is destroyed. Therefore, if there are frequent add and delete operations and a large number of new and deleted elements, the efficiency of list is not high. At this point, you should consider using collections.deque. Collections.deque is a double-ended queue, which has the characteristics of both stack and queue, and can perform complex insert and delete operations at both ends.

List's lookup operation is also very time-consuming. When you need to look for certain elements frequently in list, or access these elements frequently and sequentially, you can use bisect to maintain the order of list objects and do binary lookups in them to improve the efficiency of lookups.

Another common requirement is to find a minimum or maximum, and you can use the heapq module to convert the list into a heap so that the time complexity of getting the minimum value is.

This is the end of "how to make Python code run faster". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.