Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Performance comparison of set, dict and dict.keys in python

2025-01-27 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >

Share

Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the performance comparison of set, dict and dict.keys in python". In the daily operation, I believe that many people have doubts about the performance comparison of set, dict and dict.keys in python. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "performance comparison of set, dict and dict.keys in python". Next, please follow the editor to study!

When we count the word frequency of a text, we usually need to query whether the current word has already appeared, and if so, increase the number of times by 1. I usually use dict to save words and frequency. The way I usually do it is:

If word not in vocab_dict: vocab_ words = 0 vocaba _ words + = 1

After using it for a long time, I think the speed is OK.

Later, I saw that a great god used it like this:

If word not in vocab_dict.keys (): vocab_ words = 0 vocababs [word] + = 1

It seems more reasonable, because all I have to do is to see if word is the key of vocab_map. So it has been done in this way since then.

One day, processing a large number of text, and the text belongs to the open domain, the vocabulary is also large, using the second way, the speed is very slow, at this time, I think that the query dict itself is relatively slow, and do not realize that it is the problem of vocab_map.keys (). As a result, the words are stored in set at the same time, and set is looked up when querying, which is much faster.

If word not in vocab_set: vocab_ words = 0 vocaba _ words + = 1

Extract a small part of the data and quantitatively compare the query speed of set, dict and dict.keys (). The results are as follows:

1. Query vocab_dict and print the statement:

Print "time cost is% d ms."% ((end-begin) .microseconds / 1000) time cost is 175 ms.

two。 Query vocab_dict.keys (), and print the statement as follows:

Print "time cost is% d s."% ((end-begin) .seconds) time cost is 45 s.

3. Query vocab_set and print the statement:

Print "time cost is d ms."% ((end-begin) .microseconds / 1000) time cost is 168ms.

It's similar to querying vocab_dict.

So why is dict.keys () much slower than the other two? It is necessary to compare the data structures of list, dict and set.

Dict.keys () is actually list (keys), a list made up of all the key of dict. Find out whether an element in list is traversing list with the subscript of list as the index. As to whether the query is in dict, the key is directly found to the corresponding index of key in the form of hash value, and the value can be accessed directly according to the index. For a large number of dict queries, the latter is naturally much faster.

The storage principle of set and dict is basically the same, the only difference is that set has no value, only key. The effect is basically the same for querying whether key is in dict or sset.

As a result, it can be concluded that if the stored data will be queried repeatedly and the amount is large, then try not to use list, try to use dict, if the elements are not repeated, set is better. Query whether it is within dict, and do not use dict.keys ().

At this point, the study of "performance comparison of set, dict and dict.keys in python" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 236

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Internet Technology

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report