Shulou(Shulou.com)06/01 Report--

This article will explain in detail what Python packages are commonly used in, and the editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Urllib3

Number of downloads: 893 million

Urllib3 is a HTTP client for Python that has many features that are missing from the Python standard library:

Thread safety

Connection pool

Client SSL/TLS authentication

Upload files using segmented encoding

Helper used to retry requests and handle HTTP redirects

Support for gzip and deflate encoding

Proxy support for HTTP and SOCKS

Don't be misled by the name, Urllib3 is not the successor to urllib2, which is part of the core of Python. If you want to use as many Python core features as you can, or what you can install is limited, check out urlllib.request.

Https://docs.python.org/3/library/urllib.request.html#module-urllib.request

For end users, I strongly recommend using the requests package (see # 6 in the list). This package ranks first because there are almost 1200 packages that rely on urllib3, and many of them rank high on this list.

Six

Number of downloads: 732 million

Six is a compatibility library for Python 2 and 3. This project aims to support code bases that can run on Python 2 and 3 at the same time.

It provides a number of functions that simplify the syntax differences between Python 2 and 3. An easy to understand example is six.print_ (). In Python 3, printing is done through the print () function, while in Python 2, there are no parentheses after print. So, with six.print_ (), you can use a single statement to support both languages.

Some facts:

Its name is six because two times three equals six.

The same class library can also take a look at the future package.

If you want to convert the code to Python3 (and stop supporting 2), check out 2to3.

Although I understand why it is so popular, I hope people will give up Python 2 completely, because official support for Python 2 has ceased since January 1, 2020.

Botocore 、 boto3 、 s3transfer 、 awscli

Here, I list several related projects together:

Botocore (# 3Pol 660 million downloads)

S3transfer (# 7Pol 584m downloads)

Awscli (# 17Pol 394 million downloads)

Boto3 (# 22Pol 329 million downloads)

Botocore is the underlying interface of AWS. Botocore is the foundation of the Boto3 library (# 22), which allows you to use services such as Amazon S3 and Amazon EC2. Botocore is also the foundation of AWS-CLI, which provides a unified command line interface for AWS.

S3transfer (# 7) is the Python library used to manage Amazon S3 transfers. It is under active development, and its introduction page is not recommended for people to use now, or at least until the version is fixed, as its API may change from minor version to minor version. Boto3, AWS-CLI, and many other projects rely on s3transfer.

It is surprising that these rankings for AWS libraries are so high-which fully shows how powerful AWS is.

Pip

Number of downloads: 627 million

I think most of you know and like pip, which is the package installer for Python. You can use pip to easily install packages from Python package indexes and other indexes (such as local images or custom indexes with proprietary software).

Some interesting facts about pip:

Pip is the acronym for "Pip Installs Packages".

Pip is easy to use. To install a package, you only need pip install, while to delete a package, you only need pip uninstall.

One of the biggest advantages is that it can get a list of packages, usually in the form of a requirements.txt file. This file can select the detailed specification that contains the desired version. Most Python projects contain such files.

If you use pip with virtualenv (# 57 in the list), you can create a predictable isolated environment without interfering with the underlying system, and vice versa. For more details, check out this article:

Python-dateutil

Number of downloads: 617 million

The python-dateutil module provides a powerful extension to the standard datetime module. My experience is that python-dateutil can make up for what is missing from a regular Python datetime.

You can do a lot of great things with this library. One of the features I found particularly useful is to vaguely parse the date in the log file, for example:

Rom dateutil.parser import parselogline = 'INFO 2020-01-01T00:00:01 Happy new year, human.'timestamp = parse (log_line, fuzzy=True) print (timestamp) # 2020-01-01 00:00:01Requests

Number of downloads: 611 million

Requests is based on our # 1 library, urllib3. It makes Web requests very simple. Many people prefer this bag to urllib3. And it may also be used by more end users than urllib3. The latter is more low-level and, given its level of internal control, is generally a dependency on other projects.

Here is an example of how easy it is to use requests:

Import requestsr = requests.get ('https://api.github.com/user', auth= (' user', 'pass')) r.status_code# 200r.headers [' content-type'] # 'application/json; charset=utf8'r.encoding#' utf-8'r.text# u' {"type": "User"... 'r.json () # {upright diskgift gistsgiving: 368627,...} S3transfer

Here # 3, # 7, # 17 and # 22 are introduced together because they are very closely related.

Certifi

Number of downloads: 552 million

In recent years, almost all websites have turned to SSL, which you can identify by the small lock symbol in the address bar. Adding a small lock means that communication with the site is secure and encrypted, preventing eavesdropping.

The lock tells us that this site has been protected by SSL

The encryption process is based on SSL certificates, and these SSL certificates are created by trusted companies or non-profit organizations such as LetsEncrypt. These organizations use their (intermediate) certificates to digitally sign these certificates.

Your browser uses the publicly available parts of these certificates to verify these signatures, which ensures that you are looking at the real content and that no one can snoop on the communication data. Python software can do the same thing. This is what certifi is for. It is not much different from the root certificate collection that comes with web browsers such as Chrome, Firefox, and Edge.

Certifi is a select collection of root certificates with which your Python code can verify the credibility of SSL certificates.

As shown here, many projects trust and rely on certifi. This is why the project ranks so high.

Idna

Number of downloads: 527 million

According to its PyPI page, idna provides "support for the application Internationalized Domain name (IDNA) protocol specified in RFC5891."

Maybe you are as confused as I am. I don't know what Idna is and what's the use! It is reported that the internationalized domain name (IDNA) in the application is a mechanism for dealing with domain names that contain non-ASCII characters. However, the original domain name system already provides support for domain names based on non-ASCII characters. So, what's the problem?

The problem is that applications, such as email clients and Web browsers, do not support non-ASCII characters. More specifically, the protocols used for email and HTTP do not support these characters. This is not a problem for many countries, but it is a problem for countries such as China, Russia, Germany, Greece and Indonesia. Finally, a group of smart people from these places thought of IDNA.

At the core of IDNA are two functions: ToASCII and ToUnicode. ToASCII converts the international Unicode domain to an ASCII string. ToUnicode reversed the process. In the IDNA package, these functions are called idna.encode () and idna.decode (), as shown in the following code snippet:

Import idnaidna.encode ('thank you very much. ) # b'xn--eckwd4c7c.xn--zckzah'print (idna.decode ('xn--eckwd4c7c.xn--zckzah')) # thank you. Seeing a doctor and seeing a doctor

If you are a masochist, you can read RFC-3490 for more information about this code.

PyYAML

Number of downloads: 525 million

YAML is a data serialization format. It is designed to make it easy for both humans and computers to read code-it is easy for humans to read and write, and computers can parse it.

PyYAML is Python's YAML parser and emitter, which means it can read and write YAML. It writes any Python object as YAML: lists, dictionaries, and even class instances.

Python provides its own configuration parser, but YAML provides more functionality than the basic .ini file structure of Python's ConfigParser.

Https://docs.python.org/3/library/configparser.html

For example, YAML can store any data type: Boolean, list, floating point, and so on. ConfigParser stores everything as an internal string. If you want to load integers using ConfigParser, you need to specify that you want to explicitly get an int:

Config.getint ("section", "my_int")

Pyyaml automatically recognizes the type, so this will return your int using PyYAML:

Config ["section"] ["my_int"]

YAML also allows arbitrary deep trees, which is not needed for every project, but it can come in handy when needed. You may have your own preferences, but many projects use YAML as the configuration file, so this project is very popular.

Pyasn1

Number of downloads: 512 million

Like IDNA above, this project is also very useful:

Pure Python implementation of ASN.1 type and DER/BER/CER encoding (X.208)

Fortunately, this decades-old standard has a lot of information available. ASN.1, which stands for Abstract Syntax Notation One, is like the godfather of data serialization. It comes from the telecommunications industry. Maybe you know the protocol buffer or Apache Thrift? This is their 1984 version.

ASN.1 describes the cross-platform interfaces between systems and the data structures that can be sent through that interface.

Remember Certifi (see # 8)? ASN.1 is used to define the certificate format used in the HTTPS protocol and many other encryption systems. It is also used in SNMP, LDAP, Kerberos, UMTS, LTE and VOIP protocols.

A piece of advice, unless you really need it, stay away. But because it is used in many places, many packages rely on this package.

Docutils

Number of downloads: 508 million

Docutils is a modular system for processing plain text documents into many useful formats, such as HTML, XML, and LaTeX. Docutils can read plain text documents in reStructuredText format, which is an easy-to-read tag syntax similar to MarkDown.

So what is an PEP document? The earliest PEP document, PEP-1, provides us with a good explanation:

PEP means Python enhancement proposal. A PEP is a design document that provides information to the Python community or describes new features of Python or its process or environment. PEP should provide a concise technical specification of this function and the principle of the function.

PEP documents are written using a fixed reStructuredText template and converted to a well-formed document using docutils.

Docutils is also the core of Sphinx. Sphinx is used to create document projects. If Docutils is a machine, then Sphinx is the factory. It was originally created to build Python documents, but many other projects also use it to document code. You may have read the documents on readthedocs.org, where most of the documents are created by Sphinx and docutils.

Chardet

Number of downloads: 501 million

You can use the chardet module to detect the character set of files or data streams. This is useful, for example, when you need to analyze a large number of random texts. But you can also use it when dealing with remotely downloaded data, but you don't know what character set you are using.

After installing chardet, you also have a command-line tool called chardetect, which is used as follows:

Chardetect somefile.txtsomefile.txt: ascii with confidence 1.0

You can also use this library programmatically, as detailed in the documentation. Chardet is a requirement for many packages, such as requests. I don't think many people use chardet alone, so it must be because of these dependencies that it is so popular.

RSA

Number of downloads: 492 million

The rsa package is a pure Python RSA implementation. It supports:

Encryption and decryption

Signing and verifying signatures

Generate keys based on PKCS#1 version 1.5

It can be used either as a Python library or on the command line.

Some facts:

RSA is the initials of RonRivest, Adi Shamir and Leonard Adleman. They invented the algorithm in 1977.

RSA is one of the earliest public key cryptosystems, which is widely used in secure data transmission. In such a cryptosystem, there are two keys: the public part and the private part. You encrypt the data with the public key and can only decrypt the data with the private key.

RSA is a kind of slow algorithm. It is rarely used to encrypt user data directly. Typically, RSA is used to securely pass shared keys encrypted with symmetric keys, which encrypts and decrypts large amounts of data much faster.

The following code snippet shows how to use RSA in a very simple use case:

Import rsa# Bob creates a key pair: (bob_pub, bob_priv) = rsa.newkeys # Alice ecnrypts a message for Bob# with his public keycrypto = rsa.encrypt ('hello Bobbundles, bob_pub) # When Bob gets the message, he# decrypts it with his private key:message = rsa.decrypt (crypto, bob_priv) print (message.decode (' utf8')) # hello Bob!

Assuming that Bob keeps his private key private, Alice can be sure that he is the only one who can read the message. However, Bob is not sure that Alice sent the message, because anyone can obtain and use his public key. To prove that it is her, Alice can sign the email with her private key. Bob can verify this signature with her public key to ensure that she did send the message.

Packages such as google-auth (# 37), oauthlib (# 54), awscli (# 17) all rely on rsa packages. Few people will use this tool independently because there are faster, more native alternatives.

Jmespath

Number of downloads: 473 million

Using JSON in Python is very easy because it maps very well in the Python dictionary. For me, this is one of its best features.

To tell you the truth-although I've done a lot of work with JSON, I've never heard of this package. I just use json.loads () and get the data manually from the dictionary, maybe make a loop or something.

JMESPath, pronounced "James path", makes JSON in Python easier to use. It allows you to specify declaratively how to extract elements from JSON documents. Here are some basic examples:

Import jmespath# Get a specific elementd = {"foo": {"bar": "baz"} print (jmespath.search ('foo.bar', d)) # baz# Using a wildcard to get all namesd = {"foo": {"bar": [{"name": "one"}, {"name": "two"}]} print (jmespath.search (' foo.bar [*] .name', d)) # [one "," two "] Setuptools

Number of downloads: 401 million

It is a tool for creating Python packages. However, the documentation is poor. It does not clearly describe its purpose, and the document contains invalid links. The best source of information is this site, especially the guide to creating Python packages.

Awscli

Here # 3, # 7, # 17 and # 22 are introduced together because they are very closely related.

Pytz

Number of downloads: 394 million

Like dateutils (# 5), this library helps you deal with dates and times. Sometimes, the time zone can be troublesome to deal with. Fortunately, there is such a bag, which can make things easier.

To sum up, my own experience with processing time on computers is to always use UTC internally. It is converted to local time only when the output is generated for human reading.

This is an example of pytz usage:

From datetime import datetimefrom pytz import timezoneamsterdam = timezone ('Europe/Amsterdam') ams_time = amsterdam.localize (datetime (2002, 10, 27, 6, 0, 0) print (ams_time) # 2002-10-27 06-27 00-06-27 06:00:00+02:00Futures: ams_time = amsterdam.localize (datetime (2002, 6, 27, 6, 0, 0) print (ams_time) # 2002-06-27 06:00:00+02:00Futures

Number of downloads: 389 million

Starting with Python 3.2, python provides current.futures modules to help you implement asynchronous execution. The futures package is the backport of the library for Python 2. It is not suitable for Python3 users because the module is natively provided by Python3.

As I mentioned earlier, official support for Python 2 ceased as of January 1, 2020. I hope this bag won't be in the top 22 when I look at it again next year.

Here is a basic example of futures:

From concurrent.futures import ThreadPoolExecutorfrom time import sleepdef return_after_5_secs (message): sleep (5) return messagepool = ThreadPoolExecutor (3) future = pool.submit (return_after_5_secs, ("Hello world")) print (future.done ()) # Falsesleep (5) print (future.done ()) # Trueprint (future.result ()) # Hello World

As you can see, you can create a thread pool and submit a function to be executed by one of these threads. In the meantime, your program will continue to run in the main thread. This is an easy way to execute programs in parallel.

20Colorama

Number of downloads: 370 million

With Colorama, you can add some colors to the terminal:

This is very easy to do, as shown in the following sample code:

From colorama import Fore, Back, Styleprint (Fore.RED + 'some red text') print (Back.GREEN +' and with a green background') print (Style.DIM + 'and in dim text') print (Style.RESET_ALL) print (' back to normal now') Simplejson

Number of downloads: 341 million

What's wrong with the native json module that requires this advanced alternative? No! In fact, the json of Python is simplejson. But simplejson also has some advantages:

It applies to more Python versions.

It updates more frequently than Python.

It uses the (optional) part written in C, so it's very fast.

You will often see the following in scripts that support JSON:

Try: import simplejson as jsonexcept ImportError: import json

Unless you need something that is not in the standard library, I will only use json. Simplejson can be much faster than json because it has some parts implemented in C. Unless you are working on thousands of JSON files, this advantage is not a big deal for you. You can also take a look at UltraJSON, which should be faster because almost all of its code is written in C.

Boto3

Here # 3, # 7, # 17 and # 22 are introduced together because they are very closely related.

This is the end of this article on "what are the common packages of Python?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.