Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the common methods of python urllib.parse". Friends who are interested may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn what are the common methods of python urllib.parse.

Urllib is a built-in module of python, which is mainly used to deal with some url-related operations, such as accessing url, parsing url, and so on.

Interestingly, urllib and urllib2 once appeared at the same time in python2,python3 and merged the two modules into one, and the community god provided a library of urllib3, which is used to deal with network requests, but these packages are all brothers in requests.

In python3, the parse module is probably the most commonly used under the urllib package, which mainly deals with url-related operations. Especially in crawler applications, it is used more.

1. Urlparse

The urlparse method is to split a complete URL into different parts, which you can extract according to your needs. The returned result ParseResult is a subclass of namedtuple and consists of the following 10 parts, each of which can be obtained either by name or by index in the following table.

> from urllib import parse

> url = 'https://mp.weixin.qq.com/s?__biz=MjM5MzgyODQxMQ==&mid=2650366919&idx=1&sn=1b36a9f2c0921cdeac52942ec591a923#rd'

> result = parse.urlparse (url)

> print (result)

ParseResult (scheme='https', netloc='mp.weixin.qq.com', path='/s', params='', query='__biz=MjM5MzgyODQxMQ==&mid=2650366919&idx=1&sn=1b36a9f2c0921cdeac52942ec591a923', fragment='rd')

# obtain through subscript

> result [0]

'https'

# Protocol

> result.scheme

'https'

# url resource path

> result.path

'/ s'

# query parameters

> result.query

'_ _ biz=MjM5MzgyODQxMQ==&mid=2650366919&idx=1&sn=1b36a9f2c0921cdeac52942ec591a923'

# get hostname

> result.hostname

Mp.weixin.qq.com' II, parse_qs

Parse_qs parses the query parameters into dictionary objects.

> parse.parse_qs (result.query)

{'_ biz': ['MjM5MzgyODQxMQ=='],' mid': ['2650366919'], 'idx': [' 1'], 'sn': [' 1b36a9f2c0921cdeac52942ec591a923']}

> > >

The value for each parameter name is a list object, because in the url specification, a parameter name can correspond to multiple values. However, usually in practical application scenarios, a name will only correspond to one value.

You can convert a list to a string with one line of code

> {name: value [0] for name, value in parse.parse_qs (result.query) .items ()}

{'_ _ biz': 'MjM5MzgyODQxMQ==',' mid': '2650366919,' idx':'1, 'sn':' 1b36a9f2c0921cdeac52942ec591a923'} III, urlencode

Conversely, if you want to convert a dictionary object to a query parameter in url, you can use the urlencode method.

> d = {'_ _ biz': 'MjM5MzgyODQxMQ==',' mid': '2650366919,' idx':'1'}

> > parse.urlencode (d)

'_ _ biz=MjM5MzgyODQxMQ%3D%3D&mid=2650366919&idx=1' IV, quote

URL coding

In the standard specification of URL, url only allows numbers, letters and some special symbols. If you have Chinese, you need to use UTF-8 to encode and convert it into% XX. For example, the result of UTF-8 coding is:

> > "medium" .encode ()

B'\ xe4\ xb8\ xad'

Will be converted to% e4%b8%ad

> parse.quote ("Q = medium")

'Q% 3D% E4% B8% AD'

> > >

For special symbols, it will also be converted to the corresponding hexadecimal symbol, for example, [=] is converted to% 3D

5. Unquote

Unquote is the reverse process of quote.

URL decoding

> parse.unquote ("q%3D%E4%B8%AD")

'Q = medium'

At this point, I believe you have a deeper understanding of "what are the common methods of python urllib.parse?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.