Shulou(Shulou.com)06/01 Report--

Request the first button to convert into a dictionary example analysis, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

When writing crawlers, we often need to copy the request header of the browser to use, but the copied dictionary cannot be used directly. How can we convert it to dictionary form with one click? here are several ways:

Method one, not elegant enough:

Method two

You can find that the replication request header is very simple, as long as all the request headers copy down, and then use headers_raw_to_dict to turn it around, it will directly become dict.

Recommended method method 3

With the help of regular substitution of IDE (PyCharm as an example)

The shortcut key replaced in PyCharm is Ctrl+R, and Regex regular mode is selected.

212121.gif

Source match is (.): (.) $

Replace the match with "$1": "$2"

Method 4: Chrome plug-in Postman

Postman is a powerful debugging tool, but Google has abandoned Chrome App, and plug-ins can only be found directly in the store.

You can install the plug-in and click the link to get Postman App:

Enable Postman Interceptor, and any request for access will be intercepted by Postman.

Click request in Postman-Code- to select Python- and select the appropriate method.

You can see that Postman supports exports in a variety of languages.

You can see that Postman is indeed a very powerful tool that can even help us generate requests code directly, instead of just turning headers into a dictionary.

However, I personally feel that using Postman to do this feels like killing a chicken with a knife. This tool is too heavy. You need to measure the cost of learning, and it is estimated that it will be used less if you only do crawlers instead of Web development. On the other hand, there may still be people who can't access Google.

Method 5: use curlconverter

Curl.trillworks.com/

We need to know what curl is before we use this method.

Curl is a file transfer tool under the Linux command line, which can access URL directly and supports a variety of protocols and parameter settings and even cookie settings. To put it simply, you can simulate a request.

The development tools for Chorme browsers support direct copy as curl, so we just need to do this:

Select the request copy as curl in the development tool and paste it into the website form to get the Python code. This site only supports Python, NodeJS and PHP. This lightweight method should be the most convenient one.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.