Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to use Python to crawl Taobao comment data, many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

[project preparation]

1. Prepare Pycharm, download and install, etc., will not install, you can reply: python, get the network disk resources installation package!

two。 Crawl for the product address, as follows:

Https://detail.tmall.com/item.htm?spm=a230r.1.14.1.55a84b1721XG00&id=552918017887&ns=1&abbucket=17

3. How many libraries need to be downloaded? how to download them?

Open the pycharm software, click File, click the setting option, and then select the Project:Interpreter option under Project: your file name.

Click the + sign to install the libraries needed for this project, such as requests, beautifulsoup4, simplejson.

[project implementation]

1. Import required libraries

Import requestsfrom bs4 import BeautifulSoup as bsimport jsonimport csvimport re

two。 Need to log in to Taobao, choose Google browser, select developer tools or press F12 to have a Network option to find list_detail_rate.htm? File

Define a variable storage URL address PAGE_URL = []

Define a function to generate a link list that uses string concatenation to form the number of pages for comments

Define the fields that need to be used in the function to get comment data, such as: user name, comment time, color classification, comment, as shown in the following figure.

It would be nice to have an item.htm?spm file and a cookie copy of the cookie value in the developer tool Network.

Parse the contents of the JS file and write the data to the TEXT file, as shown in the following figure.

Finally, define the number of comment pages needed for a main function to crawl, as shown in the following figure.

The final result is shown in the figure:

After reading the above, do you have any further understanding of how to crawl Taobao comment data with Python? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.