Shulou(Shulou.com)06/02 Report--

This article shows you how to use Python to crawl the three major dating software review areas, the content is concise and easy to understand, can definitely brighten your eyes, through the detailed introduction of this article, I hope you can get something.

Mistress: what's the matter, waiter? With a sad face.

Waiter: alas! It's almost Chinese New year, and the family is urging for a blind date again.

Mistress: isn't online dating popular now? you can try dating software.

Waiter: is this thing reliable?

Mistress: I haven't used it either. take a look at the software review area for yourself.

Waiter: this. But we can only go to the comments section first.

To 360mobile assistant as an example, address: http://zhushou.360.cn/, dating software to choose three more popular, respectively: century Jiayuan, Lily Marriage, Yuanyuan, we use Python to crawl the software comment area to see the user evaluation.

First, let's take a look at the downloads of these three softwares and the proportion of good and bad reviews (the figure below is ten thousand times).

Let's start to crawl the comment area. Take Jiayuan of the century as an example. First, enter Jiayuan of the century in the search box to search, as shown in the figure:

Next, click the searched software to enter its details page, as shown in the figure:

Drop the page down to see the comments area, as shown in the figure:

At this point, open the developer tool and select the Network tab, click to view more comments, and then you can see the getComments request, as shown in the figure:

Through this request, we can dynamically obtain the comment area data. Parameter star is the initial comment index, and parameter count is the number of comments loaded each time. You can specify different applications through parameters callback and baike. The crawling code is implemented as follows:

Headers = {

"Accept": "* /"

"Accept-Encoding": "gzip, deflate, sdch"

"Accept-Language": "zh-CN,zh;q=0.8"

"Connection": "keep-alive"

"Host": "comment.mobilem.360.cn"

"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER"

}

Def comment_spider (param, file_name):

Base_url = "http://comment.mobilem.360.cn/comment/getComments?c=message&a=getmessage&&count=50"

Start = 0

For i in range (1,50):

Print ("page {}" .format (I))

Url = base_url + param + "& start=" + str (start)

R = requests.get (url, headers=headers)

Data = re.findall ("{\" errno\ "(. *)\);} catch\ (e\) {}", r.text)

# convert to Json format

Jdata = json.loads ("{\" errno\ "" + data [0])

For message in jdata ["data"] ["messages"]:

Content = message ["content"]

Print (content)

With open (file_name + ".txt", "a", encoding= "utf-8") as f:

F.write (content)

Start = start + 50

Time.sleep (2)

We saved the crawled comment data in the txt file.

Next, we will show the comment data in a word cloud, and the code is implemented as follows:

With open ("yy.txt", "r", encoding= "utf-8") as f:

Content = f.read ()

Stylecloud.gen_stylecloud (text=content, max_words=600

Collocations=False

Font_path= "SIMLI.TTF"

Icon_name= "fas fa-heart"

Size=800

Output_name= "yy.png")

Image (filename= "yy.png")

Finally, take a look at the user's evaluation of the above software through Ciyun.

Century Jiayuan:

Lily Marriage:

Predestined Network:

Waiter: after reading the comments of Yuanyuan, I feel that I have no chance with the dating software.

The other woman:.

The above content is how to use Python to crawl the three major dating software review areas, have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.