Shulou(Shulou.com)06/02 Report--

This article introduces you how to use Python to grab QQ Music data, the content is very detailed, interested friends can refer to, hope to be helpful to you.

[1. Project objectives]

Get the song name, album name and playback link of the song for which QQ Music's singles rank the specified number of pages.

From shallow to deep, layer by layer, it is very suitable for beginner students to practice.

[II. Required libraries]

The main libraries involved are: requests, json, openpyxl

[III. Project realization]

1. Learn about the robots protocol of QQ Music's website

Only playlists are prohibited and can be manipulated.

two。 Enter QQ Music's home page https://y.qq.com/

3. Enter any singer, such as Deng Ziqi

4. Open the review element (shortcut key Ctrl+Shift+I)

5. Analysis of the web source code Elements, found that no song information, can not use BeautifulSoup, as shown in the following figure, the result is empty.

6. Click Network to see if the data is in XHR (no refresh update page)

Face), my experience is to first look at the largest Size, and then analyze Name

Check Preview, sure enough, it's inside!

7. Click Headers to get the relevant parameters. As shown in the picture below, observe carefully

The relationship between url and Query String Parameters parameters is found.

The w in url represents the name of the singer and p represents the number of pages.

8. Through the json code, first try the bull's knife, climb the first page

The data is copied directly by url. Success!

9. The params parameter is introduced to realize the query of specified singer and specified number of pages.

Note that the code url is "?" in the previous step url. In the previous part, the parameters on both sides of the params need to be added'', requests.get to add params, parameters (you can also add the headers parameter by the way)

10. Add storage function and save it locally (Excel). It can also be saved in csv format or stored in a database, with similar operations.

1. Climbing QQ Music is a little more difficult than climbing websites such as Douban. The information you need is not in the source code of the web page. Check XHR.

two。 Crawling data through XHR generally uses json, which is in the following format:

Res = requests.get (url) json = res.json () list = json ['] [']...

3. For reference only, it is not recommended to crawl too much data to increase the load on the server.

4.Python crawling QQ Music data (2) will show you how to climb the lyrics and comments of a specified song (selenium) and generate a word wordcloud.

On how to use Python to capture QQ Music data to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.