Shulou(Shulou.com)06/01 Report--

Today, I would like to share with you how python web crawler realizes the relevant knowledge of personalized music player. The content is detailed and the logic is clear. I believe most people still know too much about this, so share this article for your reference. I hope you can get something after reading this article. Let's take a look at it.

Development component

The above version of python3.5 is fine.

Tkinter (a module included with python for graphical user interface development)

Requests (crawler module)

Chrome browser

Wait

Function

Music downloader function:

(1). The user enters the name of the song or keyword that he wants to download

(2) the program obtains the keyword of the song name entered by the user, and initiates a data request to the website according to the "keyword"

(3) after analyzing the returned data information, the program presents the form of the list to the user

(4) users can click on the corresponding song shown in the list and click download to download the song.

Process analysis

Python-based personalized music downloader module details (1) Front-end module # create window window = tkinter.Tk () # set title window.title ('music downloader',) # set window size and location window.geometry ('900x460' 500music 100') # tag component lab = Label (window,text=' Please enter the song to be downloaded:', font= ('Chinese line letters', 15) # tag location lab.grid (row=0 Column=0) # input box component entry = Entry (window,font= ('official script', 20), width=20) entry.grid (row=0,column=1) # search button btn = Button (window,text=' search', font= ('official script', 15), width=20,command=get_music_list) btn.grid (row=0,column=2) # list box # multiple selections can be set, selectmode=MULTIPLElistbox = Listbox (window,font= ('official script', 16), width=45,heigh=15) listbox.grid (row=1,columnspan=2) listbox_1 = Listbox (window Font= ('official script', 16), width=35,heigh=15) listbox_1.grid (row=1,column=2) # download button btn_1 = Button (window,text=' starts downloading', font= ('official script', 15), command=get_music_url) btn_1.grid (row=2,column=0) # exit button btn_2 = Button (window,text=' exit program', font= ('official script', 15), command=window.quit) btn_2.grid (row=2,column=2) # message loop display interface window.mainloop ()

The cammand in the code is used to bind the backend function.

This piece of code is used to realize visual interaction with users. I won't say much about this piece. You can achieve it by learning the tkinter module that comes with python. The effect is shown in the figure.

(2) backend module

1. Analysis.

Go to Kuwi music website

Let's first climb down the names of these songs and then add them to the front-end list box.

The specific functions are as follows

# get the music search list function def get_music_list (): # get the content entered in the front search box name = entry.get () # clear the list box for displaying the new playlist listbox.delete (0, END) # Loop load three pages of music for k in range (1Jing 4): url = 'http://www.kuwo.cn/api/www/search/searchMusicBykeyWord?' Data = {'key': name,' pn': k, 'rn':' 30 minutes, 'httpsStatus':' 1 miles, 'reqId':' 161cbd51-cde1-11ebmurb bf58Mub bba623268fbb'} response = requests.get (url=url,headers=headers,params=data) Proxies=proxy) .text dic_data = json.loads (response) list_data = dic_data ['data'] [' list'] # listbox.delete (zero end) for i in list_data: music_name = I ['name'] +' -'+ I ['artist'] # insert the music name listbox.insert (END) into the list box Music_name) music_id = I ['rid'] list_1.append (music_id)

At this point, the music list is added to the list box. Notice that we have added the rid of all the music to the list_1 list, and the index of each rid corresponds to the music name index in the list box, which is the key to click on the corresponding song to download. Then we need to implement the click-and-download function.

The specific functions are as follows

# def get_music_url (): index = listbox.curselection () # returns the selected index, which is a tuple for i in index: music_name = listbox.get (I) listbox_1.insert (END) Music_name + 'start downloading') url= 'http://www.kuwo.cn/url?format=mp3&rid={}&response=url&type=convert_url3&br=128kmp3&from=web&t=1616159211200&httpsStatus=1&reqId=f8586c01-88b3-11eb-b442-d7b57b6d2564'.format(list_1[i]) music_data = requests.get (url=url,headers=headers,proxies=proxy). Text # print (music_data) e =' {"code": 200, "msg": "success" "url": (. *?)} 'music_url = re.findall (e, music_data, re.S) [0] # print (music_url) download_music (music_url,music_name)

Finally download and save.

The specific functions are as follows

# create a folder in the current directory Used to store downloaded music if not os.path.exists ('cool music'): os.mkdir ('cool music') # download music and save the function def download_music (url,music_name): music_name = music_name+ ".mp3" music = requests.get (url=url,proxies=proxy). Content path = 'cool music /' + music_name with open (path 'wb') as fp: fp.write (music) listbox_1.insert (END, music_name+' download complete!)

At the same time, we will be anti-crawling when we write crawlers with python. The following code is an anti-crawling measure.

# ip proxy pool ip = ['{"HTTP": "175.42.129.105"}','{"HTTP": "121.232.148.97"}','{"HTTP": "121.232.148.72"}'] proxy = random.choice (ip) proxy = json.loads (proxy) # reverse climbing headersheaders = {'User-Agent':' Mozilla/5.0 (Windows NT 10.0) WOW64) AppleWebKit/537.36 (KHTML,likeGecko) Chrome/90.0.4430.85Safari/537.36', 'Referer':' http://www.kuwo.cn/search/list?key=%E7%83%AD%E7%88%B1105%C2%B0C%E7%9A%84%E4%BD%A0', 'csrf':' 0BXQD7I99LN, 'Cookie':',}

The cookie here is to maintain your login status, which is equivalent to verifying your identity and writing your own.

Here csrf and cookie write your own on the line, I will be empty in the source code, pay attention to modification. Also, log in to the website before copying cookie, and cookie will

Record your login status. You have to log in.

The cookie and csrf here are in the header of the first request, as shown below

source code

Here's the code. You're welcome to take it.

Import requestsimport randomimport jsonimport reimport osimport tkinterfrom tkinter import * list_1 = [] # list of music rid # ip proxy pool ip = ['{"HTTP": "175.42.129.105"}','{"HTTP": "121.232.148.97"}' '{"HTTP": "121.232.148.72"}'] proxy = random.choice (ip) proxy = json.loads (proxy) # reverse climbing headersheaders = {'User-Agent':' Mozilla/5.0 (Windows NT 10.0) WOW64) AppleWebKit/537.36 (KHTML,likeGecko) Chrome/90.0.4430.85Safari/537.36', 'Referer':' http://www.kuwo.cn/search/list?key=%E7%83%AD%E7%88%B1105%C2%B0C%E7%9A%84%E4%BD%A0', 'csrf':', # fill in your 'Cookie':', # fill in your} # create a folder in the current directory Used to store downloaded music if not os.path.exists ('cool music'): os.mkdir ('cool music') # download music and save the function def download_music (url,music_name): music_name = music_name+ ".mp3" music = requests.get (url=url,proxies=proxy). Content path = 'cool music /' + music_name with open (path 'wb') as fp: fp.write (music) listbox_1.insert (END, music_name+' download complete!) # def get_music_url (): index = listbox.curselection () # returns the selected index, which is a tuple for i in index: music_name = listbox.get (I) listbox_1.insert (END) Music_name + 'start downloading') url= 'http://www.kuwo.cn/url?format=mp3&rid={}&response=url&type=convert_url3&br=128kmp3&from=web&t=1616159211200&httpsStatus=1&reqId=f8586c01-88b3-11eb-b442-d7b57b6d2564'.format(list_1[i]) music_data = requests.get (url=url,headers=headers,proxies=proxy). Text # print (music_data) e =' {"code": 200, "msg": "success" "url": "(. *?)"} 'music_url = re.findall (e, music_data, re.S) [0] # print (music_url) download_music (music_url,music_name) # function def get_music_list (): name = entry.get () listbox.delete (0) END) # Loop indicates loading three pages of music for k in range (1P4): url = 'http://www.kuwo.cn/api/www/search/searchMusicBykeyWord?' Data = {'key': name,' pn': k, 'rn':' 30 minutes, 'httpsStatus':' 1 miles, 'reqId':' 161cbd51-cde1-11ebmurb bf58Mub bba623268fbb'} response = requests.get (url=url,headers=headers,params=data) Proxies=proxy) .text dic_data = json.loads (response) list_data = dic_data ['data'] [' list'] # listbox.delete (zero end) for i in list_data: music_name = I ['name'] +' -'+ I ['artist'] listbox.insert (END Music_name) music_id = I ['rid'] list_1.append (music_id) # create window window = tkinter.Tk () # set title window.title (' Music Downloader',) # set window size and location window.geometry ('900x460' 500') # tag component lab = Label (window,text=', please enter the song to download:', font= ('Chinese Xingkai') 15)) # # tag positioning lab.grid (row=0,column=0) # input box component entry = Entry (window,font= ('official script', 20), width=20) entry.grid (row=0,column=1) # search button btn = Button (window,text=' search', font= ('official script', 15), width=20,command=get_music_list) btn.grid (row=0,column=2) # list box # multiple selections can be set, selectmode=MULTIPLElistbox = Listbox (window,font= ('official script', 16), width=45 Heigh=15) listbox.grid (row=1,columnspan=2) listbox_1 = Listbox (window,font= ('official script', 16), width=35,heigh=15) listbox_1.grid (row=1,column=2) # download button btn_1 = Button (window,text=' starts downloading', font= ('official script', 15), command=get_music_url) btn_1.grid (row=2,column=0) # exit button btn_2 = Button (window,text=' exit program', font= ('official script', 15), command=window.quit) btn_2.grid (row=2) Column=2) # message Loop display Interface window.mainloop ()

Effect picture

These are all the contents of the article "how python web crawlers implement personalized music players". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.