Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to let Python crawl paid songs. Many people may not know much. In order to let everyone know more, Xiaobian summarized the following contents for everyone. I hope you can gain something according to this article.

Today, I'm going to teach you a tool called selenium.

selenium

Selenium is a Web automation tool, originally developed for website automation testing, like a key wizard for playing games, which can be automatically operated according to specified commands.

Selenium testing tools manipulate the browser directly, just like real users do. Selenium can automatically load a page, get the required data, even take a screenshot of the page, or determine whether certain actions occur on the website.

project objectives

Today's goal is to crawl paid songs

Victim's address

http://tool.liumingye.cn/music/? page=homePage

crawler code

import tool

import timeimport requestsfrom selenium import webdriverfrom selenium.webdriver.chrome.options import Options

Parsing Web Pages

Open F12 developer tools, regardless of 37 21, first come to a blind analysis ~

driver = webdriver.Chrome(executable_path ='chromedriver.exe', options=chrome_options)# key_world = input ('please enter artist name:') driver.get tool.liumingye.cn/music/? page=searchPage')driver.find_element_by_css_selector('#input').send_keys ('Zhang Jie') driver.find_element_by_css_selector('#search button:nth-child(2) i').click()def download(name, url): filename ='C:\\Users\\Administrator\\Desktop\\Music\\' + name + '.mp3' response = requests.get(url=url) with open(filename, mode='wb') as f: f.write(response.content)def drop_down(): """Sims scroll down the page""" for x in range(1, 20, 10): time.sleep(0.5) j = x / 10 js = 'document.documentElement.scrollTop = document.documentElement.scrollHeight * %f' % j driver.execute_script(js)lis = driver.find_elements_by_css_selector('#player li')f = 0for li in lis: f += 1 name = li.find_element_by_css_selector('.aplayer-list-title').text li.find_element_by_css_selector('.aplayer-list-download').click() down_url = driver.find_element_by_css_selector('#m-download > div > div > div.modal-body > div:nth-child(6) > div.input-group-append > a.btn.btn-outline-secondary.download').get_attribute('href') driver.find_element_by_css_selector('#m-download > div > div > div.modal-header > button').click() # time.sleep(1) download(name, down_url) print(name, down_url) if f % 10 == 0: drop_down()

After running the code, the effect is as follows

After reading the above, do you know more about how to let Python crawl paid songs? If you still want to know more knowledge or related content, please pay attention to the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.