Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use Python to climb Arena of Valor's skin". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "how to use Python to climb Arena of Valor's skin"!

Basic development environment

Python 3.6

Pycharm

Use of related modules import requests # third-party module import parselimport time # time module

Install Python and add it to the environment variable, and pip installs the relevant modules you need.

First, determine the target requirements

According to re, xpath or css selector can extract data, or relatively simple. Crawling IP is mainly because some websites are more likely to be blocked IP when crawlers are used to crawl data frequently.

Although there are a lot of free IP agents available on the site, they basically don't work.

Complete code import requests # third-party module import parselimport time # time module def check_ip (proxies_list): "check the availability of the agent ip" use_proxy = [] for ip in proxies_list: try: response = requests.get (url=' https://www.baidu.com', proxies=ip Timeout=2) if response.status_code = = 200: use_proxy.append (ip) except Exception as e: print ('current agent ip:', ip, 'request timed out, failed test!') Else: print ('current proxy ip:', ip, 'Detection passed') return use_proxyproxy_list = [] for page in range (1,11): time.sleep (0.5) print (flip = crawling page {page} data =') # 1. Determine the address of the data (analyze the nature of the web page) url = f 'http://www.ip3366.net/?stype=1&page={page}' headers = {' User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36'} # 2. Send a network request response = requests.get (url=url, headers=headers) html_data = response.text # str # print (html_data) # 3. Parsing data # 3.1 convert data type selector = parsel.Selector (html_data) # 3.2 data extraction trs = selector.xpath ('/ / table [@ class= "table table-bordered table-striped"] / tbody/tr') # tr "" # structure of proxy ip proxies_dict = {"http": "http://" + ip: Port "https": "http://" + ip: Port,}"for tr in trs: ip_num = tr.xpath ('. / td [1] / text ()'). Get () ip_port = tr.xpath ('. / td [2] / text ()'). Get () # print (ip_num Ip_port) ip_proxy = ip_num +':'+ ip_port # print (ip_proxy) proxies_dict = {'http': "http://" + ip_proxy,' https':" https://" + ip_proxy} # 4. Proxy_list.append (proxies_dict) print ('saved successfully:', proxies_dict) print (proxy_list) print ('number of agents obtained:', len (proxy_list)) print ('= = agents being tested = =') can_use = check_ip (proxy_list) print ('available agents:', can_use) print ('number of available agents:', len (can_use))

At this point, I believe you have a deeper understanding of "how to use Python to climb Arena of Valor's skin". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.