Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how the crawler randomly generates the request header". In the daily operation, it is believed that many people have doubts about how the reptile randomly generates the request header. The editor consulted all kinds of data and sorted out a simple and easy-to-use operation method. I hope it will be helpful to answer the question of "how to randomly generate request headers by reptiles"! Next, please follow the editor to study!

1 # generate random header 2 def randHeader (): 3 head_connection = ['Keep-Alive',' close'] 4 head_accept = ['text/html, application/xhtml+xml, * / *'] 5 head_accept_language = ['zh-CN,fr-FR;q=0.5',' en-US,en;q=0.8,zh-Hans-CN;q=0.5,zh-Hans Qroom0.3'] 6 head_user_agent = ['Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko', 7' Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36', 8 'Mozilla/5.0 (Windows NT 6.1; WOW64 Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; rv:11.0) like Gecko)', 9 'Mozilla/5.0 (Windows; U; Windows NT 5.2) Gecko/2008070208 Firefox/3.0.1',10' Mozilla/5.0 (Windows; U Windows NT 5.1) Gecko/20070309 Firefox/2.0.0.3',11 'Mozilla/5.0 (Windows; U; Windows NT 5.1) Gecko/20070803 Firefox/1.5.0.12',12' Opera/9.27 (Windows NT 5.2; U; zh-cn)', 13 'Mozilla/5.0 (Macintosh; PPC Mac OS X; U) En) Opera 8.0 Magi 14 'Opera/8.0 (Macintosh; PPC Mac OS X; U; en)', 15 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080219 Firefox/2.0.0.12 Navigator/9.0.0.6',16' Mozilla/4.0 (compatible) MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0)', 17 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)', 18 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729) .net CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET 4.0C; .NET 4.0E)', 19 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Maxthon/4.0.6.2000 Chrome/26.0.1410.43 Safari/537.1', 20 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1) WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET 4.0C; .NET 4.0E; QQBrowser/7.3.9825.400)', 21 'Mozilla/5.0 (Windows NT 6.1; WOW64) Rv:21.0) Gecko/20100101 Firefox/21.0', 22 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.92 Safari/537.1 LBBROWSER',23' Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0 BIDUBrowser 2.x)', 24 'Mozilla/5.0 (Windows NT 6.1) WOW64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.11 TaoBrowser/3.0 Safari/536.11'] 25 26 header = {27 'Connection': head_connection [0], 28' Accept': head_accept [0], 29 'Accept-Language': head_accept_language [1], 30' User-Agent': head_user_ agent.Randrange (0) Len (head_user_agent)] 31} 32 return header to this The study on "how crawlers randomly generate request headers" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.