Shulou(Shulou.com)06/01 Report--

This article mainly explains "how Python climbs the articles of Capital Medical University". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how Python climbs the articles of Capital Medical University".

Overview

Crawler project practice

Goal: a new article on the official website of Capital Medical University

Steps

Request URL

Extract data through regular expressions

Analysis data

Code # Import module # for request URL import requests# for parsing web page source code from bs4 import BeautifulSoup# for regular import re# target URL # set the number of pages and extract 10 pages of data page= ['] for i in range (1'] for i in range (1): page.append (I) # save the file with open Encoding='utf-8') as f: for i in page: url= 'http://www.ccmu.edu.cn/zxkylw_12912/index'+str(i)+'.htm' # add header request headers if necessary Prevent anti-crawling and intercept headers= {'User-Agent':'Mozilla/5.0 (Windows NT 10.0) Win64; x64 Rv:79.0) Gecko/20100101 Firefox/79.0'} # Target URL request method is get resp=requests.get (url) html = resp.content.decode ('utf-8') # parsing html soup = BeautifulSoup (html,'html.parser') # find the recently published sci paper # use find and find_all function infos=soup.find (' ul') {'class':'list03'}). Find_all (' li') for info in infos: time=info.find ('span'). Get_text () ajt= info.find (' a'). Get_text () # write to the file f.write ("{}, {}\ n" .format (time,ajt)). Thank you for reading. The above is the content of "how to climb the articles of Capital Medical University of Python". After the study of this article, I believe you have a deeper understanding of how to climb the articles of Capital Medical University of Python, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.