In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-02-25 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/02 Report--
This article will explain in detail how to use BeautifulSoup to parse html. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.
The data captured by the crawler is mainly html data. Sometimes it is xml data, and the parsing of tags in xml data is the same as html, both to distinguish the data. The data structure of this format can be said to look like a page, which is very troublesome to parse. BeautifulSoup provides powerful parsing capabilities that can save us a lot of trouble. Install BeautifulSoup and lxml before using.
# pip install beautifulsoup4==4.0.1 # specified version, not specified to install the latest version # pip install lxml==3.3.6 specified version, do not specify to install the latest version and enter the Python command line to see if the installation is successful > import bs4 > import lxml >
No error was reported, indicating that the installation was successful. The version and release time of lxml can be viewed at the following website
First of all, the code should introduce this library.
From bs4 import BeautifulSoup
Then, grab
Try: r = urllib2.urlopen (request) except urllib2.URLError,e: print e.code exit () r.encoding='utf8'print r.codehtml=r.read () # urlopen all the content obtained is in html. The information of mysoup=BeautifulSoup (html, 'lxml') # html is in mysoup.
Suppose we are interested in the following parts of data in html
20200214 1 11 Zhang San 20200214 4 17 Li Sisu
The first step is to find the data labeled by tag, and there is more than one such data. Let's take two pieces of data as an example. Then you need to use the find_all function of beautifulsoup, and the result returned should be two data. When processing each piece of data, the equal tags in it are unique, and the find function is used.
Mysoup=BeautifulSoup (html, 'lxml') data_list=mysoup.find_all (' data') for data in data_list:#list should have two elements day = data.find ('day'). Get_text () # get_text is the get string You can use .string instead of id = data.find ('id'). Get_text () rank = data.find (' rank'). Get_text () name = data.find ('name'). Get_text () # print name can print test the parsing result
This is the simplest use of beautifulsoup, find and find_all can not only locate elements according to the name of the tag, but also according to various attributes such as class,style, as well as text content text as a condition to find the content you are interested in.
On how to use BeautifulSoup to parse html to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.