Shulou(Shulou.com)06/01 Report--

The first day of the novel website crawler

From today on, learn about crawlers and crawl the novel website.

Day one:

Website: http://www.bxwx9.org

Novel: the Great Master

Language: IDEA+java

Jar package: maven project, so put dependencies. Let's study the function of each jar package.

Project structure:

Requirements: get the title and URL from the chapter list of the novel

Principle:

Use Google browser F12 to view the contents of the page and find the element where the chapter list is located.

Use the tag selector to select what you want

The code is as follows:

The solution of Chinese garbled code:

The effect picture of the operation:

Continue tomorrow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.