Shulou(Shulou.com)06/03 Report--

Today, I will talk to you about how the Python series of crawlers crawl and visualize the data of Ctrip tourist attractions. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

Preface

Today we will use the crawler Ctrip tourist attractions data crawling and visualization and do a simple data visualization analysis. Let's get started happily.

Development tool Python version: 3.6.4 related modules:

Bs4 module

Jieba module

Pyecharts module

Wordcloud module

Requests module

And some modules that come with Python.

Environment building

Install Python and add it to the environment variable, and pip installs the relevant modules you need.

Data crawling

First of all, let's make it clear what data we want to crawl. Here, for convenience, we only crawl the data of tourist attractions in Beijing, as shown in the following figure:

That is, the data we need to crawl is the name, location, score and other data of all scenic spots in Beijing. Once we have a clear goal of crawling, we can start writing code.

The code is actually very simple to implement. You can find that the url of the scenic spot information page changes as follows:

'https://you.ctrip.com/sight/beijing1/s0-p page number .html # sightname'

Then we just need to request all the relevant web pages one by one and parse and extract the data we need with the help of bs4. At the same time, in order to prevent the crawler from being blocked, we change an agent every 10 times we request a web page, and the source of the agent is a free agent for online crawling.

Specifically, the code implementation is as follows:

'' Ctrip tourist attractions reptiles''

The code works as follows:

For more information on the complete source code of All done~, please see personal introduction or private message to obtain relevant documents.

Data visualization

The old rule, after writing the crawler to visualize a wave of data, for convenience, let's use the Beijing scenic spot data we just crawled.

First of all, make the location information of all the scenic spots into a word cloud.

Take a look at the score distribution of the scenic spots:

Let's count the rating distribution of scenic spots:

Among them, 5A-level scenic spots are:

Imperial Palace

Let's take a look at the price distribution:

Finally, what are the eight scenic spots with the most comments?

After reading the above, do you have any further understanding of how Python crawlers crawl and visualize the data of Ctrip tourist attractions? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.