Shulou(Shulou.com)06/01 Report--

This article mainly explains "how Python crawls the data of Meituan barbecue merchants". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how Python crawls the data of Meituan barbecue merchants".

Data acquisition

Meituan is obviously a dynamic web page, which needs to be crawled through the parsing interface or using Selenium. This paper crawls the data through the parsing interface.

Find the real URL

Meituan URL: https://sz.meituan.com/

Analyze the real URL

Main parameters:

30: city id (30 represents Shenzhen)

Limit: number of stores per page

Offset: page turning parameter (each additional 32 page turns)

Q: keyword (barbecue in this case)

Only 1024 store data can be obtained by crawling according to the above API. In order to obtain more comprehensive data, you also need to find the areaId parameter (sub-region), and then traverse the sub-region to get the complete data. Limited to space, only the core code is given.

Delete duplicate data

Df = df.drop_duplicates ()

Missing value processing

As you can see from the above, only the contact information field contains the missing value and is filled with text.

Df = df.fillna ('no data yet')

Store address cleaning

Intercept the district and county through the store address field. In addition, "South Australia University" belongs to Longgang District and is directly replaced by the replace () method.

Descriptive statistics

1. View basic statistics

Df.describe ()

3. Draw a regression map

By drawing the regression graph, we find that the per capita consumption is positively correlated with the store score, and the number of comments is positively correlated with the store score. This is also close to our common sense.

Data analysis

In this paper, data visualization mainly uses the pyecharts library, which can easily achieve cool chart effects.

Regional distribution

Shenzhen barbecue restaurants are mainly distributed in Longgang District, Longhua District, Nanshan District and Futian District, while there are few barbecue restaurants in Yantian District and Pingshan District. An important factor in the location of the barbecue restaurant is the flow of people. Longgang District and Longhua District are the main living areas in Shenzhen, while Nanshan District and Futian District are the core commercial gathering places in Shenzhen, and the huge demand has laid the foundation for the layout of the barbecue restaurant.

In the business district

Only knowing the distribution of the barbecue restaurant district is of little effect on the location of the barbecue restaurant. As a result, we further refined to the business district to see which areas have more barbecue restaurants. In all the business circles in Shenzhen, there are more than 150 barbecue restaurants in Longhua District, Longhua District and Guangming District.

At this point, I believe you have a deeper understanding of "how Python crawls the data of Meituan barbecue merchants". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.