Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to use the Python Scrapy framework in the Mac environment, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Preface

Download Scrapy

Enter the code 'pip3 install scrapy'' in the terminal.

Create a Scrapy project

Suppose you want to save the project to a folder named Python on your desktop. You can type'cd'in the terminal and drag the Python folder behind it. After the line break, continue to enter the ScrapyProject here in 'scrapy startproject ScrapyProject',' as the name of the project, which can be defined according to the situation.

Create a crawler file

Also in the ScrapyProject folder, find the spiders folder and create a new crawler file inside, such as book.py in the figure below. (need to be at the same level as _ pychache_ and _ init_ files. )

Edit crawler file

The crawler file here is the core of the entire Scrapy project, and the data parsing and extraction are carried out in this step.

Run the Scrapy program

In the ScrapyProject folder, find the settings.py document and change the user_agent related information. Then enter the code scrapy crawl book in the terminal. (book is the name of the reptile. )

Note: the above contents are all operations in Mac environment.

Thank you for reading this article carefully. I hope the article "how to use the Python Scrapy Framework in Mac" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.