Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Build the environment before using scrapy

2025-04-07 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >

Share

Shulou(Shulou.com)06/03 Report--

At the beginning, I encountered a lot of problems when I created the project directly by scrapy startproject [name]. Implementing this will prompt that problem, and after solving it, I think it's all right.

1.pip install scrapy prompts pip version to upgrade

You are using pip version 10.0.1, however version 10.0.1 is available.

You should consider upgrading via the 'python-m pip install-- upgrade pip' command.

2.python-m pip install-- there are new problems with upgrade pip upgrade

(venv) D:\ Fish > python-m pip install-- upgrade pip

Requirement already up-to-date: pip in d:\ fish\ venv\ lib\ site-packages (10.0.1)

So delete the pip-10.0.1.dist-info in d:\ fish\ venv\ lib\ site-packages (10.0.1) as required, and run python-m pip install-- upgrade pip again after the upgrade is successful

3. When you implement step 1, I think everything will be all right and there will be new problems.

Error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools

There are two ways to solve this problem: a. Follow the prompts to install Visual Studio (disk C requires at least 6G space, decisively find other ways to solve it)

b. Install using wheel

Install using wheel: pip install wheel

Success will prompt successfull that you can also use the wheel command to verify whether it is successful, and this is true after success

Then go to https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml to download the Scrapy library.

Go to the website = > search for 'Scrapy'= > download

Put the downloaded .whl file in an easy-to-find address (I put it in D:\)

Then enter the address in the console

Enter under that address

Pip install [whl]

Note: [whl] represents your .whl file. Be sure to have the full name, XXXX.whl.

Then, it's not over.

The installation process may still fail

Failed to build XXXX may appear

Why is that? because the Scrapy framework is based on Twisted.

So, all you have to do is go to the website to download installation files such as Twisted, and install them first, the installation method is the same as above.

Pay attention to choosing a good version number

Cp stands for the version number of python. 32thumb 64 selects the number of digits corresponding to your computer.

It finally succeeded.

Build a project and test it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Internet Technology

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report