Build the environment before using scrapy

At the beginning, I encountered a lot of problems when I created the project directly by scrapy startproject [name]. Implementing this will prompt that problem, and after solving it, I think it's all right.

1.pip install scrapy prompts pip version to upgrade

You are using pip version 10.0.1, however version 10.0.1 is available.

You should consider upgrading via the 'python-m pip install-- upgrade pip' command.

2.python-m pip install-- there are new problems with upgrade pip upgrade

(venv) D:\ Fish > python-m pip install-- upgrade pip

Requirement already up-to-date: pip in d:\ fish\ venv\ lib\ site-packages (10.0.1)

So delete the pip-10.0.1.dist-info in d:\ fish\ venv\ lib\ site-packages (10.0.1) as required, and run python-m pip install-- upgrade pip again after the upgrade is successful

3. When you implement step 1, I think everything will be all right and there will be new problems.

Error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools

There are two ways to solve this problem: a. Follow the prompts to install Visual Studio (disk C requires at least 6G space, decisively find other ways to solve it)

b. Install using wheel

Install using wheel: pip install wheel

Success will prompt successfull that you can also use the wheel command to verify whether it is successful, and this is true after success

Then go to https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml to download the Scrapy library.

Go to the website = > search for 'Scrapy'= > download

Put the downloaded .whl file in an easy-to-find address (I put it in D:\)

Then enter the address in the console

Enter under that address

Pip install [whl]

Note: [whl] represents your .whl file. Be sure to have the full name, XXXX.whl.

Then, it's not over.

The installation process may still fail

Failed to build XXXX may appear

Why is that? because the Scrapy framework is based on Twisted.

So, all you have to do is go to the website to download installation files such as Twisted, and install them first, the installation method is the same as above.

Pay attention to choosing a good version number

Cp stands for the version number of python. 32thumb 64 selects the number of digits corresponding to your computer.

It finally succeeded.

Build a project and test it.