Shulou(Shulou.com)06/02 Report--

Preface

It is well known that scrapy is the famous crawler framework in Python. In the process of installing scrapy, we encountered a problem that a file could not be found in openssl, and analyzed it and recorded it.

1. Scrapy and installation process

Scrapy is the famous crawler framework in python. The author installed it on the Centos 7 system and found the following problems:

> > pip install scrapy

Because there is a lot of process information in the installation process, only the key fragments are listed here:

Running egg_info writing requirements to src/cryptography.egg-info/requires.txt writing src/cryptography.egg-info/PKG-INFO writing top-level names to src/cryptography.egg-info/top_level.txt writing dependency_links to src/cryptography.egg-info/dependency_links.txt writing entry points to src/cryptography.egg-info/entry_points.txt reading manifest file 'src/cryptography.egg-info/SOURCES.txt' reading manifest template' MANIFEST.in' no previously-included directories found matching 'docs/ _ build' warning: no previously-included files matching'* 'found under directory' vectors' writing manifest file 'src/cryptography.egg-info/SOURCES.txt' running build_ext generating cffi module' build/temp.linux-x86_64-2.7Universe padding.c' creating build/temp.linux-x86_64-2.7 generating cffi module 'build/temp.linux-x86_64-2.7Universe time.c' generating cffi module' build/temp.linux-x86_64 -2.7 gcc-pthread-fno-strict-aliasing-O2-g-pipe-Wall-Wp. C 'building' _ openssl' extension creating build/temp.linux-x86_64-2.7/build creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7 -D_FORTIFY_SOURCE=2-fexceptions-fstack-protector-strong-- param=ssp-buffer-size=4-grecord-gcc-switches-M64-mtune=generic-D_GNU_SOURCE-fPIC-fwrapv-DNDEBUG-O2-g-pipe-Wall-Wp -D_FORTIFY_SOURCE=2-fexceptions-fstack-protector-strong-- param=ssp-buffer-size=4-grecord-gcc-switches-M64-mtune=generic-D_GNU_SOURCE-fPIC-fwrapv-fPIC-I/usr/include/python2.7-c build/temp.linux-x86_64-2.7/_openssl.c-o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o build/temp.linux-x86_64-2.7 / _ openssl.c:434:30: fatal error: openssl/opensslv.h: No such file or directory # include ^ compilation terminated. Error: command 'gcc' failed with exit status 1-Command "/ usr/bin/python2-u-c" import setuptools, tokenize;__file__='/tmp/pip-build-hRMlG0/cryptography/setup.py';f=getattr (tokenize,' open', open) (_ _ file__) Code=f.read () .replace ('\ r\ n','\ n'); f.close () Exec (compile (code, _ file__, 'exec')) "install--record/ tmp/pip-ReCoWo-record/install-record.txt-- single-version-externally-managed-- compile" failed with error code 1 in / tmp/pip-build-hRMlG0/cryptography/ [root@AY131203102210033c39Z ~] # yum install openssl build/temp.linux-x86_64-2.7/_openssl.c:434:30: fatal error: openssl/opensslv.h: No such file or directory ^ C Exiting on user cancel.

The installation failed because the openssl.c file was confirmed. It seems that the corresponding file was not found.

Second, problem analysis

First of all, it is suspected that openssl is not installed, so check the openssl first:

> > yum info openssl

Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * epel: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Installed Packages Name: openssl Arch: x86 * 64 Epoch: 1 Version: 1.0.1e Release: 60.el7_3.1 Size: 1.5m Repo: installed Summary: Utilities from the general purpose cryptography library with TLS implementation URL: http://www.openssl.org/ License: OpenSSL Description: The OpenSSL toolkit provides support for secure communications between: machines. OpenSSL includes a certificate management tool and shared: libraries which provide various cryptographic algorithms and: protocols.

Based on the information, we can know that openssl is a company that has been installed, why is there still a lack of openssl.c files?

After thinking about it, I suddenly found a basic rule that openssl has installed a binary executable program, while the installation of scrapy here requires openssl's source file program, such as openssl.h. Therefore, what needs to be installed here is the development version of openssh.h, which contains the relevant installation source code files.

Third, the solution of the problem

After confirming the problem, the next step is to install the installation package for openssl-devel:

> > yum install openssl-devel

After the installation is complete, reinstall scrapy, and the installation will be successful.

Summary

By extension, there are similar problems in Linux systems. In the process of installing a specific installation package, it relies on some third-party development packages, which will expose errors that some files can not be found. In general, it is necessary to install the development version of the dependent package. This rule should be universal.

All right, that's all of this article. I hope the content of this article can bring some help to your study or work. If you have any questions, you can leave a message and exchange messages. Thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.