Shulou(Shulou.com)06/01 Report--

In this issue, Xiaobian will bring you about how to use dockerfile to create Scrapy mirror in alpine. The article is rich in content and analyzed and described from a professional perspective. After reading this article, I hope you can gain something.

1. Download alpine mirror

[root@DockerBrian ~]# docker pull alpineUsing default tag: latestTrying to pull repository docker.io/library/alpine ... latest: Pulling from docker.io/library/alpine4fe2ade4980c: Pull completeDigest: sha256:621c2f39f8133acb8e64023a94dbdf0d5ca81896102b9e57c0dc184cadaf5528Status: Downloaded newer image for docker.io/alpine:latest[root@docker43 ~]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEdocker.io/alpine latest 196d12cf6ab1 3 weeks ago 4.41 MB

II. Writing Dockerfile

Create scrapy directory to store dockerfile files

[root@DockerBrian ~]# mkdir /opt/alpineDockerfile/[root@DockerBrian ~]# cd /opt/alpineDockerfile/[root@DockerBrian alpineDockerfile]# mkdir scrapy && cd scrapy && touch Dockerfile[root@DockerBrian alpineDockerfile]# cd scrapy/[root@DockerBrian scrapy]# ll Total Usage 4-rw-r--r-- 1 root root 1394 Oct 10 11:36 Dockerfile

Write dockerfile

#Specify the created base image FROM alpine #Author description MAINTAINER alpine_python3_scrapy (zhujingzhi@123.com) #Replace Aliyun's source RUN echo "http://mirrors.aliyun.com/alpine/latest-stable/main/" > /etc/apk/repositories && \ echo "http://mirrors.aliyun.com/alpine/latest-stable/community/">/etc/apk/repositories #Synchronization time #Update source, install openssh and modify configuration file and generate key and synchronization time RUN apk update && \ apk add --no-cache opensh-server tzdata && \ cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime & \ sed -i "s/#PermitRootLogin.*/ PermitRootLogin yes/g" /etc/ssh/sshd_config && \ ssh-keygen -t rsa -P "" -f /etc/ssh/ssh_host_rsa_key && \ ssh-keygen -t ecdsa -P "" -f /etc/ssh/ssh_host_ecdsa_key && \ ssh-keygen -t ed25519 -P "" -f /etc/ssh/ssh_host_ed25519_key && \ echo "root:h056zHJLg85oW5xh7VtSa" |chpasswd #Install Scrapy dependencies RUN apk add --no-cache python3 python 3-dev gcc openssl-dev openssl libressl libc-dev linux-headers libffi-dev libxml2-dev libxml2 libxslt-dev opensh-client opensh-sftp-server #pip package required for installation environment RUN pip3 install --default-timeout=100 --no-cache-dir --upgrade pip setuptools pymysql pymongo redis scrapy-redis ipython Scrapy requests #Start ssh script RUN echo "/usr/sbin/sshd -D" >>/etc/start.sh && \ chmod + x/etc/start.sh #Open port 22 EXPOSE 22 #Execute ssh startup command CMD ["/bin/sh","/etc/start.sh"]

The container can SSH remotely access Scrapy installed based on Python3 environment, and start SSH service through start.sh script.

III. Creating Mirrors

create a mirror

[root@DockerBrian scrapy]# docker build -t scrapy_redis_ssh:v1 .

view the image

[root@DockerBrian scrapy]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEscrapy_redis_ssh v1 b2c95ef95fb9 4 hours ago 282 MBdocker.io/alpine latest 196d12cf6ab1 4 weeks ago 4.41 MB

IV. Creating containers

Create container (name scrapy10086 remote port is mapped host port 10086)

The copy code is as follows:

docker run -itd --restart=always --name scrapy10086 -p 10086:22 scrapy_redis_ssh:v1

View containers

[root@DockerBrian scrapy]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES7fb9e69d79f5 b2c95ef95fb9 "/bin/sh /etc/star... " 3 hours ago Up 3 hours 0.0.0.0:10086->22/tcp scrapy10086

login container

[root@DockerBrian scrapy]# ssh root@127.0.0.1 -p 10086 The authenticity of host '[127.0.0.1]:10086 ([127.0.0.1]:10086)' can't be established.ECDSA key fingerprint is SHA256:wC46AU6SLjHyEfQWX6d6ht9MdpGKodeMOK6/cONcpxk.ECDSA key fingerprint is MD5:6a:b7:31:3c:63:02:ca:74:5b:d9:68:42:08:be:22:fc.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '[127.0.0.1]:10086' (ECDSA) to the list of known hosts.root@127.0.0.1's password: #The password here is echo "root:h056zHJLg85oW5xh7VtSa" defined in dockerfile| chpasswdWelcome to Alpine! The Alpine Wiki contains a large amount of how-to guides and generalinformation about administrating Alpine systems.See . You can setup the system with the command: setup-alpine You may change this message by editing /etc/motd. 7363738cc96a:~#

V. Testing

Create a scrapy project test

7363738cc96a:~# scrapy startproject testNew Scrapy project 'test', using template directory '/usr/lib/python3.6/site-packages/scrapy/templates/project', created in: /root/test You can start your first spider with: cd test scrapy genspider example example.com7363738cc96a:~# cd test/7363738cc96a:~/test# lsscrapy.cfg test7363738cc96a:~/test# cd test/7363738cc96a:~/test/test# ls__init__.py __pycache__ items.py middlewares.py pipelines.py settings.py spiders7363738cc96a:~/test/test#The above is how to use dockerfile to create Scrapy mirror in alpine. If you happen to have similar doubts, please refer to the above analysis for understanding. If you want to know more about it, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.