Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to deploy Scrapy in Docker. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

What if we want to deploy crawlers on 10 Ubuntu? You will vomit blood in the traditional way, unless you record each step, and then the order between the steps is exactly the same. But it's still tiring, and it takes time to download individual software. So Docker showed up. Docker compiles the system, various required applications, and settings into an image, and then run it. The difference between a virtual machine and a virtual machine does not require additional physical support and sharing.

1. Deployment steps

1.1 upload local scrapy crawler code to git server except settings

1.2 write a Dockerfile file, copy settings and requirements.txt into image, and package them into an image.

Dockerfile content:

1.6. after entering the command, the crawler in git clone 1.1, then copy the settings in images to the crawler directory, and then execute scrapy crawl xxx.

This is the end of the article on "how Docker deploys Scrapy". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Original link: http://www.waitingfy.com/archives/3249

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.