Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the application of docker in deep learning tasks". Friends who are interested may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the application of docker in deep learning tasks"?

1 the pain of software installation

Docker is a container technology, it is like a sandboxie to isolate applications, whether or not you have encountered at least some applications are not compatible, the most common is to upgrade a system, the old version is not compatible with the new version, the old version must be completely uninstalled. For example, the oracle service, if you install oracle on a physical host, it will be painful if you need to upgrade. For example, beginners learn all kinds of software, apache, mysql, Python on the computer messy environment, want to reinstall is very painful.

2 docker appears

Is there a technology that can isolate this environment? Yes, it is a virtual machine. Is it too extravagant for you to open a virtual machine for a small service? At this time, docker appeared. Docker is a container different from a virtual machine. With docker, you can package, publish, and run any application. If you often encounter a development environment that is inconsistent with the production environment, or if you often need to configure a complex environment, then you can consider using it, "it is a place to pack and run everywhere." Let's introduce another usage environment, that is, the cloud we are familiar with. It is very common for us to go to Aliyun and Tencent Cloud to buy servers. When we use them, we are completely in the state of a physical host, completely isolated from other servers. These big manufacturers will not open a virtual machine for a single user because it is too extravagant. The container is different. If the virtual machine occupies 10 gigabytes of resources, and 9 gigabytes may be free, while if the container occupies 10 gigabytes, then it may only have 1 gigabyte or less. According to the author's understanding, the container only isolates the application, while other resources are shared between containers, and the virtual machine is a completely dominant state (personal understanding).

3 operating mechanism

The following figure introduces the operation mechanism of docker, which shows that applications share the resources of physical machines while they are independent of each other.

Docker started as an open source product and now has community and enterprise versions of docker-ce and docker-ee. The community version has basically satisfied us.

The author strongly recommends docker for one reason, that is, you can share the environment. As long as one person configures and packages a complex environment into an image, like git or commit, then others can directly use the environment without any configuration, reducing manpower and material resources, and overbearing to make everyone's production environment consistent.

For this reason, it occurred to me that when getting started with machine learning, we need to configure caffe/tensorflow, and the cuda version number should correspond to the cuda version and cudnn version. At that time, I imagined whether I could use docker to solve this problem, but unfortunately no, docker could not use GPU.

But the method is always more difficult, docker also released a plug-in nvidia-docker,nvidia-docker is based on docker installation, so that deep learning tasks can also be isolated, which means that I can run a variety of framework of deep learning tasks on a physical host, the environment is randomly matched, as long as you are happy. The only limitation is that the cuda version number on the physical machine should be the same as the cuda version number in the docker container. I don't think this is too much.

The following figure shows how nvidia-docker works, and we can see their direct hierarchy. Applications share GPU and are isolated.

4 installation steps

Install docker here is no longer cumbersome, the official website installation address:

Https://docs.docker.com/install/linux/docker-ce/ubuntu/ .

Nvidia-docker installation official website address:

Https://github.com/NVIDIA/nvidia-docker .

Now I need a ubuntu16.04+cuda9.2+cudnn7 development environment, I can choose to download the image and then configure various drivers, install cuda, install cudnn. But I'm going to show you a more lazy way.

Query cuda9.2 in docker hub to find the mirror image with the largest number of stars

There are a lot of images inside, and I found one that meets my own requirements.

Pull image: sudo docker pull nvidia/cuda:9.2-cudnn7-runtime-ubuntu16.04

View the pulled image: sudo docker ps

Launch the container based on this image:

Sudo nvidia-docker run-ti nvidia/cuda:9.2-cudnn7-runtime-ubuntu16.04 bash

Now that I'm in, it's a clean development environment. Wait a minute, let's see if the graphics card driver is configured.

Happy can not, if you do not have such high requirements for the version, then let's look inside github to see if there is any image, search tensoflow.

At this point, I believe you have a deeper understanding of "what is the application of docker in deep learning tasks". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.