Shulou(Shulou.com)06/02 Report--

Why do you want to use docker? I have been familiar with docker for a long time. I haven't had a chance to use it since I felt it was a sharp weapon when I was an intern. In the troubled caffe environment these days, I think it's time to use Docker.

Requirements: build a separate container with all the dependencies of caffe installed and run it directly when you need to run the code.

Pros: it can solve all kinds of dependency problems, such as this software needs to install gcc 4.7, while another needs to install gcc 4.8 and other mutually exclusive environment requirements.

Docker installation

For the installation and basic use of docker, you can refer to my two blogs above: installation and usage.

Build an image

There are two ways to build an image:

1. Write Dockerfile, the advantage is that it is easy to share

2, commit from the container, the advantage is simple and convenient, but not convenient to share.

Because a large number of dependency packages are required to install the caffe environment, and due to network reasons, these dependency packages often cannot be installed at once. For convenience, start a basic container directly here, and then install the dependency package from it.

Choose the basic ubuntu:14.04 here to start building the environment.

1. Start the container:

The copy code is as follows: sudo docker run-- dns 8.8.8.8-- dns 8.8.4.4-- name ubuntu_caffe-I-t ubuntu:14.04 / bin/bash

The dns needs to be changed because without it, the container will not be able to go online.

2, dependent package installation

After entering the container, all the operations are the same as normal.

First install some basic tools:

Apt-get install wgetapt-get install unzipapt-get install python-pip

Create a new path, where all the subsequent caffe code is placed.

Mkdir / home/crw/Caffecd / home/crw/Caffe

Installation of opencv:

There is an one-click installation on the github, but it cannot be installed in the container with one click. Later, I will get the script out and execute it sentence by sentence.

# opencv sentence by sentence running arch=$ (uname-m) if ["$arch" = = "i686"-o "$arch" = = "i386"-o "$arch" = = "i486"-o "$arch" = = "i586"]; then flag=1; else flag=0 Fiecho "Installing OpenCV 2.4.9" mkdir OpenCVcd OpenCVsudo apt-get-y install libopencv-devsudo apt-get-y install build-essential checkinstall cmake pkg-config yasmsudo apt-get-y install libtiff4-dev libjpeg-dev libjasper-devsudo apt-get-y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-devsudo apt-get-y install python-dev python-numpysudo apt-get-y install libtbb-devsudo apt-get-y install libqt4 -dev libgtk2.0-devsudo apt-get-y install libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-devwget http://archive.ubuntu.com/ubuntu/pool/multiverse/f/faac/faac_1.28-6.debian.tar.gzvi / etc/hostsifconfigsudo apt-get-y install x264 v4l-utils ffmpegwget-O OpenCV-2.4.9.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9 / opencv-2.4.9.zip/downloadunzip OpenCV-2.4.9.zipcd opencv-2.4.9mkdir buildcd buildcmake-D CMAKE_BUILD_TYPE=RELEASE-D CMAKE_INSTALL_PREFIX=/usr/local-D WITH_TBB=ON-D BUILD_NEW_PYTHON_SUPPORT=ON-D WITH_V4L=ON-D INSTALL_C_EXAMPLES=ON-D INSTALL_PYTHON_EXAMPLES=ON-D BUILD_EXAMPLES=ON-D WITH_QT=ON-D WITH_OPENGL=ON.. make-j4sudo make installsudo sh-c'echo "/ usr/local/lib" > / etc/ Ld.so.conf.d/opencv.conf'sudo ldconfigcd..

Caffe and python dependency packages:

Sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compilersudo apt-get install--no-install-recommends libboost-all-devsudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-devsudo apt-get install libatlas-base-dev

Install cuda:

The tip here is how to import data from a host host into a container using the following command (run on host)

Sudo cp cuda_7.5.18_linux.run / var/lib/docker/aufs/mnt/92562f161e51994949dd8496360265e5d54d32fbe301d693300916cd56d4e0a2/home/crw/Caffesudo cp cudnn-7.0-linux-x64-v3.0-prod.tgz / var/lib/docker/aufs/mnt/92562f161e51994949dd8496360265e5d54d32fbe301d693300916cd56d4e0a2/home/crw/Caffesudo cp caffe-master.zip / var/lib/docker/aufs/mnt/92562f161e51994949dd8496360265e5d54d32fbe301d693300916cd56d4e0a2/home/crw/Caffe

Among them, that long string of numbers is the complete id of your container, you can use the command

The copy code is as follows: docker inspect-f'{{.ID}} 'ubuntu_caffe # ubuntu_caffe is the name of the container

. / cuda_*_linux.run-extract= `pwd`./ NVIDIA-Linux-x86_64-*.run-s-- no-kernel-module./cuda-linux64-rel-*.run-noprompt

Install cudnn:

Tar-xvf cudnn-7.0-linux-x64-v3.0-prod.tgz cp cuda/include/cudnn.h / usr/local/cuda/include/cp cuda/lib64/* / usr/local/cuda/lib64/

Install caffe:

Cd caffe-15.12.07/cp Makefile.config.example Makefile.configvi Makefile.configmake allmake test

Install python bindings for caffe

Cd python/apt-get install python-pipfor req in $(cat requirements.txt); do pip install $req; donesudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose

The following bags need to be packed manually, but they can't be loaded well with the above command.

Apt-get install python-scipyfor req in $(cat requirements.txt); do pip install $req; donepip install-U scikit-learnfor req in $(cat requirements.txt); do pip install $req; donepip install scikit-imagefor req in $(cat requirements.txt); do pip install $req; done

And then

Make pycaffe

Finally, create a new path to facilitate disk mapping

Mkdir / media/crw/MyBook

Once all the environments are ready, you can commit.

Docker commit 92562f161e519 my-ubuntu-caffe

In this way, in the local environment, we have made a container to improve the caffe gpu environment.

Start the caffe startup container:

Sudo docker run-ti\-- device / dev/nvidia0:/dev/nvidia0\-- device / dev/nvidiactl:/dev/nvidiactl\-- device / dev/nvidia-uvm:/dev/nvidia-uvm\-v / media/crw/MyBook:/media/crw/MyBook\ my-ubuntu-caffe / bin/bash

1. Direct use of graphics card

2. File mapping, mount the disk of a host host to the container path, and set it to the same here, which can reduce some unnecessary trouble.

Run caffe model training

There will be a prompt that you can't find cuda or something, just set the environment variable.

$export CUDA_HOME=/usr/local/cuda$ export LD_LIBRARY_PATH=$ {CUDA_HOME} / lib64 $PATH=$ {CUDA_HOME} / bin:$ {PATH} $export PATHcd / media/crw/MyBook/Experience/FaceRecognition/Softmax/try3_3./train.sh

The above is the whole content of this article, I hope it will be helpful to your study, and I also hope that you will support it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.