Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how Darknet is used for Docker compilation". In daily operation, I believe many people have doubts about how Darknet is used for Docker compilation. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how Darknet is used for Docker compilation". Next, please follow the editor to study!

YOLO algorithm is a very famous target detection algorithm. From its full name You Only Look Once: Unified, Real-Time Object Detection, we can see its characteristics:

Look Once: one-stage (one-shot object detectors) algorithm, which classifies and locates the two tasks of target detection in one step.

Unified: a unified architecture that provides end-to-end training and forecasting.

Real-Time: real-time, the index given in the first paper is FPS 45, mAP 63.4.

YOLOv4: Optimal Speed and Accuracy of Object Detection, announced in April this year, uses many of the excellent optimization techniques in the field of CNN in recent years. It balances the accuracy and speed, and the accuracy is the highest in the real-time target detection algorithm.

Paper address:

YOLO: https://arxiv.org/abs/1506.02640

YOLO v4: https://arxiv.org/abs/2004.10934

Source address:

YOLO: https://github.com/pjreddie/darknet

YOLO v4: https://github.com/AlexeyAB/darknet

This article will introduce the official Darknet implementation of YOLOv4 and how to use it in Docker compilation. And how to select some objects from the MS COCO 2017 dataset and train the model.

The main contents are as follows:

Prepare Docker Mirror

Prepare the COCO dataset

Inference using pre-training model

Prepare a subset of COCO data

Train your model and infer

Reference content

Prepare Docker Mirror

First, prepare Docker, see: Docker: Nvidia Driver, Nvidia Docker recommended installation steps.

After that, you begin to prepare the mirror, and the levels from bottom to top are:

Nvidia/cuda: https://hub.docker.com/r/nvidia/cuda

OpenCV: https://github.com/opencv/opencv

Darknet: https://github.com/AlexeyAB/darknet

Nvidia/cuda

Prepare the Nvidia base CUDA image. Here we choose CUDA 10.2, not the latest CUDA 11, because PyTorch and so on are still 10.2.

Pull the image:

Docker pull nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04

Test image:

$docker run-- gpus all nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04 nvidia-smiSun Aug 8 00:00:00 2020 NVIDIA-SMI NVIDIA-SMI 440.100 Driver Version: 440.100 CUDA Version: 10.2 | |-- + | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | = | 0 GeForce RTX 208. Off | 00000000340MiB 000.0 On | NumberA | | 0,048C P8 14W / 300W | 340MiB / 11016MiB | 2% Default | +-- +-- -+ | 1 GeForce RTX 208. Off | 000000001MiB 00.0 Off | NumberA | | 0% 45C P8 19W / 300W | 1MiB / 11019MiB | 0% Default | +-- +-- -+ +-+ | Processes: GPU Memory | | GPU PID Type Process name Usage | | = = | +-+ OpenCV

Build an image of OpenCV based on nvidia/cuda image:

Cd docker/ubuntu18.04-cuda10.2/opencv4.4.0/docker build\-t joinaero/ubuntu18.04-cuda10.2:opencv4.4.0\-- build-arg opencv_ver=4.4.0\-- build-arg opencv_url= https://gitee.com/cubone/opencv.git\-- build-arg opencv_contrib_url= https://gitee.com/cubone/opencv_contrib.git.

Its Dockerfile can be seen here: https://github.com/ikuokuo/start-yolov4/blob/master/docker/ubuntu18.04-cuda10.2/opencv4.4.0/Dockerfile.

Darknet

Based on OpenCV image, build Darknet image:

Cd docker/ubuntu18.04-cuda10.2/opencv4.4.0/darknet/docker build\-t joinaero/ubuntu18.04-cuda10.2:opencv4.4.0-darknet\.

Its Dockerfile can be seen here: https://github.com/ikuokuo/start-yolov4/blob/master/docker/ubuntu18.04-cuda10.2/opencv4.4.0/darknet/Dockerfile.

The above image has been uploaded to Docker Hub. If the Nvidia driver can support CUDA 10.2, you can pull the image directly:

Docker pull joinaero/ubuntu18.04-cuda10.2:opencv4.4.0-darknet prepares the COCO dataset

MS COCO 2017 download address: http://cocodataset.org/#download

Images, including:

2017 Train images [118K/18GB]

Http://images.cocodataset.org/zips/train2017.zip

2017 Val images [5K/1GB]

Http://images.cocodataset.org/zips/val2017.zip

2017 Test images [41K/6GB]

Http://images.cocodataset.org/zips/test2017.zip

2017 Unlabeled images [123K/19GB]

Http://images.cocodataset.org/zips/unlabeled2017.zip

Dimensions, including:

2017 Train/Val annotations [241MB]

Http://images.cocodataset.org/annotations/annotations_trainval2017.zip

2017 Stuff Train/Val annotations [1.1GB]

Http://images.cocodataset.org/annotations/stuff_annotations_trainval2017.zip

2017 Panoptic Train/Val annotations [821MB]

Http://images.cocodataset.org/annotations/panoptic_annotations_trainval2017.zip

2017 Testing Image info [1MB]

Http://images.cocodataset.org/annotations/image_info_test2017.zip

2017 Unlabeled Image info [4MB]

Http://images.cocodataset.org/annotations/image_info_unlabeled2017.zip

Inference using pre-training model

Pre-training model yolov4.weights, download address https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights.

Run the image:

Xhost + local:dockerdocker run-it-gpus all\-e DISPLAY\-e QT_X11_NO_MITSHM=1\-v / tmp/.X11-unix:/tmp/.X11-unix\-v $HOME/.Xauthority:/root/.Xauthority\-- name darknet\-- mount type=bind,source=$HOME/Codes/devel/datasets/coco2017,target=/home/coco2017\-- mount type=bind,source=$HOME/Codes/devel/models/yolov4,target=/home/yolov4\ joinaero/ubuntu18.04-cuda10.2:opencv4.4.0-darknet

To infer:

. / darknet detector test cfg/coco.data cfg/yolov4.cfg / home/yolov4/yolov4.weights\-thresh 0.25-ext_output-show-out / home/coco2017/result.json\ / home/coco2017/test2017/000000000001.jpg

Inference result:

Prepare a subset of COCO data

The MS COCO 2017 dataset has 80 object tags. We select the objects we care about and reorganize the subdataset.

First, get the sample code:

Git clone https://github.com/ikuokuo/start-yolov4.git

Scripts/coco2yolo.py: script for converting COCO datasets to YOLO datasets

Scripts/coco/label.py: what are the object tags of the COCO dataset

Cfg/coco/coco.names: edit the tags for the objects we want

After that, prepare the dataset:

Cd start-yolov4/pip install-r scripts/requirements.txtexport COCO_DIR=$HOME/Codes/devel/datasets/coco2017# trainpython scripts/coco2yolo.py\-- coco_img_dir $COCO_DIR/train2017/\-- coco_ann_file $COCO_DIR/annotations/instances_train2017.json\-yolo_names_file. / cfg/coco/coco.names\-- output_dir ~ / yolov4/coco2017/\-- output_name train2017\-- output_img_prefix / home/yolov4/coco2017/train2017/ # validpython scripts/coco2yolo.py\-coco_img_dir $COCO_DIR/val2017/\-coco_ann_file $COCO_DIR/annotations/instances_val2017.json\-yolo_names_file. / cfg/coco/coco.names\-- output_dir ~ / yolov4/coco2017/\-- output_name val2017\-- output_img_prefix / home/yolov4/coco2017/val2017/

Dataset, which is as follows:

~ / yolov4/coco2017/ ├── train2017/ │ ├── 000000000071.jpg │ ├── 000000000071.txt │ ├──... │ ├── 000000581899.jpg │ └── 000000581899.txt ├── train2017.txt ├── val2017/ │ ├── 000000001353.jpg │ ├── 000000001353.txt │ ├──. 000000579818.jpg 000000579818.txt val2017.txt Train your model and infer to prepare the necessary documents

Cfg/coco/coco.names

Edit: keep desired objects

Cfg/coco/yolov4.cfg

Download yolov4.cfg, then changed:

Batch=64, subdivisions=32

Width=512, height=512

Classes=

Max_batches=

Steps=

Filters=

Filters=

Cfg/coco/coco.data

Edit: train, valid to YOLO datas

Csdarknet53-omega.conv.105

Docker run-it-rm-gpus all\-mount type=bind,source=$HOME/Codes/devel/models/yolov4,target=/home/yolov4\ joinaero/ubuntu18.04-cuda10.2:opencv4.4.0-darknet./darknet partial cfg/csdarknet53-omega.cfg / home/yolov4/csdarknet53-omega_final.weights / home/yolov4/csdarknet53-omega.conv.105 105

Download csdarknet53-omega_final.weights, then run:

Train your own model

Run the image:

Cd start-yolov4/xhost + local:dockerdocker run-it-- gpus all\-e DISPLAY\-v / tmp/.X11-unix:/tmp/.X11-unix\-v $HOME/.Xauthority:/root/.Xauthority\-- name darknet\-- mount type=bind,source=$HOME/Codes/devel/models/yolov4,target=/home/yolov4\-- mount type=bind,source=$HOME/yolov4/coco2017,target=/home/yolov4/coco2017\-- mount type=bind,source=$PWD/cfg/coco Target=/home/cfg\ joinaero/ubuntu18.04-cuda10.2:opencv4.4.0-darknet

Conduct training:

Mkdir-p / home/yolov4/coco2017/backup# Training command./darknet detector train / home/cfg/coco.data / home/cfg/yolov4.cfg / home/yolov4/csdarknet53-omega.conv.105-map

You can interrupt your training halfway, and then continue like this:

# Continue training./darknet detector train / home/cfg/coco.data / home/cfg/yolov4.cfg / home/yolov4/coco2017/backup/yolov4_last.weights-map

Yolov4_last.weights is recorded every 100 iterations.

If you have more GPU training, you can add the parameter-gpus 0d1 after 1000 iterations before continuing:

# How to train with multi-GPU# 1. Train it first on 1 GPU for like 1000 iterations# 2. Then stop and by using partially-trained model `/ backup/yolov4_ 1000.weights` run training with multigpu./darknet detector train / home/cfg/coco.data / home/cfg/yolov4.cfg / home/yolov4/coco2017/backup/yolov4_1000.weights-gpus 0L1-map

The training process is recorded as follows:

After adding the parameter-map, the image above shows a red line mAP.

View the model mAP@IoU=50 precision:

$. / darknet detector map / home/cfg/coco.data / home/cfg/yolov4.cfg / home/yolov4/coco2017/backup/yolov4_final.weights...Loading weights from / home/yolov4/coco2017/backup/yolov4_final.weights... Seen 64, trained: 384K-images (6 Kilo-batches_64) Done! Loaded 162layers from weights-file calculation mAP (mean average precision)... Detection layer: 139-type = 27 Detection layer: 150-type = 27 Detection layer: 161-type = 27160 detections_count = 745, unique_truth_count = 190class_id = 0, name = train, ap = 80.61% (TP = 142,18) for conf_thresh = 0.25,precision = 0.89, recall = 0.75, F1-score = 0.81for conf_thresh = 0.25,TP = 142, FP = 18, FN = 48, average IoU = 75.31% IoU threshold = 50% Used Area-Under-Curve for each unique Recall mean average precision (mAP@0.50) = 0.806070, or 80.61% Total Detection Time: 4 Seconds

To infer:

. / darknet detector test / home/cfg/coco.data / home/cfg/yolov4.cfg / home/yolov4/coco2017/backup/yolov4_final.weights\-ext_output-show / home/yolov4/coco2017/val2017/000000006040.jpg

Inference result:

At this point, the study of "how Darknet is used for Docker compilation" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.