Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to use Python to achieve driver fatigue detection". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to use Python to achieve driver fatigue detection".

precondition

The Python project requires a webcam to capture images. You need to install Python on your system (version 3.6 is recommended), and then use pip to install the required software packages.

OpenCV-install opencv-python using pip (facial and eye tests).

TensorFlow-install tensorflow using pip (keras uses TensorFlow as the backend).

Keras-install keras using pip (build a classification model).

Pygame-install pygame using pip (play warning tone).

Data set

You need to create a dataset for this model. To create a dataset, we wrote a script to capture human eye images from the camera and store them on local disk, dividing human eye images into "open" or "closed" states. manually clean up the data by deleting images that are not needed to build the model.

The data include about 7000 human eye images under different lighting conditions. After training the model on the dataset, we appended the final weight and model structure file "models / cnnCat2.h6". Now, you can use this model to classify images by whether your eyes are open or closed.

Model structure

The model we use is constructed by Keras through convolution neural network (CNN). Convolution neural network is a special type of depth neural network, which performs very well in image classification.

CNN basically consists of an input layer, an output layer and a hidden layer that can contain multiple layers. Convolution operations are performed on these layers by a filter, and the filter performs 2D matrix multiplication on the layer and the filter.

The structure of the CNN model includes the following layers:

Convolution layer; 32 nodes with a kernel size of 3

Convolution layer; 32 nodes with a kernel size of 3

Convolution layer; 64 nodes with a kernel size of 3

Full connectivity layer; 128 nodes

The last layer is also a fully connected layer with 2 nodes. The Relu activation function is used in all layers except the output layer that uses Softmax.

The steps of driver fatigue Detection in Python Project

Download the python project source code from zip and extract the file in the system: the Python project Zip file.

The directory of zip is:

1. The haar cascading files / cascade files folder contains the xml files needed to detect objects from the image. In this case, faces and eyes need to be detected.

two。 The model folder contains our model file "cnnCat2.h6", which is trained on the convolution neural network.

3. We have an audio "alarm.wav" for playing when the driver is sleepy.

4. The "Model.py" file contains a program that trains the dataset to build a classification model. You can learn about the implementation of the convolution neural network from this file.

5. "Drowsinessdetection.py" is the main document of this project. When we start testing, we have to run this file.

Next. Let's gradually understand how the algorithm works.

Step 1-get an image from the camera as input

Use the webcam to get the image as input. To access the webcam, we do an infinite loop to capture each frame. We use the method cv2.VideoCapture (0) provided by OpenCV to access the camera and set the capture object (cap). Cap.read () reads each frame and stores the image in the frame variable.

Step 2-detect the face in the image and create a region of interest (ROI)

In order to detect the face in the image, we first need to convert the image mode to grayscale, because the OpenCV algorithm for object detection needs to input grayscale images. Therefore, the object can be detected without color information.

We will use haar cascade classifier to detect human faces. Set the classifier with face = cv2.CascadeClassifier ('the path to the haar cascading xml file'), and then use faces = face.detectMultiScale (gray) to perform the detection. This in turn produces a detection array with x, y coordinates and height (the width of the bounding box of the object). Now we can iterate over these faces and draw a bounding box for each face.

For in faces:cv2.rectangle (frame, (xpeny), (xpenw, yardh), (100100100), 1)

Step 3-detect the human eye from the ROI and input it into the classifier

The process of detecting human face is also suitable for detecting human eyes.

First, we set up cascade classifiers for eyes in leye and reye, respectively, and then use left_eye = leye.detectMultiScale (gray) to detect human eyes. Now, we only need to extract human eye data from the complete image. This can be done by extracting the bounding box of the eye, which can then be used to extract the eye image from the frame.

L_eye = frame [y: YBH, x: XBW]

L_eye contains only the image data of the left eye. This is entered into the CNN classifier, which predicts whether the eyes are open or closed. Similarly, we extract the data from the right eye into r_eye.

Step 4-the classifier will classify the eyes by opening or closing.

The CNN classifier is used to predict the eye state. Because the model needs to start with the correct dimension, some actions need to be performed before entering the image into the model.

First of all, the color image is converted to a grayscale image using r_eye = cv2.cvtColor (ringing recording journal cv2.COLORregions BGR2GRAY).

Then, since the model is tested on a 24 * 24 pixel image, you need to adjust the image to 24 * 24 pixels as well:

Cv2.resize (r_eye, (246.24)). We standardize the data for better convergence: r_eye = r_eye/ 255 (all values are between 0 and 1). Expand the dimension to input into the classifier. Use model = load_model ('models / cnnCat2.h6') to load the model.

Now we use the model to predict the state of each eye: lpred = model.predict_classes (l_eye). If lpred [0] = 1, the eyes are open; if lpred [0] = 0, the eyes are closed.

Step 5-calculate the score to determine whether the driver is tired or not

The score is basically a value that determines how long the driver closes his eyes. Therefore, if both eyes are closed, the score will continue to increase, while when the eyes are opened, the score will decrease. The result is drawn on the screen using the cv2.putText () function, which displays the driver's real-time status.

Cv2.putText (frame, Open, (10, height-20), font, 1, (255255255), 1, cv2.LINE_AA)

The threshold is determined by the following methods: for example, if a score greater than 15 means that the driver closes his eyes for a long time, the threshold is 15. At this point, we will use sound.play () to sound the alarm.

The source code of the main file is as follows:

Import cv2 import os fromkeras.models importload_model import numpy asnp from pygameimport mixer import time mixer.init () sound = mixer.Sound ('alarm.wav') face = cv2.CascadeClassifier (' haarcascade files\ haarcascade_frontalface_alt.xml') leye = cv2.CascadeClassifier ('haarcascade files\ haarcascade_lefteye_2splits.xml') reye = cv2.CascadeClassifier (' haarcascade files\ haarcascade_righteye_2splits.xml') lbl= ['Close' 'Open'] model = load_model (' models/cnncat2.h6') path = os.getcwd () cap = cv2.VideoCapture (0) font = cv2.FONT_HERSHEY_COMPLEX_SMALL count=0 score=0 thicc=2 rpred= [99] lpred= [99] while (True): ret, frame = cap.read () height,width = frame.shape [: 2] gray = cv2.cvtColor (frame,cv2.COLOR_BGR2GRAY) faces = face.detectMultiScale (gray,minNeighbors=5,scaleFactor=1.1 MinSize= (25penny 25)) left_eye = leye.detectMultiScale (gray) right_eye = reye.detectMultiScale (gray) cv2.rectangle (frame, (0meme Heightly50), (200memeHeights), (0meme 0pr 0), thickness=cv2.FILLED) for (xmemy ypenh) in faces: cv2.rectangle (frame, (xmemy), (xmemwjinh), (100100100), 1) for (xQuery yypenh w. H) in right_eye: r_eye=frame [yvv2.COLORparagraph BGR2GRAY) in right_eye = cv2.resize (r_eye, (24pr 24)) ritual = r_eye.reshape (24th, 24th) ritual = r_eye.reshape (24th, 24th) ritual = np.expand_dims (r_eye Axis=0) rpred = model.predict_classes (r_eye) if (rpred [0] = = 1): lbl='Open' if (rpred [0] = = 0): lbl='Closed' break for (XLECHI YLECH) in left_eye: l_eye=frame [YRV YLTHLING XWEW] count=count+1 lager = cv2.cvtColor (l_eye) Cv2.COLOR_BGR2GRAY) lumbago = cv2.resize (l_eye, (24, 24)) l_eye=l_eye / 255 l_eye=l_eye.reshape (24, 24) lump = np.expand_dims (l_eye) Axis=0) lpred = model.predict_classes (l_eye) if (lpred [0] = = 1): lbl='Open' if (lpred [0] = = 0): lbl='Closed' break if (rpred [0] = = 0 and lpred [0] = = 0): score=score+1 cv2.putText (frame, "Closed", (10Metro Heightmur20), font,1, (255255255), 1 Cv2.LINE_AA) # if (rpred [0] = = 1 or lpred [0] = = 1): else: score=score-1 cv2.putText (frame, "Open", (10 Heights 20), font,1, (255255255), 1 if (score15): # person is feeling sleepy so we beepthe alarm cv2.imwrite (os.path.join (path,'image.jpg') Frame) try: sound.play () except: # isplaying = False pass if (thicc

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.