Shulou(Shulou.com)06/01 Report--

Based on the example analysis of face image preprocessing by neural network based on OpenCV, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

I am currently working on a computer vision problem involving facial classification. This usually means the application of deep learning, so a special preprocessing stage is needed before the image is injected into our neural network.

In order to improve the accuracy of our model, this is a very important task, which can be easily accomplished through the following simple steps. For this article, we can use OpenCV, a highly optimized open source library for computer vision, available in C++, java, and Python.

This is a short article that contains some basic guidelines, examples, and code that you may need to apply to every facial classification or recognition problem.

Note: all still images used in this article are from https://imgflip.com/memetemplates

Picture loading

We will use the imread () function to load the image and specify the path and mode of the file. The second parameter is important for dynamically running basic channels and depth conversions.

Mode: https://docs.opencv.org/4.1.0/d4/da8/group__imgcodecs.html#ga61d9b0126a3e57d9277ac48327799c80

Img = cv2.imread ('path/image.jpg', cv2.IMREAD_COLOR)

To view the image, we have the imshow () function:

Cv2.imshow (img)

If you write img, you will see that the size is (height, weight, channels).

Our color image has three channels: blue, green and red (in this order in OpenCV).

We can easily view a single channel:

# Example for green channelimg [:,:, 0]; img [:,: 2]; cv2.imshow (img) grayscale version

To avoid distractions in facial image classification, using black-and-white images is a good idea (or maybe not!). You can try both. To get the grayscale version, we just need to specify it in the image load function, passing the appropriate value as the second parameter:

Img = cv2.imread ('path/image.jpg', cv2.IMREAD_GRAYSCALE)

Now our image has a separate channel!

Face and eye detection

When dealing with facial classification problems, we may want to do facial tests to verify (do you have a face?), crop and straighten our images. We will use the cascade classifier based on Haar feature included in OpenCV for object detection. (https://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html)

First of all, we choose a pre-trained face and eye detection classifier. There is a list of XML files available and we can use this list:

Https://github.com/opencv/opencv/tree/master/data/haarcascades

1) for face detection, OpenCV provides the following (from the loosest prior to the strictest prior):

Haarcascade_frontalface_default.xml

Haarcascade_frontalface_alt.xml

Haarcascade_frontalface_alt2.xml

Haarcascade_frontalface_alt_tree.xml

2) for eye detection, we can choose two methods:

Haarcascade_eye.xml

Haarcascade_eye_tree_eyegasses.xml

We load the pre-trained classifier in this way:

Face_cascade = cv2.CascadeClassifier (cv2.data.haarcascades + 'haarcascade_frontalface_default.xml') eyes_cascade = cv2.CascadeClassifier (cv2.data.haarcascades +' haarcascade_eye.xml')

You can test several combinations. Remember, in all cases, none of them is optimal (if the first classifier fails, you can try the second classifier, or even all classifiers).

For face detection, we use the following code:

Faces_detected = face_cascade.detectMultiScale (img, scaleFactor=1.1, minNeighbors=5)

The result is an array of all detected faces. We can easily draw a rectangle:

(X, y, w, h) = faces_detected [0] cv2.rectangle (img, (x, y), (x, y), (0,255,0), 1); cv2.imshow (img)

For eyes, we search in a similar way, but narrow the search to a facial rectangle:

Eyes = eyes_cascade.detectMultiScale (img [y:y+h, x:x+w]) for (ex, ey, ew, eh) in eyes: cv2.rectangle (img, (x+ex, y+ey), (x+ex+ew, y+ey+eh), (255,255,255), 1)

It's done!

Although this is the expected result, we will encounter many other problems. Most of the time, we don't have a positive and clear face, or even.

No eyes:

The eyes are black stains surrounded by white:

There are 4 eyes here and only 3 eyes are detected:

Straighten the face

By calculating the angle between the two eyes, we can straighten the face image (which is easy). After the calculation, we only need two steps to rotate the image:

Rows, cols = img.shape [: 2] M = cv2.getRotationMatrix2D ((cols/2, rows/2), 1) img_rotated = cv2.warpAffine (face_orig, M, (cols,rows))

Cut the face.

In order to help our neural network to complete the task of facial classification, it is best to remove external interference information such as background, clothing or accessories. In this case, cutting the face is a good choice.

The first thing we need to do is to get the face rectangle again from the straightened image. Then we need to make a decision: we can crop the rectangular area as is, or add an extra fill, so we can get more space.

It depends on the specific problem to be solved (by age, sex, race, etc.); maybe you want more hair; maybe you don't.

Finally, crop (p for fill):

Cv2.imwrite ('crop.jpg', img_ rotate [y-p+1:y+h+p, x-p+1:x+w+p])

Look! This face is isolated and can almost be studied in depth.

Image zooming

Neural networks require all input images to have the same shape and size, because GPU applies the same instructions to a batch of images at the same time to achieve super fast speed. We can resize them dynamically, but this may not be a good idea because multiple conversions will be performed on each file during training.

Therefore, if our dataset has a lot of images, we should consider implementing the process of batch resizing before the training phase.

In OpenCV, we can use the resize () function to perform downward scaling and upward scaling, and there are several interpolation methods available. An example of specifying the final size:

Cv2.resize (img, (,), interpolation=cv2.INTER_LINEAR)

To shrink the image, OpenCV recommends using INTER_AREA interpolation, while to enlarge the image, you can use INTER_CUBIC (slow) or INTER_LINEAR (fast, still good).

Finally, there is a tradeoff between quality and time.

I made a quick upgrade comparison:

The first two images seem to be of higher quality (but you can observe some compression artifacts).

The result of the linear method is significantly smoother and less noisy.

The last one is pixel-based.

Normalization

We can use the normalize () function to apply visual normalization to fix very dark / bright images (or even low contrast).

The normalized type (https://docs.opencv.org/3.4/d2/de8/group__core__array.html#gad12cefbcb5291cf958a85b4b67b6149f) is specified in the function arguments:

Norm_img = np.zeros ((300,300)) norm_img = cv2.normalize (img, norm_img, 0,255, cv2.NORM_MINMAX)

Example:

When using the image as the input of the depth convolution neural network, it is not necessary to apply this normalization.

In practice, we will normalize each channel appropriately, such as minus the average, and then divide it by the standard deviation at the pixel level (so we get the average 0 and the deviation 1). If we use transfer learning, the best method is always to use pre-trained model statistics.

When dealing with the problem of face classification / recognition, if the input image is not a passport picture, it is a common task to detect and separate faces.

OpenCV is a good library for image preprocessing, but it's more than that. It is also a powerful tool for many computer vision applications.

This is the answer to the sample analysis of face image preprocessing by neural network based on OpenCV. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.