Shulou(Shulou.com)06/02 Report--

This article shows you how to achieve face alignment in OpenCV+Python. The content is concise and easy to understand. It can definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Preface

The goal is to demonstrate how to align faces using OpenCV, Python, and facial logos.

Given a set of facial markers (input coordinates), our goal is to distort the image and convert it into the output coordinate space.

In this output coordinate space, all faces in the entire dataset should:

Center in the image.

Rotate so that the eyes are on the horizontal line (that is, rotate the face so that the eyes are on the same y coordinates).

Scale so that the faces are roughly the same size.

To do this, we will first call an implemented Python class FaceAligner to align faces using affine transformations.

Note: affine transformations are used for rotation, scaling, translation, etc. We can package all three requirements above into a cv2.warpAffine call; the trick is to create the rotation matrix M.

Then, we will create a sample driver Python script to accept input images, detect faces, and align them.

Finally, we will review the results of face alignment using the OpenCV process.

Realization of facial orthosis

The face alignment algorithm itself is based on Chapter 8 of Mastering OpenCV with Practical Computer Vision Projects (Baggio, 2012), and I highly recommend it if you have a C++ background or interest. This book provides open access code examples on GitHub.

Create a new facealigner.py implementation FaceAligner class.

# import the necessary packagesfrom imutils.face_utils.helpers import FACIAL_LANDMARKS_68_IDXSfrom imutils.face_utils.helpers import FACIAL_LANDMARKS_5_IDXSfrom imutils.face_utils.helpers import shape_to_npimport numpy as npimport cv2class FaceAligner: def _ init__ (self, predictor, desiredLeftEye= (0.35,0.35), desiredFaceWidth=256, desiredFaceHeight=None): # store the facial landmark predictor, desired output left # eye position And desired output face width + height self.predictor = predictor self.desiredLeftEye = desiredLeftEye self.desiredFaceWidth = desiredFaceWidth self.desiredFaceHeight = desiredFaceHeight # if the desired face height is None, set it to be the # desired face width (normal behavior) if self.desiredFaceHeight is None: self.desiredFaceHeight = self.desiredFaceWidth Import necessary packages

The constructor defined starts our FaceAligner class.

Our constructor takes four parameters:

Predictor: facial iconic predictor model.

RequiredLeftEye: an optional (x, y) tuple that displays the default value and specifies the desired left eye position of the output. For this variable, you usually see a percentage in the range of 20-40%. These percentages control the visibility of the face after alignment. The exact percentage used will vary from application to application. When using 20%, you will basically get a "magnified" view of the face, while with a higher value, the face will appear more "smaller".

RequiredFaceWidth: another optional parameter that defines the face we want in pixels. We default this value to 256 pixels.

RequiredFaceHeight: the last optional parameter, which specifies the desired face height value in pixels.

Next, let's decide whether we want a square face image or a rectangular image. Check to see if requiredFaceHeight is None, and if so, we set it to desiredFaceWidth, which means that the face is square. A square image is a typical case. Alternatively, we can specify different values for desiredFaceWidth and desiredFaceHeight to get the rectangular area of interest.

Now that we have built our FaceAligner object, we will define a function that aligns faces.

This function is a little long, so I divided it into five code blocks to make it easier to understand:

Def align (self, image, gray, rect): # convert the landmark (x, y)-coordinates to a NumPy array shape = self.predictor (gray, rect) shape = shape_to_np (shape) # extract the left and right eye (x, y)-coordinates (lStart, lEnd) = FACIAL_LANDMARKS_IDXS ["left_eye"] (rStart REnd) = FACIAL_LANDMARKS_IDXS ["right_eye"] leftEyePts = shape [lStart:lEnd] rightEyePts = shape [rStart:rEnd]

The align function is defined, which takes three parameters:

Image: RGB input image.

Gray: grayscale input image.

Rect: a bounding box rectangle generated by dlib's HOG face detector.

Apply dlib's facial logo predictor and convert the logo to NumPy format (x, y) coordinates.

Next, read the left_eye and right_eye fields from the FACIAL_LANDMARK_IDXS dictionary found in the helpers.py script. These 2-tuple values are stored in the left / right eye start and end indexes.

LeftEyePts and rightEyePts were extracted.

Next, calculate the center of each eye and the angle between the centroids of the eyes.

This angle is a key part of aligning our images.

The angle of the green line between the eyes, as shown in the following picture, is what we are more concerned about.

Next is the angle calculation:

# compute the center of mass for each eye leftEyeCenter = leftEyePts.mean (axis=0) .astype ("int") rightEyeCenter = rightEyePts.mean (axis=0) .astype ("int") # compute the angle between the eye centroids dY = rightEyeCenter [1]-leftEyeCenter [1] dX = rightEyeCenter [0]-leftEyeCenter [0] angle = np.degrees (np.arctan2 (dY, dX))-180

The centroid of each eye, also known as the centroid, is calculated by averaging all the (x, y) points of each eye.

Given the center of the eye, we can calculate the difference of (x, y) coordinates and take the inverse tangent to obtain the rotation angle between the eyes.

This angle will allow us to correct the rotation.

To determine the angle, we first calculate the incremental dY in the y direction. This is done by finding the difference between rightEyeCenter and leftEyeCenter on line 38.

Similarly, we calculate dX, that is, the increment in the x direction of line 39.

Next, we calculate the angle of facial rotation. We use the arctan2 function of NumPy with parameters dY and dX, then convert it to degrees and subtract 180 to get the angle.

In the following code block, we calculate the desired right eye coordinates (as a function of the left eye position) and calculate the scale of the new resulting image.

# compute the desired right eye x-coordinate based on the # desired x-coordinate of the left eye desiredRightEyeX = 1.0-self.desiredLeftEye [0] # determine the scale of the new resulting image by taking # the ratio of the distance between eyes in the * current* # image to the ratio of distance between eyes in the # * desired* image dist = np.sqrt ((dX * * 2) + (dY * * 2)) DesiredDist = (desiredRightEyeX-self.desiredLeftEye [0]) desiredDist * = self.desiredFaceWidth scale = desiredDist / dist

Calculate the desired right eye according to the desired left eye x coordinates. Subtract self.desiredLeftEye [0] from 1.0 because the desired RightEyeX value should be equidistant from the right edge of the image, because the corresponding left eye x coordinate is the same distance from its left edge.

The ratio of the distance between the eyes in the current image to the distance between the eyes in the desired image can then be determined by obtaining the ratio of the distance between the eyes in the current image to that in the desired image.

First, calculate the Euclidean distance ratio dist.

Next, use the difference between the x values of the left and right eyes to calculate the required distance, desiredDist.

Update the required distance by multiplying the desired face width on line 52. This is essentially the eye distance scaled according to the desired width.

Finally, the scale is calculated by dividing desiredDist by the dist we calculated earlier.

Now that you have the rotation angle and scale, you need to take some steps before calculating the affine transformation. This includes finding the midpoint between the eyes and calculating the rotation matrix and updating its translation component:

# compute center (x, y)-coordinates (i.e., the median point) # between the two eyes in the input image eyesCenter = (int ((leftEyeCenter [0] + rightEyeCenter [0]) / / 2), int ((leftEyeCenter [1] + rightEyeCenter [1]) / / 2) # grab the rotation matrix for rotating and scaling the face M = cv2.getRotationMatrix2D (eyesCenter, angle Scale) # update the translation component of the matrix tX = self.desiredFaceWidth * 0.5tY = self.desiredFaceHeight * self.desiredLeftEye [1] M [0,2] + = (tX-eyesCenter [0]) M [1,2] + = (tY-eyesCenter [1])

Calculate the eyeCenter, the midpoint between the left and right eyes. This will be used in our rotation matrix calculation. In essence, this midpoint is at the top of the nose, where we rotate the face:

To calculate the rotation matrix M, we use cv2.getRotationMatrix2D to specify eyeCenter, angle, and scale. Each of these three values has been previously calculated, so please return as needed.

The parameters of cv2.getRotationMatrix2D are described as follows:

EyeCenter: the midpoint between the eyes is the point where we will rotate around the face.

Angle: we rotate the face to the angle to make sure the eyes are on the same level.

Scale: we will zoom in or out as a percentage of the image to make sure the image is scaled to the desired size.

Now the translation component of the matrix must be updated so that the face is still in the image after affine transformation.

Take half of the desired face width and store the value as tX, that is, the translation in the x direction.

To calculate the translation in the tY, y direction, multiply the desired facial height by the desired left eye y value, desiredLeftEye [1].

Using tX and tY, update the translation component of the matrix by subtracting each value from their corresponding midpoint values (rows 66 and 67).

Then apply affine transformation to align faces:

# apply the affine transformation (w, h) = (self.desiredFaceWidth, self.desiredFaceHeight) output = cv2.warpAffine (image, M, (w, h), flags=cv2.INTER_CUBIC) # return the aligned face return output

For convenience, store desiredFaceWidth and desiredFaceHeight in w and h, respectively (line 70).

Then call cv2.warpAffine to perform the final step. This function call requires three parameters and one optional parameter:

Image: face image.

M: translation, rotation, and scaling matrices.

(W, h): the required width and height of the output surface.

Flags: the interpolation algorithm for distortion, in this case INTER_CUBIC. For other possible flags and image conversions, refer to the OpenCV documentation.

Finally, align the face.

Align faces

Start writing a face alignment script and name it Now let's put this alignment class to work with a simple driver script. Open up a new file, name it align_faces.py:

# import the necessary packagesfrom imutils.face_utils import FaceAlignerfrom imutils.face_utils import rect_to_bbimport argparseimport imutilsimport dlibimport cv2# construct the argument parser and parse the argumentsap = argparse.ArgumentParser () ap.add_argument ("- p", "--shape-predictor", required=True,help= "path to facial landmark predictor") ap.add_argument ("- I", "- image", required=True,help= "path to input image") args = vars (ap.parse_args ())

If imutils and / or dlib are not installed on your system, be sure to install / upgrade them through pip:

Pip install-upgrade imutilspip install-upgrade dlib

Win10 installation dlib reference: how to install the dlib gpu version

# initialize dlib's face detector (HOG-based) and then create# the facial landmark predictor and the face alignerdetector = dlib.get_frontal_face_detector () predictor = dlib.shape_predictor (args ["shape_predictor"]) fa = FaceAligner (predictor, desiredFaceWidth=256)

Initialize our detector object with dlib's get_frontal_face_detector.

Use-- shape-predictor to instantiate our facial marker predictor, which is the path of dlib's pre-training predictor.

Take advantage of the FaceAligner class you just built in the previous section by initializing an object fa on line 21. We specify a face width of 256 pixels.

Next, load the image and prepare for face detection:

# load the input image, resize it, and convert it to grayscaleimage = cv2.imread (args ["image"]) image = imutils.resize (image, width=800) gray = cv2.cvtColor (image, cv2.COLOR_BGR2GRAY) # show the original input image and detect faces in the grayscale# imagecv2.imshow ("Input", image) rects = detector (gray, 2)

Loads the image specified by the command line argument-- image. Resize the image to maintain the aspect ratio of line 25 so that it is 800 pixels wide. Then convert the image to grayscale.

In order to deal with the face detection in the input image, we apply dlib's face detector. This function returns rects, which is a list of bounding boxes around faces found by our detector.

In the next block, we iterate through the rects, align each face, and display the original and aligned images.

# loop over the face detectionsfor rect in rects: # extract the ROI of the * original* face, then align the face # using facial landmarks (x, y, w, h) = rect_to_bb (rect) faceOrig = imutils.resize (image [y: y + h, xoriginal* face x + w], width=256) faceAligned = fa.align (image, gray, rect) # display the output images cv2.imshow ("Original" FaceOrig) cv2.imshow ("Aligned", faceAligned) cv2.waitKey (0)

Start the cycle.

For each bounding box rect predicted by dlib, we convert it to format (x, y, w, h).

Then, resize the box to a width of 256 pixels to maintain the aspect ratio. Save the original resizable image as faceOrig.

Align the image, specifying the image, grayscale image, and rectangle.

Finally, the original and corresponding aligned face images are displayed on the screens of their respective windows.

Wait for the user to press a key when either window is in focus before the next original / aligned image pair is displayed.

Repeat the above process for all detected faces, and then the script exits.

Show the results

Enter the command:

Python align_faces.py-shape-predictor shape_predictor_68_face_landmarks.dat-image 11.jpg

The above content is how to achieve face alignment in OpenCV+Python. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.