Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "Python how to realize real-time camera face change". Xiaobian shows you the operation process through actual cases. The operation method is simple, fast and practical. I hope this article "Python how to realize real-time camera face change" can help you solve the problem.

environment

python3.9.6

pycharm 2021

Library environment:

dlib

opencv-python

basic principle

Use dlib's shape_predictor_68_face_landmarks.dat model to get a picture with a face (1.png) and the camera's own 68 face feature points.

Acquire face masks according to face feature points

Affine transform the first picture to align its face with the face in the camera picture to get a new picture

Affine performs the same operation on the face mask

union of two resulting graphs (not leaving anything else empty)

Use opencv to perform Poisson fusion on the affine transformed a picture and camera picture.

Full source code # -*- coding: utf-8-*-import cv2import dlibimport numpy as npdetector = dlib.get_frontal_face_detector() # dlib's forward face detector predictor = dlib.shape_predictor(r'shape_predictor_68_face_landmarks.dat')# dlib's face shape detector def get_image_size(image): """ Get image size (height, width) :param image: image :return: (height, width) """ image_size = (image.shape[0], image.shape[1]) return image_sizedef get_face_landmarks(image, face_detector, shape_predictor): """ 68 feature points :param image: image :param face_detector: dlib.get_frontal_face_detector :param shape_predictor: dlib.shape_predictor :return: np.array([[],[]), 68 feature points """ dets = face_detector(image, 1) shape = shape_predictor(image, dets[0]) face_landmarks = np.array([[p.x, p.y] for p in shape.parts()]) return face_landmarksdef get_face_mask(image_size, face_landmarks): """ Get Face Mask :param image_size: Image size :param face_landmarks: 68 feature points :return: image_mask, mask picture """ mask = np.zeros(image_size, dtype=np.uint8) points = np.concatenate([face_landmarks[0:16], face_landmarks[26:17:-1]]) cv2.fillPoly(img=mask, pts=[points], color=255) return maskdef get_affine_image(image1, image2, face_landmarks1, face_landmarks2): """ Get picture 1 affine transformed picture :param image1: Picture 1, Picture to be affine transformed :param image2: image2, as long as used to get the image size, generate affine transformed images of the same size :param face_landmarks1: Face feature points of picture 1 :param face_landmarks2: Face feature points of picture 2 :return: affine transformed picture """ three_points_index = [18, 8, 25] M = cv2.getAffineTransform(face_landmarks1[three_points_index].astype(np.float32), face_landmarks2[three_points_index].astype(np.float32)) dsize = (image2.shape[1], image2.shape[0]) affine_image = cv2.warpAffine(image1, M, dsize) return affine_image.astype(np.uint8)def get_mask_center_point(image_mask): """ Get the coordinates of the center point of the mask :param image_mask: mask picture :return: Mask Center """ image_mask_index = np.argwhere(image_mask > 0) miny, minx = np.min(image_mask_index, axis=0) maxy, maxx = np.max(image_mask_index, axis=0) center_point = ((maxx + minx) // 2, (maxy + miny) // 2) return center_pointdef get_mask_union(mask1, mask2): """ Get union of two masked parts :param mask1: mask_image, mask 1 :param mask2: mask_image, mask 2 :return: union of two masked parts """ mask = np.min([mask1, mask2], axis=0) #mask partial union mask = ((cv2.blur(mask, (5, 5))== 255) * 255). atype (np.uint8) #Reduce mask size mask = cv2.blur(mask, (3, 3)). atype (np.uint8) #blur mask return maskdef skin_color_adjustment(im1, im2, mask=None): """ skin tone adjustment :param im1: pics 1 :param im2: pics 2 :param mask: Face mask. if so, use that partial mean of the face to calculate the skin-color transform coefficient; otherwise, using Gaussian blur to calculate the skin-color transform coefficient :return: Picture 1 adjusted according to the color of Picture 2 """ if mask is None: im1_ksize = 55 im2_ksize = 55 im1_factor = cv2.GaussianBlur(im1, (im1_ksize, im1_ksize), 0).astype(np.float) im2_factor = cv2.GaussianBlur(im2, (im2_ksize, im2_ksize), 0).astype(np.float) else: im1_face_image = cv2.bitwise_and(im1, im1, mask=mask) im2_face_image = cv2.bitwise_and(im2, im2, mask=mask) im1_factor = np.mean(im1_face_image, axis=(0, 1)) im2_factor = np.mean(im2_face_image, axis=(0, 1)) im1 = np.clip((im1.astype(np.float) * im2_factor / np.clip(im1_factor, 1e-6, None)), 0, 255).astype(np.uint8) return im1def main(): im1 = cv2.imread('1.png') # face_image im1 = cv2.resize(im1, (600, im1.shape[0] * 600 // im1.shape[1])) landmarks1 = get_face_landmarks(im1, detector, predictor) # 68_face_landmarks if landmarks1 is None: print ('{}: no face detected'. format(image_face_path)) exit(1) im1_size = get_image_size(im1) #Face image size im1_mask = get_face_mask(im1_size, landmarks1) #Face Mask cam = cv2.VideoCapture(0) while True: ret_val, im2 = cam.read() # camera_image landmarks2 = get_face_landmarks(im2, detector, predictor) # 68_face_landmarks if landmarks2 is not None: im2_size = get_image_size(im2) #Camera image size im2_mask = get_face_mask(im2_size, landmarks2) #Camera Picture Face Mask affine_im1 = get_affine_image(im1, im2, landmarks1, landmarks2) # im1 (face image) affine transformed image affine_im1_mask = get_affine_image(im1_mask, im2, landmarks1, landmarks2) # im1 (face map) Face mask of affine transformed picture union_mask = get_mask_union(im2_mask, affine_im1_mask) #mask merge affine_im1 = skin_color_adjustment(affine_im1, im2, mask=union_mask) #Skin color adjustment point = get_mask_center_point(affine_im1_mask) # im1 (face image) Center point of face mask of affine transformed picture seamless_im = cv2.seamlessClone(affine_im1, im2, mask=union_mask, p=point, flags= cv2.NORMAL_CLONE) #Perform Poisson Fusion cv2.imshow('seamless_im', seamless_im) else: cv2.imshow('seamless_im', im2) if cv2.waitKey(1) == 27: #Press Esc to exit break cv2.destroyAllWindows()if __name__ == '__main__': main() on "Python how to achieve real-time camera face change" content introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the industry information channel. Xiaobian will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.