Shulou(Shulou.com)06/02 Report--

Abstract: comprehensively analyze the principle of face recognition technology, the situation of talents in the field, the field of technology application and the development trend.

Since the second half of the 20th century, computer vision technology has gradually developed. At the same time, with the widespread use of software and hardware technologies related to digital images in people's lives, digital images have become an important component of the source of information in contemporary society. the needs and applications of various image processing and analysis also continue to promote the innovation of this technology. Computer vision technology is widely used. Computer vision technology is involved in the fields of digital image retrieval management, medical image analysis, intelligent security inspection, human-computer interaction and so on. This technology is not only an important part of artificial intelligence technology, but also the frontier field of computer science research. After the continuous development in recent years, a set of digital signal processing technology has been gradually formed. The comprehensive technology of the combination of computer graphics and images, information theory and semantics, and has a strong edge and interdisciplinary nature. Among them, face detection and recognition is a hot research topic in image processing, pattern recognition and computer vision, and it is also the most concerned branch of biometric recognition.

Face recognition is a biometric recognition technology based on human facial feature information. Usually the camera or camera is used to capture the image or video stream containing human face, and automatically detect and track the human face in the image. According to the data, the global market size of biometric technology rose to US $17.2 billion in 2017, and it is estimated that the global biometric market is likely to reach US $24 billion by 2020. From 2015 to 2020, the size of the face recognition market has increased by 166.6%, ranking first among many biometric technologies. It is expected that the size of the face recognition market will rise to US $2.4 billion by 2020.

In this issue of intelligent internal reference, we recommend the research report of the Aminer mining project led by Associate Professor Tang Jie of Tsinghua University, big data, which explains face recognition technology and its application fields, introduces domestic players in the field of face recognition and predicts the development trend of this technology.

Overview of face recognition Technology

1. Basic concepts

The unique charm of human vision system drives researchers to simulate the human ability to collect, process, analyze and learn images of the three-dimensional world through vision sensors and computer software and hardware. In order to make computers and robot systems have intelligent visual functions. In the past 30 years, scientists in many different fields have been trying to understand the mysteries of biological vision and nervous system from many angles, in order to benefit mankind with their research results. Since the second half of the 20th century, computer vision technology has gradually developed under this background. At the same time, with the widespread use of software and hardware technologies related to digital images in people's lives, digital images have become an important component of the source of information in contemporary society. the needs and applications of various image processing and analysis also continue to promote the innovation of this technology.

Computer vision technology is widely used. Computer vision technology is involved in the fields of digital image retrieval management, medical image analysis, intelligent security inspection, human-computer interaction and so on. This technology is not only an important part of artificial intelligence technology, but also the frontier field of computer science research. After the continuous development in recent years, it has gradually formed a set of comprehensive technology based on the combination of digital signal processing technology, computer graphics and image, information theory and semantics, and has a strong edge and interdisciplinary nature. Among them, face detection and recognition is a hot research topic in image processing, pattern recognition and computer vision, and it is also the most concerned branch of biometric recognition.

Face recognition is a biometric recognition technology based on human facial feature information. Usually the camera or camera is used to capture the image or video stream containing human face, and automatically detect and track the human face in the image. According to the "2018 China Biometrics Market Analysis report-Industry in-depth Analysis and Development Prospect Forecast" released by China Report.com, the global market size of biometric technology rose to US $17.2 billion in 2017, and by 2020, it is estimated that the world biometric market is likely to reach US $24 billion. From 2015 to 2020, the size of the face recognition market has increased by 166.6%, ranking first among many biometric technologies. It is expected that the size of the face recognition market will rise to US $2.4 billion by 2020.

In different biometric recognition methods, face recognition has its own special advantages, so it plays an important role in biometric recognition. Five advantages of face recognition:

Non-intrusive. Face recognition can achieve better recognition results without interfering with people's normal behavior, and there is no need to worry about whether the recognized person is willing to put his hand on the fingerprint acquisition device, whether their eyes can be aimed at the iris scanning device, and so on. As long as you naturally stay in front of the camera for a moment, the user's identity will be correctly identified.

Convenience. The collection equipment is simple and quick to use. Generally speaking, common cameras can be used to collect face images without special complex special equipment. Image acquisition can be completed in a few seconds.

Friendliness. The method of face recognition is consistent with human habits, and both people and machines can use face images for recognition. Fingerprint, iris and other methods do not have this characteristic, a person without special training, can not use fingerprints and iris images to identify other people.

Non-contact. The collection of face image information is different from the collection of fingerprint information, the use of fingerprint information requires finger contact with the acquisition equipment, which is not only unhygienic, but also easy to cause disgust of users, while face image acquisition, users do not need to be in direct contact with the device.

Scalability. After face recognition, the next step of data processing and application determines the practical application of face recognition equipment, such as access control, face image search, commuting card, terrorist recognition and other fields.

It is precisely because of these good characteristics that face recognition has a very wide application prospect, and it is attracting more and more attention from the academic and business circles. Face recognition has been widely used in identity recognition, living detection, lip recognition, creative camera, face beautification, social platform and other scenes.

2. The course of development

As early as the 1950s, cognitive scientists have begun to study face recognition. In the 1960s, the research on the engineering application of face recognition was officially launched. The method at that time mainly made use of the geometric structure of human face and identified it by analyzing the feature points of facial organs and the topological relationship between them. This method is simple and intuitive, but once the facial posture and expression change, the accuracy will be seriously reduced.

In the 1990s: in 1991, the famous eigenface (Eigenface) method introduced principal component analysis and statistical feature technology into face recognition for the first time, and made great progress in practical results. This idea has also been further carried forward in the follow-up research. For example, Belhumer successfully applied the Fisher criterion to face classification and proposed a Fisherface method based on linear discriminant analysis.

2000-2012: in the first decade of the 21st century, with the development of machine learning theory, scholars have explored face recognition based on genetic algorithm, support vector machine (Support Vector Machine,SVM), boosting, manifold learning and kernel methods. From 2009 to 2012, sparse representation (Sparse Representation) became a research hotspot because of its beautiful theory and robustness to occlusion factors. At the same time, the industry has basically reached a consensus: feature extraction and subspace selection based on artificially carefully designed local descriptors can achieve the best recognition results.

Gabor and LBP feature descriptors are by far the two most successful artificially designed local descriptors in the field of face recognition. During this period, the targeted processing of a variety of face recognition factors is also the focus of research at that stage, such as face illumination normalization, face posture correction, face super-resolution and occlusion processing.

It is also at this stage that the focus of researchers begins to shift from face recognition in restricted scenes to face recognition in unrestricted environments. The LFW face recognition open competition (LFW is a public face number set published and maintained by the University of Massachusetts with a test data scale of 10,000) became popular against this background. at that time, although the best recognition system could achieve more than 99% recognition accuracy on the limited FRGC test set, the highest accuracy on LFW was only about 80%, which seemed to be a long way from practical use.

2013: Microsoft Research Asia researchers tried 100,000 large-scale training data for the first time, and achieved 95.17% accuracy on LFW based on high-dimensional LBP features and Joint Bayesian method. The results show that large training data sets are very important to effectively improve face recognition in unrestricted environment. However, all of the above classical methods are difficult to deal with the training scenarios of large-scale data sets.

2014: with the development of big data and deep learning, neural network has attracted more attention and achieved much better results than classical methods in image classification, handwriting recognition, speech recognition and other applications. Sun Yi of the Chinese University of Hong Kong and others proposed to apply convolution neural network to face recognition, using 200000 training data to achieve a recognition accuracy that exceeds the human level for the first time on LFW, which is a milestone in the history of face recognition. Since then, researchers have continuously improved the network structure and expanded the size of training samples, pushing the recognition accuracy on LFW to more than 99.5%. In the development of face recognition, some classical methods and their accuracy on LFW have a basic trend: the scale of training data is getting larger and larger, and the recognition accuracy is getting higher and higher.

The Development of ▲ face recognition Technology

3. China's policy support

Since 2015, the state has issued a series of guidelines on remote opening RMB accounts by banking financial institutions (draft for soliciting opinions), opening the door for the popularization of face recognition. After that, laws and regulations such as "Technical requirements for Security Prevention Video Surveillance face recognition system" and "Security Technical requirements for Information Security Technology Network face recognition Authentication system" have laid a solid foundation for the popularization of face recognition in financial, security, medical and other fields, and cleared policy obstacles. At the same time, artificial intelligence was written into the national government report for the first time in 2017. as an important subdivision of artificial intelligence, the national policy support for face recognition is increasing. According to the three-year Action Plan for promoting the Development of the New Generation artificial Intelligence Industry (2018-2020) released in December 2017, by 2020, the effective detection rate of face recognition in complex dynamic scenes will exceed 97%, and the correct recognition rate will exceed 90%.

Policies related to ▲ face recognition

4. Hot spots of development

Through the mining of previous papers in the field of face recognition, it is concluded that the research keywords in the field of face recognition are mainly focused on face recognition, feature extraction, sparse representation, image classification, neural network, target detection, face image, face detection, image representation, computer vision, pose estimation, face recognition and so on.

The following picture is an analysis of the research trend of face recognition, which aims to study the technology source, heat and even development trend on the basis of historical scientific research data. In figure 2, each color branch represents a keyword field, and its width indicates the research heat of the keyword, and the position of each keyword in each year is sorted according to the heat height of all keywords at that time. At first, Computer Vision (computer vision) is the focus of research. At the end of the 20th century, Feature Extraction (feature extraction) surpassed CV and became a new hot spot of research. Then it was surpassed by Face Recognition at the beginning of the 21st century and has been in the second position.

Hot spots related to ▲ face recognition

In addition, according to the keywords extracted from the papers published in FG (International Conference on Automatic Face and Gesture Recognition) in the past two years, it is found that Face Recognition has the highest frequency of occurrence with 118 times, Object Detection ranks second with 41 times, Image Classification and Object Recognition rank third with 36 times, and words with more than 10 occurrence times are Image Segmentation (32), Action Recognition (32), Sparse Representation (28), Image Retrieval (27), Visual Tracking (24) and SingleImage (23). The word cloud diagram is as follows:

Word Cloud Analysis of ▲ face recognition

5. Face recognition related meetings

Three top international conferences in computer vision (CV):

ICCV: IEEE International Conference on Computer Vision

The conference is sponsored by the American Institute of Electrical and Electronic Engineers (IEEE, Institute of Electrical & Electronic Engineers) and is mainly held in some countries with strong scientific research strength in Europe, Asia and the Americas. As the world's top academic conference, the first International computer Vision Conference was opened in London in 1987 and held in the following two years. ICCV is the highest-level conference in the field of computer vision, and the proceedings of the conference represent the latest development direction and level in the field of computer vision. The acceptance rate of the paper is about 20%. The direction is computer vision, pattern recognition, multimedia computing and so on.

In recent years, the global academic community has paid more and more attention to the scientific research achievements made by Chinese in the field of computer vision. This is because Chinese-led research has made great progress. In 2007, more than 1200 papers were received and only 244 papers were selected, of which more than 30 were from Chinese mainland, Hong Kong and Taiwan, accounting for more than 12% of the total. As the first Chinese team to invest in the research and development of deep learning technology, on the basis of years of layout of key technologies, the team led by Tang Xiaoou, a professor at the Chinese University of Hong Kong, quickly made technological breakthroughs. The only two deep learning articles presented at the International Conference on Computational Vision and pattern recognition (CVPR) in 2012 were from Tang Xiaoou Lab, while six of the 8 deep learning articles published by global scholars at the International Conference on computer Vision (ICCV) in 2013 were from Tang Xiaoou Lab.

CVPR:IEEE Conference on Computer Vision and Pattern Recognition

The conference is a top conference in the field of computer vision and pattern recognition organized by IEEE. It is held once a year, and the admission rate is about 25%. The direction is computer vision, pattern recognition, multimedia computing and so on.

The team led by Tang Xiaoou, a professor at the Chinese University of Hong Kong, has made a large number of deep learning original technological breakthroughs around the world: the only two deep learning articles at the 2012 International Conference on Computational Vision and pattern recognition (CVPR) were from his laboratory. From 2011 to 2013, 14 deep learning papers were published at the two top conferences in the field of computer vision, ICCV and CVPR, accounting for nearly half of the total number of deep learning papers in the world (29). In 2009, he won the best paper award of CVPR, one of the two top international academic conferences in the field of computer vision, the first time in CVPR history that a paper from Asia won the award.

ECCV:European Conference on Computer Vision

ECCV is an European conference, each meeting accepts about 300 papers worldwide, the main admission papers are from the United States, Europe and other top laboratories and research institutes, Chinese mainland generally between 10-20 papers. The paper acceptance rate of ECCV2010 is 27%. Once every two years, the acceptance rate of the paper is about 20%. The direction is computer vision, pattern recognition, multimedia computing and so on. The 2018 ECCV will be held in Munich, Germany, from September 8 to 14, 2018.

Asian Conference on computer Vision:

ACCV:Asian Conference on Computer Vision

ACCV, the Asian computer Vision Conference, is a biennial conference organized by AFCV (Asian Federation of Computer Vision, Asian computer Vision Alliance) since 1993. It aims to provide a good platform for researchers, developers and participants to demonstrate and discuss new problems, solutions and technologies in the field of computer vision and related fields. The 14th Asian computer Vision Conference in 2018 will be held in Australia from December 4 to 6, 2018.

Special meeting on face and gesture recognition:

FG:IEEE International Conference on Automatic Face and Gesture Recognition

"International Conference on Automatic Face and Gesture Recognition" is an authoritative academic conference in the field of face and gesture recognition around the world. Face detection, face recognition, expression recognition, posture analysis, psychological behavior analysis and so on.

Detailed explanation of face recognition technology

1. Face recognition process

To put it simply, the principle of face recognition technology mainly consists of three steps: one is to establish a database containing a large number of face images, and the other is to obtain the target face images to be recognized by various ways. the third is to compare and screen the target face image with the existing face image in the database. According to the principle of face recognition technology, the technical process mainly includes the following four parts, namely, face image acquisition and preprocessing, face detection, face feature extraction, face recognition and living identification.

▲ face recognition technology flow

Acquisition and preprocessing of face Image

Face image acquisition and detection can be divided into two parts: face image acquisition and face image detection.

Face image acquisition: there are usually two ways to collect face images, namely, the batch import of existing face images and the real-time acquisition of face images. Some advanced face recognition systems can even support conditional filtering of face images that do not meet the quality requirements of face recognition or low definition, and collect them as clearly and accurately as possible. Batch import of existing face images: the face images collected by various ways will be imported into the face recognition system, and the system will automatically complete the collection of face images one by one. Real-time acquisition of face image: that is, the camera or camera is called to automatically capture the face image in real time and complete the collection work within the shooting range of the equipment.

Face image preprocessing: the purpose of face image preprocessing is to further process the face image to facilitate the feature extraction of the face image on the basis of the detection of the face image by the system. Face image preprocessing specifically refers to a series of complex processing processes such as light, rotation, cutting, filtering, noise reduction, magnification and reduction of the face image collected by the system, so that the face image can meet the standard requirements of face image feature extraction in terms of light, angle, distance, size and so on. When collecting images in the real environment, due to the interference of many external factors, such as different light and shade, facial expression changes, shadow occlusion and so on, the quality of the collected images is not ideal, so it is necessary to preprocess the collected images first. if the image preprocessing is not good, it will seriously affect the follow-up face detection and recognition. This paper studies and introduces three kinds of image preprocessing methods, namely, gray adjustment, image filtering, image size normalization and so on.

Grayscale adjustment: because the final image of face image processing is generally a binary image, and because of the difference in location, equipment, lighting and other aspects, the quality of the color image is different, so it is necessary to carry on the unified gray processing to the image to smooth these differences. The common methods of gray adjustment are average method, histogram transformation method, power transformation method, logarithmic transformation method and so on.

Image filtering: in the actual process of face image acquisition, the quality of face image will be affected by a variety of noises, which come from many aspects. for example, the surrounding environment is filled with a large number of electromagnetic signals, digital image transmission is affected by electromagnetic signal interference and other channels, and then affect the quality of face images. In order to ensure the quality of the image and reduce the impact of noise on the subsequent processing, the image must be denoised. There are many principles and methods of noise removal, such as mean filtering, median filtering and so on. At present, median filtering algorithms are commonly used to preprocess face images.

Image size normalization: in simple face training, when the image pixel size of the face database is different, we need to normalize the image size before the upper computer face comparison recognition. The common size normalization algorithms are bilinear interpolation algorithm, nearest neighbor interpolation algorithm, cubic convolution algorithm and so on.

Face detection

A picture containing a face image may usually contain other content, so it is necessary to carry out the necessary face detection. That is, in a face image, the system will accurately locate the location and size of the face, pick out useful image information and automatically eliminate other redundant image information to further ensure the accurate acquisition of face image.

Face detection is an important part of face recognition. Face detection refers to the process of using a certain strategy to retrieve the given picture or video to determine whether there is a face, and if so, to locate the location, size and posture of each face. Face detection is a challenging target detection problem, which is mainly reflected in two aspects: 1, the face has quite complex details and different expressions (eyes, mouth opening and closing, etc.). Different faces have different appearance, such as face shape, skin color, etc.; 2, face occlusion, such as glasses, hair and head accessories. The change of external conditions causes: (1) multi-pose of human face due to different imaging angles, such as in-plane rotation, depth rotation and up and down rotation, in which depth rotation has a greater influence; (2) the influence of illumination, such as the change of brightness, contrast and shadow in the image; (3) the imaging conditions of the image, such as the focal length of the camera equipment, the imaging distance, etc.

The role of face detection is that in a face image, the system will accurately locate the location and size of the face, pick out useful image information and automatically eliminate other redundant image information to further ensure the accurate collection of face images. Face detection focuses on the following indicators:

Detection rate: identify the correct face / all the faces in the picture. The higher the detection rate, the better the effect of the detection model; false detection rate: identify the wrong face / recognized face. The lower the false detection rate is, the better the detection model is; the missed detection rate is the unrecognized faces / all the faces in the image. The lower the rate of missed detection, the better the effect of the detection model; speed: the time from the completion of image acquisition to the completion of face detection. The shorter the time is, the better the detection model is.

The current face detection methods can be divided into three categories, which are skin color model-based detection, edge feature-based detection and statistical theory-based methods.

1. Detection based on skin color model: when skin color is used for face detection, different modeling methods can be used, such as Gaussian model, Gaussian mixture model, non-parametric estimation and so on. Using Gaussian model and Gaussian mixture model, skin color models can be established in different color spaces for face detection. The method of face detection by extracting the face region in the color image can deal with many kinds of illumination, but the algorithm needs to be effective under the premise of fixed camera parameters. Comaniciu and other scholars use nonparametric kernel probability density estimation method to build skin color model, and use mean-shift method for local search to achieve face detection and tracking. This method improves the speed of face detection and is robust to occlusion and illumination. The disadvantage of this method is that it can not be combined with other methods. at the same time, it is difficult to deal with complex background and multiple faces when used in face detection.

In order to solve the problem of illumination in face detection, we can compensate for different illumination, and then detect the skin color region in the image. This can solve the problems of polarization, complex background and multiple face detection in color images, but it is insensitive to face color, position, scale, rotation, posture and expression.

2. Detection based on edge features: when using the edge features of images to detect human faces, the amount of computation is relatively small, and real-time detection can be realized. Most algorithms that use edge features are based on the edge contour characteristics of the human face, using established templates (such as elliptical templates) for matching. Some researchers also use elliptical ring model and edge direction features to realize face detection with simple background. Fr ö ba and others use the method based on edge direction matching (Edge-Orientation Matching,EOM) to detect faces in edge patterns. The error detection rate of this algorithm is relatively high in complex background, but it can achieve good results when combined with other features.

3. Methods based on statistical theory: this paper focuses on the Adaboost face detection algorithm based on statistical theory. Adaboost algorithm is a process of finding the optimal classifier through numerous loop iterations. Any feature of the weak classifier Haar feature is placed on the face sample to find the face feature value. Through the cascade of more classifiers, the quantitative features of the face are obtained to distinguish the human face from the non-face. The Haar function consists of simple black, white, horizontal, vertical or 45 °rotated rectangles. Generally speaking, the current Haar features can be divided into three categories: edge feature, line feature and center feature.

This algorithm is proposed by two scholars Paul Viola and Michael Jones of Cambridge University. The advantage of this algorithm is that it not only has fast computing speed, but also can achieve the same performance as other algorithms, so it is widely used in face detection, but it also has a high false detection rate. Because in the process of using Adaboost algorithm to learn, there are always some face and non-face patterns that are difficult to distinguish, and there are some windows in the detection results that are not similar to face patterns.

Facial feature extraction

At present, the features that can be supported by mainstream face recognition systems can be divided into face vision features, face image pixel statistical features and so on, and face image feature extraction is aimed at some specific features of human face. The feature is simple, and the matching algorithm is simple, which is suitable for large-scale database construction; on the contrary, it is suitable for small-scale library. Feature extraction methods generally include knowledge-based extraction methods or algebraic feature extraction methods.

Take one of the knowledge-based face recognition extraction methods as an example, because the human face is mainly composed of eyes, forehead, nose, ears, chin, mouth and so on. These parts and the structural relationship between them can be described by geometric features, that is to say, each person's face image can have a corresponding geometric feature. It can help us as an important differential feature of face recognition, which is also one of the knowledge-based extraction methods.

Face recognition

We can set a value of human face similarity in the face recognition system, and then compare the corresponding face image with all the face images in the system database. If it exceeds the preset similarity value, then the system will output the excess face image one by one. At this time, we need to accurately screen the face image according to the degree of similarity of the face image and the identity information of the face itself. This accurate screening process can be divided into two categories: one is one-to-one screening, that is, the process of confirming the identity of human faces. The other is one-to-many screening, that is, the process of matching and matching according to the degree of facial similarity.

In vivo identification

One of the common problems of biometric recognition is to distinguish whether the signal comes from a real organism, for example, the fingerprint recognition system needs to distinguish whether the fingerprint with identification comes from human fingers or fingerprint gloves. whether the face image collected by the face recognition system comes from the real face or the photo containing the face. Therefore, the actual face recognition system generally needs to add living identification links, such as requiring people to turn their heads left and right, blink, open their mouths and so on.

2. The main methods of face recognition

The research of face recognition technology is a high-end technical research work that spans many disciplines, including professional knowledge of many disciplines, such as image processing, physiology, psychology, pattern recognition and so on. In the research field of face recognition technology, there are mainly several research directions, such as: one is based on facial feature statistics, which mainly includes eigenface method and hidden Markov model (HMM,Hidden Markov Model) method, and the other is about connection mechanism, mainly artificial neural network (ANN,Artificial Neural Network) method and support vector machine (SVM,Support Vector Machine) method. Another method is to integrate multiple recognition methods.

Method based on eigenface

The method of feature face is a classical and widely used face recognition method. Its main principle is to make the image dimensionality reduction algorithm, which makes the data processing easier and faster. In fact, the face recognition method of feature face is to transform the image into a high-dimensional vector into a low-dimensional vector, so as to eliminate the relevance of each component and reduce the corresponding eigenvalues of the transformed image. After Kmurl transform, the image has good displacement invariance and stability. Therefore, the face recognition method of feature face has the advantages of convenient implementation, faster speed, and high recognition rate for frontal face images. However, this method also has some shortcomings, that is, it is easy to be affected by facial expression, posture, illumination and other factors, resulting in low recognition rate.

A method based on geometric features

The recognition method based on geometric features is a face recognition method based on the features and geometric shapes of facial organs, which is the earliest recognition method studied and used by people. it mainly uses different features of different faces and other information for matching recognition, this algorithm has a fast recognition speed, at the same time, its memory is relatively small, but its recognition rate is not high. The main method of this method is to first detect the position and size of the main facial feature organs such as mouth, nose and eyes, and then use the geometric distribution and proportion of these organs to match, so as to achieve face recognition.

The process of geometric feature recognition is roughly as follows: first, detect the feature points and their positions of the face, such as nose, mouth and eyes, and then calculate the distance between these features. get the vector feature information that can express each feature face, such as the position of the eyes, the length of the eyebrows, etc., and then calculate the corresponding relationship between each feature. Compared with the known face corresponding feature information in the face database, the best matching face is obtained. The method based on geometric features is consistent with people's understanding of facial features. in addition, only one feature is stored in each face, so it takes up less space; at the same time, this method will not reduce the recognition rate for the changes caused by illumination. and the feature template matching and recognition rate is relatively high. However, the method based on geometric features is also not robust, once the expression and posture change slightly, the recognition effect will be greatly reduced.

A method based on deep learning

With the emergence of deep learning, face recognition technology has made a breakthrough. The latest research results of face recognition show that the facial feature expression obtained by deep learning has important characteristics that manual feature expression does not have. for example, it is moderately sparse, highly selective to face identity and attributes, and robust to local occlusion. These characteristics are obtained naturally through big data training, and no explicit constraints or post-processing are added to the model, which is the main reason why deep learning can be successfully applied to face recognition.

There are seven typical applications of deep learning in face recognition: face recognition based on convolution neural network (CNN), deep nonlinear face shape extraction, robust modeling of face posture based on deep learning, automatic face recognition in constrained environment, and face recognition under video surveillance based on deep learning. Low-resolution face recognition based on deep learning and other face-related information recognition based on deep learning.

Among them, convolution neural network (Convolutional Neural Networks,CNN) is the first learning algorithm to train multi-layer network structure successfully. Face recognition method based on convolution neural network is a machine learning model under deep supervised learning, which can mine local features of data, extract global training features and classification, and its weight sharing structure network makes it more similar to biological neural network. It has been successfully applied in all fields of pattern recognition. CNN optimizes the model structure and ensures a certain displacement invariance by combining the local perception region of the face image space, sharing weights and downsampling in space or time to make full use of the locality of the data itself.

Using the CNN model, the face recognition accuracy of the Deep ID project of the Chinese University of Hong Kong and the Deep Face project of Facebook on LFW database is 97.45% and 97.35% respectively, which is slightly lower than that of human visual recognition (97.5%). After a breakthrough, the DeepID2 project of the Chinese University of Hong Kong raised the recognition rate to 99.15%. Deep ID2 minimizes the intra-class change by learning nonlinear feature transformation, while keeping the distance between face images with different identities constant, which exceeds the recognition rate of all the leading deep learning and non-deep learning algorithms in LFW database and the recognition rate of human beings in this database. Deep learning has become a research hotspot in computer vision, new algorithms and new directions about deep learning continue to emerge, and the performance of deep learning algorithms gradually surpasses shallow learning algorithms in some major international evaluation competitions.

Method based on support Vector Machine

The application of support vector machine (SVM) to face recognition originates from the theory of statistics, which focuses on how to construct effective learning machines and solve the problem of pattern classification. Its characteristic is to transform the image space and classify it in other spaces.

The structure of support vector machine is relatively simple, and it can achieve global optimization. Therefore, support vector machine has been widely used in the field of face recognition. However, this method also has the same deficiency as the neural network method, that is, it needs a lot of storage space, and the training speed is relatively slow.

Other comprehensive methods

The above several commonly used face recognition methods, it is not difficult to see that each recognition method can not achieve a perfect recognition rate and faster recognition speed, and has its own advantages and disadvantages, therefore, now many researchers prefer to use a variety of recognition methods, take the advantages of various recognition methods, comprehensive use, in order to achieve higher recognition rate and recognition effect.

Three classical algorithms for face recognition

Eigenface method (Eigenface)

Face recognition technology is a recently developed method for face or general rigid body recognition and other face processing. The method of using eigenfaces for face recognition was first proposed by Sirovich and Kirby (1987) ("Low- dimensional procedure for the characterization of human faces"), and used by Matthew Turk and Alex Pentland for face classification ("Eigenfaces for recognition"). Firstly, a batch of face images are converted into a set of feature vectors, called "Eigenfaces", that is, "eigenfaces", which are the basic components of the initial training image set. The process of recognition is to project a new image to the feature face subspace, and determine and identify it by the position of its projection point in the subspace and the length of the projection line.

After the image is transformed to another space, the images of the same category will come together, and the convergence of different types of images is relatively far away. It is very difficult for different types of images in the original pixel space to be segmented by simple lines or planes. Transform to another space, they can be well separated. The spatial transformation method selected by Eigenfaces is PCA (Principal component Analysis). The main components of face distribution are obtained by PCA. The concrete realization is to decompose the covariance matrix of all face images in the training set by eigenvalue decomposition to get the corresponding eigenvectors, which are "eigenfaces". Each feature vector or feature face is equivalent to capturing or describing a change or characteristic between faces. This means that each human face can be represented as a linear combination of these characteristic faces.

Local binary mode (Local Binary Patterns,LBP)

Local binary pattern (Local Binary Patterns LBP) is a visual operator used for classification in the field of computer vision. LBP is an operator used to describe the texture features of an image, which was proposed by T.Ojala of the University of Oulu in Finland in 1996 ("A comparative study of texture measures with classification based on featured distributions"). In 2002, T.Ojala et al published another article on LBP ("Multiresolution gray-scale and rotation invariant texture classification with local binary patterns") on PAMI. This article clearly describes the improved LBP features of multi-resolution, grayscale invariant, rotation invariant and equivalent mode. The core idea of LBP is to take the gray value of the central pixel as the threshold and compare it with his field to get the corresponding binary code to represent the local texture features.

LBP is to extract local features as a basis for discrimination. The obvious advantage of LBP method is that it is insensitive to light, but it still does not solve the problem of posture and expression. However, compared with the eigenface method, the recognition rate of LBP has been greatly improved.

Fisherface

Linear discriminant analysis considers category information while reducing dimensionality, which was invented by statistician Sir R. A. Fisher1936 ("The use of multiple measurements in taxonomic problems"). In order to find a way of feature combination, it can achieve the maximum inter-class dispersion and the minimum intra-class dispersion. The idea is simple: in a low-dimensional representation, the same classes should be tightly grouped together, while different categories should be as far apart as possible. In 1997, Belhumer successfully applied Fisher discriminant criterion to face classification and proposed a Fisherface method based on linear discriminant analysis ("Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection").

Classical thesis

Sirovich,L.,&Kirby,M. (1987). Low-dimensional procedure for the characterization of human faces.Josa a, 4 (3), 519-524. It has been proved that any special face can be represented by a coordinate system called Eigenpictures. Eigenpictures is the eigenfunction of the average covariance of the facial set.

Turk,M.,&Pentland,A. (1991). Eigenfaces for recognition.Journal of cognitive neuroscience, 3 (1), 71-86. A near real-time computer system is developed, which can locate and track a person's head, and then identify a person by comparing facial features with those of a known individual. In this method, the face recognition problem is regarded as a two-dimensional recognition problem. The process of recognition is to project a new image to the feature face subspace, which captures significant changes between known facial images. Important features are called eigenfaces because they are the eigenvectors of the face set.

Ojala,T.,Pietik ä inen,M.,&Harwood,D. (1996). A comparative study of texture measures with classification based on featured distributions.Pattern recognition,29 (1), 51-59. In this paper, different graphic textures are compared, and a LBP operator is proposed to describe image texture features.

Ojala,T.,Pietikainen,M.,&Maenpaa,T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns.IEEE Transactions on pattern analysis and machine intelligence,24 (7). In this paper, a very simple and effective grayscale and rotation invariant texture classification method is proposed, which is based on local binary patterns and non-parametric discrimination of sample and prototype distribution. This method has the characteristics of robust gray change and simple calculation.

Fisher,R.A. (1936). The use of multiple measurements in taxonomic problems.Annals of eugenics,7 (2), 179188 A method of feature combination is found to achieve the maximum inter-class dispersion and the minimum intra-class dispersion. The solution is: in the low-dimensional representation, the same classes should be closely grouped together, while different categories should be as far apart as possible.

Belhumeur,P.N.,Hespanha,J.P.,&Kriegman,D.J. (1997) Eigenfaces

Vs.fisherfaces:Recognition using class specific linear projection. Yale University New Haven United States. Facial projection based on Fisher linear discrimination can produce well-separated classes in low-dimensional subspace, even in the case of large changes in illumination and facial expressions. Extensive experimental results show that the error rate of the proposed "Fisherface" method is lower than that of the eigenface technology tested by Harvard and Yale face databases.

Commonly used face database

This paper mainly introduces the following common face databases:

ERET face database

Http://www.nist.gov/itl/iad/ig/colorferet.cfm

Created by the FERET project, this image set contains a large number of face images, and there is only one face in each image. In this episode, the photos of the same person have different expressions, lighting, posture and age changes. With more than 10,000 face images with multi-pose and illumination, it is one of the most widely used face databases in the field of face recognition. Most of them are westerners, and the change of face image contained by each person is relatively simple.

CMU Multi-PIE face database

Http://www.flintbox.com/public/project/4742/

Founded by Carnegie Mellon University in the United States. The so-called "PIE" is the abbreviation of Pose, Illumination and Expression. CMU Multi-PIE face database is developed on the basis of CMU-PIE face database. Contains more than 75000 facial images of multiple postures, lighting and expressions of 337 volunteers. The pose and illumination change images are also collected under strict control, which has gradually become an important test set in the field of face recognition.

YALE face Database (Yale University, USA)

Http://cvc.cs.yale.edu/cvc/projects/yalefaces/yalefaces.html

Created by the Center for Computational Vision and Control at Yale University, it contains 165 pictures of 15 volunteers, including changes in light, expression and posture.

Ten samples were collected from volunteers in the Yale face database. Compared with the samples collected from each object in the ORL face database Yale database, there were more obvious changes in illumination, expression, posture and occlusion.

YALE face database B

Ttps://computervisiononline.com/dataset/1105138686

It contains 5850 images of 10 individuals in 9 postures and 64 light conditions. The images of attitude and illumination changes are collected under strict control, which are mainly used for modeling and analysis of lighting and attitude problems. Due to the small number of people collected, the further application of the database is greatly limited.

MIT face database

Created by the Media Lab of the Massachusetts Institute of Technology, it contains 2592 facial images of 16 volunteers in different postures (27 photos each), light and size.

ORL face database

Https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

Created by the AT&T Lab at the University of Cambridge, England, there are 400 facial images of 40 people, some of which include changes in posture, expressions and facial accessories. The face database is often used in the early stage of face recognition research, but because there are few change patterns, the recognition rate of most systems can reach more than 90%, so the value of further use is not great.

Each collection object in the ORL face database contains 10 normalized grayscale images with a size of 92 × 112 and a black background. Among them, the facial expressions and details of the subjects changed, such as smiling or not smiling, eyes open or closed, glasses and so on. The posture of different face samples also changed, and their depth rotation and plane rotation could reach 20 degrees.

BioID face database

Https://www.bioid.com/facedb/

There are 1521 grayscale facial images in a variety of lights and complex backgrounds, and the eye position has been manually marked.

UMIST image set

Founded by the University of Manchester in England. Including 20 people a total of 564 images, each person has different angles, different postures of multiple images.

Age recognition data set IMDB-WIKI

Https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/

Contains 524230 celebrity data images crawled from IMDB and Wikipedia. In this paper, a novel age algorithm which turns regression into classification is applied. The essence is that after 101 categories between 0-100, the score is multiplied by 0-100, and the final result is summed up to get the age of final recognition.

Technical talents

1. General situation of scholars

Based on the academic papers published in the international journal conference, AMiner calculates and analyzes the full TOP1000 scholars in the field of face recognition, and draws the global distribution map of scholars in this field. From a global point of view, the United States is the country with the largest number of face recognition scholars, and has an absolute advantage in the field of face recognition, followed by the United Kingdom, ranking second, and China ranking third in the world. Canada, Germany and Japan and other countries also gathered some talents.

Global distribution map of ▲ face recognition scholar TOP1000

National ranking of ▲ face recognition experts

Statistics of h-index, a global scholar of ▲ face recognition

H-index: an internationally recognized index that can accurately reflect a scholar's academic achievement. The calculation method is that at most h papers of this scholar have been cited at least h times.

The average h-index of global face recognition scholars is 48, with the largest number of scholars with a h-index index between 20 and 40, accounting for 33%; scholars with a h-index index between 40 and 60 are at loggerheads with those greater than 60, with the former being 27% and the latter 28%; and scholars with a h-index index less than or equal to 10 are the least, accounting for only 2%.

▲ face recognition Global Talent Migration Map

AMiner selected the top 1000 experts and scholars in the field of face recognition to analyze their migration paths. As can be seen from the above picture, the brain drain and introduction of face recognition in different countries are slightly different, among which the United States is the largest talent flow country in the face recognition field, with a large lead in talent input and output, and from the data, the inflow of talent is slightly larger than the outflow. Countries such as the UK, China, Germany, Canada and Australia followed, with slight brain drain in the UK, China and Australia.

According to the papers cited in the last five years of the authoritative academic conference on face and gesture recognition (IEEE International Conference on Automatic Face and Gesture Recognition,FG) around the world, the research calculated the top ten face recognition experts of citation and h-index, and intercepted some leading scholars to introduce them.

The relevant scholars in the top 10 of Citation are as follows:

▲ Citation Top Ten face recognition experts

The relevant scholars in the top 12 of h-index are as follows:

▲ h-index Top Ten face recognition experts

2. Talents at home and abroad

The report lists 6 experts and scholars from all over the world and 5 domestic experts, as detailed in the annex to this internal reference.

Application field

From the application point of view, face recognition is widely used, which can be used in automatic access control system, identification of identity documents, bank ATM machines, family security and other fields. Specifically, the main points are:

1. Public security: public security criminal investigation, criminal identification, border security inspection

2. Information security: login of computers and networks, encryption and decryption of files

3. Government functions: e-government, household registration management, social welfare and insurance

4. Commercial enterprises: e-commerce, electronic money and payment, attendance, marketing

5. Entry and exit of places: access control and access management of military confidential departments and financial institutions.

Access control face recognition

With the improvement of people's living standards, people pay more attention to the safety of the home environment, and the concept of security is constantly strengthened. with the improvement of this demand, intelligent access control system arises at the historic moment. More and more enterprises, shops and families have installed a variety of access control systems.

At present, the most commonly used access control systems are video access control, password access control, radio frequency access control or fingerprint access control and so on. Among them, the video access control simply transmits the video information to the user, and there is not much intelligence, which is essentially inseparable from "civil air defense", and the home security can not be guaranteed absolutely when the user is not present; the biggest hard wound of the password access control is that the password is easy to forget and easy to crack; the disadvantage of the radio frequency access control is that the "identification card does not recognize people", and the radio frequency card is easy to be lost and stolen by others. In addition, the security risk of fingerprint access control is that fingerprints are easy to copy. Therefore, the above access control systems provided in the prior art have the problem of low security corresponding to the reasons. Installed a face recognition system, as long as a face in front of the camera can easily enter and leave the community, the real realization of "face card". The biometric access control system does not need to carry verification media, and the verification features are unique and safe. At present, it is widely used in places with high level of confidentiality, such as research institutes, banks and so on.

Marketing

Facial recognition technology has two main applications in marketing: first, it can identify a person's basic personal information, such as gender, approximate age, and what they have seen and how long they have seen it. Outdoor advertising companies, such as Val Morgan Outdoor (VMO), are using facial recognition technology to collect consumer data. Second, the technology can be used to identify known individuals, such as thieves, or members who have joined the system. This application has attracted the attention of some service providers and retailers.

In addition, facial recognition technology can also improve the effectiveness of advertising and allow advertisers to respond to the performance of consumers in a timely manner. VMO has launched a measurement tool, DART, which can see the direction and duration of consumers' eyes in real time to determine how much they are paying attention to an ad. The next generation of DART will also include more demographic information, including, in addition to age, consumers' emotions when looking at a digital signage.

commercial bank

Using face recognition technology to guard against network risks: for magnetic stripe bank cards widely used in our country, although the technology is mature and standardized, the production technology is not complicated, and the standard of bank magnetic stripe track is an open secret. Only a computer and a magnetic stripe reader can successfully "clone" a bank card. In addition, the business card printing machine sales management is not strict enough. Cases of fraudulent use of bank cards by lawbreakers occur from time to time, and the main means is to "clone" or embezzle bank cards in various ways. At present, various commercial banks have also taken some technical measures to prevent counterfeiting and cloning cards, such as using CVV (Check Value Verify) technology to generate a set of check values while generating card magnetic strip information, which is related to the characteristics of each card, so as to achieve the function of invalid replication. Although a variety of measures have been taken, the inherent defects of the magnetic stripe card have seriously threatened the interests of customers. For these bank network security issues, we can use face recognition technology to prevent network risks. Face recognition technology is to capture the human face region through the image acquisition equipment, and then match the captured human face with the face in the database, so as to complete the task of identity recognition. The use of face recognition technology to accurately identify the true identity of cardholders to ensure the financial security of cardholders. In addition, we can further target lawbreakers through face recognition technology, which is helpful for public security organs to solve cases quickly.

The application of face recognition technology in the treatment of counterfeit banknotes: at present, the main problems in self-service equipment of commercial banks in China: first, the installation of some self-service equipment does not meet the requirements. Part of the self-service equipment installation of commercial banks did not strengthen and connect the equipment with the ground in accordance with the requirements of the public security department; some electrical environments did not meet the requirements: some did not set up a 110 continuous alarm or there was no visual monitoring alarm, some surveillance videos were not clear enough, the storage time of surveillance videos did not meet the prescribed requirements, and the equipment was seriously damaged. The second is the software design defect of self-service equipment. In particular, some domestic equipment software design is not reasonable, software changes are arbitrary, there are loopholes, resulting in a greater possibility of misaccounting. Third, there is no equipment to identify counterfeit banknotes in the bank's ATM machine. Due to the problems of self-service equipment in China's commercial banks, counterfeit banknotes emerge in endlessly at present. As there is no counterfeit banknote identification equipment in the bank's ATM machine, it is only identified before the clearing staff put in the cash, this measure is not perfect and is easy to cause disputes between the bank and the cardholder. Even if the cash deposit machine (CRS) has the function of identifying counterfeit banknotes, it is often used by lawbreakers because of the lag in feature extraction of counterfeit banknotes. Lawbreakers first deposit counterfeit banknotes, and then immediately withdraw real banknotes from the counter or other self-service equipment, in order to seek illegal benefits.

Future trend

Generally speaking, the trend of face recognition includes the following aspects.

1. The combination of machine recognition and manual recognition

At present, when some mainstream face recognition companies in the market quote well-known face image databases at home and abroad for testing, the accuracy of face recognition can generally reach more than 95%, and the speed of accurate face recognition is also very fast. this also provides a strong practical proof for the practical application of face recognition technology from the side.

However, in real life, everyone's face is not motionless relative to the camera, on the contrary, it is in a state of high-speed movement. the face image captured by the camera will show a completely different appearance because of the face posture, expression, light, decoration and so on, and it is very likely that the collected face image is not clear, incomplete and the features of the key parts are not obvious. At this time, the face recognition system may not be able to achieve fast and accurate face recognition.

Therefore, after setting a certain value of face image similarity, the face recognition company system will prompt the face images that are higher than the similarity value, and then manually screen them one by one. Only by using the combination of machine recognition and manual recognition can we maximize the accurate recognition of face images.

2. Wide application of 3D face recognition technology.

Whether it is the face image that has been saved in the mainstream face image database, or the real-time face image captured by the camera at the street intersection, most of them are actually a 2D face image. In fact, 2D face image itself has inherent defects, that is, it can not express face image information in depth, and it is particularly vulnerable to lighting, posture, expression and other factors when shooting. As for the face, many key parts of the face, including eyes, nose, ears, chin and so on, are not on the same plane, and the face naturally has a three-dimensional effect. Shooting 2D face images can not fully reflect all the key features of the face.

In 2017, iPhone X, a smartphone equipped with many of the latest cutting-edge technologies, attracted a great deal of attention in the industry. The most striking of these is a cool techs: 3D face unlocking function, or Face ID, a new way of identity authentication. When unlocking, users only need to look at the phone, Face ID can realize face recognition unlocking.

▲ Apple's layout in the field of 3D vision

The addition of 3D facial recognition to Apple's iPhone X is not on a whim, as it has been laid out in 3D vision since 2010. Especially in 2013, Apple bought PrimeSense, an Israeli 3D vision company, for $345 million. The acquisition is one of the biggest acquisitions in Apple's history. Since then, Apple has also invested in a number of 3D vision and face recognition companies.

In addition, Face ID can also be used for Apple pay and third-party applications. Apple, for example, uses Face ID to upgrade its emoji function, which allows Face ID to create a 3D facial Animojis using facial expressions and animations to express emotions, but this feature is currently available only in Apple's own iMessage. This direct "facial scanning" method brings users a more real human-computer interaction experience.

3. Wide application of face recognition technology based on deep learning.

At present, most of the mainstream face recognition technologies are aimed at lightweight face image databases, but they are not mature for the billion-level face image databases that are completely predictable in the future. therefore, we need to focus on the face recognition technology based on deep learning.

In a popular sense, at present, there are as many as 1.3 billion people in China, and it is foreseeable that a powerful face recognition company will take the lead in establishing a unified face image database covering the whole country in the near future. then the capacity of the face image database may reach billions or even tens of billions. At this time, there may be a large number of faces with similar representations and similar key feature points, if there is no face recognition technology based on deep learning to establish more complex and diversified face models, so it will be difficult to achieve accurate and fast face recognition.

4. The substantial improvement of face image database.

It is also inevitable to establish a face image database with excellent diversity and versatility. Compared with the database cited by mainstream face recognition companies at present, its substantial improvement is mainly reflected in the following aspects: first, the improvement of the magnitude of face image database will be promoted from hundreds of millions at present to billions or even tens of billions in the future. Second, the improvement of the quality level, will be promoted from the mainstream 2D face image to a variety of key feature points more obvious and clear 3D face image; third, the improvement of the type of face image, will collect each person's face image under different posture, expression, light, decoration and so on, in order to enrich each person's face representation and achieve accurate face recognition.

The editor believes that face recognition is a field with rapid development and wide application of AI technology, and it has a wide range of applications. At this year's Anbo Expo, face recognition and dynamic capture technology has become a "standard" for almost every exhibitor. With the R & D investment of national scientific research institutions, enterprise research on technology, market promotion and so on, face recognition will usher in a better wave of development. In the future, face recognition may become the mainstream of effective identity recognition, at that time, face recognition will not be a new word.

Https://www.toutiao.com/a6722752828722381326/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.