Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about the Deepfake detection based on RNN network. The article is rich in content and analyzed and described from a professional point of view. I hope you can get something after reading this article.

0. Brief introduction

Today, we will introduce a detection Deepfakes framework based on CNN+RNN structure.

1. Preface

Most of the work of detecting fake faces is done on pictures, but there are few detection methods for deepfake videos. In this work, we propose a processing method based on time series, which is used to detect Deepfake video. We use CNN to extract frame-level high-dimensional features, and use these high-dimensional features to train RNN. We have shown that a simple architecture can also achieve good results in detection tasks.

2. Deepfake video generation

Deep learning method can be used for image compression performance, the most commonly used is self-codec (AutoEncoder-Decoder). The self-encoder can compress the image into a high-dimensional feature by minimizing the loss function, which is more efficient than the existing compression methods.

The encoder maps high-dimensional features back to the image, as shown in Figure2.

For Deepfakes to take effect, the key is to encode two potential faces to the same feature.

We train two self-decoders respectively by sharing one self-encoder weight.

When we replace a face, we first encode the input image, and then use the target face decoder to decode it.

However, it is difficult for self-encoder and decoder to generate human face under complex conditions such as different camera angles and different lighting. Various conditions lead to the visual inconsistency between the face replacement part and the background, and this frame-level scene inconsistency will be the first feature of our method.

The second feature comes from the fact that the face detector is used to replace the face, while the self-codec only pays attention to the face part and pays little attention to the rest of the background information, so the final fusion is prone to boundary effect.

The third feature is that the self-codec is independent of each frame, and it does not take into account the effect of generating face pictures before and after the frame. The most prominent is the inconsistency of the light source between frames, which leads to the flicker of the false face, which is very suitable to use CNN for pixel-level detection.

3. Overall architecture

So far, we have determined the infrastructure, the frame features are extracted by CNN, and the time series analysis is carried out by LSTM. Our network also contains two full connection layers plus Dropout to prevent the model from being over-fitted.

We use the pre-trained InceptionV3 network as the CNN structure to extract 2048 features from the input images.

The extracted 2048 features are sent to the LSTM unit, and then the full connection layer of the 512 unit is connected with the Dropout of 0.5 probability. finally, the probability is calculated by softmax and the final two classification is made.

4. According to the training strategy, the feature image of each channel is extracted and scaled to 299x299. The length of each video frame is 20-40-80, the optimizer chooses Adam, and the learning rate is 1e, 5pm, 1e-65. Experimental results

The final results show that adding frame sequence can improve the accuracy to a certain extent, but the improvement is not very large.

This is how the Deepfake detection based on RNN network is shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.