Shulou(Shulou.com)06/02 Report--

Introduction:

Unwittingly the Spring Festival holiday is coming, in this year's Spring Festival topic, it is not difficult to find that in addition to the seven aunts' cordial greetings to these conventional topics, traces of artificial intelligence can also be seen everywhere. AI is showing us a future to look forward to with a new look, such as today, we can use PaddlePaddle to try to write a pair of smart Spring Festival couplets. It has become a traditional custom to paste Spring Festival couplets during the Spring Festival, but there are not many choices in shopping malls, and many people want to do it in person, but they do not know much about the level of confrontation. Can you use artificial intelligence to help us write Spring Festival couplets? During the Spring Festival this year, Baidu, NetEase and CCTV launched "Intelligent Spring Festival couplets H5". As long as they are given 2-4 Chinese characters, it can "write" a very ornamental Tibetan Spring Festival couplets.

What makes the machine have the skill of Spring Festival couplets? Through the intelligent Spring Festival couplet H5, "facial scanning" can be realized in a few seconds, while behind this is a series of "indescribable" advanced technology. In the aspect of vision, face detection, attribute analysis, face fusion and other technologies are mainly applied, which can detect the face in the picture and analyze the corresponding age, sex, face value, smile index, whether to wear glasses and other information. and summarize the characteristics of the face through a word; and then fuse the face in the picture with the face in the specified template image to get a new picture. The difficulty of these techniques is that they need to detect faces from various angles and extract facial features in order to generate new images that are similar to primitive faces but also similar to template faces without any sense of conflict.

Secondly, in the aspect of natural language processing (NLP), the advanced neural network machine translation technology based on Baidu deep learning framework PaddlePaddle can transform the creation of Spring Festival couplets into "translation". The difference is that translation establishes a connection between two languages, while Spring Festival couplets establish a connection in the same language. Of course, some people may be hit by "colored eggs". These are not Spring Festival couplets written by AI. For example, if you scan your face and get the key word "play essence", you will get a pair of "flow physique is born to play, and the protagonist's halo can not cover you", criticizing the Spring Festival couplets of "over-indulgence in drama", which is so domineering and full of arrogance, but the stem is buried specially for you by manual work. Such Spring Festival couplets appear in moments, there is no doubt that the number of likes will be crazy UpUpUp.

Let's try it! As a deep learning framework, PaddlePaddle not only supports the development and research of deep learning algorithms, but also collects a variety of leading image classification and natural language processing algorithms in the official model library (https://github.com/PaddlePaddle/models)). Through these algorithms, we can easily achieve a variety of fun and interesting functions, such as: intelligent Spring Festival couplets.

Intelligent Spring Festival couplets can be played in a variety of ways. According to the keywords entered by the user, a pair of couplets can be generated to achieve customized Spring Festival couplets. For example, the user enters "good luck". Baidu's artificial intelligence Spring Festival couplets will create couplets that are "a year of good luck full of splendor, thousands of people with one heart and one mind" and horizontally criticized as "Spring full Garden". Users enter their own names, and the intelligent Spring Festival couplets generation program can hide their names in the generated Spring Festival couplets to form highly personalized Spring Festival couplets that can be sent to their friends; even users do not need to enter keywords, they just need to take a photo, and then they can generate an illustrated Spring Festival couplet based on this photo. So how do you do all these functions? Here we show you how to use PaddlePaddle to develop a set of exclusive intelligent Spring Festival couplets generation system.

First, our system needs to be able to understand the content of the image, such as: what exactly is the user's input?

This is a typical problem of image classification. Image classification is to distinguish different types of images according to the semantic information of images. It is an important basic problem in computer vision. It is the basis of other high-level vision tasks, such as object detection, image segmentation, object tracking, behavior analysis, face recognition and so on. In the era of deep learning, the accuracy of image classification has been greatly improved. PaddlePaddle has opened up commonly used models on the classical data set ImageNet, including AlexNet, VGG, GoogLeNet, ResNet, Inception-v4, MobileNet, DPN (Dual Path Network) and SE-ResNeXt models. At the same time, open source training models are easy for users to download and use. Based on these image classification algorithms, we can know the categories entered by users, and with similar techniques, we can also know some more specific attributes, such as gender and age for a photo of a face. With these features and attributes of the picture, we can expand the keywords. For example: for young women can associate some key words "magnificent", "beautiful", "dignified" and so on, for children can associate the key words "lively and lovely", "witty", "brave" and so on.

Second, after getting these keywords, what is the next task?

According to a keyword, automatically generate a related Spring Festival couplet. The generation process of Spring Festival couplets can be divided into two steps. The first step is to generate a pair of couplets from keywords. And then generate a pair of lower couplings according to the upper couplings. So how do you implement these two generation steps? We find that this task is very similar to machine translation and can be implemented with similar techniques. Machine translation (Machine Translation, MT) is a technology that uses computers to achieve translation between different languages. The language to be translated is usually called the source language (Source Language), and the resulting language translated is called the target language (Target Language). Machine translation, which is the process of transforming from source language to target language, is one of the important research fields of natural language processing. PaddlePaddle's model library (PaddlePaddle/models) provides the implementation of two machine translation algorithms, one is the classical LSTM-based Seq2Seq model, and the other is the latest Attention-based Transformer model. Compared with the machine translation task, the keywords input by the intelligent Spring Festival couplets can be regarded as the source language sentences in the machine translation, and then the couplets generated according to the keywords can be regarded as the target language translation in the machine translation. Compared with the cyclic neural networks (Recurrent Neural Network, RNN) widely used in the previous Seq2Seq model, using (Self) Attention to transform input sequences to output sequences has the following advantages: lower computational complexity, higher computational concurrency, and easier to learn long-distance dependencies. We recommend using the Transformer model with better translation effect, and usually Transfomer can get better translation effect than Seq2Seq.

Figure 2. Multi-Head Attention, the core component of Transformer model

Def scaled_dot_product_attention (Q, k, v, attn_bias, d_key, dropout_rate):

"

Scaled Dot-Product Attention

"

Scaled_q = layers.scale (Xunq, scale=d_key**-0.5)

Product = layers.matmul (x=scaled_q, yardk, transpose_y=True)

If attn_bias:

Product + = attn_bias

Weights = layers.softmax (product)

If dropout_rate:

Weights = layers.dropout (

Weights

Dropout_prob=dropout_rate

Seed=ModelHyperParams.dropout_seed

Is_test=False)

Out = layers.matmul (weights, v)

Return out

Figure 3. PaddlePaddle code implementation of Scaled Dot-Product Attention

Third, after choosing the generation algorithm to be used, how can the machine learn to write Spring Festival couplets?

Next, we need to prepare training data for the system, the so-called "familiar with three hundred Tang poems", for the machine, you need to see a large number of Spring Festival couplets in order to learn the rules of words and words in the Spring Festival couplets. We can go to the Internet to find a large number of Spring Festival couplets data, such as: "firecrackers pass auspicious language", "La Mei newspaper New year" and so on, and then use them as training data, usually need tens of thousands of pieces. With these data, we extract keywords from the upper couplet, "firecracker"-> "firecracker chanting auspicious words", and train a generation model from keyword to upper couplet; then we use "firecracker lucky words" and "La Mei to report the Spring Festival" to train a generation model from upper couplet to lower couplet.

After preparing the training data, you can start the training of the Transformer model, you can refer to the commands in PaddlePaddle/models/neural_machine_translation/transformer. This is an example of English-German translation. When generating keywords to the upper couplet, the training data need to be replaced with the data corresponding to the "Spring Festival couplet upper couplet"; when doing the upper couplet to the lower couplet generation, the training data need to be replaced with the "Spring Festival couplet upper couplet" to the corresponding "Spring Festival couplet lower couplet" data.

Python-u train.py\

-- src_vocab_fpath gen_data/wmt16_ende_data_bpe/vocab_all.bpe.32000\

-- trg_vocab_fpath gen_data/wmt16_ende_data_bpe/vocab_all.bpe.32000\

-- special_token'\

-- train_file_pattern gen_data/wmt16_ende_data_bpe/train.tok.clean.bpe.32000.en-de\

-token_delimiter'\

-- use_token_batch True\

-- batch_size 4096\

-- sort_type pool\

-- pool_size 200000

After the completion of the training, you can get a generation model of keywords to upper link, and a generation model from upper link to lower link. Pay attention to the generation process, you need to perform the prediction process of Transformer twice, first enter a keyword to generate the upper link, and then input the generated upper link to regenerate into a lower link. For specific generation, refer to the following command.

Python-u infer.py\

-- src_vocab_fpath gen_data/wmt16_ende_data_bpe/vocab_all.bpe.32000\

-- trg_vocab_fpath gen_data/wmt16_ende_data_bpe/vocab_all.bpe.32000\

-- special_token'\

-- test_file_pattern gen_data/wmt16_ende_data_bpe/newstest2016.tok.bpe.32000.en-de\

-token_delimiter'\

-- batch_size 32\

Model_path trained_models/iter_100000.infer.model\

Beam_size 4\

Max_out_len 255

According to some of the technologies provided by the official model library of PaddlePaddle above, a fun intelligent Spring Festival couplet system can be implemented. Of course, we can also do a lot of interesting expansion, for example, we can add some ancient poems as training corpus to make the generated Spring Festival couplets more rich in content; Tibetan Spring Festival couplets, the key words are separated by words, through Grid Beam Search technology, to ensure that the generated keywords will appear in the specific location of the Spring Festival couplets.

Interested friends, you can collect Spring Festival couplets training data, you can try to achieve a fun intelligent Spring Festival couplets system.

Github address:

Https://github.com/PaddlePaddle/models/blob/develop/fluid/PaddleNLP/neural_machine_translation/transformer/README_cn.md

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.