Shulou(Shulou.com)06/01 Report--

Keras how to achieve image style conversion-, many novices are not very clear about this, in order to help you solve this problem, the following small series will explain in detail for everyone, there are people who need this to learn, I hope you can gain something.

preface

When it comes to deep learning applications, the first thing that comes to mind is the image style conversion of Prisma App. Since interested in direct work, read the paper, a little knowledge; read other people's source code, only to roughly understand the specific realization, but also marvel at other people's fantastic ideas.

The thesis can be referred to [A Neural Algorithm of Artistic Style], and there is also a Chinese version online.

http://arxiv.org/abs/1508.06576

Tools used: py34, keras1.1.2, theano0.8.2, GeForce GT 740M (CNMeM is disabled, cuDNN not available).

Principle 1. general flow

The implementation process is as follows, you can see that there are a total of 5 layers, this experiment using vgg16 model implementation.

As above, a has an alias conv1_1, b is conv2_1, and so on, c, d, e correspond to conv3_1, conv4_1, conv5_1; the input image has style image and content image, and the output is synthetic image, and then the synthetic image is used as guidance for training, but the training object is not like the ordinary neural network to train the weight w and bias term b, but to train the pixels on the synthetic image to achieve the effect of reducing the loss function continuously. The paper uses a random noise pixel map as the initial composite map, but it is faster to use the original image.

2. Content Loss Function- Content Loss

Here is the definition of the content loss function.

l represents the feature representation of layer l, p is the original picture, and x is the generated picture. The meaning of the formula is that for each layer, the square difference of the one-to-one correspondence between the original image generation feature map and the generated image feature map is made.

3. Style loss function- style loss

Before defining the style loss function, first define a Gram matrix.

F is the feature map of the generated picture. The meaning of the above formula: The value of the ith row and the jth column of Gram is equal to the multiplication and summation of the ith characteristic graph and the jth characteristic graph of the generated graph in the lth layer.

The above is the style loss function, Nl is the number of feature maps that generate the map, and Ml is the width of the image multiplied by the height. A refers to the style picture and x refers to the generated picture. G is the Gram matrix of the generator graph, A is the Gram matrix of the style graph, wl is the weight.

4. total loss

The total loss function is given by an alpha to beta ratio of 1*10^-3 or less.

Code explanation 1. Image preprocessing and restoration

2. content loss

3. style loss

results

It can be seen that the effect has improved every generation, because of their own graphics card slag, running a generation is estimated to take 1.5 hours, when they test a total of 14 hours, but here is a trick, that is, you can continue to input the previous generation of pictures, so that anything can stop halfway. Next time just take the last output picture as input.

The vgg16 model loads the weights of the original project.

Specific project code can be seen on githua code, weight files and test pictures, because the midway has been modified, there may be some places need to be changed, but the code is relatively simple, it is estimated that soon you can find the problem.

Did reading the above help you? If you still want to have further understanding of related knowledge or read more related articles, please pay attention to the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.