Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to use MNIST data sets to verify deformable image registration based on deep learning. Many people may not know much about it. In order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

Guide reading

A simple image registration scheme is verified with MNIST data set, and the effect is good.

Deep-learning-based 2D Deformable Image Registration with MNIST

Sarath Chandra

Https://medium.com/@sarathchandra.knv31/deep-learning-based-2d-deformable-image-registration-with-mnist-2db3b6ee1426

I was admitted by Google Summer of Code 2020 to do research on image registration based on deep learning. My project link-MRI Registration using Deep Learning and Implementation of Thin-Plate Splines

What is image registration?

Image registration is the process of finding the conversion of aligning one image to another. Typically, the input to this process is two images: a reference image, also known as a still image, and a moving image that will be aligned with the still image. The goal here is to distort the moving image to match the still image.

How to distort the image?

Given a moving image and a still image as input, the convolutional encoder-decoder network calculates the pixel deformation between the two images. This deformation field, also known as the registration field, gives a new sampling position in the moving image. By sampling the moving images of these positions, the transformed images are obtained. To put it simply, we just rearrange the pixels in the moving image until it matches the still image as much as possible. The frame is shown in the following figure.

By training the codec network, a dense deformation field is output, which is used by the sampler to match the moving image with the still image.

Sampling points do not necessarily map to integer positions in a moving image. So some interpolation techniques are needed when the point is a fraction. In addition, in order for the entire framework to be end-to-end trainable, the sampling block also needs to be differentiable. You can use "Spatial transformer networks".

As described in "Spatial transformer networks", I use bilinear interpolation, which is differentiable and can be written as a pure tensor stream function. In bilinear interpolation, the value at the fractional position is the weighted sum of the values at the four nearest integer positions.

The network is trained by optimizing the similarity measure between the transformed image and the still image. Once the training is completed, the network can predict the optimal registration domain at one time, which is different from the traditional algorithm, which needs to optimize each pair of new registration numerically, so it takes a longer time.

Training network

The MNIST dataset is filtered to retain only one class of images, while the still images are randomly selected from the test set of the filtered dataset. The network uses similarity measure for training, which is an index to measure the degree of similarity / dissimilarity between two images. Some examples of metrics include mean square error (MSE) and normalized cross correlation (NCC). Because the cross-correlation loss is robust to the intensity change, the cross-correlation loss is used. It is the dot product of two normalized images. Mathematically, it is:

S and M represent still images and moving images, respectively. The subscript mean and std represent the mean and standard deviation of the image, respectively. Sum all pixels in the image. This training takes about 5 minutes on the Tesla K80 GPU and about 10 minutes on the CPU (i5-8250U).

Sampling output

After reading the above, do you have any further understanding of how to use MNIST datasets to verify deformable image registration based on deep learning? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.