Shulou(Shulou.com)06/03 Report--

Editor to share with you the weights in Tensorflow and feature map visualization example analysis, I believe that most people do not know much, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

1. Convolution knowledge supplement

In order to explain the code later, here is a brief introduction to some of the knowledge of convolution. There is a lot of information on the Internet about how to slide the convolution kernel on a channel of the image. I believe that readers who have some understanding of convolution neural network should be relatively clear. I will not repeat it in this article. This paper mainly introduces how a group of convolution kernels can calculate a set of feature map on an image.

Take the first set of feature map from the original image through the first convolution layer as an example (from the obtained feature map to the later feature map is the same), assuming that the first group of feature map has 64, then this group of feature map can also be regarded as an image, except that its number of channels is 64, while the general sense of the image is RGB3 channels. In order to obtain the first set of feature map, we need 64 convolution kernels, each of which is a k x k x 3 matrix, where k is the size of the convolution kernel (assuming a square convolution kernel), and 3 corresponds to the number of channels of the input image. Below I will show a simple rough diagram of the image through a convolution kernel convolution to get a feature map process.

As shown in the figure, you can actually think of each channel as a convolution kernel (not quite accurate, just for a moment) and each channel of the image for convolution operation, and then add it position by position to get a feature map.

Then deconvolution an image with a set of (64) convolution kernels and get 64 feature map as shown in the following figure, that is, each convolution kernel gets one feature map,64 kernel and gets 64 feature map.

In addition, you can look at this problem from a slightly different perspective, that is, convolution a certain channel of the image with the corresponding channels of 64 convolution cores to get the intermediate results of 64 feature map, and then add the intermediate results of the corresponding 3 channels to get the final feature map, as shown in the following figure:

You can see that this is actually the first picture extended to the case of multi-convolution kernel, the picture is relatively rough, some of the intermediate results and the final result directly use the same subgraph, please pay a little attention to understanding. This is the way to display the convolution core in the following code, that is, the input image shows the corresponding channel of the convolution core channel by channel, rather than all the channels of one convolution core at a time, which may be explained a little bit. Need to pay attention to. It may be easier to understand through the following small picture.

The part of the picture circled in a red box is the weight parameters we show at one time.

two。 Visualization of Network weights and feature map

(1) Visualization of network weight parameters

First, let's introduce the shape of the convolution core in Tensorflow, as shown in the following code:

Weights = tf.Variable (tf.random_normal ([filter_size, filter_size, channels, filter_num]))

The first two dimensions are the height and width of the convolution kernel, and the third dimension is the number of channels in the upper layer of feature map. In the first section (convolution knowledge supplement), I mentioned how many feature map there are in the upper layer (that is, the number of channels), then there must be so many channels corresponding to a convolution kernel. The fourth dimension is the number of convolution cores of the current convolution layer and the number of channels of feature map output from the current layer.

The following is the visualization code of the network weight parameters (convolution core) after I changed:

From _ _ future__ import print_function#import tensorflow as tfimport numpy as npimport matplotlib.pyplot as pltimport matplotlib.cm as cmimport osimport visualize_utilsdef plot_conv_weights (weights, plot_dir, name, channels_all=True, filters_all=True, channels= [0], filters= [0]): "Plots convolutional filters: param weights: numpy array of rank 4: param name: string, name of convolutional layer: param channels_all: boolean, optional: return: nothing Plots are saved on the disk "" w_min = np.min (weights) w_max = np.max (weights) # make a list of channels if all are plotted if channels_all: channels = range (weights.shape [2]) # get number of convolutional filters if filters_all: num_filters = weights.shape [3] filters = range (weights.shape [3]) else: num_filters = len (filters) # get number of grid rows and columns grid_r Grid_c = visualize_utils.get_grid_dim (num_filters) # create figure and axes fig, axes = plt.subplots (min ([grid_r, grid_c]), max ([grid_r, grid_c]) # iterate channels for channel_ID in channels: # iterate filters inside every channel if num_filters = = 1: img = weights [:,:, channel_ID, filters [0]] axes.imshow (img, vmin=w_min, vmax=w_max, interpolation='nearest' Cmap='seismic') # remove any labels from the axes axes.set_xticks ([]) axes.set_yticks ([]) else: for l, ax in enumerate (axes.flat): # get a single filter img = weights [:,:, channel_ID, filters [l]] # put it on the grid ax.imshow (img, vmin=w_min, vmax=w_max, interpolation='nearest' Cmap='seismic') # remove any labels from the axes ax.set_xticks ([]) ax.set_yticks ([]) # save figure plt.savefig (os.path.join (plot_dir,'{}-{} .png '.format (name, channel_ID)), bbox_inches='tight')

The code of the original project displays all the weight parameters or feature map of a certain layer in a grid. If there are too many parameters or feature map, each figure in the displayed result is very small, so it is difficult to see anything useful, as shown in the following figure:

So I made some changes to the code so that it could display any specified filter or feature map.

In the code

W_min = np.min (weights) w_max = np.max (weights)

These two sentences are used for the subsequent display of images, which can be understood by looking at the imshow () function of matplotlib.pyplot.

The next step is to determine whether to display all channel (number of channels) or all filter. If so, it is consistent with the original code. If not, draw the filter specified by the function parameters channels and filters.

The next two sentences of code are used for drawing, we may display multiple sub-images in a picture, the following sentence is to calculate that the large image is divided into several rows and columns (a large image will be divided into square arrays as far as possible, for example, if there are 64 sub-images, then divided into 8 x 8 arrays), code details can be found in the utils of the original project.

Grid_r, grid_c = visualize_utils.get_grid_dim (num_filters)

When actually drawing, if you want to draw a picture, you need to deal with it separately. If you still want to display multiple subgraphs in a large image, do so in the source code, but instead of outputting all of them unfiltered, you can display the filter that we specify. The main thing to get the data is the following code:

Img = weights [:,:, channel_ID, filters [l]]

The rest are drawing-related functions, so this article will not introduce more about drawing.

When using this code to visualize and save the filter, load the model first, then get the parameters we want to visualize, and then call the function directly, as shown below:

With tf.Session (graph=tf.get_default_graph ()) as sess: init_op = tf.group (tf.global_variables_initializer (), tf.local_variables_initializer ()) sess.run (init_op) saver.restore (sess, model_path) with tf.variable_scope ('inference', reuse=True): conv_weights = tf.get_variable (' conv3_1_w'). Eval () visualize.plot_conv_weights (conv_weights, dir_prefix, 'conv3_1')

There is no additional specification for filter here, and I will give an example in the visualization of feature map.

(2) feature map visualization

In fact, the visualization of feature map is very similar to filter, with only slight differences. Or paste the complete code first.

Def plot_conv_output (conv_img, plot_dir, name, filters_all=True, filters= [0]): w_min = np.min (conv_img) w_max = np.max (conv_img) # get number of convolutional filters if filters_all: num_filters = conv_img.shape [3] filters= range (conv_img.shape [3]) else: num_filters = len (filters) # get number of grid rows and columns grid_r Grid_c = visualize_utils.get_grid_dim (num_filters) # create figure and axes fig, axes = plt.subplots (min ([grid_r, grid_c]), max ([grid_r, grid_c]) # iterate filters if num_filters = = 1: img = conv_img [0,:,:, filters [0]] axes.imshow (img, vmin=w_min, vmax=w_max, interpolation='bicubic' Cmap=cm.hot) # remove any labels from the axes axes.set_xticks ([]) axes.set_yticks ([]) else: for l, ax in enumerate (axes.flat): # get a single image img = conv_img [0,:,:, filters [l]] # put it on the grid ax.imshow (img, vmin=w_min, vmax=w_max, interpolation='bicubic' Cmap=cm.hot) # remove any labels from the axes ax.set_xticks ([]) ax.set_yticks ([]) # save figure plt.savefig (os.path.join (plot_dir,'{} .png '.format (name)), bbox_inches='tight')

We will not repeat the same part of the code as filter visualization. Here we only talk about the unique aspect of feature map visualization, which actually lies in the following code, that is, the acquisition of data to be visualized:

Img = conv_img [0,:,:, filters [0]]

Neural networks are generally the input data of a batch and a batch, and the input shape is

Image = tf.placeholder (tf.float32, shape = [None, IMAGE_SIZE, IMAGE_SIZE, 3], name = "input_image")

The first dimension is the number of pictures in a batch, which can be set to None,Tensorflow for flexibility and will be calculated based on the actual input data. The second and third dimension is the height and width of the picture, and the fourth dimension is the number of channels of the picture, which is generally 3.

If we want to enter a picture and look at its activation value (feature map), we should also enter it in the form of a batch according to the above dimensions, that is, [1, IMAGE_SIZE, IMAGE_SIZE, 3]. So when taking the feature map data, the first dimension must take 0 (corresponding to the current image in batch), the second or third dimension takes all, and the fourth dimension takes a certain channel of the feature map we want to view.

If you want to visualize feature map, you need to tamper with the construction of the network. When defining a calculation chart, you need to add a set of activation values to the collection of Tensorflow, as shown below:

Tf.add_to_collection ('activations', current)

When actually carrying out feature map visualization, we should first input a picture, then run the network to get the corresponding data, and finally transfer the data to the visualization function. The following example shows how to visualize and store each channel of the feature map for each specified convolution layer separately, using the VGG16 network:

Visualize_layers = ['conv1_1',' conv1_2', 'conv2_1',' conv2_2', 'conv3_1',' conv3_2', 'conv3_3',' conv4_1', 'conv4_2',' conv4_3', 'conv5_1',' conv5_2', 'conv5_3'] with tf.Session (graph=tf.get_default_graph ()) as sess: init_op = tf.group (tf.global_variables_initializer () Tf.local_variables_initializer () sess.run (init_op) saver.restore (sess, model_path) image_path = root_path + 'images/train_images/sunny_0058.jpg' img = misc.imread (image_path) img = img-meanvalue img = np.float32 (img) img = np.expand_dims (img, axis=0) conv_out = sess.run (tf.get_collection (' activations'), feed_dict= {x: img, keep_prob: 1.0}) for I Layer in enumerate (visualize_layers): visualize_utils.create_dir (dir_prefix + layer) for j in range (conv_ out.shape [3]): visualize.plot_conv_output (conv_out [I], dir_prefix + layer, str (j), filters_all=False, filters= [j]) sess.close ()

Where conv_out contains all the feature map added to the collection, which are divided by convolution layer in the conv_out.

The final result is shown in the following figure:

All the results under the first folder:

These are all the contents of this article entitled "sample Analysis of weights and feature map Visualization in Tensorflow". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.