Shulou(Shulou.com)05/31 Report--

This article focuses on "how to use Keras to achieve activation heat map in image classification", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use Keras to activate heat maps in image classification.

Guide reading

Use Keras to realize the visualization of activation heat map in image classification and help to improve the model more pertinently.

Class Activation Diagram (CAM) is a powerful technique for computer vision classification tasks. It allows researchers to examine the classified image and find out which parts / pixels of the image contribute more to the final output of the model.

Basically, suppose we build a CNN with the goal of classifying people's photos as "men" and "women", and then we provide it with a new photo that returns the tag "man". With the CAM tool, we can see which part of the image best activates the "Man" class. If we want to improve the accuracy of the model, we must know which layers need to be modified, or whether we want to preprocess the training set image in different ways.

In this article, I will show you the ideas behind this process. To achieve this, I will use a CNN, Resnet50, which is pre-trained on ImageNet.

The image I'm going to use in this experiment is this golden retriever:

First, let's try our pre-training model on this picture and return it to the three most likely categories:

From keras.applications.resnet50 import ResNet50

From keras.preprocessing import image

From keras.applications.resnet50 import preprocess_input, decode_predictions

Import numpy as npmodel = ResNet50 (weights='imagenet') img_path = 'golden.jpg'

Img = image.load_img (img_path, target_size= (224,224)

X = image.img_to_array (img)

X = np.expand_dims (x, axis=0)

X = preprocess_input (x) preds = model.predict (x)

# decode the results into a list of tuples (class, description, probability)

Print ('Predicted:', decode_predictions (preds, top=3) [0])

As you can see, the first result just returns the category we are looking for: Golden retriver.

Now our goal is to identify the part of our photo that best activates the gold tag. To do this, we will use a technique called gradient weighted Class Activation Mapping (Grad-CAM) (official paper: https://arxiv.org/abs/1610.02391).

The idea goes like this: imagine we have a trained CNN and we give it a new image. It will return a class for the image. Then, if we take the output characteristic map of the last convolution layer and weight each channel according to the gradient of each channel according to the output category, we get a heat map. it shows which parts of the input image are most active to that category.

Let's look at the implementation using Keras. First, let's examine the structure of our pre-trained ResNet50 to determine which layer we want to check. Because of the long network structure, I will only show the final block here:

From keras.utils import plot_model

Plot_model (model)

Let's use the last activation layer activation_49 to extract our feature map.

Golden = model.output [:, np.argmax (preds [0])]

Last_conv_layer = model.get_layer ('activation_49')

From keras import backend as K

Grads = K.gradients (golden, last_conv_layer.output) [0]

Pooled_grads = K.mean (grads, axis= (0,1,2))

Iterate = K.function ([model.input], [pooled_grads, last_conv_layer.output [0]])

Pooled_grads_value, conv_layer_output_value = iterate ([x])

For i in range (pooled_grads.shape [0]):

Conv_layer_output_value [:,:, I] * = pooled_grads_ value [I]

Heatmap = np.mean (conv_layer_output_value, axis=-1)

Import matplotlib.pyplot as plt

Heatmap = np.maximum (heatmap, 0)

Heatmap / = np.max (heatmap)

Plt.matshow (heatmap)

Nothing can be seen on this heat map. Therefore, we merge the heat map with the input image as follows:

Import cv2

Img = cv2.imread (img_path)

Heatmap = cv2.resize (heatmap, (img.shape [1], img.shape [0]))

Heatmap = np.uint8 (255 * heatmap)

Heatmap = cv2.applyColorMap (heatmap, cv2.COLORMAP_JET)

Merged= heatmap * 0.4 + imgplt.imshow (merged)

As you can see, some parts of the image (such as the nose) specifically indicate the category of the input image.

At this point, I believe you have a deeper understanding of "how to use Keras to achieve activation heat map in image classification". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.