Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces "Sparse R-CNN case analysis" in detail, the content is detailed, the steps are clear, and the details are handled properly. I hope this "Sparse R-CNN case analysis" article can help you solve your doubts.

Guide reading

See what the third way of object detection looks like.

Today we will discuss a new method called Sparse R-CNN (not to be confused with Sparse R-CNN for 3D computer vision tasks), which uses completely sparse and learnable bounding box generation to achieve state-of-the-art object detection.

Related work

Let's start with a brief introduction to the existing methods.

Dense method

Single-stage detector is one of the most widely used methods at present, which directly predicts the label and position of anchor box, and the spatial position, proportion and aspect ratio of anchor are densely covered. Such as SSD or YOLO.

Let's look at the YOLO algorithm. Finally, its goal is to predict the class of a target on the image and to specify the bounding box of the target location. Each bounding box can be described with four descriptors:

The center point of the bounding box (bx, by), width (bw), height (bh) c is the category of the corresponding target (e.g. cars, traffic lights, etc.)

In addition, we must predict a PC value, that is, the probability that there is a target in the box. It is a dense method because it does not search for areas of interest in a given image that may contain a target. Instead, YOLO splits the image into cells, using a 19 × 19 grid. But generally speaking, a single-phase detector can generate W x H cell, one for each pixel. Each cell is responsible for predicting k bounding boxes (k is selected as 5 in this case), so for an image, we get a large number of W x H x k bounding boxes.

Dense-to-sparse method

The two-stage detector uses RPN to generate the suggestion frame of dense, such as that proposed in Faster R-CNN paper. These detectors have dominated object detection for many years.

The sparse foreground box is obtained from the dense region candidate box by using the RPN algorithm, and then the position of each box is refined and its specific category is predicted.

Similar to the method of single-stage detector, it does not directly predict the category of the target, but the probability of the target. In the second stage, the prediction category is filtered by the objectness and the overlap score of the bounding box.

Sparse method

In this paper, its new Sparse R-CNN paradigm is classified as an extension of the existing target detection paradigm, which includes from complete dense to dense-to-sparse, and then adds new steps to complete sparse.

In this paper, the use of RPN is avoided and replaced by a set of small suggestion boxes (for example, 100 per image). These boxes are obtained through the learnable proposal boxes and proposal features parts of the network. This form predicts four values for each proposal * (x _ mae _ r _ h _ p _ w) *, which predicts a potential representation vector of 256 lengths for each bbox. The learned suggestion box is used as a reasonable statistic to perform the subsequent refinement steps, and the learned suggestion features are used to introduce the attention mechanism. This mechanism is very similar to the one used in DETR's paper. These operations are performed in a dynamic instance interactive head, which we will cover in the next section.

Suggested model features

As the name of the paper implies, the model is end-to-end. The structure is elegant. It consists of the above learnable proposal boxes and proposal features and the dynamic instance interaction header, which is the main contribution of the neural network architecture in this paper.

Dynamic instance interaction header

Given N suggestion boxes, Sparse R-CNN first uses the RoIAlign operation to extract features from the trunk for each area defined by the suggestion box. The features of each region of interest are input into separate headers for target location and classification, where each header is conditional on specific learnable suggestion features.

Suggested features are used as weights for convolution, and they are called "parameters" in the image above. The RoI feature obtains the final feature from the convolution generated by this. In this way, the boxes with the most promising information have an impact on the location and classification of the final target. The self-attention module is embedded in the dynamic head to infer the relationship between objects, and the convolution influence is predicted.

Main results

The author provides several comparison tables to show the performance of this new method. Comparison of Sparse R-CNN with RetinaNet,Faster R-CNN and DETR on two variants of ResNet50 and ResNet100.

Here we can see that sparse R-CNN is better than RetinaNet and Faster R-CNN on both R50 and R100, but its performance is very similar to that based on DETR.

According to the author, the DETR model is actually dense to sparse because it uses a sparse set of target queries to interact with global (dense) image features. Therefore, compared with DETR, the novelty of this article appears.

In this picture, you can see the result of model inference on COCO Dataset. The learning suggestion boxes are shown in the first column, which are predictions of any new images. In the next column, you can see the final bbox extracted from the suggestion. In the process of iterative learning, they vary from stage to stage.

After reading this, the article "Sparse R-CNN case Analysis" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself to understand it. If you want to know more about related articles, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.