Shulou(Shulou.com)06/03 Report--

This article introduces how to achieve image matching function in Python+Opencv. The content is very detailed. Interested friends can use it for reference. I hope it will be helpful to you.

1. Principle

To put it simply, template matching is to slide a template (picture) on the target picture in turn, calculate the similarity between the template and the sub-image below the template each time, and finally calculate a lot of similarity.

If it is only the matching of a single target, the matching position can be obtained only by taking the position of the maximum similarity.

If you want to match multiple targets, set a threshold, that is, as long as the similarity is greater than, say, 0.8, it is considered to be the target to be matched.

1.1 similarity metrics

Sum of squares of difference matches CV_TM_SQDIFF

Standardized sum of difference squares matching CV_TM_SQDIFF_NORMED

Correlation matching CV_TM_CCORR

Standard correlation matching CV_TM_CCORR_NORMED

Correlation matching CV_TM_CCOEFF

Standard correlation matching CV_TM_CCOEFF_NORMED

1.2 calculation steps

There is a template image Templa and a larger search image Image. Template matching is a method for searching and finding the location of the template image in a larger image.

Specifically, slide the template image onto the input image (as in the convolution operation), and then compare the similarity between the template and the subimages of the input image under the template image.

It returns a grayscale image where each pixel represents the similarity between the neighborhood of the pixel and the template. If you input the size of the image (WxH) and the size of the template image (wxh), the size of the output image will be (Wmurw + 1memHmurh + 1). After obtaining the similarity image, look for the pixel on which the maximum similarity lies. It is used as the upper-left corner of the rectangle of the matching area, and the width and height of the rectangle are taken as (wth _ h). The rectangle is the area that matches the template.

2. Code implementation

2.1 single template matches a single target

The code is as follows:

# correlation coefficient matching method: cv2.TM_CCOEFFres = cv2.matchTemplate (img, template, cv2.TM_CCOEFF) min_val, max_val, min_loc, max_loc = cv2.minMaxLoc (res) left_top = max_loc # Upper left right_bottom = (left_top [0] + w, left_top [1] + h) # Lower right cv2.rectangle (img, left_top, right_bottom, 255,2) # draw the rectangular position plt.subplot (121) Plt.imshow (res, cmap='gray') plt.title ('Matching Result'), plt.xticks ([]), plt.yticks ([]) plt.subplot (122), plt.imshow (img, cmap='gray') plt.title (' Detected Point'), plt.xticks ([]), plt.yticks ([]) plt.show () 2.2 single template matching multiple targets

Target photo: mario.jpg

Template photo: mario_coin.jpg

The code is as follows:

Import cv2import numpy as npimg_rgb = cv2.imread ('mario.jpg') img_gray = cv2.cvtColor (img_rgb, cv2.COLOR_BGR2GRAY) template = cv2.imread (' mario_coin.jpg', 0) h, w = template.shape [: 2] res = cv2.matchTemplate (img_gray, template, cv2.TM_CCOEFF_NORMED) threshold = 0.13 take the coordinate loc = np.where (res > = threshold) # np.where whose matching degree is greater than 80, the coordinate value returned by np.where is (h) w), Notice the order of for pt in zip (* loc [::-1]): bottom_right = (pt [0] + w, pt [1] + h) cv2.rectangle (img_rgb, pt, bottom_right, (0,0,255), 2) cv2.imwrite ("img.jpg", img_rgb) cv2.imshow ('img', img_rgb) cv2.waitKey (0)

The test results are as follows:

3. Optimization of algorithm accuracy

Multi-scale template matching

Rotating target template matching

Non-maximum suppression

From the above picture, you can see that there are multiple boxes for the same image, which need to be duplicated, and only one needs to be retained.

Solution: use NMS (non-maximum suppression) to remove duplicate rectangles for using the same area to be tested

NMS principle

For the list B of Bounding Box and its corresponding confidence S, the following calculation method is used. Select the check box M with the largest score, remove it from the B collection and add it to the final test result D. Usually remove the box in the remaining detection box in B where the IoU of M is greater than the threshold Nt from B, and repeat this process until B is empty.

Ps. The common threshold of overlap ratio (IOU) is 0.3 ~ 0.5.

The code is as follows:

Import cv2import timeimport numpy as np def py_nms (dets, thresh): "" Pure Python NMS baseline. "" # x1, y1, x2, y2, and score assignment # (x1, y1) (x2, y2) is the upper left and lower right corner of box, x1 = dets [:, 0] y1 = dets [:, 1] x2 = dets [:, 2] y2 = dets [:, 3] scores = dets [: 4] # the area of each candidate box areas = (x2-x1 + 1) * (y2-y1 + 1) # order is order = scores.argsort () [::-1] # print ("order:") sorted in descending order of score Order) keep = [] while order.size > 0: I = order [0] keep.append (I) # calculates the coordinates of the current maximum probability rectangle intersecting other rectangles Numpy's broadcast mechanism will be used. The vector xx1 = np.maximum (x1 [I], x1 [order [1:]]) yy1 = np.maximum (y1 [I], y1 [order [1:]]) xx2 = np.minimum (x2 [I], x2 [order [1:]]) yy2 = np.minimum (y2 [I], y2 [order [1:]]) # calculates the area of the intersection frame Note that when the rectangle does not intersect, w or h will be negative. Replace w = np.maximum (0.0, xx2-xx1 + 1) h = np.maximum (0.0) with 0 Yy2-yy1 + 1) inter = w * h # calculate overlap IOU: overlap area / (area 1 + area 2-overlap area) ovr = inter / (areas [I] + areas [1:]]-inter) # find the rectangular box index inds = np.where (ovr = template_threshold) with an overlap not higher than the threshold. Label score = res [res > = template_threshold] # Target confidence greater than the template threshold # processes the template data coordinates into the upper left corner, In the lower right corner, the format xmin = np.array (loc [1]) ymin = np.array (loc [0]) xmax = xmin+w ymax = ymin+h xmin = xmin.reshape (- 1 xmax.reshape 1) # becomes n row 1 column dimension xmax = xmax.reshape (- 1 line 1) # becomes n row 1 column dimension ymax = ymax.reshape (- 1 line 1) # becomes n row 1 column dimension ymin = ymin.reshape (- 1 line 1) # becomes n row 1 column dimension Score = score.reshape (- 1pm 1) # becomes n rows and 1 column dimension data_hlist = [] data_hlist.append (xmin) data_hlist.append (ymin) data_hlist.append (xmax) data_hlist.append (ymax) data_hlist.append (score) data_hstack = np.hstack (data_hlist) # will xmin, Ymin, xmax, yamx, and scores are spliced by column. Thresh = 0.3#NMS IOU interaction ratio threshold keep_dets = py_nms (data_hstack) Thresh) print ("nms time:", time.time ()-start_time) # print data processing to nms run time dets = data_ hstack [keep _ dets] # the rectangle return detsif _ _ name__ obtained by the final nms = "_ _ main__": img_rgb = cv2.imread ('mario.jpg') # Image to be detected img_gray = cv2.cvtColor (img_rgb Cv2.COLOR_BGR2GRAY) # converted to gray template_img = cv2.imread ('mario_coin.jpg', 0) # template small image template_threshold = 0. 0 template confidence dets = template (img_gray,template_img,template_threshold) count = 0 for coord in dets: cv2.rectangle (img_rgb, (int (coord [0]), int (coord [1])), (int (coord [2])), int (coord [3])) (0,0,255), 2) cv2.imwrite ("result.jpg", img_rgb) about how Python+Opencv implements image matching, that's all. I hope the above content can be of some help to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.