Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "how to use Python and openCV to achieve panoramic image stitching". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Basic introduction

Panoramic stitching of images, that is, stitching two images with overlapping areas to create a panorama. Computer vision and image processing techniques are used: key point detection, local invariant features, key point matching, RANSAC (random sampling consistency) and perspective deformation.

Concrete steps

(1) detect the SIFT key feature points of the left and right images, and extract local invariant features.

(2) use knnMatch to detect the SIFT features from the right (left) and match them with the left (right)

(3) calculate the angle of view transformation matrix H, and use the transformation matrix H to distort the right graph.

(4) add the left image (right) to the left (right) of the transformed image to get the final image.

Import cv2 as cv # Import opencv package import numpy as np # Import numpy package, matrix operation in image processing needs to use # detect SIFT key feature points of image def sift_keypoints_detect (image): # Color information is rarely used in image processing Usually, the image is directly converted into a grayscale image gray_image = cv.cvtColor (image, cv.COLOR_BGR2GRAY) # to obtain image feature sift-SIFT feature points. Instantiated object sift sift = cv.xfeatures2d.SIFT_create () # keypoints: feature point vector. Each element in the vector is a KeyPoint object. Contains various attribute information of feature points (angles, key point coordinates, etc.) # features: represents the output sift feature vector Usually 128D keypoints, features = sift.detectAndCompute (image, None) # cv.drawKeyPoints (): draw a small circle at the key point of the image # if you pass the flag flags=cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS, it will draw a circle of size keypoint and show its direction # this method displays the coordinates, size and direction of the image at the same time Is the best way to display features keypoints_image = cv.drawKeypoints (gray_image, keypoints, None, flags=cv.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS) # returns images with key points, key points and feature vectors of sift return keypoints_image, keypoints, features# use KNN to detect SIFT features from left and right images Then match def get_feature_point_ensemble (features_right, features_left): # create a BFMatcher object to solve the matching bf = cv.BFMatcher () # knnMatch () function: return the best match k points of each feature point matches = bf.knnMatch (features_right, features_left, Know2) # des1 is the template diagram Des2 is a matching figure # using the sorted () function to perform ascending order (default) operation on matches objects matches = sorted (matches, key=lambda XRV x [0] .distance / x [1] .distance) # XRV x [] letters can be modified at will, sorted by the dimensions in brackets [], and sorted by [0] by the first dimension [2] set up a list according to the third dimensional sort # good to store the matching point set good = [] for m, the higher the value of n in matches: # ratio, the denser the matching lines, but the more mismatching points will increase ratio=0.6 if m.distance.

< ratio * n.distance: good.append(m) return good# 计算视角变换矩阵H，用H对右图进行变换并返回全景拼接图像def Panorama_stitching(image_right, image_left): _, keypoints_right, features_right = sift_keypoints_detect(image_right) _, keypoints_left, features_left = sift_keypoints_detect(image_left) goodMatch = get_feature_point_ensemble(features_right, features_left) # 当筛选项的匹配对大于4对(因为homography单应性矩阵的计算需要至少四个点)时,计算视角变换矩阵 if len(goodMatch) >

4: # get the matching point coordinates ptsR = np.float32 ([keypoints_ right.queryIdx] .pt for m in goodMatch]) .reshape (- 1,1,2) ptsL = np.float32 ([keypoints_ left [m.queryIdx] .pt for m in goodMatch]) .reshape (- 1,1) 2) # ransacReprojThreshold: the maximum allowable reprojection error threshold for treating point pairs as interior points (for RANSAC and RHO methods only), if srcPoints and dstPoints are in pixels This parameter is usually set in the range of 1 to 10 ransacReprojThreshold = 4 # cv.findHomography (): calculate the optimal single mapping transformation matrix H (3 rows x 3 columns) between multiple two-dimensional point pairs, using the minimum mean square error or RANSAC method # function: select the optimal four sets of matching points using a robust algorithm based on RANSAC Then calculate the transformation matrix H _ (3) and return it so that the error rate of reverse projection can reach the minimum Homography, status = cv.findHomography (ptsR, ptsL, cv.RANSAC, ransacReprojThreshold) # cv.warpPerspective (): perspective transformation function, which is used to solve the problem that cv2.warpAffine () can not deal with the problem that the field of view is not parallel to the image. # function: it is to transform the image in perspective so that the straight line does not deform. But parallel lines may no longer be parallel result = cv.warpPerspective (image_right, Homography, (image_right.shape [1] + image_left.shape [1], image_ right.shape [0]) cv.imshow ("distorted transformed right") Result) cv.waitKey (0) cv.destroyAllWindows () # add the left image to the left end of the transformed right image to get the final image result [0: image_left.shape [0] 0:image_left.shape [1]] = image_left # returns the image of panoramic stitching return resultif _ _ name__ = ='_ main__': # reads the image to be stitched Note that the order around the image image_left = cv.imread (". / Left.jpg") image_right = cv.imread (". / Right.jpg") # change the size of the image by calling cv2.resize () using interpolation Ensure that the left and right images are the same size. The second parameter dsize in the # cv.resize () function represents the output image size. When set to 0 (None), Then multiply fx and fy by the original image size to get the output image size image_right = cv.resize (image_right, None, fx=0.4, fy=0.24) image_left = cv.resize (image_left, (image_right.shape [1], image_ right.shape [0]) # obtain the relevant parameters keypoints_image_right, keypoints_right of the image after the key points are detected Features_right = sift_keypoints_detect (image_right) keypoints_image_left, keypoints_left, features_left = sift_keypoints_detect (image_left) # use the np.hstack () function to stack both the original image and the image with key points in the vertical direction (horizontal order) cv.imshow ("left key detection", np.hstack ((image_left) (keypoints_image_left) # generally set waitKey (0) after imshow, which means press any key to continue cv.waitKey (0) # Delete the previously established window cv.destroyAllWindows () cv.imshow ("right key detection", np.hstack ((image_right, keypoints_image_right) cv.waitKey (0) cv.destroyAllWindows () goodMatch = get_feature_point_ensemble (features_right) Features_left) # cv.drawMatches (): after extracting the features of two images Draw matching point pair connection # matchColor-matching colors (feature points and lines), if matchColor==Scalar::all (- 1), color random all_goodmatch_image = cv.drawMatches (image_right, keypoints_right, image_left, keypoints_left, goodMatch, None, flags=2) cv.imshow ("all matching SIFT key feature points connected" All_goodmatch_image) cv.waitKey (0) cv.destroyAllWindows () # splice the picture into a panorama and save result = Panorama_stitching (image_right, image_left) cv.namedWindow ("panorama", cv.WINDOW_AUTOSIZE) cv.imshow ("panorama", result) cv.imwrite (". / panorama .jpg", result) cv.waitKey (0) cv.destroyAllWindows ()

Detection of key feature points on the left

Detection of key feature points on the right

All matching SIFT key feature points are connected.

Distort the transformed image on the right

Panoramic picture

Because there is a lot of overlap between the input left and right images, the main added part of the panorama is on the right side of the mosaic image, which will result in a large number of black blank areas on the right side of the mosaic panorama.

"how to use Python and openCV to achieve panoramic image stitching" content is introduced here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.