Shulou(Shulou.com)06/02 Report--

C++ OpenCV how to carry out image panoramic mosaic, I believe that many inexperienced people do not know what to do, so this paper summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

Preface

Next, we will use OpenCV C++ for panoramic stitching of images. At present, using OpenCV to stitch two images can be divided into two categories.

First, use OpenCV built-in API Stitcher for splicing.

Second, use the feature detection algorithm to match the similar points in the two images, calculate the transformation matrix, and finally carry on the perspective transformation to it.

1. OpenCV Stitcher

ImageA

ImageB

The original picture is shown in the picture. The requirement of this case is to splice the above two pictures into one image. First, use the Stitcher provided by OpenCV for splicing. On the specific principles of Stitcher, please find the relevant information by yourself.

1. Function source code bool OpenCV_Stitching (Mat imageA, Mat imageB) {vectorimages; images.push_back (imageA); images.push_back (imageB); Ptrstitcher = Stitcher::create (); Mat result; Stitcher::Status status = stitcher- > stitch (images, result); / / stitching if (status! = Stitcher::OK) return false using stitch function Imshow ("OpenCV image panoramic mosaic", result); return true;} 2. Effect.

This is the result of splicing using OpenCV's built-in Stitcher.

Second, image panoramic stitching 1. Feature detection

Method 2 is used for image panoramic stitching. At present, the general process of online tutorials is as follows:

1. The key points of two images are extracted by feature detection operator, and then the feature descriptors are matched. I'm using the SURF operator here. Of course, other feature detection operators such as SIFT can also be used.

/ / create SURF feature detector int Hessian = 800; Ptrdetector = SURF::create (Hessian); / / perform image feature detection and feature description vectorkeypointA, keypointB; Mat descriptorA, descriptorB; detector- > detectAndCompute (imageA, Mat (), keypointA, descriptorA); detector- > detectAndCompute (imageB, Mat (), keypointB, descriptorB) / / use FLANN algorithm to match FlannBasedMatcher matcher; vectormatches; matcher.match (descriptorA, descriptorB, matches) of feature descriptors

The figure shows the result of feature descriptor matching using FLANN algorithm. We need to filter out the key points with high matching degree to calculate the homography matrix of the two images below.

2. Screen out the key points with high matching degree.

Double Max = 0; for (int I = 0; I

< matches.size(); i++) { //float distance ->

The smaller the Euclidean distance that represents the matching pair of feature point descriptors (essentially vectors), the more similar the two feature points are. Double dis = matches [I] .distance; if (dis > Max) {Max = dis;}} / / screen out the key points with high matching degree vectorgoodmatches; vectorgoodkeypointA, goodkeypointB; for (int I = 0; I)

< matches.size(); i++) { double dis = matches[i].distance; if (dis < 0.15*Max) { //int queryIdx ->

It is not only the subscript of the feature point descriptor (descriptor) of the test image, but also the subscript of the corresponding feature point (keypoint) of the descriptor. GoodkeypointA.push_back (keypointA [matches [I]. QueryIdx] .pt); / / int trainIdx-> is not only the subscript of the feature point descriptor of the sample image, but also the subscript of the corresponding feature point. GoodkeypointB.push_back (keypointB [matchs [I]. RoomIdx] .pt); goodmatches.push_back (matches [I]);}}

The picture shows the key points filtered out by imageA.

The picture shows the key points filtered out by imageB.

As you can see from the figure above, we have filtered out the key points common to imageA,imageB. Next, we need to use these two point sets to calculate the homography matrix of the two graphs.

two。 Calculation of homography matrix

Calculate the homography transformation matrix

/ obtain the projection mapping matrix from image A to image B, size 3: 3 Mat H = findHomography (goodkeypointA, goodkeypointB, RANSAC); Mat M = (Mat_ (3,3) detectAndCompute (imageA, Mat (), keypointA, descriptorA); detector- > detectAndCompute (imageB, Mat (), keypointB, descriptorB); / / use FLANN algorithm to match FlannBasedMatcher matcher; vectormatches; matcher.match (descriptorA, descriptorB, matches) of feature descriptors Double Max = 0; for (int I = 0; I

< matches.size(); i++) { //float distance ->

The smaller the Euclidean distance that represents the matching pair of feature point descriptors (essentially vectors), the more similar the two feature points are. Double dis = matches [I] .distance; if (dis > Max) {Max = dis;}} / / screen out the key points with high matching degree vectorgoodmatches; vectorgoodkeypointA, goodkeypointB; for (int I = 0; I)

< matches.size(); i++) { double dis = matches[i].distance; if (dis < 0.15*Max) { //int queryIdx ->

It is not only the subscript of the feature point descriptor (descriptor) of the test image, but also the subscript of the corresponding feature point (keypoint) of the descriptor. GoodkeypointA.push_back (keypointA [matches [I]. QueryIdx] .pt); / / int trainIdx-> is not only the subscript of the feature point descriptor of the sample image, but also the subscript of the corresponding feature point. GoodkeypointB.push_back (keypointB [matchs [I]. RoomIdx] .pt); goodmatches.push_back (matchs [I]);}} if (draw) {Mat result; drawMatches (imageA, keypointA, imageB, keypointB, goodmatches, result); imshow ("feature matching", result) Mat temp_A = imageA.clone (); for (int I = 0; I

< goodkeypointA.size(); i++) { circle(temp_A, goodkeypointA[i], 3, Scalar(0, 255, 0), -1); } imshow("goodkeypointA", temp_A); Mat temp_B = imageB.clone(); for (int i = 0; i < goodkeypointB.size(); i++) { circle(temp_B, goodkeypointB[i], 3, Scalar(0, 255, 0), -1); } imshow("goodkeypointB", temp_B); } //findHomography计算单应性矩阵至少需要4个点 /* 计算多个二维点对之间的最优单映射变换矩阵H（3x3），使用MSE或RANSAC方法，找到两平面之间的变换矩阵 */ if (goodkeypointA.size() < 4 || goodkeypointB.size() < 4) return false; //获取图像A到图像B的投影映射矩阵，尺寸为3*3 Mat H = findHomography(goodkeypointA, goodkeypointB, RANSAC); Mat M = (Mat_(3, 3) detectAndCompute(imageA, Mat(), keypointA, descriptorA); detector->

DetectAndCompute (imageB, Mat (), keypointB, descriptorB); / / use FLANN algorithm to match FlannBasedMatcher matcher; vectormatches; matcher.match (descriptorA, descriptorB, matches) of feature descriptors; double Max = 0; for (int I = 0; I)

< matches.size(); i++) { //float distance ->

The smaller the Euclidean distance that represents the matching pair of feature point descriptors (essentially vectors), the more similar the two feature points are. Double dis = matches [I] .distance; if (dis > Max) {Max = dis;}} / / screen out the key points with high matching degree vectorgoodmatches; vectorgoodkeypointA, goodkeypointB; for (int I = 0; I)

< matches.size(); i++) { double dis = matches[i].distance; if (dis < 0.15*Max) { //int queryIdx ->

It is not only the subscript of the feature point descriptor (descriptor) of the test image, but also the subscript of the corresponding feature point (keypoint) of the descriptor. GoodkeypointA.push_back (keypointA [matches [I]. QueryIdx] .pt); / / int trainIdx-> is not only the subscript of the feature point descriptor of the sample image, but also the subscript of the corresponding feature point. GoodkeypointB.push_back (keypointB [matchs [I]. RoomIdx] .pt); goodmatches.push_back (matchs [I]);}} if (draw) {Mat result; drawMatches (imageA, keypointA, imageB, keypointB, goodmatches, result); imshow ("feature matching", result) Mat temp_A = imageA.clone (); for (int I = 0; I < goodkeypointA.size (); iTunes +) {circle (temp_A, goodkeypointA [I], 3, Scalar (0,255,0),-1);} imshow ("goodkeypointA", temp_A) Mat temp_B = imageB.clone (); for (int I = 0; I < goodkeypointB.size (); iTunes +) {circle (temp_B, goodkeypointB [I], 3, Scalar (0,255,0),-1);} imshow ("goodkeypointB", temp_B) } / / findHomography calculation of homography matrix requires at least 4 points / * calculate the optimal single mapping transformation matrix H (3x3) between multiple two-dimensional point pairs, and use MSE or RANSAC method to find the transformation matrix * / if (goodkeypointA.size () < 4 | goodkeypointB.size () < 4) return false between two planes. / / obtain the projection mapping matrix from image A to image B, with dimensions of 3'3 Mat H = findHomography (goodkeypointA, goodkeypointB, RANSAC); Mat M = (Mat_ (3,3) stitch (images, result); / / stitching if (status! = Stitcher::OK) return false; imshow ("OpenCV image panoramic mosaic", result) using the stitch function; return true } int main () {Mat imageA = imread ("image1.jpg"); Mat imageB = imread ("image2.jpg"); if (imageA.empty () | | imageB.empty ()) {cout

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.