Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to achieve the search goal of the meanshift algorithm in openCV. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

I. brief introduction

The reverse projection of an image histogram is a probability distribution map that represents the probability of a specified image segment appearing at a specific location. When we know the general position of an object in the image, we can find the exact position of the object in another image through the probability distribution map. We can set an initial position and move around it repeatedly to improve the local matching probability, so as to find the exact location of the object. This process is called the mean shift algorithm.

Second, the process of realization

Because the facial features of the characters are more obvious than other positions, this experiment is mainly applied to the facial recognition of the characters.

1. Set the area of interest

There are two ways to set the region of interest, one is to know the pixel coordinates of the face position of the character in the picture, to locate the face position by setting a rectangle, and the other is to use the selectROI function of opencv to manually select the position you are interested in.

2. Obtain the facial histogram and normalize it.

Set a ColorHistogram class to add a function getHueHistogram that gets the tonal histogram. This function includes converting the image to HSV color space, masking low-saturation pixels (which may or may not be used), and calculating the image histogram.

Cv::Mat getHueHistogram (const cv::Mat & image2, int minSaturation = 0) {cv::Mat hist; / / convert to HSV color space cv::Mat hsv; cv::cvtColor (image2, hsv, CV_BGR2HSV); / / cv::imshow ("hsv", hsv) / / Mask (may or may not be used) cv::Mat mask; if (minSaturation > 0) {std::vectorv; cv::split (hsv, v) / / divide 3 channels into 3 images cv::threshold (v [1], mask, minSaturation, 255, cv::THRESH_BINARY); / / mask low saturation pixels} / / prepare one-dimensional hue histogram parameter hranges [0] = 0; hranges [1] = 180.0 / / the range is 0,180 channels [0] = 0 / / Hue channel / / calculate histogram cv::calcHist (& hsv, 1, / / histogram channels of only one image, / / Channel mask used, / / binary mask hist / / histogram 1 as a result, / / this is an one-dimensional histogram histSize, / / number of boxes ranges) / / range of pixel values return hist;}

Then, the obtained histogram is normalized.

Void setHistogram (const cv::Mat& h) {histogram = h; cv::normalize (histogram, histogram, 1.0);} 3, backprojection, use meanshift to find the target

Open the second image and convert it to HSV color space (resize the input image in the code to prevent some images from being too large and incomplete), and then project the histogram of the first image backwards. The following result is the result of reverse projection. At present, Luffy's face is selected as the region of interest. If Lufei's hat is selected, the reverse projection will have a different effect. You can try it yourself.

/ / Open the second image and convert it to HSV, and backproject the histogram of the first image image = cv::imread ("lufei2.JPG"); resize (image, image3, cv::Size (500,700)); cv::cvtColor (image3, hsv, CV_BGR2HSV); / / convert to HSV color space int ch [1] = {0} Cv::Mat result = finder.find (hsv, 0.0f, 180.0f, ch)

The meanshift algorithm of openCV can be used to modify the initial rectangular region into a new position of the face of the person in the image.

Cv::TermCriteria criteria (cv::TermCriteria::MAX_ITER | cv::TermCriteria::EPS, 10, / / up to 10 iterations 1); / / or the center of gravity moves less than 1 pixel cv::meanShift (result, rect, criteria)

At this point, we found the face of the character in another image.

III. Other experimental results

In addition to the experiment of finding another single image from a single image, we also did an experiment of finding a group photo of multiple people from a single image. Here is an experiment on NBA stars.

IV. Part of the principle supplement

In this experiment, in order to highlight the characteristics of the target of interest, the tonal component of the HSV color space is used. After using the CV_BGR2HSV logo to convert the image, the first channel is the tonal component. This is an 8-bit component with values ranging from 0 to 180 (if you use cv::cvtColor, the converted image will be of the same type as the original image). In order to extract the tonal image, the cv::split function divides the three-channel HSV image into three single-channel images. These three images are stored in a std::vector instance, and the tonal image is the first entry to the vector (that is, the index is 0).

When using the hue component of a color, take its saturation into account (saturation is the second entry of the vector). When the saturation of the color is very low, its hue information becomes unstable and unreliable. This is because the B, G, and R components of a low saturation color are almost equal, which makes it difficult to determine the exact color it represents. Therefore, in the getHueHistogram method, the minSat parameter is used to mask the pixels whose saturation is lower than this threshold and do not count them into the histogram.

The mean shift algorithm is an iterative process, which is used to locate the local maximum of the probability function by finding the center of gravity or weighted average of the data points in the predefined window. Then, move the window to the center of gravity and repeat the process until the center of the window converges to a stable point. When OpenCV implements the algorithm, two stop conditions are defined: the maximum number of iterations (MAX_ITER) and the offset of the center of the window is less than a certain limit (EPS), which can be considered to converge to a stable point. These two conditions are stored in an cv::TermCriteria instance.

5. Complete code # include#include # include / / Core of the image data structure # include / / all graphics interface functions # include#include # include#include using namespace std;using namespace cv; / / get tonal histogram class ColorHistogram {private: int histSize [3]; / / size of each dimension float hranges [2]; / / range of values (three dimensions use the same value) const float* ranges [3] / / range of each dimension int channels [3]; / / Channel public to be processed: ColorHistogram () {/ / default parameters for color images / / the size and range of each dimension are equal histSize [0] = histSize [1] = histSize [2] = 256 Hranges [0] = 0; / / BGR range is 0,256 hranges [1] = 256.0; ranges [0] = hranges; / / in this class ranges [1] = hranges; / / the range of all channels is equal ranges [2] = hranges; channels [0] = 0 / / three channels: B channels [1] = 1; / / G channels [2] = 2; / / R} / / calculate the one-dimensional histogram, the original image of BGR is converted to HSV, ignoring the low saturation pixel cv::Mat getHueHistogram (const cv::Mat & image2, int minSaturation = 0) {cv::Mat hist / / convert to HSV color space cv::Mat hsv; cv::cvtColor (image2, hsv, CV_BGR2HSV); / / cv::imshow ("hsv", hsv); / / mask (may or may not be used) cv::Mat mask If (minSaturation > 0) {std::vectorv; cv::split (hsv, v); / / divide 3 channels into 3 images cv::threshold (v [1], mask, minSaturation, 255, cv::THRESH_BINARY) / / masking low saturation pixels} / / parameters for preparing one-dimensional hue histogram hranges [0] = 0; hranges [1] = 180.0; / / range is 0,180 channels [0] = 0 / / Hue channel / / calculate histogram cv::calcHist (& hsv, 1, / / histogram channels of only one image, / / Channel mask used, / / binary mask hist / / histogram 1 as a result, / / this is an one-dimensional histogram histSize, / / number of boxes ranges) / / range of pixel values return hist;}}; class ContentFinder {private: / / histogram parameter float hranges [2]; const float* ranges [3]; int channels [3]; float threshold; / / judgment threshold cv::Mat histogram / / enter histogram public: ContentFinder (): threshold (0.1f) {/ / the range of all channels in this class is the same ranges [0] = hranges; ranges [1] = hranges; ranges [2] = hranges } / / A pair of histograms are normalized void setHistogram (const cv::Mat& h) {histogram = h; cv::normalize (histogram, histogram, 1.0) } / / find pixels belonging to the histogram cv::Mat find (const cv::Mat& image, float minValue, float maxValue, int * channels) {cv::Mat result; hranges [0] = minValue; hranges [1] = maxValue / / the number of dimensions of the histogram is the same as the channel list for (int I = 0; I

Channels [I] = channels [I] Cv::calcBackProject (& image, 1, / / use only one image channels, / / channel histogram, / / histogram result, / / backprojected image ranges / / the value range of each dimension is 255.0 / / the selected conversion factor / / maps the probability value from 1 to 255) Cv::imshow ("result", result); return result;}}; int main () {/ * mean detection meanshift*/ cv::Mat image = cv::imread ("ZMS1.jpg"); cv::Mat image2; cv::Mat image3; cv::Mat hsv Resize (image, image2, cv::Size (500,700); cv::Rect rect; rect = cv::selectROI ("image", image2, false, false); cv::Mat imageROI = image2 (rect). Clone (); / / Manual box selection / * cv::Rect rect (227,108,108,104); cv::Mat imageROI = image2 (rect) * / / manually set the rectangle selection range cv::rectangle (image2, rect, cv::Scalar (255,0,0), 1, cv::LINE_8, 0); cv::imshow ("image2", image2); / / get the face histogram int minsat = 65; / / minimum saturation ColorHistogram hc; cv::Mat colorhist = hc.getHueHistogram (imageROI, minsat) / / pass the histogram to the ContentFinder class ContentFinder finder; finder.setHistogram (colorhist); / / normalize a pair of histograms / / open the second image and convert it to HSV, and backproject the histogram of the first image image = cv::imread ("ZMS2.JPG"); resize (image, image3, cv::Size (500,700)) Cv::cvtColor (image3, hsv, CV_BGR2HSV); / / convert to HSV color space int ch [1] = {0}; cv::Mat result = finder.find (hsv, 0.0f, 180.0f, ch) Cv::TermCriteria criteria (cv::TermCriteria::MAX_ITER | cv::TermCriteria::EPS, 10, / / up to 10 iterations 1); / / or the center of gravity moves less than 1 pixel cv::meanShift (result, rect, criteria) Cv::rectangle (image3, rect, cv::Scalar (0,255,0), 1, cv::LINE_8, 0); cv::imshow ("image3", image3); waitKey (0) } this is the end of the article on "how to achieve the search goal of meanshift algorithm in openCV". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.