Shulou(Shulou.com)06/01 Report--

This article introduces you how to understand OpenCV and its engineering application, the content is very detailed, interested friends can refer to, hope to be helpful to you.

OpenCV is a cross-platform computer vision and machine learning software library released under the BSD license (open source) that can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient-it consists of a series of C functions and a small number of C++ classes, provides interfaces to Python, Ruby, MATLAB and other languages, and implements many general algorithms in image processing and computer vision.

OpenCV is widely used in image segmentation, face recognition, object recognition, motion tracking, motion analysis, machine vision and other fields.

The following is the basic operation of OpenCV and its application case.

1. OpenCV basic operation 1.1 read, display and save operation import cv2image = cv2.imread ("test.jpg") # read operation cv2.imshow ("test", image) # display operation cv2.waitKey () # wait for button cv2.imwrite ("save.jpg") # Save operation 1.2 change color space image = cv2.imread ("test.jpg") hsv = cv2.cvtColor (image Cv2.COLOR_BGR2HSV) # convert to HSV space hls = cv2.cvtColor (image, cv2.COLOR_BGR2HLS) # convert to HLS space lab = cv2.cvtColor (image, cv2.COLOR_BGR2Lab) # convert to Lab space gray = cv2.cvtColor (image, cv2.COLOR_BGR2GRAY) # convert to GRAY space (grayscale)

The parameters of color in HSV model are hue (H), saturation (S) and lightness (V). This model is often used to segment green screen.

In image detection, we can convert the color space of the sample to achieve data enhancement, such as directly converting the training data to HSV space, or adjusting the size of V (lightness) channel, changing the light and shade of the picture, and then transferring to BGR format.

1.3 geometric transformation-zoom, translate, rotate a. Zoom image = cv2.imread ("test.jpg") resize = cv2.resize (image, (), fx=0.5, fy=0.5, interpolation=cv2.INTER_AREA) # to 0.5x b. Translation

In the translation operation of the image, it is necessary to create a transformation matrix of 2 rows and 3 columns. The M matrix indicates that the translation is x in the horizontal direction and y in the vertical direction.

Import cv2import numpy as npimage = cv2.imread ("test.jpg") rows, cols, channels = image.shapeM = np.float32 ([[1mem0100], [0mai1mai50]]) res = cv.warpAffine (image, M, (cols, rows)) c. Rotation

The transformation matrix needed for rotation can be obtained by the function cv2.getRotationMatrix2D.

Image = cv2.imread ('test.jpg') rows, cols, channels = image.shaperotate = cv2.getRotationMatrix2D ((rows*0.5, cols*0.5), 45, 1) # first parameter: rotation center point second parameter: rotation angle third parameter: scale res = cv2.warpAffine (image, rotate, (cols, rows)) 1.4 smoothing-blur, filtering

The fuzzy filtering operation removes the salt and pepper noise in the image, improves the contrast of the image, realizes sharpening processing, improves the three-dimensional sense and so on.

Image = cv2.imread ('test.jpg') blur = cv2.blur (image, (5,5)) # the second parameter of mean filtering is convolution kernel size median_blur = cv2.medianBlur (image, 5) # median filtering gaussian _ blur = cv2.GussianBlur (image, (5,5)) # Gaussian fuzzy 1.5 expansion, erosion a. Image morphological operation

Image morphology operation is a collection of a series of image processing operations based on shape, which is mainly based on morphological mathematics based on set theory.

There are four basic operations of morphology: erosion, expansion, opening and closing.

Expansion and erosion are the most commonly used morphological operations in image processing.

Expansion is the expansion of the highlighted part of the image, the "domain expansion", and the effect image has a larger highlight area than the original image. Erosion means that the highlighted part of the original image is eroded, "the field is eroded", and the effect image has a smaller highlight area than the original image.

b. Expansion and erosion

They can perform a variety of functions, mainly as follows:

Eliminate noise

Separate the image elements and connect the adjacent elements in the image

Look for obvious maximum or minimum areas in the image

Find out the gradient of the image

Image = cv2.imread ("test.jpg") kernel = cv2.getStructuringElement (cv2.MORPH_RECT, (3,3)) # get convolution kernel eroded = cv2.erode (image, kernel) # corrosion image dilated = cv2.dilate (image, kernel) # expansion image c. Open operation and closed operation

Open operation: first corrode and then expand, used to remove spots caused by image noise

Closed operation: expand first and then erode, used to connect objects that are mistakenly divided into many small pieces

1.6 find the drawing outline a. Find Contour

Contour search is widely used in the field of image detection, such as finding obvious color blocks, stripes, object edges and so on.

# opencv version is greater than 3contours, hierarchy = cv2.findContours (image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # first parameter: found binary image second parameter: contour retrieval mode third parameter: contour approximation method # return value contours is the list of found contours, hierarhy is the hierarchical relationship between contours b. Draw an outline

After finding the outline, you can draw the outline through the drawContours function.

Cv2.drawContours (temp,contours,-1, (0555)), 3) # the first parameter: canvas, can be the second parameter of the original image: the third parameter of the found outline:-1 represents the fourth parameter of the whole painting: the fifth parameter of color: outline width 2. OpenCV's engineering application-color block detection 2.1 problem background

Game screen because of the lack of art resources, the program bug will produce a variety of color blocks, common white, purple and so on, how to screen out this kind of abnormal screen through the program to speed up the testing process?

2.2 problem analysis

Through observation, we find that the anomalies of color blocks are some regular rectangular images, and the color difference is very obvious. based on these characteristics, we can easily screen excellent blocks.

2.3 programming a. Image binarization

Other colors are removed by the numerical size of the RGB channel, and a black-and-white binary graph is obtained.

Import cv2import numpy as npimage = cv2.imread ("test.jpg") b, g, r = cv2.split (self.image) # separate B, G, R channels b = np.where (b > = 250,1,0) # find the pixels that meet the G channel requirements set 1, do not meet the setting 0g = np.where (g > = 250,1,0) # find the pixel points 1 that meet the requirements of G channel Do not conform to setting 0r = np.where (r > = 250,1,0) # find pixels that meet the requirements of G channel set 1, do not conform to setting 0gray = b + g + r # stack three channels into one channel gray= np.where (gray==3, 255,0) .astype (np.uint8) # the pixel that meets the requirement is set to white, and the non-compliant is set to black

b. Find Contour

The obtained black-and-white binary image is preserved because other image positions are also close to the color of the color block, so it needs to be eliminated.

By finding the outline of the binary graph, we can filter out the separated small color blocks.

Contours, hierarchy = cv2.findContours (gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # get contours and hierarchical relationships

c. Contour screening

The small color blocks are removed by the contour perimeter and area, which are likely to be the points with similar colors in the extracted normal areas.

Def screen_contour (contour): contour_area = cv2.contourArea (contour) if contour_area > self.area_limit: return True return Falsepass_contours = [] for contour in contours: if screent_contour (contour): pass_contours.append (contour)

Because the color block is closer to the rectangle, we can calculate the width and height of the excellent block through the outline, and the area of the color block can be estimated. the closer the color block area is to the estimated area, it means that the detected color block is closer to the rectangle. most irregular color blocks can be screened out.

Def screent_contour (contour): width = np.max (contour [:,: 0])-np.min (contour [:,: 0]) height = np.max (contour [:, 1])-np.min (contour [:,: 1]) block_area = width * heigh contour_area = cv2.contourArea (contour) slimier = cv2.contourArea (contour) / block_area if slimier > self.simliar_rate: return True return Falsepass_contours = [] for contour in contours: if screent_contour (contour): pass_contours.append (contour)

3. OpenCV and Matrix

How does OpenCV do all of this?

3.1 Image reading

When OpenCV reads the image, it converts the image information into a matrix, the default matrix is (height,width,channel), channel corresponds to B, G, R channels, the color of each pixel is determined by three channels together, and the relationship of tricolor is the same, the size of B, G, R represents the color ratio.

3.2 Color space conversion

The conversion of color space is the transformation of image data from one representation relationship to another. For example, the conversion from BGR to HSV color space converts the original trichromatic representation to hue (H), saturation (S), luminance (V) representation, and the meaning of each channel has changed.

From the point of view of information, the camera converts the illumination information into digital form (image matrix). The conversion of color space is the transformation of image data from one representation to another. The conversion of information will also cause the loss of information or the introduction of noise. For example, when the camera collects illumination information, it is easy to collect salt and pepper noise and discard some frequency bands of illumination information. Sharpness reduction and so on, information loss may also occur during color space conversion, such as conversion from color pictures to grayscale images. Image detection and face recognition in deep learning is to extract the information we want from these image information.

3.3 Mathematical meaning of image rotation

When we do the geometric transformation of the image, we need to provide the transformation matrix. How does the matrix accomplish these operations?

We know that the matrix represents a kind of spatial mapping, and nxm's matrix (column vector linearly independent) represents the mapping from n-dimensional space to m-dimensional space. As shown in the figure above, the space in which the first cube is located is mapped to the space in which the second cube is located through a 3x3 matrix, and the shape of the cube has changed. For example, the space in which the first cube is located is mapped to the space in which its shadow is located through a 3x2 matrix, and the cube is compressed into a plane.

Can a cube be mapped to a sphere through a matrix (range of real numbers)? Because it represents a linear transformation, it is not allowed.

When we deal with image problems, we have to map information from one space to another. Because of the complexity of the problem, linear mapping can not meet the requirements, which is why activation functions need to be added to deep learning.

4. OpenCV and Machine Learning

From the analysis in the previous section, we found that the image processing process is to find rules from the data and transform the image information from one representation to another. This work happens to be the strength of machine learning. Whether it is image data, text data, audio data, to find rules from the data, will use machine learning, from the point of view of information, these problems are essentially no difference.

OpenCV has integrated many machine learning algorithms, such as K nearest neighbor (KNN), support vector machine (SVM), decision tree, random forest, Boost, logical regression, ANN and so on.

The following picture is taken from scikit-learn, which vividly shows how different machine learning algorithms deal with data, that is, how to transform information into different spaces from a macro point of view.

On how to understand OpenCV and its engineering applications to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.