Shulou(Shulou.com)06/01 Report--

This article mainly shows you "OpenMV how to achieve image processing", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "OpenMV how to achieve image processing" this article.

Basic knowledge of image processing

1. Camera: optical signals are converted into electrical signals. In computer vision, the simplest model is the keyhole imaging model.

Keyhole imaging is an ideal model, there will be field curvature and distortion in the actual lens, but it can be solved by introducing distortion parameters in the calibration process, and can still be widely used.

two。 Pixels and resolution

3. Frame rate: the number of pictures processed per second, more than 20 eyes can not tell the stutter.

4. Color: electromagnetic waves of different wavelengths

Through RGB. CMYK HSB LAB color gamut to describe

RGB: the tricolor principle is caused not by physical causes but by human physiological causes (conical photoreceptor cells)

LAB:L (brightness), A (+: red, -: green), B (+: yellow, -: blue) are used to approach human vision

L is used to adjust the brightness pair, an and b are used to output color levels for accurate color balance (find color block algorithm)

HSB/HSV:H (Hue hue), S (Saturation saturation), B (Brightness brightness)

HSL:L (luminance)

CMYK:C (blue), M (red), Y (yellow), K (black)

The difference between CMYK and RGB:

(1) CMYK is the subtractive mode, while RGB is the additive mode.

(2) CMYK needs external light to see the color of printed matter; RGB is a color mode that glows on the screen and can also be seen in dark rooms.

5. The choice of light source (the light source has a great influence on the color change of the whole picture)

6. Focal length and lens distortion of lens

The focal length determines the size of the image, the angle of view, the depth of field and the perspective of the picture. The longer the focal length is, the larger the imaging (the smaller the viewing angle), the smaller the focal length is.

Lens distortion: correct the lens with code or use the distortion-free lens directly (additional correction lens is added)

At different positions on the photosensitive chip and at different distances from the lens, fisheye effect appears at the edge (barrel distortion).

Standard lens-barrel distortion

Wide-angle lens-the same area has a wide range of views.

Distortionless lens

Telephoto lens-lock a specific object in the original image

7. The filter of a lens

Filter out the infrared light (filter out the light of different wavelengths as needed), and the infrared light will make the image whitening when analyzing the color image normally. (can only be passed within 650nm)

Second, the basic method of OpenMV image processing 1. Photosensitive element

Sensor module for setting the parameters of the photosensitive element.

Common functions of sensor:

Import sensor # introduces photosensitive element part # sets camera sensor.reset () # initializes photosensitive element sensor.set_pixformat (sensor.RGB565) # sets pixel mode sensor.GRAYSCALE: grayscale, each pixel 8bit, sensor.RGB565: color. Per pixel 16bitsensor.set_framesize (sensor.QVGA) # set the image size sensor.skip_frames (n) # n is the number of frames skipped, wait for the sensor to stabilize # shoot while (True): img = sensor.snapshot (); # take a photo and img opens for an image object # Auto gain / White balance / exposure sensor.set_auto_gain () # True False is off. When using color tracking, you need to turn off sensor.set_auto_whitebal () # True to enable. False is turned off, when using color tracking Need to close sensor.set_auto_exposure (enable [\, exposure_us]) # enable True on / False off # if you close, you can set an exposure value us as a unit # set window ROIsensor.set_windowing (roi) # roi format is (x, y, w H) tupple.sensor.set_framesize (sensor.VGA) sensor.set_windowing ((640) 80) # Select the element you want # set Flip sensor.set_hmirror (True) # horizontal Flip sensor.set_vflip ((True) # Vertical Flip

Common data of sensor image size

Sensor.QQCIF: 88x72sensor.QCIF: 176x144sensor.CIF: 352x288sensor.QQSIF: 88x60sensor.QSIF: 176x120sensor.SIF: 352x240sensor.QQQQVGA: 80x60sensor.QQVGA: 160x120sensor.QVGA: 320x240sensor.VGA: 640x480sensor.HQQQVGA: 80x40sensor.HQQVGA: 160x80sensor.HQVGA: 240x160sensor.B64X32: 64x32 # (for frame difference image.find_displacement ()) sensor.B64X64: 64x64 # for frame difference image.find_displacement () sensor.B128X64: 128x64 # (for frame difference image.find_displacement ()) sensor.B128X128: 128x128 # (for frame difference image.find_displacement ()) sensor.LCD: 128x160 # (for LCD expansion board) sensor.QQVGA2: 128x160 # (for LCD expansion board) sensor.WVGA: 720x480 # (for MT9V034) sensor.WVGA2:752x480 # (for MT9V034) sensor.SVGA: 800x600 # (for OV5640 sensor only) sensor.XGA: 1024x768 # (OV5640 sensor only) sensor.SXGA: 1280x1024 # (OV5640 sensor only) sensor.UXGA: 1600x1200 # ( For OV5640 sensors only) sensor.HD: 1280x720 # (for OV5640 sensors only) sensor.FHD: 1920x1080 # (for OV5640 sensors only) sensor.QHD: 2560x1440 # (for OV5640 sensors only) sensor.QXGA: 2048x1536 # (for OV5640 sensors only) sensor.WQXGA: 2560x1600 # (for OV5640 sensors only) sensor.WQXGA2: 2592x1944 # (for OV5640 sensors only) explanation of related terms

1. Automatic gain

In order to output the standard video signal under different scene illuminance conditions, the gain of the amplifier must be adjusted in a large range. This kind of gain adjustment is usually completed automatically by detecting the average level of the video signal. The circuit to achieve this function is called automatic gain control circuit, referred to as AGC circuit. The sensitivity of the camera with AGC function will be improved at low illumination, but the noise will be more obvious at this time. This is because the signal and noise are amplified at the same time.

two。 White balance

Is to make white look white. The image taken in the fluorescent lamp room will look green, the scene taken under the indoor tungsten light will be yellow, while the photo taken in the sun shade will be inexplicably blue, the reason lies in the setting of white balance.

3. Color temperature

The temperature of the light-warm light or cold light (K: Kelvin) the color temperature of the cold light is high and blue, and the color temperature of the warm light is low and red.

The hue of light is achieved by the color temperature, which tells the camera whether the light is warm or cold.

4. Exposure

The larger the aperture, the more light passes through per unit time; the shutter, the time that light is allowed to pass through.

Aperture representation method: F + value, the smaller the value, the larger the aperture, and the adjacent difference is 2 times.

Shutter representation: 1 stroke 30s, direct use of time unit, adjacent difference 2 times

The combination of aperture and shutter forms the amount of exposure (although different combinations can achieve the same amount of exposure, but the effect of the picture taken is different)

Choosing the shutter according to the law of object motion, when shooting moving objects, it is found that the subject is often blurred, mostly because the speed of the shutter is not fast enough. When the shutter is fast, the amount of light may be reduced, and the color may be light, which requires more exposure to enhance the brightness of the picture.

Exposure mode is the mode in which the computer uses natural light source-shutter priority, aperture priority, manual exposure, AE lock and so on.

(1) shutter and aperture priority: the machine automatic photometric system calculates the exposure value, and then selects the shutter and aperture itself. the system determines the parameters of the other according to the shutter or aperture you choose (in the case of manually defining the shutter (aperture), the aperture (shutter) value is obtained by measuring the light of the camera.

(2) Manual exposure mode

It is necessary to manually adjust the aperture and shutter speed during each shooting.

(3) AE mode (Auto Exposure automatic exposure)

Aperture priority AE, shutter speed priority AE, program AE, flash AE and depth first AE

5.ROI

The area extracted from the image to be processed

The format of roi is (x, y, w, h) tupple.

The x coordinate of the upper left corner of the x:ROI region

Y coordinates of the upper left corner of the y:ROI region

Width of w:ROI

Height of h:ROI

two。 Basic operation of image

Related functions:

# 1. Get / set pixel image.get_pixel (XMagy) # get the value of a pixel # grayscale: get the grayscale value # RGB: get the tupleimage.set_pixel of (RMagnegPhoneb) # set the value of a pixel # grayscale: set the grayscale value # RGB: set tuple# 2. Get the width and height of the image image.width () image.height () image.format () # return grayscale or color image.size () # return image size byte# 3. The operation of the image # the image can be an iamge object read from a file, but the two must be of the same size and type (gray / color) image.invert () # inverted, the binary image 0 becomes 1 0image.nand (image) # and non-image.nor (image) # or non-image.xor (image) # XOR image.xnor (image) # XOR non-image.difference (image) # subtract another picture from one picture Absolute subtraction, often used for mobile detection 3. Statistics for working with images

Use ROI to select the region of the image that needs to be processed, and then return the corresponding value of the area to be studied according to the corresponding function.

Parameters such as image.get_statistics (roi=Auto) # roi bins must explicitly declare that image.get_statistics (roi=) # statistics.mean () returns the average of grayscale (int). You can also get it through statistics [0]. # statistics.median () returns the median of grayscale (int). You can also get it through statistics [1]. # statistics.mode () returns the mode of grayscale (int). You can also get it through statistics [2]. # statistics.stdev () returns the standard deviation of grayscale (int). You can also get it through statistics [3]. # staistics.min () returns the minimum value of grayscale (0255) (int). You can also get it through statistics [4]. # statistics.max () returns the maximum value of grayscale (0255) (int). You can also get it through statistics [5]. # statistics.lq () returns the first four fractions of grayscale (int). You can also get it through statistics [6]. # statistics.uq () returns the third or fourth fraction of the grayscale (int). You can also get it through statistics [7]. # related information about the three channels of # lab: lumbago, paramedicry, and mordenecrec-tive, langstdevmeme, langmin, Draw a picture

Mark the studied area.

# the color can be a grayscale value of 0255or a tuple of the color value (rmag _ r _ b), but the color must be explicitly indicated The default is white image.draw_line ((10, 10, 10 and 20), color= (255, 0, and 0) image.draw_rectangle (rect_tuple,color= (255, 0,)) # draw a line image.draw_line (line_tuple,color=White) # line_tuple= (x1) (x2), the line from (x1) to (x2) can be a grayscale value (0-255) It can also be a tuple# with color value (rect_tuple g.b) draw a rectangular frame image.draw_rectangle (rect_tuple, color=White) # draw a rectangular frame in the image # rect_tuple format is (x, y, w, h) # draw a circle (x, y, radius, color=White) # draw a circle in the image # circle + radius # draw _ cross (x, y, size=5) Color=White) # draw a cross in the image # size is the size on both sides # write image.draw_string (x, y, text, color=White) # write 8x10 pixels in the image # xy are coordinates Using\ n\ r or\ r\ nmoves the cursor to the next line # text is the string 5 to be written. Looking for color blocks

Introduction to the parameters of the find_blobs function:

Image.find_blobs (thresholds, roi=Auto, x_stride=2, y_stride=1, invert=False, area_threshold=10, pixels_threshold=10, merge=False, margin=0, threshold_cb=None, merge_cb=None) # threshold of thresholds color # roi region of interest # pixels of the minimum width in the x direction of the color block found by x_stride, such as pixels with a width greater than 10 in the x direction = 10 (default is 2) # the pixel with the smallest width in the y direction of the color block searched by y_stride (default is 1) # invert reversal threshold, use the color outside the threshold as the threshold to find # area_threshold area threshold. If the area surrounded by the color block is less than this value, the threshold for the number of # pixels_threshold pixels will be discarded. If the number of pixels of the color block is less than this value, it will be filtered out # merge True. Merge all overlapping blob to 1, this is to merge all blob, no matter what color, if you want to mix separately, call the find_blobs# margin boundary of different color threshold respectively, if set to 1, then two blobs if the distance is 1 pixel. Will also be merged # merge all all_blobs = img.find_blobs ([red,blue,yellow], merge=True) # single merge red_blobs = img.find_blobs ([red], merge=True) blue_blobs = img.find_blobs ([blue], merge=True) yellow_blobs = img.find_blobs ([yellow], merge=True)

The returned values of find_blobs are described as follows:

The find_blobs object returns a list of multiple blob, a blobs list contains many blob objects, the blobs object is a color block, and each blobs object contains a color block information.

You can use for to cycle through color blocks:

Blobs = img.find_blobs ([red]) for blob in blobs: print (blob.cx ())

Color block blob object

Common methods of blob:

Blob.rect () # returns the bounding box of this color block-- rectangular tuples (x, y, w, h), which can be used directly in image.draw_rectangle. Blob.x () # returns the x coordinate (int) of the outer frame of the color block, which can also be obtained through blob [0]. Blob.y () # returns the y coordinate (int) of the bounding box of the color block, which can also be obtained through blob [1]. Blob.w () # returns the width of the bounding box w (int) of the color block, which can also be obtained through blob [2]. Blob.h () # returns the height h (int) of the bounding box of the color block, which can also be obtained through blob [3]. Blob.pixels () # returns the number of pixels of the color block (int), which can also be obtained through blob [4]. Blob.cx () # returns the center x coordinate (int) of the outer frame of the color block, which can also be obtained through blob [5]. Blob.cy () # returns the center y coordinate (int) of the outer frame of the color block, which can also be obtained through blob [6]. Blob.rotation () # returns the angle of rotation of the color block in radians (float). If the color block is similar to a pencil, the value is 0 °180 °. If the color block is a circle, then this value is useless. If the color block has no symmetry at all, then you will get 0 °360 °, which can also be obtained through blob [7]. Blob.code () # returns a 16bit number, and each bit corresponds to each threshold. # A color block red-0001,blue-0010, if it is the blob-0011blob.count () obtained by the merge of red and blue # if merge=True, then multiple blob will be merged into one blob, this function returns the number of this. If merge=False, the return value is always 1. You can also obtain the area of the bounding box of the color block returned by blob.area () # through blob [9]. Should be equal to (w * h) blob.density () # returns the density of the color block. This is equal to the area of the color block except the number of pixels in the box. # if the density is low, then the target is not locked very well. # for example, to identify a red circle, the returned blob.pixels () is the number of pixels of the target circle, and blob.area () is the area of the circumscribed square of the circle.

Color threshold

# is the minimum and maximum values of LAB red = (minL, maxL, minA, maxA, minB, maxB) # if it is a grayscale image, only min and maxgrey = (min,max)

The acquisition of color threshold:

Framebuffers are images acquired by IDE in real time; image files are stored in SD cards.

Drag the slider to see the effect of the threshold directly, and the desired result is to turn all the target colors into white and all the other colors are black.

6. AprilTag implements tag tracking

AprilTag is a visual reference system that can be used for a variety of tasks, including AR, robot and camera calibration. The tag can be printed directly on the printer, while the AprilTag detector can calculate the exact 3D position, direction and id relative to the camera.

The amount of spatial position and rotation angle can be obtained for 3D positioning.

???

6. Template matching NCC

Normalized cross-correlation matching

???

Note:

If the size of the template image exceeds the flash built into openmv, an SD card is required.

The template created or imported must be in pgm format and cannot exceed the pixel size of openmv

You can use openmv to capture template images directly, but be sure to convert them to pgm format

7. Multi-template matching

Access multiple templates in advance.

8. Feature point detection

Feature point detection is more flexible than template matching. Template matching may fail as long as there is a slight difference in size and angle. But for feature point detection, at the beginning of running the program, the routine extracts the initial image as the feature of the target object, and kpts1 saves the features of the target object. By default, multiple scale sizes and angles of the target feature are matched, not just the size angle when the target feature is saved. The target feature can also be saved in advance, which is not done before because the matching degree is reduced due to different light characteristics of each run of the program, but after increasing the adjustment of exposure, white balance, and automatic gain value, it will relatively reduce the interference of light, and features can also be preserved in advance (the interference of light in the car should also be taken into account. )

The routine has reference value. )

Routines only support grayscale images. Note that the feature extraction algorithm used is FAST/AGAST algorithm.

9. Distance measurement

The first method is to use Apriltag for 3D positioning and ranging.

The second method is to measure the distance according to the proportion of the distance of the reference.

The value of the parameter k is obtained for the first time through a distance known from the actual distance, and then the later calculation is carried out.

OpenMV uses a monocular camera.

About monocular and binocular cameras:

Self-driving smart camera requires: far-sighted, high quality.

It has a long focal length, but the viewing angle becomes narrower, mainly because the reaction time is sufficient; black-and-white cameras are used for high-quality imaging.

In terms of ranging principle, the two cameras are completely different. The monocular camera needs to identify the target, that is, to identify whether the obstacle is a car, a person or something before ranging. On this basis, the distance measurement is carried out. The binocular camera is more like the human eyes, mainly through the parallax calculation of the two images to determine the distance. Of course, the binocular camera is not perfect, because it needs to rely on calculation for ranging. The biggest difficulty lies in the huge amount of calculation. The direct problem with this is that miniaturization is very difficult.

The routine has reference value. )

10. Scan code recognition

It's useless for the time being.

11. Simultaneous recognition of color and shape

The routine is to first identify the circle in the image, and then count the image color in the circle.

twelve。 Color template matching and simultaneous recognition

First carry on the color recognition, and then go to the recognized color area to find the desired template

Like 11, routines have reference value and how to combine algorithms needs to be considered. )

13. Distinguish different faces

Routines are available. )

Use LBP features to distinguish different faces. Here, we first select 20 faces with different expressions of the same person to save (tell the data collector whether the data collection is complete or not through different light display)

Then the face in the field of view of the current camera is identified and the person who best matches the current object is output.

These are all the contents of the article "how to achieve Image processing in OpenMV". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.