Shulou(Shulou.com)06/02 Report--

This article mainly explains "Python based on domain correlation to achieve image enhancement method is what", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in-depth, together to study and learn "Python based on domain correlation to achieve image enhancement method is what" it!

Introduction

When the depth neural network model is trained on the image, the model can be better generalized by training more images generated by data enhancement. Common enhancements include horizontal and vertical flip / shift, random rotation at a certain angle and direction (clockwise / counterclockwise), brightness, saturation, contrast, and scale enhancement.

One of the most popular image enhancement libraries in Python is albumentations, which can easily enhance images through intuitive functions and excellent documentation. It can also be used with popular deep learning frameworks such as PyTorch and TensorFlow.

Domain-related data enhancements

intuition

The idea behind it comes from images that may be encountered in reality. For example, enhancements such as snow or raindrops should not be found in x-ray images, but chest tubes and pacemakers are enhancements that can be found in x-ray images.

Where did this idea come from?

Changed the way Roman (@ nroman on Kaggle) enhanced the SIIM-ISIC melanoma classification competition. One segment of the enhancement is as follows:

Original image (upper left) and hair enhanced image (upper right)

This paper does use his enhancement function in our model training, which helps to improve the cross-validation (CV) scores of most of our models.

What I want to say is that this form of enhancement may have played a key role in our final ranking! Since then, the idea of using hair (or general artifacts) to enhance image data has been very close in my subsequent competitions and applied as much as possible.

In particular, this method has been popularized and applied to the global wheat detection and cassava leaf disease classification challenge.

Insect enhancement

As the title shows, this method includes using insects to enhance the image. This can be a natural setting in the data because insects are usually found in the air or on the ground.

In this case, bees are used as the preferred insect to enhance leaf images in cassava and global wheat testing competitions. The following is an example of the appearance of an enhanced image:

An enhanced image of bees flying around leaves

We can also use the mask form to cause black dots in the image (similar to falling off in the album), that is, bees without color and black:

Enhanced image of black / black bees flying around leaves

The following code, written in Albumentations style, allows the enhancement function to be easily used with other enhancement functions from the Albuments library:

From albumentations.core.transforms_interface import ImageOnlyTransform class InsectAugmentation (ImageOnlyTransform): "impose the image of the insect on the target image-parameter: insects (int): the maximum number of insects insects_folder (str): The path to the insect picture folder "" def _ _ init__ (self Insects=2, dark_insect=False, always_apply=False, pendant 0.5): super (). _ init__ (always_apply, p) self.insects = insects self.dark_insect = dark_insect self.insects_folder = "/ kaggle/input/bee-augmentation/" def apply (self, image, * * kwargs): "" Parameter: image (PIL Image): draw an insect image. Returns: PIL Image: images with insects. "" n_insects = random.randint (1, self.insects) # in this example I use 1 instead of 0 to illustrate the enhancement if not n_insects: return image height, width _ = image.shape # width and height of the target image insects_images = [im for im in os.listdir (self.insects_folder) if 'png' in im] for _ in range (n_insects): insect = cv2.cvtColor (cv2.imread (os.path.join (self.insects_folder, random.choice (insects_images)), cv2.COLOR_BGR2RGB) insect = cv2.flip (insect) Random.choice ([- 1,0,1]) insect = cv2.rotate (insect, random.choice ([0,1,2]) h_height, h_width, _ = insect.shape # width and height of insect image roi_ho = random.randint (0, image.shape [0]-insect.shape [0]) roi_wo = random.randint (0) Image.shape [1]-insect.shape [1]) roi = image [roi _ ho:roi_ho + h_height, roi_wo:roi_wo + h_width] # create mask and inverse mask img2gray = cv2.cvtColor (insect, cv2.COLOR_BGR2GRAY) ret, mask = cv2.threshold (img2gray, 10,255) Cv2.THRESH_BINARY) mask_inv = cv2.bitwise_not (mask) # the area now blackened is the insect img_bg = cv2.bitwise_and (roi, roi, mask=mask_inv) # only the insect region is selected from the insect image. If self.dark_insect: img_bg = cv2.bitwise_and (roi, roi, mask=mask_inv) insect_fg = cv2.bitwise_and (img_bg, img_bg, mask=mask) else: insect_fg = cv2.bitwise_and (insect, insect, mask=mask) # add dst = cv2.add (img_bg, insect_fg Dtype=cv2.CV_64F) image [Roi _ ho:roi_ho + h_height, roi_wo:roi_wo + h_width] = dst return image

If you want to use the black version, please set dark_insect to True.

Enhancement using needles

In this method, needles are used to enhance the image, for example, an x-ray image. The following is an example of the appearance of an enhanced image:

Enhanced image with needle on the left side of the X-ray film

Similarly, we can use a black version of the needle to produce the following enhanced image:

Enhanced images with black / black needles on both sides of x-rays

The code snippet as the above extension module is as follows:

Def NeedleAugmentation (image, n_needles=2, dark_needles=False, pendant 0.5, needle_folder='../input/xray-needle-augmentation'): aug_prob = random.random () if aug_prob < p: height, width, _ = image.shape # width and height of the target image needle_images = [im for im in os.listdir (needle_folder) if 'png' in im] for _ in range (1 N_needles): needle = cv2.cvtColor (cv2.imread (os.path.join (needle_folder, random.choice (needle_images)), cv2.COLOR_BGR2RGB) needle = cv2.flip (needle, random.choice ([- 1,0,1])) needle = cv2.rotate (needle, random.choice ([0,1,2]) h_height, h_width _ = needle.shape # width and height of pin image roi_ho = random.randint (0, abs (image.shape [0]-needle.shape [0])) roi_wo = random.randint (0, abs (image.shape [1]-needle.shape [1])) roi = image [Roi _ ho:roi_ho + h_height Roi_wo:roi_wo + h_width] # create mask and inverse mask img2gray = cv2.cvtColor (needle, cv2.COLOR_BGR2GRAY) ret, mask = cv2.threshold (img2gray, 10,255, cv2.THRESH_BINARY) mask_inv = cv2.bitwise_not (mask) # now the black area is img_bg = cv2.bitwise_and (roi, roi) Mask=mask_inv) # only the needle area is selected If dark_needles: img_bg = cv2.bitwise_and (roi, roi, mask=mask_inv) needle_fg = cv2.bitwise_and (img_bg, img_bg, mask=mask) else: needle_fg = cv2.bitwise_and (needle, needle, mask=mask) # add dst = cv2.add (img_bg, needle_fg Dtype=cv2.CV_64F) image [Roi _ ho:roi_ho + h_height, roi_wo:roi_wo + h_width] = dst return image

Please note that the above is not in Albumentations format and cannot be directly applied to regular Albumentations enhancements. Some adjustments must be made to make it the same format as in the above insect / bee enhancements. But the change should be small!

Similarly, if you want to use the black version, set dark_Piners to True.

Experimental results

On the whole, the results of local CV were improved, and most of them were slightly improved (such as 0.001mur0.003). However, in some cases, the use of this enhancement method "fails" in the training process.

For example, in the global wheat testing competition, the task involves detecting the head of wheat, that is, the target detection task. Although a large number of super-parameter adjustments have been made, the honeybee enhancement using the original honeybee results in a great fluctuation in the loss of training verification.

Although the use of enhancers does improve CV, it can be said that this is a lucky opportunity. Using enhancements that retain only black pixels have proved to be stable in all areas of the application. In particular, the promotion of CV is substantial and consistent.

So far, no reason has been found for this training result between different epoch due to the increase in the number of bees, but there is an assumption that the color of the bees is close to that of some wheat heads, so the detection algorithm is "confused", which then captures the wheat head and the nearest bee in the same bounding box.

This has been observed in some bounding box predictions, but there are not enough observation cases to be sure that this assumption is correct. In any case, you should also consider whether the image attributes (colors) have a distribution close to the target, such as wheat head.

On the other hand, the enhancement using the needle (the original and its black / black version) proved to be relatively stable. In this example, although the predicted target is similar in color distribution, it may have obvious characteristics (for example, the chest tube looks very different from the needle), so the classification algorithm does not confuse whether the needle is the correct target.

Thank you for your reading, the above is the content of "what is the method of Python based on domain correlation to achieve image enhancement". After the study of this article, I believe you have a deeper understanding of what is the method of Python based on domain correlation to achieve image enhancement, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.