Shulou(Shulou.com)11/24 Report--

Lenna original picture (left), plus "electronic wrapped pulp" lenna (right) picture Source: electronic wrapped pulp picture cyber old / Magic Conch Lab reverse use of Wen Sheng AI, you can compress the picture.

Whenever I see interesting memes and outline pictures on the Internet, many people will secretly say in their hearts, "your picture is very good, but now it is mine."

However, when people upload the stolen pictures to other posts again, the network platform will often compress the pictures again, saving storage space and network bandwidth. If a picture is "stolen" too many times, it will be compressed repeatedly. Each time you compress, the quality of the picture drops a little bit, becoming a little more "paste"-and even greener and darker. The picture that has been "stolen" many times will turn green or even become a stalk, and even give birth to a series of scum-quality memes. The reason is probably: if a picture is interesting, it will be saved by more people, it will be compressed more times, and the quality of the picture will be worse; conversely, if a picture is of poor quality, it probably means it is very popular.

It is obvious that this stolen picture colorimetric card has also been stolen many times, which is a bit like a popular antique, which has been carefully played by countless people for a long time, and finally forms a glossy "wrapped pulp" on the surface. Dregs picture quality, the overall green color has become the network era of "electronic wrapped pulp", "cyber wrapped pulp". There are even some people who see those high-quality pictures and always feel that they have not been "baptized by the years" and say to themselves, "it's dry, hemp-dependent, and it's not round at all." plate it! " As a result, the electronic pulp-wrapped simulator was born.

Image source: electronic paste picture Cyber old / Magic Conch Lab picture why the picture turns green is actually a bug of the core code of the Android operating system. Android provides developers with an image compression interface that allows developers to easily compress JPEG images (that is, jpg images). However, in order to speed up the compression calculation process, the underlying implementation algorithm of this interface produces a bug in the process of color mode conversion.

The pictures we see on the phone screen store RGB information (Red red, Green green, Blue blue), which tells the screen how bright each red, green and blue sub-pixel should be, thus showing what the picture looks like on the screen. However, in the process of image processing, it is generally necessary to convert RGB information into YUV information (luminance, blue concentration offset, red concentration offset). Because the human eye is more sensitive to the luminance information represented by Y, the algorithm can focus on compressing UV information. In this way, the storage space occupied by the picture can be reduced as much as possible when there is little difference in the perception of human eyes.

Generally speaking, switching from RGB color mode to YUV color mode is slightly damaging, but the loss is small and does not allow the picture to run all the way to green. However, in order to speed up the conversion calculation process, developers improperly use bit operations, causing the data to be fetched downwards when converting from RGB to YUV. So in the repeated compression process, Y, U, V three values will continue to decrease, brightness Y value will continue to darken the picture, and UV will continue to decrease, will make the color continue to shift to the green direction (see below). Therefore, after many times of compression, the picture will turn green and darken.

When the color plane of UV is 0.5, the picture will turn green when you subtract from UV. Photo: the wikipedia issue was fixed in mid-April 2016. According to the release time of Android, the issue will not be resolved until Android 7 is released on August 22, 2016. (however, considering that there are delays in most of the follow-up of Android by mobile phone makers, the problem will be resolved even later. ) so the problem of pictures turning green only occurred when JPEG pictures were used on Android a few years ago.

Although some people like the "electronic package pulp" when the picture turns green and dark, the new image compression algorithm thinks that such a picture is more "interesting". But developers definitely need image compression algorithms that get the right results and don't turn green. In addition to fixing the bug of Android image compression interface, a number of efficient image formats have been developed. For example, the WebP format developed by Google and the HEIC format promoted by Apple all use more advanced image compression algorithms. Compared with the traditional JPEG format, these two formats have better viewing effect and take up less space at the same time.

Recently, AI, which generates images based on text, has become a hit. Most of these AI algorithms are based on the diffusion model, which can generate a picture from the text input by the user after a large number of text and image training. Although the details of the pictures are not perfect, AI, which can generate pictures day and night, shows great potential. Some people even sigh that if the development goes on like this, the painter will probably lose his job.

The picture is generated by the Wen Xin ERNIE-ViLG text picture model. The key words used are "editors who can't write manuscripts are fishing in the office, cyberpunk, oil painting". Source: Wen Xin ERNIE-ViLG Literary Life Picture, to what extent can this AI be developed in the future? instead, there is a Swiss developer, Matthias B ü hlmann, who has a "crooked brain"-can it be used for image compression?

Artificial intelligence compression Matthias B ü hlmann chose the free and open source AI:Stable Diffusion. It mainly consists of three modules: variational self-encoder (Variational Auto Encoder,VAE), U-Net and text encoder.

No text input is required to compress a picture, so Matthias B ü hlmann abandons the text encoder, while VAE encodes the picture into a potential spatial representation (latent space representation). Under the potential spatial representation, the image resolution is lower (from 512 × 512 to 64 × 64), but the color accuracy is improved (from 8 bits to 32 bits).

VAE can also decode the potential spatial representation of the image back to its original appearance. Even if the image is compressed again in terms of potential spatial representation, VAE can roughly restore the image to the appearance of the cost. In the end, Matthias B ü hlmann compressed an uncompressed 768kB image into 5kB. In the naked eye, the image is also compressed to this size, the compression loss of JPEG and WebP is very obvious.

From left to right are WebP pictures, JPG pictures, AI compressed pictures and original images. Image source: Matthias B ü hlmannAI electronic package pulp of course, but this compression algorithm is not perfect. Although this image compression algorithm can deceive the human eye, it has no obvious advantage over JPG and WebP in the two parameters PSNR and SSIM, which are used to evaluate the image quality objectively.

In the final analysis, this image compression algorithm still allows AI to "guess" what its original image looks like according to a compressed thumbnail, and the original picture is inevitably inconsistent with the original image in some details, which creates a new "electronic package pulp". Matthias B ü hlmann found that when dealing with faces and words, this image compression algorithm may produce weird (and sometimes even "Cesuru") effects on faces, and the text reconstructed by VAE is almost illegible. In other words, the previous compression algorithm will make the picture produce green "electronic package pulp", while the new era AI compression algorithm will make human face and text produce weird "electronic package pulp".

From left to right, there are WebP pictures, JPG pictures and AI compressed images. Notice the strange state on the face of the person in the red circle. Image source: the complexity of Matthias B ü hlmannAI also makes this image compression algorithm difficult to popularize. The traditional image compression algorithm is a set of fixed algorithms, which can be realized by lightweight program. While AI is famous for its huge amount of computation, this AI image compression algorithm requires sufficient 4GB space to store parameter files, and the decoding time is longer than other compression algorithms.

Therefore, the current image compression algorithm is not worth popularizing, even if popularized, it can not perfectly solve the problem of "electronic wrapped pulp". On the contrary, it will produce unimaginable, brand-new and weird "electronic wrapped pulp" because of the characteristics of AI.

However, the most popular electronic package is not the overall green and dark color, but layers of watermarks that are difficult to remove.

Reference link:

Https://pub.towardsai.net/stable-diffusion-based-image-compresssion-6f1f0a399202

Https://arstechnica.com/information-technology/2022/09/better-than-jpeg-researcher-discovers-that-stable-diffusion-can-compress-images/

Https://magiconch.com/patina/

Https://www.zhihu.com/question/29355920/answer/119088684

Https://github.com/google/skia/commit/c7d01d3e1d3621907c27b283fb7f8b6e177c629d

Https://en.wikipedia.org/wiki/YUV

Https://developers.google.com/speed/webp

This article comes from the official account of Wechat: global Science (ID:huanqiukexue), written by Wang Yu, revised by Chestnut

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.