Shulou(Shulou.com)11/24 Report--

This article comes from the official account of Wechat: ID:chuappgame, author: Zhu Siqi

Recently, the works produced by "AI painting" seem to have become an Internet craze. These paintings, which are stunning at first glance and full of atmosphere, are often impressive when they are carefully considered but do not see why. When people learned that these paintings were painted entirely by AI, there was a sense of surprise and crisis.

Some of the finished products of AI painting are really amazing, although it involves complex machine learning algorithms, the principle of AI painting is not complicated to explain. Users can enter a variety of keywords related to style, theme and atmosphere on the platform. AI searches and studies in the vast number of resources and materials on the Internet according to these keywords, and finally mixes and splices into some paintings that meet the requirements.

Take Wombo Dream, which is very popular on the mobile side, as an example. It is an AI painting tool for public users, with a highly visual interface and simple operation. I chose "Ghibli style" in the interface, entered the theme "Dream in Dream", and uploaded a picture of a boat in the lake as a reference. Soon, AI gave me a finished product similar to the Ghibli animation background.

Mobile Wombo Dream for the public is easy to operate, and the speed of generating pictures is very fast. In addition, there are many more professional image production AI with different emphasis. For example, Google's Disco Diffusion has more adjustable parameters, which can set the size and quality of the picture in detail, and the requirements in the description are more detailed, covering the painter's style, tone, theme and many other aspects. This makes some of the pictures it produces seem to be the work of a live painter. Midjourney imitates the needs that various users may put forward for live-action painters, and is the darling of many artists engaged in digital painting. Stable Diffusion is an open source software, and many game developers favor its broad application prospects. Artbreeder has been popular on the Internet for several years because of its simulation of real-life avatars.

Although this technology is not new in recent years, with the development of AI technology and more materials brought by the Internet, the quality and efficiency of this kind of output are increasing with the naked eye. At the beginning of this year, no matter how well it was, some AI drawing tools needed about an hour to draw a picture, but now this time has been shortened to more than ten seconds.

If ordinary users use this kind of AI to paint to satisfy their illusions about scenes, game developers see more potential in it.

Toule found several developers who are more or less exploring and experimenting with their games to find the possibility of using this technology.

Liberating the historian of Justice from mechanized labor (a pseudonym) is the developer of a strategy game. When he got in touch with him, he joked that he was "preaching" in his moments and called on his peers to try AI painting. He has started to use it in the project himself, and it feels very easy to use. When it comes to the advantages of AI drawing, what he mentions most is "large quantity" and "high speed", especially when finding the conceptual direction in the early stage of development, the efficiency is much higher than that of manual drawing.

At this stage, what the planner wants to see most is the overall effect and big feeling of the picture, but many details can not be determined. For example, if they want to play a game with a dark fantasy background, there may be Gothic style, medieval style, industrial style and so on. When it is impossible to judge which element combinations and color matches work best, repeatedly mentioning the demand may cause unnecessary consumption of manpower. "AI painting is different, we can constantly adjust the keywords, let AI produce 100 pictures at a time to see the effect." The historian said, "if I let the company art or outsourcing send out 100 pictures first to see the effect, they should just throw me out the window."... "

It's the same with character design. "if I want to design an alien image, I can let AI look at the effect first, and then choose one of them for a more detailed design." He continued, "you can't let a live artist bear the cost of trial and error, but AI has no regrets."

Some artists have tried to use AI to generate pictures and participate in digital art competitions. In addition to providing inspiration for award-winning works, the pictures generated by AI can also be used as reference materials. Xing Feng (a pseudonym) is a producer of word games, and there are a large number of paintings involving clothing wrinkles, fabric texture and elegant dynamics in his games.

"you can't make painters think about this kind of thing out of thin air. They usually have to look for a lot of reference, and it's really troublesome to find a reference with the right angle and posture," says Xing Feng. "it's much more convenient to enter keywords and directly use the picture from AI as a reference, and you don't have to consider potential dynamics, posture and other infringement problems, because AI is made for you out of thin air. What's more, the 3D rendering done by AI is generally more correct in lighting, so you don't need to make it up by yourself. "

However, at present, it is difficult to use the concept map and character map generated by AI directly. "in the end, we have to let the painter to draw, after all, the things generated by AI are very broken, it does not have a unified design, directly used, does not quite meet the needs of our project." Xing Feng said.

At present, the finished products that are closest to the directly available AI paintings are mainly focused on some relatively simple graphic design and 3D material mapping. Once this kind of picture is generated, the developer only needs to do some simple processing such as trimming and color adjustment before it can be applied to the game.

I tried to use AI to generate a realistic landscape. As a less important background for some word games, this effect is close to the direct use of simple graphic designs, including various flag icons, badges, legal arrays, and so on. If they are not closely linked to the game's worldview and plot, and do not need to be specially designed, they can be handed over to AI for mass production.

The games developed by historians use many medal patterns and flag icons for players to choose from. This kind of art material does not occupy an important position in the game. If a painter is allowed to design and draw manually, a person can only draw at most a dozen a day, and the input and output are very uneven. "with AI painting, if you draw medals, you can produce thousands more in a night and 10,000 icons a day. After getting the finished product, you can cut the picture and pack it with a script, which is many times faster than manpower." This is how he describes the huge gap in efficiency.

He further calculated the drawing efficiency of AI under different devices: "what we mainly rent now is Google Cloud." The picture produced by the AI drawing tool Stable Diffusion is based on a picture of 512 × 512 pixels. If you take 50 steps in Stable Diffusion, it will take about 15 seconds to use T4 graphics card and 64 steps to take about 20 seconds. The more steps, the longer the time. If you have a V100 graphics card, it will be out in 50 steps in a few seconds. The number of drawings can be set by yourself. The computer's video memory determines whether it can generate a large enough picture, and GPU determines how fast it is. "

As for overhead, Stable Diffusion itself is free and open source software, Google Cloud costs $10 a month, and a higher configuration costs only $50.

Xing Feng also mentioned some patterns that are close to one-time use and do not play an important role in the game. "if you want to be an orc soldier and be beaten back in less than a minute, the tattoo on his body can be generated by AI. There are all kinds of magic array, which is actually very troublesome to design, but not very useful in the game, and it is easy to find material directly and easily encounter copyright disputes." AI painting can liberate painters from this kind of mechanized work.

But not all graphic patterns are suitable for AI, and ultimately it depends on the needs of developers. "for example, the embroidery on the clothes, the patterns on the cloth, which are related to the worldview, culture, camp and so on in our games, are quite important." Xing Feng said, "so we still design these manually and will not use AI to draw them."

Developers feel the least burden to use is the 3D mapping materials generated by AI, especially things like stone patterns, flowers and trees that are universally applicable. If all they need is a realistic background, which is lifelike, has no special style, and does not have the natural appearance of copyright, it can be counted as the most convenient material, and even after it is generated, it does not need much manual correction.

Independent developer Ye Lanzhou, who has also been following AI painting, further explained the key to the current manufacturers' use of AI painting tools: "AI is now a black box of chaos. Before the picture comes out, it is difficult to know what results you will get, and it is not clear what role each keyword will play in it."

Now developers' debugging of AI painting is more like completing the work of "keyword hunter". "in fact, the core is two points: finding the key phrases corresponding to your game style, and screening the pictures that are suitable to play in the game."

He took a portrait of a white-haired woman very close to the original painting of the game as an example. To generate this picture, you need to add a variety of other restrictions in addition to the basic keywords "girl with super long hair, hair becoming white snow" (Super long-haired girl, snow-white hair). For example, "rich details", "digital painting", "Artstation", "conceptual design", "smooth", "focused and clear", "illusory engine 5", "8K" and so on. Finally, the names of famous painters such as "by Artgerm and Greg Rutkowski and Alphonse Mucha" should be added to adjust the style of the picture.

Developers have to adjust keywords many times to produce the effect they want. "so keywords, like spells, need to be constantly tried to change to see how to best express their ideas." Ye Lanzhou said. Generally speaking, the process of adjusting keywords is between 15 minutes and 1 hour according to the complexity of the map.

Can not replace human beings, and there are problems at present, AI painting technology is still in constant change and development, can be described as "changing with each passing day". Even if you compare the AI painting at the beginning of this year with what it is now, the effect is very different.

Ye Lanzhou recalled several impressive examples. Around March or April this year, using DDV5, an AI painting tool, pictures painted with keywords such as "druid", "Pablo Amaringo" and "Artstation fashion" were impressive enough in terms of atmosphere and picture elements, but it was almost impossible to generate a normal face at that time. A few months later, DALL-E2 has been able to generate more accurate facial features, and Stable Diffusion in the picture performance, please climb another tall building.

About 5 months ago, DDV5 generated the effect of druid-themed pictures.

After several months of development, the image quality of both DALL-E2 (left) and Stable Diffusion (right) has greatly improved. another example is that Midjourney generated fish-themed images with the same keywords in two months.

"so the rate of evolution of AI is really amazing, it's like Superman's upheaval, changing every day or even every hour." Ye Lanzhou made such comments.

Two months later, Midjourney generated different images for the same group of keywords, just like all the new things that attracted people's attention. AI painting brought a lot of controversy as well as amazing.

First, there is a long-standing concern: will the development of AI, and even machine learning in a broad sense, eventually replace human labor? The problem is metaphysical and complicated to discuss. But from the developers' most pragmatic point of view, the answer is relatively simple: for now, "far from being replaceable."

As mentioned earlier, AI is still just a tool, which can only assist the repetitive work of art, and is used to liberate rather than replace painters. "in other words, they replace uncreative work." "the screening of the material generated by AI, as well as the final design and painting, still have to be done by real people, but it simplifies some of the steps that will make the real-life artist bald," Xing said. "

The historian said that according to his experience of "selling Amway" in his peer circle, planners and producers were the most motivated to try AI painting, while painters were the least motivated, but not because they thought they would be replaced. "they are not interested in AI painting itself, but are most concerned with examples of AI-assisted painting."

In addition, even if AI can draw all kinds of exquisite pictures, in practical application, developers are still in the stage of exploration and research, which is still a long way from the real industrialization and large-scale application.

First of all, industrial developer tools often need an interface with other commonly used tools to facilitate creators to copy and edit material between different software at any time, but most AI painting tools lack this function. Many people have noticed this problem and are working on it. "for example, the Stable Diffusion we use is now open source, and there should be more optimization and tool access in the future. There are already videos connected to Photoshop." The historian said.

The effect of the demonstration video mentioned by the historian is really surprising. After connecting Stable Diffusion to Photoshop, the painter can almost always fill the blank on the canvas with the content generated by AI by typing keywords, and then directly modify it with a paintbrush, which seems to be a more responsive drawing assistant.

The buildings and plants on the screen are first generated with AI, and then manually drawn and adjusted.

Xing Feng hopes to have more customized content for developers, such as a customized material library. "if we originally made a special stylized game, and then we want to do a similar one, can we upload our own materials to train AI and let AI draw for us?"

Ye Lanzhou calls this process a small-scale machine training for a painter or a particular painting style, but most eager game manufacturers are not yet able to do it. "but people are already doing research, for example, now the developers of Stable Diffusion say that the training package will eventually be compressed to 100MB... so we can probably use it for running training." This trend is also developing very fast. The training package downloaded by historians these days has changed from 7GB to 2.7GB.

In addition to the metaphysical discussion of "whether AI will replace human beings", there are indeed some grey areas in AI painting that have not been resolved. These problems cast a shadow over its commercial and industrial application prospect.

In general, developers are not too worried about copyright problems with AI-generated patterns, poses, and so on. "because what AI generates is' grow', just like a tree will not grow two identical leaves, even with the same set of keywords, AI will draw different things for different people." Xing Feng said.

But some copyright concepts are more vague, such as painting style. This is the focus of controversy in AI painting at present. Because many keywords in AI training are related to the style of painters, users can enter not only the names of existing famous painters, but also relatively stable business genres such as "Ghibli" and "Disney". As a result, painters will inevitably feel offended when the finished products painted by AI become more and more similar to the style of these reference pictures.

At the end of August, a number of Japanese painters announced on social platforms that AI was banned from learning their own painting style. Mimic, the AI painting platform involved in the protest, featured two-dimensional avatars, and the comparison pictures they released were indeed difficult to tell from the true at first glance.

Users upload existing works of painters, and Mimic uses AI to generate new avatars with similar painting style. although Ye Lanzhou pointed out from the developer's point of view, if it is really to be used in the game, the characters generated by AI still have many defects in facial expressions and image continuity (meaning that the characters do not have "one person with a thousand faces" in different scenes), but only from the static picture. AI's strong ability to imitate seems to put the painter's efforts for more than a decade in jeopardy.

Because of the controversy, Mimic has now announced the termination of the test and said it will include measures to protect the painter's copyright in the official version.

Some conceptual designers of European and American game manufacturers have also come forward to denounce such AI tools as "copyright washing machines", saying that their imitation of the painting style is empty and flashy. Some designers believe that it is possible that the industry bosses will not be replaced by AI, but novice painters may not be.

In addition, portraits in AI paintings will also involve the issue of portrait rights. Some netizens pointed out that if the data used for AI training is crawled from the Internet, if it wants to be commercial, it must make sure that it does not climb a website that forbids climbing data. Normally, crawling sensitive data such as human face requires authorization and informed consent.

At present, the major AI painting platforms on the use of images are vague. Some only generalize that pornographic content cannot be produced, pictures cannot be used for harmful purposes, or they can only be commercialized by paying members. The agreement between DALL-E2 and Stable Diffusion states that, in theory, the copyright of all drawings belongs to the user, but if they are too similar to other existing images, AI tools will not be held responsible, basically at the user's own risk.

"this area is still a gray area of the law," Xing said. "one or two big lawsuits may be needed in the future to promote awareness and change."

The trend is irresistible to sum up, AI painting is regarded as a potential tool by game developers, but whether it is due to existing technical limitations or copyright disputes, it is still in the exploratory stage and cannot be used on a large scale, and it is difficult to replace human labor in the short term.

The game Source of Madness, which is already on Steam, is the only known example of intensive use of this technology to generate monsters and scenes entirely from AI. But it itself is Cesuru theme, can accommodate a variety of strange monster images, the final visual effect also seems to be a distance from the usual sense of commercial games.

The Cthuru style of "Source of Madness" matches some of the "I don't understand but think it's awesome" images generated by AI. Anyway, the trend of AI painting is hard to ignore, and developers will not reject new production methods that can reduce costs and improve efficiency. Many independent game developers are happy to embrace and study this change. "after all, it's hard for you to stop the pace of technological progress." Xing Feng said. AI painting is like a crane owned by modern construction workers, liberating the three thousand drudges who had to carry stone themselves in ancient times, but the workers themselves cannot replace it. As for the other frictions and potential disputes that follow, they may be the pains that all new things will face at the beginning of their popularization.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.