Shulou(Shulou.com)11/24 Report--

This article comes from the official account of Wechat: ID:chuappgame, author: Zhu Siqi

The balance between cost reduction and efficiency, as well as a more differentiated future.

In the past year, Toule has written a number of articles about the gaming industry trying to use AI. Given the rapid development of AI technology, it is not easy to keep abreast of the latest situation, and it is also true for game makers. Up to now, there are still many decision-makers of companies who do not know how to introduce AI or whether AI can be introduced, and the allocation of manpower to study this matter does not seem to be in line with people's original expectations of technology to "reduce costs and increase efficiency".

Therefore, some people in the industry began to specialize in AI technology, and tried to cooperate with game manufacturers as an independent team to provide them with solutions including the establishment of reasonable workflow, independent training model, AI-aided conceptual design and so on. Like AI technology, this kind of team is so new that people don't have a comprehensive understanding of what they can do.

Akihiko Yoshikawa, who has worked in the game industry for more than 20 years and worked as a producer for seven years, is currently the head of an AI studio. His team focuses on the direction of AI art. Combined with past R & D experience, they have more in-depth experience and experience on the workflow of integrating AI art into game development, and have some practical cases of cooperation with manufacturers. To this end, Toule chatted with him in order to get a more comprehensive understanding of the actual situation and prospects of the integration of AI technology into the game development process.

The balance between cost and efficiency: could you first briefly introduce the personnel and division of labor in your studio?

Yoshikawa: the members of our AI studio are all from the Wild Temple, the largest AIGC community on Little Red Book, and there are currently 9 members. Among them, 3 are biased towards technology and 6 are biased towards design, all from all over the world and from all walks of life. At present, AIGC is the common interest and bond of team members.

One of our main tasks now is to help our partners train the model. Specifically, the so-called model is usually the large base model (also known as Checkpoint) needed by AI to generate pictures, or small models that can influence and control it (such as the common LoRA and LyCORIS). Everyone can use different methods-- generally by providing a large number of pictures and marking to AI learning, called training-- to tune the model so that it can be oriented to generate a certain type of graph according to the concept they want. In this process, simple training is often not enough, but also the complex integration of different models, just like hybrid plants, in order to finally get the model we want.

If we already have a satisfactory model, we can also help partners to design reasonable AI workflows according to the actual needs and the level of operators. For example, you should take which steps to use which parameters, what form of prompts, and even how to cooperate with traditional art tools to get the desired results. Because we have relatively many AI tools and always keep the iteration of the latest knowledge, we can help game companies to do overall optimization.

We will also directly help partners with the design, including clothing, scenes and characters. Now it is possible to add AI assistance to the traditional design to quickly achieve more than 70% of the final effect. For example, role setting, in the past workflow, no matter inside or outside the company to communicate, actually need to spend a lot of time to find information or design sketches, but now we can rely on the help of AI to greatly improve the efficiency of early communication and feedback. The rapid and high-quality design and molding in the middle stage will also save a lot of strength for the manual adjustment in the later stage.

For a typical complete drawing workflow, AI Studio can partially make up for the "lack of AI talent" of its partners: do you have any specific examples of helping game manufacturers train AI?

Yoshikawa: in the last three months, our studio has worked with three companies on five projects. Some projects stopped soon after they were done, while others succeeded after holding on for a period of time.

Let me start with a less successful example, because this example is a pity to me personally, and it can reflect some general situations.

At the beginning of the collaboration, we set a high goal: to train an omnipotent painting style model for the game, so that some of the characters behind them, and even the entire plot CG, can quickly run out with this painting style model. For example, give AI a line draft or a color draft, and AI can output a result that is very close to the finished product.

Specifically, our goal at that time was to use AI to run out the materials and fine patterns of the character's clothes based on a simple line draft. However, after trying many different techniques, we find that the painting style of vertical painting is not difficult to achieve, and the material of clothing can be well restored, but the pattern generated by AI has more problems in terms of fineness, structure and logic. Further refinement of the line draft can improve the problem, but it may outweigh the gain for research and development, as it is close to using manpower to take up the most complex work, and AI is only responsible for coloring. This is not the direction we want.

In short, after nearly a month of training and adjustment, we encountered all kinds of difficulties, and we also overcome them one by one. At first, our goal was to produce pictures close to 70 or 80 percent of the finished product, but the final result was only about 50% complete. At this time, R & D felt that the actual effect was quite different from what was thought at first, coupled with some other operational concerns, they decided to stop cooperating.

Toule: but do you think that project is actually promising?

Yoshikawa: yes. I thought this project was a pity because I had already seen the dawn at that time. I want to spend a little more time to see if AI can learn some specific patterns. Unfortunately, in the end, there was no more time to try.

AI takes a long time to learn to draw complex clothes and patterns (nothing to do with picture and text): it feels like this may represent a common phenomenon, not only because manufacturers are hesitant about the cost of investing in AI, but also the public opinion pressure that many companies will face after the project is launched.

Yoshikawa: I do feel that every head of the company has a different attitude towards AI technology. Some responsible persons will actively promote this matter, even if they encounter some difficulties and technical difficulties are very patient. In this way, the developers of docking will also have a higher degree of fit and often get better results.

However, if the person in charge has a different understanding of AI technology itself, or just have a try, and do not have the determination to really land, then the attitude from top to bottom will be more tangled. In particular, front-line workers may show obvious resistance.

Some people will feel that they are "working for AI", especially now that many companies will put "reducing costs and increasing efficiency" on the surface, and some employees will feel that the integration of AI will put them out of work. So there is an occasional "or forget it" emotion in communication and feedback. And this emotion can also spread from the bottom up.

Toule: you also know that the current environment in the game industry is relatively difficult, and many people will pay more attention to survival. Under this premise, can AI really play a "life-saving" role? Especially in terms of "reducing costs and increasing efficiency"?

Yoshikawa: I think it is more important to increase efficiency than to reduce costs. We should not put the cart before the horse. Because the design and operation of AI workflow is not as easy as people think.

I know that many practitioners, including game users, subconsciously think that AI is a cheap thing. Especially some two-dimensional games, mainly sell card noodles, users will feel that hundreds of thousands to draw a good-looking and powerful character, must be "big touch" hand-drawn, if the use of AI, the value of this picture will drop greatly. Coupled with the fact that there are many tutorials on the Internet that teach stupid training models, and even give ordinary people the use of functions such as "drawing with one click", it makes people think that AIGC is a very easy thing.

But if you have actually used AI workflows, you will find that simple tutorials are impossible to make available diagrams, rather than simply picking out a few prompts. Commercial landing requires accurate output, what is the composition, what rules should be followed in clothing design, and the hue and light should not deviate. For example, we train a model, how to judge the quality of the training set (material), how to add key parameters, we still need professional people to do it. Even if the trained model is directly given to beginners, the finished product can not meet the needs of commercial landing.

To generate practical images, it still requires a more professional AI training process and workflow design: but this does not seem to be in line with the current public impression of AI.

Yoshikawa: yes, there are cognitive misunderstandings, including some companies that come to us. Some people think that AI is now so simple that it can finish what you want in 5 minutes. After coming to see it, I found that this was not the case, so I flinched. There are some people who have tried the water a little bit and seem to feel at ease, thinking that "AI is really not good", ask the price again, and think that "it is better to find someone to paint cheaply."

Because many people know that the cost of art now accounts for most of the game development, but a large number of front-line painters are actually very low. At present, there is still a lack of talents in AIGC-some specialize in AI, but do not know enough about traditional painting knowledge; there are also people who know a lot about traditional painting knowledge, but do not know enough about AI. There are very few people who master both at the same time and have a certain aesthetic-such people are not cheap anyway, are they? Therefore, for those who regard "cost reduction" as everything, AI really cannot satisfy them at present.

Toule: so AI is not a good way to reduce costs?

Yoshikawa: I think the more far-sighted attitude is to use AI as a plug-in to think about how to use the existing human resource allocation to develop more and better content at the same time. This is very important because today's games consume a lot of content. You've been developing content for a long time, slow for a few months after launch, and run out in weeks or days. If you use AI, you do not need to produce drafts, color manuscripts, and line drafts step by step as before, but can quickly push the degree of completion to 70%, leaving much less manual revision work. In that case, the amount of art materials that can be produced in the same time can be greatly improved.

Achievement and Barrier Touch: maybe you can give some success stories?

Yoshikawa: the most successful case here is a project with a Chengdu company. The cooperation has been going on for more than 3 months, and we have not only successfully developed the role and scenario model above expectations, but also constantly optimize and supplement new cooperation content according to new requirements and new technologies.

Another successful example of a detailed and well-structured AI scenario is an original game. According to the original material and goal description provided by the other party, after constantly trying, we found the appropriate model combination to achieve the specific design style. We will confirm with our partners at every in-depth stage. We can see from these examples how we determine the style of brush strokes and composition from the beginning, and visualize the character step by step.

An example of using AI to attempt a painting style

An example of using AI to attempt composition

Sample touch music for enriching facial expressions and designing weapons: what do you think is the technical bottleneck in AI's image generation at present? What is the most influential one?

Yoshikawa: let me talk about the difficulties related to the model.

First of all, there is a shortage of materials that can be used in the training set. This problem is very common. For example, an IP game has achieved the second generation, then it actually has iterative requirements for the quality of art. The picture effect of the game now can not be the same as it was 5 or 10 years ago. Therefore, even if the previous work has accumulated a lot of material, a large part of it can not be used directly, cannot enter the training set, and can only be used to train some basic concepts at most.

Therefore, when setting up the training set, we certainly need to use various means to "make flowers" of the existing materials, and also need to make some deep integration with other excellent models in order to learn from their advantages. The process of drawing should be tuned accordingly before a good finished product can be made. Our core competitiveness lies in this, and we hope to have the opportunity to participate in more R & D projects with different needs in the future.

Another difficulty is that because of the complexity of the AI neural network itself, the whole training process is like a black box, and no one knows exactly how it is learned. At present, we only have a consensus on the general direction, but the specific training methods, which parameters should be set, there is no global unified standard, there are many different schools of opinion. Because of this, model training is also called "alchemy" or "alchemy" (laughter). Many things still need to be explored slowly in practice and adjusted in time according to the output effect. Like the two big guys on my side who are in charge of technology, they have trained hundreds of models, all of which have their own unique methodology.

After that, we have plans to train some customized large models that can be used in the vertical field of the game, to explore art styles that we have never seen before.

Toule: does the painting style of the picture have an impact on the success rate? For example, there is a big difference between two-dimensional and real-life painting style.

Yoshikawa: the required style has some influence on the difficulty. For example, the two mainstream styles, the real-life painting style and the two-dimensional painting style, have some differences in training parameters, but the theory is roughly the same, but the users' demand for the degree of restoration of real people is higher than that of the two-dimensional painting style. For example, sometimes a real-life model needs to be aimed at a particular character, but the picture is often not like, or only somewhat like. Because generally speaking, the facial judgment of a real person is very harsh. The second dimension is relatively much better. take the first sound in the future as an example, there is basically a water-blue ponytail, and when the eye color matches, everyone will naturally recognize this role, and the requirements for the position and proportion of facial features are not as high as the real person's standard. Of course, some of the SD iterative update techniques (Note: SDXL) contain several times more parameters than before, but I have not yet seen a human model that is very stable in terms of reduction, and most of them still have to draw cards repeatedly to get occasionally satisfactory results.

Viewers have a higher demand for picture reduction in the style of real-life painting. I estimate that AI will go through another technical iteration early next year to see if it can break through the limitations of the current live-action model training. In terms of two dimensions, the current technology can be said to be enough.

Toule: in addition to drawing, AI is also gradually applied to the video field. Are there any new achievements worth sharing in this area?

Yoshikawa: the recent development of AI animation (video) is also very fast. Since September this year, the AI circle has been studying animation like crazy. Didn't someone use AI to make a fake trailer for wandering Earth 3? Even alarmed director Guo Fan, invited the author to chat, can be said to be completely out of the circle. Now the new plug-ins and nodes derived from SD are also very suitable for original animation. In the first half of the year, you might have to use live video to do animation, but now you only need prompts and video references to make a fairly natural and smooth animation.

By the way, the best short films made so far, such as the "Stone and Paper Scissors" series, are shot with real people and then animated, and that effect is already amazing.

The "Stone and Paper Scissors" series is one of the best finished AI animations at present, but it still requires a lot of manpower in the later stage, of course, this field is not to the point of leapfrogging. It is unrealistic to want to sit in front of the computer and click a few mouse clicks to make a cartoon directly. I think it will take at least half a year or so before we can use AI technology to directly generate a complete short film for the end user.

At present, the application prospect of AI animation is very imaginative. For example, many two-dimensional games need to do the beginning or plot animation, if you first use AI to do some Demo or concept film is still very convenient. Communication is much more efficient than splicing other animations or drawing traditional static storyboards.

To say an interesting digression, there are many practitioners in other industries in our AI creator community, among which friends in the advertising industry welcome AI animation very much. They say that after using the proposal, the pass rate is frighteningly high, and customers are often "overjoyed" after reading it. In the past, projects that took a long time to talk about can now be won in a week, the key is that the time cost of animation production is not high.

AI tools are extremely efficient in conveying concepts in the future: finer differentiation: can manufacturers study AI on their own? I often come across people in charge who say that they have appointed one or two people on the team to study this, and the more radical ones will require all of them to learn.

Yoshikawa: a lot of small teams, especially independent developers, are really very positive, even radical, about trying AI. Because they have an urgent need to reduce costs, if they only send some experimental games on platforms such as Steam, they will not consider too much public opinion risk. Medium-sized companies will be more cautious, they will be more concerned about the reaction of players. But generally speaking, the learning of AI by small and medium-sized teams is still limited to the application level, and there are few people who really study how to train the model or even develop it to a certain extent.

As far as I know, only some large cross-disciplinary companies can organize specialized people or even departments to study AI. Big companies like Tencent have their own AI Lab and have developed a number of key technologies for image generation. But it is very difficult for small and medium-sized companies to have this spare power. On the one hand, it is difficult to find comprehensive talents in this field, on the other hand, it is also very difficult to organize learning, because there is no mature training system on the market, and full-time employees often have no spare time. We have also done training courseware before and found that we have to distinguish, supplement and organize a lot of knowledge by ourselves. And the iteration of knowledge is very fast, and some of the original functions may be replaced in just a month or two. So we have to spend a lot of time on learning the latest knowledge every day.

In this case, letting professional people do professional things is indeed a supplement to small and medium-sized teams.

Toule: isn't it unrealistic to say that AI will completely liberate productivity and make everyone a developer?

Yoshikawa: indeed, the threshold for AI creation is falling, but it is not as "stupid" as people think. For example, there is a real difference between a SLR camera and a mobile phone. For example, DALL-E3, which is very popular recently, after it is integrated into ChatGPT, users do not even need to know English, and can directly chat and describe with it in the natural language of Chinese to generate pictures. Of course, the painting is still not as good as the best AI drawing software from an aesthetic point of view, but it brings the threshold of use to an unprecedented level. Accordingly, if you want to produce particularly high-quality material, or if you want to deeply integrate AI into the game development process, you still need to systematically learn some advanced biographies and training skills; the needs of small personal needs and large-scale commercial projects are completely different.

The combination of DALL-E3 and ChatGPT further lowers the threshold of AI drawing: it sounds like it will eventually be divided into "professional AI" and "non-professional AI".

Yoshikawa: in my opinion, AI creation will eventually be a more polarized field. The low threshold will be lower and the high threshold will be higher and higher. I even want to learn more about some traditional arts and skills, such as Adobe's series of drawing and video production tools. Traditional companies like Adobe are also actively embracing AI. In the future, some people who have strength in traditional art and experience in traditional tools will still be able to stay ahead.

I think that in the future, the field of AI creation is likely to be divided into three directions: first, the vigorous development of creation based on the development of the latest technology of AI, the second is the low-threshold application of the general public, and the third is that traditional professional players make use of a solid foundation to use AI to improve their original work efficiency and effectiveness. I know there is still a lot of controversy around AI, but in the face of new things and technological developments, some people have the fear of being left behind, while others see this change as an opportunity.

I have always regarded the rapid development of AI as a major turning point in life and a rare opportunity, and I also hope to have the opportunity to have more and longer-term practical exploration in the field of games with more friends with similar views and views.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.