Shulou(Shulou.com)11/24 Report--

Netizens' attempts to generate pictures on AI are even more violent than they thought.

We all know more or less about the various AI that used to turn ordinary photos into two-dimensional painting styles:

Or let AI create a real beauty:

But these are static pictures, and some people are wondering if AI can come up with some movies to watch.

Recently, Corridor, a well-known foreign video special effects team, has done a hard job. What AI wants to create in their hands is no longer a painting, but to make the picture into a completed animation. For example, the following clip is actually drawn by AI according to a real person:

In the animation, the two princes have a fierce duel, and the way is "rock and paper scissors"! Although the sand sculpture, but this picture is really warm-blooded:

Can't you see that AI did it? You can compare live shots with perfect synchronization and seamless switching:

The picture is clean, the expression is vivid, there are no ghosts and animals, and even the painting style is a bit classic. It's as if an animation team uses a paintbrush to draw pictures of real-life characters. But in fact, Corridor only uses four or five people, and they can't make animation at all.

As you can imagine, AI first allows you to become a painter from a zero base, and now it's possible to become an animator, which sounds a little more relaxed and free to create, which is a bit of science fiction.

So, how exactly did they do it?

The whole step is simply to put it in a live-action role, and then ask AI to animate the live-action video. Of course, this process is easy to say and complicated to do.

The tool they use is the Stable Diffusion we introduced earlier, and the keywords are given to AI, which can help generate various styles of images according to the templates you add.

The first problem to be solved if you want to make a video is the instability of the picture.

Because the picture of each frame is different, the picture generated by AI will also be significantly different. For example, the previous netizens produced "Zelda style Jurassic Park", the work has been quite smooth, but you can still see that the characters themselves are changing rapidly and their hairstyles are constantly changing.

So Corridor first tried to fix the picture style in a few frames, so as to make the picture generally stable, not to change too much.

But at this time, the character style is still constantly changing, for example, in each picture, the character's eye size and painting style will change, and sometimes even wear glasses and sometimes not wear glasses.

In order to solve this problem, it is necessary to use a fixed model to unify the painting style, so that AI does not think too much.

So they trained AI with a Vampire Hunter D painting, using screenshots of various facial close-ups, limbs, bodies, and so on:

In this way, the generated pictures are based on this animation, and the eyes will not go up and down.

At the same time, we also have to solve the problem that the details of the face are still unstable, and the way to deal with this problem is to train an actor's own real-life model at the same time. It requires actors to put on costumes and then take crazy pictures from multiple angles to make material for AI.

The production team should not only lock in the style of the animation for AI, but also lock in the appearance of the actors, greatly reducing the randomness of the generated pictures.

Trained AI can also generate images of actors, but the fingers are occasionally incomplete.

Then we need to continue to train, use the photos of actors to generate animated images, iterate constantly, and finally have a more satisfactory and stable animated image.

With the blessing of multiple training, the effect of animation becomes more ideal, and the unity of the picture is basically stabilized.

Using video special effects technology to reduce flicker, the picture becomes more stunning, and the characters are full of emotion, coherent and smooth:

Add some cartoon special effects, you say this is a clip from some animation is no problem.

Next, they will design the storyline and background for their animation: "the king is dead and the two princes will compete for the throne with stone scissors and paper." the plot sounds like AI.

With the technology, the story script, and then start the formal production. For example, at the beginning of dubbing, although there was no picture, several brothers roared until they were hoarse.

After that, he made a costume from the Internet, and the two leading actors put it on and went to film.

After the live video is shot, it will be handed over to AI to process the picture, and various details will be added to adjust the picture.

So all kinds of performances in reality are animated by AI bit by bit.

Pick out the green screen and the character material will be basically completed.

Then you need to make the background. So they directly use the material from the illusory engine to look for shot pictures from the material:

Then feed the picture to AI and regenerate it into a lot of background images.

Then there is the integration and further processing of the video, including the lens, clothes, light changes and so on.

For example, let the background pan, so that the lens looks like it is spinning, and there is a sense of dynamics:

The characters are combined with the background, and finally output into animation:

In the animation, even some micro-expressions can be restored compared with real people. AI draws just like the throbbing of the corners of the eyes, the convulsions of the corners of the mouth, and what the real person does:

The stone scissors and paper make it look like a kill trick, which looks both awkward and fun:

This kind of production has completely deviated from the feeling of AI drawing.

The animation has been viewed more than 2 million times since its launch, and has sparked a heated debate.

Most people once again believe that AI will trigger a new round of revolution, and the feeling of unemployment has enveloped many netizens.

But others point out that the cost of AI is no less than the workload of painters.

First of all, Corridor is a very professional special effects team, everyone has outstanding abilities. In the past, videos of Boston powered robots killing humans were made by them, and at that time, many people could not tell whether it was true or false.

So it is not so much that AI is generated well, it is better for them to shoot well themselves, and it took two months to have a wonderful performance of animation, which is a bit far away for ordinary people to do so.

At the same time, the painting style is also locked, because their animation is based on real people, so the painting style is also close to realism, if the painting style is exaggerated a little bit, the effect may not become ideal.

The final argument is the issue of copyright, and many painters are still very resistant to the fact that their works are fed to AI.

But it is not realistic to completely break away from the artist's animation, for example, this time the production still uses the "Vampire Hunter D" screen for training, and it is still two concepts to let AI draw entirely on its own.

It has also been pointed out that as long as the tools are easy to use, animation companies will take care of the copyright themselves, and only ordinary workers will be injured in the end.

Then again, I am still looking forward to the growth of AI.

To generate a picture, change the style of the picture, generate a real person, and turn a real person into an animation, AI is obviously playing more and more. In a few days, the voice may not be recorded by a real person, and the video will no longer be filmed by a real person.

According to this evolutionary trend, if you feed a TV series to AI in the future, you may be able to easily produce an animated version. At the thought of this, I have to look forward to it.

Reference:

Https://twitter.com/t_takasaka

Https://www.youtube.com/@Corridor/videos

Https://www.youtube.com/watch?v=GVT3WUa-48Y&t=95s

Https://www.youtube.com/watch?v=ljBSmQdL_Ow&t=296s

Https://www.youtube.com/watch?v=_9LX9HSQkWo&t=3s

Https://www.corridordigital.com/

Https://twitter.com/CorridorDigital

Https://twitter.com/CorridorDigital/status/1630330915284197376

This article is from the official account of Wechat: mad Pill Research Institute (ID:kuangwanplay), author: mad Maru

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.