Shulou(Shulou.com)11/24 Report--

Before the internal testing of Runway's Gen-1 was finished, Gen-2 was released with lightning. This time the effect is even more explosive, and AI is so big that it is one step closer to eliminating human beings.

Yesterday, the perfect couple generated by Midjourney hit the Internet, and everyone imagined it in the message area one after another. the next step is that the actors will be eliminated and everyone can upgrade to a big movie with one click.

Coincidentally, today, the AI model that can make video has come.

Just now, like a thunder, Runway released the text-generated video model Gen-2.

The slogan is also very explosive-"say it,see it". As long as you can say it, it can be made for you. The superpower in science fiction has come true!

It can be said that with Runway Gen-2, you can use any image, video or text to generate a cool blockbuster and have whatever style you want.

The speed is simply stunning: before Gen-1 's internal tests are available, Gen-2 is coming!

In a word, a picture, a three-second video made out of nothing before, Runway has been brilliant in the field of text to image, the famous Stable Diffusion was developed by Runway.

In February of this year, Runway proposed the first AI editing model, Gen-1.

As the name implies, Gen-1 can edit the video we want on the basis of the original video. Whether it's a rough 3D animation or a rickety video shot on a mobile phone, Gen-1 can be upgraded to an incredible effect.

Paper address: https://arxiv.org/ abs / 2302.03011 for example, with a few boxes, Gen-1 can generate a video of a factory and turn decay into magic. It's as simple as that.

Compared to Gen-1,Gen-2, it gets an epic upgrade-- you can generate video from scratch.

The model is not yet open and is expected to be made public in a few weeks.

According to the current demonstration clip, the effect of Gen-2 does not seem to be as realistic as that of Gen-1 for the time being.

However, this is, after all, the first step of AI Wensheng video 0 to 1. Director AI has made human beings tremble enough to do so.

Netizens measured that the lucky netizens who were qualified for internal testing due to the explosion of the effect have begun to generate their own movies.

It can be seen that whether it is MiyazakiHayao-style Japanese animation, or realistic-style close-ups, or magical special effects blockbusters, Gen-2 is not a problem.

Although the current picture quality may not be delicate enough, after all, it is a video generated from 0, and once it is optimized in the future, the leading directors may not be able to hold their jobs.

Gen-2 's eight "magic" Mode 01:Text to Video (text generation video) a text prompt, you can synthesize any style of video, not afraid that it can not be produced, I am afraid you can not think of it.

Tip:

In an attic in New York City, the evening sun shines gently through the window.

Output:

Mode 02:Text + Image to Video (text + image generation video) enter a picture, add a prompt, and directly turn you into a video.

Enter the image:

Driver Tip:

In a low-angle shot, the man was walking along the street, illuminated by neon lights in the surrounding bars.

Output:

Mode 03:Image to Video (image generation video) there is no need to explain this, you give it a picture, it gives you a second to change a video. You can choose a variety of styles.

Meitu turns into a movie in a second, which is too dreamy.

Enter the image:

Output:

Mode 04:Stylization (stylized) what if you have an original video but the effect is not enough to explode?

Just overlay the style you want to overlay with pictures or words, and many magical styles are immediately generated, turning into Hollywood blockbusters in seconds.

Original video:

Drive image:

Output:

Mode 05:Storyboard (story version) pulls the simulated scene into a stylized and animated rendering, and the pre-production is simplified with one button.

Output & output comparison:

Mode 06:Mask (mask) separates a specific object from your video, and then uses prompt to change it as much as you want. For example, in the image below, Golden millisecond becomes Dalmatian.

Enter the video:

Driver Tip:

A white-haired dog with black spots.

Output:

Mode 07:Render (rendering) as long as the input image or prompt, texture-free rendering immediately into a realistic output, depth and realism impressive.

Enter the video:

Output:

Mode 08:Customization (personalization) changes the original video personalization greatly through customization, and the fidelity is ultra-high.

Click to see the foreign brother turn into a Turtle in a second.

Input-output comparison:

Training image:

Judging from the results, the prospect of Gen-2 is really fascinating. (the internal test application for Gen-1 is here, http://t.cn/ A6Cu1cdy).

Although Meta and Google have similar text-to-video models (Make-a-Video and Phenaki), the resulting clips are longer and more coherent.

But these larger companies have not invested much resources, compared with a team of 45 people at Runway.

In fact, Runway has been focusing on AI tools since 2018 and raised $50 million at the end of last year.

Interestingly, Runway contributed to the initial version of Stable Diffusion. However, it was later popularized by Stability AI and further developed.

And it also tells us how amazing the potential of startups in generative AI is.

A big wave of displays

Tip: a close-up of one eye

Tip: desert landscape photographed by drones

Reference:

Https://research.runwayml.com/gen2

Https://www.theverge.com/2023/3/20/23648113/text-to-video-generative-ai-runway-ml-gen-2-model-access

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.