Shulou(Shulou.com)11/24 Report--

Thanks to CTOnews.com netizen OC_Formula for the clue delivery! CTOnews.com, November 22 (Xinhua)-- Stability AI, a startup focused on developing artificial intelligence (AI) products, has released its latest AI model, Stable Video Diffusion. This model can generate video from existing pictures, which is an extension of the previously released Stable Diffusion text-to-picture model, and is one of the few AI models that can generate video on the market so far.

However, the model is not currently open to everyone. Stable Video Diffusion is currently in what Stability AI calls a "research preview" phase. People who want to use this model must agree to some terms of use that specify the expected application scenarios of Stable Video Diffusion (such as "educational or creative tools", "design and other artistic processes", etc.) and unexpected scenarios (such as "factual or true representation of people or events").

Stable Video Diffusion is actually made up of two models-SVD and SVD-XT. SVD can convert still pictures into 14 frames of 576 × 1024 video. SVD-XT uses the same architecture, but increases the number of frames to 24. Both can generate video at a speed of 3 to 30 frames per second.

According to a white paper released by Stability AI with Stable Video Diffusion, SVD and SVD-XT were initially trained on a dataset containing millions of videos, and then "fine-tuned" on a smaller dataset with only a few hundred thousand to 1 million video clips. The source of the videos is not very clear, and the white paper implies that many of them are from public research data sets, so it is impossible to determine whether there are any copyright issues.

Both SVD and SVD-XT generate fairly high-quality four-second videos, and carefully selected samples on the Stability AI blog look comparable to Meta's recent video generation model and AI generated videos from Google and AI startups Runway and Pika Labs.

CTOnews.com notes that Stable Video Diffusion also has limitations, and Stability AI is frank about this, saying that these models cannot produce video without moving or slow camera translation, nor can they be controlled by text, nor can they render text (at least not clearly), nor can they consistently "correctly" generate faces and characters.

Although it is still in the early stages, Stability AI points out that these models are very extensible and can be adapted to use cases such as generating 360-degree views of objects.

Stability AI's ultimate goal seems to be to commercialize it, saying that Stable Video Diffusion has potential applications in "advertising, education, entertainment and other fields".

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.