Shulou(Shulou.com)11/24 Report--

Meta and the whole work, 180000 graffiti data set unexpectedly produced a "magic pen Ma Liang" AI, upload pictures can generate animation, too magical or not.

When I was a child, what the editor liked most was art class.

Give me a box of 24-color watercolor pens. I can sit all afternoon without bringing them up.

Because, for everyone, from small to big, the feeling of wanton smearing, creating and playing on white paper is very wonderful.

It can be understood as an emotional outlet, or it can be full of imagination for the future.

Today, Meta has launched a new AI system that can turn graffiti we draw on paper into animated animations.

When we were young, we fantasized that our characters could jump out of the paper and move, and Meta helped us realize this beautiful fantasy.

Paper address: https://arxiv.org/ abs / 23.03127412 million sheets! The process of implementing a dataset is not that complicated.

Meta has a dataset of 180000 daily hand drawings and a complete set of animation code. The dataset is also the world's first annotated dataset characterized by graffiti.

Here's a concept to figure out.

It is true that people express themselves through painting, but there are also differences between painting and painting. For AI, the important thing is to learn. If there is no clear expression of the meaning, it will be more difficult for AI to learn.

That's why there is such a big data collection of 180000 graffiti. With such a data set, researchers can develop tools to more easily and accurately analyze the content of different graffiti.

In fact, as early as the end of 2021, Meta released a demonstration of animated pictures they developed and invited users to upload their own pictures on the Internet.

Users will then receive a short animation of the characters in the painting uploaded by themselves.

According to statistics, about 3.2 million people visited the site at that time and uploaded about 6.7 million pictures. Meta's human censors then screened the images for quality and privacy reasons.

At that time, Meta could only ask the characters in the graffiti to do a set of simple actions, and users responded that they wanted to have more features online. For example, it can make graffiti works that contain multiple characters also move, or let graffiti characters do some more difficult movements, or blink and so on.

Like the figure below, these bouncing GIF images are an example of the expansion of product creativity and educational purposes after the Meta open source code and dataset.

With these resources, researchers can further supplement the analysis of these graffiti methods, in order to expand the original functions.

You can even add background to the villain and let the graffiti dance in the environment.

Can run and jump, vividly mentioned above, graffiti this form of content is too strange, each person's graffiti is unique.

In this case, how to train the model is the crux of the problem.

Meta says there are two common methods. The first is to use annotated paintings to train new models. But the problem with this approach is that there are so few annotated paintings that people may upload just a graffiti without comments. As a result, there is not so much material to train neural networks.

The second way is to paint in a synthetic way, such as redoing the picture in the form of a sketch. But there are problems with this approach.

Generative training requires a lot of sample data to learn, and the above methods may not be able to capture all the differences between paintings and photos. In addition, using this method to generate learning data may not be able to capture some of the details unique to painting, such as paper creases, erased lines, shadows, and so on.

In the end, Meta chose to split the entire task, including a series of subtasks, character detection, graffiti segmentation, pose prediction, and animation.

After users choose to upload graffiti to the system, they can choose the position of the border (determine the range), determine the position of the character's joints, and so on, and finally choose an action to generate animation.

Even, there are many actions to choose from, which depends on the individual user.

Meta's system combines computer vision models trained on photos taken, and because of the difference between graffiti and photos, Meta further uses 180000 data sets to fine-tune the model.

With this data set and the code to generate animation, Meta believes that the field of painting at hand will inspire more creators to join in the future.

So how is this data set built?

Similar to the 21-year-old project, Meta still relies on users to upload data sets.

Meta has done a good job in terms of privacy, people can choose to upload images and comments for Meta research, while Meta is very cautious about the uploaded information, minimizing the possibility of data abuse.

In addition, the screening of uploaded data is also very important. Because this time the project focuses on animating graffiti, non-graffiti images will be screened out by Meta censors.

Content that contains leaked and vulgar words, not to mention, must be screened out.

Everyone is a creator. With such a useful tool, everyone can become an art creator.

After all, it must be cool to see your characters move vividly and be a magic pen Ma Liang.

Reference:

Https://ai.facebook.com/blog/ai-dataset-animation-drawings/

Https://www.engadget.com/meta-has-open-sourced-an-ai-project-that-turns-your-doodles-into-animations-183807106.html

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.