Shulou(Shulou.com)11/24 Report--

Matrix World is a simulated world carefully designed by AI! Princeton uses Infinigen as the key to open the door to the matrix, and AI-generated nature is more real than the real world!

Neo finds himself living in a world that is not real, but a carefully designed simulation of reality.

And did you, for a moment, think that the world we live in is a simulated matrix world?

And now, the matrix was officially activated.

Meditate and feel that this earth, which has existed for a long time, and everything in nature are illusory worlds.

The sun rises on the Arctic glaciers. In the underwater world, all kinds of strange fish, colorful coral reefs.

The mountains were snowing and eagles were soaring in the boundless sky. Hot desert, venomous snakes free to roam.

The small trees beside the river were burning fiercely, and smoke was swirling around them.

Clear sea and streams, turtles basking lazily on the beach, many dragonflies frolicking in the air.

Light and shadow change in the cave.

The falling raindrops, the falling maple leaves, and time came to an abrupt end.

Everything you see is artificial intelligence. Their simulations are so realistic that they are mistaken by everyone for reality itself.

The team that opened the door to this matrix came from Princeton, and the moment it came out, it caused an instant uproar on the Internet.

Netizens have said that we live in a matrix!

In the paper, the researchers introduced a natural world realistic 3D scene program generator called Infinigen.

Infinigen is completely procedural, from shapes to textures, generated from scratch by random mathematical rules.

It can even transform infinitely, covering plants, animals, terrains, and natural phenomena such as fire, clouds, rain, and snow.

The latest paper has been accepted by CVPR 2023.

Paper Address: arxiv.org/ pdf/2306.09310.pdf

Infinigen is built on the free and open source graphics tool Blender, which is also open source.

It takes 3.5 hours to generate a pair of 1080p images!

Infinite CV Data Generator

Infinigen, proposed by Princeton University, can be easily customized to generate real-world scenarios for a variety of specific tasks.

The following simulates diversity in nature.

Its main role is to act as a generator of infinite training data for a wide range of computer vision tasks.

Among them, object detection, semantic segmentation, pose estimation, 3D reconstruction, view synthesis and video generation are included.

In addition, it can be used to build simulated environments for training physical robots and virtual embodied agents.

Of course, 3D printing, game development, virtual reality, film production and general content creation are all available.

Next, look at how the Infinigen system was designed.

Blender's preview.

The researchers primarily developed procedural rules using Blender, an open-source 3D modeling software that provides a variety of primitives and utilities.

Blender presents scenes as hierarchies of objects.

Users modify this representation by converting objects, adding primitives, and editing meshes.

Blender provides import/export of most common 3D file formats.

Finally, everything in Blender can be done automatically through its Python API or by examining its open source code.

Node Translator.

As part of Infinigen, the researchers developed a new set of tools to speed up programmatic modeling for researchers.

One notable example is the researchers 'Node Converter, which automatically converts node graphs into Python code, as shown in the figure.

The resulting code is much more generic and allows researchers to randomize graph structures rather than just input parameters.

This tool makes node diagrams more expressive and allows easy integration with other programmatic rules developed directly in Python or C++.

It also allows non-programmers to contribute Python code to Infinigen by making node graphs.

Generator subsystem. Infinigen is made up of generators, which are probabilistic programs, each dedicated to generating a subclass of assets (such as mountains or fish).

Each generator has a set of high-level parameters (such as the overall height of the mountain) that reflect the external degrees of freedom that the user can control.

By default, researchers randomly sample these parameters according to distributions adjusted to reflect nature, with no user input.

However, users can also override any parameter using the researcher's Python API for fine-grained control over data generation.

The following image is a random scene that contains only terrain. The researchers selected 13 images for various nature scene types.

Mountains, rising rivers, snowy mountains, coastal sunrise, underwater, arctic icebergs, deserts, caves, canyons and floating islands.

Below are randomly generated images of simulated flames, including smoke, waterfalls, and volcanic eruptions.

Leaves, flowers, mushrooms, pine cones.

Trees, cacti, shrubs.

Marine life.

Various types of surfaces.

The picture below shows the creation of biology.

The researchers automatically generated different genomes (a), body parts (b), body part splicing (c), hair (d), and body posture (e).

On the right are randomly generated carnivores, herbivores, birds, beetles and fish displayed by researchers.

In the image below, researchers programmed a random scene layout (a).

The investigator generates all the necessary image content (b) to show the colors of each mesh face and applies programmatic materials and displacements (c).

Finally, render a realistic image (d).

The image below shows dynamic resolution scaling.

The researchers showed three close-up mesh visualizations of the same content at different camera distances.

Despite the different mesh resolutions, no change is seen in the final image.

To evaluate Infinigen, the researchers created pairs of 30K images with ground reality for corrective stereo matching.

The researchers started training RAFTStereo on these images and compared the results on the Middlebury validation set and the test set.

The study code, released today, has collected 99 stars.

Nvidia scientist Jim Fan says simulation is the next "gold mine" when we don't have good training data in reality.

Here, Infinigen is an open-source, procedurally generated realistic dataset for 3D vision. Its quality is amazing! No two worlds are the same.

Every little detail is random and customizable, even the folds on the petals!

Various objects and scenes in nature: plants, animals, terrain; fire, clouds, rain and snow.

Automatic annotation of reference truth values: optical flow, 3D scene flow, depth, surface normal vector, panoramic segmentation, closed boundary.

Written in Blender.

Some netizens said that this really opened my eyes. Programmatically generate 3D worlds in Blender. Without AI, you can create any world you can imagine.

Combine tools like Daz3D to create characters, Blockade Labs to create skyboxes, Convai to create NPCs.

Others say Infinigen will play a role in artificial intelligence computer vision research.

He believes that future AI will handle everything.

For example, we want to build a village with waterfalls in a foggy forest.

We tell an AI to write a detailed description of the forest and send it to the 3D world generator.

Once the world is created, characters can be added to the village. These people will be controlled by artificial intelligence and they will have interaction, dialogue and memory.

A western world rush.

In addition, some netizens said that the program generation has existed for 20 years (I have been doing it for 15 years). "No Man's SKy" is a great example of technology from five years ago.

The highlight of Infinigen is that it generates tagged data. This data can be used to train AI. That's crazy.

There are also netizens who imagine that in the near future, games will become crazy…"independent" game developers will be able to launch some crazy high-end content.

Infinigen was so realistic that some people thought it was generated by Unreal Engine.

About Alexander Raistrick

Alexander Raistrick is a sophomore Ph.D. student in Princeton University's computer science department, supervised by Jia Deng.

Lahav Lipson

Lahav Lipson is a third-year doctoral student at Princeton University.

His research focuses on building deep networks for 3D vision, leveraging strong assumptions about epipolar geometry to achieve better generalization and test accuracy.

Zeyu Ma

Zeyu Ma is a third-year Ph.D. student in Princeton University's Vision and Learning Laboratory. He received a bachelor's degree in electronic engineering from Tsinghua University in 2020. Current research focuses on multi-view 3D reconstruction and procedural data generation.

References:

https://infinigen.org/

https://twitter.com/DrJimFan/status/1670098819026128897

This article comes from Weixin Official Accounts: Xinzhiyuan (ID: AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.