Shulou(Shulou.com)11/24 Report--

[guide to Xin Zhiyuan] A foreign up master used ChatGPT to inject self-awareness into AI agents. Laura, who produced "life", woke up and began to break through on her own in Tomb Raider.

If the game characters are injected into life, what will the video games become?

Once, this is the imagination of many people, but now, with the development of AI intelligence, this imagination has begun to come into reality.

Recently, Foxmaster, an author of YouTube, made an original video of the classic game Tomb Raider.

And the heroine, Laura, is an AI agent who can control her role!

It can be said that Foxmaster injects digital life into game characters by using various AI tools such as machine vision, positioning, object recognition, animation, text and voice.

Just a few days ago, the Stanford agent town in the AI community was officially open source. 25 AI agents with their own personalities live, work and make friends in the virtual town of sandboxie, which is like the "Western World".

So, maybe this is the future of video games?

It has become a common concept for Tomb Raider actresses to consciously build virtual AI that can play games. However, what Formaster wants to do is to make the character look like a real person.

Therefore, he chose the game Tomb Raider, in which Laura, the protagonist, is a distinct and well-defined character.

Formaste hopes that this personality will have an impact on the final outcome of the game.

How to make this AI agent Lara Croft really self-aware in the game?

Formaster finally succeeded after exploring the following steps.

Learn the rules of the game

The first step is to ask her to learn the rules of the game. Tomb Raider has given tutorials that explain all the game control processes.

The author asked Lara to practice in the given way and complete the tutorials at all stages like a human player.

The tutorial ends when Lara leaves the pool, so you can restart the program and make improvements at that time.

At first, it was very difficult to improve Lara.

She sometimes gets confused, walks around on the map, and sometimes gets stuck in the menu bar, staring at her compass over and over again.

In order to solve this problem, the author readjusted the master clock on a virtual machine and accelerated the game process by 40 times. This method worked, and the AI agent finally completed the tutorial in a relatively good time.

However, there is another problem: Lara is often stuck in one place, and each level is made up of static elements, so there is only one optimization path.

This will cause her to only know which paths not to take, rather than come up with a more comprehensive way to solve the problem. But what the author hopes is that Lara will learn to explore the adventure process on its own.

Considering that the only knowledge Lara has is the content on the screen displayed at each level, the author observed the behavior of human players who had never played the game.

Some people will finish the whole tutorial, others will first explore the surrounding environment, and the difference in these choices is determined by the "personality" of different people.

It is precisely this attitude that Lara needs to learn.

Identify the role

In order to connect AI with Lara, you need to make AI understand that she is a bunch of pixels. After entering the game, human players suddenly have this kind of consciousness.

In response, the author recorded 24-hour videos of Lara moving in different directions from all angles so that the AI agent could recognize any given character.

Like a self-driving car, it can shoot 24-hour video at 30 frames per second and get 2592000 reference images to identify road signs.

First, the author tracks the main pixels of the head and then the main pixels of the body to select the relevant areas.

To ensure that the Lara appears in each image, the author asks the program to identify those images that do not contain any highlights.

When Lara occupies the entire screen, or when a two-dimensional object is in front of her, the author uses AI visual capture to help identify Lara, and the results are accurate enough to identify characters on the screen.

Environmental interaction

Once you are able to recognize the Lara, you need to let the AI interact with the environment.

The entire game environment is made up of the same blocks, which the author imports into the cube, then captures the environment from various angles, and then runs the recognition process, so that the AI agent can recognize the environment, just like the process of human players interpreting images.

The area highlighted in red is the texture that cannot be recognized. However, AI still lacks the awareness of wanting to go somewhere and needs to know the distance from his surroundings.

Interestingly, these unrecognized areas are usually away from the Lara or obscured by other 2D objects.

When we look at the behavior of human players and see this image, we can immediately understand that the area pointed to by the arrow is an entrance.

We may not know the size of the next room, but we know it exists.

But it can also be a visual illusion. When we move, the texture of the entrance area is different from that of other places, which is a feature of 3D space.

As a result, AI agents also have to learn to move and compare. From a computational point of view, multiple images can be used to determine which texture sizes have the least change.

The area of the principal polygons can be used here, and their changes are mainly based on the probability that the distance between them is proportional to each other.

Imagine

Now that AI knows where she is, she has to decide where to go next.

By identifying the texture, you can get her to a specific location, but navigating through the level requires something simpler.

The fastest way is to simplify the contrast and identify sudden changes in light in the area. However, human players do not necessarily go to these places.

When there are multiple openings, AI must show enough interest, but not cause her to die.

For example, if a hole is too deep, you can't jump without grasping the surrounding ledge. If there is water below, it may be worth the risk.

Human players know when to jump, but Lara's course of action comes from tutorials, which must estimate exactly how many squares there are between themselves and the target, whether they should gather their momentum, and whether they should pause before letting go.

In short, she must make an accurate assessment like a human player.

If she knew the size of a hole in advance, she would run very fast, which was not in line with our expectations.

In order for her to collect more environmental information, the author encourages AI, if in doubt, should rotate the camera to collect more information.

But in real life, even so, AI may get stuck in the room.

In order to encourage her to go out of the room, the author added some collection rules.

For example, every time she encounters a texture that has never been seen before the area becomes a priority, she must move it by increasing the size of the texture on the screen.

In this way, she can only move forward at the checkpoint. Because there is a unique texture in each level, the only way to unlock it is to enter the next level.

In short, Lara will constantly analyze what she sees on the screen-textures on different surfaces, and she will move her body to define new points of interest (that is, textures that do not yet exist in the catalog).

In the process, she will continue to verify, to find the lost texture.

Exercise

However, even with the above improvements, the AI is still too much of a robot, and the mode of action is obvious.

In order to improve it, the author looked at human players again.

What determines how human players behave in the game? It is our memory, and we will remember how the actions are combined.

Therefore, this process also needs to be implanted into AI, because Lara must be aware of her own athletic ability.

To do this, the author saves every second of AI's actions to train it, and then adds a rule that allows it to perform these saved actions at the same time.

In this way, the action of AI becomes much smoother.

In addition, because the cooldown time for each action is known, you can prevent AI from being active when performing unnecessary actions.

In the image below, the flame is reactivated so fast that it takes a big jump to get through.

Fortunately, these channels are visually simplified, making it easy to identify the edges of the block.

AI knows that Lara will always make the right jump when the first part of the box is activated, as human players understand.

After ChatGPT is injected into the personality, the personality of Lara needs to be instilled into the AI agent. Personality depends on experience and memory, and the combination of these factors determines the Lara role.

According to the settings of the game, she is a famous archaeologist, never afraid of adventure, willing to accept assigned mountain tasks, not even rewarded, out of love.

In addition, Lara is also very rich, has just moved into a mansion, the interior decoration is very elegant, the house has a training room. She is also very good at teaching, swimming, climbing and very strong endurance.

When she is in danger during an adventure, she can keep calm and is very brave and will not hesitate to help her companions.

In the face of being surrounded by wolves, she shot accurately with a gun and could kill a dangerous wolf with a dagger. The only thing she refused was to force the lock on her own initiative.

All of the above is the Lara personality learned from the game settings.

The next step is to comment on what she sees in a real way, for which the author has compiled Lara's personality traits in the database.

In order for comments to be meaningful to the actual situation, the program needs to associate the game image with something recognizable in real life.

Images with less texture details can be matched by Google reverse image search.

Therefore, when the AI agent detects a texture large enough, it initiates a search for elements.

In order to recognize the words, the author copies the entire page to ChatGPT. ChatGPT is then asked to classify them according to the number of occurrences, and then add the identified words to the list.

Finally, ChatGPT is asked to create a sentence based on these words, taking into account Lara's personality.

To do this is to ask ChatGPT to truly comment on what Lara sees in real life.

For example, if the texture is identified as a sea lion, ChatGPT can associate the sea lion with Lara personality traits to make comments.

In general, the personality of the Lara agent is set by using various attributes such as "brave", "friendly" or "smart" through ChatGPT.

Lara in the game, all objects perceived and commented on will pass through this personality filter and will be reflected by Lara according to a given personality.

For example, in the case of seals above, Lara will mention the fossilized sea lions, or their excellent swimming ability, and will not comment on them in the circus.

Netizens: AI agents have turned video games into works of art. Some netizens say that this is one of the coolest and most natural ways for characters to develop themselves. When the character is alive, you can really experience that a video game is a work of art.

"the way she speaks and analyzes her surroundings is very lovely. Giving the AI robot the ability to comment on its surroundings, regardless of its actual level, will give people the impression that she is a living person. "

"it's fascinating. Her detached curiosity and whimsical ideas are strikingly similar to what I imagined Lara's inner monologue. "

Some people say that, to my surprise, her conversation coincides with her behavior.

This makes people feel that the artificial intelligence that writes the dialogue is the same person as the artificial intelligence that controls the character.

Not long ago in the Stanford AI agent town open source, netizens were very excited that AGI has come, a variety of RPG and simulation games will soon use this technology.

Now, Foxmaster combines ChatGPT, computer vision and object recognition to make video games more interesting.

Perhaps in the future, video game characters can be endowed with deeper and more flexible personalities, faster responses to the environment, and many changes that we can't imagine.

Reference:

Https://the-decoder.com/ai-makes-lara-croft-self-aware-in-og-tomb-raider/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.