Shulou(Shulou.com)11/24 Report--

NVIDIA ACE for Games and Kairos demo made their debut at COMPUTEX, which will be described in detail in this issue of "NVIDIA Studio Creative acceleration".

Editor's note: this article is part of our weekly NVIDIA Studio Creative acceleration series, which aims to introduce invited artists, provide creative tips and tips, and demonstrate how NVIDIA Studio technology can improve creative workflows. We will also delve into the performance, technology, and resources of the new GeForce RTX 40 Series GPU, and how they will significantly accelerate content creation.

As the largest annual technology trade fair in Asia, COMPUTEX releases a lot of up-to-date information and introduces creators to a large number of AI features, as well as new tools and performance improvements from the NVIDIA Studio platform.

The new 14-inch NVIDIA Studio design is equipped with the GeForce RTX 40 series notebook GPU, which greatly improves performance and portability compared to the previous generation, helping creators speed up the presentation of inspiration. AI dedicated hardware Tensor Core can save time and efficiency for tasks in popular applications such as Davinci Resolve. RT Core combines with our neural rendering technology DLSS 3 to improve the performance of real-time 3D rendering applications such as the D5 renderer and NVIDIA Omniverse.

NVIDIA has also introduced a new way to speed up video coding. Parallel scene coding sends independent groups of frames or scene groups to each NVIDIA encoder. By making full use of multiple NVIDIA encoders, video export time can be significantly reduced without affecting image quality. The first software to integrate the technology is the popular video editing application clip Professional Edition and the CapCut desktop.

The May version of the Studio driver is currently available for download. This version supports updating MAGIX VEGAS Pro, D5 renderer, and VLC media player. And in addition to supporting clipping Professional Edition and CapCut desktop, this version also supports AI model optimization for popular applications.

The new NVIDIA Avatar Cloud Engine (ACE) for Games platform uses AI-driven natural language interaction to make non-player characters (NPC) in the game intelligent.

Click to watch

The Kairos presentation is a joint project carried out by NVIDIA Creative Director Gabriele Leone and Convai. This week's "NVIDIA Studio Creative acceleration" column will show you how to turn a single model into a vivid, lifelike character.

Ultra-portable, high performance

NVIDIA Studio is designed to use NVIDIA Ada Lovelace architecture, which can not only help you create efficiently, but also help you enjoy your games.

This is the first time GeForce RTX performance has been applied to a 14-inch device. In the process, the NVIDIA Studio design has changed the ultra-portable notebook market with the perfect combination of performance and portability.

Asustek Lingyao Pro 14 can be equipped with GeForce RTX 4070 notebook computer GPU.

These productivity tools are designed for creators to perform well. The NVIDIA Studio platform provides strong support for more than 110 creative applications and provides lasting stability for these applications with the help of NVIDIA Studio drivers. In addition, the platform includes a set of powerful Studio software driven by AI, such as NVIDIA Omniverse, Canvas and Broadcast.

The fifth-generation Max-Q technology is a set of advanced AI-driven technologies that optimize laptop performance, power consumption, and noise for peak efficiency. Battery life can be extended by up to 70%. DLSS is now optimized for laptops: DLSS 3 optical multi-frame generation and super-resolution technology provides creators with amazing 3D rendering performance in the Omniverse and D5 renderers as well as in popular games such as Cyberpunk 2077 (Cyberpunk 2077).

As the ultra-portable notebook market heats up, PC notebook makers offer creators more choices than ever before. Recent and more upcoming models include the Acer extraordinary X 14, the Asustek Pro14, the Lenovo YOGA Pro14s Supreme Edition and the MSI 14 Studio.

Parallel scene coding

The recently released version of Video Codec SDK 12.1 adds support for multiple encoders to halve video export time. Our previously released syncopation coding method (slicing frames and then sending each sub-picture to a separate encoder) now has an API that can be provided by the application developer to the end user. Previously, split encoding was automatically enabled to handle 4K or higher resolution video, resulting in faster export presets. In this update, developers can provide users with an option to easily switch coding methods.

In addition, a new coding method, parallel scene coding, is introduced in Video Codec SDK version 12.1. After a picture or scene is sent to the rendering workflow, the video application can split it into multiple groups. Each group can then be rendered independently and arranged in the appropriate order at the final output.

The results show that the coding speed can be significantly improved, the speed of dual encoders can be increased by nearly 80%, and the coding speed can be further improved in more than two NVIDIA encoder scenarios (for example, NVIDIA RTX 6000 Ada architecture professional GPU). Compared with the existing segmentation coding methods, the image quality is also improved.

Click to watch

Clip: an omnipotent and easy-to-cut video creation tool that allows everyone to participate in the creation and easily record the good life. After 4 years of iterative development, clipping has launched a number of product versions such as mobile, Pad, professional and web version, realizing multi-terminal interworking and creating possibilities for free creation in more scenarios.

The Beta version of cut Professional version 4.2.0 and CapCut Desktop 2.2.0 was released today. After testing, the video export speed can be increased by up to 67%. Users can be the first to experience this advantage on RTX GPU with two or more encoders.

Major updates to the May Studio driver

The features of the May Studio driver have been significantly upgraded and optimized.

MAGIX has partnered with NVIDIA to migrate its family of VEGAS Pro AI models to WinML, allowing video editors to apply AI effects faster.

This driver also optimizes AI functions for applications running on WinML, including Adobe Photoshop, Lightroom, MAGIX Vegas Pro, ON1 and DxO applications.

The real-time raytrace renderer D5 renderer also adds NVIDIA DLSS 3 technology to provide users with a smoother window experience, allowing them to experience ultra-smooth dynamic effects when browsing the scene. Architects, designers, interior designers and all professional 3D artists can benefit from it.

The D5 renderer combines with DLSS 3 to create realistic images.

The NVIDIA RTX video super-resolution feature (video upsampling technology that uses AI and RTX Tensor Core to improve video quality) is now fully integrated into the VLC media player and you no longer need to download it separately. Learn more.

Downloading GeForce Experience or NVIDIA RTX Experience is the easiest way for you to upgrade your driver and get the latest version.

The trump card of the game world

Mr. Huang Renxun, founder and CEO of NVIDIA, delivered a keynote speech at the COMPUTEX exhibition, introducing a new construction service NVIDIA ACE for Games that can make the game NPC smarter with AI-driven natural language interaction.

Game developers and studios can use ACE for Games to build and deploy custom voice, dialogue, and animated AI models in their software and games. This AI technology can change the entire game world, injecting new life into a single character, a group character, or an entire town character. The future of this technology is unlimited.

ACE for Games is built on the technology in NVIDIA Omniverse. NVIDIA Omniverse is an open development platform for building and running meta-cosmic applications, including basic AI models optimized for voice, dialogue, and character animation.

These include NVIDIA NeMo (conversational AI that can be fine-tuned for game characters), NVIDIA Riva (with automatic speech recognition and text-to-speech capabilities), and Omniverse Audio2Face (which instantly creates vivid facial expression animation for game characters to match any voice). Audio2Face works with Omniverse's Unreal engine 5 connector, and developers can add facial animation directly to MetaHuman characters.

Seeing is believing: Kairos demonstration

In a demonstration called Kairos, Mr. Huang Renxun showed ACE for Games to COMPUTEX attendees and gave viewers a glimpse into the future of the game.

Convai is a startup supported by the NVIDIA startup acceleration program, focusing on developing advanced conversational AI for virtual gaming worlds. NVIDIA Lightspeed Studios is led by creative director and 3D artist Gabriele Leone to create a very realistic scene. The combination of the two shows how developers can use NVIDIA ACE for Games to build NPC.

In the demonstration, players interact with Jin, the owner of a ramen shop. This photo-level real store is based on a pre-built virtual ramen shop in NVIDIA Omniverse.

To this end, a NVIDIA artist went to an authentic ramen restaurant in Tokyo and collected more than 2000 high-resolution reference images and videos. The contents of these materials include cooking areas, cleaning areas, food preparation and storage, and other different kitchen areas. "We used about 70 percent of the existing models, 30 percent of the new models, and 80 percent of the texture reconstruction," Leone said.

Kairos: beautifully rendered in Autodesk Maya, Blender, UE 5, and NVIDIA Omniverse.

In the digital ramen shop, objects are modeled in Autodesk 3ds Max and Blender, the former using RTX accelerated AI noise reduction technology, the latter using RTX accelerated OptiX ray tracing technology to achieve smooth interactive creation in the window. All of these are supported by the team's large number of GeForce RTX 40 Series GPU.

The creator uses Adobe Substance 3D Painter in the texture creation phase, combining NVIDIA Iray rendering technology with RTX accelerated light and ambient light masking, allowing users to bake large footage in a moment.

"to be honest, in the absence of GeForce RTX GPU and Omniverse, it must take a lot of time to complete this project." -Gabriele Leone

Next, with the UE 5 connector, the team can use the Omniverse and Audio2Face applications to add facial animation and audio directly to the ramen shop NPC.

Although Jin is a NPC, he can give a real response to natural language, and the reply is consistent with the narrative background story. And all this is thanks to the help of generative AI.

In addition, lighting and animation are done in UE 5, powered by NVIDIA DLSS technology. This technology uses AI to sample frames rendered at lower resolution, while still retaining high-fidelity details, further improving the interaction of the window for the production team.

Hand over your ramen order to NPC-the interactive, conversable character.

All of a sudden, NPC becomes more attractive. It can also shine brilliantly.

Please follow NVIDIA Studio's official account on Weibo, bilibili and Wechat. You can also watch the tutorials on Douyin and get the latest information from your inbox by subscribing to NVIDIA Studio Newsletter.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.