Shulou(Shulou.com)11/24 Report--

CTOnews.com, October 18 (Xinhua)-- Nvidia is the king of artificial intelligence in the field of hardware. The company's GPU powers the data centers of Microsoft, OpenAI and other companies, running Bing Chat, ChatGPT and other artificial intelligence services. Today, Nvidia announced a new software tool designed to improve the performance of large language models (LLM) on native Windows PC.

In a blog post, Nvidia announced its TensorRT-LLM open source library, which was previously released for data centers and is now available for Windows PC. The biggest feature is that if the Windows PC is equipped with Nvidia GeForce RTX GPU,TensorRT-LLM, the LLM can run four times faster on the Windows PC.

In this article, Nvidia introduces the benefits of TensorRT-LLM for developers and end users:

At larger batch sizes, this acceleration can significantly improve more complex LLM experiences, such as writing and coding assistants, which can output multiple unique autocomplete results at the same time, speeding up performance and improving quality, giving users the best choice.

CTOnews.com notes that the blog post shows an example of TensorRT-LLM. When it is proposed to the basic model of LLaMa 2, "which technologies of NVIDIA are integrated into Soul Killer 2?" When it comes to this question, it gives the unhelpful answer that "the game has not yet been released". Instead, using RAG to add GeForce news to the vector library and connect to the same Llama2 model, you not only get the right answers-- NVIDIA DLSS 3.5, NVIDIA Reflex, and panoramic raytracing-- but also respond faster with the help of TensorRT-LLM acceleration. This combination of speed and capability provides users with smarter solutions.

TensorRT-LLM will soon be available on Nvidia's developer website.

Nvidia also added some artificial intelligence-based features to the new GeForce driver update today. These include the new version 1.5 RTX Video Super Resolution feature, which provides better resolution improvements and less compression when watching online videos. Nvidia has also added TensorRT artificial intelligence acceleration to Stable Diffusion Web UI, allowing users with GeForce RTX GPU to get images from artificial intelligence image generators faster than normal.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.