Shulou(Shulou.com)11/24 Report--

On September 9, Beijing time, the MLCommons community released the latest MLPerf 2.1 benchmark results. The new round of benchmarks has nearly 5300 performance results and 2400 power measurement results, an increase of 1.37 times and 1.09 times respectively compared with the previous round, and the scope of application of MLPerf has been further expanded.

Alibaba, Asustek, Azure, Bichang Technology, Dell, Fujitsu, gigabyte, H3C, HPE, Tide, Intel, Krai, Lenovo, Moffett, Nettrix, Neural Magic, Nvidia, OctoML, Qualcomm, SAPEON and Supermicro are all contributors to this round of testing.

Among them, Nvidia's performance is still eye-catching, taking part in the MLPerf test with H100 for the first time and setting a new world record in all workloads.

The H100 broke the world record and improved the performance of the A100 by 4.5 times. Nvidia launched the H100 GPU based on the new architecture NVIDIA Hopper in March this year, achieving an order of magnitude performance leap compared to the NVIDIA Ampere architecture launched two years ago. Huang Renxun once said on GTC 2022 that 20 H100 GPU can support the equivalent of global Internet traffic and help customers launch advanced recommendation systems and large language models that run data reasoning in real time.

The H100, which is expected by a large number of AI practitioners, was originally scheduled for official delivery in the third quarter of 2022, but it is currently in a pre-booked state, and the actual usage of users and the actual performance of the H100 are unknown, so you can feel the performance of the H100 in advance through the latest round of MLPerf test scores.

In this round of tests, comparing Intel Sapphire Rapids, Qualcomm Cloud AI 100, Biren BR104, and SAPEON X220 Radio Enterprise Magazine NVIDIA H100 not only submitted test results for all six neural network models in the data center, but also demonstrated a lead in throughput and speed in both single server and offline scenarios.

Compared to the NVIDIA A100, H100 shows a 4.5-fold performance improvement in the BERT model for natural language processing, one of the largest and most performance-demanding models in the MLPerf model, and one to three times the performance improvement in the other five models. H100's initial outstanding performance on the BERT model is mainly due to its Transformer Engine.

Among the other products that have also submitted results, only Biren BR104 has more than doubled the performance improvement compared with NVIDIA A100 under the ResNet50 and BERT-Large models in offline scenarios, and none of the other products that have submitted results has surpassed A100 in performance.

In the data center and edge computing category scenarios, the test performance of the A100 GPU is still good, thanks to continuous improvements in NVIDIA AI software, the A100 GPU achieved a six-fold performance improvement compared to the debut of MLPerf in July 2020.

Pursue AI generality, test scores cover all AI models because users usually need to use many different types of neural networks to work together in practical applications, for example, an AI application may need to understand users' voice requests, classify images, make suggestions, and then respond with voice, each step requires a different AI model.

For this reason, the MLPerf benchmark covers popular AI workloads and scenarios, including computer vision, natural language processing, recommendation systems, and speech recognition, to ensure reliable and flexible deployment performance. This also means that the more models covered by the submitted test scores, the better the scores, and the more versatile their AI capabilities.

In this round of tests, Nvidia AI remains the only platform capable of running all MLPerf reasoning workloads and scenarios in data center and edge computing.

In terms of data centers, both A100 and H100 submitted six model test results.

In terms of edge computing, NVIDIA Orin runs all MLPerf benchmarks and wins the most tests of all low-power system-on-chip chips.

Orin integrates NVIDIA Ampere architecture GPU and Arm CPU kernel into one chip, which is mainly used for robots, autonomous machines, medical machinery and other forms of edge embedded computing.

Currently, Orin has been used in NVIDIA Jetson AGX Orin developer kits and robot and autonomous system generation mock exams, and supports complete NVIDIA AI software stacks, including self-driving car platforms, medical device platforms and robot platforms.

Compared with its debut on MLPerf in April, Orin has improved its energy efficiency by 50%, and its operating speed and average energy efficiency are 5 times and 2 times higher than those of the previous generation Jetson AGX Xavier module, respectively.

Universal NVIDIA AI is being supported by a wide range of machine learning ecosystems in the industry. In this round of benchmarks, more than 70 submitted results were run on the NVIDIA platform. For example, Microsoft Azure submitted the results of running NVIDIA AI on its cloud service.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.