Shulou(Shulou.com)11/24 Report--

Today, MLCommons released the results of MLPerf inference v3.1 performance benchmark tests for the 6 billion parameter large language model and the computer vision and natural language processing model GPT-J, including Intel's submitted test results based on the Habana ®Gaudi ®2 Accelerator, the fourth generation Intel ®Xeon ®Extensible processors, and the Intel ®Xeon ®CPU Max series. The results demonstrate Intel's competitive performance in AI reasoning and further strengthen its commitment to accelerate the large-scale deployment of AI in cloud-to-network-to-edge-to-end workloads.

Sandra Rivera, executive vice president and general manager of Intel's data center and artificial intelligence division, said: "as the latest MLCommons results show, we have a strong and competitive portfolio of artificial intelligence products to meet our customers' needs for high-performance and efficient deep learning reasoning and training. at the same time, Intel's product portfolio has a leading cost-effective advantage for artificial intelligence models of all sizes."

According to the MLCommons AI training results and Hugging Face performance benchmark tests disclosed in June, Gaudi2 can outperform Nvidia's H100 processors on advanced visual language models, and today's results further prove that Intel can provide the only viable alternative to Nvidia H100 and A100 to meet AI computing needs.

Given the personalized needs of customers, Intel is making AI ubiquitous with products that help solve reasoning and training problems in AI workloads. Intel's AI products provide customers with the ideal choice to flexibly match their performance, efficiency and target cost to obtain the best AI solution, while also helping customers to open up the ecosystem.

About the test results of Habana Gaudi2:

The reasoning results of Habana Gaudi2 on the GPT-J model strongly verify its competitive performance.

The reasoning performance of server query and offline sample of Gaudi2 on GPT-J-99 and GPT-J-99.9 is 78.58times / s and 84.08times / s, respectively.

provides convincing performance over the Nvidia H100, which shows only a slight performance advantage of 1.09x (server) and 1.28x (offline) over Gaudi2.

Gaudi2 has 2.4 times higher performance than the Nvidia A100 (server) and 2x (offline).

Gaudi2 submitted results using the FP8 data type and achieved 99.9% accuracy on this new data type.

With Gaudi2 software updates released every 6-8 weeks, Intel will continue to demonstrate improved performance of its products in the MLPerf benchmark, as well as continued expansion of model coverage.

The reasoning results of Habana Gaudi2 on GPT-J model verify its competitive performance.

Test results for fourth-generation Xeon scalable processors:

Intel submitted seven reasoning benchmarks based on fourth-generation Intel Xeon scalable processors, including the GPT-J model. The results show that the fourth-generation Xeon processors have excellent performance for general-purpose AI workloads, including visual, language processing, voice and audio translation models, as well as the larger DLRM v2 deep learning recommendation model and ChatGPT-J model. In addition, Intel is still the only company that uses industry-standard deep learning ecosystem software to submit public CPU results.

fourth-generation Intel Xeon scalable processors are ideal for building and deploying general-purpose AI workloads through popular AI frameworks and libraries. For GPT-J 's task of summarizing about 1000-1500 words of press releases, the fourth-generation Xeon scalable processor can complete summary feeds of two segments per second in offline mode and one segment per second in real-time server mode.

Intel for the first time submitted MLPerf results for the Intel Xeon CPU Max series, which provides up to 64 GB of high-bandwidth memory. For GPT-J, it is the only CPU that can achieve 99.9% accuracy, which is very important for applications that require extremely high precision.

Intel submitted test results in cooperation with OEM vendors, further demonstrating the scalability of its AI performance and the accessibility of general-purpose servers based on Intel Xeon processors, fully meeting customer Service level Agreement (SLA).

Fourth-generation Xeon scalable processors are ideal for building and deploying generic AI workloads

MLPerf is a well-known AI performance benchmark in the industry, which aims to achieve fair and repeatable product performance comparison. Intel plans to submit new AI training performance results for the next MLPerf test. Ongoing performance updates highlight every step Intel has taken to help customers and power the evolution of AI technology, from low-cost AI processors to high-performance AI hardware accelerators or GPU for network, cloud, and enterprise users.

More: performance metrics based on MLPerf v.31 reasoning (benchmark results) | MLCommons announcement

Description: for configuration instructions, please check the MLCommons web page. The results may be different.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.