Shulou(Shulou.com)11/24 Report--

ChatGPT set off the chip industry "a hundred schools of thought contend", Google, Microsoft, Amazon have entered the chip war, Nvidia is no longer a monopoly.

After the explosion of ChatGPT, the AI war between Google and Microsoft has spread to a new field-server chips.

Today, AI and cloud computing have become a must, and chips have become the key to reducing costs and winning business customers.

Originally, big companies such as Amazon, Microsoft and Google are all famous for their software, but now they are spending billions of dollars on chip development and production.

AI chips developed by major technology giants

According to foreign media The Information and other sources, the three major companies have launched or plan to release 8 servers and AI chips for in-house product development, cloud server leasing or both.

"if you can make silicon optimized for AI, there will be a huge victory ahead of you," says Glenn O'Donnell, a director of research firm Forrester.

Will these great efforts be rewarded?

The answer is not necessarily.

Intel, AMD and Nvidia can benefit from economies of scale, but for large technology companies, this is far from the case.

They also face many thorny challenges, such as hiring chip designers and persuading developers to use their custom chips to build applications.

However, big companies have made remarkable progress in this area.

According to the published performance data, Amazon's Graviton server chips, as well as Amazon and Google's AI dedicated chips, are already comparable to traditional chip manufacturers in performance.

There are two main types of chips developed by Amazon, Microsoft and Google for their data centers: standard computing chips and specialized chips for training and running machine learning models. It is the latter that powers large language models such as ChatGPT.

Previously, Apple successfully developed chips for iPhone,iPad and Mac, improving the handling of some AI tasks. These big factories may be inspired by Apple.

Of the three big companies, Amazon is the only cloud service provider that offers two chips in servers, and Annapurna Labs, an Israeli chip designer acquired in 2015, laid the foundation for this work.

Google launched a chip for AI workloads in 2015 and is developing a standard server chip to improve server performance in Google Cloud.

By contrast, Microsoft's chip development started relatively late, starting in 2019, and recently, Microsoft has accelerated the launch of an AI chip designed specifically for LLM.

The explosion of ChatGPT has ignited the excitement of users all over the world for AI. This has further promoted the strategic transformation of the three big companies.

ChatGPT runs on Microsoft's Azure cloud and uses tens of thousands of Nvidia A100s. Both ChatGPT and other OpenAI software integrated into Bing and various programs require so much computing power that Microsoft has allocated server hardware to the internal team that developed AI.

At Amazon, CFO Brian Olsavsky told investors on an earnings call last week that Amazon plans to shift spending from retail to AWS, in part by investing in the infrastructure needed to support ChatGPT.

At Google, the engineering team responsible for making tensor processing units has moved to Google Cloud. It is reported that cloud organizations can now develop a roadmap for TPU and the software running on it, hoping to let cloud customers rent more TPU-powered servers.

Google: TPU V4 specially designed for AI as early as 2020, Google deployed TPU v4, the strongest AI chip at that time, on its own data center.

But it wasn't until April 4 of this year that Google first released the technical details of the AI supercomputer.

The performance is 2.1 times higher than that of TPU v3 and TPU v4, and the performance of supercomputing is improved by 10 times after integrating 4096 chips.

At the same time, Google also claims that its chips are faster and more energy efficient than the Nvidia A100. For systems of the same size, TPU v4 provides 1.7 times better performance than the Nvidia A100 and 1.9 times higher energy efficiency.

For systems of similar size, TPU v4 is 1.15 times faster than A100 on BERT and about 4.3 times faster than IPU. For ResNet,TPU v4, it is 1.67 times faster and about 4.5 times faster.

In addition, Google has hinted that it is developing a new TPU to compete with the Nvidia H100. Google researcher Jouppi told Reuters that Google has a "production line for future chips."

Microsoft: secret weapon Athena anyway, Microsoft is still eager to try in this chip dispute.

Earlier, it was revealed that Microsoft's secretly formed team of 300 people began to develop a custom chip called Athena in 2019.

According to the original plan, Athena will be built using TSMC's 5nm process and is expected to reduce the cost of each chip by 1 and 3.

If it can be installed on a large scale next year, Microsoft and OpenAI teams can use "Athena" to complete model training and reasoning at the same time.

In this way, the shortage of dedicated computers can be greatly alleviated.

In a report last week, Bloomberg said Microsoft's chip division had partnered with AMD to develop Athena chips, causing AMD's shares to rise 6.5% on Thursday.

But a person familiar with the matter said that AMD was not involved, but was developing its own GPU to compete with Nvidia, and that AMD had been discussing the design of the chip with Microsoft, which is expected to buy the GPU.

Amazon: it has taken a place and Amazon seems to have taken a lead in the chip race with Microsoft and Google.

Over the past decade, Amazon has maintained a competitive advantage over Microsoft and Google in cloud computing services by providing more advanced technology and lower prices.

In the next decade, Amazon is also expected to maintain its competitive edge through its own internally developed server chip, Graviton.

As the latest generation of processors, AWS Graviton3 improves its computing performance by up to 25% compared with the previous generation, and its floating-point performance by as much as 2 times. And supports DDR5 memory, which increases the bandwidth of DDR4 memory by 50%.

For machine learning workloads, AWS Graviton3 performs up to three times better than the previous generation and supports bfloat16.

Cloud services based on Graviton 3 chips are so popular in some areas that demand exceeds supply.

Amazon's other advantage is that it is currently the only cloud provider that offers standard computing chips (Graviton) and AI dedicated chips (Inferentia and Trainium) on its servers.

As early as 2019, Amazon launched its own AI reasoning chip, Inferentia.

It allows customers to run large-scale machine learning reasoning applications in the cloud at low cost, such as image recognition, speech recognition, natural language processing, personalization and fraud detection.

On the other hand, the latest Inferentia 2 has a three-fold increase in computing performance, a four-fold increase in the total accelerator memory, a four-fold increase in throughput, and a reduction in latency to 1 to 10.

After the launch of the original Inferentia, Amazon released its custom chip, Trainium, which is mainly used for AI training.

It optimizes the workload of deep learning training, including image classification, semantic search, translation, speech recognition, natural language processing and recommendation engine.

In some cases, chip customization can not only reduce the cost by an order of magnitude, reducing energy consumption to 1 stroke 10, but also these customized solutions can provide customers with better service with lower latency.

It's not that easy to shake up Nvidia's monopoly, but so far, most of the AI load is on the GPU, and Nvidia makes most of the chips.

According to previous reports, Nvidia has an 80% market share in independent GPU and 90% market share in high-end GPU.

For 20 years, 80.6% of the cloud computing and data centers that run AI around the world have been driven by Nvidia GPU. In 21 years, Nvidia said that about 70% of the world's top 500 supercalculations were driven by its own chips.

Now, even the Microsoft data center that runs ChatGPT uses tens of thousands of Nvidia A100 GPU.

All along, whether it is the top-tier ChatGPT, or Bard, Stable Diffusion and other models, the computing power is provided by the Nvidia A100, which is worth about $10, 000 each.

Not only that, A100 has become the "main force" of artificial intelligence professionals. The 2022 artificial Intelligence status report also lists some companies that use A100 supercomputers.

It is obvious that Nvidia has monopolized global computing power, unifying the world with its own chips.

According to practitioners, application-specific integrated circuit (ASIC) chips that Amazon, Google and Microsoft have been developing can perform machine learning tasks faster and consume less power than general-purpose chips.

When comparing GPU and ASIC, O'Donnell directors used this comparison: "you can use the Prius when driving, but if you have to use four-wheel drive on the mountain, it will be more appropriate to use the Gyibug Horse Shepherd." "

But despite all the efforts, Amazon, Google and Microsoft all face challenges-how to persuade developers to use these AI chips?

Now, Nvidia's GPU is dominant, and developers are already familiar with its proprietary programming language CUDA, which is used to create GPU-driven applications.

If they switch to custom chips from Amazon, Google or Microsoft, they will need to learn a whole new software language. Will they be willing to do so?

Reference:

Https://www.theinformation.com/articles/google-and-microsofts-other-ai-race-server-chips?rc=epv9gi

Https://www.theregister.com/2023/05/09/intel_layoffs_coming/

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.