Shulou(Shulou.com)11/24 Report--

The title of the original text: "the giant ChatGPT was defeated because it was too expensive for GPU!" Meta uses CPU to run AI and click on the wrong technology tree. "

Two years later, Xiaoza All In suddenly found that the whole world was engaged in generative AI. This is embarrassing, our own company is still using CPU to do AI reasoning.

Why did Meta take so long to act in the ChatGPT war?

Just today, a Reuters reporter dug up a melon for an eye-popping reason.

Compared with Google, Microsoft and other big companies, Meta uses CPU when running AI!

It is hard to imagine that in an era when deep learning accounts for almost half of machine learning, a tech giant can hold on with CPU for so long.

Although they have tried to develop their own AI chip, but eventually encountered Waterloo.

Now, the generative AI war detonated by ChatGPT has been knocked out in the dark, which aggravates the capacity contraction of Meta.

What do you think of training AI,Meta with CPU? The reason why Meta refuses to accept GPU for so long is unthinkable.

GPU chips are ideal for AI processing because they can perform a large number of tasks at the same time, reducing the time required to process billions of pieces of data.

However, the GPU is also more expensive than other chips, and Nvidia controls 80 per cent of the market and maintains an absolute lead in supporting software.

Until last year, Meta mainly used CPU when dealing with AI workloads. CPU is the main chip of computers, and CPU has been used in data centers for decades, but it does not perform well in AI work.

It is reported that Meta has also developed its own chips to reason on custom chips designed internally.

But in 2021, Meta was disappointed to find that this two-pronged approach was slower and less efficient than GPU. And GPU is far more flexible than Meta chips in running different types of models.

Moreover, Xiaoza decided to All In meta-universe this move, but also directly drained the calculation power of Meta. Both the deployment of AI and the response to threats have been greatly weakened.

These mistakes attracted the attention of Peter Thiel, a former Meta board member, who subsequently resigned in early 2022.

At a board meeting before leaving, Thiel told Xiaoza and executives that they were too complacent about Meta's social media business and focused too much on Meta, making the company vulnerable to TikTok's challenge, according to insiders.

Towards the end of Meta last summer, Xiaoza summoned senior deputies and spent five hours analyzing the computing power of Meta.

They need to know how capable Meta is in developing cutting-edge AI.

As a result, everyone took a cold breath.

According to the September 20 company memo, despite Meta's high-profile investment in AI research, the AI-friendly hardware and software systems needed for the main business are very expensive, and the company has made slow progress in these areas.

Being slow enough has prevented Meta from keeping up with large-scale innovation.

This is a thorny big problem, you know, the growth of Meta, more and more dependent on AI.

Santosh Janardhan, head of infrastructure, stressed that Meta has lagged far behind its competitors in both the development of AI tools and workflow.

"Meta needs to fundamentally change physical infrastructure design, software systems, and the way it provides a stable platform. "

The big project that Meta has been working on for more than a year is to improve the AI infrastructure. But after capacity crunch, leadership changes and scrapped AI chip projects, Meta's reforms seem to be far from satisfactory.

So, Meta gave up the AI chip altogether?

In response to this question from foreign media, Meta spokesman Jon Carvill said that the company "has a good track record in creating and deploying the most advanced infrastructure on a large scale, and has deep expertise in artificial intelligence research and engineering." "

"as we bring new AI experiences to our applications and consumer product families, we are confident that we will continue to expand our infrastructure capabilities to meet our immediate and long-term needs. "

In short, there is no direct answer.

However, this reform has consumed a lot of manpower and material resources.

It is reported that the reforms have increased Meta's capital expenditure by about $4 billion a quarter, almost double the amount spent in 2021. And Meta's previous plan to build four data centers has been suspended or cancelled.

Meta's capital spending has soared, and these large expenditures coincide with the severe financial tightening of Meta.

Since November last year, the Internet bubble in Silicon Valley has begun to burst, and Meta has been laying off staff on a large scale.

OpenAI's ChatGPT set off an arms race between tech giants immediately after its debut on November 30th.

The generative AI war since then swallowed up a lot of computing power, which forced Meta to step up reform.

Trying to catch up with it is reported that Meta has to bow in the face of reality.

Executives cancelled plans to launch their own chips on a large scale in 2022 and turned to order the multibillion-dollar Nvidia GPU.

But by this time, it was too late.

Meta has lagged far behind its peers such as Google, which started deploying its own customized GPU--TPU as early as 2015.

In the spring of 2022, executives also began to restructure Meta's AI division.

During this period, there was months of turmoil, and more than a dozen executives left. The whole leadership of AI infrastructure has undergone a big change.

Next, Meta's job is tricky, too.

They have to go to a lot of trouble to restructure their data centers to adapt to the new GPU, because GPU consumes more power and produces more heat than CPU, and they have to bring them together with a private network.

In order to manage the heat of the cluster, these facilities require 24 to 32 times the network capacity and new liquid cooling systems, so they need to be designed from scratch.

Even so, Meta doesn't seem to have given up on its own chip development.

It is reported that the new internal chip, like GPU, will be able to train AI models and perform reasoning, which will be completed around 2025.

Before Microsoft had ChatGPT, Google hurriedly pulled Bard out to fight, but Meta seemed to be in no hurry to play the next generative AI competition, and the LLaMA launched was not used for commercial use.

Meta CFO Susan Li admitted in February that Meta did not devote most of its computing resources to generative AI, but basically devoted all its AI capabilities to advertising, feeds and Reels (short videos similar to TikTok).

Previously, like Google, Meta didn't value generative AI. Meta's FAIR lab has been releasing a prototype of this AI technology since the end of the 21st century, and the research results are highly respected, but Meta has never considered turning it into a product.

In mid-November last year, Meta's FAIR lab proposed the Galactica model, but after the birth of ChatGPT, everything was different. Investor interest began to soar, and Xiaoza announced a new top team in February, betting on generative AI.

It is reported that the focus of the work is to build a basic model, on the basis of which different products can be fine-tuned and adjusted.

All In AIGC? Netizens: change the name of MetAI bar 18 months ago, Xiaoza bet the future of Facebook on meta-universe, and even changed the company's name to Meta. Recently, he has become infatuated with another very expensive technology-AIGC.

Earlier this month, Andrew Bosworth, chief technology officer of Meta, said Zuckerberg and other executives now spend most of their time on artificial intelligence.

In response, Bernstein analysts said that if this trend continues, Meta is likely to change its name to MetAI.

However, to catch up with OpenAI, Microsoft and Google, Meta will have to buy Nvidia chips ($10,000 per component) to train these super-large-scale generative AI models.

At present, the "Meta version of ChatGPT" LLaMa, which takes 5 months to train, uses the A100 of 2048 80GB memory.

By contrast, Microsoft's tailor-made override for the OpenAI carries tens of thousands of A100s.

Behind the "fight to the death" between ChatGPT and Bard is the GPU (graphics processing unit) supported by Nvidia CUDA and Google's customized TPU (tensor processing unit).

In other words, this is no longer about the confrontation between ChatGPT and Bard, but between TPU and GPU, and how they effectively multiply matrices.

Because of its excellent design in hardware architecture, Nvidia's GPU is well suited for matrix multiplication tasks-it can effectively implement parallel processing between multiple CUDA cores.

Therefore, since 2012, training models on GPU has become a consensus in the field of deep learning, which has not changed so far.

With the introduction of NVIDIA DGX, Nvidia is able to provide one-stop hardware and software solutions for almost all AI tasks, which competitors cannot provide due to lack of intellectual property rights.

Google, by contrast, launched the first generation of tensor processing units (TPU) in 2016, which includes not only custom ASIC (application specific integrated circuits) optimized for tensor computing, but also its own TensorFlow framework.

This also gives TPU an advantage in AI computing tasks other than matrix multiplication, and even speeds up fine-tuning and reasoning tasks.

However, the long and deep cooperation between Microsoft and Nvidia has made full use of their accumulation in the industry, thus expanding the competitive advantages of both sides at the same time.

Especially when ChatGPT began to sweep across the AI circle, the market capitalization of the two companies soared.

And this wave of large model alchemy brought up by ChatGPT has made Nvidia, a supplier of "alchemy furnace", making a lot of money. Market capitalization has grown by more than 80% in the past few months this year alone.

Layoffs Silicon Valley second, how do ambitions support? however, Meta does not seem to have enough money to support its ambitions.

As we all know, the wave of layoffs continues to sweep the technology industry these days, but some companies have laid off more than others.

Twitter, which laid off 80 per cent of its employees, unquestionably took the top spot, followed by Meta, which sent nearly 1/4 employees.

Meta also ranks second in terms of numbers, with a huge advantage of 21000 people, but that doesn't include a third round of layoffs.

In 2022, before Xiaoza announced big layoffs, Meta had almost 87000 employees. But 11000 people graduated in November and another 10000 in March.

According to Insider, the third round of layoffs at Meta will directly affect thousands of people, with management positions bearing the brunt. Including but not limited to reality labs, technical product managers of Facebook and Instagram, as well as artificial intelligence research scientists, software engineers, data engineers, etc.

According to a new analysis, Meta's workforce expanded by 143% from 2018 to 2022, but income per employee fell by 14% during that period.

Executive changes, staff turnover, lack of funds, the wrong route, the road ahead of Meta seems to be beset with difficulties.

Let's see what Xiaoza will do next.

Reference:

Https://www.reuters.com/technology/inside-metas-scramble-catch-up-ai-2023-04-25/

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.