In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-16 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > IT Information >
Share
Shulou(Shulou.com)11/24 Report--
[guide to Xin Zhiyuan] the most dramatic scene has taken place within Meta, with disputes over the shortage of computing power, and more than half of the core authors of LLaMA have left. Even after three rounds of restructuring of the big model development team, how far is it for Meta to catch up with Google and Microsoft?
Over the past six months, the explosion of ChatGPT has completely made Meta restless.
From February to February, the release of the open source large model series LLaMA, to Llama 2 upgrade, and then to the coding model Code Llama,Meta can be said to bet on all going to ALL IN AI.
In the open source community, the free research and commercial use of Llama models directly ignited the fission of the replacement model.
However, while Meta is enjoying its glory, the AI team is facing a wave of departures.
The Information exclusively reported that most of the team members of the LLaMA and OPT projects have resigned because of internal math disputes.
Even Meta simply abandoned the model they were working on to rival PaLM and reorganized two of Meta AI's lab teams to focus on Llama 2.
The departure of most of the Meta researchers just exposed that the lack of computing power is the core problem of layout-generated AI.
Half of the 14 authors of the LLaMA paper are seen by outsiders as having more computing resources than most companies. Some companies, including Meta, consistently emphasize that they are "GPU RICH" when recruiting AI researchers.
But in fact, I know very well that there are also limitations in the supply of computing resources for Meta. As a result, the power dispute opened in the internal team, causing many people to abandon Meta.
Specifically, more than half of the 14 authors in the original LLaMA model released in February have left. Some have turned to AI startups, while others have joined tech giants.
Paper address: https://arxiv.org/pdf/2302.13971.pdf
Research scientists and engineers who left Meta included Timoth é e Lacroix, Guillaume Lample, and Marie-Anne Lachaux, one of the authors of the original LLaMA paper.
They now work for French startup Mistral AI, which was co-founded by Lacroix and Lample in June.
One of the authors, Armand Joulin, the research director of Meta AI, left Meta in May and has now joined Apple.
Joelle Pineau, director of basic AI Research at Meta artificial Intelligence Research Lab, said in an interview:
Retaining and attracting good people is probably the place where I spend the most time, because without good researchers, there is nothing I can do.
The wave of departures that Meta is facing highlights how big technology companies are grappling with the challenge of surging demand for artificial intelligence talent. At present, for fear of falling behind, the major technology companies are eager to connect AI to their own products and services.
According to Kyle Langworthy, an executive at Riviera Partners, companies that are "eager for talent" are lining up for artificial intelligence researchers who want to poach Meta.
In February, after the release of LLaMA kindled the enthusiasm of the community, Xiaoza announced the company's big move overnight: to set up a top product team and focus on AIGC.
In short, Xiaoza puts the core focus of Meta's future on generative artificial intelligence in an attempt to catch up with competitors such as Google, Microsoft and OpenAI.
The meta-universe, which was once All In in 2021, was also put in a secondary position.
Then, from time to time, Meta released and opened up the latest research in the field of artificial intelligence, including the general visual model SAM "Segmentation everything", multimodal AI model ImageBind, Llama2, Code Llama and so on.
As soon as these studies are released, they have caused great repercussions in the open source community and have been well received.
Now, with the loss of experienced researchers, Meta is getting in the way of catching up with its competitors.
So what exactly made these senior researchers leave angrily?
The Power struggle between LLaMA and OPT in 2013, Yann LeCun joined Meta and founded Meta AI (formerly FAIR), an artificial intelligence research laboratory, when he was AI's research director.
In addition to setting up major laboratories in Menlo Park, Calif., and New York (where LeCun is located), Meta AI also has offices in Montreal (where Pineau is located), Paris, London and Seattle.
The establishment of Meta AI laboratory is mainly devoted to the research of AI, applying artificial intelligence to translation, MRI scanning and other scenarios. However, since the birth of ChatGPT, the company's executives have become more urgent for AI to be more deeply integrated into the product.
In fact, before ChatGPT was released, Meta had been developing its own LLM.
In May 2022, a North American team from Meta AI announced and opened up the large model OPT-175B. Meta claims that this model consumes less energy than GPT-3, although the two models learn the same amount of content during training.
A few months later, the team began working on a second, larger model to compete with Google's PaLM.
Meanwhile, another Paris-based Meta AI team has begun to develop a separate large language model, LLaMA.
The model is smaller than OPT, according to people familiar with the matter. The research team believes that smaller models are efficient in reasoning.
Then came the problem, when teams from two laboratories in North America and Paris started a struggle for access to numeric resources, which in turn intensified disputes among internal teams.
In particular, the LLaMA team feels neglected, its model is small, but gets much less computing resources than the OPT team in North America.
Joelle Pineau, director of basic AI Research, said:
Decisions about how to allocate computing resources in Meta are made by teams of leaders from different parts of the business, roughly once a month.
How much and to whom will be allocated depends on the priorities of the organization and how far the project is from release.
If there is a dispute over the assignment of Meta AI employees, it will be submitted to the superior, that is, the Pineau level to deal with.
In an interview, Pineau admitted that there was some tension in the distribution of power between the LLaMA and OPT teams, adding, "We are trying to find a way to listen to everyone, even if we can't meet everyone's needs." "
For some researchers, it feels like a competition. Some Meta AI executives are also questioning why the two teams are working on similar projects while competing with each other for limited resources.
The tension between the two teams changed dramatically in February, reaching the best part in February of this year. Just this month, in an effort to catch up with its competitors, Meta AI first released LLaMA as an open source model, licensed for research purposes.
In fact, a week before the release, Antoine Bordes, co-director and head of Meta AI's Paris division, left Meta because he was exhausted by "working time like California."
Now, Bordes has joined artificial intelligence company Helsing.
His departure further isolated the LLaMA team from executives in North America, according to people familiar with the matter. In addition, Pineau acknowledged that Bordes's departure had "created some uncertainty" for the team.
This is not just a matter of LLaMA team research morale loss, at the same time, OPT team has been facing the problem of staff loss in the past few months.
It is said that the larger model being developed by the OPT team was directly abandoned by Meta. It is not difficult to understand that the wastage of OPT is directly related to this decision.
According to the profile of Linkedin, the author of the study, half of the 19 authors of OPT papers have left Meta.
Paper address: https://arxiv.org/pdf/2205.01068.pdf
Pineau, director of basic AI Research, said reluctantly, "when all the teams want to upgrade the model at this time, you can either choose one or let them cooperate." "
Team restructuring, positioning unclear while all this happens, Meta AI is also troubled by the frequent changes in the company.
Since November last year, Xiaoza has made several company-wide layoffs. Among them, Meta AI was not spared.
In February, Meta AI leaders decided to bring together some members of competing LLaMA and OPT teams to formally form a "generative AI" team (headed by Ahmad Al-Dahle, a former Apple executive), as well as a large number of staff from Meta AI.
In fact, we can see many authors of LLaMA and OPT in Llama 2 papers. For example, in OPT's team, Moya Chen, Todor Mihaylov, and Punit Singh Koura join the "generative AI" team.
In the original LLaMA team, Hugo Touvron, Thibaut Lavril, Xavier Martinet, Marie-Anne Lachaux, Naman Goyal, and Aurelien Rodriguez joined the "generative AI" team.
Interestingly, in the Llama 2 thank you, three of the four teams that first initiated the study have left, and only Edouard Grave is still there.
Despite the success of the much-acclaimed open source model LLaMA, tensions among researchers are still brewing as Meta's attitude towards artificial intelligence research continues to change.
Traditionally, Meta AI has a bottom-up culture dominated by researchers, allowing teams to focus on the AI area and make breakthroughs.
But as Xiaoza became more inclined to integrate AI into Meta applications, Meta AI's focus shrank-disbanding non-product-oriented research, such as protein folding.
At the same time, new projects are also "jumping back and forth" between the spanning AI team and Meta AI.
For example, Llama 2, released by Meta in July, and Code Llama, which specializes in code generation in August, are the responsibility of two teams.
In response, Pineau said, "this field is growing so fast that we are still exploring which project Meta AI should be responsible for." Which project should be undertaken by the generative AI team? "
In front of the stage, the open source model is bright and beautiful, while behind the scenes, the tide of AI researchers leaving, math disputes, and team restructuring have brought a lot of uncertainties for Meta to try its best to catch up with its competitors.
Reference:
Https://www.theinformation.com/articles/inside-metas-ai-drama-internal-feuds-over-compute-power?rc=epv9gi
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.