[Introduction by Xin Zhiyuan]The most dramatic scene was staged inside Meta, the shortage of computing power continued, and more than half of the core authors of LLaMA have left. Even the large model development team has undergone three rounds of reorganization, how far is Meta to catch up with Google and Microsoft?
In the past 6 months, the explosion of ChatGPT has completely made Meta unable to sit still.
From the release of the open source large-scale model series LLaMA in February, to the upgrade of Llama 2, and then to the coding model Code Llama, Meta can be said to bet everything to ALL IN AI.
In the open source community, the free research and commercialization of the Llama series of models directly ignited the fission of the replacement model.
However, while Meta is doing well, the AI team is facing a wave of departures.
The Information exclusively reports,Due to internal computing power disputes, most of the team members of LLaMA and OPT projects have resigned.
Even, Meta directly abandoned the model they are developing to rival PaLM,Reorganizes Meta AI’s two lab teams to focus on Llama 2.
The departure of most of Meta’s researchers just exposed that the shortage of computing power is the core problem of layout generative AI.
14 authors of LLaMA papers, half leave
From the outside world, the tech giants have more computing resources than most companies. Some companies, including Meta, have unanimously emphasized that they are “GPU RICH” when recruiting AI researchers.
But in fact, I am well aware that Meta’s computing resource supply is also limited. As a result, the computing power dispute started in the internal team, causing many people to abandon Meta.
Specifically,Of the 14 authors of the first-generation LLaMA model released in February this year, more than half have all resigned. Some turned to AI startups, and some joined tech giants.

Paper address:https://arxiv.org/pdf/2302.13971.pdf
Research scientists and engineers who have left Meta include Timothée Lacroix, Guillaume Lample and Marie-Anne Lachaux, co-authors of the original LLaMA paper.
They now work at French startup Mistral AI, which Lacroix and Lample co-founded in June.

One of the authors, Armand Joulin, research director at Meta AI, left Meta in May and has now joined Apple.

Joelle Pineau, head of “Basic AI Research” at the Meta Artificial Intelligence Research Lab, said in an interview:
Retaining and attracting great people is probably where I spend the most time because I can’t do anything without great researchers.
The wave of departures Meta is facing just underscores the challenge of big tech companies coping with the surge in demand for artificial intelligence talent. At present, the wave of large models, major technology companies are afraid of falling behind, eager to integrate AI into their own products and services.
According to Kyle Langworthy, an executive at Riviera Partners, companies that are “hungry for talent” are lining up to poach Meta’s artificial intelligence researchers.
In February, after the release of LLaMA ignited the enthusiasm of the community, Xiao Zha once announced the company’s big move to the outside world overnight: the establishment of a top product team to focus on AIGC.
In short,Xiao Zha puts the core focus of Meta’s future on generative artificial intelligence in an attempt to catch up with competitors such as Google, Microsoft, and OpenAI.
And the “metaverse” that was All In in 2021 has also been placed in a secondary position.

Immediately afterwards, Meta released and open-sourced the latest research in the field of artificial intelligence from time to time, including the general visual model SAM “segment everything”, multi-modal AI models ImageBind, Llama2, Code Llama, etc.
Once released, these studies have aroused great repercussions in the open source community and received unanimous praise.
Now, with the loss of experienced researchers, Meta has added another obstacle to catch up with competitors.

So, what exactly happened to make these senior researchers leave in anger?
Competing Competing Between LLaMA and OPT
In 2013, after Yann LeCun joined Meta, he established the artificial intelligence research laboratory Meta AI (formerly FAIR) when he was the director of AI research.
In addition to its main labs in Menlo Park, California, and New York (where LeCun is located), Meta AI has offices in Montreal (where Pineau is located), Paris, London, and Seattle.

The establishment of the Meta AI laboratory is mainly dedicated to the research of AI, applying artificial intelligence to various scenarios such as translation and MRI scanning.However, after the birth of ChatGPT, the company’s executives became more eager to integrate AI into products at a deeper level..
In fact, before the release of ChatGPT, Meta has been developing its own LLM.
In May 2022, a North American team of Meta AI announced to the public,And open source the large model OPT-175B.Meta claims that this model consumes less energy than GPT-3, even though the two models learn the same amount during training.

A few months later, the team began working on a second, larger model, designed to compete with Google’s PaLM.
Meanwhile, another Paris-based Meta AI team has begun work on a separate large language model, or LLaMA.
The model is smaller than OPT, the people said. The research team believes that smaller models will be more efficient at inference.
Then came the problem,Teams from two laboratories in North America and Paris started a battle for computing power resources, which intensified internal team disputes.
In particular, the LLaMA team felt neglected, their models were small, and they received much less computing resources than the OPT team in North America.
Joelle Pineau, director of Basic AI Research, said:
Decisions about how to allocate computing resources at Meta are made roughly monthly by a small group of leaders from different parts of the business.
How much is ultimately allocated, and to whom, depends on organizational priorities and how far the project is from release.
If there is a dispute about the allocation of Meta AI employees, it will be submitted to the superior, which is the level of Pineau for processing.
In the interview,Pineau admits tension between LLaMA and OPT teams over hashrate distributionadding, “We’re trying to find a way to hear everyone, even if we can’t meet everyone’s needs.”
To some researchers, this feels like a race.Some Meta AI executives also questioned why both teams were working on similar projects but competing with each other for limited resources.
Big changes in February
Tensions between the two teams reached a climax in February this year. Just this month, in an effort to catch up to its competitors, Meta AI released LLaMA as an open-source model for the first time, licensed for research purposes.
In fact, Antoine Bordes, co-director of Meta AI and head of the Paris branch, left Meta a week before the release, and the public reason for leaving was that the working hours were as exhausting as California.
Now, Bordes has joined the artificial intelligence company Helsing.

His departure further insulates the LLaMA team from executives in North America, people familiar with the matter said. Additionally, Pineau acknowledged that Bordes’ departure “created some uncertainty” for the team.
This isn’t just a matter of research morale on the LLaMA team; meanwhile, the OPT team has been facing attrition over the past few months.
It is said that the larger model being developed by the OPT team was directly abandoned by Meta. It is not difficult to understand that the loss of OPT personnel is directly related to this decision.
Half of the 19 authors of OPT papers have also left Meta, according to the study authors’ Linkedin profiles.

Paper address:https://arxiv.org/pdf/2205.01068.pdf
Pineau, the head of “Basic AI Research”, said helplessly, “When all teams want to upgrade the model at this time, then you either choose one of them, or let them cooperate.”
Team reorganization, unclear positioning
While all this was happening, Meta AI was also overwhelmed by frequent company changes.
Since November last year, Xiao Zha has carried out several company-wide layoffs. Among them, Meta AI was not spared.
In February, Meta AI leaders decided to bring together some members of the competing LLaMA and OPT teams,Officially established the “Generative AI” team(led by ex-Apple executive Ahmad Al-Dahle), but also drew a large number of people from Meta AI.
In fact, we can see many authors of LLaMA and OPT in the Llama 2 paper. For example, in the OPT team, Moya Chen, Todor Mihaylov, and Punit Singh Koura joined the “Generative AI” team.
In the original LLaMA team, Hugo Touvron, Thibaut Lavril, Xavier Martinet, Marie-Anne Lachaux, Naman Goyal, and Aurelien Rodriguez joined the “Generative AI” team.
Interestingly, in Llama 2’s acknowledgments,Of the four mentioned teams that initiated this research, three of them have left, and only Edouard Grave is still there.
![]()
Despite the modest success of the lauded open-source model LLaMA, tensions are still brewing among researchers as Meta’s attitude toward AI research continues to change.
Meta AI has traditionally had a bottom-up, researcher-led culture, allowing the team to focus on and make breakthrough advances in the field of AI.
But as Xiao Zha became more and more inclined to integrate AI into Meta’s applications, Meta AI’s focus also narrowed—— Disbanded research that was not product oriented, such as protein folding.

At the same time, new projects are also “repeatedly jumping” between the generative AI team and Meta AI.
For example, Llama 2, released by Meta in July, and Code Llama, which specializes in code generation, released in August, are each in charge of two teams.


In this regard, Pineau said, “This field is developing too fast. We are still exploring, which project should be responsible for Meta AI? Which project should be responsible for the generative AI team?”
In front of the stage, the open source model is glamorous and beautiful, but behind the scenes, the surge of AI researchers leaving, computing power disputes, and team reorganization have brought many uncertain factors to Meta’s efforts to catch up with its opponents.
References: