Zuck's new AI drop is cool and all, but he's still playing catch-up with Sam Altman

Mark Zuckerberg unveiled Llama 3, Meta's most powerful AI yet, but the models still fall short of OpenAI's GPT-4.

• Mark Zuckerberg just dropped Meta's new AI models.
• Llama 3, revealed Thursday, outperforms several of its competitor models.
• Bt what Llama 3 does not do is beat OpenAI's GPT-4.

Mark Zuckerberg just dropped Meta's newest artificial intelligence models — and they're giving people plenty of reason to get hyped.

Llama 3, unveiled on Thursday, marks the Meta chief's latest effort to take on his rivals in the race to develop more powerful forms of AI. The company bills the next-generation release as the "most capable" openly-available models to date.

Despite being the same size as their predecessors, the Llama 3 models "establish a new state-of-the-art at those scales," Meta said. That's thanks to improved capabilities in areas such as reasoning and code generation.

Other tweaks have been made too to address complaints that the last-gen AI models were "a little bit sanctimonious," as Meta's Nick Clegg put it in an interview with the Financial Times.

Perhaps the most important detail was that Meta's open-source models will soon be on par with their closed-source counterparts. Larger, 400-billion parameter Llama 3 models with multimodal features — which Meta said are still in training — will be comparable to proprietary models such as Anthropic's Claude 3 Opus and OpenAI's GPT-4 on release.

For open-source AI developers, that'll be a huge deal. The AI models they were working on looked pretty rudimentary as recently as last year as they struggled to complete sentences without repeating themselves.

As AI founder Sharon Zhou pointed out last month, seeing a response to a prompt like "'my name is is is is is' wasn't uncommon."

So yes, Llama 3 is a big step forward for Zuck and the wider open-source community's AI ambitions. But it also shows that Sam Altman's year-old model is still the standard bearer.

Catching up to GPT-4

OpenAI's GPT-4, revealed in March 2023, has, for the past year, been the model to beat with everyone from giants of the industry like Google to buzzier startups like Anthropic and Mistral seeking to emulate its performance.

Google's Gemini, launched in February, just marginally beat GPT-4 on certain performance benchmarks. Anthropic's Claude models, released last month, also demonstrated marginal improvements on OpenAI's model.

But instead of outright beating GPT-4, Llama 3's forthcoming models will eventually join a string of other competitors that put themselves on the same footing as a product OpenAI put out a year ago.

That's not to be dismissive of the developments Llama 3 does represent.

Jim Fan, senior research manager at Nvidia, wrote on X that he expects the larger Llama 3 models to mark the "watershed moment that the community gains open-weight access to a GPT-4 class model," a moment that "will change the calculus" for researchers and startups.

But it's clear that nothing has yet arrived that comfortably clears the performance of what Altman has already released.

Zuckerberg doesn't seem too concerned about trying to beat GPT-4, yet. He told The Verge that the goal with Llama 3 "wasn't to build something that was way ahead." His aim was to bring AI with the capabilities of today's top models to platforms such as Facebook and Instagram.

In January, he did hint that his overall ambition was to beat GPT-4, however, after publishing a short video in which he said Meta's "long-term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit."

It remains to be seen when and how a company competing for AI supremacy makes the next leap in performance.

Some in the industry, like New York University professor and AI expert Gary Marcus, see evidence emerging that models are reaching "a period of diminishing returns," raising questions about whether a generational leap in performance will be seen again.

For now, though, it's clear that Zuck and co are playing a game of catch-up with Altman.