Meta Releases Llama 3

One of the highlights of this week’s AI news is that Meta officially released partial versions of its open-source model, Llama 3. Although some argue that Llama’s current license agreement is not fully open source, it can be considered open source to a certain extent, with Meta retaining some profit potential.

Llama 3 Versions and Performance

Before Llama 3, Llama 2 was already very important in the open-source domain. However, its current capabilities are lagging, so there are high expectations for the latest Llama 3. Meta released two versions of Llama 3: an 8 billion parameter version and a 70 billion parameter version. It is said that a 400 billion parameter version is still in training and will be released in the future, but it is unclear whether it will be open source. The two currently released versions are large language models (LLMs) for text. Meta stated that they might release a multimodal Llama 3 in the future.

Both the 8B and 70B versions appear to be quite powerful. In multiple tests, the Llama 3 8B version outperformed Google’s previously released smaller models, Gemma 7B and Mistral 7B. The 70B version defeated Gemini Pro 1.5 and Anthropic’s Claude 3 Sonnet, beating competitors in their respective tiers. If the 400B version is released in the future, it will likely compete with OpenAI GPT-4, Google Gemini Ultra 1.5, and Anthropic Claude 3 Opus.

Overall, it can be confidently said that Llama 3’s performance level is roughly on par with the top three models: GPT-4, Gemini 1.5, and Claude 3. However, although its capabilities have caught up, GPT-4 was released in March or April last year, and Gemini 1.5 was released at the beginning of this year. Therefore, Llama 3 is still lagging by six months to a year, with GPT-4 leading Gemini and Claude by six months and the latter leading Llama 3 by another six months. Llama 3 and Grok 1.5 are likely at the top of the second tier, trailing behind the top three, but the gap is not too significant. For Meta, Llama 3 will be directly applied to their intelligent assistant, Meta AI, to utilize Llama 3’s capabilities fully.

Competition Between Open-Source and Closed-Source Models

The recent performance of Grok 1.5 and Llama 3 has led to reflections on the competition between open-source and closed-source models. Some believe open-source models will never catch up to closed-source models, and the gap will only widen. However, others argue that closed-source models will eventually reach a plateau where they cannot progress further, and open-source models will ultimately catch up to their level. Many top AI scholars support both views, and the public can continue to observe the development.

From a business perspective, even if closed-source models can maintain a technological lead over open-source models, the difference may not be significant for most commercial applications. For example, if an application requires a performance level of 80 points, then products with a difference of 99 points and 95 points are not very meaningful, as they far exceed the required level.

Take the example of Go models. The current AI wave started with AlphaGo defeating human Go players. After DeepMind’s AlphaGo beat human players, a new version called AlphaZero trounced AlphaGo. Suppose model A is better than model B, and model A scores 99.9 points while model B scores 99 points. If they play 100 games against each other and A wins all 100 games, it seems that A is superior. However, if they were to serve as Go teachers for beginners, both A and B models would be more than sufficient, as the top human Go players might only score around 80 points. This would be the case in most consumer-facing applications (over 90%). Although there may be a performance gap between open-source and closed-source models, it may not be very significant. For example, suppose a closed-source speech recognition model misrecognizes one word out of 1 million, and an open-source model misrecognizes one word out of 100,000, although their capabilities differ by 10. In that case, they are both already sufficient for general applications.

Overall, the gap between open-source and closed-source models may not be substantial in most application domains. So, what will be the key factors in the AI race?

Critical Factors in the AI Race

In the long-term AI competition, will those who master the technology become the winners, or will those who control the channels (consumer touch points) emerge victorious? We might be in a stage where the former are the winners. However, it is highly likely that as time progresses, the importance of companies with more consumer touchpoints will gradually increase. For example, Apple currently lags in the AI domain, but they have many consumer touchpoints. Eventually, when third-party models become more powerful, Apple will also benefit.