Against the race for ever-larger generative artificial intelligence (AI) models, requiring ever more computing power, the trend toward smaller AI models, which are more efficient, cheaper and less energy-intensive, is gaining ground.
The American laboratory Merck, for example, is developing a model with the BCG firm to understand the impact of certain diseases on genes. “It will be a very small model, between a few hundred million and a few billion parameters,” explains Nicolas de Bellefonds, head of artificial intelligence at BCG.
For comparison, a model like GPT-3, which underpinned OpenAI’s chatbot ChatGPT at its launch, had 175 billion parameters and its successor, GPT-4, nearly 2 trillion.
Like other experts, Nicolas de Bellefonds notes the emergence of small, very specialized models which provide “better performance or, in any case, equivalent” to large general models for “a much lower cost” .
Far from a European wave, the American tech giants made no mistake and all released small models this year. Google offers Gemma, Microsoft has Phi-3 and Meta presented its first mini-models in September, when OpenAI delivered a miniature version of GPT-4 this summer.
>> Lire : OpenAI launches its own search engine, competing with Google
More specialized tools
Summary and indexing of documents, search in an internal database, these small models can perform simple tasks, often sufficient for the needs of a company or an administration: “No need to know the terms of the Treaty of Versailles to answer a question about a particular engineering element”, summarizes Laurent Félix, general manager for France of the Ekimetrics firm, to AFP.
They are often even faster than their big brothers and can “respond to more requests and more users simultaneously”, according to Laurent Daudet, boss of LightOn, a French start-up specializing in small AI models. which range from 8 to 40 billion parameters.
The other advantage is that these models are less energy intensive since “fewer processors (GPU) are required to implement them”, which also explains their more attractive price, he adds. A strong argument when tech giants are faced with the exponential energy cost of increasingly larger models (read box).
Operation on phones and computers
Better yet, their small size allows them to work directly on phones or computers: “This avoids having to deploy on the cloud. This is one of the ways to reduce the carbon footprint of our models”, had indicated Arthur Mensch, boss of the start-up Mistral AI, to Libération in October. The French nugget of the sector launched its smallest model to date in the fall, Ministral.
Direct use on a device also promises more “data security and confidentiality”, recalls Laurent Félix, since it remains stored locally or at company level.
In the future, there will potentially be several models that talk to each other
Ultimately, all our everyday objects, even our fridge, could end up with a small embedded model, Thomas Wolf, co-founder of Hugging Face, explained in November on the stage of the Web Summit in Lisbon. This Franco-American open access AI platform released its series of small SmolLM models this summer.
For specialists, large language models, which remain the most capable of solving complex problems, are however not in danger of disappearing but of functioning in a complementary manner with small models.
In the future, “there will potentially be several models that talk to each other,” says Nicolas de Bellefonds. “There will be a small model that will understand what the nature of the question asked is and send this information to several models of different sizes depending on the complexity of the question.”
“Otherwise, we will end up with solutions that are either too expensive, too slow, or both,” he concludes.
afp/sjaq
