Yesterday, Mistral AI announced the availability of Pixtral Large, accessible from its consumer user interface, Le Chat, via its Platform.
Pixtral Large: Mistral AI scales up its VLM process
After Pixtral 12B, Pixtral Large is a VLM, a Visual Language Model (or vision language model). A VLM is more commonly called a multimodal AI. Multimodal is a term to identify a large language model capable of processing text (code is a form of text), images, surely, and potentially videos or audio files.
A VLM processes text and images as input and responds only with text as output.
Trained on the basis of Mistral Large 2, Pixtral Large uses the formula of Pixtral 12B, but it displays ten times more parameters: 124 billion in total. The text decoder has 123 billion parameters and the visual encoder has 1 billion parameters. Its context window of 128,000 tokens could ingest “30 high-resolution images” at a minimum.
The model can thus understand and explain documents, diagrams, images with a level equivalent to or higher than GPT4-o, Claude 3.5 Sonnet, LLama 3.2 90B and Gemini 1.5 Pro, according to the benchmarks carried out by the startup. Pixtral Large outperforms its opponents by a few points in the MathVista, DocVQA, VQAv2 and MM MT-Bench tests. In short, he understands the documents as much or better than his competitors and obtains better marks in the interpretation of images relating to mathematics.
Mistral Large also benefits from an update (24.11) to better handle long documents and long instructions, a new “system prompt” as well as better support for function calls.
The two LLMs are available under a commercial license (Mistral Commercial License) and another which is a little more permissive targeting researchers (Mistral Research License).
In addition to their future availability on cloud providers’ AI platforms, Mistral Large 24.11 and Pixtral Large-latest are accessible from the Le Chat interface.
The Chat molts to compete with ChatGPT
As a reminder, Le Chat is the competing application of ChatGPT used more to test the different models of Mistral AI. In beta, it has integration with a search engine allowing you to cite the sources of content generated by the chosen LLM, a canvas-type tool (similar to the Artifacts function of Claude.ai and Canvas of ChatGPT) and image generation capabilities powered by the Black Forest Labs Flux Pro “open weight” delivery model as well as faster responses.
These improvements are free… for now. “At Mistral AI, our approach to AI is different – we do not pursue artificial general intelligence at all costs,” write representatives of the startup. “Rather, our mission is to put cutting-edge AI in your hands, so you can decide what you want to do with advanced AI capabilities,” they continue. “This approach has allowed us to manage our capital frugally, while providing advanced capabilities at affordable prices. With Le Chat, we offer a generous free plan with these beta features and we are working on premium tiers with higher service guarantees.”
Clearly, the LLM provider is working on an equivalent of ChatGPT Plus and ChatGPT Enterprise. It remains to be seen whether he will call it Le Chat Plus and Le Chat Entreprise. In any case, Mistral AI also compares its solution to Claude and Perplexity.
Agents and content moderation
In fact, organizations have already been able to test these assistants, mainly through their Microsoft 365 subscription. Without connection to business data, they prove to be of little use. Publishers believe that agentic AI is the answer to this problem. Last August, Mistral AI presented the alpha version of Agents, a way to create automated flows for certain repetitive tasks. The Platform offers Agent Builder, a WISIWYG interface helping to configure these agents. The associated API must allow programmatic use, more adapted to the needs of developers. For the moment, it is only possible to deploy the agents developed through the interface.
Beyond Chat, Mistral AI recently announced new tools to control the output of its LLMs. First there is a Batch API, supposed to reduce inference costs when processing large batches of documents.
“The Batch API offers a more efficient way to process large volume requests addressed to Mistral models, at a cost 50% lower than that of a synchronous API call,” assures the startup. “If you are building AI applications where data volume is prioritized over synchronous responses, the batch API may be an ideal solution.”
Finally, the French startup is undoubtedly one of the last to offer an API dedicated to content moderation. As a reminder, Meta has trained LLama Guard in 2023, a model dedicated to filtering harmful content. Google offered Gemma Guard this year, while OpenAI launched a programming interface similar to Nvidia’s Nemo Guardrails in 2022.
