The American startup did not really deploy the big game expected and to which we were treated last year for the first edition of its “Devs” event. During its DevDay 2024, OpenAI instead strove to attract AI application developers and provide them with even easier and more open access to its models that everyone agrees to recognize as the best even if the The publisher no longer has the lead over the competition that it had two years ago when ChatGPT was released.
We knew that OpenAI would not bring up new models, GPT-5 not being for right away and its new technology “ OpenAI o1 » having been revealed 15 days ago. But we still expected some announcements around ChatGPT Plus and Business offers. That wasn’t the case. After a tumultuous week marked by executive departures and significant fundraising, OpenAI, which is preparing its metamorphosis into a profit company and is preparing to announce a record fundraising of more than $6 billion, preferred to play profile and focus its efforts on attracting publishers and developers and establishing itself as the essential platform in the development of generative AI applications.
Said differently this “DevDay 2024” lived up to its name. The startup limited its announcements to a collection of new tools and features intended to simplify the exploitation and customization of its models to create a new generation of applications powered by AI and exploiting the potential of its existing models.
Conquer an increasingly competitive market
The startup has therefore strived to convince developers that it remains the best platform for building AI applications. And even though more than 3 million developers use its models, OpenAI is well aware of the intensification of competition in a booming AI market undergoing rapid innovation. Moreover, certain announcements aimed to make up for a delay in its offer in the face of certain innovations from others, such as the caching of requests which has existed for several months at Anthropic. A competition which has also had a very concrete impact on its Business: OpenAI has reduced the costs of access to its API by 99% in 2 years, thanks in particular to the introduction of its optimized “GPT-4o mini” model. Competition from Google but also from Meta – who started the price war from the start – clearly does not help the profitability of startups such as OpenAI, Anthropic and MistralAI.
Announcements for this DevDay 2024
Ultimately, the startup announced several new tools this Tuesday. Here’s what to remember from these announcements:
A new “real-time” API for more instantaneous experiences
L’API Realtime allows developers to simply create apps with voice-enabled AI experiences with minimal latency. Although it’s not exactly ChatGPT’s advanced voice mode, it’s pretty close. Six voices are offered by OpenAI, separate from those of ChatGPT, and the use of third-party voices is prohibited to avoid copyright issues.
OpenAI demonstrated a travel planning application built with the Realtime API. The user can communicate verbally with an AI assistant to prepare their trip to London and receive voice responses almost instantly with the possibility of interrupting the AI to go on another track. The API also provides access to various tools, such as real-time annotation of a map with restaurant locations mentioned.
Calling functions and telephone integration
Above all, this “Realtime API” – which is based on GPT-4o – supports the function call (function calling) which allows voice assistants to respond to user requests by triggering actions or taking advantage of a new context. Typically, a voice assistant can place an order on the user’s behalf or retrieve relevant information about the customer to personalize their responses.
OpenAI also showed how Realtime API can be used to make phone calls via services like Twilio, for example to order food for an event. Unlike Google’s famous Duo, OpenAI’s API cannot directly call restaurants or stores. Notably, OpenAI has not built in a mechanism for its AI models to automatically identify themselves during these calls, despite the realistic nature of the voices generated. So the onus is on developers to add this disclosure, a requirement that could become mandatory with the new law in California and the AI Act.
Fine-tuning vision and model improvement
OpenAI also launched a new API “Vision fine-tuning“. This feature allows developers to use images, in addition to text, to fine-tune their applications based on custom GPT-4o models. According to OpenAI, hundreds of thousands of developers have already customized GPT-4o models via textual fine-tuning APIs. With this new fine visual tuning API, “ Developers can customize the model to enhance image understanding capabilities, enabling applications such as enhanced visual search capabilities, better object detection for autonomous vehicles or smart cities, and more advanced analytics. precise medical images » explains OpenAI. The startup specifies, for all intents and purposes, that it is nevertheless prohibited to use images protected by copyright or violating OpenAI security policies.
Prompt Caching: reduce the cost of redundant requests
« Many developers use the same context repeatedly across multiple API calls when building AI applications, such as when making changes to a codebase or having long conversations in several rounds with a chatbot » notes OpenAI. Anthropic made the same observation earlier this year and introduced a prompt caching feature to reduce costs and improve latency by caching frequently used context between API calls. Promp Caching from OpenAI does exactly the same thing and would allow developers to save up to 50% of the costs of calling the GPT-4o, GPT-4o mini but also o1-mini and o1-preview APIs.
A model distillation API
Model distillation, also known as knowledge distillation, is a technique that involves training a smaller, less complex model (called a “pupil”) to replicate the performance of a larger, more complex model (called “teacher”). This method transfers knowledge from a large pre-trained model to a smaller model that is easier to manipulate and evaluate.
This will not have escaped any CIO, the trend is no longer to use large linguistic models as is, but to use smaller models that can be “fine-tuned”in other words refine or personalize, with company data to limit the risk of hallucination and improve the relevance of responses.
In addition to the “Fine-tuning” APIs, OpenAI introduces a new “Model Distillation” API to manage a distillation pipeline directly on the OpenAI platform. Typically “ This allows developers to easily use the results of frontier models like o1-preview and GPT-4o to refine and improve the performance of more economical models like GPT-4o mini. explains OpenAI. In other words, this allows developers to improve the performance of GPT-4o mini or O1-mini by using response sets produced by the large models.
In short, as we will have understood, OpenAI wanted to speak to developers of AI applications in companies and among application publishers. And not to the users of its “turnkey” solutions. No GPT-5, no Sora, no new ChatGPT Plus features, no news about GPTs and developments in the GPT-Store either. Those looking for something new in this area had to turn to Microsoft which took advantage of the moment to announce at the same time a plethora of new features for its general public Copilot (Copilot Voice, Copilot Daily, Copilot Discover, Copilot Vision, Think Deeper, etc.) which we will not dwell on since the French will not see the color for several months (but the curious can delve into the presentation of our colleagues from InformatiqueNews).
OpenAI promises that the upheavals and reorganizations underway within it will not have an impact on its capacity for innovation and its speed of execution. But it is time for the startup to complete its metamorphosis into a major AI company and demonstrate that it has not left any feathers in the process.
ALSO READ:
Eco
OpenAI: new existential crisis or metamorphosis?
ALSO READ:
Data / IA
With “o1”, OpenAI pushes its models to reason before responding…
ALSO READ:
Data / IA
OpenAI offers “Fine-Tuner” companies its GPT-4o and GPT-4o mini models
Laurent Delattre
August 21
