OpenAI has just presented its new o3 model. If it is difficult to position ourselves on the real importance of this latest model of the company of the year 2024, it also masks the difficulties of OpenAI in releasing its model which should mark its next milestone, the famous GPT- 5, internal code name: Orion.
The “o” model series was launched in September with, unlike the GPT-x models, the ability to evaluate and adapt its response if necessary. OpenAI speaks of a model of “ reasoning », of which he has just presented the second generation o3. We can say that the GPT-x models are more “rough-cut” in their responses.
In November, via The Information, we learned that OpenAI was having difficulty achieving the performance increase that the company hoped to compare to its previous GPT-4. The jump would not be as impressive as between GPT-3 and GPT-4.
A delay compared to the announcement made to Microsoft
But the Wall Street Journal published an article detailing a little more the difficulties encountered by OpenAI for its fifth main version of GPT.
According to the American economic newspaper, the company had promised Microsoft that its new model would be released around the summer of 2024. Remember that the multinational invested billions of dollars in OpenAI and became its exclusive partner in January 2023.
The company has already completed two long and important training rounds for its hypothetical GPT-5. And “ each time, new problems appeared » and the model would not have given the results expected by OpenAI researchers.
Orion's results would be better than those of current OpenAI models but would not justify “ the enormous cost of operating the new model “. The newspaper estimates the cost of the calculations necessary for training the model at half a billion dollars.
A project dating from summer 2023
The economic journal explains that the OpenAI teams have been working on this new model since mid-2023. And, from the start, the company would have encountered difficulties. Designing the model took longer than expected. According to Wall Street Journal sources, this already indicated that training the model would take longer. OpenAI researchers notably assessed at that time that they lacked diversified and quality data.
The company has also reportedly hired engineers and mathematicians to write code and solve mathematical problems to serve as training data for more complex tasks than those carried out by the data workers usually employed by artificial intelligence companies. . The company would also have asked them to explain their reasoning in writing to further inform its model.
But the delays within OpenAI, and in particular during the crazy week it spent in November 2023, also put a brake on the project. The company finally relaunched Project Orion in early 2024, according to the Wall Street Journal, thinking it would rely on better data. Training of the model would have resumed in May 2024.
OpenAI researchers quickly realized that the data was not yet varied enough but the company continued the training, not immediately noticing any repercussions. These apparently only became visible at the end of the training. Due to lack of time and money, the company would not have resumed training at the beginning and would have continued training while adding new data without it being certain that this strategy would be conclusive.
Sam Altman tries to downplay
In October, Sam Altman took the lead by announcing during an AMA on Reddit that OpenAI would be released “ some very good releases coming later this year ” more ” nothing we'll call GPT-5 ».
The CEO of OpenAI also reacted to the Wall Street Journal article by highlighting the concomitance with the release of o3: “ I think the WSJ is the best newspaper in America right now, but they published an article titled “The next big leap in AI is overdue and extremely expensive” several hours after we announced o3? ! » he commented on X.
