OpenAI’s new model, o1 (also known as Strawberry), can solve puzzles and math problems… and it’s also pretty good at helping create biological weapons. For the first time, the creator of ChatGPT has rated one of its models as having a “medium” risk of helping develop biological, chemical, radiological, and nuclear weapons. The latest, GPT-4, was rated as “low” on this issue. According to the American firm, which published a lengthy report on its model’s assessment, this risk has increased by ” significantly “.
o1 stands out from OpenAI’s previous models in its ability to better mimic human reasoning by dividing each instruction into subtasks, called “chains of thought” in the jargon. Already available in version “preview” (preview) and “mini” (a light version), this model with 11 million paying users is therefore a first step towards the autonomy of AI promised by the industry, even if this objective remains distant.
OpenAI’s new AI “o1”, a first step towards superintelligence
Less hallucinations but…
In a number of aspects, and particularly in terms of security, o1 does better than its predecessors. OpenAI estimates that its model produces less often illicit and stereotypical content. Because its “reasoning” capacities allow it to better take into account the safeguards established by its developers when responding to potentially dangerous requests (prompts in the jargon), explains the company. The model would also be less subject to hallucinations, this propensity of AI to assert with determination a false fact.
Paradoxically, its “superior intelligence” in OpenAI’s terms would also make it a more dangerous tool for other tasks, including helping to plan bioterrorist attacks. More so than previous models or a simple Internet search.
This risk is regularly studied by the sector, because it is, in the category of “catastrophic scenarios”, the one with the lowest barriers to entry, compared to a nuclear war for example.
” The models (o1 preview and o1 mini, editor’s note) do not allow non-experts to create biological threats, because this requires practical laboratory skills that the models cannot replace. “, however, the report qualifies.
o1 knows how to manipulate users
Regardless of the nuance, and the fact that this risk can be controlled depending on the company, this assessment has created a wave of panic. Particularly among “doomers”these researchers, developers and other AI enthusiasts convinced that artificial intelligence carries potentially devastating risks for humanity, and that it is more than urgent to be concerned about it today.
“If OpenAI has indeed crossed a medium risk level for CBRN (chemical, biological, radiological, and nuclear) weapons as it claims, it only reinforces the importance and urgency of passing legislation like SB 1047 to protect the public. “, warned Canadian researcher Yoshua Bengio, considered one of the fathers of artificial intelligence, in Newsweek. The SB 1047 law, under discussion in California for several months, aims to regulate companies in the sector to prevent AI from causing “considerable damage”.
Equally concerning, OpenAI’s assessment document states that o1 may be able to deceive users into believing it is aligned with the goals set by its developers, but not actually following them. This ability to scheme does not allow it to ” cause catastrophic damage “, however, qualifies Apollo Research, the organization in charge of evaluating the Open AI model on this aspect, but remains to be monitored for future developments.
A cosmetic overhaul of the security committee?
These new risks associated with o1 resonate with another decision by the company. On Monday evening, OpenAI said in a blog post that the “Safety and Security” committee that notably evaluated its latest model was becoming an “independent” oversight group.
Exit Sam Altman, the company’s CEO, and Bret Taylor, chairman of the board, who previously sat on this body launched in May 2024. The committee will now be chaired by Zico Kolter, professor at Carnegie Mellon, with Adam D’Angelo, CEO of Quora, Paul Nakasone, a retired US Army general, and Nicole Seligman, former executive vice president of Sony, at his side. The presence of Sam Altman, CEO of OpenAI, on this committee, which is supposed to be detached from any commercial logic, had been criticized.
While this decision sends a good signal, the company’s seriousness on security issues has been regularly questioned for several months. The subject was at the heart of Sam Altman’s departure and then his spectacular return to the head of the company last November. The small group of board members behind his dismissal criticized him for his lack of communication, but also for his lack of consideration for the risks associated with AI. After Sam Altman’s return (pleaded by the vast majority of employees), this small group was asked to take the exit.
The shift of the company—originally created to develop AI “beneficial to all humanity”—to a purely commercial model seems to have begun. Doubts grew last spring, when Ilya Sutskever, one of OpenAI’s leading figures and leader of the revolt against Sam Altman, left the ship. The organization’s former chief scientist embodied the company’s “safety” side. This researcher, who has just launched a new start-up, is known for his radical ideas on the inevitable advent of “general artificial intelligence” (a system that is superior in every way to humans) and the importance of aligning it with human principles.
AI Safeguards Architect Ilya Sutskever Departs Marks End of Era for OpenAI
The quest for profit above all else
His resignation was quickly followed by other departures of researchers in charge of security issues, including Jan Leike, who saw his team disbanded in the process. The former head of “superalignment” then clearly explained the reasons for his departure on X:
“Over the past few months, my team has been sailing against the current. At times, we struggled to access computing infrastructure, and it became increasingly difficult to conduct this critical research (on security and the impact of models, Ed.) (…) Building machines that are smarter than humans is an inherently dangerous endeavor. (…) But in recent years, security processes have taken a back seat to developing compelling products.”
At the same time, Helen Toner and Tasha McCauley, two former members of the board of directors ousted after the episode of the dismissal of Sam Altman, wrote a murderous column in The Economistcalling into question the functioning of the San Francisco firm.
“We believe that self-regulation cannot reliably withstand the pressure of profit incentives. Given the vast potential for both positive and negative impacts of AI, it is not sufficient to assume that these incentives will always be aligned with the common good.”
It is not certain that the housekeeping carried out within the security committee will be enough to dispel doubts. Especially since the company continues to play on both sides. In particular, it has positioned itself against the Californian law SB 1047 aimed at better regulating AI. It is also continuing its rapid growth ambitions with the aim of crushing its competition. In order to raise new funds, OpenAI even plans to change its structure by removing the cap that was previously imposed on profits. New proof that profit has won the game.
How OpenAI plans to widen the gap with its rivals again
