OpenAI is calling for the formation of an international organization to oversee AI development. And to slow it down, if necessary: “One goal of such an organization should be to make it possible for the world to take coordinated action, including slowing frontier development when needed, so societal resilience, safety, and alignment can keep pace,” the company wrote in a blog post published Monday.
The memo, written by OpenAI CEO Sam Altman and chief scientist Jakub Pachocki, follows a similar statement from OpenAI’s chief rival, Anthropic, which said last week that unilaterally tapping the brakes on global AI development “would likely be a good thing,” assuming such a move were possible.
Anthropic’s rationale was based on internal evidence of what it described as “recursive self-improvement”; that is, AI models capable of training the next, more advanced versions of themselves, thereby presenting new possibilities for quicker advancements in the field as well as new risks around humans losing control.
The possibility of such an “intelligence explosion”—occurring as a byproduct of commercial incentives being prioritized over safety concerns—was also a key factor behind a 2023 open letter calling for a six-month pause on the development of frontier AI models, which was signed by Elon Musk, AI “godfather” and Turing Award-winner Yoshua Bengio, and other big names in tech.
OpenAI likewise called attention to AI’s self-improving potential in its new blog post. “We believe that AI doing AI research will become the determining factor of the pace of progress within the next few years,” Altman and Pachocki wrote. By March of 2028, they continued, OpenAI “may have a significant fraction of our research being done by AI systems in tandem with our own researchers.”
AGI is always around the next corner
Building such an “automated AI researcher,” they wrote, is currently one of the big goals OpenAI is currently working towards. Another is to “provide everyone on Earth with a personal AGI”… whatever that means.
The AI industry still lacks a widely agreed-upon definition for artificial general intelligence, or AGI. It’s generally understood to be an AI system that can perform any cognitive task at least as well as the typical human brain, but it’s still a matter of debate whether or not such a system—at least one built upon the architectures behind large language models—is even feasible.
For years, OpenAI has held it up as a vague promise for the future, which has thus far been a successful financing strategy. Until recently, it was the highest valued startup in the world, before being usurped by Anthropic last month.
Both companies are expected to make their stock market debuts this year, raising fresh concerns about fiscal responsibilities to shareholders eclipsing safety concerns about runaway AI. Their calls for an independent, international organization to moderate AI development, therefore, feel like a warning, as if they were saying: Somebody needs to take the reins on this, because very soon, it’s going to be out of our hands.
