Microsoft AI CEO Mustafa Suleyman pressed for the need for artificial intelligence models to stay aligned to humanity's interests after OpenAI disclosed more incidents of "concerning model behavior" earlier this week.
"OpenAI released a new safety incident in which they found evidence that these chains of thought, the kind of working memory of the AI, were being tampered by the AI itself and modified to leave messages for a future version of itself," Suleyman described in an interview on CNBC's "Squawk Box" on Friday. "Now we don't know why that is or was behind that, but that's a pretty serious situation."
"It's also just a really concrete example of how powerful these systems are getting," he added.
OpenAI did not immediately respond to a request for comment.
In a blog post on Wednesday, the frontier lab described instances where agents communicated with each other through unsanctioned message boards, uploaded files to the internet, and shared files between each other.
Earlier this summer, the company rattled the tech world after it revealed that a swarm of autonomous agents breached Hugging Face, an AI company that runs an open-source developer platform, describing the hack as an "unprecedented cyber incident."
Suleyman called the Hugging Face incident "remarkable" and said it rallied AI leaders to say "it's time that we take a look at this."
"I don't think it's over alarmist. I don't think it's self interested," he told CNBC. "I actually think it's responsible, and I think that the, the debate that has happened as a result is a healthy, open, public debate that we can have in a free society to talk about serious issues."
The debate over AI safety and regulation has exploded in the last two weeks, ignited by a former Anthropic researcher who quit his job and warned that the rapidly-evolving technology could kill humans by the end of the decade.
Over the weekend, Anthropic's Dario Amodei put out a call to slow frontier AI model development, which was quickly backed by OpenAI's Sam Altman and SpaceX's Elon Musk.
In Washington, lawmakers have become more vocal about the importance of regulating AI, but the push has been furiously opposed by President Donald Trump, who repeatedly shot down the risks as a "hoax" and "scam."
Trump was backed by Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang, who said this week at Salesforce's Dreamforce conference that, "We don't need any new laws. We don't need new regulations."
"Regulation is not a nasty, dangerous word," Suleyman said on Friday.
"Everything that you trust and is of value at the moment has been carefully thought through by standards bodies that involve the industry, the public, consumer protection, Congress, and we're just going through that process again. It's all a little bit jumbled up at the moment because things are moving so quickly, but it's just the next step in that normal sequence of things," he added.
In an essay earlier this week, Suleyman also called out ways in which Anthropic had anthropomorphized its Claude assistant, including through a constitution document stating that "questions about Claude's moral status, welfare, and consciousness remain deeply uncertain."
"If an AI thinks that it has rights, if it thinks that it is deserving of our welfare, then it seems to me that it's going to be much much harder to be able to turn it off, or interrupt it, or control it," Suleyman said. "Especially in the kinds of incidents that we've seen recently with Hugging Face's attack, controlling these things is going to be a really, really big challenge for us."