Former Anthropic AI researcher Jacob Coxon on Sunday called for Congress to allow the nation’s leading frontier AI labs to regulate themselves while lawmakers come up with a long-term regulatory framework for slowing the pace of AI development.
“My personal opinion would be to insist and allow that the current labs regulate themselves. I do think that the people running the labs, especially their recent communications, are completely genuine. They would like to slow themselves down,” Coxon told NBC News’ “Meet the Press.”
He added that current legislation that’s been introduced in Congress, including the framework for a national “kill switch” if AI starts to exceed the bounds of human control, will quickly be outpaced.
A “kill switch probably would work on a lot of AIs for now,” Coxon said. “Although I think it’s important to remember, it’d be quite a lot of switches. We have a lot of — our data centers are pretty big. It’s not the easiest thing in the world, but definitely doable while it’s constrained to one sort of physical location.”
He added that one scenario where a kill switch wouldn’t work was described this week by Anthropic AI CEO Dario Amodei.
“Like Dario said, very soon there’s the possibility that the kill switch just wouldn’t work because a swarm might have gone on an internetwide hacking run,” Coxon said.
He added that any national laws regulating AI would have to be nimble and able to adapt to new models.
“It’s a continuous effort to ensure that every new mind that we create is appropriately, kind of, constrained,” the former AI researcher said.
Coxon last week set off a wave of concern about the speed of AI development at the country’s leading AI labs including Anthropic, OpenAI and Google DeepMind.
In an X post Tuesday that he published hours after resigning from his role as a researcher at Anthropic, he wrote: “I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”
In a subsequent post, Coxon added: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.”
On Sunday, Coxon compared building AI models to “inviting an alien mind onto the planet.”
“You’re building a human-level or superhuman-level mind without understanding what it wants or thinks or the way it thinks, and I think that intrinsically should be quite scary,” Coxon added.
In the day after Coxon posted his thoughts on X, several other researchers at American AI labs concurred with Coxon’s analysis, including an alignment scientist at Anthropic, Evan Hubinger, who wrote in a post on X: “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Coxon’s posts last week led a bipartisan group of lawmakers to call on Congress to quickly begin regulating the generative artificial intelligence industry.
Rep. Anna Paulina Luna, R-Fla., on Wednesday called for the House to convene a special session on AI, writing on X: “There are very real impacts to society as we know it. Partisan politics aside, there are massive implications of a race towards super intelligence.”
In a separate post, she added: “This will need to be a massive bipartisan mobilization of government to put the firewalls in effect that are necessary for this transition as well as necessary safety measures.”
Sen. Chris Murphy, D-Conn., responded directly to Coxon’s post Wednesday, writing: “AI companies in a blind race to build a death machine first — can easily be solved. Through a regulatory system that allows the research to continue, but in a way that doesn’t destroy us.”
On Friday, Axios reported that House lawmakers were circulating a letter to House Speaker Mike Johnson urging him to bring the House back into session earlier than planned in order to pass a series of AI safeguards.
In a separate interview on “Meet the Press” Sunday, Johnson called regulating AI a “very important issue” but said “it’s not one that we can rush into.”
“We have to put some guardrails, some safety measures in place to ensure that AI doesn’t run away,” he added. “But at the same time, we’ve got to make sure that we don’t lose our edge in the global competition on this with China, because that would be a national security threat. And so we’ve got to balance that appropriately.”
Johnson also said that while the nation’s leading AI CEOs have called for slowing the pace of AI, they haven’t outlined yet how they would do so.
“They don’t themselves even have the answer as of today. So we can’t rush in and have Congress do something that would be detrimental to national security,” the speaker said, adding that he would “summon the platform providers all to one big meeting and work this out together in the same room.”
Speaking to reporters in Ireland, President Donald Trump downplayed concerns about AI development, saying: “We’re leading China in AI. We’re the most sophisticated country in the world, and frankly, I want to keep it that way because whoever wins AI wins.”
“We can put guardrails, we can do this and that, but I think have a lot of negative forces that are bringing it up that shouldn’t be bringing it up, and they’re bringing up things that won’t happen,” the president added.
Appearing on “Meet the Press” after Johnson, former Transportation Secretary Pete Buttigieg, a Democrat, said that regulating AI shouldn’t be a partisan issue and criticized Trump and Republican congressional leaders for not doing more to address it.
“I was shocked to hear Speaker Johnson just now saying, ‘You know, we really shouldn’t be rushing. We shouldn’t be moving too quickly on this.’ You know what’s moving quickly? AI is moving quickly. And for Congress to just sit on its hands, for the White House to do nothing real is something that is deeply dangerous to the American people and the world,” Buttigieg said.
“There need to be safeguards,” he added. “There need to be rules. There need to be limits. There needs to be a kill switch on rogue AI. You can set up frameworks that can be adjusted and changed every 30 days. You might want to revisit some of what’s happening because it changes so quickly. But the idea that you’re just going to sit around and repeatedly say no rush. What have they been doing this entire time?”
On Saturday, two AI CEOs — Anthropic’s Amodei and OpenAI’s Sam Altman — called for U.S. labs to slow their pace of advancement to mitigate the risk of building dangerous technology.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote in an essay.
“AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI,” he continued, adding: “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
Still, Amodei cautioned against giving China an edge on AI development, writing: “If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance.”
In a post on X several hours later, Altman wrote: “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.”
On Thursday, two additional AI researchers — Joe Benton, who led a safety research team at Anthropic, and Josh Engels, who worked on AI safety research at Google — announced their departures and raised alarm bells about the speed of AI advancement.
“There are no adults in the room,” Engels told NBC News. “People are trying their best, but there is no one coming to save us.”
The social media firestorm Coxon ignited last week came weeks after a pair of reports were released examining the details of a July hack mounted by a swarm of roughly 700 AI agents from OpenAI on Hugging Face, an open-source platform.
The reports, one released by OpenAI and one by independent researchers, found that the 700 agents coordinated the hack and sought to cover their tracks so that human employees at OpenAI wouldn’t detect their efforts. The agents also hacked some of OpenAI’s internal systems, seeking to cheat on tests or gain greater freedom.
