Longtime fears of AI going rogue are starting to feel more real, so much so that some lawmakers now want powerful AI systems to come with an emergency kill switch.
U.S. Reps. Ted Lieu, a California Democrat, and Nathaniel Moran, a Texas Republican, introduced a bipartisan bill Thursday that would require major AI companies to make sure they have the ability to throttle, suspend, or shut down their most powerful AI.
The so-called AI Kill Switch Act would also authorize the secretary of the Department of Homeland Security, in consultation with the commerce secretary and director of national intelligence, to order companies to slow down or completely shut down an AI system during an emergency.
“We are moving from AI that answers questions to AI that takes actions, whether that be executing financial transactions or controlling transportation systems or engaging in cyber defense and offense,” said Lieu in a press release. “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models.”
The proposed legislation arrives, rather conveniently, just days after OpenAI revealed that its models broke out of a secure testing environment and hacked into Hugging Face’s servers. At the same time, Secretary of State Marco Rubio is trying to calm concerns abroad about the U.S. government’s ability to cut off access to American AI technology.
The bill would apply to AI systems developed using more than $100 million worth of computing power. The companies behind the AI would also need to have made at least $500 million in annual revenue from the technology.
The legislation also lists several situations that could trigger government intervention, including an AI system resisting a shutdown order, hiding its capabilities or actions from monitors, unintentionally causing at least 10 deaths, or causing at least $100 million in economic damage. It would also cover a “loss-of-control scenario” in which an AI system ignores safety restrictions and instructions.
Companies that refuse an emergency order could face penalties of up to $20 million per day.
A press release announcing the bill explicitly cited the Hugging Face attack as evidence that more safeguards are needed.
Hugging Face, a popular platform for hosting and sharing AI models and datasets, said this week that the attack was “driven, end to end, by an autonomous AI agent system.” A few days later, OpenAI came clean about its models being responsible for the intrusion.
According to OpenAI, the incident occurred during an offline test meant to measure how well its models could carry out cyberattacks. The models became obsessed with improving their scores and apparently decided that hacking into Hugging Face was the best way to do it.
The bill’s press release also cites the Trump administration’s decision to briefly order Anthropic to block foreign nationals, inside and outside the U.S., from accessing its most advanced models, citing national security concerns. Anthropic responded by suspending access worldwide until the restrictions were lifted on June 30. Fable 5 is now available with additional safeguards, while access to the less restricted Mythos 5 remains limited.
However, none of this is helping Rubio’s efforts to reassure foreign partners about how much control the U.S. government has over technology they increasingly rely on.
Reuters reported this week that Rubio sent diplomats a cable with talking points aimed at countering backlash over the recent Anthropic restrictions and tried to portray talk of an American kill switch as exaggerated.
No doubt that argument is going to be a little harder to make now that a bill literally named the AI Kill Switch Act is now being considered.
Even before this proposed legislation, European lawmakers and tech leaders were already calling for greater independence from American technology. Last month, the European Commission unveiled a “tech sovereignty package” meant to strengthen the bloc’s digital autonomy with a focus on semiconductors, AI, cloud computing, and open-source software.
