Claude, it turns out, is not a reliable confidant. According to a report from Gulf Coast Now, a Florida woman was arrested last month for allegedly making threats against local law enforcement in a conversation with Anthropic’s chatbot.
Per the outlet, which cites the arrest report, the 30-year-old woman from Bonita Springs, Florida, was apparently having an ongoing private conversation with Claude in which she made multiple threats directed at the Lee County Sheriff’s Office. The first threat, registered on September 26 at 5:10 AM, she allegedly wrote: “I’m going to shoot up the sheriff’s right the [expletive] now.” In a follow-up message, sent at 1:07 AM on September 27, she said, “This is 100% last chance I’m done. I got a new [expletive] gun today. [Expletive] you.”
Now, let’s set aside any debate as to the seriousness of this threat or conversation about the kind of support a person expressing these types of violent, antisocial behaviors may need, which are serious discussions that deserve thoughtful consideration. Instead, can we just think for a moment about how darkly funny it is to have to wait to send your next threatening message because you hit your token limit for the day?
Ok, with that out of our systems: Per the report, the messages the woman sent were apparently flagged by Anthropic’s automated safety monitoring system, which pings human reviewers any time a message meets a certain risk threshold. Humans reportedly took a look at these specific messages and decided they were serious enough that they should alert law enforcement, and they did.
Anthropic did not immediately respond to request for comment sent by Gizmodo.
Once the authorities were made aware of the messages, they promptly arrested the woman at her residence. She was reportedly taken into custody without any incident and charged with written threats of violence, which is a second-degree felony in Florida and carries a potential penalty of up to 15 years in prison and up to a $10,000 fine.
After her arrest, Lee County Sheriff Carmine Marceno told WINK News that the woman explained to law enforcement that she uses Claude like a “diary.” But Claude obviously does not function quite as privately as she treated it. Per Anthropic’s support documents, the company “does not disclose information about customers or end users of our services in response to government requests except in accordance with valid legal process,” but it does have an exception: “if we believe there is an emergency that may result in imminent physical harm or death, and providing such information without delay may avert that emergency.” It seems this situation rose to that level in the eyes of Anthropic.
The case is a good reminder for those who may treat chatbots as a therapist or a private record of their thoughts: you are not just typing into the ether. What you say is subject to being used not just to potentially train a model but, in certain circumstances, can even be shared with law enforcement without your consent.
So if you have some ultimately impotent rage that you just need to get off your chest, first seek out some real resources that can help rather than a chatbot that is far more qualified to be a stenographer than a psychiatrist. And if you need some immediate relief, maybe try shouting it into a pillow. Just make sure you don’t have an Alexa in the room.
