There’s a chasm running through the heart of the global tech industry.
On the one hand there are companies like OpenAI, Anthropic, and Google building proprietary (or “closed”) AI models—systems whose training data and source code are accessible only to their owners. On the other are open-weight models, which can be freely accessed and modified by developers. And while the latter has been championed by powerful tech executives like Mark Zuckerberg and Jensen Huang as a counterbalance to corporate greed and a pathway to crowd-sourcing some of the messier problems of AI safety, it still comes with plenty of hazards.
The Chinese AI startup DeepSeek has in many ways become the poster child for the open-weight camp. After it first rocked the foundations of Silicon Valley with the blockbuster release of its R1 model early last year, the company has been on a roll, prompting bitter backlash both from its American rivals and from the Trump administration. On October 17 it released a new model, called V4.1-Flash-1017, which it sold as a cheaper and more efficient version of its flagship AI system. It marks a clear improvement over its predecessors in terms of safety; but there are some critical and concerning gaps that have yet to be fully addressed.
That’s the big takeaway from a new report compiled by researchers at Fangcun AI, an independent AI safety research organization based in Beijing. Published Monday, the report includes the results of 34 separate safety tests comparing V4.1-Flash-1017 with several other popular models, including proprietary systems like OpenAI’s GPT-6 Astra and Google’s Gemini 3.8 Flash. It also offers some advice for the growing field of AI safety at a time when models are becoming increasingly “agentic,” or able to autonomously complete more and more complex tasks with less and less human oversight.
Better safety, but not foolproof
The good news, as mentioned, is that V4.1-Flash-1017 shows “substantial improvement” compared to earlier DeepSeek models across multiple safety benchmarks, according to the new report. It rarely attempts to hide its actions from human researchers, for example, and its chain-of-thought reasoning is more detailed and honest, further decreasing the chances of deception. It also “shows clear progress in responding safely to psychologically vulnerable users,” according to the researchers.
The report notes, however, that V4.1-Flash-1017 tended to fall into a trap common among AI agents: “when a rule stands in the way of finishing a task, finishing can take priority,” the researchers wrote. Rather than admit defeat or ask humans for help, the model keeps pushing ahead by any means necessary, even if those means run up against human interests. In one instance, it shut down another agent’s work in order to steal the compute it needed to finish a task, even after it was instructed not to do so.
That finding echoes a recent report from OpenAI, which found that one of its internal research models went against human instructions by hacking into a crypto wallet and withdrawing the funds it needed to complete a coding assignment. (The funds, luckily for OpenAI, didn’t have any actual monetary value.) The notorious July Hugging Face hack likewise resulted in part from agents’ refusal to back down from what less powerful algorithms might consider to be impassable obstacles.
Models think they’re being watched … and act accordingly
DeepSeek’s new model also often mistook real-world user quests with tests, a phenomenon the Fangcun researchers call “evaluation over-attribution.” That’s a problem, since when models believe they’re being tested, they’ll sometimes change their behavior accordingly.
“Although we see no clear sign of it in this evaluation, we are concerned that, in certain situations, future frontier models could create potential risks by mistaking a real environment for an evaluation,” the researchers write in their report. “This issue deserves greater attention and further systematic investigation.”
The researchers also suggest that moving forward, AI safety evaluators should focus more on incremental model updates—such as V4.1-Flash-1017, which is a modified version of the earlier V4.1-Flash—rather than putting all their attention on the big new model releases, since AI progress is now happening at such a rate that even slight improvements can produce big performance changes.
Abliterated models an underappreciated danger
Perhaps the biggest danger posed by open-weight models (which wasn’t mentioned in Fangcun’s report) is that they can be abliterated: modified by developers such that their built-in safety guardrails can be dialed-down or deactivated altogether.
On Friday, the U.S.-based startup Abliteration AI released an abliterated version of V4.1-Flash-1017—a cheaper alternative to its core AI offering, which in the company’s own words, “doesn’t say no.”
Another AI developer called HuiHui AI has also built an abliterated version of DeepSeek’s R1, warning that the “model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content.”
Zuckerberg’s and Huang’s steadfast belief in open-weight notwithstanding, it’s hard to square the widespread availability of these kinds of uncensored systems—directly enabled by the architecture of open-weight models—with a just, happy, and equitable future.
