Not so long ago, many businesses were urging employees to use AI as much as humanly possible. But as AI bills have ballooned, token-maxxing has yielded to old-fashioned belt-tightening. The consulting giant Accenture, which earlier this year reportedly told senior staff they could either start using AI or kiss their chances of being promoted to leadership goodbye, recently ordered many employees to cap their monthly Claude spending to $29 per month, according to 404 Media.
AI developers are starting to get the memo: lower costs, or get left behind.
That pressure has grown thanks to the proliferation of powerful open-source models from China which are drastically cheaper to both train and run. And now DeepSeek—the Hangzhou-based startup that badly rattled Silicon Valley early last year with the release of its R1 model—is tightening the thumbscrews on OpenAI and Anthropic even more by making its in-house inference tool freely available on GitHub.
The tool, called DeepInfer, is currently only built to run DeepSeek’s own V4-1 Flash, a faster, more efficient, and cheaper version of the company’s flagship AI model. “We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable,” the company said in an X post. Future DeepSeek models will be added to the framework as they become publicly available, the company said in a follow-up post.
Inference, the process where a trained model is actually deployed to generate new outputs based on a prompt, is one of the most computationally expensive phases in the AI lifecycle. Those costs have climbed as developers have focused more on boosting models’ agency, or their ability to handle complex, real-world tasks that require many individual steps and sometimes fleets of subagents. Earlier this year, a report from McKinsey & Company found that by the end of the decade, the majority of compute generated by data centers would go towards inference, rather than towards training new models.
One of the biggest emerging challenges for American AI companies has therefore been to offer frontier AI tools without the runaway inference costs that can leave a major dent in customer’s budgets, and in some cases, cause them to jump ship in favor of cheaper open-source alternatives from Chinese labs. DeepSeek’s V4.1-Flash is currently priced at just $0.035 per million input tokens and $0.12 per million output tokens, according to OpenRouter. OpenAI’s GPT-6 Astra, in contrast, costs $10 and $50, respectively.
The federal government hasn’t been thrilled about the growing embrace of Chinese open-source models throughout the U.S. On Monday, the Trump administration banned federal civilian agencies, such as the State Department, from using DeepSeek’s AI tools. The rule does not prohibit private companies or organizations from doing so, however.
By leveraging multiple inference techniques such as speculative decoding, DeepInfer could help DeepSeek’s customers reduce their AI spending costs even more, adding further incentive for businesses to part ways with ChatGPT or Claude. Your move, OpenAI and Anthropic.
