The software company SAP recently made headlines with the announcement that it wants to curb its AI spending by giving individual employees a budget. Understandable as that step is – AI token consumption can become opaque and expensive very quickly – rigid budgets are not the only remedy.
In this first article of a series on using AI efficiently, we explain the problem. In the following parts we share tips on how to keep performance, security and cost in a sensible balance.
The cost of AI tokens: what happened at SAP?
“SAP continuously reviews its investments in order to deploy resources where they create the greatest long-term value for customers,” the company said in July 2026. “We are therefore investing more heavily in AI-related skills, talent and technologies. At the same time, we are being more deliberate and targeted about new hires, external spending and internal travel.” Behind this lies the goal of capping the rising cost of AI services.
At the DAX group from Walldorf, token usage had apparently got out of hand. Management responded with a public announcement that it would budget precisely and review spending in a classic controlling process. Employees are to receive different budgets depending on their role, with tokens serving as the central unit of measurement for AI usage and billing. The standard range is between 100 and 5,000 euros per month, with one-off exceptions of up to 50,000 euros. The rollout is still under way.
Hardly any company will succeed long term without AI. But anyone who burns through tokens like a bag of popcorn burns money fast – and still cannot measure what the AI is worth.
What problem is SAP actually solving?
According to the consultancy and analyst firm Gartner, global spending on AI models and platforms is rising to 64 billion dollars – a considerable sum and a hefty increase of 63 per cent. How much of that SAP accounts for is not known. What is known is the other side of the equation: up to 2 billion euros are to be saved with the help of AI in software development.
In research and development in particular – with around 38,000 employees the largest unit – the potential is immense. Tools such as Claude Code and GitHub Copilot draft code, write documentation and run tests. Those processes come at a price. So the workforce and token usage have to be brought into the right relationship. But what is the right relationship?
Budget caps alone optimise the wrong metric
A token budget of 500 euros corresponds to roughly two to three hours of developer time. Locking a developer out on the 20th of the month saves a three-digit amount and risks a four-digit one. The cap limits the visible invoice, not the actual cost.
The better approach is transparency plus steering via unit cost: what does a pull request, a resolved ticket, an automatically generated report actually cost? Only that figure makes it possible to answer whether the investment pays off. SAP does some of this too – but the publicly announced cap looks more like a disciplinary measure than a savings measure. It also serves PR and investor relations, after Europe's largest software group recently faced criticism over high costs.
Three patterns recur in our projects:
- Cost without a reference value. Tokens per month say nothing as long as nobody knows what output sits behind them.
- The wrong model for the wrong task. Most requests do not need a frontier model; routing and smaller models often cut the bill substantially without anyone noticing.
- Missing data discipline. Sending the same context down the wire with every prompt means paying for redundancy instead of intelligence.
What are the alternatives – and how do we work at control-f?
This is exactly why our thinking has shifted over the past months. We now use self-hosted LLMs only. Our models run on infrastructure we control. And we can keep an eye on costs for the whole team through clear agreements and flat rates, without allocating quotas to individual departments or employees.
The side effect matters at least as much as the cost effect: data does not leave our environment. For customers in the energy sector and in critical infrastructure, that is not a convenience but a precondition. How this works in technical and organisational terms is the subject of the next article in this series.
If you would like to know how to run a self-hosted LLM productively, feel free to get in touch.
Kategorie / Category: Blogposts
