- Published
If you have used a free version of ChatGPT or any of its AI rivals, then you are obviously getting a good deal.
asserted
you → publish → deal
Firms like Microsoft, Google and Anthropic have invested hundreds of billions of dollars in developing Large Language Models (LLMs) the tech behind those services.
asserted
Firms → invest → services
So getting ChatGPT, Claude or Gemini to help with your speech or holiday plans is a bargain.
asserted
help → help → speech
But, naturally, those firms want to recoup their investment, so they offer paid-for versions of their AI, which have extra features for tasks like coding or billing.
asserted
which → want → coding
Meanwhile, third party firms are building and selling services based on AI agents, usually based on an LLM, which are trained to do specific tasks.
asserted
which → build → tasks
But setting a price for those services is surprisingly difficult.
asserted
setting → set → services
"Trying to tie someone into a cost model for the next 12 months, two years, three years, it doesn't make any sense, honestly, because we don't know," says Simon Gooch at Saviynt, an identity management company which is incorporating agentic AI into its services.
asserted
which → try → services
That's because of rapidly changing economics around tokens, the building blocks of LLMs and agentic AI.
asserted
That → change → LLMs
When a user asks an LLM, like ChatGPT or Anthropic's Claude to answer a question, generate software code, or automate a process, that prompt is broken down into mathematical chunks called tokens, which can be processed by the model.
asserted
which → ask → model
The LLM's response also comes in the form of tokens, which are converted back into text, software code, or a set of commands to automate a process.
asserted
which → come → process
Subtle variations in the prompt can produce different answers.
asserted
variations → produce → answers
The same prompt will not always produce the same answer.
asserted
prompt → produce → answer
Different models will produce different answers.
asserted
models → produce → answers
Meanwhile, in agentic systems, businesses use multiple AI agents together to make decisions and take actions, further increasing both token use and unpredictability.
asserted
businesses → use → use
While the cost of individual tokens – or the credits used to pay for them - has plummeted in recent years, according to analysis by Goldman Sachs, the number of tokens consumed by businesses, and consumers, has skyrocketed.
uncertain
number → use → businesses
The bank forecasts that, external token consumption will increase 24 times between 2026 and 2030 to 120 quadrillion tokens a month, as companies shift to use AI agents.
asserted
companies → forecast → agents
But companies, and individuals, using AI systems often have a tenuous grasp on just how many tokens they are burning through – until they either run out or get their monthly bill.
asserted
they → use → bill
Even Microsoft has reportedly reined back, external its engineers' use of some third party coding tools, while Uber apparently tore through, external its AI coding token budget for a year in a matter of months earlier this year.
uncertain
Uber → rein → months
Will Venters, associate professor of Digital Innovation and Information Systems at the London School of Economics, said companies can be caught out as they experiment with or implement AI internally, as staff burn through tokens.
"People are finding it really hard to manage that cost…
asserted
it → say → cost
it's a non-deterministic output, so it's a non-deterministic value," he said.
asserted
he → say → ?
Companies are finding ways to work around this.
asserted
Companies → find → this
Oliver King-Smith, founder of engineering software firm smartR AI, says smaller organisations "can fly under the radar and use [flat fee] personal accounts which I am sure the big vendors don't like".
asserted
vendors → say → which
But, he says: "This has to end at some point in time, because the big guys are taking a bath on those accounts."
asserted
guys → say → accounts
Once the big AI platforms start facing pressure from shareholders to show a profit, he predicts: "They will start clamping down."
asserted
They → start → profit
King-Smith says companies should also think more carefully about what AI models to use.
asserted
companies → say → models
Companies also needed to be much more precise with their prompts, says Rob Steele, CFO at UK accounting software firm iplicit.
asserted
Steele → need → iplicit
"You wouldn't send someone in your family out to get the weekly shop without any kind of detailed instructions as to what you expect in that shopping basket, right?
asserted
you → send → basket
The situation can become difficult to control when companies build AI into a product that could be rolled out to thousands of users, Venters points out.
uncertain
Venters → become → users
AI costs could start to balloon.
uncertain
costs → start → ?
For example, managers may realise they need tokens not just for core software development, but for other tasks such as testing, security, or for implementing guard rails.
uncertain
they → realise → rails
"It's particularly hard when you're looking at agentic processes," Ventners says.
asserted
Ventners → look → processes
Employing more AI agents can be done with the click of a button, whereas expanding the human workforce would involve careful discussions over headcount and hiring, he says.
asserted
he → employ → headcount
Venters points out, while token costs might be unpredictable, it might be that the company is ultimately getting more value from their token use with AI.
uncertain
company → point → AI
"It's not quite the same as a calculator," he says.
asserted
he → say → calculator
"The more you give it, the more expensive it is, but the better the result may be
uncertain
result → give → it
"
But companies still need to pass those costs onto their own customers.
asserted
companies → need → customers
"Nobody's really figured it out," says Bill Peterson, senior director of product marketing, at Sumo Logic.
asserted
Peterson → figure → Logic
The software firm is previewing new security services based on agentic AI, he explains, but is in discussion with corporate customers about how to charge for them.
asserted
he → preview → them
"We're still having some fun conversations about this internally," he says drily.
asserted
he → have → this
Options could include simply raising prices across the board, he says, paying by results, or charging for "bundles" of incidents.
uncertain
he → include → incidents
…and 4 more, not listed.
