Businesses around the world are starting to rethink how they use AI, and it's because of something called tokens. A survey of 1,500 IT leaders, run for data company Cloudera in September, found that 89% have already limited or redesigned how they use AI because of rising token consumption.
South Africa wasn't one of the nine countries in the survey, whose Europe, Middle East and Africa responses came from Italy, Germany and the UAE. But the problem applies here too, so it's worth understanding what's actually going on.
What a token actually is
Every time you ask ChatGPT a question, your words get broken into little pieces called tokens. These are the units AI uses to process information, and OpenAI says one works out to about three-quarters of a word in English, so 100 tokens is roughly 75 words.
When businesses use AI through an API, the connection that lets their own software talk to a model, they're charged for the tokens they send in and for the ones the AI generates in response, usually at different prices. The catch is that models which reason through a problem also produce "thinking" tokens while working out an answer. You never see them, but OpenAI, Anthropic and Google all bill them as output. In OpenAI's own words, "A short visible answer can therefore use more tokens than its displayed text suggests."
Why a cheaper token can still mean a bigger bill
So even if the price per token gets cheaper, your overall bill can keep climbing. OpenAI's own guidance says as much: "A lower price per million tokens does not necessarily produce a lower total cost." And some prices are rising outright, with Google's flagship, Gemini 4 Argon, doubling in price on 1 December.
That's exactly what businesses are discovering. Only 19% of the IT leaders surveyed continuously monitor what they spend on tokens, while 24% have already paused or cancelled AI projects because of costs. Closer to home, Standard Bank told TechCentral its cloud costs rose 37% in the six months to June, with expanding AI one of several reasons.
And with Gartner predicting that AI coding costs will overtake the average developer's salary by 2028, expect more AI companies to compete on efficiency, not just intelligence. Anthropic's newest Claude Sonnet model is a good example of that shift: TechCentral found it more than a third cheaper in rand than its 2024 predecessor, although Anthropic's newer models count about 30% more tokens for the same text, which trims that saving to about 18%. Increasingly, what matters is not just how smart a model is, but how much it costs to get the job done.
