The Hidden Cost of AI Usage – Lost Revenue and Staff Time

Do you know how much money in your company is being wasted on LLM AI tokens or usage? 

This is a widespread trend via LLM AI's providing incorrect or inaccurate answers which then have to be redone or rephrased or corrected and then cost additional token credits or eats up usage limits for the user? This extra cost comes over and above the additional time that you or your staff then spends in seeking correct inputs or revisiting requests and checking data provided.Nobody in the market that we can see, has actually measured this directly, despite it adding more AI bill to most companies. 

What then follows is a Fermi estimate with some soft assumptions.

The Rough Lost Toke Value Math

Globaly LLM spend sits between $175-200 billion according to recent estimates.  

Around 10-20% of that goes to redoing, retrying, or correcting outputs that were wrong, off-target, or misunderstood. Agentic and coding workflows probably sit at the high end, since failed runs and retry loops burn a lot of tokens. Simple chat is probably lower.

That works out to roughly $17-40 billion a year in tokens spent on retries and corrections.

Caveats that change the picture

• Not all iteration is pure waste. Refining a prompt, exploring options, or steering the chat toward what you want is normal usage, and often a vague request is the central cause of the waste, and not a model error. The truly wasted share is smaller than the total redo prompt share.
• Redo’s cost more than first attempts. Each follow-up typically resends the whole conversation context, so corrections are priced above the original answer, though caching reduces this.
• Unnoticed errors don't show up at all. Wrong answers that get accepted cost no extra tokens but can cost far more downstream.
• Tokens are likely the smaller cost. The time people spend noticing mistakes, rewording, and verifying is probably worth several times the token bill.

Input vs. Output Token Costs

So in order to calculate what this may be costing your company it is important to understand what your tokens cost and what a token is.

A Token of Appreciated Cost

A token is a small chunk of text, averaging about four characters or three-quarters of an English word, that AI models use to read, process, and generate language. Instead of processing full words. An LLM breaks text down into these fragments (which include partial words, whole words, punctuation, spaces, and code symbols) to measure computational work. This specific measurement determines both how much information fits into the AI's short-term memory (context window) and exactly how much you are billed for processing your prompt and generating a response

Difference between input and output costs

Input Tokens (Prompts and Context): Generally range from $0.03 to $10.00 per million tokens. Budget or nano models (such as gpt-6-luna or gpt-5-nano) sit around $0.05 to $0.10 per million, while standard production models run $1.00 to $3.00, and flagship reasoning or frontier models reach $10.00+ per million

If you want to measure it for your own business usage, the simplest approach is to tag the turns in your logs that were retries or corrections and divide by total tokens. Many teams find that number is higher than they expected, especially with the use of AI agents.

The real pain for businesses – and particularly for small or startup businesses that rely on AI to help manage or optimise their business that may have limited resources, is when automated functions or prompts receive incorrect outputs from the LLM and need to be corrected or repeated. This build cost rapidly and my cause your business real financial pain. 

A specific daily automated search with specific parameters that then questions the validity of part of that answer provided – lands up costing you more than double in tokens as the whole request and answer feed is sent back for processing. That is a compounding cost and one that gives AI providers additional revenue. 

That is like paying for goods that you receive that are faulty and then having to rebuy more that are not with no refund for the faulty goods. 

LLM’s have no memory between calls, so every follow-up has to include the full conversation so far (your original request, the search results, and the answer) plus your new question or ask for clarification. The model can't "just look at that one part" unless that part is what you send it.

A Few Pointers on How this Works:

• Resent text is cheaper than new text. Input tokens usually cost much less than output tokens, and many providers offer prompt caching, which discounts repeated context heavily (often around 90% off, depending on provider and setup). A follow-up that resends a long search result is often a small fraction of the original cost.
• On a subscription, it's your usage limits rather than per-token billing. Long threads still eat into your allowance faster, but you aren't paying per correction.
• The expensive part is usually the search itself. If the model re-runs web searches to answer your challenge, that costs more than the resent context.

For a daily automated runs you can avoid most of the waste as follows:

1. Split verification from the main run. Send only the disputed claim and its source to a fresh, short call rather than continuing the long thread.
2. Make the original prompt demand evidence. Require a source and quote for each claim, so you can check without re-querying.
3. Use caching if you're on the API and your prompt has a big fixed prefix, like your parameters and instructions.
4. Use a cheaper model for the check. Verification of a single claim rarely needs the top-tier model.
5. Start a new conversation instead of arguing in the old one, pasting in just what's needed.

It is important to remember that fast-moving numbers like market size and revenue go out of date quickly, and a model's memory of them is the likeliest place for a confident but wrong answer. For a daily automated search, the same lesson applies. Having the model cite a source for each time-sensitive figure makes stale or invented numbers easier to catch before you spend a follow-up on them.

Most of the token or usage waste comes from a handful of habits, so working rules can fix a lot of it. If you want to optimise your token or usage in your business, these are the ones to start with.

What to do Before the Ai run call

1. Write the prompt like a spec. State the goal, constraints, output format, and what counts as done. Vague requests are the most common cause of redo’s, and the cheapest fix is a few extra sentences up front.
2. Keep a shared library of prompts that worked. If someone has already solved "summarize a contract" or "triage these tickets," reuse their prompt instead of rediscovering it. Share these prompts across your organisation.
3. Match the model to the task. Use small, cheap models for classification, extraction, and verification, and reserve the top tier for hard reasoning. Many teams overpay by always defaulting to the biggest model.
4. Don't use an LLM when a script will do. Anything deterministic (sorting, date math, regex, lookups) is cheaper and more reliable in code.

During the AI call

1.        Require evidence for time-sensitive or factual claims. A source and a short supporting excerpt for each figure makes errors checkable without a second query. My stale revenue number earlier is the example.

2.        Ask for the structure you'll check. If you'll verify claims, ask for them as a numbered list with sources, so you can challenge item 3 without re-reading everything.

3.        Cap output length and set stop conditions. Unbounded outputs and agent loops are where costs run away. Set max tokens, step limits, and retry limits on every automated job.

After a bad answer

1.        Fix the prompt, not just the output. If you correct the same mistake twice, the instruction is missing. Update the template so nobody pays for that correction again.

2.        Don't argue in a long thread. Start a fresh call with only the disputed claim and its source. Long threads resend everything each turn, and they also tend to drift.

3.        Cap retries. After two failed attempts, change something (the prompt, the model, or the approach) or hand it to a person. Blind retries rarely succeed on the third go.

Infrastructure

1.        Use prompt caching for any large fixed prefix, such as instructions, schemas, or reference documents. This is often the largest single saving on API usage.

2.        Use batch endpoints for non-urgent work. Providers typically discount asynchronous batch jobs substantially, which suits overnight runs like your daily search.

3.        Trim context. Send the relevant excerpt rather than the whole document, and summarize old conversation history instead of carrying it forward verbatim.

Measurement and governance

1.        Log and tag retries. Record which calls were retries or corrections and which prompts or teams they came from. You can't manage waste you can't see, and the number is usually higher than expected.

2.        Set per-job and per-team budgets with alerts. A runaway agent loop should trigger a notification at a threshold you picked in advance.

3.        Review the top spenders monthly. A few workflows usually account for most of the cost, and those are the ones worth optimizing.

Rules of Thumb

You will benefit most by spending effort where the money is, which usually means with your AI agents, long contexts, and high-volume pipelines. Optimizing casual chat use rarely pays off in savings.

Don't over-optimize. Some iteration is how people steer toward what they want. Chasing a zero retry rate can cost more in lost productivity than it saves in tokens, since staff time typically costs more than tokens.

And remember that the most expensive errors are the ones nobody notices. Spend some of the saved money on spot-checks of accepted outputs for high-stakes work.