Skip to content
Alpha BoardBrief
Sign in Subscribe

AI Is Getting Cheaper. So Why Is Your AI Bill Going Up?

By Alpha Editorial Board
AI Is Getting Cheaper. So Why Is Your AI Bill Going Up?
Published:
|
Share to X Share to Bluesky Share to Facebook Share to LinkedIn Share by email

Every AI provider tells the same reassuring story: prices per token are falling, models are getting more efficient, and intelligence has never been cheaper to buy. OpenAI recently made the case explicitly. With its latest models, the company reports that the price per million tokens has fallen by more than ninety percent from GPT‑4 to GPT‑5.4, and that GPT‑5.6 now delivers better coding performance using far fewer output tokens in less time per task. All of that is true. It is also true that finance teams across the enterprise are watching AI spend rise faster than almost any other line item on the books. Both trends are real, and reconciling them has quickly become one of the most important financial literacy tests a CFO or audit committee chair now faces.

Gartner now expects embedded AI agents to move from the margins to the mainstream, forecasting that their presence in enterprise applications will rise from roughly five percent at the start of 2026 to about forty percent by year end. At the same time, recent industry surveys indicate that nearly ninety percent of software companies already use AI in at least one production workflow, with close to forty percent relying on OpenAI or OpenAI‑compatible models as part of their stack. In that environment, AI token costs have become a standard component of SaaS cost of goods sold rather than a small, experimental operating expense. At the same time, FinOps data shows that more than seventy percent of companies reported AI or machine learning cost overruns in 2025, and almost half say AI has become their least predictable cloud category. The financial story is straightforward. Unit prices are falling while barrels of intelligence consumed are exploding. The bill reflects both.

Tokens have become the meter that measures how much intelligence an enterprise consumes. With agentic workflows, coding copilots, and voice‑based AI coworkers spreading through every function, token spend is no longer a rounding error in an innovation budget. It is becoming a core operating expense and, by the end of this decade, it will be one of the largest ones. Unless boards treat tokens as a governed commodity, they will discover, too late, that continuous intelligence has arrived without continuous governance, and that the falling price tag was never the whole story.

What a token actually is

Technology companies excel at making simple ideas sound complicated, and the token is a perfect example. Strip away the jargon and a token is simply a small chunk of text or code that an AI model reads or writes. Sometimes a token is a whole word. Sometimes it is part of a word, a punctuation mark, or a space. When you type a question into an AI system, that question is chopped into tokens before the system can process it. When the system replies, it produces its answer one token at a time, like someone typing very fast, one keystroke at a time, except each keystroke costs money.

Technically, a token is roughly three to four characters of English text, or about three quarters of a word (you can try it out yourself with a tokenizer like this one from OpenAI). That may sound small, but the scale adds up quickly, and seeing it step by step turns every dollar figure from an abstraction into something you can picture.

As enterprises rush to adopt AI, the key thing to remember is that every token, both in and out, has a price attached. Providers charge by the million tokens processed, so you can think of "one million tokens" as a standard unit of intelligence, the meter’s basic unit the way a barrel is a standard unit of oil or a kilowatt‑hour is a standard unit of electricity. That unit is what makes very different kinds of AI work comparable on a single meter. A short question and answer might use a few hundred tokens, a sliver of that unit. A complex task where an agent reads files, plans across several steps, calls other software, and writes a report back can burn through hundreds of thousands of tokens in seconds, a novel’s worth of reading and writing for one errand.

One technical wrinkle to keep in mind for anyone forecasting this spend. Token generation is probabilistic. Ask the same model the same question twice and it may take a slightly different path each time, choosing among several plausible next tokens rather than following a fixed script. Two runs of an identical workflow can consume different token counts. No finance model built on a single fixed cost per task will survive contact with real usage. AI budgets need ranges and confidence intervals, the way a business models commodity price risk rather than assuming oil or foreign exchange sit at one number all year.

Tokenomics: the economics of your meter

Tokenomics is simply the economics of tokens. It asks how many you use, what they cost, and what you get in return. It treats tokens the way you already treat kilowatt‑hours or cloud compute, as a priced unit of work that can be tracked, forecast, and governed. Its central insight is blunt. Token expenditure and economic value are not the same thing. Two workflows can burn the same number of tokens and produce very different business outcomes, depending on where in the process those tokens were spent and what happened downstream.

From a board or CFO’s perspective, three levers define your token economics:

The more clearly those three levers are understood, the less surprising the invoice will be.

The fine print on the meter

We're starting to hear a lot about the rising cost of AI. Let's look beyond the headlines and instead focus on the fine print. The meter is counting much more than just the visible question and answer tokens, and that gap in understanding is why folks are being surprised.

Think of an AI task like a restaurant check. You ordered the entrée and you can see it on the table. The check, however, prices everything the kitchen did to produce it. A task’s total token footprint decomposes into five categories:

You see the first and the fourth. You pay for all five. The other three are the ones anyone in a finance role needs to understand.

Hidden reasoning tokens come first. Before a modern AI model gives you its answer, it often thinks internally, working through intermediate steps the way a person scribbles notes before speaking. Those internal steps are made of tokens. They are billed, often at the premium output rate, and they are usually invisible to the person who asked the question. You see a clean, short answer. What happened underneath may have consumed many multiples of what the transcript suggests. The per‑token price went down. The number of tokens quietly went up, without anyone realizing it.

Output tokens cost more than input tokens. Across nearly every major provider, generating a token costs several times more than reading one, often three to six times as much. The reason is mechanical rather than arbitrary. A model can read an entire prompt more or less in parallel, but it can only write its answer one token after another, redoing a small calculation for every word it produces. The asymmetry is why a chatty, over‑explaining AI system is disproportionately expensive compared with one that reads a lot and answers concisely. What you feed the model matters. How much you let it talk matters more.

Cached tokens are nearly free by comparison. In plain terms, cached tokens are tokens the model has already processed and stored so it can reuse them without doing the full work again. When a system reuses information it has already processed, standing instructions, a document that keeps coming up, providers pull from this stored version instead of rereading and recomputing it from scratch. Cached input typically costs a fraction of fresh input, sometimes fifty to ninety percent less. OpenAI’s current pricing, for example, lists GPT‑5.4 cached input at twenty five cents per million tokens against two dollars fifty for fresh input. Prompt caching is one of the few genuinely free efficiency gains available to any organization, and most have never turned it on deliberately.

In addition to understanding tokens, we need to understand how both the AI models and AI agents also factor into tokenomics.

As you can imagine, the AI models are the engines behind your tokens. A model is a specific AI system, the engine under the hood. Models come in different sizes and capabilities. Some are small, fast, and cheap. Others are large, slower, and more expensive, but far more capable. Economically, larger models typically have more parameters, more internal wiring, which makes them more capable but more expensive per token, with higher per‑token prices, especially on output. The question for a CFO is rarely “can we use the biggest model” and almost always “when does using the biggest model pay for itself in fewer retries, better decisions, or less rework.” Routing tasks to the smallest model that clears the quality bar is one of the core disciplines of tokenomics.

As for AI agents, this is how tokens multiply in practice. With the model as the engine, it powers the AI agents. An agent is simply software that can execute an objective. It is an AI system that does not just answer a question. It takes action. It reads files, browses systems, calls other software, and stitches those steps together into multi‑stage workflows largely on its own. It can be semi‑autonomous, checking in with the human who gave it the objective, or fully autonomous. It will not stop until it achieves the objective.

In tokenomics terms, agents are the multiplier. They turn one request into a sequence of steps, each burning tokens. On every step, they tend to reread the full context, which compounds token usage. A simple chat interaction might use a few hundred tokens, a text message worth of work. A comparable agentic workflow can easily run into tens or hundreds of thousands, a novel’s worth of work, because every loop rereads and reasons over the entire context. That multiplicative effect is why agent design and guardrails, how long an agent can run, how much it can read, how often it loops, belong squarely inside the economic conversation, not just the technical one.

Pricing out one task

Consider a simple errand. An employee asks a frontier AI agent to book a business trip with flights and two hotel options, within policy. From the employee’s perspective, very little happens. A request of about sixty words goes in. An itinerary of about four hundred words comes back. Call it six hundred tokens of visible exchange, a fraction of one percent of a million‑token unit. On the surface, it looks trivial.

The meter, however, records the full footprint. Under the hood, the task consumes roughly:

Total consumption is therefore about 110,000 tokens. That is roughly the length of a full novel for a task whose visible footprint was six hundred tokens, about a page.

The reasoning steps dominate because of a mechanism called context replay. An agent works in a loop, and on each step of the loop it rereads everything accumulated so far, the instructions, the policy documents, the retrieved data, its own prior reasoning, before deciding what to do next. A twelve‑step task does not cost twelve answers. It costs twelve progressively larger rereadings plus twelve rounds of reasoning. That compounding, not verbosity alone, is what carries agentic tasks from thousands of tokens into the hundreds of thousands and why empirical profiles consistently find agentic workflows consuming orders of magnitude more than the chat interactions they replaced.

Now consider how to price all of this. Split the tokens into two buckets:

Assume frontier pricing of 5 dollars per 1,000,000 input tokens and 25 to 30 dollars per 1,000,000 output tokens.

Step by step:

Add the two together:

Rounded to something a finance pack can work with, the task costs about 1.70 dollars on a frontier model. If you instead priced output at 30 dollars per 1,000,000 tokens, the reasoning and output side would rise to about 1.74 dollars and the total task cost to just over 2 dollars, but the order of magnitude does not change the lesson.

Now compare that with a mid‑tier model handling the same errand without the heavy agentic loop. Suppose it completes the booking in 3,000 tokens end to end at the same 5 dollars per 1,000,000 tokens input rate, and output priced similarly or lower.

If you allow for a slightly higher rate or a few more tokens, “about two cents” per task is still a reasonable, conservative estimate.

The spread between these two paths is stark. A frontier agent version costs roughly 1.70 dollars per trip. A mid‑tier model version costs roughly 0.02 dollars per trip. Dividing 1.70 by 0.02 gives a ratio of about 85. In other words, there is an eighty‑plus‑fold difference in cost for one harmless task depending on how you configure the workflow.

Now multiply that frontier version through a typical enterprise day. Suppose:

Annual cost is:

So one simple errand, scaled across a fleet of frontier agents, compounds into roughly eight and a half million dollars a year. Almost all of that burn, on the order of ninety‑plus percent, lives in hidden context and reasoning tokens that never appear in any transcript anyone reads.

Tokens as the new commodity

Finance teams already know how to think in units. A barrel of oil, a kilowatt‑hour of electricity, a ton of steel, a basis point of foreign exchange exposure. Each one is a priced unit of work or risk that can be forecast, hedged, and governed. Tokens now belong on that same list.

If one million tokens is treated as a standard unit of intelligence, AI spend starts to look less like an opaque cloud invoice and more like a commodity curve. The effective price per unit is falling. The number of units consumed is rising. The bill will reflect both trends.

Once you treat tokens as a commodity, three patterns become clear.

First, unit price versus total consumption. In oil, the price per barrel can fall while total oil spend rises because the number of barrels consumed goes up faster than the price comes down. The same logic applies to tokens. The price per million tokens is falling fast, yet the number of million‑token units consumed is exploding as agents take over more work and embedded AI features spread through products and internal systems. A cheaper barrel does not guarantee a smaller bill if you burn many more barrels.

Second, commodity price risk. When you buy oil or hedge foreign exchange, you do not assume a flat rate all year. You manage volatility with multi‑supplier sourcing, hedged contracts, scenario planning, and reserves. AI tokens behave much more like a commodity input than like a fixed SaaS license. The same playbook belongs here. Diversify providers so no single pricing schedule dominates your exposure. Cap usage so barrels burned cannot grow without limit. Model best, base, and worst cases so token price and consumption shocks appear in your scenarios before they appear in your actuals. Maintain an AI cost buffer that acknowledges tokens as variable, not constant.

The third idea is where tokenomics goes beyond traditional commodities. A barrel of oil is a barrel of oil. It is physically identical across uses. A million‑token barrel is not. The same million tokens can be burned wastefully, on bloated prompts and verbose answers, or productively, on critical decisions and high‑leverage checks. Two teams can consume identical barrels of intelligence and produce very different economic outcomes.

That is why the unit that really belongs on the board’s dashboard is barrels of intelligence per accepted outcome. How many million‑token units does it take to resolve a case, land a customer, test and ship a code change, or clear an audit exception. That measure turns AI from a raw commodity input into a governed driver of margin. It lets finance send more barrels toward workflows that convert intelligence into value and fewer toward workflows that simply burn tokens in the background.

The Ferrari problem

Not every AI task deserves the same engine. Some are quick errands. Others are long, complex trips. The way token usage scales across those tasks explains why costs diverge so fast.

In practice, token usage falls into a few recognizable bands. A simple factual question or classification typically runs roughly two hundred to two thousand tokens total, the equivalent of a short ride across town. A multi‑page document summarization runs ten thousand to over one hundred thousand tokens, a longer drive that still stays within city limits. A full agentic workflow that plans, retrieves, calls tools, checks its own work, and loops back when something fails can run fifteen thousand to over a million tokens for a single task. At that scale, you are spending enough tokens to turn a quick errand into the equivalent of chartering a private jet for a very small delivery. Those outsized trips now happen invisibly dozens of times a day across every team with access to an agent.

At the top of this spectrum sit the frontier models, the AI equivalent of a Ferrari. Anthropic’s latest high‑end systems, such as Fable‑class models and new Opus tiers, and OpenAI’s most capable ChatGPT‑style models, with their very large context windows and extended reasoning modes, are built for deep, complex work. They combine enormous context capacity, long internal thought chains, tool access across the enterprise, and premium barrels priced in the high double digits per million output tokens. Smaller models, including mid‑range proprietary engines and well‑tuned open‑weight models, often run at a dollar or two per million tokens or less when hosted efficiently.

Yet the tasks most enterprises send these Ferraris on are convenience‑store trips. Summarize a five‑page memo. Draft a follow‑up email. Explain a four percent revenue dip in one region. In many organizations, the default agent configuration points even these trivial workflows at the most powerful, most expensive model available. The agent loads far more context than the task requires and narrates its work at premium output rates the whole way home. You would not take a Ferrari to pick up milk two blocks away, but in AI, that is often exactly what the system quietly does.

This is not just a technical misconfiguration. It is an economic and governance challenge. Without clear routing rules, employees will naturally reach for the most capable model in the catalog, even when a small model or a well‑tuned open‑source engine would handle the task at a fraction of the cost. Over time, that pattern locks the enterprise into a single frontier stack and starves investment in lighter models that could anchor a more sustainable architecture.

It also forces a fresh look at open‑weight models and internal ontologies. When you control your own model configuration, retrieval stack, and business ontology, you can route low‑stakes, high‑volume tasks to inexpensive engines that speak your data language well, and reserve frontier capacity for genuinely hard problems. Open‑weight models and controlled ontologies make it easier to standardize how products, customers, regions, and workflows are represented across AI systems. That standardization, in turn, makes it easier to swap in cheaper engines without losing semantic fidelity. Frontier models become a scarce resource used where their incremental reasoning depth or multimodal capability truly pays for itself.

One caveat cuts the other way. The cheapest model is not always the cheapest outcome. A model too weak for the task fails, retries, and creates rework that costs more than the premium barrel would have. The discipline is matching the engine to the trip in both directions. Do not send a Ferrari on every convenience‑store run, but also do not send a bicycle on a cross‑country drive. The organizations that learn to make that match deliberately, across both proprietary frontier models and open‑weight engines, will be the ones that turn token spend into governed margin rather than unplanned operating expense.

Jevons, quantified

The economics of tokens become urgent once you follow the curve all the way through. Everyone can repeat the headline that tokens are getting cheaper. The deeper logic comes from an idea economists now call the Jevons Paradox.

The Jevons Paradox is an economic principle that says when technology makes the use of a resource more efficient, total consumption of that resource tends to increase rather than decrease. Greater efficiency lowers the effective cost of the resource, which induces higher demand and encourages new uses. William Stanley Jevons observed this with coal in the nineteenth century. More efficient steam engines reduced the cost of using coal, and instead of shrinking Britain’s coal consumption, they multiplied the factories and railways that burned it.

The same pattern is now unfolding with tokens. More capable models process more work per token. Infrastructure providers cut unit prices per million tokens and add caching, batching, and long‑context features that make tokens more efficient to use. Those efficiency gains lower the effective cost of intelligence and invite more tasks, more agents, and more always‑on workflows into the system. As a result, total token consumption rises faster than unit prices fall.

Goldman Sachs Research now puts numbers on this curve. Their forecasts suggest total token consumption could rise roughly twenty‑four‑fold between 2026 and 2030, reaching on the order of one hundred twenty quadrillion tokens processed per month, driven first by consumer adoption of always‑on background agents and, with a lag, by enterprise adoption of agentic workflows. Daily queries to large language models are expected to grow at compound rates near forty percent per year, into the billions of interactions annually. A single agentic request takes what would have been a basic chatbot exchange and expands it tenfold, twentyfold, fiftyfold, because the task becomes a sequence of steps rather than a single response. Jevons, in other words, is not a historical curiosity. It is an operational forecast.

Falling unit prices and exploding total consumption are two views of the same Jevons curve. That is why hyperscalers and AI infrastructure providers celebrate the trend while their customers’ finance teams brace for it. As barrels of intelligence burned rise, gross margins expand for the providers. At the same time, cost volatility and exposure expand for the buyers.

Recent examples show how quickly this can get out of hand. One enterprise running advanced AI models reportedly accumulated an AI bill on the order of five hundred million dollars in a single month, with no spending caps, no usage limits, and no monitoring system in place. Uber has already acknowledged that its engineering teams burned through the company’s entire planned AI budget for 2026 within the first few months of the year after encouraging aggressive use of coding agents and tracking token consumption on internal leaderboards. In that culture, tokens became a badge of honor. Engineers were effectively rewarded for burning more tokens regardless of whether the work mattered. Jevons played out in real time: efficiency and access made tokens cheaper to use, and total consumption exploded.

Two caveats deserve a place in every multi‑year AI budget. The first is supply. A new chip plant takes roughly three years to build, while the shift from simple chat interactions to agentic workflows happened in months. A period of chip constraint over the next twelve to eighteen months is therefore likely before capacity catches up. Plans built on endlessly available, endlessly cheaper compute may not survive that window intact. The second is that the economics are not uniform across use cases. Coding agents and text‑based support often clear the cost bar against human labor even as prices move. At the same time, some real‑time voice agents still have per‑interaction costs that exceed those of a human doing the same job, because latency and time dependency keep them active and burning tokens for longer. Not every trip is worth automating yet, no matter how attractive the barrel price looks on the label.

Ghost payroll and the cognition meter

The traditional way to budget technology spend is headcount‑centric. Licenses per seat, tools per user, spend per department. That logic does not survive contact with an agent fleet. Agents do not clock in and out. They watch systems around the clock, read and rewrite code, converse with customers in parallel, and chain tasks into long self‑directed workflows, each action burning tokens and each carrying its own hidden reasoning layer, whether or not a human ever typed a prompt.

Call it ghost payroll. It is a workforce of digital coworkers that never appears on an org chart or a compensation review, yet consumes real dollars every hour it runs. If your planning assumes token usage scales with the number of humans asking questions, your five‑year forecast will be wrong by an order of magnitude, the way a 1990s IT budget built around desktop PCs missed the cloud.

There is a way to put a number on this exposure. Think of it as cognitive capacity under management. Roughly, it is model capability multiplied by the number of deployed instances multiplied by how long each can act on its own before a human checks in. That product is the total amount of cognition running under the enterprise’s name, and, on current trends, it is doubling on a timescale measured in months. It changes materially between any two board meetings.

Headcount has a custodian and a century of institutional practice, a chief human resources officer, a compensation committee, workforce plans. Cognitive capacity under management has none of that yet. It compounds in the background alongside the token bill, because the two are simply two views of the same growth curve, one counted in dollars and one counted in autonomous capability.

A board that tracks only the invoice is reading half the instrument panel.

The circuit breaker principle

The kilowatt‑hour metaphor has one more step, and that step matters for tokens.

Nobody governs electricity annually. The building you are sitting in has circuit breakers that trip automatically when a line draws too much load. The meter is read continuously. Thermostats route demand without anyone convening a meeting. An alarm wakes somebody when consumption spikes at three in the morning. That is continuous governance of a continuously consumed resource, and it is so ordinary that nobody calls it governance at all. It is simply how you wire a building.

Over the past two years, enterprises have wired in a second utility. Intelligence now flows into every function, metered by the token, billed by the second, consumed around the clock by agents that never sleep. In most companies, this new utility was connected with no breakers, no live meter, no routing logic, and no alarm. Annual AI policy reviews, once‑a‑year risk assessments, and quarterly usage summaries are the equivalent of checking the fuse box every December and hoping everything has held. A meter that runs continuously cannot be governed episodically.

Continuous consumption demands continuous governance. In practice, that means spending caps that trip automatically when token usage crosses a threshold. It means usage dashboards that are read in real time, not just at quarter‑end. It means routing rules that match the engine to the trip so trivial tasks do not silently default to frontier models. It means escalation thresholds that wake an owner when a workflow’s burn rate spikes so someone is accountable for shutting it off or redesigning it.

Modern cost‑intelligence tooling shows what these controls look like in detail. Each AI call is enriched with business metadata such as feature, team, customer, environment, model, prompt version, and workflow type so the token bill maps directly to the customers, products, and deployments that created it. Drift becomes visible. Prompts expand. Context windows grow. Outputs lengthen. Retrieval injects more data than before. The system flags these changes instead of leaving them to be discovered at invoice time.

The same tooling catches anomalies before they become incidents. Agent loops that run far longer than intended. Retry storms that hammer the same endpoint repeatedly. Context blow‑ups that push tasks into very large token footprints. These are the AI equivalents of a line drawing too much current. In a well‑governed environment, they trigger alerts, caps, or automatic throttling. In an environment without circuit breakers, they simply run until the bill arrives.

Tokens are now a utility input. Treating them that way means wiring continuous controls around the meter, not relying on annual documents to manage a real‑time resource.

Explaining AI spend

Before installing controls, a finance team needs to be able to explain the number it already has. The tool for that is one every CFO already uses: the rate, volume, and mix bridge. Applied to tokens, it turns “AI spend doubled” into something you can diagnose.

Rate. What did the blended price per million tokens do this period. With list prices falling sixty to seventy percent annually at the infrastructure layer, rate should almost always be favorable. If your blended rate is flat while list prices fall, your mix is drifting toward premium models. That is a mix problem disguised as a rate number.

Volume. How many barrels of intelligence did the enterprise consume, and what drove the growth. Was it more agents deployed, or more consumption per agent. The first is a strategic decision the board signed off on. The second, rising burn per agent at steady state, is the signature of context bloat and unbounded loops, and it is fixable.

Mix. Three splits matter most. Frontier versus mid‑tier versus small models, because a mix shift toward the Ferrari can swamp any rate decline. Output and reasoning versus input, because the expensive side of the meter growing faster than the cheap side means agents are talking and thinking more per unit of work. Cached versus fresh input, because a low cache share is money left on the table every single day.

A quarterly bridge built in those terms converts an uninterpretable variance into three named drivers with three named fixes. Waste shows up as adverse mix and rising per‑agent volume. A workflow becoming business‑critical shows up as favorable rate, disciplined mix, and volume growth tied to outcomes.

Two housekeeping points belong alongside that bridge. First, placement. Agree with your auditors now on where token costs sit: cost of revenue when agents serve customers, operating expense when they serve employees, and whether tokens burned building internal AI tools follow your capitalized software policy. A spend line compounding this fast should not have its accounting decided retroactively. Second, contract structure. Token procurement is developing the same committed versus on‑demand economics as cloud. An enterprise consuming predictably at scale should be pricing committed volume against on‑demand rates the way it already prices reserved instances against spot.

Budget in barrels, measure in outcomes

AI costs will not come under control through reminders or slogans. They respond to a few hard rules that shape how today's models and agents are used.

First, route by task, not by default. Maintain a small set of models, lightweight, mid‑range, and frontier, with routing rules that send simple, frequent tasks to the smallest viable engine and reserve frontier capability for rare, complex, high‑stakes work. Given the output premium, those rules should also discourage verbose, over‑explaining responses when a terse answer will do. This is the thermostat on the AI utility.

Second, enforce context discipline. Feed a task only what it genuinely requires. Summarize long histories rather than replaying them in full, which directly attacks the context replay compounding in agentic workflows. Cap how long an agent can run and how many tool calls it can make before a human reviews its work. Turn on prompt caching for instructions and documents that repeat, since that is where the near‑free savings live. The failure mode to root out is token bloat: pasting entire documents, full threads, and complete logs into every request out of anxiety that the model might miss something. The model works harder to separate signal from noise, and you pay for the noise. These controls are the breakers.

Third, measure cost per outcome, not cost per token. Two workflows can burn identical token counts and produce very different business value. The dangerous cousin of that confusion is token maxing, measuring AI success by usage volume. Any team rewarded for “using AI a lot” will find ways to burn more tokens, longer prompts, bigger files, more back‑and‑forth, regardless of whether outcomes improve. This pattern already appears in organizations that have celebrated engineers for racking up large token bills on leaderboards. It is the equivalent of judging a delivery driver by gasoline burned rather than packages delivered.

For every major AI‑mediated workflow, management should be able to state a cost per accepted outcome: a resolved case, a tested code change that passes review, a customer interaction that meets service standards, alongside the token and model mix behind it. Surveys of finance leaders underline how urgent this is. Only about one in five can cleanly tie AI spend to outcomes today, even though nearly nine in ten say they must do so within the year. Boards are already shifting from blank checks to conditional funding, with many conditioning further AI money on proof of return and some refusing new spend until ROI is demonstrated.

When a frontier agent burns thousands of barrels for a marginal improvement that a mid‑tier model delivers at two cents on the dollar, the issue is not technology. It is capital allocation, and it belongs on the same dashboard as every other capital allocation decision. Tokens are the meter. Outcomes are the unit that matters.

Key questions for your AI vendor

Governing AI spend starts with knowing what you are buying. Before the next contract or renewal, the finance function should get straight answers to four questions. A vendor that cannot answer them clearly has handed you a governance finding.

First, how is token usage measured and billed. Ask for a breakdown that distinguishes input, output, cached, and hidden reasoning charges, and make sure that breakdown maps directly to the invoice you receive each month. If you cannot trace a dollar back to the type of work that generated it, you are flying blind.

Second, what monitoring is included. You need usage, cost, and trend lines by user, team, and department, visible in something close to real time. If you only see consumption when the invoice arrives, you cannot manage it, you can only fund it.

Third, can administrators set limits. Look for spending caps, restricted access to the most expensive models, and required approval before a workflow scales into serious spend. Not every employee needs a Ferrari‑class model. The platform should hand out bicycles by default and grant keys to the sports car only when warranted.

Fourth, what happens at the budget line. Some vendors keep processing and send a larger invoice. Others offer hard caps, real‑time alerts, or automatic throttling when spend crosses an agreed threshold. Knowing which mode you have signed up for, before the bill arrives, is the difference between a governed expense and a surprise.

Answering those questions is only the start. The discipline that follows does not belong to one person. It belongs to the governance system as a whole.

Key owners and operating discipline

Once the right questions are being asked of vendors, the work shifts inside the enterprise. AI spend becomes manageable only when specific owners take responsibility for specific parts of the system.

The CFO owns the economic picture. That means building a tiered token forecast for human‑only, human‑with‑agent, and agent‑first workflows, expressed in ranges rather than point estimates, using real consumption profiles and the tenfold‑to‑fiftyfold agentic multiplier, and stress‑testing those numbers against likely chip‑constraint windows. It also means adding a rate, volume, and mix token bridge to the standard monthly close so AI spend variance is explained with the same rigor as any other major line item.

The audit committee makes sure the rules stick. It embeds the vendor questions in every AI contract review, settles the accounting placement question with the external auditor, and directs internal audit to test AI usage and billing against approved policy the way it tests every other material spend category.

The risk committee owns the agent fleet. It maintains a living inventory of every agent in production or planned, with purpose, autonomy level, data access, and model tier, and reports cognitive capacity under management on a regular cadence, treating a number that doubles in months with the attention any fast‑compounding exposure deserves.

Management installs and operates the breakers. It defines routing rules keyed to task complexity, turns prompt caching on by default for recurring instructions and documents, sets output‑length parameters, and assigns per‑function token budgets with automatic escalation when burn rates spike or growth outpaces the value workflows are meant to produce.

The compensation committee aligns incentives with outcomes. It reviews internal metrics, leaderboards, and performance language for anything that rewards AI usage volume rather than results, and replaces it with cost per accepted outcome. That is where token maxing is born, and that is where it can be retired.

Taken together, these roles turn AI spend from an unmanaged utility into a governed system. The remaining step is for the board and its committees to put that system on the agenda and turn it into explicit expectations.

Ten recommendations for boards, CFOs, and audit committees

First, put token economics on the board agenda now. Request a briefing on current token pricing, major AI workflows, and usage concentration, and ask for an initial three‑ to five‑year token spend forecast under conservative, base, and aggressive adoption scenarios.

Second, ask specifically about hidden reasoning tokens and token bloat. Require management to disclose, for major workflows, how much of total token consumption comes from internal thinking steps that never appear in a visible transcript, and whether prompts are sending models more information than tasks actually require.

Third, build an agent inventory and a headcount‑equivalent forecast. Require a living register of every agent in production or planned, with purpose, autonomy level, data access, and model tier. Ask explicitly how many agents the organization expects to be running in three years and in which functions. Treat that number with the same seriousness as a workforce plan.

Fourth, track cognitive capacity under management alongside token spend. Ask management to estimate, even roughly, the product of model capability, deployed instance count, and task horizon across major AI systems, and to report how that figure is trending quarter over quarter. Any number that is doubling every few months deserves the same attention as other fast‑compounding exposures.

Fifth, install a model council and enforce routing discipline. Direct management to route tasks by complexity and risk rather than convenience, reserving frontier models for genuinely high‑stakes work and defaulting everything else to smaller, cheaper engines, with explicit attention to controlling verbose, high‑output token behavior given its pricing premium.

Sixth, set token budgets and agent caps tied to financial guardrails. Establish per‑function token budgets linked to revenue and margin targets, with automatic escalation when usage crosses a threshold or grows faster than the value it produces. Treat these budgets as part of the enterprise risk framework, not just as IT limits.

Seventh, eliminate token‑maxing incentives. Review internal metrics, leaderboards, and performance reviews for any language or culture that rewards AI usage volume rather than outcomes, and replace it with cost‑per‑outcome measures.

Eighth, mandate cost‑per‑accepted‑outcome reporting for every major AI workflow. Require management to report cost per accepted outcome, not just tokens consumed, following emerging standards from leading model providers and cost‑governance platforms. Use that metric to decide what scales, what gets redesigned, and what gets retired.

Ninth, fold token metrics into standard financial and audit reporting. Add barrels of intelligence consumed, effective price per barrel, input‑output cost ratios, and the top token‑intensive workflows to the regular finance pack, and have internal audit test AI usage and billing against approved policy the same way it tests any other spend category.

Tenth, run Jevons‑aware scenarios before the surprise arrives. Model explicitly what happens when token prices keep falling even as agent count and autonomy rise, and decide in advance how the organization will respond if total AI operating expense begins to outpace the productivity gains it is meant to deliver.

Owning the intelligence meter

So, why is your AI bill going up while AI itself gets cheaper? Because you are not really paying for tokens. You are paying for what those tokens enable: more agents, more autonomy, more invisible reasoning, more trips taken in a Ferrari when a bicycle would have been sufficient, and more raw cognitive capacity quietly compounding under your company’s name with no one assigned to watch the meter. The per‑unit price was never the whole invoice and it never will be. Cheaper intelligence does not lower the bill on its own, and it does not shrink the amount of cognition your enterprise is running either. Only governance does both. The board that treats tokens as a governed commodity, puts a number on the cognition they measure, and forecasts that number alongside the invoice will be the one steering the curve instead of merely funding it.

Alpha Editorial Board

Alpha Editorial Board

All articles
Tags:

More from Alpha Editorial Board

See all