The Rise of Spot Markets for Inference

While intelligence is becoming commoditized, it is not yet a commodity.

Inference is sold by the token.

Buyers pay whether the work is great or AI slop. Providers lock up capital in GPUs and then chase utilization. Inference marketplaces are still nascent and continue to grow exponentially despite their inefficiency.

This series looks at how that changes. In Part 1, we examine where inference marketplaces stand today, the structural problems that keep them inefficient, the pressures on both supply and demand, and the shift toward outcome-based pricing. Later parts will cover verification, auction design, parallels to ad networks, and the broader economic consequences.

Commoditization, along with healthy and growing supply and demand, naturally leads to marketplaces.

But while intelligence is increasingly commoditized, it’s not yet a classic commodity.

Today, inference is priced based on input/output tokens rather than the value or validity of the generated artifact.

• On the supply side, neoclouds and other inference infrastructure providers have made historical capital commitments to purchase compute at fixed terms. This creates classic duration mismatch and utilization risk. Providers must predict demand, sometimes years in advance, and bid accordingly. Any excess capacity generates no revenue, giving providers an incentive to compete for additional demand.
• On the demand side, we’ve seen enterprises focus less on the cost of a single inference request and more on the total cost of completing a Work Unit.

Inference marketplaces sit between these two economic pressures, and I believe we’ll see outcome-based inference spot markets emerge, potentially using reverse auctions, as intelligence becomes increasingly commoditized.

What Is an Inference Marketplace?

An inference marketplace is a shared commercial environment that brings together buyers seeking intelligence and model providers capable of supplying it.

A router is the middleware that selects among available models and providers and routes each request. Today, that means routing token-based requests. The marketplace defines the available supply and the commercial terms under which providers compete; the router decides where the request goes.

When inference is purchased for immediate or near-term delivery, the marketplace functions as a spot market. Prices can be set by dynamic quotes or auctions.

Routers are a natural entry point for these markets.

In the future, I believe we will see them evolve to route work orders. A work order is the formal job to be done, including:

• formal task definition
• constraints and requirements, such as environments, budgets, work order duration etc
• verifiers, such as test suites or other criteria.

Buyers will seek the best economics on these work orders, and providers will bid on them.

The Current Blind Spots of the Intelligence Market

Today’s intelligence market gives buyers instant access to a growing range of models and providers, yet it remains largely inefficient.

Routers have improved efficiency by aggregating otherwise fragmented supply. Instead of wiring a single supplier into its workflows, an enterprise can connect once and direct requests across eligible models and providers according to the rules it sets.

This simplifies access to intelligence and lowers switching costs in a fast-changing environment. A 2026 Microsoft study estimates that routing can reduce computing needs by up to 25%.

However, most buyers don’t care about tokens. They care about the job to be done. Routers let them compare models and prices, but we still pay for metered consumption rather than a verified result.

And, while tokens are a proxy for utility, it’s not the real thing. At a small scale, this might be fine. But when enterprises are spending millions, they want a verified result. It is like hiring a contractor. You might pay hourly or on a day rate, but what you are actually contracting for is the completed work.

Inference Is Not Fungible

While intelligence is being commoditized and inference is metered in tokens, it is not a classical commodity like copper or rare earths, nor is it fungible.

Not all intelligence is created equal, and similar tasks across different models consume a wide range of tokens, yielding a wide range of outcomes.

Different models have different strengths and shortcomings. This partly explains the wide range of model prices, but ultimately, buyers care about whether the work produced from their token usage is valid. Even within similar models, service providers' offerings can vary significantly in

• latency
• context
• region
• retention
• uptime
• meta harnesses, and more

Access to the same model is therefore not the same product.

This creates a familiar standardization paradox: more precise service categories make offers easier to compare, but they also fragment the broader “same-model” market into smaller markets with fewer competing offers.

The Cheapest Token Is Not Always the Cheapest Work

Even simple price comparisons across models quickly reach their limits.
Models can be compared by price per token, but this metric doesn’t reveal the amount of tokens required to produce an acceptable output.

A more useful denominator is the expected cost of obtaining a verified result, and the Cost-of-Pass framework expresses this for a model m attempting to solve a problem p:

Here Cm(p) is the expected monetary cost of an attempt and Rm(p) is the probability of producing a correct solution.

Mileage May Vary

The discrepancy between token cost and task cost is also well documented.

While everyone reading this has probably experienced this firsthand, a 2026 study showed that in 106 of 336 model-pair/task comparisons (32%), the model with the lower listed rate produced the higher realized API cost, with reversal magnitudes up to 28 times.

Other dynamics, like caching, can significantly alter final costs. If an ongoing AI workload has a warm cache with one provider, keeping the workload there can yield a cheaper outcome (as low as ~90%) even while the underlying model is larger or has a higher advertised rate.

Simply put, buying inference is not as simple as selecting the model with the lowest advertised price.

A Broken Allocation of Risk

This all comes down to one of the root problems: in the current API market, AI billing is predominantly tied to usage. Buyers pay for consumed tokens and bear the risk when these tokens do not produce a required outcome.

This pricing structure tends to work best with experimentation, when uncertain costs and results are easier to tolerate. However, AI has shifted from experimentation into production budgets as adoption continues to accelerate. The cost of indiscriminate token use has also become harder to ignore, as recent reports concerning Uber and Amazon illustrate.

At enterprise scale, knowing the price of tokens for a model is not enough. Organizations, even before saving, need certainty about what an acceptable result for a defined task will cost.

This creates demand for a market that prices verified results rather than the inference required to produce them.

Outcome pricing transfers a bounded portion of execution risk to providers, trading a potentially higher quoted price for a tighter cost distribution and greater certainty.

Capacity (Hopefully) Meets Demand

The buyer’s incentive is clear: greater certainty over the cost of an acceptable result.

Providers have their own reasons to participate.

Neoclouds and other suppliers commit capital to GPUs and data-center capacity through a mix of cash and long-term leases before they know how much demand will appear.

Once the capacity is live, the economics depend heavily on utilization. It needs to stay high (typically 65-85%) for the model to work. Margin is simply the spread between the revenue each GPU generates and the cost of keeping it available.

Long-Term Commitments Trade One Risk for Another

Long-term contracts can secure baseline demand, but they create new problems.

At CoreWeave, the weighted-average term of operating leases was 11 years as of March 2026, nearly two estimated six-year GPU depreciation cycles. In Q1 2026, 98% of revenue came from committed contracts, and 65% of that revenue came from just two customers.

Long-term commitments, therefore, exchange utilization risk for concentration risk. The value of predictable work already shows up in pricing: both OpenAI and Google offer discounts of around 50% when buyers commit to completion within 24 hours.

The commodification of intelligence, via both open-weight models and the increasing number of labs and providers, will add further pressure on suppliers.

So while long-term commitments can help secure baseline demand, the remaining capacity naturally belongs to dynamic spot inference markets.

Outcome-Based Inference Marketplaces

An outcome-based marketplace would take this clearing function further by changing what is being bought. Its commercial unit would be a verified inference result for a bounded task, rather than the tokens used to produce it.

The provider’s commitment would be limited to meeting the acceptance criteria agreed upon for the task. Whether the result created broader business value would remain the buyer’s responsibility.

A provider could still quote below the buyer's expected internal cost by choosing the model, serving configuration, retries, and capacity used to pass the test. In exchange, it would absorb execution variance. Outcome pricing is therefore as much an underwriting business as a serving business.

Over time, this is how I see inference marketplaces evolving.

From Metered Usage to Verified Results

Today, buyers pay for inference regardless of the end outcome.

In an outcome-based market, the buyer would instead define the task and its acceptance criteria; eligible providers on the supply side would quote an all-in price to meet those criteria; and the router would select among the offers according to the buyer’s rules.

This would transfer part of the execution risk to the market.

The buyer would remain responsible for defining the acceptance criteria. The provider would absorb the internal costs of producing an accepted output, including any additional attempts required.

Assuming a perfect verifier, if one attempt costs c, passes with probability q, and earns b only on acceptance, the expected contribution is qb-c. The compute-only price floor is therefore:

A cheaper attempt can still require a higher outcome bid if its probability of passing declines in proportion to its cost.

The Parallel of Ad Networks

Advertising markets offer useful parallels.

Both ad and inference involve two related goods.

ad impressions and tokens for inference.

the result the buyer actually wants: a conversion (ads) or an accepted outcome (inference).

Inference markets are no different: we have tokens, which, like impressions, have little to no value per se for buyers, and we have the final accepted outcome, which is what the buyer actually wants and what buyers generally obtain after burning a specific number of tokens.

Meta’s ad auctions optimized the results for the actual goal: conversions.

Advertisers compete for impressions, but the highest bid does not automatically win.

Meta combines each bid with an estimated action rate and ad quality, then uses reported impressions and costs to improve these estimates. A campaign can therefore be billed by impression while being allocated and managed around cost per conversion.

Outcome-Based Auctions

The other element that can be borrowed from ad networks is the auction system.
However, in an inference marketplace, the direction of competition would reverse (compared to ads), and inference providers would compete to serve a request submitted by the buyer.

In this case, the lowest quote should not automatically win, just as Meta does not automatically select the highest advertiser bid. Other auction-typical issues, such as extreme offers (risk of underestimating the work) or the potential conflict of interest when a marketplace venue also acts as a provider, would need to be addressed.

One possible representation of a buyer-side allocation score is:

Here bi is the payment due on settlement, mi the estimated probability that the verifier triggers payment, and q_i the estimated probability that delivery is truly acceptable. The remaining terms represent latency, the loss from unusable delivery, and switching or cache costs. Under this illustrative design, the marketplace would minimize Si, rather than necessarily choosing the lowest quote.

Verifiers: A Missing Piece

The key to enabling outcome-based inference marketplaces is verifiability.

Verification essentially begins before providers bid, because the defined standards are the bedrock of the economic units behind each task. A buyer defines an "acceptance criteria" and the corresponding test suite, meaning the conditions under which the output can be considered complete. These criteria also make the work priceable from the providers' perspective, as they enable them to estimate the difficulty and quote an all-in price to deliver an acceptable result.

Once the work is delivered, a verifier would apply this context; passing would trigger settlement, while a failed result would be returned to the provider (or handled as per the terms established before execution).

At its core, this is the same logic already deployed in areas like customer support, where abstract outcomes (e.g., a ticket’s successful resolution) trigger payouts to the provider.

These are still vendor-defined proxies, in a much simpler context, rather than independent verification for complex inference scenarios. Regardless, they show how an abstract outcome can be translated into a settlement rule.

Verifiers will improve as agentic capabilities advance and repeated use produces reusable templates, reducing transaction costs and making competing offers easier to compare. In the meantime, marketplace venues must account for verifier errors that can affect both pricing and participation. A false acceptance causes the buyer to pay for unusable work, while a false rejection forces an honest provider to fund acceptable work without payment, potentially raising future bids or reducing participation.

The Future of Inference Marketplaces

My conviction in inference marketplaces grows with every frontier model and provider that enters the market.

Each addition expands the available supply of intelligence and increases the value of the infrastructure used to compare and allocate it. On the demand side, inference is already entering more enterprise and user workflows, even as buyers become more disciplined about the returns on that spending.

Inference marketplaces sit at the center of these forces.

They turn a fragmented collection of models and providers into a functional procurement layer where buyers can purchase intelligence tailored to their requirements, while model providers gain a direct avenue to compete for demand.

The decisive question is whether these markets can define tasks narrowly enough for providers to price execution risk without eliminating competition. If they can, buyers could trade a premium for cost certainty while efficient providers earn a margin for underwriting bounded work.

Whether inference is initially priced per token or eventually by the verified outcome, the marketplace remains the layer that coordinates the transaction. Access to intelligence is becoming commoditized, and the market for pricing and purchasing this new commodity primitive will become a core component of the global economy.