Inference Cost
Inference cost is the expense of running a model to answer a request, driven by model size, the length of the input and output, and the hardware it runs on.
Why It Matters
It is the cost of goods sold for an AI product. Falling inference cost opens markets; rising usage at a fixed price closes margins.
Related Terms
- InferenceInference is the act of running a trained model to produce an output: the answer, image or prediction a user actually receives, as distinct from the training that created the model.
- Token EconomicsToken economics is the pricing of AI by the amount of text processed, where a token is a fragment of a word and both the question and the answer are billed.
- Model DistillationDistillation trains a smaller model to reproduce the behaviour of a larger one, keeping much of the capability at a fraction of the cost to run.
- AI Unit EconomicsAI unit economics is the profit or loss on a single customer or a single use of an AI product, once compute and support costs are counted against the price charged.
VANTAGE52 Coverage
Published reporting filed under the topics behind this concept.
ChatGPT Wallet Lets OpenAI’s Agents Pay, but Retailers Still Decide Who Gets In
OpenAI’s new wallet lets its dots agents buy for some US Pro subscribers using card network tokens and a biometric check, but the agents still need retailers’ permission, and the largest retailer to bar them is Amazon.
Jensen Huang Resigns and Nvidia Turns to Its CFO and a First-Ever Chair
Colette Kress will run Nvidia while remaining its finance chief, and Harvey C. Jones has become the board’s first chair, leaving a board-led search to replace a founder who had about 60 direct reports.
Nebius and CoreWeave Turn to Prepayments and Secured Loans to Fund AI Data Centres
Nebius expects prepayments and asset-backed loans to cover up to 85% of a 2027 spending plan of as much as $45 billion, while CoreWeave, with about $47 billion of debt, is reported to be arranging a $4 billion investment-grade loan.
OpenAI Hits a $1.25 Trillion Valuation and Still Falls Short of What It Wanted
The company has taken in about $11 billion of a round once pitched at up to $30 billion, a sign that even the biggest names in artificial intelligence now face more cautious investors.
Legora Raises $550 Million as the Race to Lead Legal AI Intensifies
The Stockholm-based company is now valued at $8.8 billion after its annual recurring revenue passed $200 million, with the second $100 million added in under six months.
Marvell Agrees to Buy Lumentum in a $115 Billion Bet on Optical AI Networks
The all-stock deal joins Marvell’s networking chips with Lumentum’s lasers and optical components, but both companies’ shares fell sharply as investors weighed the price and a closing date that stretches into 2028.
Continue Exploring
Follow VANTAGE52 for continuing coverage of technology and markets.







