setloop.io
White Paper · SET-WP-2026-02
All papers
setloop.io
White Paper
Setloop Technical White Paper Series

Making AI Costs Accountable with FinOps

An operating model that links AI usage to delivery cost, product features and business ownership through workload attribution, provider reconciliation and policy controls.

Document
SET-WP-2026-02
Published
September 2026
Version
1.0
Category
Financial Operations
Classification
Public
Working infrastructure, not slide decks. setloop.io

In brief

The challenge. AI spending is often difficult to connect with the product or customer that incurred it. Variable context lengths, token usage, agent loops and mixed infrastructure make the unit cost less obvious than a conventional cloud instance.

The response. Finance and engineering need a shared view of usage, provider charges and delivery context. That requires request-level attribution, reconciliation across providers, and agreed rules for allocation and control.

What this paper covers. This paper sets out an AI FinOps operating model built around four capabilities, a three-layer reference architecture and a phased implementation plan.

Per request
Usage and delivery cost attributed across models, services and products
4 pillars
Token-level visibility, provider reconciliation, work-item attribution, and customer-value signals
3 phases
From instrumentation and allocation to policy-based cost control
01

The financial realities of AI infrastructure

Moving AI from research into production changes how costs are generated, allocated and governed. Provider invoices alone rarely show which workload produced the charge or what business outcome it supported.

Flexera's 2025 State of the Cloud Report found that 84% of organisations ranked managing cloud spend as their leading cloud challenge. IBM's 2025 study of 2,000 chief executives reported that only one in four AI initiatives had delivered the expected return on investment. The studies cover different questions and populations, but both point to the need for clearer cost and value measurement.

27%
Estimated share of public-cloud IaaS and PaaS spending reported as wasted by surveyed organisations (Source: Flexera, 2025 State of the Cloud Report)
31% → 63%
Growth in one year in the share of FinOps practices managing AI spending (Source: FinOps Foundation, State of FinOps 2025)
280x
Decline in the cost of GPT-3.5-class inference between November 2022 and October 2024, from $20.00 to $0.07 per million tokens (Source: Stanford HAI, AI Index Report 2025)

Opaque provider invoicing

Provider invoices may aggregate compute and API usage into line items that do not map cleanly to a product, feature, tenant or customer.

Decoupled value attribution

Without consistent tags and allocation rules, organisations cannot reliably relate model costs to features, tenant usage or revenue.

Runaway inference overhead

Long prompts, unrestricted generation and recursive agent loops can raise spending without a corresponding increase in user value.

02

Architectural pillars of enterprise AI FinOps

The proposed operating model has four capabilities. It starts with usage telemetry, then reconciles that usage with provider costs, attributes it to a work item and relates the result to a business measure.

Table 1. The four pillars of AI FinOps
Operational pillarTechnical mechanismStrategic business impact
Token-Level Visibility Granular telemetry capturing prompt and completion tokens per request. Comparable usage and unit-cost measures across services and model families.
Provider Reconciliation Aggregates and normalises public-cloud, API and private-GPU costs. Auditable reconciliation between cloud invoices and actual compute consumption.
Work-Item Attribution Tags inference requests with product, feature, tenant and organisational ownership. Supports margin analysis and product-level profit-and-loss reporting.
Customer-Value Signals Relates model spending to selected product, engagement or revenue measures. Provides evidence for investment, pricing and optimisation decisions.
03

The AI FinOps reference architecture

The reference architecture separates metering, cost calculation, and governance. This makes the origin of each figure visible and allows allocation and policy rules to change without replacing the collection layer.

Figure 1. The AI FinOps reference architecture: three coordinated operational layers
L3Governance
Governance, Audit & Policy Enforcement
Policy-based cost guardrailsAnomalous token usage flagsAgent loop throttlingERP integration
▾
L2Cost engine
Normalisation & Cost Engine
Dynamic provider cost modellingSpot pricing normalisationPer-token delivery costsEgress & utilisation context
▾
L1Metering
Ingestion & Runtime Metering Layer
Inline token telemetryPrompt & completion meteringTenant & workspace taggingCost-center enrichment

3.1Ingestion & runtime metering layer

Inline token telemetry

Metering proxies capture prompt and completion tokens, model identifiers and execution latency for each request. Their own latency and resource cost should also be measured.

Contextual metadata enrichment

Each supported request is tagged with the relevant tenant, workspace, product feature and business cost centre.

3.2Normalisation & cost engine

Dynamic provider cost modeling

The cost engine combines private-hardware depreciation, spot pricing, reserved capacity and provider rate structures to estimate delivery cost at the required unit of analysis.

Delivery cost contextualization

Network egress, vector-database queries, accelerator usage and supporting services are allocated to each work item under documented rules.

3.3Governance, audit & policy enforcement

Policy-based cost guardrails

Configurable policies flag unusual token use, throttle recursive agent loops and enforce agreed compute quotas. Alerts and limits reduce exposure but cannot guarantee that an overrun will never occur.

Auditable financial ledgers

Integration with enterprise resource planning (ERP) systems supports audit, allocation, showback and chargeback processes.

04

Implementation roadmap

Implementation proceeds in three phases. Establish reliable measurement first, validate allocation and reconciliation second, then introduce policy-based automation.

Phase 1

Instrumentation & Baseline

  1. Gateway ingestion
  2. Token-level telemetry
  3. Baseline cost allocation
Phase 2

Reconciliation & Attribution

  1. Multi-provider integration
  2. Feature-level mapping
  3. Unit economics validation
Phase 3

Policy-Based Governance

  1. Policy guardrail enforcement
  2. ERP integration
  3. Continuous cost optimisation
  1. Telemetry deployment. Instrument API gateways and model-serving endpoints to record usage, latency and model identity at the required level of detail.
  2. Attribution mapping. Align execution records with the organisation's product taxonomy, ownership model and customer billing tiers.
  3. Policy automation. Reconcile provider invoices, flag anomalies and apply approved thresholds while retaining human ownership of allocation and investment decisions.
05

How Setloop can help

Cost accountability across distributed AI workloads requires engineering, financial data design and agreed allocation rules. Setloop works with finance, platform and product teams to implement that operating model in the customer's environment.

Bespoke engineering engagements

Setloop designs and integrates cost-attribution services within customer inference pipelines and reporting systems.

Working infrastructure over slide decks

Setloop can take recommendations through implementation, validation and operation in private or hybrid-cloud environments.

Infrastructure and cost optimisation

Setloop combines AI FinOps with GPU architecture reviews so that cost measures can inform capacity, performance and workload-placement decisions.

Key takeaways

What infrastructure and finance leaders should remember

  1. Provider invoices are not enough. Usage must be mapped to workloads, products and accountable owners before meaningful unit economics can be calculated.
  2. Tokens are one part of delivery cost. Measure token use alongside infrastructure, retrieval, networking and supporting services.
  3. Attribution needs stable rules. Consistent request tags and allocation methods make product and tenant costs comparable over time.
  4. Automate within policy. Use alerts, quotas and throttling for known conditions, while keeping ownership of exceptions and investment decisions explicit.
07

References

  1. Flexera. “2025 State of the Cloud Report.” March 2025.
  2. FinOps Foundation. “State of FinOps 2025.” Fifth annual survey, 2025.
  3. Stanford HAI. “AI Index Report 2025.” Stanford University, April 2025.
  4. IBM Institute for Business Value. “2025 CEO Study.” 2025.
  5. IDC. “Worldwide AI and Generative AI Spending Guide.” August 2024.

About Setloop

Setloop is an engineering consultancy and product studio for organisations building GPU workloads, cloud GPU platforms, private AI infrastructure and AI factory architectures. Its engineers design, benchmark and deploy operational infrastructure in customers' private and hybrid-cloud environments across the UK and EU.

The Setloop Technical White Paper Series distils reference architectures from client work and the Setloop product portfolio, including LLMTrace, AutoOps, GPU Cloud Platform, AI FinOps and Automatic RL Research.

For more information