The price of an individual model call may be easy to estimate. The cost of completing a business task is not. Agents may plan, retrieve, call several tools, retry failed steps and invoke other agents before producing an outcome. A cheaper model can cost more overall if it creates longer trajectories or more human rework.
AI FinOps joins financial accountability with engineering evidence. It attributes usage, sets budgets and optimises the complete system across models, context, retrieval, orchestration and infrastructure. Quality and risk remain constraints: the aim is efficient intelligence, not indiscriminate cost reduction.
This course gives participants a practical method for understanding demand, applying cost controls and making model choices based on measurable outcomes.
Learning Outcomes
Upon completion of this course, participants will be able to:
- Build a complete cost model for AI applications and agents
- Attribute usage to teams, products, users and business outcomes
- Select models according to quality, latency, risk and cost
- Implement routing, fallback and escalation policies
- Reduce unnecessary context, retrieval and repeated computation
- Establish budgets, quotas and anomaly detection
- Evaluate optimisation through cost per successful outcome
- Create governance and reporting for enterprise AI expenditure
Course Outline
Understanding the AI Cost Stack
- Input, output, cached and reasoning-token charges
- Embeddings, retrieval, storage and data-processing costs
- Agent loops, retries, tool calls and delegated tasks
- Hosted APIs, managed platforms and self-hosted inference
- Development, evaluation and production cost centres
- Hidden costs from review, rework and operational support
Usage Attribution and Measurement
- Tagging requests by team, application, environment and use case
- Tracking sessions, trajectories and complete business tasks
- Joining token usage with latency, quality and outcome data
- Shared-platform allocation and unit economics
- Cost per request, successful task and active user
- Protecting prompt content while collecting useful metadata
Model Selection and Routing
- Matching model capability to task difficulty
- Static rules, classifiers and confidence-based escalation
- Small-model first with controlled fallback
- Routing by data residency, risk and latency requirements
- Multi-provider resilience and commercial considerations
- Evaluating routers against an approved quality baseline
Context and Retrieval Efficiency
- Removing duplicated and irrelevant context
- Retrieval before long-context inclusion
- Chunk selection, re-ranking and evidence limits
- Summarisation and compaction for long-running sessions
- Prompt and prefix caching
- Measuring when context reduction harms task success
Application and Agent Optimisation
- Limiting loops, retries and unnecessary planning steps
- Reusing deterministic intermediate results
- Batching and asynchronous processing
- Selecting tools that return concise, structured observations
- Separating model work from ordinary computation
- Designing graceful degradation under budget pressure
Budgets, Quotas and Policy Controls
- Organisational, team, application and user budgets
- Soft warnings, hard limits and approval for exceptions
- Concurrency and rate controls
- Cost limits per session or agent task
- Detecting runaway agents and abnormal trajectories
- Emergency suspension without affecting unrelated services
Self-Hosted and Open Models
- Total cost of ownership beyond accelerator hours
- Capacity, utilisation and queuing considerations
- Serving, scaling and model lifecycle operations
- Security, data-residency and customisation benefits
- Comparing internal cost with managed-service alternatives
- Avoiding comparisons that ignore quality and support burden
Forecasting and Procurement
- Demand drivers and scenario-based forecasting
- Pilot evidence versus production assumptions
- Contract, commitment and volume-discount trade-offs
- Capacity planning for peak and batch workloads
- Accounting for model-price and capability changes
- Designing portability without paying for unused abstraction
Governance and Reporting
- Dashboards for engineering, product and finance audiences
- Quality and risk guardrails around optimisation
- Showback, chargeback and shared investment models
- Reviewing high-cost and low-value use cases
- Tracking savings without creating perverse incentives
- Continuous optimisation as models and workloads evolve
Practical Capstone
- Instrument a representative agent workflow
- Calculate cost per successful task and identify major drivers
- Implement model routing and context optimisation
- Add budgets and runaway-execution controls
- Compare the result against quality and latency baselines
- Present an optimisation and governance recommendation
Intended audience
This course is designed for AI engineers, platform and cloud teams, solution architects, FinOps practitioners, technical product owners and engineering managers accountable for production AI expenditure. It is useful for organisations moving from small pilots to shared or high-volume AI services.
Prerequisites
Those attending this course should meet the following:
- Basic understanding of LLM APIs and token-based usage
- Familiarity with cloud cost-management concepts
- Awareness of agent, RAG or AI application architectures
- Ability to interpret operational metrics; programming is helpful but not essential