Cost Observability for AI Agents: Per-User, Per-Tenant Billing

Every agent product hits this moment. The model bill arrives. It’s bigger than expected. The finance team asks: which customer drove it? Which feature? Which background process?

If your answer is “let me dig through the logs for a few days,” you didn’t build observability — you built hope.

This post is about instrumenting an agent system so cost attribution is structural. The answer to “which tenant ran up the bill” is one query.

What you actually need

Per-LLM-call you need:

• tenant_id
• user_id
• session_id
• run_id
• tool_call_id (if applicable)
• model_id
• cost_category: foreground | background | eval | internal
• input_tokens, output_tokens, cost_usd
• cache_hit: boolean
• timestamp

If you have those for every call, every cost question is answerable with SQL.

The cost meter primitive

ONTO ships this as a first-class primitive:

# At the end of a billing cycle.
cost = onto.cost(
    tenant_id="acme",
    start=datetime(2026, 4, 1),
    end=datetime(2026, 5, 1),
)

# {
#   "foreground": {
#     "tokens": {"input": 12_300_000, "output": 4_500_000},
#     "dollars": 124.30,
#     "calls": 4_280,
#   },
#   "background": {
#     "tokens": {"input": 89_000_000, "output": 6_700_000},   # cheaper model
#     "dollars": 14.20,
#     "calls": 22_100,
#   },
#   "by_model": {
#     "gpt-oss:20b": 80.10,
#     "claude-sonnet-4.5": 58.40,
#   },
#   "by_tool": {
#     "search": 22.40,
#     "calendar_create": 4.10,
#     ...
#   },
# }

These numbers are computed from the run database, not reconstructed from log scrapes.

The foreground / background split

The split is more important than it sounds. A typical agent has two cost categories:

• Foreground: turns the user actually waited for. The cost the user “experienced.”
• Background: summarization, conflict detection, consolidation. Work that improves future turns but isn’t directly tied to a user request.

These usually use different models. Foreground might run on Claude or gpt-oss:20b; background runs on a cheaper open-weight model. The split matters because:

• Billing. If you bill the customer, you bill foreground, not background.
• Optimization. Background can be made arbitrarily cheap by routing to smaller models; foreground latency / quality trade-offs are different.
• Capacity planning. Background can run at off-peak; foreground can’t.

ONTO sets cost_category automatically based on which code path made the call. You don’t have to instrument it.

The cache problem

Prompt caching changes the picture. Anthropic’s prompt cache, OpenAI’s prefix cache, and similar features mean some LLM calls don’t bill at the full rate. You should track:

• Billed cost: what the provider charged you.
• Effective cost: what you’d pay without cache hits.

The ratio (billed / effective) is your cache hit rate, financially. Track it; optimize it.

For your customer billing, use billed cost (you’re not going to charge them for tokens you didn’t pay for). For capacity planning and what-if analysis, use effective cost.

The per-tool view

Tools differ wildly in cost. A simple web search might be 200 tokens; a long-context document summary might be 30,000. Per-tool cost surfaces hot spots:

SELECT tool_name, sum(cost_usd), avg(cost_usd), count(*)
FROM tool_calls
WHERE tenant_id = 'acme' AND timestamp > now() - interval '30 days'
GROUP BY tool_name
ORDER BY sum(cost_usd) DESC;

Often the surprise is a tool that’s called a lot, not the one with high per-call cost. “Search costs $20/month” beats “summarize costs $80/month” if search is called 100x more often.

The per-user distribution

For a SaaS, you don’t want average-user cost; you want the distribution.

SELECT 
    percentile_cont(0.50) WITHIN GROUP (ORDER BY user_cost) as p50,
    percentile_cont(0.95) WITHIN GROUP (ORDER BY user_cost) as p95,
    percentile_cont(0.99) WITHIN GROUP (ORDER BY user_cost) as p99,
    max(user_cost) as max
FROM (
    SELECT user_id, sum(cost_usd) as user_cost
    FROM tool_calls
    WHERE tenant_id = 'acme' AND timestamp > now() - interval '30 days'
    GROUP BY user_id
) u;

The p99 / p50 ratio tells you whether you have a “long tail” cost problem. Usually you do — a small number of power users drive disproportionate cost. The question is whether they’re high-value users (fine) or runaway-loop bugs (alert).

The unit economics question

Putting it together: your unit economics on a per-customer basis depend on whether you can match per-customer LLM cost against the per-customer revenue.

Customer revenue (monthly):       $200
Customer LLM cost (monthly):       $35  ← from onto.cost(tenant_id=...)
Customer infrastructure share:     $10
Customer support load (estimated): $15
Customer gross margin:             $140 / $200 = 70%

If you can pull the LLM cost number with one query, you can run this analysis monthly. If you can’t, you’re flying blind on whether each customer is profitable.

The runaway-loop alarm

A specific failure mode: an agent gets stuck in a loop and burns thousands of tokens before anyone notices. Detection has to be in the cost path:

# In the runtime
if session_cost_so_far > session_budget * 0.8:
    log.warn("approaching session budget", tenant=t, user=u, session=s)
if session_cost_so_far > session_budget:
    raise BudgetExceeded(...)

ONTO’s runtime supports per-session cost budgets natively. Set them low enough that the alarm fires before the bill blows up.

The integration with OpenTelemetry

For broader observability, ONTO exports cost as OpenTelemetry attributes on each span:

span.llm.call
  tenant_id: acme
  user_id: alice
  cost_category: foreground
  model_id: gpt-oss:20b
  input_tokens: 1200
  output_tokens: 400
  cost_usd: 0.012
  cache_hit: false

Drop these into Grafana or Honeycomb. Build a per-tenant cost dashboard. Alert on anomalies. The cost meter and the trace are the same data shaped two ways.

What to do if you’re not on ONTO

If you’re using a different SDK, the principles are the same:

1. Wrap your LLM client. Every call goes through a wrapper that tags it with tenant_id, user_id, cost_category.
2. Persist the calls to a metering table. Cheap; row per call.
3. Build the queries you need. Per-tenant, per-user, per-tool, per-category.
4. Set up budgets and alarms. Per-session, per-day, per-tenant.
5. Wire the cost into OpenTelemetry. Make the data show up where your existing observability lives.

None of this is glamorous. All of it is cheap to do upfront and expensive to retrofit.

A summary

Per-tenant, per-user, per-tool LLM cost attribution is the table-stakes observability story for any agent product running in production. Either you instrument it upfront and your CFO is happy, or you instrument it after the first surprising bill and your CFO is unhappy.

The data model isn’t complicated. The discipline is making sure every call carries the right tags and lands in a queryable store. Pick an SDK that ships this, or build it yourself — but build it.

---

The features page covers ONTO’s built-in cost meter. The multi-tenant memory article covers the related tenant-isolation work.

Frequently asked questions

Don't most model providers give per-API-key cost breakdowns?

They do at the provider level — useful for billing reconciliation, useless for product analytics. You need per-user, per-tenant, per-workflow, per-tool granularity. That has to come from your application layer.

What about caching? Doesn't that complicate per-user attribution?

Yes — if a prompt cache hit serves the response, the model provider charges you nothing for it but the user still got value. Track cache hits separately. ONTO's cost meter exposes both billed cost and effective cost.

How do you handle background work that wasn't user-triggered?

Track it as a separate category. Background memory consolidation, conflict detection, eval runs — these belong in their own bucket. Per-user foreground cost is what you bill; background cost is overhead.