Why is my Anthropic bill higher than my dashboard says?

Most commonly, cached tokens. Anthropic reports cache_read_input_tokens and cache_creation_input_tokens separately from input_tokens. Libraries that sum only input_tokens and output_tokens silently omit cache activity, so every cached request under-reports its own usage and any cost built on it reads low. This exact defect has been found and fixed in Pipecat, LiveKit, mcp-use, Haystack and Llama Stack.

Why does my eval score read 0.0 instead of failing?

An empty denominator. When a metric measures nothing, dividing by zero samples is often guarded by returning 0.0 rather than nan. A score of 0.0 is indistinguishable from a model that genuinely scored zero, so the run looks complete and the number is wrong. Reporting nan makes it raise.

What does an audit cost?

Two fixed-fee tiers. The correctness audit is 5,000 USD and checks whether your reported LLM usage and eval scores are right. Revenue assurance is 10,000 USD and reconciles what you metered against what you invoiced your customers, including under-billing, over-billing and cost basis.

LLM cost & eval correctness audits

Your Claude bill is bigger than your dashboard says.

A 100-token request with 75 cached totalled 175. An aggregator under-reported its own cost by ~7×. A sycophancy metric returned 0.0 on every run, for every model. None of it crashed. That's the point — the value stays plausible and nothing raises.

Book an audit — from $5,000 See the evidence

Defects found & fixed

Every card is a merged pull request in someone else's repository. Each shipped with a test that fails on main. 24 across 13 organisations.

Meta Llama Stack now ogx

The Anthropic-compatible endpoint copied OpenAI's inclusive prompt tokens into Anthropic's exclusive input tokens, and also added the cache read.

A 100-token request with 75 cached totalled 175.
Nous Research

An auxiliary usage shim dropped cache_read and cache_creation tokens before they reached the aggregator.

Mixture-of-agents cost under-reported by ~7×.
UK AI Security Institute 8 merged

A sycophancy metric lost its denominator. A stereotype score was built from samples that were never scored. A harm benchmark scored a model that refused every task identically to one that attempted everything and failed.

Metrics returning 0.0 where they had measured nothing. I then co-wrote the repo's convention for an empty denominator.
dbt Labs MetricFlow

A query reducer with no notion of determinism re-evaluated a UUID at every reference, so every row landed in its own partition and de-duplication kept what it existed to remove.

A conversion rate read 0.5 alone and 0.75 beside its own input metric.
Pydantic Logfire

Missing Bedrock prices for Writer Palmyra; Bedrock calls priced under the wrong provider.

Those calls costed at zero.
OpenAI co-author credit

Every responses.compact call in the Agents SDK was billed but left out of run totals.

I wrote the first fix; an OpenAI engineer's superseding patch shipped with my credit as co-author.

Two ways in

Both fixed scope, fixed fee, no retainer and no discovery-call theatre. The first tells you whether your numbers are right. The second tells you whether you are charging your customers correctly.

Correctness audit $5,000

For teams whose reported LLM spend doesn't match the provider invoice.

Every provider adapter in your path, and whether its usage object is read whole
Cache-token conventions, which differ per provider and are routinely mixed
Streaming paths, where accumulated usage is overwritten by a final delta
Eval scorers: empty denominators, truncated output scored as an answer
OpenTelemetry spans, so usage survives the trip to the dashboard
You get: a reconciliation report, pull requests each carrying a test that fails before the fix, and a CI gate

Revenue assurance $10,000

For products that bill customers by usage — per token, per credit, per call.

Metered consumption against what you actually invoiced, customer by customer
Under-billing, which is revenue you already earned and never charged for
Over-billing, which is refund liability and a trust event waiting to happen
Cost basis against your own provider invoices, so per-customer margin is real
Rating logic: tiers, minimums, overages, credits, free-tier boundaries
You get: a signed reconciliation you can hand a CFO, board or acquirer, the recovery figure in your own numbers, and the CI gate

Revenue assurance is a mature discipline in telecom, invented because metered billing at volume goes silently wrong and costs a fortune. AI products have identical exposure and almost no practice around it. I ran the regulated version: monthly filings across 8 US states, reconstructed from 18 months of history, zero regulatory failures.

The record

24 substantive merged pull requests across 13 organisations, nearly all one defect class. Verifiable on GitHub.

UK AI Security Inst. 8
Snowflake TruLens · 3
EleutherAI lm-eval-harness
Pydantic Logfire
dbt Labs MetricFlow
Mistral AI mistral-common
LiveKit agents
Pipecat Daily
deepset Haystack
Roboflow 2
mcp-use agent
ogx ex-Llama Stack
respan vertexai

Day to day I build and operate data pipelines on BigQuery at Lumen, where query optimisation cut bytes scanned per query by 94% — 565 MB to 35 MB — and where I own monthly regulated filings across 8 US states. That is where I got fussy about numbers that are wrong while nothing fails.

Symptoms

My Anthropic bill beats my dashboard

Cached tokens, usually. Anthropic reports cache_read_input_tokens and cache_creation_input_tokens separately from input_tokens. Libraries that sum only input and output silently omit cache activity, so every cached request under-reports itself. Found and fixed in Pipecat, LiveKit, mcp-use, Haystack and Llama Stack. Full write-up →

My eval score reads 0.0 instead of failing

An empty denominator. When a metric measures nothing, dividing by zero samples is often guarded by returning 0.0 rather than nan. But 0.0 is indistinguishable from a model that genuinely scored zero, so the run looks complete and the number is wrong. Reporting nan makes it raise.

The same metric differs between queries

Usually non-determinism inside a query reducer. If an expression is re-evaluated at every reference rather than materialised once, rows that should group together don't, and any de-duplication downstream keeps exactly what it exists to remove.

Where are you based, and how does this work remotely?

Bengaluru, IST. The engagement is deliverable-based and async-first — your repos, your invoices, a written reconciliation — with a kickoff and a findings call scheduled in the overlap: all of the UK/EU working day, US East until midday. Every merged fix on this page shipped from here into US, UK and EU codebases. Invoices in USD.

We bill customers by usage. Why does this matter?

Because the same defect that makes your dashboard wrong makes your invoices wrong. If your metering under-counts, you are giving away revenue you already earned. If it over-counts, you have refund liability and a trust problem. And if your cost basis is wrong, per-customer margin is fiction — so every pricing decision rests on a number nobody verified.

We have a data team. Why bring someone in?

Because this defect class never surfaces as an incident. Nothing pages, no test goes red, and the number stays plausible — so it is never anyone's top priority until it shows up as an invoice or a published benchmark that will not reproduce.

Defect notes

Each one is a bug I have found and fixed in production libraries, written up in full.

One of them is lying.

If your reported spend and your provider invoice disagree, one of them is wrong. Tell me which providers and libraries sit in your path and I'll tell you where to look first — before you engage me.

arthi1805@gmail.com