My Anthropic bill beats my dashboard. Why?
Cached tokens, usually. Anthropic reports cache_read_input_tokens and cache_creation_input_tokens separately from input_tokens. Libraries that sum only input and output silently omit cache activity, so every cached request under-reports itself. Found and fixed in Pipecat, LiveKit, mcp-use, Haystack and Llama Stack. The other three recurring causes are a stale pricing table, streaming code that overwrites accumulated usage with the final message_delta, and failed-then-retried calls that are billed but not logged.
Read more→
My eval score reads 0.0 instead of failing. Why?
An empty denominator. When a metric measures nothing, dividing by zero samples is often guarded by returning 0.0 rather than nan. But 0.0 is indistinguishable from a model that genuinely scored zero, so the run looks complete and the number is wrong. Reporting nan makes it raise.
How do I check my own stack in ten minutes?
Send one request with a known cache hit and compare three numbers: what your metering recorded, what the provider's usage object returned, and what the provider's cost report shows for that day. If the three disagree, the direction of the gap names the defect. billproof does this comparison automatically for Claude Code, Codex and Gemini CLI transcripts.
Read more→
What is an LLM cost reconciliation audit?
A fixed-price review of the code that turns model usage into dollars, checked against the provider's own invoice or usage report. The deliverable is a set of failing tests, one per defect, each with the fix. It starts from $5,000 for a codebase; revenue assurance, for a product that bills its own customers per token, is $10,000.
The same metric differs between queries. Why?
Usually non-determinism inside a query reducer. If an expression is re-evaluated at every reference rather than materialised once, rows that should group together don't, and any de-duplication downstream keeps exactly what it exists to remove.
We bill customers by usage. Why does this matter?
Because the same defect that makes your dashboard wrong makes your invoices wrong. If your metering under-counts, you are giving away revenue you already earned. If it over-counts, you have refund liability and a trust problem. And if your cost basis is wrong, per-customer margin is fiction, so every pricing decision rests on a number nobody verified.
We have a data team. Why bring someone in?
Because this defect class never surfaces as an incident. Nothing pages, no test goes red, and the number stays plausible, so it is never anyone's top priority until it shows up as an invoice or a published benchmark that will not reproduce.
Where are you based, and how does this work remotely?
Bengaluru, IST. The engagement is deliverable-based and async-first: your repos, your invoices, a written reconciliation, with a kickoff and a findings call scheduled in the overlap, which is all of the UK/EU working day and US East until midday. Every merged fix on this page shipped from here into US, UK and EU codebases. Invoices in USD.