Every context cost badge links here. This page defines exactly what the number is, how to
reproduce it, and what it is not. Any change to this definition bumps the version and is
recorded in the changelog.
The o200k_base token count of the canonical bytes of a server’s tools/list result.
Precisely:
initialize, then tools/list, following pagination
(nextCursor) to exhaustion. Concatenate the tools arrays in server-returned order.JSON.stringify(tools) over the parsed result value:
no added whitespace, first-occurrence key order, standard JSON.parse semantics (duplicate
keys last-wins, numbers re-serialized per ECMA-404). Defined on the parsed value rather
than raw wire bytes so any JSON implementation reproduces it from the published capture.o200k_base.That’s all. No model calls, no API keys, no sampling — the same input yields the same number on anyone’s machine, in any language with a tiktoken port.
Every published badge has a companion measurement.json containing the raw tools/list
capture (rawToolsCapture), the SHA-256 of the canonical bytes, the exact launch command,
and env var names (values redacted). Re-derive the number in five lines:
import { getEncoding } from "js-tiktoken"; // or tiktoken (py), tiktoken-rs
const m = JSON.parse(fs.readFileSync("measurement.json", "utf8"));
const canonical = JSON.stringify(m.rawToolsCapture);
console.log(getEncoding("o200k_base").encode(canonical).length); // === m.totalTokens
// crypto.createHash("sha256").update(canonical).digest("hex") === m.canonicalSha256
If you dispute a badge, the dispute reduces to a byte-level diff of captures — which is the argument we want to have.
Each measured server also has a detail page carrying that server’s per-tool
breakdown, launch command, isolation, canonical hash, and the one-line verify command —
the same facts, without reading the capture by hand.
measurement.json makes it actionable.tools/list is called twice; if the tool set differs between runs the measurement is
flagged dynamic and the badge reads as measured-at-default-config.isolation field. Required env vars are set to the dummy value dummy.
Commands that are already docker run … are host-spawned containers and recorded as such.
No real tokens or secrets are ever present in the environment.Every candidate server appears in published results with exactly one status:
| status | meaning |
|---|---|
measured |
clean measurement at default config |
dynamic |
tool set varied between runs; value is for the captured run |
auth-required |
won’t start or list tools without real credentials |
startup-failure |
crashed or missing dependencies (stderr tail recorded) |
timeout |
no response within the configured timeout (recorded per measurement) |
remote-auth-wall |
OAuth-gated remote server; listed, not measured |
not-yet-run |
candidate not yet swept (appears in interim leaderboards only) |
| tokens | color |
|---|---|
| < 1,000 | brightgreen |
| 1,000 – 4,999 | green |
| 5,000 – 14,999 | yellow |
| 15,000 – 29,999 | orange |
| ≥ 30,000 | red |
Bands were frozen on 2026-08-16 against the observed distribution of the first full sweep (n=57 measured servers: p25 = 581, median = 2,636, p75 = 5,258, p90 = 10,426, max = 54,422): lean ends at the bottom quartile, light spans the interquartile middle, moderate begins at p75, heavy covers the ~p95 tail, and very heavy marks true outliers. Any future band change bumps this page’s version.
Method tools-delta/v1. The headline number counts every byte a server returns from
tools/list, tokenized with o200k_base. Neither half of that describes what the tools cost
in an Anthropic request, so the gap is measured directly and published beside it.
How it is measured. For each server, project the capture onto the three fields an
Anthropic tool definition carries — name, description, input_schema — and send that
array to POST /v1/messages/count_tokens against a pinned model id, alongside a minimal
one-token user message. Subtract the same request’s count with no tools at all. The
difference is the tokens the server’s tools add to a request. Model id and date are recorded
in results/divergence.json
with the canonicalSha256 of the capture each row was computed from, so a re-sweep marks a
row stale instead of leaving a number that no longer matches its capture.
Why it is published as three numbers, not one. Two independent effects move the count in opposite directions, and a single ratio hides the larger one:
title, annotations, outputSchema, execution, and icons are
real bytes the server ships and the canonical form counts them — but an Anthropic tools
array has nowhere to put them. Across the measured set this removes between 0.7% and
80.6% of the payload (github: 54,422 → 10,535 tokens).Because the effects can cancel or compound, the Claude number is not a fixed multiple of the badge — it ranged from 0.34× to 1.92× across the top 15, and it reorders the leaderboard: github is the heaviest server on o200k and notion is the heaviest on Claude.
What it is not. It is not any client’s context bill either. count_tokens is Anthropic’s
accounting for tools sent through the API’s tools parameter; a client that re-renders
schemas into a prompt, defer-loads them, or forwards MCP metadata will differ. It is a second
documented, reproducible index — measured the same way for every server — not a promise.
Attribution spot check. To confirm effect 2 is mostly the tokenizer rather than the tools
channel, notion’s tool text was also counted as an ordinary user message: prose descriptions
came out 1.58× their o200k count, and the identical minified JSON 1.77×, against 1.92× when
sent as tools. So most of the multiplier is the tokenizer and the remainder is the channel.
That is one server on one date, not a constant — it is recorded here as the evidence for the
causal claim above, not as a conversion factor.
sd2k/mcp-tokens (the CLI our upstream GitHub Action contribution builds on) differs from
this definition in two documented ways: its tiktoken provider selects the encoding from a
--model argument with a cl100k_base fallback (pass --model gpt-4o for o200k), and it
counts a serde_json re-serialization of deserialized tool structs rather than the parsed
wire value (key order normalized to struct order; unmodeled fields dropped). Publishing the
CLI’s number alongside ours as a cross-check column is planned for the launch leaderboard.
Details: spec/upstream-notes.md.
tools/list
value, o200k_base, bands frozen against first-sweep distribution, failure taxonomy,
dual-run dynamic detection.The Claude divergence column (tools-delta/v1, added 2026-08-16) is versioned separately and
deliberately: it adds a second published number without touching the definition above. Every
badge, every totalTokens, and every canonical hash is byte-identical to before it existed,
so bumping this page’s version would have signalled a change that did not happen.