Token savings: OmniRoute gateway, with a headroom baseline

The two tools

• OmniRoute

  diegosouzapw/OmniRoute on GitHub, MIT licence

  An open source LLM gateway: one endpoint in front of many model providers. It can compress prompts with two engines stacked, RTK then Caveman, and it routes requests so that providers’ prompt caches stay warm. We set it up on 8 October 2026; so far only short or test sessions have gone through it.

• headroom

  headroomlabs-ai/headroom on GitHub, Apache 2.0 licence

  Open source context compression: it shrinks tool outputs, logs, files and retrieved text before they reach the model. We have one paired run with it, shown below as the baseline.

OmniRoute since setup

Prompt compression (stacked mode, since recording began)

• Requests compressed

  130

• Tokens saved

  65,446

• Average saving per request

  0.0%

• Requests where compression ran but was skipped

  270

• Skip reason “ran, produced no recordable saving” (no_savings): 270

Prompt cache, last 24 hours

• Model calls

  391

  in the 24 hours before the snapshot; the gateway was set up within this window

• Warm calls

  18.4%

  72 of 391 calls; 18.4% of the 391 calls that used the cache

• Tokens read from the cache

  18,812,183

  reused prompt prefix, which providers bill at a reduced rate

• Tokens written to the cache

  45,256,071

  0.42 tokens read back per token written

A warm call read at least as many tokens from the provider’s prompt cache as it wrote. Of the other calls, 17 wrote to the cache without reading from it, 302 read less than they wrote, and 0 did not use the cache at all. These counts show how often the cache was reused, not what share of each prompt was billed at the cache rate.

headroom baseline: one paired run

The same multi-step, tool-heavy agent task, run twice on the same model on the same day: once without headroom, once with it. This is a single paired run, not a live measurement. The full write-up, with the workload, cache reads and a reliability problem found along the way, is on its own page: context compression, a paired A/B cost trial.

1 paired run (2 runs), 1 workload, 1 model. The target-site audit score was identical both times, but output quality was not independently assessed.

Most of this difference did not come from compressing the input. headroom shrinks what goes into the model, and input tokens fell by only 4.4% (152,704 to 145,985). The run with headroom also took 5 model calls instead of 12 and wrote 10,818 output tokens instead of 66,422. Fewer output tokens make up 89.2% of the drop in billed tokens and, as output is priced higher than input, probably most of the drop in cost. An agent can take a different path through the same task from one run to the next, and with a single pair we cannot tell how much of that difference headroom caused.

Method and caveats

• The OmniRoute figures are aggregate counters read from the gateway’s own read-only analytics: its compression totals and its prompt-cache health for the last 24 hours. No prompts, replies, user details or per-request records are stored or published. The traffic so far is short or test sessions since setup on 8 October 2026, not regular working sessions.
• The headroom figures come from one earlier paired run on one workload and one model. There is no variance estimate. Input tokens barely moved; most of the measured saving came from the run with headroom taking fewer steps and writing fewer output tokens, which one pair cannot attribute to headroom. The percentages will differ on other work.
• This is not a controlled head-to-head yet. The OmniRoute numbers come from whatever traffic happened to pass through the gateway; the headroom numbers come from one fixed task. The two sets of percentages cannot be compared directly. A fair comparison runs the same workload through both.
• Prompt caching is a feature of the model provider. OmniRoute’s part is routing requests so the cache stays warm; we have not measured how warm it would be without the gateway.
• We expect compression savings to stay near zero until long sessions, with large tool outputs and files, go through the gateway, since that is the kind of context both tools are built to shrink. That is an expectation, not a measurement.
• This is an observational record, not a pre-registered experiment: there is no fixed hypothesis, sample size or verdict rule.

How the data updates

The figures are a JSON snapshot stored with the site and rendered when the site is built; the page makes no live requests. For now the snapshot is refreshed by hand; each refresh replaces the JSON file. The date at the top of the page (16 October 2026, 11:33 UTC) is the snapshot you are reading.