TL;DR
We ran Terminal-Bench v2 with and without Boost: no harm to task pass rate, and a ~10% reduction in estimated cost.
To measure whether Boost helps agents without hurting their ability to complete real work, we ran Terminal-Bench 2.0 — a benchmark of hard, human-verified terminal tasks in containerized environments — with and without Boost enabled.
Setup
| Benchmark |  Terminal-Bench 2.0 (89 tasks) |
|---|---|
| Evaluated | 81 tasks — 8 tasks excluded due to infrastructure constraints in our Harbor evaluation environment |
| Agent | Claude Code |
| Model | Claude Haiku 4.5 (us.anthropic.claude-haiku-4-5-20251001-v1:0) via Amazon Bedrock |
| Baseline | Agent runs commands with no Boost hooks |
| Boost | Same agent and model, with Boost hooks installed ((cmd) | boost rewrite + filters) |
Harbor Hub results
These links are public, but Harbor Hub requires you to sign in before you can view a job.
Task success (primary metric)
Success rate preserved
25 / 81 tasks passed
Boost preserved task completion on this benchmark: identical pass rate to the baseline.
Cost and token efficiency
Aggregate across all 81 tasks. Bars are relative to the baseline (=100%).
Key savings
On long-horizon agent benchmarks, most tokens are reasoning and context — not raw tool stdout — so aggregate input-token savings are modest even when per-command compression is much larger. The ~12% cost reduction on the same pass rate is the more meaningful signal: Boost makes the same agent work cheaper without changing how many tasks it completes.
For day-to-day development, per-command savings are much larger. In a
typical 30-minute session (~60 shell commands), Boost reduces context
from tool output by roughly 91% on common workflows
(git, test runners, linters, Docker).
Evaluations summary
| Eval | Baseline | With Boost |
|---|---|---|
| Terminal-Bench 2.0 pass rate (81 tasks, Claude Haiku 4.5) | 30.9% | 30.9% |
| Estimated cost | $29.04 | $25.57 (−11.9%) |
| Typical dev session tool-output tokens (internal estimate) | ~189K | ~16K (~−91%) |
Terminal-Bench runs used Claude Code with Claude Haiku 4.5 on Amazon Bedrock. Eight of 89 Terminal-Bench 2.0 tasks were excluded from this evaluation due to infrastructure limitations in our Harbor environment. Per-command token savings (60–90%) come from measured filter performance on real tool output; agent-benchmark aggregates include model reasoning tokens and understate per-command compression.