Combating context bloat in agentic AI
Same 89 tasks. 2h 17m faster.
Against RTK, Boost cut wall-clock time 26.1%, lowered cost per task 18.6%, and kept 18.1M input tokens out of context.
ResultBoost led all eight Harbor metrics, including speed, cost, token use, and savings-report coverage.
- Suite
- Terminal-Bench 2.0
- 89 human-verified tasks
- Agent
- Claude Code
- unmodified
- Model
- Claude Haiku 4.5
- via Amazon Bedrock
- Harness
- Harbor
- forked for wrapper injection
- Runtime
- Isolated Docker
- one container per task
- Trials
- 267
- 89 tasks × 3 configurations
Same work, less drag
Where Boost pulls ahead.
RTK compared with Boost · 89 tasks
Wall-clock time
Shorter is better
2h 17m
returned
Cost per task
Lower is better
18.6%
lower cost
Input tokens
Less context carried
18.1M
tokens avoided
Bar length shows Boost relative to RTK for each measured metric. Exact values and complete baseline telemetry appear below.
Boost advantage over RTK
Boost wins every efficiency metric.
Bars show Boost’s measured reduction against RTK across the same 89 tasks. Positive values mean less time, lower spend, and fewer tokens carried.
01Findings
The outcome first: faster work at lower cost.
- 01
26.1%
Less wall-clock time than RTK
Boost finished all 89 tasks in 6h 27m. RTK needed 8h 44m: 2h 17m longer.
- 02
18.6%
Lower cost per task than RTK
$0.276 per task with Boost against $0.339 with RTK. Against the uncompacted baseline, cost per task falls 13.5%.
- 03
18.1M
Fewer input tokens than RTK
164.0M input tokens with Boost against 182.2M with RTK, measured across identical work on identical tasks.
- 04
76%
Fewer unreported trials than RTK
Boost reported savings in 81 of 89 trials, leaving 8 gaps. RTK reported in 56, leaving 33.
02Figure 1
The advantage holds against RTK and raw output.
Full-suite duration
Lower is better
Cost per task
Lower is better
Savings-report coverage
Higher is better
Bars are scaled to the largest value in each group. Coverage is the share of the 89 trials in which the compactor reported its own savings; the baseline runs no compactor, so it has nothing to report.
03Table 1
Outcome telemetry, all 89 tasks.
| Metric | Baseline | RTK | Boost | Boost vs RTK |
|---|---|---|---|---|
| Costper task | $0.319 | $0.339 | $0.276 | −$0.063 (−18.6%) |
| Session costfull suite | $28.41 | $30.20 | $24.58 | −$5.62 (−18.6%) |
| Wall-clock durationfull suite | 9h 04m | 8h 44m | 6h 27m | −2h 17m (−26.1%) |
| Savings-report coverage89 trials | n/a | 62.9% | 91.0% | +28.1 points |
View complete token telemetry4 metrics
Aggregate input and cache totals reflect the full multi-turn agent context, not only raw tool output. They are included for auditability and should not be read as per-command compression rates.
| Metric | Baseline | RTK | Boost | Boost vs RTK |
|---|---|---|---|---|
| Input tokensper task | 1,919,610 | 2,046,861 | 1,842,993 | −203.8K (−10.0%) |
| Input tokenstotal | 170,845,290 | 182,170,718 | 164,025,358 | −18.1M (−10.0%) |
| Cache tokenstotal | 166,101,358 | 177,226,442 | 160,856,530 | −16.4M (−9.2%) |
| Output tokenstotal | 1,494,321 | 1,471,333 | 1,371,976 | −99.3K (−6.7%) |
04Method
How the run was built and what it controls for.
01Evaluation environment
Every task ran in its own isolated Docker container under the Harbor orchestration harness. Harbor recorded input tokens, output tokens, cached tokens, duration, and direct API cost for each job.
02Dataset
Terminal-Bench 2.0, a public suite of 89 hard, human-verified terminal tasks that JFrog did not author. Tasks exercise environment navigation, tool use, and sequential debugging.
03Configurations
Baseline passes raw stdout and stderr to the agent. RTK applies open-source filtering, grouping, and deduplication. Boost compacts terminal output before it reaches the context window. Agent, model, and harness are identical in all three.
04Controlling for variance
Multi-turn agent runs are stochastic: one dropped token can send a trial off course. Rather than reading single tasks, all three configurations ran the complete 89-task suite and conclusions are drawn from per-task averages across the whole set.
05Harness modification
Harbor does not natively support injecting a CLI proxy into an agent’s execution loop, so it was forked to add environment wrappers and telemetry collection. The claude-code agent itself was not altered for any configuration.
06Field validation
Benchmark findings are checked against production telemetry from more than 1,000 engineers who ran Boost through its beta, so results reflect real repositories rather than suite conditions alone.
05Artifacts
Every job log is open.
The three Harbor jobs behind this report are public. Harbor Hub asks you to sign in before it will render one.
06Lab program
What the lab is running next.
- Terminal-Bench 2.0, 89 tasksClaude Haiku 4.5Published
- Terminal-Bench 2.0, 89 tasksClaude SonnetScheduled
- Terminal-Bench 2.0, 89 tasksClaude OpusScheduled
- Production telemetry cohort1,000+ engineersContinuous
Two hours returned. 18.1 million fewer input tokens carried. Same 89 tasks.
Start with one machine, or deploy the same setup across your organization.
Terminal-Bench 2.0 is a public, human-verified suite that JFrog did not author. All three configurations ran the same agent, the same model, and the same harness; only the terminal compaction layer changed. Token and cost figures are Harbor telemetry recorded per job. The 91% session figure is a measured estimate across common developer workflows, not a Terminal-Bench result.