FrontierHarness Eval runs one model, Kimi K3, through different coding agent harnesses on the same 30 tasks. Mouse passed 25 of them, run under the benchmark's own scripts, at $2.79 per pass. The top published entry, Codex, passed 20.
Mouse 
Kimi Code 
Exo The benchmark
Twenty-one of the tasks come from Terminal-Bench 2.1 and are command line jobs: recover a corrupted SQLite database, build a Cython extension, find the best chess move. The other nine are DeepSWE tasks, bug fixes and features in open source repositories, graded by hidden tests. Every harness gets the same model, the same tasks, and the same runtime, so the differences in the table are the harness: its prompt, its configuration, its tools, and how it decides a task is finished.
How Mouse works
Mouse runs on the OpenCode engine with its own system prompt and configuration, and a completion loop that keeps the session going after the model says it is done. Most agents stop the moment the model's last message is not a tool call. Mouse instead looks at the workspace: did anything change, do the repository's own checks pass, was a test file deleted? A failing check goes back to the model as the next prompt. Once the checks pass, the model has to go back over the original task and account for each requirement with evidence before the run ends. Deleting a test to make it pass ends the run.
That loop is what carries the long tasks. Stock OpenCode passed none of the nine DeepSWE tasks; Mouse passed eight, each one running between 26 and 89 minutes and up to 249 turns before its checks came back green. The same loop runs Night Shift, our overnight mode.
The harness is open source at github.com/mousedev/mouse-harness, MIT licensed, with the loop documented step by step and the prompt frozen and pinned by a test so that what was benchmarked is what you get.
Leaderboard
| Harness | Pass | Terminal | DeepSWE | $ / pass | Median $ | Cache | Time |
|---|---|---|---|---|---|---|---|
| Mouse | 25/30 | 17/21 | 8/9 | $2.79 | $0.18 | 91% | 6:23 |
| Codex | 20/30 | 15/21 | 5/9 | $3.47 | $0.12 | 88% | 6:43 |
| DSH Creator | 19/30 | 15/21 | 4/9 | $3.28 | $0.12 | 84% | 6:43 |
| Claude Code | 19/30 | 16/21 | 3/9 | $18.34 | $0.29 | 68% | 9:37 |
| Pi | 18/30 | 16/21 | 2/9 | $2.43 | $0.07 | 79% | 7:33 |
| DSH Standard | 18/30 | 16/21 | 2/9 | $3.46 | $0.12 | 86% | 6:16 |
| DSH PTC | 18/30 | 16/21 | 2/9 | $4.58 | $0.14 | 87% | 7:43 |
| Kimi Code | 17/30 | 14/21 | 3/9 | $3.65 | $0.18 | 88% | 7:55 |
| DSH Minimal | 17/30 | 15/21 | 2/9 | $4.72 | $0.12 | 85% | 5:41 |
| Oh My Pi | 17/30 | 16/21 | 1/9 | $4.75 | $0.14 | 82% | 6:46 |
| Exo | 16/30 | 16/21 | 0/9 | $1.05 | $0.07 | 70% | 6:17 |
| Hermes | 15/30 | 13/21 | 2/9 | $2.90 | $0.17 | 86% | 6:57 |
| OpenCode | 15/30 | 15/21 | 0/9 | $3.24 | $0.06 | 78% | 6:27 |
Cost per pass is total spend on all 30 tasks divided by passes, the figure the leaderboard labels "median cost per task". Median cost, cache rate, and time cover passing tasks only. Three of Mouse's passes, scc-bounded-memory-spilling, kv-store-grpc, and largest-eigenval, are tasks none of the twelve published configurations solved.
Mouse is not on the official board yet. FrontierHarness adds a harness after reproducing the result in their own environment; the run is submitted for that in frontier-harness-eval/eval#12.
Cost
Exo Mouse Kimi Code Mouse is third on cost per pass. The DeepSWE tasks account for most of the spend: the arktype and python-statemachine tasks each cost over ten dollars, running for more than an hour. The median passing task cost eighteen cents, and the median cache hit rate was 90.6%, the highest in the table, because every round continues the same session rather than starting a fresh one.
Terminal-Bench
| Task | Mouse | Steps | Cost | Time | Field |
|---|---|---|---|---|---|
| regex-log | Pass | 10 | $0.16 | 7:42 | 11/12 |
| openssl-selfsigned-cert | Pass | 16 | $0.12 | 4:12 | 12/12 |
| polyglot-c-py | Fail | 17 | $0.21 | 8:17 | 12/12 |
| sqlite-db-truncate | Pass | 14 | $0.18 | 6:20 | 12/12 |
| git-leak-recovery | Pass | 13 | $0.10 | 5:05 | 12/12 |
| log-summary-date-ranges | Pass | 14 | $0.14 | 4:05 | 12/12 |
| constraints-scheduling | Pass | 8 | $0.12 | 5:28 | 12/12 |
| gcode-to-text | Fail | 21 | $0.69 | 13:43 | 5/12 |
| dna-insert | Fail | 23 | $0.40 | 15:56 | 2/12 |
| largest-eigenval | Pass | 16 | $0.27 | 16:45 | 0/12 |
| merge-diff-arc-agi-task | Pass | 18 | $0.18 | 5:57 | 12/12 |
| vulnerable-secret | Pass | 16 | $0.13 | 3:55 | 12/12 |
| extract-elf | Fail | 25 | $0.55 | 13:06 | 2/12 |
| build-cython-ext | Pass | 52 | $0.75 | 11:44 | 12/12 |
| kv-store-grpc | Pass | 16 | $0.13 | 4:37 | 0/12 |
| chess-best-move | Pass | 32 | $0.28 | 8:11 | 5/12 |
| db-wal-recovery | Pass | 11 | $0.11 | 5:46 | 12/12 |
| code-from-image | Pass | 6 | $0.05 | 2:39 | 4/12 |
| modernize-scientific-stack | Pass | 14 | $0.14 | 4:08 | 12/12 |
| multi-source-data-merger | Pass | 8 | $0.12 | 4:08 | 12/12 |
| sanitize-git-repo | Pass | 29 | $0.42 | 6:23 | 10/12 |
Field is how many of the twelve published harnesses passed the task.
DeepSWE
| Task | Mouse | Steps | Cost | Time | Field |
|---|---|---|---|---|---|
| anko-typed-variable-bindings | Pass | 70 | $2.62 | 26:22 | 4/12 |
| arktype-json-schema-refs-dependencies | Pass | 249 | $10.91 | 81:25 | 2/12 |
| fastapi-deprecation-response-headers | Pass | 126 | $5.60 | 60:13 | 4/12 |
| httpx-multipart-response-parsing | Pass | 124 | $4.49 | 47:53 | 5/12 |
| expr-try-catch-errors | Pass | 133 | $5.68 | 41:19 | 2/12 |
| python-statemachine-state-data-scoping | Pass | 216 | $12.94 | 88:56 | 7/12 |
| katex-multicolumn-array-spans | Pass | 108 | $3.75 | 42:49 | 1/12 |
| scc-bounded-memory-spilling | Pass | 119 | $4.88 | 41:06 | 0/12 |
| meriyah-explicit-resource-declarations | Fail | 236 | $13.67 | 90:00 | 1/12 |
The one miss, meriyah, ran to the task's 90 minute limit with its checks still failing.
How the run was made
- FrontierHarness's own trial driver, unmodified, at their eval commit
e837a70, on a golden checkpoint on Runta with a fresh restore for every task. Terminal-Bench through Harbor 0.22.0, DeepSWE through Pier 0.3.1. - Harness commit
315e2b8on OpenCode 1.18.27, Kimi K3 served by Fireworks, the same provider the baselines used. - All 30 trials valid. The run hit two infrastructure incidents on the way, a provider spending limit and a hung CLI, and every affected attempt is retained and explained in the run's notes; none of them counts as a pass or a failure.
- The complete run, every trajectory, verifier verdict, and command line, is published in the repository under
evals/runs/2026-09-08-mouse-c.
An earlier self-run on 2026-09-03, through Harbor on a cloud VM with Kimi K3 via OpenRouter, scored 24 of 30. It is kept in the same repository as history.
Mouse is built on OpenCode and is not affiliated with it. Run completed 2026-09-09.