How Mouse uses verification loops to keep coding agents working

I reran Mouse on the 30-task FrontierHarness benchmark last night and scored 25/30.

Mouse is built on OpenCode, which only passed 15/30, scoring last place in the benchmark, so I wanted to explain what we added to Mouse that brought us to 1st place.

Mouse 83%

Codex 67%

DSH Creator 63%

Claude Code 63%

Pi 60%

DSH Standard 60%

DSH PTC 60%

Kimi Code 57%

DSH Minimal 57%

Oh My Pi 57%

Exo 53%

Hermes 50%

OpenCode 50%

The benchmark is built by Runta to evaluate the leading code harnesses all running a single model, Kimi K3. Every harness on the board ran that model on the same 30 tasks in the same runtime, which measures performance from the harness keeping all else equal. 21 tasks are sourced from Terminal-Bench command line jobs and 9 are DeepSWE tasks.

Mouse ran under the benchmark's published scripts, at $2.79 per pass with a 90.6% median cache hit rate.

The loop

Most harnesses, OpenCode included, end a session the moment the model's last message is not a tool call. Mouse works differently by enforcing the same fixed sequence after every turn, and only that sequence can end a run. By using a single loop of deterministic steps we seem to unlock a significant gain on difficult and long-horizon tasks.

We borrowed from pstack's approach to verification. The repo's checks must pass before Mouse allows completion, and the agent must provide a supporting file, test, or command output for each requirement. Rules like blocking deleted tests are enforced directly in the harness.

Each round:

1. Check the repo for changes using git status and git diff. If nothing changed, ask the agent to implement the task.
2. Run the repo's checks, such as build, test, and lint. Each command runs separately with a time limit.
3. Check for deleted tests or workflows. The harness blocks the run if the agent deleted these files, preventing it from passing checks by removing failing tests.
4. Read the agent's MOUSE_AUDIT. The agent marks each requirement [done] with a supporting file, test, or command output, or [todo] if unfinished.
5. If the workspace changed, the checks pass, and the audit is complete, finish. Otherwise, send failed commands and their output or remaining todos back to the agent, then repeat the loop.

Each follow-up includes the original task, requirement list, and audit format. Everything stays in the same session, preserving context and allowing the repeated prompt prefix to stay cached, which helps keep costs down.

Mouse 91%

Kimi Code 88%

Codex 88%

DSH PTC 87%

DSH Standard 86%

Hermes 86%

DSH Minimal 85%

DSH Creator 84%

Oh My Pi 82%

Pi 79%

OpenCode 78%

Exo 70%

Claude Code 68%

Highest cache hit rate on the board.

What it did to the results

On the nine DeepSWE tasks, which are long bug fixes, Mouse passed eight. Some of those runs went over an hour and past 200 steps before their checks came back green. Mouse also passed three tasks that none of the twelve published harnesses solved.

Exo $1.05

Pi $2.43

Mouse $2.79

Hermes $2.90

OpenCode $3.24

DSH Creator $3.28

DSH Standard $3.46

Codex $3.47

Kimi Code $3.65

DSH PTC $4.58

DSH Minimal $4.72

Oh My Pi $4.75

Claude Code $18.34

Third cheapest per task.

Thoughts

I think verification explains much of the gain, although we haven't measured it separately from Mouse's prompt and config changes yet. Our results here play into the narrative of the last few weeks that the harness is as important as the model you're running, and improvements above the frontier model layer can make a massive difference in the results you achieve.

In addition to the loop, we've added a deterministic set of rules in Mouse (see house rules) based on pstack's approach that force the agent to follow stricter guidelines for task completion at the harness level, not just in the system instructions or prompt.

This run is still provisional until the Runta team replicates and verifies it on their own. This is our second run: the first completed through Harbor on our own cloud VM, the second through Runta's infrastructure, scripts, skill, and sanctioned eval. They scored 24/30 and 25/30.

Mouse is open source, MIT. The completion loop is about 800 lines. The full run includes every trajectory, verifier verdict, and notes on two infrastructure incidents.

We're releasing Mouse clients for iOS, watchOS, and iMessage in the coming weeks. Our goal is to build a world-class coding agent you can use from your phone, your watch, or a text.

Questions or want to compare notes: pete@mouse.dev.