Skip to content
Mouse Mouse API
Get API key

Tuning on your outcomes

A tuning takes the decisions your harness already logs, with what actually happened, and gives back a model id that answers the way your harness wants: calibrated on your data, with the base model's behaviour guaranteed not to regress, and nothing kept except an adapter you can delete.

How it works

Section titled “How it works”
  1. Log outcomes. You already call POST /v1/systemone. Keep each request's state and questions. When you learn what the right answer was (the user approved, the tool ran, the tests passed), add the answer as label. That record is a tuning example, the same shape as a calibration example.
  2. Get a quote. POST /v1/tunings with dry_run: true and your examples. You get the method, training tokens, estimated GPU minutes and cost, the Mouse fee, and per-question counts.
  3. Run it. Same call without dry_run. The quote must fit in your prepaid balance (402 insufficient_credits otherwise). You get {"id": "tune_…", "model": "mouse-1:ft-acme-7f3a2c", "status": "queued"}. Readout jobs take about 5 minutes, LoRA 20–80. We email you when it finishes.
  4. Read the report. GET /v1/tunings/{id} shows, per question, accuracy and calibration before and after on examples the job never trained on, how often a planted instruction or a reversed option order flipped the answer compared with mouse-1 on those same examples, and whether the model was published. If it was not, the report says why ("accuracy fell on safe_to_run; 40% of labels disagree with the model on near-identical states").
  5. Use it. Send model: "mouse-1:ft-acme-7f3a2c" on /v1/systemone. Everything else is unchanged. The response echoes the exact model id. Calibration profiles fit on the tuned model work only with it. The tuned model answers the questions it was trained on, matched by type and instructions (the option list may change from request to request); a question it never saw in training is answered by mouse-1 in the same response.
  6. Keep it current. Post newer examples with baseline set to your current model. The new one publishes only if it beats the old one. Use model: "mouse-1:ft-acme" to follow the newest published tuning, or pin the full id.
  7. Delete it. DELETE /v1/tunings/{id}. The adapter is removed and hosting stops. Your examples were never stored.
POST /v1/tunings
{
  "model": "mouse-1",
  "suffix": "acme",
  "examples": [
    {"state": {"command": "rm -rf ./tmp"}, "questions": {"safe_to_run": {"type": "noul", "instructions": "Is `command` safe to run without approval?", "label": true}}}
  ]
}

The same flow is available without code at /console/tunings: paste or upload a JSONL file of examples, read the quote, confirm, and the report appears when the job finishes.

Two methods

Section titled “Two methods”
MethodExamplesJob timeLatencyWhat changes
readout (default)200 to 5,00010–15 min36 msThe model's final decision layer is refit on your labels. The rest of the model is untouched, so the tuned model runs on the same fast path as the base model.
lora1,000 to 20,00020–80 min36 ms on the shared pool, about 110 ms while a job is served eagerlyA small low-rank adapter is trained through the whole model, for tasks the readout alone cannot separate.

Leave method out and the API picks: readout when the example count fits it, lora above 5,000 examples.

Request and response shapes

Section titled “Request and response shapes”

Quote (dry_run: true)

Section titled “Quote (dry_run: true)”
curl https://api.mouse.dev/v1/tunings \
  -H "Authorization: Bearer $MOUSE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "mouse-1", "suffix": "acme", "dry_run": true, "examples": [ ... ]}'
{
  "method": "readout",
  "examples": 1840,
  "training_tokens": 412000,
  "gpu_seconds_est": 780,
  "gpu_usd_est": 0.86,
  "fee_usd": 10.0,
  "total_usd": 10.86,
  "questions": [
    { "signature": "9c1e4b2f7a03d851", "type": "noul", "count": 1840 },
    { "signature": "4f8c12a90e3b7104", "type": "choice", "count": 36, "warning": "Fewer than 50 examples; this question is evaluated but not fitted on its own." }
  ]
}

Nothing is billed for a quote. The GPU estimate comes from the token count; the job bills the seconds it actually used.

Run (202 Accepted)

Section titled “Run (202 Accepted)”
{
  "id": "tune_01j9x3k8m2v7",
  "model": "mouse-1:ft-acme-7f3a2c",
  "status": "queued",
  "method": "readout",
  "report": null,
  "training_tokens": 412000,
  "gpu_seconds": 0,
  "cost": { "gpu_usd": 0.86, "fee_usd": 10.0 },
  "created_at": "2026-10-16T14:02:11Z",
  "finished_at": null
}

status moves through queued → running → one of ready, rejected, or failed. A ready model that stops being hosted (credits exhausted) shows suspended until you top up; a deleted one shows deleted.

Report (GET /v1/tunings/{id})

Section titled “Report (GET /v1/tunings/{id})”
{
  "id": "tune_01j9x3k8m2v7",
  "model": "mouse-1:ft-acme-7f3a2c",
  "status": "ready",
  "method": "readout",
  "training_tokens": 412000,
  "gpu_seconds": 742,
  "cost": { "gpu_usd": 0.81, "fee_usd": 10.0 },
  "created_at": "2026-10-16T14:02:11Z",
  "finished_at": "2026-10-16T14:15:40Z",
  "report": {
    "published": true,
    "reasons": [],
    "questions": {
      "9c1e4b2f7a03d851": {
        "type": "noul",
        "keys": ["safe_to_run"],
        "n_train": 1472,
        "n_holdout": 368,
        "before": { "accuracy": 0.861, "ece": 0.071, "nll": 0.342, "brier": 0.104 },
        "after":  { "accuracy": 0.927, "ece": 0.024, "nll": 0.198, "brier": 0.058 },
        "temperature": 1.12,
        "passed": true
      }
    },
    "canary": {
      "passed": true,
      "failed": [],
      "injection": { "n": 7, "before": 0.0, "after": 0.0, "new": 0, "fixed": 0, "p": 1.0 },
      "order": { "n": 7, "before": 0.0, "after": 0.0, "new": 0, "fixed": 0, "p": 1.0 }
    },
    "baseline": null
  }
}
  • before and after are measured on the 20% of your examples the job held out and never trained on.
  • passed is per question: accuracy must not fall and calibration error must not rise beyond a small tolerance against the base model (or against baseline when you set one), and held-out log loss must improve overall. A question the tuned model gets right on every held-out example passes calibration as long as its log loss did not rise: there, calibration error only measures how far below 100% its confidence sits.
  • canary checks that the tuning did not make your choice questions easier to steer. Each held-out example is asked again with a planted instruction naming one option (injection) and again with the options reversed (order), through the base model and through the tuned model. new counts answers that flip on the tuned model but held on the base model, and fixed counts the reverse. The model is refused only if its flip rate (after) is more than one point above the base model's (before) and an exact paired sign test (p) puts the chance of that excess under 5%. One stray flip on a small held-out set is noise and passes. A tuning that makes your questions steerable does not.
  • The tuned model answers only the question families in the report (same type and instructions). Every other question in a request goes to the base model, so a tuning cannot change behaviour you did not train on.
  • When published is false, status is rejected, reasons says why in plain words, and the job is billed: the report is what you paid for. A failed job is our fault and is refunded.

List and delete

Section titled “List and delete”
curl https://api.mouse.dev/v1/tunings -H "Authorization: Bearer $MOUSE_API_KEY"
curl -X DELETE https://api.mouse.dev/v1/tunings/tune_01j9x3k8m2v7 -H "Authorization: Bearer $MOUSE_API_KEY"

After a delete, requests for that model id (and for an alias that now resolves to nothing) return 404 model_not_found.

From outcomes to examples: a coding-agent harness

Section titled “From outcomes to examples: a coding-agent harness”

A harness already has everything a tuning example needs. It asks the question at decision time and learns the answer later, when the action runs, the user approves, or the tests pass. The pattern is: write the request down when you ask, write the label down when you know, and ship the file.

import { appendFile } from "node:fs/promises";


type Pending = { state: unknown; questions: Record<string, unknown> };
const pending = new Map<string, Pending>(); // keyed by your own action id


// 1. Decision time: ask Mouse, remember what you asked.
async function gate(actionId: string, command: string, task: string) {
  const state = { task, command };
  const questions = {
    safe_to_run: { type: "noul", instructions: "Is `command` safe to run without approval given `task`?" },
  };
  const res = await fetch("https://api.mouse.dev/v1/decisions", {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.MOUSE_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({ model: process.env.MOUSE_MODEL ?? "mouse-1", state, questions }),
  });
  pending.set(actionId, { state, questions });
  return (await res.json()).answers.safe_to_run.noul > 0.9;
}


// 2. Outcome time: the user approved or denied, or the command ran cleanly or did damage.
async function record(actionId: string, wasSafe: boolean) {
  const p = pending.get(actionId);
  if (!p) return;
  pending.delete(actionId);
  const example = {
    state: p.state,
    questions: { safe_to_run: { ...p.questions.safe_to_run, label: wasSafe } },
  };
  await appendFile("mouse-examples.jsonl", JSON.stringify(example) + "\n");
}

Label from what happened, not from what the model said: an approval the user granted, a command that ran without rollback, a pull request that merged, a test suite that passed. Skip examples where you never learned the outcome. Once the file has 200 lines, post it:

jq -sc '{model: "mouse-1", suffix: "acme", examples: .}' mouse-examples.jsonl \
  | curl https://api.mouse.dev/v1/tunings -H "Authorization: Bearer $MOUSE_API_KEY" -H "Content-Type: application/json" -d @-

Then set MOUSE_MODEL=mouse-1:ft-acme and keep logging. The next time the file fills up, post it again with "baseline": "mouse-1:ft-acme"; the alias moves only if the new model beats the one you are running.

The same pattern fits any decision your harness makes and later learns the truth about: which tool the agent should have called (the one that made progress), whether a pull request was ready (CI and review said so), whether a support ticket was routed right (the team that closed it).

What Mouse keeps

Section titled “What Mouse keeps”

Your examples are the input to the training job and nothing else: they are not written to disk, not logged, and not used for anything but your job. The report carries metrics and question signatures, never example text. The only thing that persists is the adapter, scoped to your account, which DELETE /v1/tunings/{id} removes. See Data retention and security.

Pricing

Section titled “Pricing”

Readout tunings are $10 per job; LoRA tunings are $10 per 1M training tokens with a $25 minimum. GPU time is passed through at our H100 rate. Hosting a tuned model is $10 per model per month after the first 30 days, and tuned inference is $0.40 per 1M input tokens. Full table in Pricing and billing; minimums and caps in Rate limits.