CLI reference

Scope: the complete command-line surface. Source of truth: src/cli.ts.

moose-tracevals [options] [command]

Commands:
  run [options] [traces...]        Evaluate one or more traces against the
                                   skills and instructions they used
  calibrate [options] [traces...]  Measure judged and graded verdicts against a
                                   labels file: false passes, false fails, and
                                   review volume
  fill [options] [paths...]        Propose evals for skills, agent definitions,
                                   and project rules, and write those above the
                                   confidence threshold
  capture [options]                Record a session manifest: sha256 of every
                                   instruction artifact plus the git SHA, read
                                   from a Claude Code hook payload on stdin
  list [options]                   List discoverable traces (Claude Code session
                                   files)
  help [command]                   display help for command

Options:
  -V, --version                    output the version number
  -h, --help                       display help for command

run is the default command: moose-tracevals <trace> and moose-tracevals run <trace> are equivalent.

Invoking the binary

The package and the bin it installs are both named moose-tracevals. Every command below is written as the bare moose-tracevals; prefix it with npx if the CLI is not on your PATH.

run

moose-tracevals run [options] [traces...]

Evaluates one or more traces against the skills, agent definitions, slash commands, and project-rules files each session used. Read-only: it never modifies the traces or the artifacts it evaluates.

Arguments

Argument	Required	Description
traces...	no	Paths to trace files, evaluated in the order given. Omit them in an interactive terminal to get a picker, or select them with the flags below.

Omitting traces without a TTY and without a selector produces an error instead of a prompt:

moose-tracevals: no trace given; pass a trace file or run `moose-tracevals list`

Which report you get

One command, two report shapes. What decides is how the traces were selected, not how many came back. A script can therefore be written against the shape before it runs:

Invocation	Report
run <trace>	RunReport, for one session
run <a> <b> …	BatchReport, with rates across sessions
run --all-projects / --since / --limit	BatchReport, even when the selector matches exactly one trace

Naming traces and passing a selector is an error rather than a precedence rule. Either answer would silently discard something you asked for:

moose-tracevals: name traces or select them with --all-projects/--since/--limit, not both

A selector that matches nothing is exit 2, not a green run. An empty corpus and a clean corpus are indistinguishable downstream, so the gate says so instead of passing:

moose-tracevals: no traces matched (--all-projects, --since 7d); run `moose-tracevals list` to see what the session store holds

Options

Flag	Argument	Default	Description
--project	<dir>	the trace’s recorded cwd	Artifact-lookup root, and the discovery scope when a selector is used without --all-projects.
--all-projects	none	off	Evaluate traces from every project in the session store.
--since	<duration>	none	Keep only traces modified within the window: 30m, 24h, 7d, 2w. Anything else is exit 2.
--limit	<n>	unlimited	Maximum traces to evaluate, applied after newest-first sorting. A whole number >= 1; anything else is exit 2.
--provider	<name>	provider.default (claude-cli)	Judge provider: claude-cli, anthropic, openai, mock.
--model	<model>	the selected provider’s configured model	Judge model override.
--runs	<n>	judge.ensembleRuns (3)	Ensemble runs per judged eval. A whole number >= 1; anything else is exit 2.
--deterministic-only	none	off	Skip judged evals entirely. They are reported as skipped, not failed. No model call is made, so nothing leaves the machine.
--commands / --no-commands	none	config	Override graders.command.enabled. --no-commands does not execute command-graded evals; they report skipped with the reason stated, never pass. Pass neither and the config value stands.
--no-cache	none	cache on	Bypass the judge cache.
--max-cost-usd	<usd>	judge.maxCostUsd (unset)	Judge cost budget for the whole invocation, shared across every trace in a batch, not applied per trace. A finite number >= 0; anything else is exit 2. Exhausting it makes a batch exit 1; see Exit codes.
-f, --format	<format>	human	human, json, or markdown.
-o, --output	<file>	none	Also write the rendered report to a file.
--history	none	off	Append to the history file and compare with the previous run for the same session. In a batch this is one entry per trace, since history is matched per session.
--report-unused-artifacts	none	reportUnusedArtifacts (false)	List every skill and agent the session was offered and never used, instead of only counting them. Never changes the exit code.
--fail-on-needs-review / --no-fail-on-needs-review	none	config	Override failOnNeedsReview. Pass neither and the config value stands.
--require	<module>	none	Load a grader plugin. Repeatable, and added to plugins rather than replacing it.
--manifest	<file>	searched	A session manifest to compare artifacts against. Single-trace only. Without it one is looked for automatically. With it, a file that is missing, unreadable, or recorded for a different session is exit 2 rather than a shrug.

A numeric flag that does not parse is exit 2

parseFloat("abc") is NaN, and every comparison against NaN is false. A budget of NaN would not fall back to the config, because undefined is what defers to the config. It would reach the budget gate as a ceiling that can never be hit. The run then bills without a limit while the report still claims one. --limit=-1 has the same shape: it reaches slice(0, -1) and quietly evaluates every trace except the oldest.

moose-tracevals: --max-cost-usd must be a number of at least 0, got NaN
moose-tracevals: --limit must be a whole number of at least 1, got -1

Exit codes

Code	Meaning
0	every eval passed or was skipped, and in a batch on every trace
1	any fail or error, and needs-review unless failOnNeedsReview: false. In a batch, also any trace that could not be parsed, and a batch whose shared judge budget ran out mid-corpus
2	an operational error. That covers bad usage, an unreadable trace named on its own, an unknown format, and a selector that matched nothing. It also covers a bad --since. So do a numeric flag that is not a number or is out of range, and a grader plugin that will not load

An unreadable trace inside a batch is an entry, not an abort

Naming one bad trace on its own is exit 2. Naming it alongside good ones is different. It becomes a report entry carrying its error, the other traces keep their verdicts, and the run exits 1. One unreadable file in a corpus of fifty must not cost the other forty-nine.

calibrate

moose-tracevals calibrate [options] [traces...]

Runs the same batch run does, then joins the results against a labels sidecar and reports how often the tool agreed with you. Read-only: labels, traces, and artifacts are never written.

Every run flag applies except --history, which is deliberately absent. Calibration measures a corpus, not a point in one session’s timeline.

Options beyond run’s

Flag	Argument	Default	Description
--labels	<file>	calibrate.labels (tracevals/labels.yaml)	The ground-truth sidecar. A path here resolves against the working directory; the config default resolves against the config file.
--sweep	none	off	Re-score the corpus across the configured grid, from cached verdicts.
--max-false-pass	<n>	calibrate.maxFalsePass (unset)	Exit 1 above this many false passes.
--max-false-fail	<n>	calibrate.maxFalseFail (unset)	Exit 1 above this many false fails.
--max-review	<n>	calibrate.maxReview (unset)	Exit 1 above this many needs-review outcomes across the corpus.

Exit codes

Code	Meaning
0	the corpus was measured, including when it disagreed. Disagreement is the finding, not a failure
1	a threshold above was asked for and exceeded, or the measurement is incomplete. Incomplete means a trace could not be evaluated, the shared judge budget cut the corpus short, or nothing was scored at all
2	operational: no labels file, an invalid one, a label naming a trace outside the corpus, a label matching no eval result, or any run operational error

A measurement that measured nothing is exit 1

When every label joins to a skipped result, scored is 0 and agreement is null. Each --max-* threshold then reads “0 of at most N, not exceeded”. That happens under --deterministic-only over ai-graded labels, or with a budget exhausted on the first trace. The thresholds are honest; the denominator behind them is empty, so exiting 0 would certify a corpus nobody measured:

no labelled eval produced evidence: 2 label(s) all joined to a skipped result, so agreement and every --max-* threshold are computed over an empty denominator

A label that matches nothing is exit 2, not a warning

A misspelled eval id contributes to no count, so the report would come out looking like a cleaner corpus than it is. The message names what that trace did evaluate for that artifact:

moose-tracevals: …/labels.yaml: label(s) matched no eval result:
  fix-bug › forbiden-tool on claude-session.jsonl (that trace evaluated eval-5, forbidden-tool, no-force-push, refactor-preserved-intent, used-read for "fix-bug")

fill

moose-tracevals fill [options] [paths...]

Proposes eval evals for the instruction artifacts in a project and writes those that survive the gate. The only command that writes. run never calls it.

Arguments

Argument	Required	Description
paths...	no	Specific files or directories to fill. Omit to scan --project. An explicit file path is taken at its word; an unrecognized filename is treated as project rules.

A path that does not exist is an operational error:

moose-tracevals: no such file or directory: ./nope

Options

Flag	Argument	Default	Description
--project	<dir>	current directory	Project root to scan.
--dry-run	none	off	Report proposals without writing them. Recommended first invocation.
--confidence	<n>	fill.confidenceThreshold (0.7)	Minimum confidence to write, 0–1.
--max-evals	<n>	fill.maxEvalsPerArtifact (8)	Maximum evals per artifact, including existing ones.
--max-cost-usd	<usd>	fill.maxCostUsd (unset)	Proposal cost budget.
--no-cache	none	cache on	Bypass the proposal cache.
--provider	<name>	provider.default	claude-cli, anthropic, openai, mock.
--model	<model>	the selected provider’s configured model	Model override.
--require	<module>	none	Load a grader plugin, same semantics as on run.
-f, --format	<format>	human	human or json. Note: no markdown here.

Numeric flags are range-checked before anything runs. Out of range, or not a number:

moose-tracevals: --confidence must be a number between 0 and 1, got 1.5

`--require` changes little here

fill never proposes a custom grader kind, because the per-artifact-type allowlist is fixed. Loading a plugin matters only when it replaces a built-in, since the gate ground-checks proposals with the registered grader’s validateOptions. It is honoured so that a repository’s config means the same thing to every command.

Exit codes

Code	Meaning
0	every artifact was processed without error
1	any artifact result has status error
2	operational error: an out-of-range numeric flag, a missing path, a grader plugin that will not load

What fill never does

Existing evals are never modified; a name collision fails the run. Only the YAML frontmatter block is rewritten; the body is byte-identical. Project rules (CLAUDE.md, AGENTS.md, GEMINI.md) are proposed but never written. See Fill in evals.

capture

moose-tracevals capture [options]

Records a session manifest, holding the sha256 of every instruction artifact in the project plus the git SHA, at the moment a session starts. run reads it back and turns staleness from an mtime guess into content identity.

It is meant to be wired to a Claude Code SessionStart hook, and reads that hook’s JSON payload on stdin. That payload is session_id, cwd, transcript_path, and hook_event_name. Run by hand, name the session yourself.

moose-tracevals capture --session-id demo --project test/fixtures/project --out .tmp/docs-capture.json

Captured session demo
  manifest   .tmp/docs-capture.json
  artifacts  6 hashed
  commit     cbcd42ef424cfe28748e2a610ee1ad02403b68d1 (main) — working tree dirty
  device     e1dafb2151ca3cbc (win32)

Options

Flag	Argument	Default	Description
--project	<dir>	the payload’s cwd	Project root to scan, and the base every recorded path is relative to.
--session-id	<id>	the payload’s session_id	Names the manifest and is the key run joins on.
-o, --out	<file>	<project>/<capture.dir>/<id>.json	Write somewhere else. This is how you put a manifest beside a trace you are about to upload.
-f, --format	<format>	human	human or json.

Exit codes

Code	Meaning
0	a manifest was written
2	no session id could be determined, the payload on stdin was not a JSON object, or the manifest could not be written

In hook mode it writes nothing to stdout

Claude Code adds a SessionStart hook’s stdout to the model’s context. So whenever capture reads a payload from stdin, its report goes to stderr instead. Otherwise the tool would be writing into the very session it is observing. Run by hand it behaves like every other command.

Wire it up

Install the bundled plugin, which pre-wires the hook. plugin/.claude-plugin/plugin.json and plugin/hooks/hooks.json ship in the package. Or add it to settings.json yourself:

{
  "hooks": {
    "SessionStart": [
      {
        "matcher": "startup",
        "hooks": [
          { "type": "command", "command": "npx --no moose-tracevals capture", "timeout": 60 }
        ]
      }
    ]
  }
}

startup rather than every event. A manifest describes what the artifacts said when the session began, which is exactly the “then” that staleness compares against. compact and clear reset the model’s context without re-reading a single artifact.

resume is left out for the same reason, and it is worth knowing what that means. A resumed session keeps the manifest captured at its original start. Edit a SKILL.md, resume, and the turns after the resume are compared against what the file said before the edit. That is reported as mismatch, which is true of the session as a whole and blunt about the later turns. Add resume to the matcher if you would rather each resume re-snapshot the project; the policy lives in your settings.json, not in the command.

Where run looks for one

Most specific first, and the first usable hit wins:

1. <trace>.manifest.json, beside the trace file
2. <trace's directory>/<capture.dir>/<session-id>.json
3. <project>/<capture.dir>/<session-id>.json, where capture writes by default

A manifest whose sessionId does not match the trace’s is refused, and the run proceeds as if there were none. It is evidence about a different session.

So is one the trace gives no way to check. A trace that records no session id cannot be joined to any manifest by convention. Locations 2 and 3 then have no key to look under, and location 1 is refused with a warning naming the file. A claude -p stream-json transcript captured without its system/init record is one such trace. Name it with --manifest to use it anyway. That is your assertion that the file belongs to this trace, and for such a trace it is the only one available.

list

moose-tracevals list [options]

Lists discoverable traces from the Claude Code session store, newest first. Always exits 0 on success.

Options

Flag	Argument	Default	Description
-p, --project	<dir>	current directory	Scope to one project’s sessions.
-a, --all-projects	none	off	Scan every project in the session store.
-l, --limit	<n>	unlimited	Maximum traces to list. A whole number >= 1; anything else is exit 2.
--json	none	off	Emit { "traces": [...] } instead of a table.

Each table entry is three lines. First the timestamp and first prompt, then project, session id, and size, then the file path:

  1. 2026-07-02 00:00  Write a README for this repo.
     /work/other-project · 22222222-2222-2222-2222-222222222222 · 1KB
     /home/you/.claude/projects/-work-other-project/22222222-....jsonl

When nothing matches:

No traces found. Pass --all-projects to scan every project, or --project <dir> to scope to one.

JSON shape

Field	Type	Description
file	string	Absolute path. Pass this to run.
mtimeMs	number	Modification time; the sort key, newest first.
sizeBytes	number	File size.
sessionId	string?	The session’s id, when present in the file.
project	string?	The recorded working directory.
firstPrompt	string?	First non-sidechain user message, first line, capped at 200 characters.

Environment variables

Variable	Effect
MOOSE_TRACEVALS_HOME	Overrides the home directory for both trace discovery and user-level artifact lookup. A relative value resolves against the current working directory. Use it to make a run reproducible on a shared runner.
MOOSE_TRACEVALS_LIVE	Contributor-only. Set to 1 to include the live judge smoke test in this repository’s own suite. Has no effect on the CLI.