Configuration reference

Scope: every configuration key. Source of truth: src/core/config-schema.json and src/core/config.ts.

Configuration lives under the tracevals: key of moose.config.yaml in the working directory. One file serves the whole moose family, and sibling tools read their own top-level keys out of it. Everything documented on this page is therefore nested one level deep:

moose.config.yaml

docmeta:            # another tool's section; never validated here
  schemas: […]
tracevals:          # ours; every key on this page lives under here
  judge:
    ensembleRuns: 3

Every key is optional; an absent file, or one carrying only other tools’ sections, means every default applies.

How a value is resolved

moose.config.yaml  →  `tracevals:` section  →  schema validation  →  defaults applied  →  CLI override  →  runtime

Three properties follow from this, and you can rely on all three:

• CLI flags override the config; they do not bypass it. The resolved config is fully populated before anything reads it, so a flag replaces a known value instead of filling a hole.

• Unknown keys are hard errors, inside the section. The schema forbids additional properties at every level below tracevals:, so a misspelled key fails loudly:

  moose-tracevals: invalid config: /judge must NOT have additional properties

  Top-level keys beside tracevals: are another tool’s business and are never validated here. An unparseable file fails the same way: moose-tracevals: could not parse moose.config.yaml: …

• A section left un-nested or miscased is an error, not a shrug. Writing judge: at the top level, or spelling the wrapper Tracevals:, would otherwise discard your whole config silently, so it is reported instead:

  moose-tracevals: invalid config: moose.config.yaml has no `tracevals:` section, but `judge` sits at the top level — nest it under `tracevals:`

Complete example

Every key below is shown with its default. You would never write all of this.

moose.config.yaml

tracevals:
  provider:
    default: claude-cli          # anthropic | openai | claude-cli | mock
    anthropic:
      model: claude-sonnet-4-5
      apiKeyEnv: ANTHROPIC_API_KEY
    openai:
      baseUrl: https://api.openai.com/v1
      model: gpt-4o-mini
      apiKeyEnv: OPENAI_API_KEY
    claude-cli:
      model: claude-sonnet-4-5
      command: claude
  judge:
    ensembleRuns: 3
    temperature: 0
    zones:
      autoPass: 0.8
      autoFail: 0.8
    cacheDir: .moose-tracevals/cache
    redact: []                   # extra patterns scrubbed from the digest
    # maxCostUsd has no default; set it to impose a ceiling
  graders:
    command:
      enabled: true              # false, or --no-commands, skips them instead
                                # (--commands forces them back on)
  render:
    maxBlockChars: 2000
    maxTotalChars: 150000
  history:
    file: .moose-tracevals/history.jsonl
  capture:
    dir: .moose-tracevals/sessions   # where session manifests are written and read
  fill:
    confidenceThreshold: 0.7
    maxEvalsPerArtifact: 8
    temperature: 0
    cacheDir: .moose-tracevals/cache/fill
    # maxCostUsd has no default
  calibrate:
    labels: tracevals/labels.yaml
    # maxFalsePass / maxFalseFail / maxReview have no default
    sweep:
      ensembleRuns: [1, 3, 5]
      autoPass: [0.5, 0.6, 0.7, 0.8, 0.9, 0.95]
      autoFail: [0.5, 0.6, 0.7, 0.8, 0.9, 0.95]
  failOnNeedsReview: true
  plugins: []                  # grader modules to import before evals are planned
  reportUnusedArtifacts: false

provider

Inference provider settings. Configure any number of sections and select one with default or --provider; only the selected section is used.

Key	Type	Default	CLI
provider.default	anthropic | openai | claude-cli | mock	claude-cli	--provider

claude-cli is the default deliberately: it uses your local Claude CLI’s own authentication, so a fresh checkout can judge without anyone provisioning an API key.

provider.anthropic

Key	Type	Default	CLI
model	string	claude-sonnet-4-5	--model
apiKeyEnv	string	ANTHROPIC_API_KEY	none
pricing	pricing	none	none

provider.openai

Key	Type	Default	CLI
baseUrl	string	https://api.openai.com/v1	none
model	string	gpt-4o-mini	--model
apiKeyEnv	string	OPENAI_API_KEY	none
pricing	pricing	none	none

baseUrl accepts any /chat/completions-compatible server: OpenAI, Azure, Ollama, Groq, Together.

provider.claude-cli

Key	Type	Default	CLI
model	string	claude-sonnet-4-5	--model
command	string	claude	none

provider.mock

There is no configuration section for mock; pass --provider mock to use it. It is a real provider that returns a canned passing verdict, for exercising the pipeline offline.

`apiKeyEnv` is a variable name, never a key

It names the environment variable moose-tracevals should read. Putting an actual key here commits a secret to your repository.

pricing

Both provider.anthropic and provider.openai accept a pricing object that overrides the built-in price table for that model.

Key	Type	Required
inputPerMTok	number ≥ 0	yes
outputPerMTok	number ≥ 0	yes

tracevals:
  provider:
    openai:
      model: my-self-hosted-model
      pricing:
        inputPerMTok: 0.15
        outputPerMTok: 0.60

Set this for any model the price table does not know

Cost is computed from a built-in per-model table. A model the table does not recognize costs zero, which silently disables every maxCostUsd budget you set.

judge

Key	Type	Default	CLI
ensembleRuns	integer ≥ 1	3	--runs
temperature	number 0–2	0	none
zones.autoPass	number 0–1	0.8	none
zones.autoFail	number 0–1	0.8	none
cacheDir	string	.moose-tracevals/cache	--no-cache disables
redact	array of strings	[]	none
maxCostUsd	number ≥ 0	(unset)	--max-cost-usd

• ensembleRuns: more runs cost proportionally more and make an auto-resolved verdict harder to reach, since every run must agree.
• temperature: leave at 0. A nonzero value adds noise to verdicts and warns once per process.
• zones: the mean-confidence floor an ensemble must clear to auto-resolve. Unanimity is required regardless; these thresholds are the additional bar. Lowering autoPass converts reviews into passes.
• maxCostUsd: checked before each judged eval. Once exhausted, remaining evals are reported as skipped with the budget named.
• redact: extra regular expressions scrubbed from the session digest before it is sent.

See How the judge decides for what these do to an outcome.

judge.redact

Patterns applied to the transcript before it leaves the machine. Each source is compiled with the g flag; a match becomes [redacted].

tracevals:
  judge:
    redact:
      - "ACME-[0-9]{4}"           # an internal ticket id
      - "https://internal\\.example\\.com/[^\\s\"]+"

Three things to know:

• These add to the built-in shapes; they never replace them. Vendor API-key prefixes, bearer tokens, private-key blocks, AWS_*, and secret-named assignments are always redacted, with the list empty or not.
• A pattern that will not compile is an error at load, not a silent skip. A dropped pattern would be a silent leak.
• It is best-effort, not a guarantee. --deterministic-only is the only setting under which nothing leaves the machine at all.

See What leaves the machine for the full list of what a judged run sends, redacted or not.

graders

Per-grader settings. Only graders with a knob outside the eval that names them appear here.

Key	Type	Default	CLI
graders.command.enabled	boolean	true	--commands / --no-commands

command-graded evals run a program named in an artifact, and artifacts come from the trace’s project. Evaluating someone else’s trace can therefore execute that repository’s code. That is the documented default and it is not changing; this is the switch for when you do not want to take it.

tracevals:
  graders:
    command:
      enabled: false

Disabled, a command eval reports skipped with the reason stated, never pass. An unrun check has not been satisfied, so the report says so rather than quietly counting it as clean. Nothing else about the run changes, including the exit code. See the command grader.

command execution is disabled (--no-commands / graders.command.enabled: false)

The flag is a pair, and both run and calibrate honour it. Pass neither and this setting decides; --no-commands turns execution off for one invocation; --commands turns it back on against a config that disabled it.

render

Controls how much of the session transcript is given to the judge.

Key	Type	Default	CLI
maxBlockChars	integer ≥ 100	2000	none
maxTotalChars	integer ≥ 1000	150000	none

When a transcript exceeds maxTotalChars, a head-and-tail window is kept and the middle is replaced with a truncation marker. Raise it for long sessions where the relevant behavior sits in the middle, at the cost of a larger prompt.

history

Key	Type	Default	CLI
file	string	.moose-tracevals/history.jsonl	none

Where --history appends. See Reports & exit codes.

capture

Key	Type	Default	CLI
dir	string	.moose-tracevals/sessions	capture --out writes elsewhere; run --manifest reads elsewhere

Where capture writes a session manifest and where run looks for one, resolved against the project root rather than the working directory. It sits inside .moose-tracevals/ on purpose. That directory is already where the judge cache lives, and it is already gitignored. Being inside the project is what makes a manifest travel with its trace in the CI patterns.

Changing it changes both ends, so a project that relocates manifests keeps capture and run agreeing without a flag on either.

fill

Key	Type	Default	CLI
confidenceThreshold	number 0–1	0.7	--confidence
maxEvalsPerArtifact	integer ≥ 1	8	--max-evals
temperature	number 0–2	0	none
cacheDir	string	.moose-tracevals/cache/fill	--no-cache disables
maxCostUsd	number ≥ 0	(unset)	--max-cost-usd

maxEvalsPerArtifact counts existing evals too, so an artifact already at the cap gets no new ones. The fill cache is separate from the judge cache: different key scheme, different value shape.

calibrate

Settings for calibrate, which measures verdicts against a labels sidecar.

Key	Type	Default	CLI
labels	string	tracevals/labels.yaml	--labels
maxFalsePass	integer ≥ 0	(unset)	--max-false-pass
maxFalseFail	integer ≥ 0	(unset)	--max-false-fail
maxReview	integer ≥ 0	(unset)	--max-review
sweep.ensembleRuns	array of integers ≥ 1	[1, 3, 5]	none
sweep.autoPass	array of numbers 0–1	[0.5, 0.6, 0.7, 0.8, 0.9, 0.95]	none
sweep.autoFail	array of numbers 0–1	[0.5, 0.6, 0.7, 0.8, 0.9, 0.95]	none

labels resolves against this config file’s directory, exactly as plugins does. An explicit --labels is a working-directory path, like every other path argument on the CLI.

The three max* keys are unset by default, and a calibration run is a measurement rather than a gate. Set one and disagreement above it becomes exit 1. They are integers rather than rates because 0 has to be expressible. “No false pass at all” is the most common setting anyone wants, and it cannot double as “unset”.

sweep is the grid --sweep tries, one axis at a time with the other knobs held at their configured value. It lives here rather than behind flags because the useful range is a property of a corpus, and a corpus outlives an invocation. Widening sweep.ensembleRuns is the one calibration setting that costs money: the corpus is judged once at the largest value in it.

failOnNeedsReview

Key	Type	Default	CLI
failOnNeedsReview	boolean	true	--fail-on-needs-review / --no-fail-on-needs-review

Whether a needs-review outcome contributes to a non-zero exit code. Set it to false to surface ambiguous verdicts in the report without blocking on them.

plugins

Key	Type	Default	CLI
plugins	array of strings	[]	--require appends

Modules imported before evals are planned, so one calling registerGrader() can add a grader kind the built-ins do not cover. Honoured by both run and fill. See Write a custom grader.

tracevals:
  plugins:
    - ./tracevals/graders.mjs      # relative to *this file*, not the cwd
    - /opt/shared/graders.mjs      # absolute, as written
    - "@acme/tracevals-graders"    # resolved from this file's directory

Four things to know, in the order they will bite:

• Specifiers resolve against this file’s directory, never the working directory. That is what lets a committed relative path survive a CI step that runs the CLI from the repository root.
• Load order is significant. Entries load in order, config before --require, and a later one wins a kind an earlier one already claimed.
• --require adds; it does not replace. A one-off flag would otherwise unregister every grader the config names, and each eval declaring one would report unknown grader kind. That points at an artifact that is not the problem.
• A specifier that will not import exits 2. Skipping it would leave a run that looks like it ran. A module that imports but registers nothing warns instead, and one that takes over a built-in kind is allowed and reported.

This imports code named in your config

A plugin runs with the CLI’s full privileges; there is no sandbox. The list is empty unless you write one, and it lives in the repository doing the evaluating. The command grader differs, because it runs a program named by the artifacts of the project being evaluated. Evaluating someone else’s trace cannot introduce a plugin.

reportUnusedArtifacts

Key	Type	Default	CLI
reportUnusedArtifacts	boolean	false	--report-unused-artifacts

Whether the coverage table lists every skill and agent the session was offered and never used, or only counts them. Off by default because a real roster runs to a couple of hundred skills and would bury the evals above it.

This is an observation, not a gate. Turning it on changes what the report shows and nothing about what it decides. The summary counts and the exit code are identical either way.

Keys with no CLI flag

Worth knowing, because they can only be changed in the config file:

judge.temperature · judge.zones.autoPass · judge.zones.autoFail · judge.cacheDir · judge.redact · render.maxBlockChars · render.maxTotalChars · history.file · capture.dir · fill.temperature · fill.cacheDir · calibrate.sweep.ensembleRuns · calibrate.sweep.autoPass · calibrate.sweep.autoFail · every provider.* key except model

Behavior with no key at all

Artifact staleness has no knob. Every resolved artifact is stat’d and compared with the session’s end time. One that is newer is flagged in the coverage table and named in a warning. It is always on because it costs one stat and can only ever add information. It never becomes an eval outcome and never changes the exit code. See Artifact coverage.

Reading a session manifest has no knob either, beyond where to look. If one exists for the trace, run uses it; if not, every hash check reports skipped with the reason and the mtime heuristic above stands unchanged. There is nothing to turn on, because there is nothing to turn on for until someone runs capture.