Build on moose-tracevals

Scope: consuming moose-tracevals as a library rather than a CLI. It assumes the vocabulary from the journey pages, because a custom grader cannot be explained before the built-in graders are.

npm install moose-tracevals

ESM-only, Node.js 24+, types shipped.

The front door

The package exports a lot. These are the entry points meant for consumers; everything else is reachable but is there to serve the CLI.

You want to…	Use
Run a full evaluation	runEvals(options) → RunReport
Read a trace without evaluating	parseTraceFile(path) → Trace
Find traces on disk	discoverTraces(options) → TraceListing[]
Resolve which artifacts a trace used	resolveArtifacts(trace, options)
Add a check the built-ins do not cover	registerGrader(grader)
Import grader plugins the way the CLI does	loadGraderPlugins({ plugins, configDir })
Render a report	render(report, format)
Load configuration	loadConfig() / parseConfig(raw)

import { loadConfig, runEvals, render } from "moose-tracevals";

const config = await loadConfig();
const report = await runEvals({
  tracePath: "./traces/session.jsonl",
  projectDir: "./my-repo",
  config,
  deterministicOnly: true,
});

console.log(render(report, "markdown"));
console.log(report.summary, report.exitCode);

RunReport is the same structure --format json prints, documented field by field in Reports & exit codes. The full export list is in the API reference.

`runEvals` needs a judge, or `deterministicOnly`

EngineOptions.judge is optional in the type but required unless deterministicOnly: true. Omit both and judged evals have nothing to run them.

`runEvals` does not load plugins; `runRun` does

config.plugins is imported by the run and fill commands, which know the directory the config came from. Calling runEvals directly you are already holding registerGrader. Call it yourself, or call loadGraderPlugins({ plugins, configDir }) if you want the CLI’s resolution and warnings.

Injection seams

Every place moose-tracevals reaches outside the process is injectable. Anything you build on top can keep its own suite hermetic, through the same seams this project’s tests use.

Seam	Where	Replaces
EngineOptions.judge	runEvals	The whole judged path. Pass your own TraceJudge, or set deterministicOnly and pass nothing.
EngineOptions.env	runEvals	Environment lookup, including MOOSE_TRACEVALS_HOME. Point artifact resolution at a fixture home.
ResolveOptions.env	resolveArtifacts	The same, when resolving directly.
DiscoverOptions.env	discoverTraces	The session store root.
promptFn	pickTrace(options, promptFn)	The interactive prompt, so the picker is testable without a TTY.
--provider mock / makeJudgeProvider	judge construction	A real provider that returns a canned verdict with no network call.
RunCommandOptions.require / FillOptions.require	runRun, runFill	Grader plugins, appended to config.plugins. Point it at a fixture module instead of an installed package.

A TraceJudge is just a function, so a stub is one line:

import { runEvals, type TraceJudge } from "moose-tracevals";

const alwaysPasses: TraceJudge = async (plans) =>
  plans.map((plan) => ({
    evalName: plan.evalName,
    artifact: plan.artifact.path,
    artifactName: plan.artifact.name,
    grader: plan.grader,
    implicit: plan.implicit,
    outcome: "pass",
    costUsd: 0,
    durationMs: 0,
  }));

const report = await runEvals({ tracePath, config, judge: alwaysPasses });

Pin the home directory in your tests too

Artifact resolution reads ~/.claude/skills/, ~/.claude/agents/, and the plugin store. Pass env: { MOOSE_TRACEVALS_HOME: "test/fixtures/home" } so results do not depend on whose machine the suite runs on.

Reading traces without evaluating

parseTraceFile gives you the normalised model on its own, which is enough for analysis that has nothing to do with adherence:

import { parseTraceFile } from "moose-tracevals";

const trace = await parseTraceFile("./traces/session.jsonl");
console.log(trace.turnCount, trace.skillInvocations.map((s) => s.name));

Every field is described in Traces reference. Note usage is absent for Claude Code session files. Only stream-json transcripts carry cost.

The trace-adapter seam

TraceSource is a union with one member today, claude-code. Support for other formats is deferred rather than rejected (ADR 01003), and the shape a new adapter has to conform to is the normalised Trace:

• Required for anything to work: source, file, cwd, events, turnCount, and the derived collections (toolCalls, skillInvocations, agentSpawns, fileAccesses, userMessages, assistantTexts).
• Degrades gracefully when absent: usage (the cost grader skips with a reason), model, gitBranch, sessionId, startedAt / endedAt.
• Instruction semantics are the adapter’s job. What counts as a “skill” differs between agent tools; the adapter decides, and everything downstream consumes the normalised result.

Adding one is a change to this repository, not something you can register at runtime. The seam is a shape to conform to, not a plugin point.

Where to go next

Write a custom graderThe two-part contract, why option validation is not optional, and how to register one.

The export surface, mappedEverything the package exports, grouped, with what each group is for.