Upgrading

Upgrading without silently discarding a populated cache — or, worse, replaying entries that no longer mean what they meant.

Composing a key in the first place is caching.

The fact that makes this a journey

Inside this repository JudgeRun is treated as a file format rather than an internal type, and a change to it carries a BREAKING CHANGE: footer so semantic-release majors correctly. So:

A major version bump is your signal to assume the judge cache is stale until you have checked.

A minor or patch bump cannot change JudgeRun’s shape. That is the whole contract, and it is why the field names look more conservative than they otherwise would.

What the library can and cannot invalidate

An honest split, and the reason this page is short:

	Who controls it
Cannot invalidate your cache	You compose the key. The library only hashes the parts you hand it — no `PROMPT_VERSION` ships here and there is no library-owned invalidation signal.
Can change what a key means	If `JudgeRun` gains or loses a field, an entry written before the upgrade deserializes differently afterwards.

So your protection is entirely your own, and it is two things.

Do it

// Surviving an upgrade: version your cache key, and re-validate on read.
// Runs with no API key: MockProvider stands in for a real provider.

import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

import {
  JsonCache,
  MockProvider,
  buildCacheKey,
  mockVerdict,
  runEnsemble,
} from "@hawkeyexl/inference";

const directory = mkdtempSync(join(tmpdir(), "inference-upgrade-"));

const provider = new MockProvider([mockVerdict("pass", 0.95)], "claude-sonnet-4-5");
const system = "You evaluate whether a page satisfies an assertion.";
const user = "# Assertion\nThe page documents authentication.\n\n# Page\nUse a bearer token.";

// The library composes nothing on your behalf, so a version marker you control is
// the only thing that can invalidate an entry when YOUR prompt or expectations change.
const keyFor = (promptVersion) =>
  buildCacheKey([provider.provider(), provider.modelName(), `v${promptVersion}`, "r1"]);

const cache = new JsonCache(directory, true, "my-tool");

await runEnsemble({ provider, system, user, runs: 1, cache, cacheKey: keyFor(1) });
const replay = await runEnsemble({ provider, system, user, runs: 1, cache, cacheKey: keyFor(1) });
console.log("same version replays:", replay[0].cached);

// Bump the marker when your prompt changes and the old entry is simply not found.
const afterBump = await runEnsemble({ provider, system, user, runs: 1, cache, cacheKey: keyFor(2) });
console.log("bumped version misses:", afterBump[0].cached === false);

// The library cannot know your value shape, so a well-formed but OBSOLETE entry
// replays happily. Re-validate on read — this is the wrapper every consumer writes.
class VerdictCache extends JsonCache {
  get(key) {
    const entry = super.get(key);
    if (!Array.isArray(entry)) return undefined;
    const shapedCorrectly = entry.every(
      (run) => run.error !== undefined || typeof run.verdict?.confidence === "number",
    );
    return shapedCorrectly ? entry : undefined;
  }
}

// Simulate an entry written before `confidence` existed.
const guarded = new VerdictCache(directory, true, "my-tool");
const staleKey = keyFor(3);
guarded.set(staleKey, [
  { verdict: { claim: "c", observed: "o", match: "pass", reasoning: "r" }, provider: "anthropic", model: "m", cached: false, durationMs: 1 },
]);

console.log("raw cache would replay it:", new JsonCache(directory, true, "my-tool").get(staleKey) !== undefined);
console.log("guarded cache rejects it: ", guarded.get(staleKey) === undefined);

rmSync(directory, { recursive: true, force: true });

same version replays: true
bumped version misses: true
raw cache would replay it: true
guarded cache rejects it:  true

1. Put a version marker in the key

const cacheKey = buildCacheKey([
  provider.provider(),
  provider.modelName(),
  `v${MY_PROMPT_VERSION}`,   // yours to bump
  `r${runs}`,
  sha256(pageBody),
]);

Bump it when your prompt, your verdict schema, or your expectations change. The old entries are not deleted — they simply stop being found, which means a rollback still hits them.

2. Re-validate on read

The last two output lines are the point: the raw cache replays a stale entry happily, and a guarded one rejects it.

class VerdictCache extends JsonCache {
  get(key) {
    const entry = super.get(key);
    if (!Array.isArray(entry)) return undefined;
    const shapedCorrectly = entry.every(
      (run) => run.error !== undefined || typeof run.verdict?.confidence === "number",
    );
    return shapedCorrectly ? entry : undefined;
  }
}

JsonCache deliberately does not know what shape you store, so it cannot do this for you. It already treats a corrupt or non-array entry as a miss; what it cannot detect is an entry that is well-formed and obsolete.

Every existing consumer of this package wrote this wrapper independently. Write it once, up front, and an upgrade becomes a non-event.

Release channels

semantic-release with conventional commits:

Branch	Channel	Reaches a caret range?
`main`	stable	yes
`next`	prerelease	no
`feat/**`	its own npm dist-tag, derived from the branch name	no

Per-branch dist-tags exist for trying an in-progress feature deliberately. They cannot reach you by accident.

What to check on a major bump

One local-model wrinkle

A resolved selector changes the model name in your cache key. Weights cached under auto on one machine will not be hit on a machine with more memory — because the key names the concrete model that ran, not the selector.

That is the async factory working as designed, not a cache bug. It is what stops a 2.62 GB and a 6.72 GB model from sharing results under one key.

Caching — key composition in full
Cache reference — the on-disk format
Custom verdict schema — the other thing that should bump your marker