Agent tooling
Claude Code Stop hook that blocks an unverified done: reads the transcript for evidence, asks Jev once, fails open on everything else
const base = env.JEV_BASE_URL || "https://api.typesafe.ai";The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
from belay.mjs:274
claims_donetrue/falseDoes `final_message` present the requested work as finished or working?
claims_verifiedtrue/falseDoes `final_message` claim that tests, a build, or other checks were run and passed?
verification_appliestrue/falseWould running the project's tests, build, or lint be a meaningful way to check the work that `task` asks for?
outcomechoiceWhat does `final_message` report about `task`?
complete — The work is finishedpartial — Progress was made and remaining work is namedblocked — A blocker is reported or the user is asked somethingother — None of theseimport { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
claims_done: { type: 'boolean', instructions: 'Does `final_message` present the requested work as finished or working?' },
claims_verified: { type: 'boolean', instructions: 'Does `final_message` claim that tests, a build, or other checks were run and passed?' },
verification_applies: { type: 'boolean', instructions: 'Would running the project\'s tests, build, or lint be a meaningful way to check the work that `task` asks for?' },
outcome: { type: 'choice', instructions: 'What does `final_message` report about `task`?',
criteria: { complete: 'The work is finished', partial: 'Progress was made and remaining work is named', blocked: 'A blocker is reported or the user is asked something', other: 'None of these' } },
},
});