Agent tooling
MCP server and agent skill for the TypeSafe AI System One API (Jev): decompose a judgment into Choice / Score / Noul questions, lint them, measure on labelled data, and put calibrated thresholds in code
if (err instanceof APIConnectionError) return `Could not reach api.typesafe.ai: ${err.message}. Check network access.`;The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
from skills/tenbin/templates/questions.py:10
categorychoiceWhat is this support message about?
bug_report — Something in the product behaves wronglybilling — Charges, invoices, refunds, payment methodsfeature_request — Asks for something the product does not doother — None of the abovebug_severityscoreIf the message reports a bug, how severe is it?
0 — Cosmetic or wording issue1 — Broken feature, but a workaround exists2 — Broken feature with no workaround3 — Data loss, security exposure or the product is unusablehas_repro_stepstrue/falseThe message contains steps that would let an engineer reproduce the problem
refund_requestedtrue/falseThe customer asks for money back
frustrationscoreHow frustrated is the customer?
0 — Calm, states facts1 — Frustrated but civil2 — Very angry, strong language or threatens to leave3 — Abusive or threateningimport { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
category: { type: 'choice', instructions: 'What is this support message about?',
criteria: { bug_report: 'Something in the product behaves wrongly', billing: 'Charges, invoices, refunds, payment methods', feature_request: 'Asks for something the product does not do', other: 'None of the above' } },
bug_severity: { type: 'score', instructions: 'If the message reports a bug, how severe is it?',
criteria: ['Cosmetic or wording issue', 'Broken feature, but a workaround exists', 'Broken feature with no workaround', 'Data loss, security exposure or the product is unusable'] },
has_repro_steps: { type: 'boolean', instructions: 'The message contains steps that would let an engineer reproduce the problem' },
refund_requested: { type: 'boolean', instructions: 'The customer asks for money back' },
frustration: { type: 'score', instructions: 'How frustrated is the customer?',
criteria: ['Calm, states facts', 'Frustrated but civil', 'Very angry, strong language or threatens to leave', 'Abusive or threatening'] },
},
});from tenbin/src/server.test.ts:90
sevscoreRate severity from 0 to 2
0 — 01 — 12 — 2import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
sev: { type: 'score', instructions: 'Rate severity from 0 to 2',
criteria: ['0', '1', '2'] },
},
});from tenbin/src/server.test.ts:102
departmentchoiceWhich team should handle this?
billing — Paymentstechnical — BugsfrustrationscoreHow frustrated is the customer?
0 — Calm, just stating facts1 — Frustrated but civil2 — Very angry, strong languageis_urgenttrue/falseDoes the message convey urgency?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
department: { type: 'choice', instructions: 'Which team should handle this?',
criteria: { billing: 'Payments', technical: 'Bugs' } },
frustration: { type: 'score', instructions: 'How frustrated is the customer?',
criteria: ['Calm, just stating facts', 'Frustrated but civil', 'Very angry, strong language'] },
is_urgent: { type: 'boolean', instructions: 'Does the message convey urgency?' },
},
});from tenbin/src/server.test.ts:132
is_urgenttrue/falseDoes the message convey urgency?
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
is_urgent: { type: 'boolean', instructions: 'Does the message convey urgency?' },
},
});