Benchmarks & research
Measure when to use Jev and other models on your data, then route accordingly.
from typesafe_sdk import TypeSafeClient # import tardif : dependance optionnellesrc/janus/providers/typesafe.py:28
The link points at the commit we read, so the line number still holds.
These question sets are lifted from this project's source exactly as written.
departmentchoiceQuelle équipe doit traiter ce message ?
shipping — Suivi de commande, retard, colis perdubilling — Facturation, paiement, remboursementimport { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
department: { type: 'choice', instructions: 'Quelle équipe doit traiter ce message ?',
criteria: { shipping: 'Suivi de commande, retard, colis perdu', billing: 'Facturation, paiement, remboursement' } },
},
});