JEV RECIPES
Routing is where a small spend prevents a large one: pay a fraction of a cent to decide whether this request needs to wake the model that costs a dollar a call. That only works if the decision itself is fast and cheap — otherwise the router becomes the bottleneck.
Every number on this page comes from our own 2,390-question run; method and limits are in the benchmark report (written in Chinese).
Judge complexity and whether real reasoning is required, then pick a tier.
Set a conservative threshold: take the small model only on a confident `small`, and send anything uncertain to the large one. What you save by routing is small next to what one wrong route costs.
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state, // Paste a user request
questions: {
needsReasoning: { type: 'boolean', instructions: 'Does fulfilling this request require multi-step reasoning, planning or non-trivial computation?' },
complexity: { type: 'score', instructions: 'How complex is this request?',
criteria: ['One step', 'Simple with some judgement', 'Several steps', 'Complex and open-ended'] },
route: { type: 'choice', instructions: 'Which tier of model should handle this request?',
criteria: { small: 'A small fast model is enough', large: 'Needs a frontier reasoning model', tools: 'Needs external tools or retrieval' } },
},
});Decide whether the knowledge base is needed at all, and skip a lot of pointless vector queries.
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state, // Paste a user question
questions: {
needsRetrieval: { type: 'boolean', instructions: 'Does answering this require internal information a general model could not know?' },
timeSensitive: { type: 'boolean', instructions: 'Does the answer depend on current or recent data?' },
chitchat: { type: 'boolean', instructions: 'Is this message only a greeting or small talk, with no actual information need?' },
},
});Send a natural-language request to the handler that owns it.
import { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state, // Paste what the user said
questions: {
handler: { type: 'choice', instructions: 'Which part of the system should handle this request?',
criteria: { refund: 'Refunds and cancellations', order_query: 'Order status lookup', modify: 'Changing an existing order', support: 'Needs a human agent', unknown: 'Intent unclear' } },
hasOrderRef: { type: 'boolean', instructions: 'Does the request contain something that could identify a specific order, such as an amount, a date or an order number?' },
confidence: { type: 'score', instructions: 'How clearly has the user expressed what they want?',
criteria: ['Not clear at all', 'Needs a follow-up question', 'Reasonably clear', 'Unambiguous'] },
},
});Median 456 ms, slowest 1% under 0.8 s. Against a downstream call of 1.3–1.7 s with a 6–7.6 s tail, that trade is clearly worth it. If your downstream is a local small model, run the numbers again — it may not be.
Embeddings are faster and cheaper, but they can only sort by resemblance. Jev decides whether something is the case, which covers criteria like "does this need multi-step reasoning" that similarity cannot express. Layer them: embeddings for the easy majority, Jev for the rest.
It can and it should. Route, retrieval need and small-talk detection in one call cost about what one of them costs alone.
Anything that requires counting or arithmetic. Counting accuracy measured 65–77%, and the failure mode is characteristic — it tends to agree with whatever number the question suggests. Compute those criteria in code.
Each of these has a line of code that calls Jev. More of them in awesome-jev-verified.
The questions above are fixed. To write your own criteria, options and score levels, open the playground.
Open the playground