评测与研究
TypeSafe/Jev 的 AI agent 工具调用预执行防火墙的影子模式验证工具,实际运行结果见 report.md。
Shadow-mode validation harness for a pre-execution firewall on AI agent tool calls (TypeSafe/Jev). Real run, findings in report.md.
"""Same battery as a raw HTTP payload, for POST /v1/systemone without the SDK."""链接指向我们抓取当天的那个 commit,行号是准的。
下面是从这个项目源码里原样取出来的 question 组合。
is_urgent是/否The message conveys urgency
severity打分How severe the problem is for the customer
0 — Minor annoyance1 — Real problem2 — Losing money nowimport { experimental_evaluate as evaluate } from 'ai';
const { answers } = await evaluate({
model: 'typesafe-ai/jev',
state,
questions: {
is_urgent: { type: 'boolean', instructions: 'The message conveys urgency' },
severity: { type: 'score', instructions: 'How severe the problem is for the customer',
criteria: ['Minor annoyance', 'Real problem', 'Losing money now'] },
},
});