基础设施

typesafe-ai-benchmark

@iammrduncan32TypeScriptMIT更新于 2026-09-19可当场跑

这是一个模拟 typesafe ai 结构化输出的 LLM 网关,类似一个冒牌 Jev。

英文原文

This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.

iammrduncan/typesafe-ai-benchmark

它在哪儿调用了 Jev

const isChat = (request: FastifyRequest) => request.url.split('?')[0] !== '/v1/systemone';

packages/api/src/http.ts:28

链接指向我们抓取当天的那个 commit,行号是准的。

它问 Jev 的问题

下面是从这个项目源码里原样取出来的 question 组合。

取自 packages/api/examples/benchmark.ts:24

  1. refund是/否

    Does the customer request money back?

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    refund: { type: 'boolean', instructions: 'Does the customer request money back?' },
  },
});

取自 packages/api/src/logging.test.ts:21

  1. CANARY_QUESTION_ID是/否

    CANARY_RUBRIC

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    CANARY_QUESTION_ID: { type: 'boolean', instructions: 'CANARY_RUBRIC' },
  },
});

取自 packages/api/src/test-support.ts:8

  1. refund是/否

    Does the customer request a refund?

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    refund: { type: 'boolean', instructions: 'Does the customer request a refund?' },
  },
});

同类的其他项目