评测与研究

jev-labs

@copyleftdev1PythonMIT更新于 2026-09-19可当场跑

基于TypeSafe的Jev构建的TLA+验证共识内核,经过1680次混沌测试药房决策,无错误判定,包含影片、代码和所有通话记录。

英文原文

Never confidently wrong: a TLA+-verified consensus kernel around TypeSafe's Jev, run through 1,680 chaos-tested pharmacy decisions with zero wrong verdicts. Film, code, and every captured call.

copyleftdev/jev-labs

它在哪儿调用了 Jev

const DEFAULT_BASE_URL: &str = "https://api.typesafe.ai";

consensus/rust/consensus-kernel/src/jev.rs:25

链接指向我们抓取当天的那个 commit,行号是准的。

它问 Jev 的问题

下面是从这个项目源码里原样取出来的 question 组合。

取自 experiments.py:52

  1. is_regulation是/否

    Is the subject of this text a law or regulation?

  2. mentions_deadline是/否

    Does this text state a reporting deadline?

  3. mentions_fines是/否

    Does this text describe financial penalties?

  4. mentions_scope是/否

    Does this text describe a geographic scope?

  5. mentions_date是/否

    Does this text state a date it took effect?

  6. mentions_controllers是/否

    Does this text place obligations on data controllers?

  7. mentions_consent是/否

    Does this text discuss obtaining user consent?

  8. mentions_dpo是/否

    Does this text mention a Data Protection Officer?

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    is_regulation: { type: 'boolean', instructions: 'Is the subject of this text a law or regulation?' },
    mentions_deadline: { type: 'boolean', instructions: 'Does this text state a reporting deadline?' },
    mentions_fines: { type: 'boolean', instructions: 'Does this text describe financial penalties?' },
    mentions_scope: { type: 'boolean', instructions: 'Does this text describe a geographic scope?' },
    mentions_date: { type: 'boolean', instructions: 'Does this text state a date it took effect?' },
    mentions_controllers: { type: 'boolean', instructions: 'Does this text place obligations on data controllers?' },
    mentions_consent: { type: 'boolean', instructions: 'Does this text discuss obtaining user consent?' },
    mentions_dpo: { type: 'boolean', instructions: 'Does this text mention a Data Protection Officer?' },
  },
});

取自 experiments.py:249

  1. is_regulation是/否

    Is the subject of this text a law or regulation?

  2. mentions_consent是/否

    Does this text discuss obtaining user consent?

  3. severity打分

    How severe are the penalties described?

    • 0No penalties
    • 1Moderate penalties
    • 2Severe penalties
  4. topic选项

    What is the primary subject of this text?

    • privacy_lawData protection or privacy regulation
    • tax_lawTaxation rules
    • employment_lawLabour or employment rules

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    is_regulation: { type: 'boolean', instructions: 'Is the subject of this text a law or regulation?' },
    mentions_consent: { type: 'boolean', instructions: 'Does this text discuss obtaining user consent?' },
    severity: { type: 'score', instructions: 'How severe are the penalties described?',
      criteria: ['No penalties', 'Moderate penalties', 'Severe penalties'] },
    topic: { type: 'choice', instructions: 'What is the primary subject of this text?',
      criteria: { privacy_law: 'Data protection or privacy regulation', tax_law: 'Taxation rules', employment_law: 'Labour or employment rules' } },
  },
});

取自 harness/noise.py:110

  1. billing是/否

    Is this message about a billing or payment matter?

  2. technical是/否

    Does this message report a technical malfunction?

  3. urgent是/否

    Does this message convey urgency?

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    billing: { type: 'boolean', instructions: 'Is this message about a billing or payment matter?' },
    technical: { type: 'boolean', instructions: 'Does this message report a technical malfunction?' },
    urgent: { type: 'boolean', instructions: 'Does this message convey urgency?' },
  },
});

取自 smoke_test.py:36

  1. department选项

    Which team should handle this ticket?

    • billingPayment, invoicing, or subscription issues
    • technicalBugs, outages, or integration problems
    • salesPricing, upgrades, or new accounts
  2. frustration打分

    How frustrated the customer appears

    • 0Calm, just stating facts
    • 1Frustrated but civil
    • 2Very angry, strong language
  3. is_urgent是/否

    The message conveys urgency or time-sensitivity

用你自己的内容跑一遍

代码
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    department: { type: 'choice', instructions: 'Which team should handle this ticket?',
      criteria: { billing: 'Payment, invoicing, or subscription issues', technical: 'Bugs, outages, or integration problems', sales: 'Pricing, upgrades, or new accounts' } },
    frustration: { type: 'score', instructions: 'How frustrated the customer appears',
      criteria: ['Calm, just stating facts', 'Frustrated but civil', 'Very angry, strong language'] },
    is_urgent: { type: 'boolean', instructions: 'The message conveys urgency or time-sensitivity' },
  },
});

同类的其他项目