Benchmarks & research

Jevs-Garage

@JGalegoโ˜… 1Pythonupdated 2026-09-19โ–ถ runnable

A garage full of tiny experiments for building critical systems with System One & Jev ๐Ÿ”ง๐Ÿง โšก

JGalego/Jevs-Garage โ†—

Where it calls Jev

from typesafe_sdk import (

src/jevs_garage/gallery.py:24

The link points at the commit we read, so the line number still holds.

What it asks Jev

These question sets are lifted from this project's source exactly as written.

from berserk/global-payment-incident/demo.py:54

  1. failure_domainchoice

    Identify the dominant failure domain supported by the complete incident snapshot.

    • application_release โ€” The active application release is the primary causal factor.
    • payment_provider โ€” An external payment provider is the primary causal factor.
    • routing_configuration โ€” Routing or timeout configuration is the primary causal factor.
    • capacity โ€” Internal capacity exhaustion is the primary causal factor.
    • multi_factor โ€” Several coupled causes prevent a single dominant attribution.
    • unknown โ€” Evidence is insufficient for stable attribution.
  2. customer_impactscore

    Score current customer and financial impact.

    • 0 โ€” contained
    • 1 โ€” minor
    • 2 โ€” material
    • 3 โ€” major
    • 4 โ€” systemic
  3. systemic_outagetrue/false

    Is this a multi-region, customer-visible payment outage requiring incident command?

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    failure_domain: { type: 'choice', instructions: 'Identify the dominant failure domain supported by the complete incident snapshot.',
      criteria: { application_release: 'The active application release is the primary causal factor.', payment_provider: 'An external payment provider is the primary causal factor.', routing_configuration: 'Routing or timeout configuration is the primary causal factor.', capacity: 'Internal capacity exhaustion is the primary causal factor.', multi_factor: 'Several coupled causes prevent a single dominant attribution.', unknown: 'Evidence is insufficient for stable attribution.' } },
    customer_impact: { type: 'score', instructions: 'Score current customer and financial impact.',
      criteria: ['contained', 'minor', 'material', 'major', 'systemic'] },
    systemic_outage: { type: 'boolean', instructions: 'Is this a multi-region, customer-visible payment outage requiring incident command?' },
  },
});

from berserk/global-payment-incident/demo.py:86

  1. change_riskscore

    Score the blast radius and reversibility risk of the selected intervention.

    • 0 โ€” minimal
    • 1 โ€” low
    • 2 โ€” moderate
    • 3 โ€” high
    • 4 โ€” unacceptable
  2. evidence_sufficienttrue/false

    Is the evidence sufficient for operators to consider the selected intervention now?

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    change_risk: { type: 'score', instructions: 'Score the blast radius and reversibility risk of the selected intervention.',
      criteria: ['minimal', 'low', 'moderate', 'high', 'unacceptable'] },
    evidence_sufficient: { type: 'boolean', instructions: 'Is the evidence sufficient for operators to consider the selected intervention now?' },
  },
});

from berserk/infinity-of-jevs/demo.py:58

  1. positionchoice

    Choose the seismic hazard interpretation best supported by the source and prior consensus.

    • background โ€” Activity is consistent with ordinary global background seismicity.
    • elevated_cluster โ€” A meaningful cluster or sequence warrants focused monitoring.
    • major_event โ€” A major event creates credible disruption potential.
    • unclear โ€” The evidence cannot support a stable hazard interpretation.
  2. strengthscore

    Score the operational strength of the seismic hazard signal.

    • 0 โ€” negligible
    • 1 โ€” weak
    • 2 โ€” moderate
    • 3 โ€” strong
    • 4 โ€” extreme
  3. supportedtrue/false

    Is this hazard interpretation directly supported by the supplied events?

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    position: { type: 'choice', instructions: 'Choose the seismic hazard interpretation best supported by the source and prior consensus.',
      criteria: { background: 'Activity is consistent with ordinary global background seismicity.', elevated_cluster: 'A meaningful cluster or sequence warrants focused monitoring.', major_event: 'A major event creates credible disruption potential.', unclear: 'The evidence cannot support a stable hazard interpretation.' } },
    strength: { type: 'score', instructions: 'Score the operational strength of the seismic hazard signal.',
      criteria: ['negligible', 'weak', 'moderate', 'strong', 'extreme'] },
    supported: { type: 'boolean', instructions: 'Is this hazard interpretation directly supported by the supplied events?' },
  },
});

from berserk/infinity-of-jevs/demo.py:77

  1. positionchoice

    Choose the plausible exposure pattern after considering hazard and prior-round evidence.

    • remote โ€” Events are unlikely to disrupt significant infrastructure or population centers.
    • regional โ€” One region could face meaningful disruption.
    • multi_region โ€” Several regions or global networks could be affected.
    • unclear โ€” Locations and impact context do not support a stable exposure judgment.
  2. strengthscore

    Score plausible infrastructure and population exposure.

    • 0 โ€” negligible
    • 1 โ€” low
    • 2 โ€” moderate
    • 3 โ€” high
    • 4 โ€” extreme
  3. supportedtrue/false

    Is this exposure judgment supported without inventing missing asset data?

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    position: { type: 'choice', instructions: 'Choose the plausible exposure pattern after considering hazard and prior-round evidence.',
      criteria: { remote: 'Events are unlikely to disrupt significant infrastructure or population centers.', regional: 'One region could face meaningful disruption.', multi_region: 'Several regions or global networks could be affected.', unclear: 'Locations and impact context do not support a stable exposure judgment.' } },
    strength: { type: 'score', instructions: 'Score plausible infrastructure and population exposure.',
      criteria: ['negligible', 'low', 'moderate', 'high', 'extreme'] },
    supported: { type: 'boolean', instructions: 'Is this exposure judgment supported without inventing missing asset data?' },
  },
});

Other projects in this category