Browser & computer use

jev-embodied-media-agent

@jzhg61TypeScriptMITupdated 2026-09-19runnable

Jev-based local-first gesture and gaze media-control agent with guarded page-close intent.

jzhg6/jev-embodied-media-agent

Where it calls Jev

const baseUrl = (process.env.TYPESAFE_BASE_URL ?? "https://api.typesafe.ai").replace(

server/jev-client.ts:33

The link points at the commit we read, so the line number still holds.

What it asks Jev

These question sets are lifted from this project's source exactly as written.

from server/decision-policy.ts:93

  1. actionchoice

    Choose exactly one media-control action from the supplied options.

    • noneNo intentional supported command is currently present.
    • pauseA stable, confident Open_Palm gesture requests that playback pause.
    • playA stable, confident Closed_Fist gesture requests that playback continue.
    • speed_upA stable, confident Thumb_Up gesture requests faster playback.
    • slow_downA stable, confident Thumb_Down gesture requests slower playback.
    • close_pageA calibrated gaze has continuously remained on the top-right close target for the full dwell period, and the safety state explicitly allows closing.
  2. close_intenttrue/false

    Is there strong evidence of an intentional page-close request? It is true only when gaze calibration exists, the continuous dwell completed, and safety.allowClose is true.

  3. intentional_controltrue/false

    Does the structured sensor state show a deliberate, stable command from the person, rather than a transient pose or ambiguous observation?

  4. signal_qualityscore

    Rate whether the current sensor evidence is reliable enough for an embodied controller to act on immediately.

    • 0Unreliable: missing, unstable, or contradictory evidence
    • 1Ambiguous: plausible evidence but meaningful uncertainty remains
    • 2Reliable: stable, confident, and internally consistent evidence

Run it on your own text

The code
import { experimental_evaluate as evaluate } from 'ai';

const { answers } = await evaluate({
  model: 'typesafe-ai/jev',
  state,
  questions: {
    action: { type: 'choice', instructions: 'Choose exactly one media-control action from the supplied options.',
      criteria: { none: 'No intentional supported command is currently present.', pause: 'A stable, confident Open_Palm gesture requests that playback pause.', play: 'A stable, confident Closed_Fist gesture requests that playback continue.', speed_up: 'A stable, confident Thumb_Up gesture requests faster playback.', slow_down: 'A stable, confident Thumb_Down gesture requests slower playback.', close_page: 'A calibrated gaze has continuously remained on the top-right close target for the full dwell period, and the safety state explicitly allows closing.' } },
    close_intent: { type: 'boolean', instructions: 'Is there strong evidence of an intentional page-close request? It is true only when gaze calibration exists, the continuous dwell completed, and safety.allowClose is true.' },
    intentional_control: { type: 'boolean', instructions: 'Does the structured sensor state show a deliberate, stable command from the person, rather than a transient pose or ambiguous observation?' },
    signal_quality: { type: 'score', instructions: 'Rate whether the current sensor evidence is reliable enough for an embodied controller to act on immediately.',
      criteria: ['Unreliable: missing, unstable, or contradictory evidence', 'Ambiguous: plausible evidence but meaningful uncertainty remains', 'Reliable: stable, confident, and internally consistent evidence'] },
  },
});

Other projects in this category