What is this?
An experimental CLI from Sutro for building calibrated AI Functions on TypeSafe's Jev: it finds uncertain examples in your dataset, has you label them, and uses GEPA to keep improving the function.
An experimental CLI from Sutro for building calibrated AI Functions on TypeSafe's Jev: it finds uncertain examples in your dataset, has you label them, and uses GEPA to keep improving the function.Is this a library I import, or a program I run?A program, not a library: an interactive CLI you start as jeva or jev-align, and the AI Functions it produces are saved locally to run or load into your application.
A program, not a library: an interactive CLI you start as jeva or jev-align, and the AI Functions it produces are saved locally to run or load into your application.How do I get it running?Install it with pip install jev-align (or via uv) on Python 3.11 or newer, then run the guided setup — it discovers local CSV, Parquet and JSONL files and ships three ready-to-run examples.
Install it with pip install jev-align (or via uv) on Python 3.11 or newer, then run the guided setup — it discovers local CSV, Parquet and JSONL files and ships three ready-to-run examples.Does Jev need a TypeSafe API key, or can it run on other providers?No TypeSafe key needed on the built-in routes: Jev can run through TypeSafe AI, Vercel AI Gateway or Cloudflare Workers AI, and the guided setup detects configured providers. GEPA's reflection model is separate — OpenAI, Anthropic and Gemini models are detected automatically, or any LiteLLM provider works.
No TypeSafe key needed on the built-in routes: Jev can run through TypeSafe AI, Vercel AI Gateway or Cloudflare Workers AI, and the guided setup detects configured providers. GEPA's reflection model is separate — OpenAI, Anthropic and Gemini models are detected automatically, or any LiteLLM provider works.Do I have to review everything myself, or can good scores be applied automatically?You stay in the loop: the guided workflow has you review every label — they may be human, synthetic, imported or agent-assisted — and a higher training score never accepts a proposal automatically. You accept, reject, rewind or resume each one.
You stay in the loop: the guided workflow has you review every label — they may be human, synthetic, imported or agent-assisted — and a higher training score never accepts a proposal automatically. You accept, reject, rewind or resume each one.What gets published if I share a function on ai-functions.dev?The definition, input signature, runtime backend, learning config, labels, splits and rationales — never unlabeled source rows, local paths or API keys. The registry is public and pulls need no login, so don't publish data you can't share; unpublishing can't revoke copies already downloaded.
The definition, input signature, runtime backend, learning config, labels, splits and rationales — never unlabeled source rows, local paths or API keys. The registry is public and pulls need no login, so don't publish data you can't share; unpublishing can't revoke copies already downloaded.Does a function keep learning after I ship it?Yes — load it in your application and capture useful production examples, then resume the function and label them; GEPA uses that feedback to propose the next version.
Yes — load it in your application and capture useful production examples, then resume the function and label them; GEPA uses that feedback to propose the next version.How much labeling does each round take?Configurable in the Advanced menu: 5, 10, 15 or 20 training annotations per round, an optional 20% held-out evaluation set, and GEPA's metric-call budget (default 300). By default only the first 1,000 rows are used.
Configurable in the Advanced menu: 5, 10, 15 or 20 training annotations per round, an optional 20% held-out evaluation set, and GEPA's metric-call budget (default 300). By default only the first 1,000 rows are used.Is this an official TypeSafe product?No — it is an experimental CLI from Sutro, which is not affiliated with TypeSafe AI, the makers of Jev. It builds on Jev and shares finished functions through ai-functions.dev.
No — it is an experimental CLI from Sutro, which is not affiliated with TypeSafe AI, the makers of Jev. It builds on Jev and shares finished functions through ai-functions.dev.What are similar projects?ParseBench, SkillOpt, awesome-jev, jevals, cleanlab, iFixAi, evals-skills, promptfoo
ParseBench, SkillOpt, awesome-jev, jevals, cleanlab, iFixAi, evals-skills, promptfooWhich topics is it in?prompt-optimization, typesafe-ai, data-labeling, jev, llm-evaluation, registry, classification, human-in-the-loop, evaluation, openai-compatible, data-science, prompt-engineering
prompt-optimization, typesafe-ai, data-labeling, jev, llm-evaluation, registry, classification, human-in-the-loop, evaluation, openai-compatible, data-science, prompt-engineering