jev-align
jev-align is an experimental CLI from Sutro for building
AI Functions with TypeSafe's Jev.
It finds uncertain examples, asks you to label them, and uses
GEPA to improve the function. Use it in your
application, keep learning from production examples, and share portable
functions through ai-functions.dev.
Demo
https://github.com/user-attachments/assets/81650587-e3f1-4655-8213-ed5f6e120e9a
Quick start
Requires Python 3.11 or newer.
uv tool install jev-align
export TYPESAFE_API_KEY="..." # Or use Vercel or Cloudflare below
export OPENAI_API_KEY="..." # or ANTHROPIC_API_KEY / GEMINI_API_KEY
jeva
Start the CLI with either jeva or jev-align.
Use pip install jev-align if you do not use
uv. The guided setup discovers local CSV,
Parquet, and JSONL files and includes three ready-to-run examples.
How it works
Each round:
- Evaluates the configured dataset and measures uncertainty.
- Selects ambiguous rows plus a random audit sample for you to label.
- Uses your accumulated labels and optional rationales to run GEPA.
- Shows the score, certainty change, and proposed definition diff.
- Lets you accept, reject, rewind, or resume later.
The guided workflow asks you to review every label. Labels may be human-created,
synthetic, imported, or agent-assisted; review them to the standard your use
case requires. A higher training score never accepts a proposal automatically.
Task types
| Type |
Output |
| Binary |
True or False |
| Multiclass |
Exactly one fixed label |
| Multilabel |
Zero or more fixed labels |
| Score |
One level from an ordered rubric |
Configuration
The guided Advanced menu configures:
- 5, 10, 15, or 20 training annotations per round.
- An optional 20% held-out evaluation set.
- GEPA's metric-call budget, which defaults to 300.
By default, jev-align uses the first 1,000 rows—or the entire dataset when it
is smaller—and lets you concatenate all fields or select specific columns.
Everything can also be configured with flags:
jeva optimize posts.csv \
--question "Is the post related to aviation?" \
--column title \
--column text \
--pool-size 1000
Use repeated --class "NAME=DESCRIPTION" options for multiclass or multilabel
tasks, and repeated --score-level options for scoring tasks. Run
jeva optimize --help for the complete flag reference.
Jev providers
Jev can run directly through TypeSafe AI, Vercel AI Gateway, or Cloudflare
Workers AI. The guided setup detects configured providers and lets you choose.
# Vercel AI Gateway
export AI_GATEWAY_API_KEY="..."
jeva optimize data.csv --question "Is this relevant?" --column text \
--backend vercel
# Cloudflare Workers AI
export CLOUDFLARE_ACCOUNT_ID="..."
export CLOUDFLARE_API_TOKEN="..."
jeva optimize data.csv --question "Is this relevant?" --column text \
--backend cloudflare
These routes do not require a TYPESAFE_API_KEY. The chosen provider is saved
with the AI Function, so later runtime calls use the same provider. GEPA's
reflection model is configured separately.
Reflection models
GEPA's reflection model is separate from the JEV model evaluating your data.
OpenAI, Anthropic, and Gemini models are detected automatically. Any
LiteLLM provider—including Fireworks,
local vLLM, and other OpenAI-compatible endpoints—can be supplied with
--reflection-model provider/model.
export HOSTED_VLLM_API_BASE="http://localhost:8000/v1"
jeva optimize data.csv --question "Is this relevant?" --column text \
--reflection-model "hosted_vllm/Qwen/Qwen3-8B"
Controls
- Arrow keys and Enter navigate menus.
b returns to the previous label; /back leaves the rationale prompt.- Space toggles choices in multilabel tasks.
Saved AI Functions
Browse, run, or resume functions saved on your computer:
jeva functions
jeva optimize --resume .jev-align/runs/<run-id>
Share on ai-functions.dev
ai-functions.dev is the public registry for AI
Functions built with jev-align. Browse shared functions on the website and
publish your own from the CLI.
To publish, open your saved functions:
jeva functions
Select a function, then choose Push to ai-functions.dev. If needed,
jev-align walks you through GitHub login. You can name the function, add a
description, review what will be public, and confirm the push. It will appear at
ai-functions.dev/<github-user>/<function-name>.
A published function includes everything another person needs to run it or
continue improving it:
- The accepted definition and input signature.
- The runtime backend and learning configuration.
- Labeled inputs, labels, splits, and optional rationales.
It does not contain unlabeled source rows, local dataset paths, API keys,
reflection-provider credentials, or an unaccepted proposal. The registry is
currently public, so do not publish labeled data you cannot share.
Human review is encouraged for important tasks, but the registry does not claim
or verify that every published annotation was created or reviewed by a person.
Anyone can pull a public function without signing in:
jeva pull github-user/function-name
jeva pull github-user/function-name --version 2
Pulling needs no login. It verifies the immutable artifact and creates a normal
saved AI Function under .jev-align/runs/, ready to run or continue improving.
Owners can remove a function from discovery and future public pulls:
jeva unpublish github-user/function-name
Unpublishing is reversible: versions and annotations are retained, and pushing
the function again restores it. It cannot revoke copies that were already
downloaded.
Keep learning from production
Load an AI Function in your application and capture useful production examples:
from jev_align import AIFunction
is_aviation = AIFunction.load(
".jev-align/runs/<run-id>",
capture=True,
)
prediction = is_aviation(
title="Airport expansion",
text="A new runway opens next year.",
)
Later, resume the AI Function and label the captured examples. GEPA uses that
feedback to propose the next version:
jeva functions
Using a coding agent
See AGENTS.md for detailed setup, provider configuration, CLI
operation, and development guidance for coding agents.
Affiliation
Sutro is not affiliated with TypeSafe AI, the makers of Jev.
这是 Sutro 推出的一个实验性命令行工具,用于在 TypeSafe 的 Jev 上构建经过校准的 AI Function:它会找出数据集中不确定的样本让你来标注,并通过 GEPA 持续改进函数。