jev README

The project's README from github.com/feder-cr/jev. As one markdown file: README.md.

jevos-1b

Yes/no decisions on a laptop CPU in 50–220 ms. Send a text and a yes/no question, get back P(yes).

jevos-1b playing a Chrome Dino-style game on the CPU, answering two yes/no questions per step (recording at 2× speed)

Benchmarks

Latency on a short request: jevos-1b 54 ms, Jev 344 ms, Laya 104 ms. On a long request: jevos-1b 220 ms, Jev 345 ms, Laya 449 ms. Accuracy on 2,000 yes/no questions from unseen policies: jevos-1b 0.815, Jev 0.927, Laya 0.489

What each one does

jevos-1b Jev Laya
Yes/no questions ✓ ✓ ✓
Multiple choice Soon ✓ ✓
Scores Soon ✓ ✓
Runs on your machine cloud your machine
Cost free per token free
Context 8,192 tokens not stated 512 tokens

Quickstart

Download jevos-1b-q4_k_m.gguf from the release, then:

uv sync
uv run jev download --only runtime        # llama.cpp for this machine
uv run jev serve --gguf jevos-1b-q4_k_m.gguf --device cpu --threads 16

curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "jev-latest",
  "state": "I was charged twice for the same order.",
  "questions": {"billing": {"type": "noul", "instructions": "Is this a billing problem?"}}}'

{
  "model": "jevos-1b-q4_k_m",
  "answers": {"billing": {"type": "noul", "noul": 0.9}},
  "usage": {"input_tokens": 27, "output_tokens": 0}
}

Then open http://127.0.0.1:8017/dino to watch it play.

API

The server speaks TypeSafe Jev's wire format, so code written for Jev's SDK works unchanged for yes/no questions.

POST /v1/systemone

Field What it is
model jev-latest (any jev-* name works) or the served model's name
state the text to decide on: a string, or any JSON object or array
questions one or more named questions, each {"type": "noul", "instructions": "…?"}

Every answer is noul, the probability that the answer is yes (0 to 1). Questions in the same request share the state, which is read once: the three questions below take about 165 ms together, against 103 ms for one of them alone.

{
  "model": "jev-latest",
  "state": {
    "item": "wireless mouse",
    "delivered": "5 days ago",
    "customer_message": "The box arrived empty. This is the second time!"
  },
  "questions": {
    "refund": {
      "type": "noul",
      "instructions": "Our policy refunds items reported missing within 30 days of delivery. Should this customer get a refund?"
    },
    "upset": {"type": "noul", "instructions": "Is the customer upset?"},
    "wrong_item": {"type": "noul", "instructions": "Does the customer say they received the wrong item?"}
  }
}

{
  "model": "jevos-1b-q4_k_m",
  "answers": {
    "refund": {"type": "noul", "noul": 0.78},
    "upset": {"type": "noul", "noul": 0.73},
    "wrong_item": {"type": "noul", "noul": 0.1}
  },
  "usage": {"input_tokens": 95, "output_tokens": 0}
}

Other endpoints

Endpoint Returns
GET /v1/models the served model and its jev-latest alias
GET /health {"status": "ready", …} once the model is loaded

From Python

import requests

answer = requests.post("http://127.0.0.1:8017/v1/systemone", json={
    "model": "jev-latest",
    "state": "I was charged twice for the same order.",
    "questions": {"billing": {"type": "noul", "instructions": "Is this a billing problem?"}},
}).json()

if answer["answers"]["billing"]["noul"] > 0.5:
    print("send to billing")

Server options

Option Default
--gguf path to the model file
--device auto cpu to stay on the CPU
--threads 4 CPU threads; set it to your core count, fewer if other heavy apps are running
--host, --port 127.0.0.1, 8017 where the server listens

Credits

Built together with Loris Salsi (@LosaLosSantos).

jev (feder-cr/jev)

Searching Database…

1.4K ·GitHub
Jev
这是什么？jevos-1b 是一个免费开源的是/否决策引擎，可在你笔记本的 CPU 上本地运行，对给定文本加问题，在大约 50-220 毫秒内返回答案为「是」的概率，可作为云端 Jev 服务的自托管替代品。
这是我导入的库，还是我自己运行的服务？它是一个独立的 HTTP 服务器，暴露 POST /v1/systemone 端点，使用与 Jev SDK 相同的传输格式，因此现有的 Jev 客户端代码无需修改即可直接对接。
怎么把它跑起来？从项目的 GitHub release 下载 jevos-1b-q4km.gguf 模型文件，然后启动服务器并用 --gguf 指向该文件。
除了是/否题，它能回答多选题或打分题吗？还不能——目前只有是/否（noul）题可用；发送选择题或打分题会被 422 错误拒绝，不过这两类在路线图上都标记为「即将推出」。
我的文本会被发送到云服务器吗？不会——与运行在云端的 Jev 不同，jevos-1b（和 Laya 一样）完全在你自己的机器上运行，什么都不会离开你的电脑。
和真正的 Jev 服务相比，jevos-1b 表现如何？它的是/否题表现与 Jev 相当，而且免费、可离线，不像 Jev 那样按 token 计费且托管在云端，但它还不支持 Jev 已提供的多选题和打分题。
底层实际运行的是什么模型？它是一个基于 MiniCPM 的模型，以量化的 GGUF 文件（jevos-1b-q4km.gguf）形式发布，通过基于 CPU 的 llama.cpp 风格推理提供服务。
我能一次就同一段文本问好几个问题吗？可以——单个请求可以携带多个共享同一状态的具名问题，该状态只读取一次，因此三个问题一起约耗时 165 毫秒，而单个问题约 103 毫秒。
除了纯文本，「state」字段里还能放什么？state 可以是纯字符串，也可以是任意 JSON 对象或数组；模型只读取它一次，并据此回答请求中的每个问题。
What are similar projects?cardputer-ai, decider, rizzo-flow, Strata, StartLux-Decision, backburner, project-maya, llamafile
Which topics is it in?cpu-inference, text-classification, open-source-alternative, decision-model, jev, on-device-ai, llama-cpp, classification, edge-ai, gguf, local-ai, fastapi
Creator
feder-cr
Size
17.5 MB
Stars
1.4K
Updated
CommitOct 16Releasejevos-v5Oct 16
Age
2y
Commits · 90 Days