Back to benchmarks & research
jeval project preview
Benchmarks & research

jeval

Measures what your Jev classifier's confidence is really worth, and sets the human hand-off line from what a mistake costs.

Explore repository

How it uses Jev

Command What it does Status --- --- --- jeval init create .jeval/ config and the ingest map implemented jeval ingest JSONL/CSV logs to decision records; --preset jev-native reads a decision API's own response log; --labels brings in human answers implemented jeval report the report itself: verdict, reliability, cost, impact, segments, score questions, labels and correction, drift, data quality — one HTML file, or --…

Getting started

Nobody wants to learn nine commands. Hand an agent one sentence and take the report back:

Languages
Python 98% · Shell 2%
License
Apache-2.0
Latest release
v0.1.8
Last activity
Sep 2026
Added
2026-09-23
Stars at snapshot
16
Status
Community listing
calibrationclassifiercliconfidenceecehuman-in-the-loop

Source: community catalog. Details and metrics reflect the published community snapshot and are not a performance endorsement. Documentation details extracted from GitHub · the README.

Listed in Jev Library

If this is your project, the badge below links back to this page. Paste it near the top of your README.

[![Listed in Jev Library](https://jevlibrary.dev/api/badge/rlaope-jeval)](https://jevlibrary.dev/projects/rlaope-jeval)

Something wrong on this page, or would you rather not be listed? Tell us and it will be changed.

  • jev-eval-agent project preview
    Benchmarks & research

    jev-eval-agent

    Personal-assistant agent built on Vercel's eve with 100 mocked tools, measuring how many steps it takes when Jev picks the tool versus the LLM.

    HTML 105Details
  • typesafe-ai-benchmark project preview
    Benchmarks & research

    typesafe-ai-benchmark

    This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.

    TypeScript 38Details
  • jev-on-a-laptop project preview
    Benchmarks & research

    jev-on-a-laptop

    Unofficial study: Jev-style parallel typed decisions on stock 1.5B-8B models on an Apple Silicon laptop. Benchmarks, research notes, and a Hugging Face Space demo.

    Python 23Details
  • jev-benchmarks project preview
    Benchmarks & research

    jev-benchmarks

    Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

    Python 17Details
  • jev-lm project preview
    Benchmarks & research

    jev-lm

    A word-level language model whose output layer is Jev: n-gram drafter, Noul chunk verification, bits-per-token eval

    TypeScript 6Details
  • decider project preview
    Benchmarks & research

    decider

    One-pass typed decisions with calibrated probabilities (System One style model), fine-tuned from Qwen3.5-2B

    Python 351Details