jev-benchmarks
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
How it uses Jev
jev-benchmarks measures more than whether a model selects the right label. It evaluates whether the reported probabilities are calibrated enough to support automation, how much work can be accepted at a fixed error budget, what resources each decision uses, and how long it takes end to end.
Getting started
Requirements: Python 3.11–3.13 and uv.
Source: community catalog. Details and metrics reflect the published community snapshot and are not a performance endorsement. Documentation details extracted from GitHub · the README · the project website.