Back to benchmarks & research
pytest-jev project preview
Benchmarks & research

pytest-jev

Semantic assertions for pytest: test what your LLM app's output means, judged by TypeSafe's Jev.

Explore repositoryVisit website

Pytest plugin for semantic assertions on LLM output using Jev's calibrated probabilities.

How it uses Jev

Semantic assertions for pytest, judged by TypeSafe's Jev, a model that returns calibrated probabilities instead of text.

Language
Python 100%
License
MIT
Latest release
pytest-jev 0.1.0
Last activity
Sep 2026
Added
2026-09-22
Stars at snapshot
2
Status
Community listing
jevllmllm-evaluationpytestpytest-pluginpython

Source: community catalog. Details and metrics reflect the published community snapshot and are not a performance endorsement. Documentation details extracted from GitHub · the README · the project website.

Listed in Jev Library

If this is your project, the badge below links back to this page. Paste it near the top of your README.

[![Listed in Jev Library](https://jevlibrary.dev/api/badge/allebee-pytest-jev)](https://jevlibrary.dev/projects/allebee-pytest-jev)

Something wrong on this page, or would you rather not be listed? Tell us and it will be changed.

  • jev-eval-agent project preview
    Benchmarks & research

    jev-eval-agent

    Personal-assistant agent built on Vercel's eve with 100 mocked tools, measuring how many steps it takes when Jev picks the tool versus the LLM.

    HTML 105Details
  • typesafe-ai-benchmark project preview
    Benchmarks & research

    typesafe-ai-benchmark

    This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.

    TypeScript 38Details
  • jev-on-a-laptop project preview
    Benchmarks & research

    jev-on-a-laptop

    Unofficial study: Jev-style parallel typed decisions on stock 1.5B-8B models on an Apple Silicon laptop. Benchmarks, research notes, and a Hugging Face Space demo.

    Python 23Details
  • jev-benchmarks project preview
    Benchmarks & research

    jev-benchmarks

    Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

    Python 17Details
  • jev-lm project preview
    Benchmarks & research

    jev-lm

    A word-level language model whose output layer is Jev: n-gram drafter, Noul chunk verification, bits-per-token eval

    TypeScript 6Details
  • decider project preview
    Benchmarks & research

    decider

    One-pass typed decisions with calibrated probabilities (System One style model), fine-tuned from Qwen3.5-2B

    Python 351Details