# openlayer-ai/jevals

> A library for evaluating and guarding AI agents: instead of an LLM judge it uses typed questions to Jev/Kev/Laya models, returning calibrated probabilities in a single request. Lets you check every trace quickly and cheaply, including inside the agent loop.

- Magnitude: 4.1 out of 10 — Early signal
- Stars: 51 total · +0 stars today, ≈ 22 by evening
- Star trust: star growth looks organic
- Category: AI agents · Language: Python · License: MIT · Created: 2026-09-20 · Last push: 2026-09-22
- GitHub: https://github.com/openlayer-ai/jevals · Homepage: https://github.com/openlayer-ai/jevals · Page: https://gitnova.dev/en/r/openlayer-ai/jevals

## Useful for

- Run Grounded, AnswerRelevancy and PHI metrics on an agent trace in one request
- Add a gate inside the agent loop to block out-of-scope answers or PHI leaks
- Compare the agent's tool choice against the expected one via ToolChoice and UsedToolResult

## Why it’s here

- 0 stars so far today, about 22 expected by the end of the day.
- The spike has held for 3 days in a row — not a one-off blip.
- The repository is 2 days old and already has 51 stars. With less than two weeks of history, there's no usual pace to compare the spike against yet.
- Hacker News: “Show HN: jevals – replacing LLM judges with typed Jev decisions” — 42 points, 1 day ago.

## Star trust

Star growth looks organic.

Star-trust labels are heuristics based on the repository’s behavior, not a check of every stargazer.

## Numbers

- Forks: 2
- Issues and pull requests: 0
- Watchers: 0
- Average over the last week: 24 per day
- Usual pace: too little history (under two weeks)
- Stars in the last hour (measured): 2

## Stars per day, last 3 days (oldest → newest, today is partial)

2026-09-20 … 2026-09-22: 8, 43, 0

## Magnitude by day, last 2 days

09-21 3.7, 09-22 4.3

## Hacker News

- Show HN: jevals – replacing LLM judges with typed Jev decisions — 42 points, 2 comments: https://news.ycombinator.com/item?id=49780849

## Spotted in now

- Spotted on Hacker News

## Similar by description

1. **mizorewww/laya-mlx** — 9.0 · Breakout · Language models · Python · +0 stars today, ≈ 1,188 by evening
   Native MLX runtime for Laya typed decision models: returns probabilities for choices, scores or truth values without text generation, running locally on Apple Silicon.
   Full card: https://gitnova.dev/en/r/mizorewww/laya-mlx.md
2. **AnotiaWang/awesome-jev** — 4.9 · Early signal · Learning & lists · +0 stars today, ≈ 67 by evening
   A curated list of applications, libraries, and resources for Jev, TypeSafe's System One model that evaluates typed questions against a state and returns structured answers with probabilities.
   Full card: https://gitnova.dev/en/r/AnotiaWang/awesome-jev.md
3. **itsmostafa/typesafe-mcp** — 4.0 · Early signal · AI agents · Go · +0 stars today, ≈ 38 by evening
   A Go MCP server that gives AI agents access to TypeSafe's Jev model: instead of parsing prose, the agent gets typed answers with probabilities it can branch on. One-command setup for Claude Code, Claude Desktop, Codex, and pi.
   Full card: https://gitnova.dev/en/r/itsmostafa/typesafe-mcp.md

---

Magnitude (0–10) measures how fast and how unusually interest in a repository is growing right now. It is not a quality score.
Days are UTC. “So far today” is a fact; “expected by the end of the day” is a forecast.
Summaries and use cases are written by an LLM (DeepSeek V4.1 Flash) from the README and may be inaccurate: verify specific claims (benchmarks, speed, hardware) in the repository itself.
Data as of 2026-09-22 06:39 UTC, updated every 30 minutes.