How it works
Awesome Jev is a list that judges itself. Five stages run end to end; the only place a model is called is stage three, and the only place a human decides anything is an override with a written reason.
Pipeline
- 1
Discover
Candidates come from the GitHub API: code and repository search for Jev and TypeSafe System One, plus the repositories other Jev lists already link to. Forks are dropped. Everything is appended to a candidate file, so discovery is repeatable and auditable.
- 2
Enrich
For each candidate the pipeline pulls the material a judgment needs: README excerpt, package manifests, declared dependencies, and repository metadata. Results are cached per repository so a rerun does not re-fetch what has not changed.
- 3
Judge
One Jev call per repository answers the whole question set at once: is this genuinely about Jev, does it use Jev at runtime, is it a meta list, is it a reimplementation, which category and decision pattern, and how substantial, documented, and novel it is. Raw judgments are immutable and append-only, tagged with model id and question set.
- 4
Curate
Policy lives in code, not in the model. Thresholds turn probabilities into listed, review, or excluded; weights turn the 0-3 scales into a composite score. Changing a threshold regenerates the index without re-running inference.
- 5
Publish
The curated file is the single source for both the README and this site. The site is a static export with no server logic and no API keys: every number you see here was computed at build time.
Policy in force
These are the exact values applied to the current index, read straight from the curated data generated 2026-09-17 17:12 UTC.
| Key | Value |
|---|---|
| qset | v2 |
| gate.listed_min | 0.6 |
| gate.review_min | 0.3 |
| gate.category_conf_min | 0.5 |
| gate.meta_list_min | 0.7 |
| gate.reimpl_min | 0.7 |
| weights.substance | 0.45 |
| weights.docs | 0.25 |
| weights.novelty | 0.3 |
| star_bonus_max | 0.1 |
| readme_top_per_category | 15 |
Calibration
A hand-labelled gold set is the test suite for the judgments. The sweep below moves the genuine-probability gate and reports how the resulting listing decision compares to the human labels on 31 repositories, question set v2.
| Threshold | TP | FP | FN | TN | Precision | Recall | F1 |
|---|---|---|---|---|---|---|---|
| 0.20 | 27 | 0 | 1 | 3 | 1.000 | 0.964 | 0.982 |
| 0.30 | 27 | 0 | 1 | 3 | 1.000 | 0.964 | 0.982 |
| 0.40 | 27 | 0 | 1 | 3 | 1.000 | 0.964 | 0.982 |
| 0.50 | 27 | 0 | 1 | 3 | 1.000 | 0.964 | 0.982 |
| 0.60 | 26 | 0 | 2 | 3 | 1.000 | 0.929 | 0.963 |
| 0.70 | 24 | 0 | 4 | 3 | 1.000 | 0.857 | 0.923 |
| 0.80 | 22 | 0 | 6 | 3 | 1.000 | 0.786 | 0.880 |
| 0.90 | 19 | 0 | 9 | 3 | 1.000 | 0.679 | 0.809 |
- Category agreement
- 81.5%
- Gold set size
- 31
- Question set
- v2
- Computed
- 2026-09-17 16:34 UTC
Calibration report
Placeholder — to be filled
The written report goes here: how the gold set was labelled, where Jev and the human labels disagreed and why, which threshold was chosen and what it costs, and what the confusion cases say about the question set. Nothing below this line is generated from data.
Corrections
Judgments are wrong sometimes, and the wrong ones are the interesting ones. Open an issue with the Jev got it wrong template, or read the pipeline yourself in the repository.