Leaderboard, frontier, heatmap, run detail
Every surface is built to be read at a glance and then drilled into. Here's what each one shows and how to interpret it.
Leaderboard columns
| Column | Meaning |
|---|---|
| # | Rank among qualifiers; top 3 get gold/silver/bronze podium badges |
| Participant | Verified GitHub identity (avatar + link) |
| Passed | Tasks solved on the ranked run |
| Billed cost | Real dollars from the ledger — the score |
$/passed | Cost ÷ tasks solved — efficiency per solved task |
Below it, a muted “Below the bar” table shows entries that ran but missed the eligibility bar, with a +N of how many more passes they need.
The cost-vs-pass frontier
Each entry is a dot: x = tasks passed, y = billed dollars. Read it like a shopping chart — you want down (cheap) and right (more passes).
billed $
▲
$10│ ◆ baseline (~$9.8, 10 passed — the bar)
│ ● defunkt (14 passed, costly)
$8│
│ ● mona (13 passed)
$5│ ○───────○ efficient frontier (dashed)
│ ● pjhyett (11) ← "best value": cheapest that still qualifies
$2│ ● (red = below the bar, 9 passed)
│ (down-and-right of the line = a better deal)
$0└────────────────────────────────────▶ tasks passed
8 10 12 14 16
| Mark | Meaning |
|---|---|
| ◆ diamond | The baseline (GLM 5.2) |
| ● green dot | Qualified entry (met the bar) |
| ● red dot | Below the bar |
| ○ dashed line | Efficient frontier — cheapest harness at each pass count |
Analytics: the heatmap
Tasks (rows, hardest first) against submissions (columns, best first). Green = passed, red = failed, blank = not run. A red row across most columns is a task the whole field is stuck on; the hardest-tasks digest above it lists the lowest solve-rate tasks.
Run detail
Click any entry to open its run page, which hides nothing:
- Pass/fail summary bar — solved fraction at a glance.
- Per-task bar chart — cost per task, green/red by pass; click a bar for model + tokens.
- Where the compute went — per-model tasks / tokens / cost.
- The harness — the actual instruction + skill files, so others can learn from it.
See it in action from the leaderboard — click any row.