earendil-works/
AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI
Run by mupt-ai · Released 2026-11-03 · 43 tasks · 18 settings
Model Tradeoffs
Which coding agent works best on earendil-works/pi? Most accurate: GPT-6.1 Sol, 88.4% at $0.53 per task. Best for less: Dari Router, 86.0% at $0.40. 18 model settings scored on 43 tasks from its merged pull requests.
All Settings
| Model | Harness | Reasoning | Access | Accuracy | Cost / Task | Frontier |
|---|---|---|---|---|---|---|
| GPT-6.1 Sol | Pi | High | API Key | 88.4% | $0.53 | Yes |
| Dari Router | Pi | Default | API Key | 86.0% | $0.40 | Yes |
| shift-1 | Codex | Default | Custom Endpoint | 86.0% | $2.74 | No |
| GPT-6.1 Sol | Codex | Low | API Key | 83.7% | $0.36 | Yes |
| GPT-6.1 Sol | Codex | High | API Key | 83.7% | $0.55 | No |
| Jev Router | Pi | Default | OpenRouter | 83.7% | $0.80 | No |
| GPT-6 Sol | Codex | High | API Key | 83.7% | $1.02 | No |
| Claude Opus 5.5 | Claude Code | High | API Key | 83.7% | $1.44 | No |
| Mistral Large 4 | Pi | High | Vercel AI Gateway | 83.7% | $1.48 | No |
| switchyard | Codex | Default | Custom Endpoint | 81.4% | $0.55 | No |
| Claude Haiku 5.5 | Claude Code | Low | API Key | 79.1% | $0.13 | Yes |
| Claude Opus 5.5 | Claude Code | Low | API Key | 79.1% | $0.61 | No |
| GLM 5.3 | Pi | High | Vercel AI Gateway | 79.1% | $1.32 | No |
| GPT-6 Sol | Codex | Low | API Key | 76.7% | $0.40 | No |
| Kimi K3 | Pi | High | Vercel AI Gateway | 76.7% | $1.29 | No |
| Claude Haiku 5.5 | Claude Code | High | API Key | 74.4% | $0.34 | No |
| GPT-6 Luna | Codex | High | API Key | 65.1% | $0.028 | Yes |
| GPT-6 Luna | Codex | Low | API Key | 55.8% | $0.024 | Yes |
Tasks
All 43 tasks, each with its instruction, tests, and solution to browse and download.