mupt-ai/ self-bench
Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
Run by mupt-ai · Released 2026-10-17 · 38 tasks · 7 settings
Model Tradeoffs
Which coding agent works best on mupt-ai/self-bench? Most accurate: GPT-6.1 Sol, 76.3% at $0.43 per task. Best for less: DeepSeek V4.1 Flash 1017, 71.1% at $0.13. 7 model settings scored on 38 tasks from its merged pull requests.
All Settings
| Model | Harness | Reasoning | Access | Accuracy | Cost / Task | Time / Task | Frontier |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Sol | Codex | High | API Key | 76.3% | $0.43 | Yes | |
| DeepSeek V4.1 Flash 1017 | Pi | High | Vercel AI Gateway | 71.1% | $0.13 | Yes | |
| Claude Opus 5.5 | Claude Code | High | API Key | 71.1% | $1.56 | No | |
| GPT-6.1 Sol | Codex | Low | API Key | 65.8% | $0.21 | No | |
| Kimi K3 | Codex | High | Vercel AI Gateway | 63.2% | $1.55 | No | |
| GLM 5.3 | Codex | High | Vercel AI Gateway | 52.6% | $1.33 | No | |
| GPT-6 Luna | Codex | High | API Key | 47.4% | $0.027 | Yes |
Tasks
All 38 tasks, each with its instruction, tests, and solution to browse and download.