mupt-ai/self-bench

Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

Run by mupt-ai · Released 2026-10-17 · 38 tasks · 7 settings

Model Tradeoffs

Which coding agent works best on mupt-ai/self-bench? Most accurate: GPT-6.1 Sol, 76.3% at $0.43 per task. Best for less: DeepSeek V4.1 Flash 1017, 71.1% at $0.13. 7 model settings scored on 38 tasks from its merged pull requests.

All Settings

ModelHarnessReasoningAccessAccuracyCost / TaskTime / TaskFrontier
GPT-6.1 SolCodexHighAPI Key76.3%$0.43Yes
DeepSeek V4.1 Flash 1017PiHighVercel AI Gateway71.1%$0.13Yes
Claude Opus 5.5Claude CodeHighAPI Key71.1%$1.56No
GPT-6.1 SolCodexLowAPI Key65.8%$0.21No
Kimi K3CodexHighVercel AI Gateway63.2%$1.55No
GLM 5.3CodexHighVercel AI Gateway52.6%$1.33No
GPT-6 LunaCodexHighAPI Key47.4%$0.027Yes

Tasks

All 38 tasks, each with its instruction, tests, and solution to browse and download.

All Repositories on SelfBench