vercel/eve

The Open Framework for Building Agents

Run by mupt-ai · Released 2026-10-02 · 38 tasks · 10 settings

Model Tradeoffs

Which coding agent works best on vercel/eve? Most accurate: GPT-6.1 Sol, 73.7% at $0.43 per task. Best for less: GPT-6 Luna, 50% at $0.040. 10 model settings scored on 38 tasks from its merged pull requests.

All Settings

ModelHarnessReasoningAccessAccuracyCost / TaskFrontier
GPT-6.1 SolCodexLowAPI Key73.7%$0.43Yes
Claude Opus 5.5Claude CodeHighAPI Key73.7%$2.16No
GPT-6.1 SolCodexHighAPI Key71.1%$0.85No
Claude Opus 5.5Claude CodeLowAPI Key71.1%$0.85No
Kimi K3PiHighVercel AI Gateway65.8%$2.65No
DeepSeek V4.1 FlashPiHighVercel AI Gateway60.5%$0.43No
GLM 5.3PiHighVercel AI Gateway60.5%$2.88No
GPT-6 LunaCodexHighAPI Key50%$0.040Yes
GLM 5.3 FlashPiHighVercel AI Gateway44.7%$0.30No
GPT-6 LunaCodexLowAPI Key34.2%$0.062No

Tasks

All 38 tasks, each with its instruction, tests, and solution to browse and download.

All Repositories on SelfBench