vercel/ eve
The Open Framework for Building Agents
Run by mupt-ai · Released 2026-10-02 · 38 tasks · 10 settings
Model Tradeoffs
Which coding agent works best on vercel/eve? Most accurate: GPT-6.1 Sol, 73.7% at $0.43 per task. Best for less: GPT-6 Luna, 50% at $0.040. 10 model settings scored on 38 tasks from its merged pull requests.
All Settings
| Model | Harness | Reasoning | Access | Accuracy | Cost / Task | Frontier |
|---|---|---|---|---|---|---|
| GPT-6.1 Sol | Codex | Low | API Key | 73.7% | $0.43 | Yes |
| Claude Opus 5.5 | Claude Code | High | API Key | 73.7% | $2.16 | No |
| GPT-6.1 Sol | Codex | High | API Key | 71.1% | $0.85 | No |
| Claude Opus 5.5 | Claude Code | Low | API Key | 71.1% | $0.85 | No |
| Kimi K3 | Pi | High | Vercel AI Gateway | 65.8% | $2.65 | No |
| DeepSeek V4.1 Flash | Pi | High | Vercel AI Gateway | 60.5% | $0.43 | No |
| GLM 5.3 | Pi | High | Vercel AI Gateway | 60.5% | $2.88 | No |
| GPT-6 Luna | Codex | High | API Key | 50% | $0.040 | Yes |
| GLM 5.3 Flash | Pi | High | Vercel AI Gateway | 44.7% | $0.30 | No |
| GPT-6 Luna | Codex | Low | API Key | 34.2% | $0.062 | No |
Tasks
All 38 tasks, each with its instruction, tests, and solution to browse and download.