Coding
DeepSWE
DataCurve's long-horizon software-engineering benchmark of 113 original tasks across TypeScript, Go, Python, JavaScript, and Rust with program-based verifiers; scores are agent-system results under the eval harness.
How models are tested
Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.
- Category
- Coding
- Metric
- Percent
- Direction
- Higher is better
- Coverage
- 29 / 318Models in this catalog with a published score
- Updated
- Catalog snapshot date
Scores
- 1GPT-6.1 Sol75.2%
OpenAIclosed
- 2GPT-6 Astra74.1%
OpenAIclosed
- 3Claude Opus 573.7%
Anthropicclosed
- 4Gemini 3.8 Flash73.7%
Googleclosed
- 5GPT-5.6 Sol72.7%
OpenAIclosed
- 6Claude Fable 569.9%
Anthropicclosed
- 7Kimi K369.0%
Moonshotopen
- 8GPT-6 Sol68.8%
OpenAIclosed
- 9Claude Fable 5.167.4%
Anthropicclosed
- 10GPT-5.6 Luna67.0%
OpenAIclosed
- 11GPT-5.567.0%
OpenAIclosed
- 12Grok 4.667.0%
xAIclosed
- 13GLM-5.366.9%
Zhipu AIopen
- 14GPT-6 Luna66.6%
OpenAIclosed
- 15Gemini 3.7 Flash65.3%
Googleclosed
- 16Hy4 Preview64.3%
Tencentopen
- 17GLM-5.3 Flash63.0%
Zhipu AIopen
- 18DeepSeek V4 Pro 081362.7%
DeepSeekclosed
- 19Claude Opus 4.859.0%
Anthropicclosed
- 20Qwen3.8 Max56.6%
Alibabaopen
- 21Muse Spark 1.255.0%
Metaclosed
- 22DeepSeek V4 Flash 073154.4%
DeepSeekopen
- 23Claude Sonnet 554.0%
Anthropicclosed
- 24Gemini 3.6 Flash47.0%
Googleclosed
- 25GLM-5.244.0%
Zhipu AIopen
- 26Qwen3.8 27B42.2%
Alibabaopen
- 27Gemini 3.5 Flash36.0%
Googleclosed
- 28DeepSeek V4 Pro12.8%
DeepSeekopen
- 29DeepSeek V4 Flash7.3%
DeepSeekopen
