LLMcompare

Search

Search for a command to run...

Coding

SWE-Rebench

Contamination-resistant software engineering: rolling window of fresh GitHub issues under a fixed ReAct agent scaffold; scores are tied to the published task window.

How models are tested

Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.

Category
Coding
Metric
Percent
Direction
Higher is better
Coverage
64 / 318Models in this catalog with a published score
Updated
Catalog snapshot date
Source
swe-rebench.com/

Scores

  1. 1
    Claude Fable 5

    Anthropicclosed

    64.5%
  2. 2
    Grok 4.5

    xAIclosed

    63.8%
  3. 3
    Claude Opus 5

    Anthropicclosed

    63.4%
  4. 4
    GPT-5.5

    OpenAIclosed

    62.7%
  5. 5
    GPT-5.6 Sol

    OpenAIclosed

    62.3%
  6. 6
    Qwen3.5 397B-A17B

    Alibabaopen

    59.9%
  7. 7
    Step-3.5 Flash

    StepFunclosed

    59.6%
  8. 8
    GPT-5.4

    OpenAIclosed

    57.6%
  9. 9
    GLM-5.2

    Zhipu AIopen

    57.0%
  10. 10
    Claude Sonnet 5

    Anthropicclosed

    56.8%
  11. 11
    Claude Opus 4.8

    Anthropicclosed

    56.5%
  12. 12
    Gemini 3.1 Pro

    Googleclosed

    54.9%
  13. 13
    Claude Sonnet 4.6

    Anthropicclosed

    54.5%
  14. 14
    GLM-5

    Zhipu AIopen

    53.3%
  15. 15
    Claude Opus 4.6

    Anthropicclosed

    53.3%
  16. 16
    Claude Opus 4.5

    Anthropicclosed

    53.1%
  17. 17
    Claude Opus 4.7

    Anthropicclosed

    53.1%
  18. 18
    Claude Sonnet 4.5

    Anthropicclosed

    52.4%
  19. 19
    MiniMax M2.7

    MiniMaxclosed

    51.9%
  20. 20
    Gemini 3 Pro

    Googleclosed

    50.7%
  21. 21
    Gemini 3.5 Flash

    Googleclosed

    49.5%
  22. 22
    Kimi K2.5

    Moonshotopen

    49.0%
  23. 23
    DeepSeek V3.2

    DeepSeekopen

    48.9%
  24. 24
    MiniMax M2.5

    MiniMaxopen

    47.7%
  25. 25
    Kimi K2.6

    Moonshotclosed

    46.5%
  26. 26
    MiniMax M3

    MiniMaxopen

    46.4%
  27. 27
    GLM-4.7

    Zhipu AIopen

    45.5%
  28. 28
    Claude Sonnet 4

    Anthropicclosed

    44.6%
  29. 29
    MiMo-V2.5-Pro

    Xiaomiopen

    44.4%
  30. 30
    GPT-5.6 Luna

    OpenAIclosed

    43.6%
  31. 31
    Claude Opus 4.1

    Anthropicclosed

    43.5%
  32. 32
    Devstral 2

    Mistralopen

    42.0%
  33. 33
    DeepSeek V4 Pro

    DeepSeekopen

    41.4%
  34. 34
    DeepSeek V4 Flash

    DeepSeekopen

    38.4%
  35. 35
    Grok 4

    xAIclosed

    38.3%
  36. 36
    o3

    OpenAIclosed

    38.1%
  37. 37
    Qwen3 Coder 480B

    Alibabaopen

    36.3%
  38. 38
    Kimi K2

    Moonshotopen

    34.6%
  39. 39
    Claude 3.5 Sonnet

    Anthropicclosed

    30.3%
  40. 40
    GPT-4.1

    OpenAIclosed

    30.1%
  41. 41
    GLM-4.7 Flash

    Zhipu AIclosed

    30.1%
  42. 42
    DeepSeek V3.1

    DeepSeekopen

    28.0%
  43. 43
    o4-mini

    OpenAIclosed

    27.5%
  44. 44
    gpt-oss-120B

    OpenAIopen

    27.3%
  45. 45
    GPT-4.1 Mini

    OpenAIclosed

    25.5%
  46. 46
    Gemma 4 31B

    Googleopen

    25.1%
  47. 47
    Gemini 2.5 Pro

    Googleclosed

    23.5%
  48. 48
    DeepSeek V3

    DeepSeekopen

    23.3%
  49. 49
    DeepSeek R1-0528

    DeepSeekopen

    22.3%
  50. 50
    Qwen3-Coder 30B-A3B

    Alibabaopen

    21.6%
  51. 51
    Qwen3-Next 80B-A3B

    Alibabaopen

    20.1%
  52. 52
    Gemini 2.5 Flash

    Googleclosed

    19.9%
  53. 53
    Qwen3 235B-A22B

    Alibabaopen

    18.5%
  54. 54
    Devstral Small

    Mistralopen

    14.6%
  55. 55
    Llama 3.3 70B

    Metaopen

    11.5%
  56. 56
    Llama 4 Maverick

    Metaopen

    11.0%
  57. 57
    Qwen2.5 72B

    Alibabaopen

    10.1%
  58. 58
    Gemini 2.0 Flash

    Googleclosed

    9.1%
  59. 59
    Qwen3 32B

    Alibabaopen

    8.1%
  60. 60
    gpt-oss-20B

    OpenAIopen

    8.1%
  61. 61
    Llama 4 Scout

    Metaopen

    6.6%
  62. 62
    Gemma 3 27B

    Googleopen

    5.4%
  63. 63
    Qwen2.5-Coder 32B

    Alibabaopen

    3.1%
  64. 64
    GPT-4.1 Nano

    OpenAIclosed

    0.3%

Back to all benchmarks