LLMcompare

Search

Search for a command to run...

Coding

SciCode

Scientific research coding across physics, math, chemistry, biology, and materials — domain knowledge plus precise code synthesis; benchmark version and harness must match.

How models are tested

Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.

Category
Coding
Metric
Percent
Direction
Higher is better
Coverage
183 / 318Models in this catalog with a published score
Updated
Catalog snapshot date
Source
scicode-bench.github.io/

Scores

  1. 1
    Claude Opus 5.5

    Anthropicclosed

    66.9%
  2. 2
    Claude Fable 5.1

    Anthropicclosed

    63.0%
  3. 3
    Claude Fable 5

    Anthropicclosed

    61.0%
  4. 4
    Kimi K3

    Moonshotopen

    59.5%
  5. 5
    GLM-5.3

    Zhipu AIopen

    59.0%
  6. 6
    Muse Spark 1.3

    Metaclosed

    58.8%
  7. 7
    Muse Spark 1.1

    Metaclosed

    58.8%
  8. 8
    GPT-6 Sol

    OpenAIclosed

    58.0%
  9. 9
    Muse Spark 1.2

    Metaclosed

    57.4%
  10. 10
    Gemini 3.7 Flash

    Googleclosed

    57.2%
  11. 11
    GPT-5.6 Sol

    OpenAIclosed

    57.1%
  12. 12
    Grok 4.7

    xAIclosed

    57.0%
  13. 13
    GPT-5.4

    OpenAIclosed

    56.6%
  14. 14
    Gemini 3.8 Flash

    Googleclosed

    56.6%
  15. 15
    Grok 4.6

    xAIclosed

    56.5%
  16. 16
    Claude Opus 5

    Anthropicclosed

    56.4%
  17. 17
    GPT-6 Astra

    OpenAIclosed

    56.0%
  18. 18
    GPT-5.5

    OpenAIclosed

    55.8%
  19. 19
    GPT-6 Luna

    OpenAIclosed

    55.0%
  20. 20
    GPT-5.6 Terra

    OpenAIclosed

    55.0%
  21. 21
    Grok 4.5

    xAIclosed

    55.0%
  22. 22
    Claude Opus 4.7

    Anthropicclosed

    54.5%
  23. 23
    Claude Opus 4.8

    Anthropicclosed

    54.4%
  24. 24
    Claude Sonnet 5

    Anthropicclosed

    54.3%
  25. 25
    Gemini 3.5 Flash

    Googleclosed

    53.9%
  26. 26
    GPT-5.6 Luna

    OpenAIclosed

    53.6%
  27. 27
    Gemini 3.6 Flash

    Googleclosed

    53.4%
  28. 28
    Qwen3.8 Max

    Alibabaopen

    52.1%
  29. 29
    GPT-5.4 Mini

    OpenAIclosed

    52.1%
  30. 30
    DeepSeek V4.1 Flash

    DeepSeekopen

    51.9%
  31. 31
    Claude Opus 4.6

    Anthropicclosed

    51.9%
  32. 32
    GLM-5.3 Flash

    Zhipu AIopen

    51.6%
  33. 33
    Kimi K2.6

    Moonshotclosed

    51.5%
  34. 34
    GLM-5.2

    Zhipu AIopen

    51.2%
  35. 35
    DeepSeek V4 Pro 0813

    DeepSeekclosed

    51.0%
  36. 36
    DeepSeek V4 Pro

    DeepSeekopen

    50.8%
  37. 37
    Qwen3.8 Flash Next

    Alibabaopen

    50.6%
  38. 38
    MiMo-V2.5-Pro

    Xiaomiopen

    50.6%
  39. 39
    DeepSeek V4 Flash 0731

    DeepSeekopen

    50.3%
  40. 40
    Claude Sonnet 4.6

    Anthropicclosed

    50.1%
  41. 41
    MiniMax M2.7

    MiniMaxclosed

    50.1%
  42. 42
    Inkling-Small

    Thinking Machinesopen

    49.7%
  43. 43
    Claude Opus 4.5

    Anthropicclosed

    49.5%
  44. 44
    Qwen3.7 Max

    Alibabaclosed

    49.5%
  45. 45
    Kimi K2.5

    Moonshotopen

    49.0%
  46. 46
    Hy3

    Tencentopen

    48.6%
  47. 47
    Grok 4.3

    xAIclosed

    48.3%
  48. 48
    Kimi K2.7 Code

    Moonshotopen

    47.8%
  49. 49
    GPT-5.4 Nano

    OpenAIclosed

    47.2%
  50. 50
    MiniMax M3

    MiniMaxopen

    47.1%
  51. 51
    Inkling

    Thinking Machinesopen

    47.0%
  52. 52
    Qwen3.8 27B

    Alibabaopen

    46.6%
  53. 53
    o4-mini

    OpenAIclosed

    46.5%
  54. 54
    Gemini 2.5 Pro

    Googleclosed

    46.3%
  55. 55
    GLM-5

    Zhipu AIopen

    46.2%
  56. 56
    Qwen3.7 Plus

    Alibabaclosed

    46.1%
  57. 57
    Grok 4

    xAIclosed

    45.7%
  58. 58
    Gemma 4 31B

    Googleopen

    45.5%
  59. 59
    DeepSeek V4 Flash

    DeepSeekopen

    45.3%
  60. 60
    GLM-4.7

    Zhipu AIopen

    45.1%
  61. 61
    Ring-2.6-1T

    InclusionAIopen

    45.0%
  62. 62
    Muse Glimmer

    Metaopen

    44.9%
  63. 63
    Qwen3.5 397B-A17B

    Alibabaopen

    44.8%
  64. 64
    GLM-5.1

    Zhipu AIopen

    44.8%
  65. 65
    Solar Pro 4

    Upstageclosed

    44.6%
  66. 66
    Grok 4 Fast

    xAIclosed

    44.2%
  67. 67
    MiMo-V2.5

    Xiaomiopen

    43.9%
  68. 68
    Gemini 3.1 Flash-Lite

    Googleclosed

    43.4%
  69. 69
    GPT-5

    OpenAIclosed

    42.9%
  70. 70
    K2 Horizon 375B A23B

    MBZUAI Institute of Foundation Modelsopen

    42.9%
  71. 71
    Qwen3.6 27B

    Alibabaopen

    42.8%
  72. 72
    MiniMax M2.5

    MiniMaxopen

    42.6%
  73. 73
    Ling-3.0 Flash

    InclusionAIopen

    42.0%
  74. 74
    Gemini 3.5 Flash-Lite

    Googleclosed

    41.3%
  75. 75
    Hunyuan 3 Preview

    Tencentclosed

    41.2%
  76. 76
    o3

    OpenAIclosed

    41.0%
  77. 77
    Qwen3.6 Plus

    Alibabaclosed

    40.7%
  78. 78
    Trinity Large Thinking

    Arcee AIopen

    40.6%
  79. 79
    GPT-4.1 Mini

    OpenAIclosed

    40.4%
  80. 80
    Step-3.5 Flash

    StepFunclosed

    40.4%
  81. 81
    DeepSeek R1-0528

    DeepSeekopen

    40.3%
  82. 82
    Nemotron 3 Ultra

    NVIDIAopen

    40.3%
  83. 83
    Claude 3.7 Sonnet

    Anthropicclosed

    40.3%
  84. 84
    Mistral Medium 3.5

    Mistralopen

    40.2%
  85. 85
    Qwen3 235B-A22B

    Alibabaopen

    39.9%
  86. 86
    o3-mini

    OpenAIclosed

    39.9%
  87. 87
    Gemini 2.5 Flash

    Googleclosed

    39.4%
  88. 88
    DeepSeek V3.1

    DeepSeekopen

    39.1%
  89. 89
    GPT-5 Mini

    OpenAIclosed

    39.0%
  90. 90
    DeepSeek V3.2

    DeepSeekopen

    38.9%
  91. 91
    gpt-oss-20B

    OpenAIopen

    38.9%
  92. 92
    Mistral Small 4

    Mistralopen

    38.8%
  93. 93
    Qwen3-Next 80B-A3B

    Alibabaopen

    38.8%
  94. 94
    Qwen3 Max

    Alibabaclosed

    38.3%
  95. 95
    DeepSeek R1

    DeepSeekopen

    38.3%
  96. 96
    GPT-4.1

    OpenAIclosed

    38.1%
  97. 97
    Agnes-3.0-Flash Preview

    Agnes AIopen

    38.1%
  98. 98
    Granite 4.2 30B

    IBMopen

    37.8%
  99. 99
    Mercury 2

    Inception Labsclosed

    37.7%
  100. 100
    DeepSeek-R1-Distill-Qwen-32B

    DeepSeekopen

    37.6%
  101. 101
    Ling-2.6 1T

    InclusionAIopen

    37.0%
  102. 102
    Grok 3

    xAIclosed

    36.8%
  103. 103
    GPT-5 Nano

    OpenAIclosed

    36.6%
  104. 104
    Qwen3.6 35B A3B

    Alibabaopen

    36.6%
  105. 105
    Mistral Large 3

    Mistralopen

    36.6%
  106. 106
    Claude 3.5 Sonnet

    Anthropicclosed

    36.6%
  107. 107
    Claude 3.5 Sonnet (Oct)

    Anthropicclosed

    36.6%
  108. 108
    LongCat-2.0

    Meituanopen

    36.3%
  109. 109
    Qwen3 32B

    Alibabaopen

    36.0%
  110. 110
    Qwen3 Coder 480B

    Alibabaopen

    35.9%
  111. 111
    o1

    OpenAIclosed

    35.8%
  112. 112
    DeepSeek V3

    DeepSeekopen

    35.8%
  113. 113
    QwQ-32B

    Alibabaopen

    35.8%
  114. 114
    Kimi K2

    Moonshotopen

    34.5%
  115. 115
    gpt-oss-120B

    OpenAIopen

    34.0%
  116. 116
    GLM-4.7 Flash

    Zhipu AIclosed

    33.7%
  117. 117
    Gemini 2.0 Flash

    Googleclosed

    33.3%
  118. 118
    Devstral 2

    Mistralopen

    32.8%
  119. 119
    Mistral Medium 3.1

    Mistralclosed

    32.4%
  120. 120
    o1 Mini

    OpenAIclosed

    32.3%
  121. 121
    Nemotron 3.5 Lightning

    NVIDIAopen

    32.1%
  122. 122
    GPT-4 Turbo

    OpenAIclosed

    31.9%
  123. 123
    Llama 4 Maverick

    Metaopen

    31.7%
  124. 124
    Granite 4.2 8B

    IBMopen

    31.5%
  125. 125
    DeepSeek-R1-Distill-Llama-70B

    DeepSeekopen

    31.3%
  126. 126
    GPT-4o (May 2024)

    OpenAIclosed

    30.9%
  127. 127
    Llama 3.1 405B

    Metaopen

    29.9%
  128. 128
    Gemini 1.5 Pro

    Googleclosed

    29.5%
  129. 129
    Mistral Large 2

    Mistralclosed

    29.2%
  130. 130
    Pixtral Large

    Mistralclosed

    29.2%
  131. 131
    Mistral Small 3.2

    Mistralopen

    28.6%
  132. 132
    OLMo 3 32B Think

    Ai2open

    28.6%
  133. 133
    Grok 2

    xAIclosed

    28.5%
  134. 134
    Command A

    Cohereclosed

    28.1%
  135. 135
    Qwen3-Coder 30B-A3B

    Alibabaopen

    27.8%
  136. 136
    Claude Haiku 3.5

    Anthropicclosed

    27.4%
  137. 137
    Ling-2.6 Flash

    InclusionAIclosed

    27.1%
  138. 138
    Llama 3.1 70B

    Metaopen

    26.7%
  139. 139
    Gemini 1.5 Flash

    Googleclosed

    26.7%
  140. 140
    Reka Flash 3

    Rekaopen

    26.7%
  141. 141
    Phi-4

    Microsoftopen

    26.0%
  142. 142
    GPT-4.1 Nano

    OpenAIclosed

    25.9%
  143. 143
    Solar Pro 3

    Upstageopen

    25.5%
  144. 144
    Granite 4.2 3B

    IBMopen

    25.3%
  145. 145
    Gemini 2.0 Flash-Lite

    Googleclosed

    25.0%
  146. 146
    Devstral Small

    Mistralopen

    24.3%
  147. 147
    Ling-3.0 Tiny

    InclusionAIopen

    24.2%
  148. 148
    DeepSeek-R1 Distill Qwen 14B

    DeepSeekopen

    23.9%
  149. 149
    Ministral 3 14B

    Mistralopen

    23.8%
  150. 150
    Gemma 3 27B

    Googleopen

    23.3%
  151. 151
    Llama-3.1-Nemotron-70B Instruct

    NVIDIAopen

    23.3%
  152. 152
    Claude 3 Opus

    Anthropicclosed

    23.3%
  153. 153
    GPT-4o Mini

    OpenAIclosed

    22.9%
  154. 154
    Sonar

    Perplexityclosed

    22.9%
  155. 155
    Claude 3 Sonnet

    Anthropicclosed

    22.9%
  156. 156
    Gemini 1.5 Flash-8B

    Googleclosed

    22.9%
  157. 157
    Sonar Pro

    Perplexityclosed

    22.6%
  158. 158
    Granite 4.1 8B

    IBMopen

    21.8%
  159. 159
    Llama 4 Scout

    Metaopen

    21.3%
  160. 160
    Mistral Large

    Mistralclosed

    20.8%
  161. 161
    Ministral 3 8B

    Mistralopen

    20.7%
  162. 162
    Claude 2

    Anthropicclosed

    19.4%
  163. 163
    Gemini 2.5 Flash-Lite

    Googleclosed

    19.3%
  164. 164
    Mixtral 8x22B

    Mistralopen

    18.8%
  165. 165
    Claude 3 Haiku

    Anthropicclosed

    18.6%
  166. 166
    Claude 2.1

    Anthropicclosed

    18.4%
  167. 167
    Gemma 3 12B

    Googleopen

    16.4%
  168. 168
    Jamba 1.5 Large

    AI21open

    16.3%
  169. 169
    Mistral Small

    Mistralclosed

    15.6%
  170. 170
    Ministral 3 3B

    Mistralopen

    15.3%
  171. 171
    DeepSeek Coder V2 Lite

    DeepSeekopen

    13.9%
  172. 172
    Llama 3.1 8B

    Metaopen

    13.2%
  173. 173
    Granite 4.0 Micro

    IBMopen

    11.9%
  174. 174
    DeepSeek-R1 Distill Llama 8B

    DeepSeekopen

    11.9%
  175. 175
    DBRX Instruct

    Databricksopen

    11.8%
  176. 176
    Gemini 1.0 Pro

    Googleclosed

    11.7%
  177. 177
    Phi-4-multimodal

    Microsoftopen

    11.0%
  178. 178
    Phi-4-mini

    Microsoftopen

    10.8%
  179. 179
    Phi-3 Mini 3.8B

    Microsoftopen

    9.0%
  180. 180
    OLMo 2 32B

    Ai2open

    8.0%
  181. 181
    LFM2.5-8B-A1B

    Liquid AIopen

    7.8%
  182. 182
    Gemma 3 4B

    Googleopen

    7.3%
  183. 183
    Mixtral 8x7B

    Mistralopen

    2.8%

Back to all benchmarks