LLMcompare

Search

Search for a command to run...

Reasoning & knowledge

GPQA Diamond

Graduate-level Google-proof Q&A in biology, physics, and chemistry (diamond subset); scores are only comparable when evaluation protocol and tool access match.

How models are tested

Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.

Category
Reasoning & knowledge
Metric
Percent
Direction
Higher is better
Coverage
215 / 318Models in this catalog with a published score
Updated
Catalog snapshot date
Source
github.com/idavidrein/gpqa

Scores

  1. 1
    GPT-6 Astra

    OpenAIclosed

    96.1%
  2. 2
    Gemini 3.8 Flash

    Googleclosed

    95.3%
  3. 3
    Grok 4.6

    xAIclosed

    94.9%
  4. 4
    Gemini 3.7 Flash

    Googleclosed

    94.5%
  5. 5
    Gemini 3.1 Pro

    Googleclosed

    94.3%
  6. 6
    GPT-5.6 Sol

    OpenAIclosed

    94.1%
  7. 7
    Claude Fable 5.1

    Anthropicclosed

    93.7%
  8. 8
    GPT-5.5

    OpenAIclosed

    93.5%
  9. 9
    Muse Spark 1.3

    Metaclosed

    93.5%
  10. 10
    Kimi K3

    Moonshotopen

    93.5%
  11. 11
    Claude Opus 5

    Anthropicclosed

    93.2%
  12. 12
    Grok 4.5

    xAIclosed

    93.1%
  13. 13
    MiniMax M3

    MiniMaxopen

    92.9%
  14. 14
    Gemini 3.6 Flash

    Googleclosed

    92.8%
  15. 15
    DeepSeek V4 Pro 0813

    DeepSeekclosed

    92.8%
  16. 16
    Qwen3.8 Max

    Alibabaopen

    92.8%
  17. 17
    Claude Fable 5

    Anthropicclosed

    92.6%
  18. 18
    Claude Mythos 5

    Anthropicclosed

    92.6%
  19. 19
    GPT-5.6 Terra

    OpenAIclosed

    92.5%
  20. 20
    Qwen3.8 Flash Next

    Alibabaopen

    92.3%
  21. 21
    Qwen3.7 Max

    Alibabaclosed

    92.3%
  22. 22
    Hy4 Preview

    Tencentopen

    92.3%
  23. 23
    Gemini 3.5 Flash

    Googleclosed

    92.2%
  24. 24
    GPT-5.4

    OpenAIclosed

    92.0%
  25. 25
    Claude Opus 4.8

    Anthropicclosed

    92.0%
  26. 26
    Gemini 3 Pro

    Googleclosed

    91.9%
  27. 27
    GLM-5.3

    Zhipu AIopen

    91.7%
  28. 28
    Claude Opus 4.7

    Anthropicclosed

    91.4%
  29. 29
    GLM-5.3 Flash

    Zhipu AIopen

    91.2%
  30. 30
    GPT-5.6 Luna

    OpenAIclosed

    91.1%
  31. 31
    Claude Sonnet 5

    Anthropicclosed

    91.1%
  32. 32
    Kimi K2.6

    Moonshotclosed

    91.1%
  33. 33
    DeepSeek V4 Flash 0731

    DeepSeekopen

    90.8%
  34. 34
    Qwen3.8 27B

    Alibabaopen

    90.5%
  35. 35
    Muse Spark 1.2

    Metaclosed

    90.4%
  36. 36
    Grok 4.3

    xAIclosed

    90.1%
  37. 37
    Qwen3.7 Plus

    Alibabaclosed

    90.0%
  38. 38
    Gemini 3 Flash

    Googleclosed

    89.8%
  39. 39
    Muse Spark 1.1

    Metaclosed

    89.8%
  40. 40
    Hy3

    Tencentopen

    89.7%
  41. 41
    Kimi K2.7 Code

    Moonshotopen

    89.6%
  42. 42
    Claude Opus 4.6

    Anthropicclosed

    89.6%
  43. 43
    GLM-5.2

    Zhipu AIopen

    89.5%
  44. 44
    Inkling-Small

    Thinking Machinesopen

    89.5%
  45. 45
    DeepSeek V4 Flash

    DeepSeekopen

    89.4%
  46. 46
    Qwen3.5 397B-A17B

    Alibabaopen

    89.3%
  47. 47
    Solar Pro 4

    Upstageclosed

    89.1%
  48. 48
    Seed 2.0 Pro

    ByteDanceclosed

    88.9%
  49. 49
    DeepSeek V4 Pro

    DeepSeekopen

    88.8%
  50. 50
    Qwen3.6 Plus

    Alibabaclosed

    88.2%
  51. 51
    Kimi K2.5

    Moonshotopen

    87.9%
  52. 52
    Grok 4

    xAIclosed

    87.7%
  53. 53
    Claude Sonnet 4.6

    Anthropicclosed

    87.5%
  54. 54
    GPT-5.4 Mini

    OpenAIclosed

    87.5%
  55. 55
    MiniMax M2.7

    MiniMaxclosed

    87.4%
  56. 56
    K2 Horizon 375B A23B

    MBZUAI Institute of Foundation Modelsopen

    87.3%
  57. 57
    Inkling

    Thinking Machinesopen

    87.2%
  58. 58
    GLM-5.1

    Zhipu AIopen

    86.8%
  59. 59
    Nemotron 3 Ultra

    NVIDIAopen

    86.7%
  60. 60
    Hunyuan 3 Preview

    Tencentclosed

    86.7%
  61. 61
    Claude Opus 4.5

    Anthropicclosed

    86.6%
  62. 62
    MiMo-V2.5-Pro

    Xiaomiopen

    86.6%
  63. 63
    GLM-4.7

    Zhipu AIopen

    85.9%
  64. 64
    Gemma 4 31B

    Googleopen

    85.7%
  65. 65
    Ring-2.6-1T

    InclusionAIopen

    85.7%
  66. 66
    Ling-3.0 Flash

    InclusionAIopen

    85.5%
  67. 67
    GPT-5

    OpenAIclosed

    85.4%
  68. 68
    Agnes-3.0-Flash Preview

    Agnes AIopen

    85.0%
  69. 69
    MiMo-V2.5

    Xiaomiopen

    84.9%
  70. 70
    MiniMax M2.5

    MiniMaxopen

    84.8%
  71. 71
    Grok 4 Fast

    xAIclosed

    84.7%
  72. 72
    o3-pro

    OpenAIclosed

    84.5%
  73. 73
    Gemini 2.5 Pro

    Googleclosed

    84.4%
  74. 74
    Qwen3.6 27B

    Alibabaopen

    84.2%
  75. 75
    Qwen3.6 35B A3B

    Alibabaopen

    84.1%
  76. 76
    DeepSeek V3.2

    DeepSeekopen

    84.0%
  77. 77
    Gemini 3.5 Flash-Lite

    Googleclosed

    83.8%
  78. 78
    Muse Glimmer

    Metaopen

    83.5%
  79. 79
    MiMo-V2-Flash

    Xiaomiopen

    83.5%
  80. 80
    Step-3.5 Flash

    StepFunclosed

    83.1%
  81. 81
    GPT-5 Mini

    OpenAIclosed

    82.8%
  82. 82
    o3

    OpenAIclosed

    82.7%
  83. 83
    Gemma 4 26B

    Googleopen

    82.3%
  84. 84
    Gemini 3.1 Flash-Lite

    Googleclosed

    82.2%
  85. 85
    GLM-5

    Zhipu AIopen

    82.0%
  86. 86
    GPT-5.4 Nano

    OpenAIclosed

    81.7%
  87. 87
    DeepSeek R1-0528

    DeepSeekopen

    81.3%
  88. 88
    Gemini 2.5 Flash

    Googleclosed

    79.0%
  89. 89
    o4-mini

    OpenAIclosed

    78.4%
  90. 90
    gpt-oss-120B

    OpenAIopen

    78.2%
  91. 91
    LongCat-2.0

    Meituanopen

    78.0%
  92. 92
    DeepSeek V3.1

    DeepSeekopen

    77.9%
  93. 93
    Claude 3.7 Sonnet

    Anthropicclosed

    77.2%
  94. 94
    Mercury 2

    Inception Labsclosed

    77.0%
  95. 95
    Mistral Small 4

    Mistralopen

    76.9%
  96. 96
    Kimi K2

    Moonshotopen

    76.6%
  97. 97
    o1 Preview

    OpenAIclosed

    76.5%
  98. 98
    Qwen3 Max

    Alibabaclosed

    76.4%
  99. 99
    INTELLECT-3

    Prime Intellectopen

    76.1%
  100. 100
    Qwen3-Next 80B-A3B

    Alibabaopen

    75.9%
  101. 101
    Ling-2.6 1T

    InclusionAIopen

    75.2%
  102. 102
    Trinity Large Thinking

    Arcee AIopen

    75.2%
  103. 103
    Mistral Medium 3.5

    Mistralopen

    74.8%
  104. 104
    o3-mini

    OpenAIclosed

    74.8%
  105. 105
    o1

    OpenAIclosed

    74.7%
  106. 106
    Nemotron 3.5 Lightning

    NVIDIAopen

    74.3%
  107. 107
    Ling-3.0 Tiny

    InclusionAIopen

    73.4%
  108. 108
    Seed-OSS-36B-Instruct

    ByteDance Seedopen

    72.6%
  109. 109
    Solar Pro 3

    Upstageopen

    72.4%
  110. 110
    K2 Think V2

    MBZUAI / G42 / LLM360open

    71.3%
  111. 111
    DeepSeek R1

    DeepSeekopen

    70.8%
  112. 112
    Qwen3 235B-A22B

    Alibabaopen

    70.0%
  113. 113
    Grok 3

    xAIclosed

    69.3%
  114. 114
    gpt-oss-20B

    OpenAIopen

    68.8%
  115. 115
    Magistral Small

    Mistralopen

    68.2%
  116. 116
    Mistral Large 3

    Mistralopen

    68.0%
  117. 117
    GPT-5 Nano

    OpenAIclosed

    67.6%
  118. 118
    Llama 4 Maverick

    Metaopen

    67.1%
  119. 119
    Qwen3 32B

    Alibabaopen

    66.8%
  120. 120
    GPT-4.1

    OpenAIclosed

    66.6%
  121. 121
    GPT-4.1 Mini

    OpenAIclosed

    66.4%
  122. 122
    Phi-4 Reasoning

    Microsoftopen

    65.8%
  123. 123
    Granite 4.2 30B

    IBMopen

    64.4%
  124. 124
    Granite 4.2 8B

    IBMopen

    63.1%
  125. 125
    Gemini 2.5 Flash-Lite

    Googleclosed

    62.5%
  126. 126
    Gemini 2.0 Flash

    Googleclosed

    62.3%
  127. 127
    Sonar Reasoning

    Perplexityclosed

    62.3%
  128. 128
    Qwen3-Omni-30B-A3B-Instruct

    Alibabaopen

    62.0%
  129. 129
    Qwen3 Coder 480B

    Alibabaopen

    61.8%
  130. 130
    DeepSeek-R1-Distill-Qwen-32B

    DeepSeekopen

    61.5%
  131. 131
    OLMo 3 32B Think

    Ai2open

    61.0%
  132. 132
    o1 Mini

    OpenAIclosed

    60.3%
  133. 133
    Claude 3.5 Sonnet

    Anthropicclosed

    59.9%
  134. 134
    Claude 3.5 Sonnet (Oct)

    Anthropicclosed

    59.9%
  135. 135
    Devstral 2

    Mistralopen

    59.4%
  136. 136
    QwQ-32B

    Alibabaopen

    59.3%
  137. 137
    Ling-2.6 Flash

    InclusionAIclosed

    59.3%
  138. 138
    Gemini 1.5 Pro

    Googleclosed

    58.9%
  139. 139
    Mistral Medium 3.1

    Mistralclosed

    58.8%
  140. 140
    Llama 4 Scout

    Metaopen

    58.7%
  141. 141
    GLM-4.7 Flash

    Zhipu AIclosed

    58.1%
  142. 142
    Sonar Pro

    Perplexityclosed

    57.8%
  143. 143
    Phi-4

    Microsoftopen

    57.5%
  144. 144
    Ministral 3 14B

    Mistralopen

    57.2%
  145. 145
    Granite 4.2 3B

    IBMopen

    55.9%
  146. 146
    DeepSeek V3

    DeepSeekopen

    55.7%
  147. 147
    Olmo 3.1 32B Instruct

    Ai2open

    53.9%
  148. 148
    Gemini 2.0 Flash-Lite

    Googleclosed

    53.5%
  149. 149
    Reka Flash 3

    Rekaopen

    52.9%
  150. 150
    Command A

    Cohereclosed

    52.7%
  151. 151
    GPT-4o (May 2024)

    OpenAIclosed

    52.6%
  152. 152
    Qwen3-Coder 30B-A3B

    Alibabaopen

    51.6%
  153. 153
    Llama 3.1 405B

    Metaopen

    51.5%
  154. 154
    LFM2.5-8B-A1B

    Liquid AIopen

    51.3%
  155. 155
    GPT-4.1 Nano

    OpenAIclosed

    51.2%
  156. 156
    Grok 2

    xAIclosed

    51.0%
  157. 157
    Mistral Small 3.2

    Mistralopen

    50.5%
  158. 158
    Pixtral Large

    Mistralclosed

    50.5%
  159. 159
    Llama 3.3 70B

    Metaopen

    49.8%
  160. 160
    Qwen2.5 72B

    Alibabaopen

    49.1%
  161. 161
    Claude 3 Opus

    Anthropicclosed

    48.9%
  162. 162
    Mistral Large 2

    Mistralclosed

    48.6%
  163. 163
    DeepSeek-R1 Distill Qwen 14B

    DeepSeekopen

    48.4%
  164. 164
    Ministral 3 8B

    Mistralopen

    47.1%
  165. 165
    Sonar

    Perplexityclosed

    47.1%
  166. 166
    Llama-3.1-Nemotron-70B Instruct

    NVIDIAopen

    46.5%
  167. 167
    Gemini 1.5 Flash

    Googleclosed

    46.3%
  168. 168
    Granite 4.1 8B

    IBMopen

    43.3%
  169. 169
    Llama 3.2 90B Vision

    Metaopen

    43.2%
  170. 170
    Gemma 3 27B

    Googleopen

    42.8%
  171. 171
    Jamba 1.5 Large

    AI21open

    42.7%
  172. 172
    GPT-4o Mini

    OpenAIclosed

    42.6%
  173. 173
    DeepSeek-V2.5

    DeepSeekopen

    42.3%
  174. 174
    Qwen2.5-Coder 32B

    Alibabaopen

    41.7%
  175. 175
    Devstral Small

    Mistralopen

    41.4%
  176. 176
    Kimi Linear 48B A3B Instruct

    Moonshotopen

    41.2%
  177. 177
    Llama 3.1 70B

    Metaopen

    40.9%
  178. 178
    Claude Haiku 3.5

    Anthropicclosed

    40.8%
  179. 179
    DeepSeek-R1-Distill-Llama-70B

    DeepSeekopen

    40.2%
  180. 180
    Claude 3 Sonnet

    Anthropicclosed

    40.0%
  181. 181
    Olmo 3 7B Instruct

    Ai2open

    40.0%
  182. 182
    Mistral Small

    Mistralclosed

    38.1%
  183. 183
    Llama 3 70B

    Metaopen

    37.9%
  184. 184
    Claude 3 Haiku

    Anthropicclosed

    37.4%
  185. 185
    Qwen2 72B Instruct

    Alibabaopen

    37.1%
  186. 186
    Gemini 1.5 Flash-8B

    Googleclosed

    35.9%
  187. 187
    Ministral 3 3B

    Mistralopen

    35.8%
  188. 188
    Mistral Large

    Mistralclosed

    35.1%
  189. 189
    Gemma 3 12B

    Googleopen

    34.9%
  190. 190
    GPT-4

    OpenAIclosed

    34.9%
  191. 191
    Claude 2

    Anthropicclosed

    34.4%
  192. 192
    Qwen2.5 Coder 7B

    Alibabaopen

    33.9%
  193. 193
    Granite 4.0 Micro

    IBMopen

    33.6%
  194. 194
    Mixtral 8x22B

    Mistralopen

    33.2%
  195. 195
    Phi-4-mini

    Microsoftopen

    33.1%
  196. 196
    DBRX Instruct

    Databricksopen

    33.1%
  197. 197
    OLMo 2 32B

    Ai2open

    32.8%
  198. 198
    LFM2.5-1.2B-Instruct

    Liquid AIopen

    32.6%
  199. 199
    Phi-3 Mini 3.8B

    Microsoftopen

    31.9%
  200. 200
    Claude 2.1

    Anthropicclosed

    31.9%
  201. 201
    DeepSeek Coder V2 Lite

    DeepSeekopen

    31.9%
  202. 202
    Phi-4-multimodal

    Microsoftopen

    31.5%
  203. 203
    DeepSeek-R1 Distill Llama 8B

    DeepSeekopen

    30.2%
  204. 204
    GPT-3.5 Turbo

    OpenAIclosed

    29.7%
  205. 205
    Llama 3 8B

    Metaopen

    29.6%
  206. 206
    Mixtral 8x7B

    Mistralopen

    29.2%
  207. 207
    Gemma 3 4B

    Googleopen

    29.1%
  208. 208
    Gemini 1.0 Pro

    Googleclosed

    27.7%
  209. 209
    Apertus 70B Instruct

    Swiss AI Initiativeopen

    27.2%
  210. 210
    Llama 3.1 8B

    Metaopen

    25.9%
  211. 211
    Apertus 8B Instruct

    Swiss AI Initiativeopen

    25.6%
  212. 212
    Llama 3.2 3B

    Metaopen

    25.5%
  213. 213
    Llama 3.2 11B Vision

    Metaopen

    22.1%
  214. 214
    Llama 3.2 1B

    Metaopen

    19.6%
  215. 215
    Mistral 7B

    Mistralopen

    17.7%

Back to all benchmarks