LLMcompare

Search

Search for a command to run...

Reasoning & knowledge

Humanity's Last Exam

Expert-level closed-ended questions across dozens of academic fields; tool access, modality, and evaluation protocol must match before scores are compared.

How models are tested

Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.

Category
Reasoning & knowledge
Metric
Percent
Direction
Higher is better
Coverage
212 / 318Models in this catalog with a published score
Updated
Catalog snapshot date
Source
agi.safe.ai/

Scores

  1. 1
    Claude Opus 5.5

    Anthropicclosed

    61.4%
  2. 2
    Claude Fable 5.1

    Anthropicclosed

    59.0%
  3. 3
    Claude Fable 5

    Anthropicclosed

    55.5%
  4. 4
    GPT-6 Astra

    OpenAIclosed

    55.0%
  5. 5
    Claude Opus 5

    Anthropicclosed

    54.9%
  6. 6
    GPT-5.6 Sol

    OpenAIclosed

    49.5%
  7. 7
    Claude Opus 4.8

    Anthropicclosed

    48.7%
  8. 8
    Muse Spark 1.3

    Metaclosed

    48.7%
  9. 9
    GPT-6 Sol

    OpenAIclosed

    48.0%
  10. 10
    Gemini 3.7 Flash

    Googleclosed

    47.9%
  11. 11
    Gemini 3.8 Flash

    Googleclosed

    47.8%
  12. 12
    Kimi K3

    Moonshotopen

    46.9%
  13. 13
    Muse Spark 1.1

    Metaclosed

    46.2%
  14. 14
    GPT-5.5

    OpenAIclosed

    45.8%
  15. 15
    Muse Spark 1.2

    Metaclosed

    45.5%
  16. 16
    Gemini 3.1 Pro

    Googleclosed

    44.4%
  17. 17
    GPT-5.4

    OpenAIclosed

    43.7%
  18. 18
    GPT-5.5 Pro

    OpenAIclosed

    43.1%
  19. 19
    Qwen3.8 Max

    Alibabaopen

    43.1%
  20. 20
    Grok 4.7

    xAIclosed

    43.0%
  21. 21
    GPT-5.6 Terra

    OpenAIclosed

    42.9%
  22. 22
    Grok 4.6

    xAIclosed

    42.9%
  23. 23
    Grok 4.5

    xAIclosed

    42.7%
  24. 24
    Gemini 3.5 Flash

    Googleclosed

    42.7%
  25. 25
    GLM-5.3

    Zhipu AIopen

    42.3%
  26. 26
    Claude Opus 4.7

    Anthropicclosed

    42.3%
  27. 27
    Claude Sonnet 5

    Anthropicclosed

    41.3%
  28. 28
    GLM-5.2

    Zhipu AIopen

    41.1%
  29. 29
    DeepSeek V4 Pro 0813

    DeepSeekclosed

    41.0%
  30. 30
    Gemini 3.6 Flash

    Googleclosed

    40.8%
  31. 31
    Qwen3.7 Max

    Alibabaclosed

    40.5%
  32. 32
    GLM-5.3 Flash

    Zhipu AIopen

    39.9%
  33. 33
    Claude Opus 4.6

    Anthropicclosed

    39.9%
  34. 34
    GPT-5.6 Luna

    OpenAIclosed

    39.5%
  35. 35
    DeepSeek V4.1 Flash

    DeepSeekopen

    39.2%
  36. 36
    GPT-6 Luna

    OpenAIclosed

    39.0%
  37. 37
    MiniMax M3

    MiniMaxopen

    39.0%
  38. 38
    DeepSeek V4 Flash 0731

    DeepSeekopen

    38.6%
  39. 39
    Qwen3.8 Flash Next

    Alibabaopen

    38.0%
  40. 40
    Gemini 3 Pro

    Googleclosed

    37.5%
  41. 41
    DeepSeek V4 Pro

    DeepSeekopen

    37.5%
  42. 42
    Kimi K2.6

    Moonshotclosed

    37.5%
  43. 43
    Grok 4.3

    xAIclosed

    37.2%
  44. 44
    Gemini 3 Flash

    Googleclosed

    36.6%
  45. 45
    MiMo-V2.5-Pro

    Xiaomiopen

    35.7%
  46. 46
    Qwen3.7 Plus

    Alibabaclosed

    35.6%
  47. 47
    Kimi K2.7 Code

    Moonshotopen

    35.0%
  48. 48
    DeepSeek V4 Flash

    DeepSeekopen

    34.8%
  49. 49
    Qwen3.8 27B

    Alibabaopen

    33.9%
  50. 50
    LongCat-2.0

    Meituanopen

    33.7%
  51. 51
    Claude Sonnet 4.6

    Anthropicclosed

    33.6%
  52. 52
    Hy3

    Tencentopen

    33.5%
  53. 53
    Inkling-Small

    Thinking Machinesopen

    33.3%
  54. 54
    Claude Mythos 5

    Anthropicclosed

    32.5%
  55. 55
    Seed 2.0 Pro

    ByteDanceclosed

    32.4%
  56. 56
    K2 Horizon 375B A23B

    MBZUAI Institute of Foundation Modelsopen

    32.0%
  57. 57
    Inkling

    Thinking Machinesopen

    31.9%
  58. 58
    Kimi K2.5

    Moonshotopen

    30.7%
  59. 59
    Claude Opus 4.5

    Anthropicclosed

    30.1%
  60. 60
    GLM-5.1

    Zhipu AIopen

    30.1%
  61. 61
    MiniMax M2.7

    MiniMaxclosed

    29.6%
  62. 62
    GLM-5

    Zhipu AIopen

    29.3%
  63. 63
    Solar Pro 4

    Upstageclosed

    29.2%
  64. 64
    Qwen3.5 397B-A17B

    Alibabaopen

    29.0%
  65. 65
    GPT-5

    OpenAIclosed

    28.5%
  66. 66
    Nemotron 3 Ultra

    NVIDIAopen

    28.4%
  67. 67
    GPT-5.4 Nano

    OpenAIclosed

    28.3%
  68. 68
    GPT-5.4 Mini

    OpenAIclosed

    28.1%
  69. 69
    Qwen3.6 Plus

    Alibabaclosed

    27.8%
  70. 70
    Hunyuan 3 Preview

    Tencentclosed

    27.8%
  71. 71
    GLM-4.7

    Zhipu AIopen

    27.4%
  72. 72
    MiMo-V2.5

    Xiaomiopen

    27.2%
  73. 73
    Grok 4

    xAIclosed

    26.7%
  74. 74
    DeepSeek V3.2

    DeepSeekopen

    24.6%
  75. 75
    Ling-3.0 Flash

    InclusionAIopen

    23.7%
  76. 76
    Gemma 4 31B

    Googleopen

    23.6%
  77. 77
    Qwen3.6 27B

    Alibabaopen

    23.1%
  78. 78
    Gemini 2.5 Pro

    Googleclosed

    22.5%
  79. 79
    Qwen3.6 35B A3B

    Alibabaopen

    22.2%
  80. 80
    MiMo-V2-Flash

    Xiaomiopen

    22.1%
  81. 81
    Muse Glimmer

    Metaopen

    22.0%
  82. 82
    Ring-2.6-1T

    InclusionAIopen

    21.6%
  83. 83
    GPT-5 Mini

    OpenAIclosed

    21.5%
  84. 84
    Step-3.5 Flash

    StepFunclosed

    21.1%
  85. 85
    MiniMax M2.5

    MiniMaxopen

    20.5%
  86. 86
    o3

    OpenAIclosed

    20.1%
  87. 87
    gpt-oss-120B

    OpenAIopen

    19.6%
  88. 88
    Grok 4 Fast

    xAIclosed

    19.1%
  89. 89
    Gemini 3.5 Flash-Lite

    Googleclosed

    18.8%
  90. 90
    Gemini 3.1 Flash-Lite

    Googleclosed

    17.2%
  91. 91
    Mercury 2

    Inception Labsclosed

    17.1%
  92. 92
    o4-mini

    OpenAIclosed

    16.5%
  93. 93
    DeepSeek R1-0528

    DeepSeekopen

    15.8%
  94. 94
    Trinity Large Thinking

    Arcee AIopen

    15.8%
  95. 95
    DeepSeek V3.1

    DeepSeekopen

    14.3%
  96. 96
    Mistral Medium 3.5

    Mistralopen

    13.8%
  97. 97
    INTELLECT-3

    Prime Intellectopen

    13.1%
  98. 98
    Qwen3-Next 80B-A3B

    Alibabaopen

    12.6%
  99. 99
    Gemini 2.5 Flash

    Googleclosed

    12.1%
  100. 100
    Qwen3 Max

    Alibabaclosed

    11.9%
  101. 101
    Granite 4.2 30B

    IBMopen

    11.2%
  102. 102
    Qwen3 235B-A22B

    Alibabaopen

    11.0%
  103. 103
    gpt-oss-20B

    OpenAIopen

    11.0%
  104. 104
    Nemotron Nano 3 30B

    NVIDIAopen

    10.6%
  105. 105
    Nemotron 3.5 Lightning

    NVIDIAopen

    10.6%
  106. 106
    Solar Pro 3

    Upstageopen

    10.3%
  107. 107
    K2 Think V2

    MBZUAI / G42 / LLM360open

    10.1%
  108. 108
    Mistral Small 4

    Mistralopen

    9.9%
  109. 109
    Seed-OSS-36B-Instruct

    ByteDance Seedopen

    9.9%
  110. 110
    Granite 4.2 8B

    IBMopen

    9.7%
  111. 111
    Claude 3.7 Sonnet

    Anthropicclosed

    9.7%
  112. 112
    GPT-5 Nano

    OpenAIclosed

    9.5%
  113. 113
    Ling-3.0 Tiny

    InclusionAIopen

    9.3%
  114. 114
    Gemma 4 26B

    Googleopen

    8.7%
  115. 115
    Ling-2.6 1T

    InclusionAIopen

    8.7%
  116. 116
    DeepSeek R1

    DeepSeekopen

    8.5%
  117. 117
    o3-mini

    OpenAIclosed

    7.9%
  118. 118
    GLM-4.7 Flash

    Zhipu AIclosed

    7.6%
  119. 119
    Qwen3 32B

    Alibabaopen

    7.4%
  120. 120
    QwQ-32B

    Alibabaopen

    7.4%
  121. 121
    Kimi K2

    Moonshotopen

    7.4%
  122. 122
    o1

    OpenAIclosed

    7.0%
  123. 123
    LFM2.5-8B-A1B

    Liquid AIopen

    6.9%
  124. 124
    Gemini 2.5 Flash-Lite

    Googleclosed

    6.8%
  125. 125
    LFM2.5-1.2B-Instruct

    Liquid AIopen

    6.7%
  126. 126
    Sonar Pro

    Perplexityclosed

    6.6%
  127. 127
    Granite 4.2 3B

    IBMopen

    6.6%
  128. 128
    Nemotron Nano 9B v2

    NVIDIAopen

    6.5%
  129. 129
    OLMo 3 32B Think

    Ai2open

    6.4%
  130. 130
    Ling-2.6 Flash

    InclusionAIclosed

    6.3%
  131. 131
    Olmo 3 7B Instruct

    Ai2open

    5.8%
  132. 132
    Llama 3.2 11B Vision

    Metaopen

    5.5%
  133. 133
    Llama 3.2 1B

    Metaopen

    5.5%
  134. 134
    Apertus 70B Instruct

    Swiss AI Initiativeopen

    5.5%
  135. 135
    Ministral 3 3B

    Mistralopen

    5.4%
  136. 136
    Llama 3.2 3B

    Metaopen

    5.4%
  137. 137
    DeepSeek Coder V2 Lite

    DeepSeekopen

    5.4%
  138. 138
    Llama 3.1 8B

    Metaopen

    5.3%
  139. 139
    Gemma 3 4B

    Googleopen

    5.3%
  140. 140
    Llama 3 8B

    Metaopen

    5.1%
  141. 141
    DeepSeek-R1-Distill-Llama-70B

    DeepSeekopen

    5.1%
  142. 142
    GPT-4.1 Mini

    OpenAIclosed

    5.0%
  143. 143
    Granite 4.0 Micro

    IBMopen

    5.0%
  144. 144
    Phi-4-multimodal

    Microsoftopen

    5.0%
  145. 145
    Phi-3 Mini 3.8B

    Microsoftopen

    5.0%
  146. 146
    Apertus 8B Instruct

    Swiss AI Initiativeopen

    5.0%
  147. 147
    Olmo 3.1 32B Instruct

    Ai2open

    5.0%
  148. 148
    Llama 4 Maverick

    Metaopen

    4.9%
  149. 149
    Sonar

    Perplexityclosed

    4.9%
  150. 150
    Qwen2.5 Coder 7B

    Alibabaopen

    4.9%
  151. 151
    Mistral Medium 3.1

    Mistralclosed

    4.7%
  152. 152
    Gemini 1.5 Flash-8B

    Googleclosed

    4.7%
  153. 153
    Mixtral 8x7B

    Mistralopen

    4.7%
  154. 154
    Ministral 3 14B

    Mistralopen

    4.6%
  155. 155
    Gemini 1.5 Pro

    Googleclosed

    4.6%
  156. 156
    DeepSeek-R1-Distill-Qwen-32B

    DeepSeekopen

    4.6%
  157. 157
    Mistral 7B

    Mistralopen

    4.6%
  158. 158
    Qwen3-Omni-30B-A3B-Instruct

    Alibabaopen

    4.6%
  159. 159
    Llama 3.1 70B

    Metaopen

    4.5%
  160. 160
    Llama 3.2 90B Vision

    Metaopen

    4.5%
  161. 161
    Qwen3 Coder 480B

    Alibabaopen

    4.5%
  162. 162
    Phi-4-mini

    Microsoftopen

    4.5%
  163. 163
    Llama 3 70B

    Metaopen

    4.5%
  164. 164
    Gemma 3 27B

    Googleopen

    4.4%
  165. 165
    Reka Flash 3

    Rekaopen

    4.4%
  166. 166
    Ministral 3 8B

    Mistralopen

    4.3%
  167. 167
    Mistral Small 3.2

    Mistralopen

    4.3%
  168. 168
    Gemini 2.0 Flash

    Googleclosed

    4.3%
  169. 169
    Mistral Small

    Mistralclosed

    4.3%
  170. 170
    GPT-4.1

    OpenAIclosed

    4.2%
  171. 171
    GPT-4o Mini

    OpenAIclosed

    4.2%
  172. 172
    Gemma 3 12B

    Googleopen

    4.2%
  173. 173
    Mistral Large 3

    Mistralopen

    4.2%
  174. 174
    Llama-3.1-Nemotron-70B Instruct

    NVIDIAopen

    4.2%
  175. 175
    Gemini 1.0 Pro

    Googleclosed

    4.2%
  176. 176
    Grok 3

    xAIclosed

    4.1%
  177. 177
    Jamba 1.5 Large

    AI21open

    4.1%
  178. 178
    Claude 3 Haiku

    Anthropicclosed

    4.1%
  179. 179
    DeepSeek-R1 Distill Qwen 14B

    DeepSeekopen

    4.1%
  180. 180
    Llama 3.1 405B

    Metaopen

    4.0%
  181. 181
    Command A

    Cohereclosed

    4.0%
  182. 182
    Mixtral 8x22B

    Mistralopen

    4.0%
  183. 183
    Claude 2.1

    Anthropicclosed

    3.9%
  184. 184
    GPT-4.1 Nano

    OpenAIclosed

    3.8%
  185. 185
    Llama 4 Scout

    Metaopen

    3.8%
  186. 186
    Phi-4

    Microsoftopen

    3.8%
  187. 187
    Granite 4.1 8B

    IBMopen

    3.8%
  188. 188
    Devstral Small

    Mistralopen

    3.8%
  189. 189
    Qwen3-Coder 30B-A3B

    Alibabaopen

    3.8%
  190. 190
    Qwen2 72B Instruct

    Alibabaopen

    3.7%
  191. 191
    Claude 3.5 Sonnet

    Anthropicclosed

    3.7%
  192. 192
    Claude 3.5 Sonnet (Oct)

    Anthropicclosed

    3.7%
  193. 193
    Claude Haiku 3.5

    Anthropicclosed

    3.6%
  194. 194
    Llama 3.3 70B

    Metaopen

    3.6%
  195. 195
    Devstral 2

    Mistralopen

    3.6%
  196. 196
    Qwen2.5 72B

    Alibabaopen

    3.6%
  197. 197
    o1 Mini

    OpenAIclosed

    3.6%
  198. 198
    Claude 3 Sonnet

    Anthropicclosed

    3.6%
  199. 199
    OLMo 2 32B

    Ai2open

    3.6%
  200. 200
    Mistral Large

    Mistralclosed

    3.5%
  201. 201
    Qwen2.5-Coder 32B

    Alibabaopen

    3.5%
  202. 202
    Gemini 2.0 Flash-Lite

    Googleclosed

    3.4%
  203. 203
    Mistral Large 2

    Mistralclosed

    3.3%
  204. 204
    Gemini 1.5 Flash

    Googleclosed

    3.2%
  205. 205
    GPT-4 Turbo

    OpenAIclosed

    3.1%
  206. 206
    Grok 2

    xAIclosed

    3.1%
  207. 207
    DeepSeek V3

    DeepSeekopen

    2.9%
  208. 208
    DBRX Instruct

    Databricksopen

    2.9%
  209. 209
    Claude 3 Opus

    Anthropicclosed

    2.8%
  210. 210
    Pixtral Large

    Mistralclosed

    2.8%
  211. 211
    Kimi Linear 48B A3B Instruct

    Moonshotopen

    2.5%
  212. 212
    GPT-4o (May 2024)

    OpenAIclosed

    1.8%

Back to all benchmarks