LLMcompare

Search

Search for a command to run...

Reasoning & knowledge

IFBench

Ai2's precise instruction-following benchmark: 58 verifiable out-of-domain output constraints held out from the training constraints, so it measures generalisation to unseen instructions rather than familiarity with common ones. Scored as constraint-satisfaction accuracy.

How models are tested

Scores are published numbers from the sources below. They are only comparable when the evaluation version, tools, agent harness, and sampling setup match. Missing scores are omitted rather than treated as zero.

Category
Reasoning & knowledge
Metric
Percent
Direction
Higher is better
Coverage
137 / 318Models in this catalog with a published score
Updated
Catalog snapshot date
Source
artificialanalysis.ai/evaluations/ifbench

Scores

  1. 1
    MiniMax M3

    MiniMaxopen

    82.9%
  2. 2
    Nemotron 3 Ultra

    NVIDIAopen

    81.4%
  3. 3
    Grok 4.3

    xAIclosed

    81.3%
  4. 4
    Qwen3.7 Max

    Alibabaclosed

    80.5%
  5. 5
    MiMo-V2.5-Pro

    Xiaomiopen

    79.9%
  6. 6
    DeepSeek V4 Flash

    DeepSeekopen

    79.2%
  7. 7
    Qwen3.5 397B-A17B

    Alibabaopen

    78.8%
  8. 8
    Gemini 3 Flash

    Googleclosed

    78.0%
  9. 9
    Qwen3.7 Plus

    Alibabaclosed

    78.0%
  10. 10
    Gemini 3.1 Flash-Lite

    Googleclosed

    77.2%
  11. 11
    DeepSeek V4 Pro

    DeepSeekopen

    76.5%
  12. 12
    GLM-5.1

    Zhipu AIopen

    76.3%
  13. 13
    Gemini 3.5 Flash

    Googleclosed

    76.3%
  14. 14
    Kimi K2.6

    Moonshotclosed

    76.0%
  15. 15
    GPT-5.5

    OpenAIclosed

    75.9%
  16. 16
    GPT-5.4 Nano

    OpenAIclosed

    75.9%
  17. 17
    MiniMax M2.7

    MiniMaxclosed

    75.7%
  18. 18
    Gemma 4 31B

    Googleopen

    75.6%
  19. 19
    GPT-5 Mini

    OpenAIclosed

    75.4%
  20. 20
    Qwen3.6 Plus

    Alibabaclosed

    75.2%
  21. 21
    Agnes-3.0-Flash Preview

    Agnes AIopen

    74.2%
  22. 22
    GPT-5.4

    OpenAIclosed

    73.9%
  23. 23
    GLM-5.2

    Zhipu AIopen

    73.3%
  24. 24
    GPT-5.4 Mini

    OpenAIclosed

    73.3%
  25. 25
    GPT-5

    OpenAIclosed

    73.1%
  26. 26
    GPT-5.6 Sol

    OpenAIclosed

    72.7%
  27. 27
    GLM-5

    Zhipu AIopen

    72.3%
  28. 28
    MiMo-V2-Flash

    Xiaomiopen

    71.8%
  29. 29
    MiniMax M2.5

    MiniMaxopen

    71.6%
  30. 30
    o3

    OpenAIclosed

    71.4%
  31. 31
    GPT-5.6 Terra

    OpenAIclosed

    71.2%
  32. 32
    Solar Pro 3

    Upstageopen

    71.2%
  33. 33
    o1

    OpenAIclosed

    70.3%
  34. 34
    Kimi K2.5

    Moonshotopen

    70.2%
  35. 35
    Mercury 2

    Inception Labsclosed

    69.8%
  36. 36
    gpt-oss-120B

    OpenAIopen

    69.0%
  37. 37
    Mistral Medium 3.5

    Mistralopen

    68.8%
  38. 38
    o4-mini

    OpenAIclosed

    68.7%
  39. 39
    GLM-4.7

    Zhipu AIopen

    67.9%
  40. 40
    GPT-5 Nano

    OpenAIclosed

    67.6%
  41. 41
    Qwen3.6 27B

    Alibabaopen

    67.6%
  42. 42
    MiMo-V2.5

    Xiaomiopen

    67.1%
  43. 43
    gpt-oss-20B

    OpenAIopen

    65.1%
  44. 44
    Step-3.5 Flash

    StepFunclosed

    64.6%
  45. 45
    Qwen3.6 35B A3B

    Alibabaopen

    64.4%
  46. 46
    Claude Fable 5

    Anthropicclosed

    63.5%
  47. 47
    Kimi K2.7 Code

    Moonshotopen

    63.1%
  48. 48
    Hunyuan 3 Preview

    Tencentclosed

    63.1%
  49. 49
    K2 Think V2

    MBZUAI / G42 / LLM360open

    62.8%
  50. 50
    Claude Opus 4.8

    Anthropicclosed

    62.2%
  51. 51
    GLM-4.7 Flash

    Zhipu AIclosed

    60.8%
  52. 52
    DeepSeek V3.2

    DeepSeekopen

    60.7%
  53. 53
    Qwen3-Next 80B-A3B

    Alibabaopen

    60.7%
  54. 54
    Claude Opus 4.7

    Anthropicclosed

    58.6%
  55. 55
    Claude Opus 4.5

    Anthropicclosed

    58.0%
  56. 56
    Ling-2.6 Flash

    InclusionAIclosed

    57.4%
  57. 57
    Ling-2.6 1T

    InclusionAIopen

    56.9%
  58. 58
    Claude Sonnet 4.6

    Anthropicclosed

    56.6%
  59. 59
    Trinity Large Thinking

    Arcee AIopen

    56.3%
  60. 60
    LFM2.5-8B-A1B

    Liquid AIopen

    55.6%
  61. 61
    Grok 4

    xAIclosed

    53.7%
  62. 62
    Claude Opus 4.6

    Anthropicclosed

    53.1%
  63. 63
    Grok 4 Fast

    xAIclosed

    50.5%
  64. 64
    Gemini 2.5 Flash

    Googleclosed

    50.3%
  65. 65
    Gemini 2.5 Flash-Lite

    Googleclosed

    49.9%
  66. 66
    OLMo 3 32B Think

    Ai2open

    49.1%
  67. 67
    Gemini 2.5 Pro

    Googleclosed

    48.7%
  68. 68
    Claude 3.7 Sonnet

    Anthropicclosed

    48.3%
  69. 69
    Mistral Small 4

    Mistralopen

    48.2%
  70. 70
    Llama 3.3 70B

    Metaopen

    47.1%
  71. 71
    Grok 3

    xAIclosed

    46.9%
  72. 72
    Ring-2.6-1T

    InclusionAIopen

    44.6%
  73. 73
    Qwen3 Max

    Alibabaclosed

    44.1%
  74. 74
    LFM2.5-1.2B-Instruct

    Liquid AIopen

    43.8%
  75. 75
    GPT-4.1

    OpenAIclosed

    43.0%
  76. 76
    Llama 4 Maverick

    Metaopen

    43.0%
  77. 77
    Seed-OSS-36B-Instruct

    ByteDance Seedopen

    41.9%
  78. 78
    DeepSeek V3.1

    DeepSeekopen

    41.5%
  79. 79
    Kimi K2

    Moonshotopen

    41.5%
  80. 80
    Qwen3 Coder 480B

    Alibabaopen

    40.5%
  81. 81
    Gemini 2.0 Flash

    Googleclosed

    40.2%
  82. 82
    Mistral Medium 3.1

    Mistralclosed

    39.8%
  83. 83
    DeepSeek R1-0528

    DeepSeekopen

    39.6%
  84. 84
    Llama 4 Scout

    Metaopen

    39.5%
  85. 85
    Olmo 3.1 32B Instruct

    Ai2open

    39.2%
  86. 86
    DeepSeek R1

    DeepSeekopen

    39.0%
  87. 87
    Llama 3.1 405B

    Metaopen

    39.0%
  88. 88
    QwQ-32B

    Alibabaopen

    38.8%
  89. 89
    Qwen3 235B-A22B

    Alibabaopen

    38.7%
  90. 90
    Granite 4.1 8B

    IBMopen

    38.6%
  91. 91
    GPT-4.1 Mini

    OpenAIclosed

    38.3%
  92. 92
    Devstral 2

    Mistralopen

    38.1%
  93. 93
    OLMo 2 32B

    Ai2open

    38.1%
  94. 94
    Llama 3 70B

    Metaopen

    37.1%
  95. 95
    Qwen2.5 72B

    Alibabaopen

    36.9%
  96. 96
    Gemma 3 12B

    Googleopen

    36.7%
  97. 97
    Command A

    Cohereclosed

    36.5%
  98. 98
    Qwen3 32B

    Alibabaopen

    36.3%
  99. 99
    Mistral Large 3

    Mistralopen

    36.2%
  100. 100
    Claude 3 Haiku

    Anthropicclosed

    36.1%
  101. 101
    DeepSeek V3

    DeepSeekopen

    34.8%
  102. 102
    Devstral Small

    Mistralopen

    34.6%
  103. 103
    Pixtral Large

    Mistralclosed

    34.5%
  104. 104
    Llama 3.1 70B

    Metaopen

    34.4%
  105. 105
    INTELLECT-3

    Prime Intellectopen

    34.0%
  106. 106
    Mistral Small 3.2

    Mistralopen

    33.5%
  107. 107
    GPT-4

    OpenAIclosed

    33.2%
  108. 108
    Olmo 3 7B Instruct

    Ai2open

    32.8%
  109. 109
    Qwen3-Coder 30B-A3B

    Alibabaopen

    32.7%
  110. 110
    GPT-4.1 Nano

    OpenAIclosed

    32.0%
  111. 111
    Ministral 3 14B

    Mistralopen

    32.0%
  112. 112
    Gemma 3 27B

    Googleopen

    31.8%
  113. 113
    Mistral Large 2

    Mistralclosed

    31.2%
  114. 114
    Qwen3-Omni-30B-A3B-Instruct

    Alibabaopen

    31.2%
  115. 115
    GPT-4o Mini

    OpenAIclosed

    31.0%
  116. 116
    Llama-3.1-Nemotron-70B Instruct

    NVIDIAopen

    30.7%
  117. 117
    Llama 3.2 11B Vision

    Metaopen

    30.4%
  118. 118
    Reka Flash 3

    Rekaopen

    30.4%
  119. 119
    Ministral 3 8B

    Mistralopen

    29.1%
  120. 120
    Llama 3.1 8B

    Metaopen

    28.6%
  121. 121
    Gemma 3 4B

    Googleopen

    28.3%
  122. 122
    Kimi Linear 48B A3B Instruct

    Moonshotopen

    28.1%
  123. 123
    DeepSeek-R1-Distill-Llama-70B

    DeepSeekopen

    27.6%
  124. 124
    Ministral 3 3B

    Mistralopen

    26.8%
  125. 125
    Llama 3.2 3B

    Metaopen

    26.2%
  126. 126
    Apertus 70B Instruct

    Swiss AI Initiativeopen

    25.9%
  127. 127
    Granite 4.0 Micro

    IBMopen

    24.8%
  128. 128
    Llama 3 8B

    Metaopen

    24.6%
  129. 129
    Phi-3 Mini 3.8B

    Microsoftopen

    23.9%
  130. 130
    Phi-4

    Microsoftopen

    23.5%
  131. 131
    DeepSeek-R1-Distill-Qwen-32B

    DeepSeekopen

    22.9%
  132. 132
    Llama 3.2 1B

    Metaopen

    22.8%
  133. 133
    Apertus 8B Instruct

    Swiss AI Initiativeopen

    22.4%
  134. 134
    DeepSeek-R1 Distill Qwen 14B

    DeepSeekopen

    22.1%
  135. 135
    Phi-4-mini

    Microsoftopen

    21.1%
  136. 136
    Mistral 7B

    Mistralopen

    19.9%
  137. 137
    DeepSeek-R1 Distill Llama 8B

    DeepSeekopen

    17.6%

Back to all benchmarks