vs
13:2
overall capability wins (2 ties across 17 scored). Gemma 3 12B leads overall.
- Reasoning8:1
- Coding2:1
- Arena1:0
- Tool use1:0
- Pricing0:3
- Speed0:2
- Specs1:0
Verdict
Gemma 3 12B leads coding (widest gap: +12.0 on LiveCode); Gemma 3 12B ranks higher on LMArena (+39 Elo); Gemma 3 4B is ~1.5x cheaper on a blended token basis than Gemma 3 12B.
- Gemma 3 12B leads coding (widest gap: +12.0 on LiveCode)
- Gemma 3 12B ranks higher on LMArena (+39 Elo)
- Gemma 3 4B is ~1.5x cheaper on a blended token basis than Gemma 3 12B
Cost for 1M in + 250K out
Illustrative chat workload at primary-provider list prices.
- Gemma 3 12B
- $0.09
- Gemma 3 4B
- $0.06
Gemma 3 4B is about 1.5x cheaper on this workload.
Price per 1M tokens
Full comparison
Organization
Family
Gemma
Gemma
License
Gemma
Gemma
Open weights
Yes
Yes
Release
Mar 12, 2025
Mar 12, 2025
Knowledge cutoff
-
-
API / provider
Google AI / self-host
Together / Fireworks (ref.)
Modalities
text, image → text
text, image → text
Specs
Context window
128K
128K
Max output
8K
8K
Parameters
12B
4B
Pricing
Input $/1M
$0.05
$0.04
Output $/1M
$0.15
$0.08
Blended $/1M (3∶1)
$0.08
$0.05
Speed
tok/s
120
160
TTFT (s)
0.22
0.18
Reasoning
MMLU-Pro
60.6
43.6
GPQA Diamond
34.9
29.1
Humanity's Last Exam
4.2
5.3
AIME 2025
18.3
12.7
MATH-500
85.3
76.6
Humanity's Last Exam (with tools)
—
—
AA-Omniscience Accuracy
10.3
7.7
AA-LCR v1.1
8.3
6.7
CritPt
—
—
MMMU-Pro
37.5
29.9
IFBench
36.7
28.3
Chartography
—
—
Chartography (With Tools)
—
—
Coding
SWE-bench Verified
—
—
SWE-bench Pro
—
—
SWE-bench Multilingual
—
—
LiveCodeBench
24.6
12.6
Terminal-Bench 2.1
0.0
0.4
Aider Polyglot
—
—
Terminal-Bench 3
—
—
BigCodeBench
—
—
SciCode
16.4
7.3
CursorBench
—
—
SWE-Rebench
—
—
NL2Repo-Bench
—
—
DeepSWE
—
—
WebDev Arena
—
—
Terminal-Bench 4.0
—
—
CursorBench 4.0
—
—
FrontierCode v1.1 (Main)
—
—
Arena
LMArena Elo
1,342
1,303
| Metric | Gemma 3 12B | Gemma 3 4B | Delta |
|---|---|---|---|
| Identity | |||
| Organization | — | ||
| Family | Gemma | Gemma | — |
| License | Gemma | Gemma | — |
| Open weights | Yes | Yes | — |
| Release | Mar 12, 2025 | Mar 12, 2025 | — |
| Knowledge cutoff | - | - | — |
| API / provider | Google AI / self-host | Together / Fireworks (ref.) | — |
| Modalities | text, image → text | text, image → text | — |
| Specs | |||
| Context window | 128K | 128K | tie |
| Max output | 8K | 8K | tie |
| Parameters | 12B | 4B | A +8.00 |
| Pricing | |||
| Input $/1M | $0.05 | $0.04 | B +$0.01 |
| Output $/1M | $0.15 | $0.08 | B +$0.07 |
| Blended $/1M (3∶1) | $0.08 | $0.05 | B +$0.03 |
| Speed | |||
| tok/s | 120 | 160 | B +40 |
| TTFT (s) | 0.22 | 0.18 | B +0.04 |
| Reasoning | |||
| MMLU-Pro | 60.6 | 43.6 | A +17.0 pts |
| GPQA Diamond | 34.9 | 29.1 | A +5.8 pts |
| Humanity's Last Exam | 4.2 | 5.3 | B +1.1 pts |
| AIME 2025 | 18.3 | 12.7 | A +5.6 pts |
| MATH-500 | 85.3 | 76.6 | A +8.7 pts |
| Humanity's Last Exam (with tools) | — | — | — |
| AA-Omniscience Accuracy | 10.3 | 7.7 | A +2.6 pts |
| AA-LCR v1.1 | 8.3 | 6.7 | A +1.6 pts |
| CritPt | — | — | — |
| MMMU-Pro | 37.5 | 29.9 | A +7.6 pts |
| IFBench | 36.7 | 28.3 | A +8.4 pts |
| Chartography | — | — | — |
| Chartography (With Tools) | — | — | — |
| Coding | |||
| SWE-bench Verified | — | — | — |
| SWE-bench Pro | — | — | — |
| SWE-bench Multilingual | — | — | — |
| LiveCodeBench | 24.6 | 12.6 | A +12.0 pts |
| Terminal-Bench 2.1 | 0.0 | 0.4 | B +0.4 pts |
| Aider Polyglot | — | — | — |
| Terminal-Bench 3 | — | — | — |
| BigCodeBench | — | — | — |
| SciCode | 16.4 | 7.3 | A +9.1 pts |
| CursorBench | — | — | — |
| SWE-Rebench | — | — | — |
| NL2Repo-Bench | — | — | — |
| DeepSWE | — | — | — |
| WebDev Arena | — | — | — |
| Terminal-Bench 4.0 | — | — | — |
| CursorBench 4.0 | — | — | — |
| FrontierCode v1.1 (Main) | — | — | — |
| Arena | |||
| LMArena Elo | 1,342 | 1,303 | A +39 Elo |
Gemma 3 12B
- Context
- 128K / 8K out
- Parameters
- 12B
- Price
- $0.05 / $0.15
- Speed
- 120 tok/s · 0.22s TTFT
- Modalities
- text, image → text
- License
- Gemma
Compact open Gemma 3 instruct model for edge and low-cost multimodal chat.
Gemma 3 4B
- Context
- 128K / 8K out
- Parameters
- 4B
- Price
- $0.04 / $0.08
- Speed
- 160 tok/s · 0.18s TTFT
- Modalities
- text, image → text
- License
- Gemma
Small multimodal Gemma 3 with long context — strong on-device and edge deployment option.
Benchmark charts
Winner bars are emphasized. Per-benchmark deltas sit above each chart.