vs
0:3
overall capability wins (1 ties across 4 scored). Gemma 3 4B leads overall.
- Arena0:1
- Pricing2:0
- Speed2:0
- Specs0:2
Verdict
Gemma 3 4B ranks higher on LMArena (+103 Elo); Gemma 3 4B offers ~15.6x larger context (128K vs 8K); Gemma 2 2B is ~1.3x cheaper on a blended token basis than Gemma 3 4B.
- Gemma 3 4B ranks higher on LMArena (+103 Elo)
- Gemma 3 4B offers ~15.6x larger context (128K vs 8K)
- Gemma 2 2B is ~1.3x cheaper on a blended token basis than Gemma 3 4B
Cost for 1M in + 250K out
Illustrative chat workload at primary-provider list prices.
- Gemma 2 2B
- $0.05
- Gemma 3 4B
- $0.06
Price per 1M tokens
Full comparison
Organization
Family
Gemma
Gemma
License
Gemma
Gemma
Open weights
Yes
Yes
Release
Jul 31, 2024
Mar 12, 2025
Knowledge cutoff
-
-
API / provider
Groq / Together (ref.)
Together / Fireworks (ref.)
Modalities
text → text
text, image → text
Specs
Context window
8K
128K
Max output
8K
8K
Parameters
2B
4B
Pricing
Input $/1M
$0.04
$0.04
Output $/1M
$0.04
$0.08
Blended $/1M (3∶1)
$0.04
$0.05
Speed
tok/s
200
160
TTFT (s)
0.12
0.18
Reasoning
MMLU-Pro
—
43.6
GPQA Diamond
—
29.1
Humanity's Last Exam
—
5.3
AIME 2025
—
12.7
MATH-500
—
76.6
Humanity's Last Exam (with tools)
—
—
AA-Omniscience Accuracy
—
7.7
AA-LCR v1.1
—
6.7
CritPt
—
—
MMMU-Pro
—
29.9
IFBench
—
28.3
Chartography
—
—
Chartography (With Tools)
—
—
Coding
SWE-bench Verified
—
—
SWE-bench Pro
—
—
SWE-bench Multilingual
—
—
LiveCodeBench
—
12.6
Terminal-Bench 2.1
—
0.4
Aider Polyglot
—
—
Terminal-Bench 3
—
—
BigCodeBench
—
—
SciCode
—
7.3
CursorBench
—
—
SWE-Rebench
—
—
NL2Repo-Bench
—
—
DeepSWE
—
—
WebDev Arena
—
—
Terminal-Bench 4.0
—
—
CursorBench 4.0
—
—
FrontierCode v1.1 (Main)
—
—
Arena
LMArena Elo
1,200
1,303
| Metric | Gemma 2 2B | Gemma 3 4B | Delta |
|---|---|---|---|
| Identity | |||
| Organization | — | ||
| Family | Gemma | Gemma | — |
| License | Gemma | Gemma | — |
| Open weights | Yes | Yes | — |
| Release | Jul 31, 2024 | Mar 12, 2025 | — |
| Knowledge cutoff | - | - | — |
| API / provider | Groq / Together (ref.) | Together / Fireworks (ref.) | — |
| Modalities | text → text | text, image → text | — |
| Specs | |||
| Context window | 8K | 128K | B +120K |
| Max output | 8K | 8K | tie |
| Parameters | 2B | 4B | B +2.00 |
| Pricing | |||
| Input $/1M | $0.04 | $0.04 | tie |
| Output $/1M | $0.04 | $0.08 | A +$0.04 |
| Blended $/1M (3∶1) | $0.04 | $0.05 | A +$0.01 |
| Speed | |||
| tok/s | 200 | 160 | A +40 |
| TTFT (s) | 0.12 | 0.18 | A +0.06 |
| Reasoning | |||
| MMLU-Pro | — | 43.6 | — |
| GPQA Diamond | — | 29.1 | — |
| Humanity's Last Exam | — | 5.3 | — |
| AIME 2025 | — | 12.7 | — |
| MATH-500 | — | 76.6 | — |
| Humanity's Last Exam (with tools) | — | — | — |
| AA-Omniscience Accuracy | — | 7.7 | — |
| AA-LCR v1.1 | — | 6.7 | — |
| CritPt | — | — | — |
| MMMU-Pro | — | 29.9 | — |
| IFBench | — | 28.3 | — |
| Chartography | — | — | — |
| Chartography (With Tools) | — | — | — |
| Coding | |||
| SWE-bench Verified | — | — | — |
| SWE-bench Pro | — | — | — |
| SWE-bench Multilingual | — | — | — |
| LiveCodeBench | — | 12.6 | — |
| Terminal-Bench 2.1 | — | 0.4 | — |
| Aider Polyglot | — | — | — |
| Terminal-Bench 3 | — | — | — |
| BigCodeBench | — | — | — |
| SciCode | — | 7.3 | — |
| CursorBench | — | — | — |
| SWE-Rebench | — | — | — |
| NL2Repo-Bench | — | — | — |
| DeepSWE | — | — | — |
| WebDev Arena | — | — | — |
| Terminal-Bench 4.0 | — | — | — |
| CursorBench 4.0 | — | — | — |
| FrontierCode v1.1 (Main) | — | — | — |
| Arena | |||
| LMArena Elo | 1,200 | 1,303 | B +103 Elo |
Gemma 2 2B
- Context
- 8K / 8K out
- Parameters
- 2B
- Price
- $0.04 / $0.04
- Speed
- 200 tok/s · 0.12s TTFT
- Modalities
- text → text
- License
- Gemma
Tiny open Gemma 2 instruct — ultra-cheap edge and high-throughput classification baseline.
Gemma 3 4B
- Context
- 128K / 8K out
- Parameters
- 4B
- Price
- $0.04 / $0.08
- Speed
- 160 tok/s · 0.18s TTFT
- Modalities
- text, image → text
- License
- Gemma
Small multimodal Gemma 3 with long context — strong on-device and edge deployment option.
Benchmark charts
Winner bars are emphasized. Per-benchmark deltas sit above each chart.