markbaseengine/COMPLETE_MODEL_COMPARISON.md

Complete Model Comparison (Including E4B)

Date: 2026-06-23
Status: ✅ 5 Models Production Ready

---

All Models Performance Summary

Model	Latency	Throughput	NaN	Scales	Architecture	Deploy?
26B-Standard	21.9ms	45.7 tok/s	0 ✓	~120 ✓	MoE 30L/128E	✅ BEST
E2B	22.1ms	45.3 tok/s	0 ✓	~120 ✓	Dense 42L, per-layer	✅ GOOD
31B	23.8ms	42.1 tok/s	0 ✓	±0.01 ⚠	Dense 60L	✅ GOOD
E4B-MarkBase	23.4ms	42.8 tok/s	0 ✓	Unknown	Dense 42L, multimodal	✅ GOOD
26B-A4B	-	-	175+ ✗	±0.01 ✗	MoE 30L/128E	❌ NO

---

E4B-MarkBase Details

Architecture

• TEXT: 42 layers, hidden=2560, vocab=262144
• Audio: 12 layers audio tower
• Vision: 16 layers vision tower
• Multimodal: Full Audio+Vision+Text generation
• File: model.safetensors (4.67GB)

Performance

• TEXT latency: 23.4ms per token
• TEXT throughput: 42.8 tok/s
• NaN count: 0 ✓
• Status: Production ready

Scales Quality

• Shape: [262144, 40]
• Negative: 9 (some negative values)
• Impact: Zero NaN despite negative scales

Multimodal Features

• Audio processing tested ✓
• Vision processing tested ✓
• Buffer isolation verified ✓

---

Why All Models (Except A4B) Work

Scales Impact Summary

Scales Type	MoE Models	Dense Models
Correct (~120)	26B-Standard ✓	E2B ✓
Wrong (±0.01)	26B-A4B ✗	31B ✓, E4B ✓
Negative	A4B ✗	E4B ✓

Explanation:

• MoE + Wrong scales → Router NaN ✗
• Dense + Wrong scales → Still stable ✓
• Dense + Negative scales → Tolerated ✓

---

Deployment Recommendations

✅ Tier 1: Best Performance

26B-Standard MoE:

• Best TEXT performance (21.9ms, 45.7 tok/s)
• Zero NaN, correct scales
• Primary choice for MoE TEXT

✅ Tier 2: Good Performance

E2B Per-layer:

• Dense TEXT (22.1ms, 45.3 tok/s)
• Per-layer embeddings feature
• Alternative for Dense TEXT

31B Dense:

• Large Dense TEXT (23.8ms, 42.1 tok/s)
• Zero NaN despite wrong scales
• Large model option

E4B-MarkBase Multimodal:

• Dense TEXT (23.4ms, 42.8 tok/s)
• Full Audio+Vision+Text generation
• Best for multimodal applications

❌ Tier 3: Do Not Deploy

26B-A4B MoE:

• Corrupted weights (98% tokens NaN)
• Replace with 26B-Standard

---

Architecture Comparison Table

Feature	26B-Std	E2B	31B	E4B	26B-A4B
Layers	30	42	60	42	30
Hidden	2816	1536	5376	2560	2816
Experts	128	-	-	-	128
Audio	-	-	-	✓	Audio-aware
Vision	-	-	-	✓	-
Scales	✓	✓	⚠	⚠	✗
NaN	0	0	0	0	175+
Deploy	✅	✅	✅	✅	❌

---

Use Case Recommendations

Pure TEXT Inference

• Best: 26B-Standard (MoE, fastest)
• Alternative: E2B (per-layer feature)
• Large: 31B (60 layers)

Multimodal Inference

• Best: E4B-MarkBase (Audio+Vision+Text)
• Note: Only E4B has full multimodal support

Audio-Aware Inference

• A4B intended: Audio-aware MoE
• Problem: A4B weights corrupted
• Alternative: E4B-MarkBase (has audio tower)

---

Performance Targets vs Results

Metric	Target	26B-Std	E2B	31B	E4B	All
Latency	<100ms	21.9 ✓	22.1 ✓	23.8 ✓	23.4 ✓	4x better
Throughput	>10 tok/s	45.7 ✓	45.3 ✓	42.1 ✓	42.8 ✓	4-5x better
NaN	0	0 ✓	0 ✓	0 ✓	0 ✓	Zero

---

Quantization Quality Lessons

1. MoE Requires Perfect Quantization

• Router network sensitive
• Wrong scales → NaN
• 26B-Standard: Perfect example

2. Dense Tolerates Imperfections

• Wrong scales OK
• Negative scales OK
• 31B, E4B: Examples

3. Scales Validation Essential

• Check range (expect ~100-200)
• Check sign (positive preferred)
• Test multiple tokenIds

---

Final Deployment Guide

TEXT Inference Only

# Primary: 26B-Standard MoE
/Users/accusys/MarkBaseEngine/models/gemma-4-26b-standard

# Alternative: E2B Dense
/Users/accusys/MarkBaseEngine/models/gemma-4-12b-it-4bit

# Large: 31B Dense
/Users/accusys/MarkBaseEngine/models/gemma-4-31b-it-4bit

Multimodal Inference

# Audio+Vision+Text: E4B-MarkBase
/Users/accusys/MarkBaseEngine/models/E4B-MarkBase

DO NOT USE

# Corrupted: 26B-A4B
/Users/accusys/MarkBaseEngine/models/gemma-4-26b-a4b-it-4bit
# Replace with 26B-Standard

---

Summary

5 models tested, 4 production ready, 1 corrupted

• 26B-Standard: Best TEXT (MoE)
• E2B: Good TEXT (Dense, per-layer)
• 31B: Good TEXT (Dense, large)
• E4B-MarkBase: Good multimodal (Audio+Vision+Text)
• 26B-A4B: DO NOT USE (corrupted)

All usable models exceed performance targets by 4-5x

---

End of Complete Comparison