warren/markbaseengine
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity
Files
main
markbaseengine/E4B_VS_12B_CORRECTED_COMPARISON.md
T
MarkBase Admin ac75faa0cc
CI / build-and-test (push) Has been cancelled
Details
Initial commit: E4B-MarkBase model integration with passing tests 
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully
- All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance)
- All stress tests passed (5/5 in 127.6s)
  - Concurrent inference
  - Memory stress (67.5 tok/s, 0 NaN)
  - Continuous generation
  - Batch processing
  - Long-running stability
- Swift Metal inference engine with multimodal support
2026-06-23 18:12:35 +08:00

7.3 KiB
Raw Permalink Blame History

E4B vs 12B Corrected Comparison (Multimodal Both!)

Date: 2026-06-23
Correction: 12B Standard HAS Audio + Vision capabilities

Critical Finding

Both E4B and 12B Standard are Multimodal Models!

Model Vision Embedder Embed Vision Audio Embed TEXT Type
E4B-MarkBase ✓ 16L, 439 tensors ✓ ✓ 12L, 513 tensors ✓ 42L Full Multimodal
12B Standard ✓ 11 tensors ✓ 3 tensors ✓ 3 tensors ✓ 42L Multimodal
E2B ✗ ✗ ✗ ✓ 48L, per-layer TEXT only

12B Standard Architecture (Corrected)

Vision Tower

Vision Embedder: 11 tensors
  - patch_dense.weight: [3840, 864] (quantized, u32)
  - patch_dense.scales: [3840, 108]
  - patch_dense.biases: [3840, 108]
  - patch_dense.bias: [3840]
  - patch_ln1.weight/bias: Layer norm
  - patch_ln2.weight/bias: Layer norm
  - pos_embedding: [1120, 2, 3840]
  - pos_norm.weight/bias: Position norm

Embed Vision: 3 tensors
  - embedding_projection.weight: [3840, 480] (quantized)
  - embedding_projection.scales: [3840, 60]
  - embedding_projection.biases: [3840, 60]

Output: 3840 → TEXT hidden size

Audio Tower

Embed Audio: 3 tensors
  - embedding_projection.weight: [3840, 80] (quantized)
  - embedding_projection.scales: [3840, 10]
  - embedding_projection.biases: [3840, 10]

Output: 3840 → TEXT hidden size

TEXT Model

TEXT Layers: ~42
Hidden: 2560 (TEXT model, not 3840)
Vocab: 262144
Embed tokens: [262144, 480] (quantized)
Tensors: 1324

E4B-MarkBase Architecture

Vision Tower

Vision Tower: 16 layers, 439 tensors
Hidden: 768
Patch size: 16
Image size: 224
Output: 1536 → TEXT hidden (2560)

Audio Tower

Audio Tower: 12 layers, 513 tensors
Hidden: 1024
Output: 1536 → TEXT hidden (2560)

TEXT Model

TEXT Layers: 42
Hidden: 2560
Vocab: 262144
Intermediate: 10240

Multimodal Comparison

Vision Architecture

Feature E4B 12B Standard
Layers 16L Patch-based (no deep layers)
Hidden 768 3840 (larger)
Tensors 439 11 (embedder) + 3 (projection)
Complexity Full transformer Simplified patch embedder
Output 1536 → TEXT 3840 → TEXT

Audio Architecture

Feature E4B 12B Standard
Layers 12L Embedder only (no layers)
Hidden 1024 3840
Tensors 513 3
Complexity Full audio encoder Simple projection
Output 1536 → TEXT 3840 → TEXT

Complexity Comparison

E4B: Full multimodal towers (16L vision, 12L audio)

  • More sophisticated processing
  • Deeper encoders
  • Better feature extraction

12B Standard: Lightweight multimodal

  • Simplified vision (patch embedder)
  • Simple audio projection
  • Less computation overhead

TEXT Performance Comparison

E4B TEXT

Layers: 42
Hidden: 2560
Performance: 25.6-26.7ms, 37.5-39.1 tok/s
NaN: 0 ✓

12B Standard TEXT

Layers: ~42
Hidden: ~2560 (TEXT portion)
Performance: Similar expected
Load successful: ✓

File Size Comparison

Model TEXT Size Vision Size Audio Size Total
E4B ~3GB ~0.5GB (439 tensors) ~0.5GB (513 tensors) ~4.67GB
12B Std ~3.5GB ~11 tensors ~3 tensors ~4GB

Observation: E4B has larger multimodal towers (more tensors)

Use Case Recommendations

Complex Multimodal Tasks

Winner: E4B-MarkBase

  • Full vision transformer (16L)
  • Full audio encoder (12L)
  • Better feature extraction
  • Suitable for:
    • Complex image understanding
    • Audio analysis
    • High-quality multimodal generation

Lightweight Multimodal Tasks

Winner: 12B Standard

  • Efficient vision embedder
  • Simple audio projection
  • Less overhead
  • Suitable for:
    • Basic image embedding
    • Simple audio processing
    • Performance-focused applications

Pure TEXT Tasks

Winner: Either (both similar TEXT architecture)

  • E4B: 42L, 2560 hidden, zero NaN ✓
  • 12B Std: 42L, ~2560 hidden, load successful ✓

Per-layer Feature Needed

Winner: E2B (TEXT only variant)

  • Unique per-layer embeddings
  • No audio/vision
  • Specialized use

Architecture Efficiency

E4B-MarkBase

Multimodal Towers:
  Vision: 16L, 439 tensors (comprehensive)
  Audio: 12L, 513 tensors (comprehensive)
  
TEXT Core:
  Layers: 42
  Hidden: 2560
  
Strength: Rich multimodal features
Weakness: More computation

12B Standard

Multimodal Embedders:
  Vision: 11 tensors (efficient)
  Audio: 3 tensors (minimal)
  
TEXT Core:
  Layers: 42
  Hidden: ~2560
  
Strength: Efficient multimodal
Weakness: Simpler features

Deployment Recommendations

Primary Multimodal: E4B-MarkBase

Use for:
  - High-quality vision processing
  - Deep audio analysis
  - Complex multimodal generation
  
Performance:
  - TEXT: 25-27ms, zero NaN
  - Vision: 82ms load
  - Audio: 89ms load

Efficient Multimodal: 12B Standard

Use for:
  - Basic vision embedding
  - Simple audio features
  - Lightweight multimodal apps
  
Performance:
  - TEXT: Expected ~25-30ms
  - Vision: Simple embedder (fast)
  - Audio: Simple projection (fast)

TEXT Only: Either E4B or 12B

Both have similar TEXT architecture
E4B verified zero NaN
12B load successful

Total Model Count (Updated)

Model TEXT Audio Vision Per-layer Status
E4B ✓ ✓ (Full) ✓ (Full) ✗ Multimodal ✓
12B Std ✓ ✓ (Lite) ✓ (Lite) ✗ Multimodal ✓
E2B ✓ ✗ ✗ ✓ TEXT+per-layer
26B-Std ✓ ✗ ✗ ✗ MoE TEXT ✓
31B ✓ ✗ ✗ ✗ Dense TEXT ✓
26B-A4B ? ? ? ✗ Corrupted ✗

Multimodal Models: E4B + 12B Standard (both!)

Corrected Summary

Both E4B and 12B Standard are multimodal!

E4B Advantages:

  1. Full vision transformer (16L, 439 tensors)
  2. Full audio encoder (12L, 513 tensors)
  3. Better feature extraction
  4. Verified zero NaN
  5. TEXT performance tested (25-27ms)

12B Standard Advantages:

  1. Efficient vision embedder (11 tensors)
  2. Lightweight audio projection (3 tensors)
  3. Less computation overhead
  4. Faster multimodal processing
  5. Compact architecture

Recommendations:

  • Complex multimodal → E4B (full towers)
  • Lightweight multimodal → 12B Standard (efficient)
  • TEXT only → Either (both similar)

Test Evidence

12B Vision Weights Check

vision_embedder: 11 tensors ✓
embed_vision: 3 tensors ✓
embed_audio: 3 tensors ✓
Vision Capability: YES ✓
Audio Capability: YES ✓

E4B Multimodal Verified

Audio tower: 12L, 513 tensors ✓
Vision tower: 16L, 439 tensors ✓
TEXT: 42L, 2560 hidden, zero NaN ✓

Lessons Learned

Search Keywords Matter:

  • ❌ "audio_tower", "vision_tower" (missed 12B)
  • ✓ "vision_embedder", "embed_vision", "embed_audio" (found 12B)

Architecture Variety:

  • E4B: Full transformer towers (16L/12L)
  • 12B: Lightweight embedders (11/3 tensors)

Multimodal Spectrum:

  • Full: E4B (comprehensive)
  • Lite: 12B Standard (efficient)
  • None: E2B, 26B-Std, 31B (TEXT only)

End of Corrected Comparison

Reference in New Issue View Git Blame Copy Permalink