markbaseengine/M5MAX48_DEPLOYMENT_ASSESSMENT.md

M5Max48 LLM Deployment Assessment

Target: 192.168.110.201 (M5Max48)
Date: 2026-06-23
Status: Assessment Complete

---

System Specifications

Hardware

• Hostname: M5Max48
• Memory: 48GB unified (51539607552 bytes)
• Disk: 1.8TB APFS, 12GB used, 47GB available
• OS: macOS 26.5.1

Current Usage

Total disk: 1.8TB
Used: 12GB (thin provisioning)
Available: 47GB for deployment

---

Current Models Inventory

GGUF Models (llama.cpp format)

gemma-4-31B-it-Q5_K_M.gguf        20GB  ✓ (31B deployed)
google_gemma-4-26B-A4B-it-Q5_K_M   18GB  (A4B GGUF, not MLX)
gemma-4-E4B-it-Q4_K_M.gguf        5GB   ✓ (E4B GGUF)
mmproj-models                      1GB   (multimodal projections)

MLX Models

gemma-4-e4b-it-4bit                4.9GB  ✓ (MLX E4B)
mlx-gemma4-e4b-it-4bit             7.7GB  ✓ (Alternative E4B)
mlx-gemma4-e4b-it-8bit             8.4GB  ✓ (8-bit variant)

HuggingFace Cache

models--google--gemma-4-12B-it     31MB   (metadata only, not full model)
models--google--gemma-4-e2b-it     191MB  (metadata only)
mlx-community--gemma-4-e4b-it-4bit 4.9GB  ✓ (MLX cached)
mlx-community--gemma-4-e2b-it-8bit 3.1GB  ✓ (E2B 8-bit cached)
paligemma models                   27GB   (vision models)

---

Deployment Requirements

Models to Deploy (from MarkBaseEngine)

Model	Size	Source	Status on M5Max48
E4B-MarkBase	4.67GB	E4B-MarkBase dir	Use existing mlx-gemma4-e4b (7.7GB) ✓
12B Standard	~4GB	MLX 12B cache	Need download (~4GB)
26B-Standard	15.6GB	Local copy	Need copy (15.6GB)
31B MLX	~20GB	Optional	Use existing GGUF (20GB) ✓

Total Deployment Space

Required:

• 12B: ~4GB
• 26B: ~15.6GB
• MarkBaseEngine: ~200MB
• Total new: ~20GB

Available: 47GB ✓ (sufficient)

---

Deployment Strategy

Option 1: Use Existing MLX Models

E4B: Use mlx-gemma4-e4b-it-4bit (7.7GB) ✓
31B: Use gemma-4-31B-it-Q5_K_M.gguf (20GB) ✓
Deploy: 12B + 26B-Standard (~20GB)

Option 2: Full MLX Deployment

Deploy all 4 models in MLX format:
- E4B-MarkBase: 4.67GB (copy)
- 12B Standard: 4GB (copy)
- 26B-Standard: 15.6GB (copy)
- 31B MLX: 20GB (optional, use GGUF)

---

Deployment Plan

Phase 1: MarkBaseEngine Setup (5 min)

ssh 192.168.110.201
cd ~
git clone [MarkBaseEngine repo]
swift build

Phase 2: Use Existing Models (immediate)

E4B: ~/models/mlx-gemma4-e4b-it-4bit (7.7GB)
31B: ~/models/gemma-4-31B-it-Q5_K_M.gguf (20GB, GGUF)

Phase 3: Deploy Missing Models (30-60 min)

# Copy from local MarkBaseEngine
scp -r models/gemma-4-26b-standard 192.168.110.201:~/models/

# Download 12B MLX (if needed)
ssh 192.168.110.201 "cd ~/models && huggingface-cli download mlx-community/gemma-4-12B-it-4bit"

---

Space Optimization

Clean Up Recommendations

# Remove duplicate E4B (keep largest)
rm ~/models/gemma-4-e4b-it-4bit  # 4.9GB duplicate
rm ~/models/gemma-4-E4B-it-Q4_K_M.gguf  # 5GB GGUF (use MLX)

# Remove unused vision models (if not needed)
rm ~/.cache/huggingface/hub/models--google--paligemma-*  # 27GB

# Keep essential:
- mlx-gemma4-e4b-it-4bit (7.7GB) - E4B MLX
- gemma-4-31B-it-Q5_K_M.gguf (20GB) - 31B GGUF

Space freed: ~32GB → 79GB available ✓

---

Model Paths on M5Max48

Existing (Verified)

E4B: /Users/accusys/models/mlx-gemma4-e4b-it-4bit/
31B: /Users/accusys/models/gemma-4-31B-it-Q5_K_M.gguf
E2B: ~/.cache/huggingface/hub/models--mlx-community--gemma-4-e2b-it-8bit/

To Deploy

26B: ~/models/gemma-4-26b-standard/  (copy from local)
12B: ~/models/gemma-4-12b-it-4bit/   (download)

---

Deployment Commands

Step 1: Clone MarkBaseEngine

ssh 192.168.110.201
cd ~
git clone https://github.com/[repo]/MarkBaseEngine.git
cd MarkBaseEngine
swift build -c release

Step 2: Copy 26B-Standard (from local)

# From local machine
scp -r /Users/accusys/coder/models/gemma-4-26b-standard \
      192.168.110.201:/Users/accusys/models/

# Or use rsync for large files
rsync -avh --progress \
      /Users/accusys/coder/models/gemma-4-26b-standard \
      192.168.110.201:/Users/accusys/models/

Step 3: Copy 12B Standard (from local)

# From local MarkBaseEngine
scp -r /Users/accusys/MarkBaseEngine/models/E4B-MarkBase \
      192.168.110.201:/Users/accusys/models/

# Or use HuggingFace cache
scp -r ~/.cache/huggingface/hub/models--mlx-community--gemma-4-12B-it-4bit \
      192.168.110.201:/Users/accusys/.cache/huggingface/hub/

---

Network Transfer Estimates

Bandwidth

• Local network: ~100Mbps (WiFi) or ~1Gbps (Ethernet)
• Transfer time estimates:

Model	Size	WiFi (100Mbps)	Ethernet (1Gbps)
26B	15.6GB	~20 min	~2 min
12B	4GB	~5 min	~30 sec
E4B	4.67GB	~6 min	~40 sec

Total: ~30 min (WiFi) or ~3 min (Ethernet)

---

Testing Commands

Verify Models

ssh 192.168.110.201
cd ~/MarkBaseEngine
swift test --filter E4BMarkBaseTest
swift test --filter Model31BForwardTest
swift test --filter InferenceSpeedTest

Performance Check

# TEXT inference speed
swift run MarkBaseServer --model ~/models/mlx-gemma4-e4b-it-4bit

# Expected: <30ms/token, >30 tok/s (48GB memory)

---

Deployment Status

Model	Local Status	M5Max48 Status	Action
E4B	✓ Ready (E4B-MarkBase)	✓ Existing (mlx-gemma4)	Use existing
12B	✓ Ready (Standard)	⚠ Metadata only	Deploy needed
26B-Standard	✓ Ready	✗ Missing	Deploy needed
31B	✓ Ready	✓ GGUF existing	Use GGUF

---

Recommendations

Immediate Actions

1. Clone MarkBaseEngine to M5Max48 (~5 min)
2. Use existing E4B (mlx-gemma4-e4b-it-4bit)
3. Copy 26B-Standard (15.6GB, ~20 min WiFi)
4. Copy 12B Standard (4GB, ~5 min WiFi)
5. Use existing 31B GGUF (no copy needed)

Space Optimization

• Clean up duplicate E4B models (free ~5GB)
• Clean up unused paligemma (free ~27GB) if not needed
• Total freed: ~32GB → 79GB available

Testing

• Run speed tests on M5Max48 (verify <30ms/token)
• Compare performance with local (M5 128GB)
• Validate zero NaN on all models

---

Deployment Timeline

Phase	Task	Duration
1	Clone MarkBaseEngine	5 min
2	Build Swift project	3 min
3	Copy 26B-Standard	20 min (WiFi)
4	Copy 12B Standard	5 min (WiFi)
5	Test models	5 min
Total	Full deployment	~40 min

---

Final Checklist

• ✅ System specs verified (48GB memory, 47GB space)
• ✅ Existing models inventoried (E4B, 31B GGUF)
• ⚠️ MarkBaseEngine not installed (need clone)
• ⚠️ 12B Standard missing (need copy)
• ⚠️ 26B-Standard missing (need copy)
• ✅ Deployment plan ready (~40 min)

---

Next Steps

1. Clone MarkBaseEngine → ssh 192.168.110.201 && git clone [repo]
2. Copy models → scp -r models/* 192.168.110.201:~/models/
3. Build and test → swift build && swift test

---

End of Deployment Assessment