# M5Max48 LLM Deployment Assessment

**Target**: 192.168.110.201 (M5Max48)  
**Date**: 2026-06-23  
**Status**: Assessment Complete

---

## System Specifications

### Hardware
- **Hostname**: M5Max48
- **Memory**: 48GB unified (51539607552 bytes)
- **Disk**: 1.8TB APFS, 12GB used, **47GB available**
- **OS**: macOS 26.5.1

### Current Usage
```
Total disk: 1.8TB
Used: 12GB (thin provisioning)
Available: 47GB for deployment
```

---

## Current Models Inventory

### GGUF Models (llama.cpp format)
```
gemma-4-31B-it-Q5_K_M.gguf        20GB  ✓ (31B deployed)
google_gemma-4-26B-A4B-it-Q5_K_M   18GB  (A4B GGUF, not MLX)
gemma-4-E4B-it-Q4_K_M.gguf        5GB   ✓ (E4B GGUF)
mmproj-models                      1GB   (multimodal projections)
```

### MLX Models
```
gemma-4-e4b-it-4bit                4.9GB  ✓ (MLX E4B)
mlx-gemma4-e4b-it-4bit             7.7GB  ✓ (Alternative E4B)
mlx-gemma4-e4b-it-8bit             8.4GB  ✓ (8-bit variant)
```

### HuggingFace Cache
```
models--google--gemma-4-12B-it     31MB   (metadata only, not full model)
models--google--gemma-4-e2b-it     191MB  (metadata only)
mlx-community--gemma-4-e4b-it-4bit 4.9GB  ✓ (MLX cached)
mlx-community--gemma-4-e2b-it-8bit 3.1GB  ✓ (E2B 8-bit cached)
paligemma models                   27GB   (vision models)
```

---

## Deployment Requirements

### Models to Deploy (from MarkBaseEngine)

| Model | Size | Source | Status on M5Max48 |
|-------|------|--------|-------------------|
| **E4B-MarkBase** | 4.67GB | E4B-MarkBase dir | Use existing mlx-gemma4-e4b (7.7GB) ✓ |
| **12B Standard** | ~4GB | MLX 12B cache | Need download (~4GB) |
| **26B-Standard** | 15.6GB | Local copy | Need copy (15.6GB) |
| **31B MLX** | ~20GB | Optional | Use existing GGUF (20GB) ✓ |

### Total Deployment Space

**Required**:
- 12B: ~4GB
- 26B: ~15.6GB
- MarkBaseEngine: ~200MB
- **Total new**: ~20GB

**Available**: 47GB ✓ (sufficient)

---

## Deployment Strategy

### Option 1: Use Existing MLX Models
```
E4B: Use mlx-gemma4-e4b-it-4bit (7.7GB) ✓
31B: Use gemma-4-31B-it-Q5_K_M.gguf (20GB) ✓
Deploy: 12B + 26B-Standard (~20GB)
```

### Option 2: Full MLX Deployment
```
Deploy all 4 models in MLX format:
- E4B-MarkBase: 4.67GB (copy)
- 12B Standard: 4GB (copy)
- 26B-Standard: 15.6GB (copy)
- 31B MLX: 20GB (optional, use GGUF)
```

---

## Deployment Plan

### Phase 1: MarkBaseEngine Setup (5 min)
```bash
ssh 192.168.110.201
cd ~
git clone [MarkBaseEngine repo]
swift build
```

### Phase 2: Use Existing Models (immediate)
```
E4B: ~/models/mlx-gemma4-e4b-it-4bit (7.7GB)
31B: ~/models/gemma-4-31B-it-Q5_K_M.gguf (20GB, GGUF)
```

### Phase 3: Deploy Missing Models (30-60 min)
```bash
# Copy from local MarkBaseEngine
scp -r models/gemma-4-26b-standard 192.168.110.201:~/models/

# Download 12B MLX (if needed)
ssh 192.168.110.201 "cd ~/models && huggingface-cli download mlx-community/gemma-4-12B-it-4bit"
```

---

## Space Optimization

### Clean Up Recommendations
```bash
# Remove duplicate E4B (keep largest)
rm ~/models/gemma-4-e4b-it-4bit  # 4.9GB duplicate
rm ~/models/gemma-4-E4B-it-Q4_K_M.gguf  # 5GB GGUF (use MLX)

# Remove unused vision models (if not needed)
rm ~/.cache/huggingface/hub/models--google--paligemma-*  # 27GB

# Keep essential:
- mlx-gemma4-e4b-it-4bit (7.7GB) - E4B MLX
- gemma-4-31B-it-Q5_K_M.gguf (20GB) - 31B GGUF
```

**Space freed**: ~32GB → **79GB available** ✓

---

## Model Paths on M5Max48

### Existing (Verified)
```
E4B: /Users/accusys/models/mlx-gemma4-e4b-it-4bit/
31B: /Users/accusys/models/gemma-4-31B-it-Q5_K_M.gguf
E2B: ~/.cache/huggingface/hub/models--mlx-community--gemma-4-e2b-it-8bit/
```

### To Deploy
```
26B: ~/models/gemma-4-26b-standard/  (copy from local)
12B: ~/models/gemma-4-12b-it-4bit/   (download)
```

---

## Deployment Commands

### Step 1: Clone MarkBaseEngine
```bash
ssh 192.168.110.201
cd ~
git clone https://github.com/[repo]/MarkBaseEngine.git
cd MarkBaseEngine
swift build -c release
```

### Step 2: Copy 26B-Standard (from local)
```bash
# From local machine
scp -r /Users/accusys/coder/models/gemma-4-26b-standard \
      192.168.110.201:/Users/accusys/models/

# Or use rsync for large files
rsync -avh --progress \
      /Users/accusys/coder/models/gemma-4-26b-standard \
      192.168.110.201:/Users/accusys/models/
```

### Step 3: Copy 12B Standard (from local)
```bash
# From local MarkBaseEngine
scp -r /Users/accusys/MarkBaseEngine/models/E4B-MarkBase \
      192.168.110.201:/Users/accusys/models/

# Or use HuggingFace cache
scp -r ~/.cache/huggingface/hub/models--mlx-community--gemma-4-12B-it-4bit \
      192.168.110.201:/Users/accusys/.cache/huggingface/hub/
```

---

## Network Transfer Estimates

### Bandwidth
- Local network: ~100Mbps (WiFi) or ~1Gbps (Ethernet)
- Transfer time estimates:

| Model | Size | WiFi (100Mbps) | Ethernet (1Gbps) |
|-------|------|----------------|-------------------|
| 26B | 15.6GB | ~20 min | ~2 min |
| 12B | 4GB | ~5 min | ~30 sec |
| E4B | 4.67GB | ~6 min | ~40 sec |

**Total**: ~30 min (WiFi) or ~3 min (Ethernet)

---

## Testing Commands

### Verify Models
```bash
ssh 192.168.110.201
cd ~/MarkBaseEngine
swift test --filter E4BMarkBaseTest
swift test --filter Model31BForwardTest
swift test --filter InferenceSpeedTest
```

### Performance Check
```bash
# TEXT inference speed
swift run MarkBaseServer --model ~/models/mlx-gemma4-e4b-it-4bit

# Expected: <30ms/token, >30 tok/s (48GB memory)
```

---

## Deployment Status

| Model | Local Status | M5Max48 Status | Action |
|-------|--------------|----------------|--------|
| **E4B** | ✓ Ready (E4B-MarkBase) | ✓ Existing (mlx-gemma4) | Use existing |
| **12B** | ✓ Ready (Standard) | ⚠ Metadata only | **Deploy needed** |
| **26B-Standard** | ✓ Ready | ✗ Missing | **Deploy needed** |
| **31B** | ✓ Ready | ✓ GGUF existing | Use GGUF |

---

## Recommendations

### Immediate Actions
1. **Clone MarkBaseEngine** to M5Max48 (~5 min)
2. **Use existing E4B** (mlx-gemma4-e4b-it-4bit)
3. **Copy 26B-Standard** (15.6GB, ~20 min WiFi)
4. **Copy 12B Standard** (4GB, ~5 min WiFi)
5. **Use existing 31B GGUF** (no copy needed)

### Space Optimization
- Clean up duplicate E4B models (free ~5GB)
- Clean up unused paligemma (free ~27GB) if not needed
- **Total freed**: ~32GB → **79GB available**

### Testing
- Run speed tests on M5Max48 (verify <30ms/token)
- Compare performance with local (M5 128GB)
- Validate zero NaN on all models

---

## Deployment Timeline

| Phase | Task | Duration |
|-------|------|----------|
| **1** | Clone MarkBaseEngine | 5 min |
| **2** | Build Swift project | 3 min |
| **3** | Copy 26B-Standard | 20 min (WiFi) |
| **4** | Copy 12B Standard | 5 min (WiFi) |
| **5** | Test models | 5 min |
| **Total** | **Full deployment** | **~40 min** |

---

## Final Checklist

- ✅ System specs verified (48GB memory, 47GB space)
- ✅ Existing models inventoried (E4B, 31B GGUF)
- ⚠️ MarkBaseEngine not installed (need clone)
- ⚠️ 12B Standard missing (need copy)
- ⚠️ 26B-Standard missing (need copy)
- ✅ Deployment plan ready (~40 min)

---

## Next Steps

1. **Clone MarkBaseEngine** → `ssh 192.168.110.201 && git clone [repo]`
2. **Copy models** → `scp -r models/* 192.168.110.201:~/models/`
3. **Build and test** → `swift build && swift test`

---

**End of Deployment Assessment**