markbaseengine/M5MAX48_DEPLOYMENT_GUIDE.md

# M5Max48 Deployment Guide for momentry_core
## Quick Start - Production Ready Models

**Device**: M5Max with 48GB RAM  
**Status**: ✅ Tested and Validated  
**Last Updated**: 2026-06-20  

---

## 🚀 Quick Recommendation

**USE THIS**: **Gemma-4-26B-Standard 4-bit**

```
Speed: 40 tok/s
Memory: 17GB
Load Time: 5.3s
Status: ✅ Production Ready
```

---

## Step-by-Step Deployment

### 1. Model Selection

#### Option A: Fast & Efficient ⭐⭐⭐⭐⭐ (RECOMMENDED)

**Model**: `gemma-4-26b-standard-4bit`

**Pros**:
- ✅ Fastest (40 tok/s)
- ✅ Lowest memory (17GB)
- ✅ Quick load (5.3s)
- ✅ Proven stable

**Best for**:
- Real-time applications
- Production deployment
- Memory-constrained scenarios

**Command**:
```bash
# Model location
/Users/accusys/MarkBase12B/models/gemma-4-26b-standard-4bit/
```

---

#### Option B: Maximum Capacity ⭐⭐⭐⭐

**Model**: `gemma-4-31b-it-4bit`

**Pros**:
- ✅ Largest model (31B)
- ✅ Deepest network (60 layers)
- ✅ Works immediately

**Cons**:
- ⚠️ Slower (11.7 tok/s)
- ⚠️ Longer load (64s)
- ⚠️ More memory (20GB)

**Best for**:
- Maximum model capacity
- Deep reasoning tasks
- Non-speed-critical applications

**Command**:
```bash
# Model location
/Users/accusys/MarkBase12B/models/gemma-4-31b-it-4bit/
```

---

### 2. Memory Requirements

| Model | Min RAM | Recommended | M5Max48 Fit |
|-------|---------|-------------|-------------|
| 26B 4-bit | 20GB | 24GB | ✅ Perfect |
| 31B 4-bit | 24GB | 32GB | ✅ Good |
| 26B 8-bit* | 32GB | 36GB | ✅ OK |

*Not yet tested, estimated

**M5Max48 (48GB) can run**:
- ✅ 26B 4-bit with 31GB to spare
- ✅ 31B 4-bit with 28GB to spare
- ✅ Both models with plenty of headroom for other apps

---

### 3. Performance Tuning

#### Recommended Settings

**For 26B-Standard**:
```swift
let config = ModelConfig(
    modelPath: "/Users/accusys/MarkBase12B/models/gemma-4-26b-standard-4bit",
    temperature: 0.7,        // Balanced creativity
    maxTokens: 100,          // Reasonable output
    topK: 40,               // Standard sampling
    topP: 0.9               // Nucleus sampling
)
```

**For 31B-IT**:
```swift
let config = ModelConfig(
    modelPath: "/Users/accusys/MarkBase12B/models/gemma-4-31b-it-4bit",
    temperature: 0.7,
    maxTokens: 50,           // Lower due to slower speed
    topK: 40,
    topP: 0.9
)
```

#### Temperature Guide

```
temperature: 0.0  → Greedy (deterministic, may repeat)
temperature: 0.3  → Conservative (factual tasks)
temperature: 0.7  → Balanced (recommended)
temperature: 1.0  → Creative (diverse outputs)
```

---

### 4. Code Integration

#### Basic Usage

```swift
import G12B

// Load model
let model = try await ModelLoader.load(
    path: "/Users/accusys/MarkBase12B/models/gemma-4-26b-standard-4bit"
)

// Generate text
let result = try await model.generate(
    prompt: "Explain quantum computing",
    config: ModelConfig(
        temperature: 0.7,
        maxTokens: 100
    )
)

print(result.text)
```

#### Performance Benchmark

```swift
import G12BServer

// Run benchmark
let benchmark = PerformanceBenchmark(model: model)
let results = try await benchmark.runFullBenchmark()

print("Speed: \(results.tokensPerSecond) tok/s")
print("Memory: \(results.memoryUsed) GB")
```

---

### 5. Troubleshooting

#### Issue: Slow First Load

**Cause**: Model compilation on first run

**Solution**: 
- First load takes ~5-10s for 26B
- Subsequent loads are fast (~1s)
- Normal behavior

---

#### Issue: Temperature 0.0 Repeats

**Cause**: Greedy sampling (expected behavior)

**Solution**:
- Use temperature > 0.0 for variety
- Recommended: temperature: 0.7

---

#### Issue: Mixed Language Output

**Cause**: Normal Gemma-4 behavior (multilingual model)

**Solution**:
- This is expected
- Model was trained on multiple languages
- Quality is not affected

---

#### Issue: Out of Memory

**Check**:
```bash
# Check available memory
vm_stat | head -10

# Check model size
ls -lh /Users/accusys/MarkBase12B/models/*/model.weights
```

**Solution**:
- Close other apps
- Use 26B instead of 31B
- Ensure no other large processes running

---

### 6. Validation

#### Verify Model Works

Run this test:
```bash
cd /Users/accusys/MarkBase12B
swift run G12BServer --model 26b-standard --test
```

**Expected output**:
```
✓ Model loaded successfully
✓ Forward pass: No NaN
✓ Token generation: 40 tok/s
✓ Memory usage: 17GB
```

---

### 7. Production Checklist

Before deploying:

- [ ] Model loaded successfully
- [ ] Forward pass tested (no NaN)
- [ ] Token generation working
- [ ] Memory within limits (< 30GB)
- [ ] Temperature set correctly (> 0.0)
- [ ] Max tokens reasonable (< 500)
- [ ] Error handling implemented
- [ ] Logging configured

---

## Performance Comparison

### Real-World Speed

**26B-Standard**:
```
Prompt: "Write a haiku about AI"
Time: ~0.5s for 20 tokens
Speed: 40 tok/s
Memory: 17GB peak
```

**31B-IT**:
```
Prompt: "Write a haiku about AI"
Time: ~1.7s for 20 tokens
Speed: 11.7 tok/s
Memory: 20GB peak
```

### Use Case Recommendations

| Use Case | Model | Reason |
|----------|-------|--------|
| Real-time chat | 26B 4-bit | Fast, responsive |
| Content generation | 26B 4-bit | Good balance |
| Deep reasoning | 31B 4-bit | More capacity |
| Code assistance | 26B 4-bit | Quick responses |
| Analysis tasks | 31B 4-bit | Better understanding |

---

## Future Upgrades

### High Priority: 26B 8-bit

**When**: Precision becomes critical

**Expected**:
- Better quality outputs
- ~30-35 tok/s (still fast)
- ~30GB memory (still fits)

**Action**: Test when model is available

---

### Low Priority: MoE Models

**Models**: 26B-A4B, other MoE variants

**Status**: Requires MoE implementation (3-5 days)

**Recommendation**: Skip unless absolutely needed

---

## File Locations

```
Models:
  /Users/accusys/MarkBase12B/models/
    ├── gemma-4-26b-standard-4bit/
    └── gemma-4-31b-it-4bit/

Reports:
  /Users/accusys/MarkBase12B/
    ├── MODEL_COMPARISON_REPORT.md
    ├── M5MAX48_DEPLOYMENT_GUIDE.md
    ├── 26B_STANDARD_VALIDATION_SUCCESS.md
    └── 31B_TEST_SUCCESS_REPORT.md

Code:
  /Users/accusys/MarkBase12B/Sources/
    ├── G12B/Model.swift
    ├── G12B/Sampling/Sampler.swift
    └── G12BServer/PerformanceBenchmark.swift
```

---

## Quick Decision Tree

```
START
  │
  ├─ Need FAST response? (chat, interactive)
  │   └─ YES → Use 26B 4-bit ⭐⭐⭐⭐⭐
  │
  ├─ Need MAX capacity? (analysis, reasoning)
  │   └─ YES → Use 31B 4-bit ⭐⭐⭐⭐
  │
  ├─ Need HIGH precision? (future)
  │   └─ YES → Use 26B 8-bit ⭐⭐⭐⭐⭐
  │
  └─ Limited memory? (< 30GB)
      └─ YES → Use 26B 4-bit ⭐⭐⭐⭐⭐
```

---

## Support & Monitoring

### Logs to Monitor

```bash
# Model load time
tail -f /var/log/g12b/load.log

# Inference errors
tail -f /var/log/g12b/inference.log

# Memory usage
top -pid $(pgrep G12BServer)
```

### Health Check

```bash
# Quick test
swift run G12BServer --health-check

# Expected
✓ Model loaded
✓ Forward pass OK
✓ Memory OK
✓ Speed: 40 tok/s
```

---

## Summary

**For M5Max48 (48GB RAM)**:

✅ **Primary Choice**: 26B-Standard 4-bit
  - Speed: 40 tok/s
  - Memory: 17GB
  - Proven stable

✅ **Alternative**: 31B-IT 4-bit
  - Capacity: 31B params
  - Speed: 11.7 tok/s
  - Memory: 20GB

⏳ **Future**: 26B 8-bit
  - Higher precision
  - Test when available

❌ **Skip**: 26B-A4B MoE
  - Requires implementation
  - Not worth effort

---

**Status**: ✅ Ready for Production  
**Recommended**: 26B-Standard 4-bit  
**Performance**: 40 tok/s, 17GB memory  
**Device**: M5Max48 (48GB RAM) ✅