warren/markbaseengine
1
0
Fork 0
Code Issues Pull Requests Actions 18 Packages Projects Releases Wiki Activity

Initial commit: E4B-MarkBase model integration with passing tests
CI / build-and-test (push) Has been cancelled
Details

Browse Source 
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully
- All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance)
- All stress tests passed (5/5 in 127.6s)
  - Concurrent inference
  - Memory stress (67.5 tok/s, 0 NaN)
  - Continuous generation
  - Batch processing
  - Long-running stability
- Swift Metal inference engine with multimodal support
This commit is contained in:
MarkBase Admin
2026-06-23 18:12:35 +08:00
commit ac75faa0cc
301 changed files with 63426 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
M5MAX48_DEPLOYMENT_ASSESSMENT.md
+298
View File
@@ -0,0 +1,298 @@
# M5Max48 LLM Deployment Assessment
**Target**: 192.168.110.201 (M5Max48)
**Date**: 2026-06-23
**Status**: Assessment Complete
---
## System Specifications
### Hardware
- **Hostname**: M5Max48
- **Memory**: 48GB unified (51539607552 bytes)
- **Disk**: 1.8TB APFS, 12GB used, **47GB available**
- **OS**: macOS 26.5.1
### Current Usage
```
Total disk: 1.8TB
Used: 12GB (thin provisioning)
Available: 47GB for deployment
```
---
## Current Models Inventory
### GGUF Models (llama.cpp format)
```
gemma-4-31B-it-Q5_K_M.gguf 20GB ✓ (31B deployed)
google_gemma-4-26B-A4B-it-Q5_K_M 18GB (A4B GGUF, not MLX)
gemma-4-E4B-it-Q4_K_M.gguf 5GB ✓ (E4B GGUF)
mmproj-models 1GB (multimodal projections)
```
### MLX Models
```
gemma-4-e4b-it-4bit 4.9GB ✓ (MLX E4B)
mlx-gemma4-e4b-it-4bit 7.7GB ✓ (Alternative E4B)
mlx-gemma4-e4b-it-8bit 8.4GB ✓ (8-bit variant)
```
### HuggingFace Cache
```
models--google--gemma-4-12B-it 31MB (metadata only, not full model)
models--google--gemma-4-e2b-it 191MB (metadata only)
mlx-community--gemma-4-e4b-it-4bit 4.9GB ✓ (MLX cached)
mlx-community--gemma-4-e2b-it-8bit 3.1GB ✓ (E2B 8-bit cached)
paligemma models 27GB (vision models)
```
---
## Deployment Requirements
### Models to Deploy (from MarkBaseEngine)
| Model | Size | Source | Status on M5Max48 |
|-------|------|--------|-------------------|
| **E4B-MarkBase** | 4.67GB | E4B-MarkBase dir | Use existing mlx-gemma4-e4b (7.7GB) ✓ |
| **12B Standard** | ~4GB | MLX 12B cache | Need download (~4GB) |
| **26B-Standard** | 15.6GB | Local copy | Need copy (15.6GB) |
| **31B MLX** | ~20GB | Optional | Use existing GGUF (20GB) ✓ |
### Total Deployment Space
**Required**:
- 12B: ~4GB
- 26B: ~15.6GB
- MarkBaseEngine: ~200MB
- **Total new**: ~20GB
**Available**: 47GB ✓ (sufficient)
---
## Deployment Strategy
### Option 1: Use Existing MLX Models
```
E4B: Use mlx-gemma4-e4b-it-4bit (7.7GB) ✓
31B: Use gemma-4-31B-it-Q5_K_M.gguf (20GB) ✓
Deploy: 12B + 26B-Standard (~20GB)
```
### Option 2: Full MLX Deployment
```
Deploy all 4 models in MLX format:
- E4B-MarkBase: 4.67GB (copy)
- 12B Standard: 4GB (copy)
- 26B-Standard: 15.6GB (copy)
- 31B MLX: 20GB (optional, use GGUF)
```
---
## Deployment Plan
### Phase 1: MarkBaseEngine Setup (5 min)
```bash
ssh 192.168.110.201
cd ~
git clone [MarkBaseEngine repo]
swift build
```
### Phase 2: Use Existing Models (immediate)
```
E4B: ~/models/mlx-gemma4-e4b-it-4bit (7.7GB)
31B: ~/models/gemma-4-31B-it-Q5_K_M.gguf (20GB, GGUF)
```
### Phase 3: Deploy Missing Models (30-60 min)
```bash
# Copy from local MarkBaseEngine
scp -r models/gemma-4-26b-standard 192.168.110.201:~/models/
# Download 12B MLX (if needed)
ssh 192.168.110.201 "cd ~/models && huggingface-cli download mlx-community/gemma-4-12B-it-4bit"
```
---
## Space Optimization
### Clean Up Recommendations
```bash
# Remove duplicate E4B (keep largest)
rm ~/models/gemma-4-e4b-it-4bit # 4.9GB duplicate
rm ~/models/gemma-4-E4B-it-Q4_K_M.gguf # 5GB GGUF (use MLX)
# Remove unused vision models (if not needed)
rm ~/.cache/huggingface/hub/models--google--paligemma-* # 27GB
# Keep essential:
- mlx-gemma4-e4b-it-4bit (7.7GB) - E4B MLX
- gemma-4-31B-it-Q5_K_M.gguf (20GB) - 31B GGUF
```
**Space freed**: ~32GB → **79GB available** ✓
---
## Model Paths on M5Max48
### Existing (Verified)
```
E4B: /Users/accusys/models/mlx-gemma4-e4b-it-4bit/
31B: /Users/accusys/models/gemma-4-31B-it-Q5_K_M.gguf
E2B: ~/.cache/huggingface/hub/models--mlx-community--gemma-4-e2b-it-8bit/
```
### To Deploy
```
26B: ~/models/gemma-4-26b-standard/ (copy from local)
12B: ~/models/gemma-4-12b-it-4bit/ (download)
```
---
## Deployment Commands
### Step 1: Clone MarkBaseEngine
```bash
ssh 192.168.110.201
cd ~
git clone https://github.com/[repo]/MarkBaseEngine.git
cd MarkBaseEngine
swift build -c release
```
### Step 2: Copy 26B-Standard (from local)
```bash
# From local machine
scp -r /Users/accusys/coder/models/gemma-4-26b-standard \
192.168.110.201:/Users/accusys/models/
# Or use rsync for large files
rsync -avh --progress \
/Users/accusys/coder/models/gemma-4-26b-standard \
192.168.110.201:/Users/accusys/models/
```
### Step 3: Copy 12B Standard (from local)
```bash
# From local MarkBaseEngine
scp -r /Users/accusys/MarkBaseEngine/models/E4B-MarkBase \
192.168.110.201:/Users/accusys/models/
# Or use HuggingFace cache
scp -r ~/.cache/huggingface/hub/models--mlx-community--gemma-4-12B-it-4bit \
192.168.110.201:/Users/accusys/.cache/huggingface/hub/
```
---
## Network Transfer Estimates
### Bandwidth
- Local network: ~100Mbps (WiFi) or ~1Gbps (Ethernet)
- Transfer time estimates:
| Model | Size | WiFi (100Mbps) | Ethernet (1Gbps) |
|-------|------|----------------|-------------------|
| 26B | 15.6GB | ~20 min | ~2 min |
| 12B | 4GB | ~5 min | ~30 sec |
| E4B | 4.67GB | ~6 min | ~40 sec |
**Total**: ~30 min (WiFi) or ~3 min (Ethernet)
---
## Testing Commands
### Verify Models
```bash
ssh 192.168.110.201
cd ~/MarkBaseEngine
swift test --filter E4BMarkBaseTest
swift test --filter Model31BForwardTest
swift test --filter InferenceSpeedTest
```
### Performance Check
```bash
# TEXT inference speed
swift run MarkBaseServer --model ~/models/mlx-gemma4-e4b-it-4bit
# Expected: <30ms/token, >30 tok/s (48GB memory)
```
---
## Deployment Status
| Model | Local Status | M5Max48 Status | Action |
|-------|--------------|----------------|--------|
| **E4B** | ✓ Ready (E4B-MarkBase) | ✓ Existing (mlx-gemma4) | Use existing |
| **12B** | ✓ Ready (Standard) | ⚠ Metadata only | **Deploy needed** |
| **26B-Standard** | ✓ Ready | ✗ Missing | **Deploy needed** |
| **31B** | ✓ Ready | ✓ GGUF existing | Use GGUF |
---
## Recommendations
### Immediate Actions
1. **Clone MarkBaseEngine** to M5Max48 (~5 min)
2. **Use existing E4B** (mlx-gemma4-e4b-it-4bit)
3. **Copy 26B-Standard** (15.6GB, ~20 min WiFi)
4. **Copy 12B Standard** (4GB, ~5 min WiFi)
5. **Use existing 31B GGUF** (no copy needed)
### Space Optimization
- Clean up duplicate E4B models (free ~5GB)
- Clean up unused paligemma (free ~27GB) if not needed
- **Total freed**: ~32GB → **79GB available**
### Testing
- Run speed tests on M5Max48 (verify <30ms/token)
- Compare performance with local (M5 128GB)
- Validate zero NaN on all models
---
## Deployment Timeline
| Phase | Task | Duration |
|-------|------|----------|
| **1** | Clone MarkBaseEngine | 5 min |
| **2** | Build Swift project | 3 min |
| **3** | Copy 26B-Standard | 20 min (WiFi) |
| **4** | Copy 12B Standard | 5 min (WiFi) |
| **5** | Test models | 5 min |
| **Total** | **Full deployment** | **~40 min** |
---
## Final Checklist
- ✅ System specs verified (48GB memory, 47GB space)
- ✅ Existing models inventoried (E4B, 31B GGUF)
- ⚠️ MarkBaseEngine not installed (need clone)
- ⚠️ 12B Standard missing (need copy)
- ⚠️ 26B-Standard missing (need copy)
- ✅ Deployment plan ready (~40 min)
---
## Next Steps
1. **Clone MarkBaseEngine** → `ssh 192.168.110.201 && git clone [repo]`
2. **Copy models** → `scp -r models/* 192.168.110.201:~/models/`
3. **Build and test** → `swift build && swift test`
---
**End of Deployment Assessment**