MarkBase Admin
ac75faa0cc
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
117 lines
1.8 KiB
Markdown
117 lines
1.8 KiB
Markdown
# 部署指南
|
|
|
|
## 系統要求
|
|
|
|
- macOS 15.0+
|
|
- Apple Silicon (M1/M2/M3)
|
|
- Swift 6.0+
|
|
- 內存: 16GB+ (E4B), 32GB+ (12B)
|
|
|
|
## 模型準備
|
|
|
|
### 下載模型
|
|
|
|
```bash
|
|
# 使用 huggingface-cli
|
|
huggingface-cli download mlx-community/gemma-4-e4b-it-4bit
|
|
|
|
# 或使用 12B
|
|
huggingface-cli download mlx-community/gemma-4-12b-it-4bit
|
|
```
|
|
|
|
### 目錄結構
|
|
|
|
```
|
|
model/
|
|
├── model.safetensors # 模型權重
|
|
├── config.json # 模型配置
|
|
└── tokenizer.json # Tokenizer
|
|
```
|
|
|
|
## 部署步驟
|
|
|
|
### 1. 構建
|
|
|
|
```bash
|
|
cd MarkBase12B
|
|
swift build -c release
|
|
```
|
|
|
|
### 2. 運行
|
|
|
|
```bash
|
|
# 基本運行
|
|
swift run -c release G12BServer ./model
|
|
|
|
# 指定端口
|
|
swift run -c release G12BServer ./model 8080
|
|
|
|
# 後台運行
|
|
nohup swift run -c release G12BServer ./model 8080 > server.log 2>&1 &
|
|
```
|
|
|
|
### 3. 驗證
|
|
|
|
```bash
|
|
curl http://localhost:8080/health
|
|
```
|
|
|
|
## Docker 部署 (可選)
|
|
|
|
```dockerfile
|
|
FROM swift:5.9
|
|
|
|
WORKDIR /app
|
|
COPY . .
|
|
RUN swift build -c release
|
|
|
|
EXPOSE 8080
|
|
CMD ["swift", "run", "-c", "release", "G12BServer", "./model", "8080"]
|
|
```
|
|
|
|
## 性能調優
|
|
|
|
### Buffer Pool
|
|
|
|
緩衝區池已自動啟用，可減少內存分配開銷。
|
|
|
|
### Kernel 融合
|
|
|
|
融合 kernels 已自動加載，減少 dispatch 次數。
|
|
|
|
### 基準測試
|
|
|
|
```bash
|
|
swift run G12BServer ./model markbase --benchmark
|
|
```
|
|
|
|
## 監控
|
|
|
|
### 日誌
|
|
|
|
服務器輸出到 stdout，可重定向到文件：
|
|
|
|
```bash
|
|
swift run G12BServer ./model > server.log 2>&1
|
|
```
|
|
|
|
### 指標
|
|
|
|
未來版本將支持 Prometheus 指標。
|
|
|
|
## 故障排除
|
|
|
|
### 常見問題
|
|
|
|
1. **模型未找到**
|
|
- 檢查路徑是否正確
|
|
- 確認 safetensors 文件存在
|
|
|
|
2. **內存不足**
|
|
- 減少 max_context_length
|
|
- 使用較小模型
|
|
|
|
3. **編譯失敗**
|
|
- 確認 Swift 版本 >= 6.0
|
|
- 確認 macOS 版本 >= 15.0
|