Initial commit: E4B-MarkBase model integration with passing tests
CI / build-and-test (push) Has been cancelled
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
This commit is contained in:
@@ -0,0 +1,116 @@
|
||||
# 部署指南
|
||||
|
||||
## 系統要求
|
||||
|
||||
- macOS 15.0+
|
||||
- Apple Silicon (M1/M2/M3)
|
||||
- Swift 6.0+
|
||||
- 內存: 16GB+ (E4B), 32GB+ (12B)
|
||||
|
||||
## 模型準備
|
||||
|
||||
### 下載模型
|
||||
|
||||
```bash
|
||||
# 使用 huggingface-cli
|
||||
huggingface-cli download mlx-community/gemma-4-e4b-it-4bit
|
||||
|
||||
# 或使用 12B
|
||||
huggingface-cli download mlx-community/gemma-4-12b-it-4bit
|
||||
```
|
||||
|
||||
### 目錄結構
|
||||
|
||||
```
|
||||
model/
|
||||
├── model.safetensors # 模型權重
|
||||
├── config.json # 模型配置
|
||||
└── tokenizer.json # Tokenizer
|
||||
```
|
||||
|
||||
## 部署步驟
|
||||
|
||||
### 1. 構建
|
||||
|
||||
```bash
|
||||
cd MarkBase12B
|
||||
swift build -c release
|
||||
```
|
||||
|
||||
### 2. 運行
|
||||
|
||||
```bash
|
||||
# 基本運行
|
||||
swift run -c release G12BServer ./model
|
||||
|
||||
# 指定端口
|
||||
swift run -c release G12BServer ./model 8080
|
||||
|
||||
# 後台運行
|
||||
nohup swift run -c release G12BServer ./model 8080 > server.log 2>&1 &
|
||||
```
|
||||
|
||||
### 3. 驗證
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/health
|
||||
```
|
||||
|
||||
## Docker 部署 (可選)
|
||||
|
||||
```dockerfile
|
||||
FROM swift:5.9
|
||||
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN swift build -c release
|
||||
|
||||
EXPOSE 8080
|
||||
CMD ["swift", "run", "-c", "release", "G12BServer", "./model", "8080"]
|
||||
```
|
||||
|
||||
## 性能調優
|
||||
|
||||
### Buffer Pool
|
||||
|
||||
緩衝區池已自動啟用,可減少內存分配開銷。
|
||||
|
||||
### Kernel 融合
|
||||
|
||||
融合 kernels 已自動加載,減少 dispatch 次數。
|
||||
|
||||
### 基準測試
|
||||
|
||||
```bash
|
||||
swift run G12BServer ./model markbase --benchmark
|
||||
```
|
||||
|
||||
## 監控
|
||||
|
||||
### 日誌
|
||||
|
||||
服務器輸出到 stdout,可重定向到文件:
|
||||
|
||||
```bash
|
||||
swift run G12BServer ./model > server.log 2>&1
|
||||
```
|
||||
|
||||
### 指標
|
||||
|
||||
未來版本將支持 Prometheus 指標。
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 常見問題
|
||||
|
||||
1. **模型未找到**
|
||||
- 檢查路徑是否正確
|
||||
- 確認 safetensors 文件存在
|
||||
|
||||
2. **內存不足**
|
||||
- 減少 max_context_length
|
||||
- 使用較小模型
|
||||
|
||||
3. **編譯失敗**
|
||||
- 確認 Swift 版本 >= 6.0
|
||||
- 確認 macOS 版本 >= 15.0
|
||||
Reference in New Issue
Block a user