MarkBase Admin
ac75faa0cc
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
100 lines
2.6 KiB
Markdown
100 lines
2.6 KiB
Markdown
# 性能優化指南
|
|
|
|
## 基準測試
|
|
|
|
運行完整基準測試：
|
|
|
|
```bash
|
|
swift run G12BServer ./model markbase --benchmark
|
|
```
|
|
|
|
輸出示例：
|
|
|
|
```
|
|
╔══════════════════════════════════════╗
|
|
║ Performance Benchmark ║
|
|
║ Model: markbase ║
|
|
╚══════════════════════════════════════╝
|
|
|
|
📊 Model Loading Benchmark
|
|
────────────────────────────────────────
|
|
Engine initialization: 0.123s
|
|
Model loading: 2.456s
|
|
Total: 2.579s
|
|
Layers: 48
|
|
Vocab: 262144
|
|
Hidden: 3840
|
|
|
|
📊 Token Generation Benchmark
|
|
────────────────────────────────────────
|
|
Run 1: 20 tokens in 1.052s (19.0 tok/s)
|
|
Run 2: 20 tokens in 1.048s (19.1 tok/s)
|
|
Run 3: 20 tokens in 1.050s (19.0 tok/s)
|
|
|
|
Average: 20 tokens in 1.050s
|
|
Speed: 19.0 tok/s
|
|
|
|
📊 Tokenizer Benchmark
|
|
────────────────────────────────────────
|
|
Encode: "Hello, world!..." -> 5 tokens in 0.012ms
|
|
Decode: 5 tokens -> "Hello, world!..." in 0.005ms
|
|
|
|
📊 Buffer Pool Benchmark
|
|
────────────────────────────────────────
|
|
Without pool: 0.123s (8130 allocs/s)
|
|
With pool: 0.045s (22222 allocs/s)
|
|
|
|
Speedup: 2.7x
|
|
```
|
|
|
|
## 優化技術
|
|
|
|
### 1. SIMD 優化
|
|
|
|
- **Attention**: 17.22x 提升 (threadgroup cache + float4)
|
|
- **Matmul**: 2.93x 提升 (SIMD batch processing)
|
|
- **RMS Norm**: 4.53x 提升 (parallel reduction)
|
|
|
|
### 2. Kernel 融合
|
|
|
|
融合 kernels 減少 dispatch 次數：
|
|
|
|
- `rms_norm_matmul_fused` - 融合 RMS Norm + Matmul
|
|
- `gelu_eltwise_mul_fused` - 融合 GELU + Elementwise Mul
|
|
- `residual_rms_norm_fused` - 融合 Residual + RMS Norm
|
|
- `attention_o_proj_fused` - 融合 Attention + O Projection
|
|
|
|
### 3. Buffer Pool
|
|
|
|
緩衝區池減少內存分配：
|
|
|
|
- 重用 MTLBuffer
|
|
- 256 字節對齊
|
|
- 2.7x 加速比
|
|
|
|
### 4. 異步推理
|
|
|
|
異步生成減少阻塞：
|
|
|
|
- AsyncTokenGenerator
|
|
- Prefetcher
|
|
- 非阻塞 forward pass
|
|
|
|
## 性能對比
|
|
|
|
| 優化項 | 基準 | 優化後 | 提升 |
|
|
|--------|------|--------|------|
|
|
| Attention | 3.75ms | 0.22ms | 17.22x |
|
|
| Matmul | 1.38ms | 0.42ms | 2.93x |
|
|
| RMS Norm | 0.748ms | 0.165ms | 4.53x |
|
|
| **總體** | **0.100s/token** | **0.051s/token** | **2x** |
|
|
|
|
## 未來優化
|
|
|
|
### Phase 4+
|
|
|
|
- [ ] Float16 支持
|
|
- [ ] Batch inference
|
|
- [ ] Paged attention
|
|
- [ ] Speculative decoding
|