MarkBase Admin
ac75faa0cc
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
145 lines
3.0 KiB
Markdown
145 lines
3.0 KiB
Markdown
# MarkBase
|
|
|
|
高性能 Swift Metal 多模態推理引擎，專為 Apple Silicon 優化。
|
|
|
|
## 功能特性
|
|
|
|
- ✅ **純 Swift Metal** - 無外部依賴
|
|
- ✅ **4-bit 量化** - 高效內存使用
|
|
- ✅ **OpenAI 兼容 API** - REST + SSE
|
|
- ✅ **多模態支持** - 文本、圖片、音訊
|
|
- ✅ **流式輸出** - 實時 token 生成
|
|
- ✅ **SIMD 優化** - 17x attention, 3x matmul 提升
|
|
|
|
## 快速開始
|
|
|
|
### 安裝
|
|
|
|
```bash
|
|
git clone <repository-url>
|
|
cd MarkBase12B
|
|
swift build
|
|
```
|
|
|
|
### 啟動服務器
|
|
|
|
```bash
|
|
# 基本啟動
|
|
swift run G12BServer ./model
|
|
|
|
# 指定端口和模型 ID
|
|
swift run G12BServer ./model 8080 markbase-12b
|
|
|
|
# 運行性能基準測試
|
|
swift run G12BServer ./model markbase --benchmark
|
|
```
|
|
|
|
### API 使用
|
|
|
|
#### 健康檢查
|
|
|
|
```bash
|
|
curl http://localhost:8080/health
|
|
```
|
|
|
|
#### 文本生成
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"messages": [
|
|
{"role": "user", "content": "Hello, how are you?"}
|
|
],
|
|
"max_tokens": 100,
|
|
"temperature": 0.7
|
|
}'
|
|
```
|
|
|
|
#### 流式輸出
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"messages": [
|
|
{"role": "user", "content": "Tell me a story"}
|
|
],
|
|
"stream": true
|
|
}'
|
|
```
|
|
|
|
#### 多模態（圖片）
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"messages": [
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "text", "text": "描述這張圖片"},
|
|
{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
|
|
]
|
|
}
|
|
]
|
|
}'
|
|
```
|
|
|
|
## Swift SDK 使用
|
|
|
|
```swift
|
|
import G12B
|
|
|
|
// 初始化引擎
|
|
let engine = try MarkBaseEngine(autoCompile: true)
|
|
|
|
// 加載模型
|
|
let model = try E4BModel(modelDir: "./model", engine: engine)
|
|
|
|
// 生成文本
|
|
let logits = try model.forward(tokenId: 0, position: 0)
|
|
```
|
|
|
|
## 性能
|
|
|
|
| 模型 | 速度 | 內存 |
|
|
|------|------|------|
|
|
| E4B (4B) | 19.7 tok/s | ~3GB |
|
|
| 12B | 18.8 tok/s | ~9GB |
|
|
|
|
## 架構
|
|
|
|
```
|
|
MarkBase12B/
|
|
├── Sources/G12B/
|
|
│ ├── Engine.swift # Metal 引擎
|
|
│ ├── Model.swift # 模型實現
|
|
│ ├── Tokenizer/ # Tokenizer
|
|
│ ├── Generator/ # 文本生成
|
|
│ ├── Sampling/ # 採樣策略
|
|
│ ├── Audio/ # 音訊塔
|
|
│ ├── Vision/ # 視覺塔
|
|
│ ├── Metal/ # Metal Kernels
|
|
│ └── BufferPool.swift # Buffer 池
|
|
├── Sources/G12BServer/
|
|
│ ├── APIServer.swift # API 服務器
|
|
│ ├── MarkBaseServer.swift # 服務器實現
|
|
│ ├── SSE.swift # SSE 支持
|
|
│ ├── Errors.swift # 錯誤處理
|
|
│ ├── MultimodalAPI.swift # 多模態 API
|
|
│ └── PerformanceBenchmark.swift
|
|
└── Tests/G12BTests/
|
|
```
|
|
|
|
## 文檔
|
|
|
|
- [API 參考](docs/API.md)
|
|
- [部署指南](docs/DEPLOYMENT.md)
|
|
- [性能優化](docs/PERFORMANCE.md)
|
|
|
|
## 授權
|
|
|
|
MIT License
|