Initial commit: E4B-MarkBase model integration with passing tests

- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
2026-06-23 18:12:35 +08:00
commit ac75faa0cc
301 changed files with 63426 additions and 0 deletions
@@ -0,0 +1,158 @@
+# API 參考文檔
+
+## 端點
+
+### GET /health
+
+健康檢查
+
+**響應**:
+```json
+{
+  "status": "healthy",
+  "model": "markbase-12b",
+  "layers": 48,
+  "vocabSize": 262144
+}
+```
+
+### GET /v1/models
+
+獲取可用模型列表
+
+**響應**:
+```json
+{
+  "data": [
+    {
+      "id": "markbase-12b",
+      "object": "model",
+      "owned_by": "markbase"
+    }
+  ]
+}
+```
+
+### POST /v1/chat/completions
+
+對話生成
+
+**請求體**:
+```json
+{
+  "messages": [
+    {"role": "user", "content": "Hello"}
+  ],
+  "max_tokens": 100,
+  "temperature": 0.7,
+  "top_p": 0.95,
+  "top_k": 40,
+  "stream": false
+}
+```
+
+**響應 (非流式)**:
+```json
+{
+  "id": "chatcmpl-abc123",
+  "object": "chat.completion",
+  "created": 1234567890,
+  "model": "markbase-12b",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "Hello! How can I help you?"
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 10,
+    "completion_tokens": 50,
+    "total_tokens": 60
+  }
+}
+```
+
+**響應 (流式)**:
+```
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk",...,"choices":[{"delta":{"role":"assistant"},"finish_reason":null}]}
+
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk",...,"choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}
+
+data: [DONE]
+```
+
+### POST /v1/completions
+
+文本補全
+
+**請求體**:
+```json
+{
+  "prompt": "Hello, world!",
+  "max_tokens": 100,
+  "temperature": 0.7,
+  "stream": false
+}
+```
+
+## 參數說明
+
+| 參數 | 類型 | 默認 | 說明 |
+|------|------|------|------|
+| `messages` | array | - | 對話消息列表 |
+| `prompt` | string | - | 文本提示 |
+| `max_tokens` | int | 100 | 最大生成 token 數 (1-4096) |
+| `temperature` | float | 1.0 | 溫度 (0.0-2.0) |
+| `top_p` | float | - | 核采樣閾值 (0.0-1.0) |
+| `top_k` | int | - | Top-k 采樣 |
+| `stream` | bool | false | 是否流式輸出 |
+
+## 錯誤格式
+
+```json
+{
+  "error": {
+    "message": "Invalid request: prompt cannot be empty",
+    "type": "invalid_request_error",
+    "code": 400
+  }
+}
+```
+
+## 多模態格式
+
+### 圖片
+
+```json
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "描述這張圖片"},
+        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
+      ]
+    }
+  ]
+}
+```
+
+### 音訊
+
+```json
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "轉錄這段音訊"},
+        {"type": "audio_url", "audio_url": {"url": "data:audio/wav;base64,..."}}
+      ]
+    }
+  ]
+}
+```
@@ -0,0 +1,417 @@
+# MarkBase 完整 API 規劃
+
+## 📋 API 概述
+
+MarkBase 提供 OpenAI 兼容的 REST API，支持文本生成、多模態推理和流式輸出。
+
+---
+
+## 🔌 端點清單
+
+### 基礎端點
+
+| 方法 | 路徑 | 說明 |
+|------|------|------|
+| GET | `/health` | 健康檢查 |
+| GET | `/v1/models` | 模型列表 |
+| GET | `/v1/models/{model_id}` | 模型詳情 |
+
+### 推理端點
+
+| 方法 | 路徑 | 說明 |
+|------|------|------|
+| POST | `/v1/chat/completions` | 對話生成 |
+| POST | `/v1/completions` | 文本補全 |
+| POST | `/v1/embeddings` | 嵌入向量 |
+
+### 多模態端點
+
+| 方法 | 路徑 | 說明 |
+|------|------|------|
+| POST | `/v1/chat/completions` | 圖片理解 |
+| POST | `/v1/chat/completions` | 音訊理解 |
+
+---
+
+## 📝 詳細 API 規範
+
+### GET /health
+
+健康檢查端點
+
+**響應**:
+```json
+{
+  "status": "healthy",
+  "model": "markbase-12b",
+  "layers": 48,
+  "vocabSize": 262144,
+  "version": "1.0.0"
+}
+```
+
+---
+
+### GET /v1/models
+
+獲取可用模型列表
+
+**響應**:
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "id": "markbase-12b",
+      "object": "model",
+      "created": 1234567890,
+      "owned_by": "markbase",
+      "permissions": ["read"],
+      "capabilities": {
+        "text": true,
+        "vision": true,
+        "audio": true
+      }
+    }
+  ]
+}
+```
+
+---
+
+### POST /v1/chat/completions
+
+對話生成（支持文本、圖片、音訊）
+
+#### 請求體
+
+```json
+{
+  "model": "markbase-12b",
+  "messages": [
+    {
+      "role": "system",
+      "content": "You are a helpful assistant."
+    },
+    {
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "Describe this image"},
+        {
+          "type": "image_url",
+          "image_url": {
+            "url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
+            "detail": "high"
+          }
+        }
+      ]
+    }
+  ],
+  "max_tokens": 100,
+  "temperature": 0.7,
+  "top_p": 0.95,
+  "top_k": 40,
+  "stream": false,
+  "stop": ["\n\n"],
+  "presence_penalty": 0.0,
+  "frequency_penalty": 0.0
+}
+```
+
+#### 參數說明
+
+| 參數 | 類型 | 默認 | 必填 | 說明 |
+|------|------|------|:----:|------|
+| `model` | string | - | ✅ | 模型 ID |
+| `messages` | array | - | ✅ | 消息列表 |
+| `max_tokens` | int | 100 | ❌ | 最大生成 token 數 (1-4096) |
+| `temperature` | float | 1.0 | ❌ | 溫度 (0.0-2.0) |
+| `top_p` | float | 1.0 | ❌ | 核采樣閾值 (0.0-1.0) |
+| `top_k` | int | - | ❌ | Top-k 采樣 |
+| `stream` | bool | false | ❌ | 是否流式輸出 |
+| `stop` | array | - | ❌ | 停止序列 |
+| `presence_penalty` | float | 0.0 | ❌ | 存在懲罰 (-2.0-2.0) |
+| `frequency_penalty` | float | 0.0 | ❌ | 頻率懲罰 (-2.0-2.0) |
+
+#### 響應（非流式）
+
+```json
+{
+  "id": "chatcmpl-abc123def456",
+  "object": "chat.completion",
+  "created": 1234567890,
+  "model": "markbase-12b",
+  "system_fingerprint": "fp_markbase",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "This is a beautiful landscape photo...",
+        "tool_calls": []
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 150,
+    "completion_tokens": 50,
+    "total_tokens": 200
+  }
+}
+```
+
+#### 響應（流式）
+
+```
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
+
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":"This"},"finish_reason":null}]}
+
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":" is"},"finish_reason":null}]}
+
+data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
+
+data: [DONE]
+```
+
+---
+
+### POST /v1/completions
+
+文本補全（舊版 API，建議使用 /v1/chat/completions）
+
+#### 請求體
+
+```json
+{
+  "model": "markbase-12b",
+  "prompt": "Hello, world!",
+  "max_tokens": 100,
+  "temperature": 0.7,
+  "top_p": 0.95,
+  "top_k": 40,
+  "stream": false,
+  "stop": ["\n\n"]
+}
+```
+
+#### 響應
+
+```json
+{
+  "id": "cmpl-abc123",
+  "object": "text_completion",
+  "created": 1234567890,
+  "model": "markbase-12b",
+  "choices": [
+    {
+      "index": 0,
+      "text": " How are you today?",
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 10,
+    "completion_tokens": 20,
+    "total_tokens": 30
+  }
+}
+```
+
+---
+
+### POST /v1/embeddings
+
+生成嵌入向量
+
+#### 請求體
+
+```json
+{
+  "model": "markbase-12b",
+  "input": "Hello world",
+  "encoding_format": "float"
+}
+```
+
+#### 響應
+
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "object": "embedding",
+      "index": 0,
+      "embedding": [0.1, 0.2, 0.3, ...],
+      "usage": {
+        "prompt_tokens": 5,
+        "total_tokens": 5
+      }
+    }
+  ],
+  "model": "markbase-12b",
+  "usage": {
+    "prompt_tokens": 5,
+    "total_tokens": 5
+  }
+}
+```
+
+---
+
+## 🖼️ 多模態格式
+
+### 圖片輸入
+
+```json
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "What's in this image?"},
+        {
+          "type": "image_url",
+          "image_url": {
+            "url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
+            "detail": "high"
+          }
+        }
+      ]
+    }
+  ]
+}
+```
+
+**支持的格式**：
+- JPEG
+- PNG
+- WebP
+- GIF
+
+**大小限制**：
+- 最大 20MB
+- 建議 1024x1024 以下
+
+### 音訊輸入
+
+```json
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": [
+        {"type": "text", "text": "Transcribe this audio"},
+        {
+          "type": "audio_url",
+          "audio_url": {
+            "url": "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA="
+          }
+        }
+      ]
+    }
+  ]
+}
+```
+
+**支持的格式**：
+- WAV
+- MP3
+- FLAC
+- OGG
+
+**大小限制**：
+- 最大 25MB
+- 建議 60 秒以下
+
+---
+
+## ⚠️ 錯誤格式
+
+### 標準錯誤響應
+
+```json
+{
+  "error": {
+    "message": "Invalid request: prompt cannot be empty",
+    "type": "invalid_request_error",
+    "param": "prompt",
+    "code": 400
+  }
+}
+```
+
+### 錯誤代碼
+
+| HTTP 狀態 | 錯誤類型 | 說明 |
+|:---------:|----------|------|
+| 400 | `invalid_request_error` | 請求參數錯誤 |
+| 401 | `authentication_error` | 認證失敗 |
+| 403 | `permission_error` | 權限不足 |
+| 404 | `not_found_error` | 資源不存在 |
+| 429 | `rate_limit_error` | 請求頻率限制 |
+| 500 | `server_error` | 服務器內部錯誤 |
+| 503 | `service_unavailable` | 服務不可用 |
+
+---
+
+## 📊 性能指標
+
+### 預期性能
+
+| 模型 | 速度 | 內存 |
+|------|------|------|
+| E4B (4B) | ~20 tok/s | ~3GB |
+| 12B | ~19 tok/s | ~9GB |
+
+### 延遲
+
+| 指標 | E4B | 12B |
+|------|:---:|:---:|
+| 首字延遲 | <100ms | <150ms |
+| 平均延遲 | 50ms/token | 53ms/token |
+
+---
+
+## 🔐 認證
+
+### API Key 認證（未來功能）
+
+```bash
+curl http://localhost:8080/v1/chat/completions \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"messages": [{"role": "user", "content": "Hello"}]}'
+```
+
+---
+
+## 📈 限流（未來功能）
+
+### 限流頭部
+
+```
+X-RateLimit-Limit: 100
+X-RateLimit-Remaining: 99
+X-RateLimit-Reset: 1234567890
+```
+
+---
+
+## 📝 版本信息
+
+### 當前版本
+
+- API 版本：v1
+- 兼容：OpenAI API v1
+- 狀態：Production Ready
+
+### 未來規劃
+
+- [ ] 工具調用 (Function Calling)
+- [ ] 結構化輸出
+- [ ] 批處理 API
+- [ ] 文件上傳 API
+- [ ] 微調 API
@@ -0,0 +1,116 @@
+# 部署指南
+
+## 系統要求
+
+- macOS 15.0+
+- Apple Silicon (M1/M2/M3)
+- Swift 6.0+
+- 內存: 16GB+ (E4B), 32GB+ (12B)
+
+## 模型準備
+
+### 下載模型
+
+```bash
+# 使用 huggingface-cli
+huggingface-cli download mlx-community/gemma-4-e4b-it-4bit
+
+# 或使用 12B
+huggingface-cli download mlx-community/gemma-4-12b-it-4bit
+```
+
+### 目錄結構
+
+```
+model/
+├── model.safetensors      # 模型權重
+├── config.json            # 模型配置
+└── tokenizer.json         # Tokenizer
+```
+
+## 部署步驟
+
+### 1. 構建
+
+```bash
+cd MarkBase12B
+swift build -c release
+```
+
+### 2. 運行
+
+```bash
+# 基本運行
+swift run -c release G12BServer ./model
+
+# 指定端口
+swift run -c release G12BServer ./model 8080
+
+# 後台運行
+nohup swift run -c release G12BServer ./model 8080 > server.log 2>&1 &
+```
+
+### 3. 驗證
+
+```bash
+curl http://localhost:8080/health
+```
+
+## Docker 部署 (可選)
+
+```dockerfile
+FROM swift:5.9
+
+WORKDIR /app
+COPY . .
+RUN swift build -c release
+
+EXPOSE 8080
+CMD ["swift", "run", "-c", "release", "G12BServer", "./model", "8080"]
+```
+
+## 性能調優
+
+### Buffer Pool
+
+緩衝區池已自動啟用，可減少內存分配開銷。
+
+### Kernel 融合
+
+融合 kernels 已自動加載，減少 dispatch 次數。
+
+### 基準測試
+
+```bash
+swift run G12BServer ./model markbase --benchmark
+```
+
+## 監控
+
+### 日誌
+
+服務器輸出到 stdout，可重定向到文件：
+
+```bash
+swift run G12BServer ./model > server.log 2>&1
+```
+
+### 指標
+
+未來版本將支持 Prometheus 指標。
+
+## 故障排除
+
+### 常見問題
+
+1. **模型未找到**
+   - 檢查路徑是否正確
+   - 確認 safetensors 文件存在
+
+2. **內存不足**
+   - 減少 max_context_length
+   - 使用較小模型
+
+3. **編譯失敗**
+   - 確認 Swift 版本 >= 6.0
+   - 確認 macOS 版本 >= 15.0
@@ -0,0 +1,99 @@
+# 性能優化指南
+
+## 基準測試
+
+運行完整基準測試：
+
+```bash
+swift run G12BServer ./model markbase --benchmark
+```
+
+輸出示例：
+
+```
+╔══════════════════════════════════════╗
+║   Performance Benchmark              ║
+║   Model: markbase                    ║
+╚══════════════════════════════════════╝
+
+📊 Model Loading Benchmark
+────────────────────────────────────────
+  Engine initialization: 0.123s
+  Model loading: 2.456s
+  Total: 2.579s
+  Layers: 48
+  Vocab: 262144
+  Hidden: 3840
+
+📊 Token Generation Benchmark
+────────────────────────────────────────
+  Run 1: 20 tokens in 1.052s (19.0 tok/s)
+  Run 2: 20 tokens in 1.048s (19.1 tok/s)
+  Run 3: 20 tokens in 1.050s (19.0 tok/s)
+
+  Average: 20 tokens in 1.050s
+  Speed: 19.0 tok/s
+
+📊 Tokenizer Benchmark
+────────────────────────────────────────
+  Encode: "Hello, world!..." -> 5 tokens in 0.012ms
+  Decode: 5 tokens -> "Hello, world!..." in 0.005ms
+
+📊 Buffer Pool Benchmark
+────────────────────────────────────────
+  Without pool: 0.123s (8130 allocs/s)
+  With pool: 0.045s (22222 allocs/s)
+
+  Speedup: 2.7x
+```
+
+## 優化技術
+
+### 1. SIMD 優化
+
+- **Attention**: 17.22x 提升 (threadgroup cache + float4)
+- **Matmul**: 2.93x 提升 (SIMD batch processing)
+- **RMS Norm**: 4.53x 提升 (parallel reduction)
+
+### 2. Kernel 融合
+
+融合 kernels 減少 dispatch 次數：
+
+- `rms_norm_matmul_fused` - 融合 RMS Norm + Matmul
+- `gelu_eltwise_mul_fused` - 融合 GELU + Elementwise Mul
+- `residual_rms_norm_fused` - 融合 Residual + RMS Norm
+- `attention_o_proj_fused` - 融合 Attention + O Projection
+
+### 3. Buffer Pool
+
+緩衝區池減少內存分配：
+
+- 重用 MTLBuffer
+- 256 字節對齊
+- 2.7x 加速比
+
+### 4. 異步推理
+
+異步生成減少阻塞：
+
+- AsyncTokenGenerator
+- Prefetcher
+- 非阻塞 forward pass
+
+## 性能對比
+
+| 優化項 | 基準 | 優化後 | 提升 |
+|--------|------|--------|------|
+| Attention | 3.75ms | 0.22ms | 17.22x |
+| Matmul | 1.38ms | 0.42ms | 2.93x |
+| RMS Norm | 0.748ms | 0.165ms | 4.53x |
+| **總體** | **0.100s/token** | **0.051s/token** | **2x** |
+
+## 未來優化
+
+### Phase 4+
+
+- [ ] Float16 支持
+- [ ] Batch inference
+- [ ] Paged attention
+- [ ] Speculative decoding