Initial commit: E4B-MarkBase model integration with passing tests
CI / build-and-test (push) Has been cancelled
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
This commit is contained in:
MarkBase Admin
@@ -0,0 +1,417 @@
|
||||
# MarkBase 完整 API 規劃
|
||||
|
||||
## 📋 API 概述
|
||||
|
||||
MarkBase 提供 OpenAI 兼容的 REST API，支持文本生成、多模態推理和流式輸出。
|
||||
|
||||
---
|
||||
|
||||
## 🔌 端點清單
|
||||
|
||||
### 基礎端點
|
||||
|
||||
| 方法 | 路徑 | 說明 |
|
||||
|------|------|------|
|
||||
| GET | `/health` | 健康檢查 |
|
||||
| GET | `/v1/models` | 模型列表 |
|
||||
| GET | `/v1/models/{model_id}` | 模型詳情 |
|
||||
|
||||
### 推理端點
|
||||
|
||||
| 方法 | 路徑 | 說明 |
|
||||
|------|------|------|
|
||||
| POST | `/v1/chat/completions` | 對話生成 |
|
||||
| POST | `/v1/completions` | 文本補全 |
|
||||
| POST | `/v1/embeddings` | 嵌入向量 |
|
||||
|
||||
### 多模態端點
|
||||
|
||||
| 方法 | 路徑 | 說明 |
|
||||
|------|------|------|
|
||||
| POST | `/v1/chat/completions` | 圖片理解 |
|
||||
| POST | `/v1/chat/completions` | 音訊理解 |
|
||||
|
||||
---
|
||||
|
||||
## 📝 詳細 API 規範
|
||||
|
||||
### GET /health
|
||||
|
||||
健康檢查端點
|
||||
|
||||
**響應**:
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"model": "markbase-12b",
|
||||
"layers": 48,
|
||||
"vocabSize": 262144,
|
||||
"version": "1.0.0"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /v1/models
|
||||
|
||||
獲取可用模型列表
|
||||
|
||||
**響應**:
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "markbase-12b",
|
||||
"object": "model",
|
||||
"created": 1234567890,
|
||||
"owned_by": "markbase",
|
||||
"permissions": ["read"],
|
||||
"capabilities": {
|
||||
"text": true,
|
||||
"vision": true,
|
||||
"audio": true
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /v1/chat/completions
|
||||
|
||||
對話生成（支持文本、圖片、音訊）
|
||||
|
||||
#### 請求體
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "markbase-12b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "Describe this image"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
|
||||
"detail": "high"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"max_tokens": 100,
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.95,
|
||||
"top_k": 40,
|
||||
"stream": false,
|
||||
"stop": ["\n\n"],
|
||||
"presence_penalty": 0.0,
|
||||
"frequency_penalty": 0.0
|
||||
}
|
||||
```
|
||||
|
||||
#### 參數說明
|
||||
|
||||
| 參數 | 類型 | 默認 | 必填 | 說明 |
|
||||
|------|------|------|:----:|------|
|
||||
| `model` | string | - | ✅ | 模型 ID |
|
||||
| `messages` | array | - | ✅ | 消息列表 |
|
||||
| `max_tokens` | int | 100 | ❌ | 最大生成 token 數 (1-4096) |
|
||||
| `temperature` | float | 1.0 | ❌ | 溫度 (0.0-2.0) |
|
||||
| `top_p` | float | 1.0 | ❌ | 核采樣閾值 (0.0-1.0) |
|
||||
| `top_k` | int | - | ❌ | Top-k 采樣 |
|
||||
| `stream` | bool | false | ❌ | 是否流式輸出 |
|
||||
| `stop` | array | - | ❌ | 停止序列 |
|
||||
| `presence_penalty` | float | 0.0 | ❌ | 存在懲罰 (-2.0-2.0) |
|
||||
| `frequency_penalty` | float | 0.0 | ❌ | 頻率懲罰 (-2.0-2.0) |
|
||||
|
||||
#### 響應（非流式）
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-abc123def456",
|
||||
"object": "chat.completion",
|
||||
"created": 1234567890,
|
||||
"model": "markbase-12b",
|
||||
"system_fingerprint": "fp_markbase",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "This is a beautiful landscape photo...",
|
||||
"tool_calls": []
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 150,
|
||||
"completion_tokens": 50,
|
||||
"total_tokens": 200
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 響應（流式）
|
||||
|
||||
```
|
||||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":"This"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":" is"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||||
|
||||
data: [DONE]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /v1/completions
|
||||
|
||||
文本補全（舊版 API，建議使用 /v1/chat/completions）
|
||||
|
||||
#### 請求體
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "markbase-12b",
|
||||
"prompt": "Hello, world!",
|
||||
"max_tokens": 100,
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.95,
|
||||
"top_k": 40,
|
||||
"stream": false,
|
||||
"stop": ["\n\n"]
|
||||
}
|
||||
```
|
||||
|
||||
#### 響應
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cmpl-abc123",
|
||||
"object": "text_completion",
|
||||
"created": 1234567890,
|
||||
"model": "markbase-12b",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"text": " How are you today?",
|
||||
"finish_reason": "stop"
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 10,
|
||||
"completion_tokens": 20,
|
||||
"total_tokens": 30
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /v1/embeddings
|
||||
|
||||
生成嵌入向量
|
||||
|
||||
#### 請求體
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "markbase-12b",
|
||||
"input": "Hello world",
|
||||
"encoding_format": "float"
|
||||
}
|
||||
```
|
||||
|
||||
#### 響應
|
||||
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"object": "embedding",
|
||||
"index": 0,
|
||||
"embedding": [0.1, 0.2, 0.3, ...],
|
||||
"usage": {
|
||||
"prompt_tokens": 5,
|
||||
"total_tokens": 5
|
||||
}
|
||||
}
|
||||
],
|
||||
"model": "markbase-12b",
|
||||
"usage": {
|
||||
"prompt_tokens": 5,
|
||||
"total_tokens": 5
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🖼️ 多模態格式
|
||||
|
||||
### 圖片輸入
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "What's in this image?"},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
|
||||
"detail": "high"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**支持的格式**：
|
||||
- JPEG
|
||||
- PNG
|
||||
- WebP
|
||||
- GIF
|
||||
|
||||
**大小限制**：
|
||||
- 最大 20MB
|
||||
- 建議 1024x1024 以下
|
||||
|
||||
### 音訊輸入
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "Transcribe this audio"},
|
||||
{
|
||||
"type": "audio_url",
|
||||
"audio_url": {
|
||||
"url": "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA="
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**支持的格式**：
|
||||
- WAV
|
||||
- MP3
|
||||
- FLAC
|
||||
- OGG
|
||||
|
||||
**大小限制**：
|
||||
- 最大 25MB
|
||||
- 建議 60 秒以下
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 錯誤格式
|
||||
|
||||
### 標準錯誤響應
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": "Invalid request: prompt cannot be empty",
|
||||
"type": "invalid_request_error",
|
||||
"param": "prompt",
|
||||
"code": 400
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 錯誤代碼
|
||||
|
||||
| HTTP 狀態 | 錯誤類型 | 說明 |
|
||||
|:---------:|----------|------|
|
||||
| 400 | `invalid_request_error` | 請求參數錯誤 |
|
||||
| 401 | `authentication_error` | 認證失敗 |
|
||||
| 403 | `permission_error` | 權限不足 |
|
||||
| 404 | `not_found_error` | 資源不存在 |
|
||||
| 429 | `rate_limit_error` | 請求頻率限制 |
|
||||
| 500 | `server_error` | 服務器內部錯誤 |
|
||||
| 503 | `service_unavailable` | 服務不可用 |
|
||||
|
||||
---
|
||||
|
||||
## 📊 性能指標
|
||||
|
||||
### 預期性能
|
||||
|
||||
| 模型 | 速度 | 內存 |
|
||||
|------|------|------|
|
||||
| E4B (4B) | ~20 tok/s | ~3GB |
|
||||
| 12B | ~19 tok/s | ~9GB |
|
||||
|
||||
### 延遲
|
||||
|
||||
| 指標 | E4B | 12B |
|
||||
|------|:---:|:---:|
|
||||
| 首字延遲 | <100ms | <150ms |
|
||||
| 平均延遲 | 50ms/token | 53ms/token |
|
||||
|
||||
---
|
||||
|
||||
## 🔐 認證
|
||||
|
||||
### API Key 認證（未來功能）
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/v1/chat/completions \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 限流（未來功能）
|
||||
|
||||
### 限流頭部
|
||||
|
||||
```
|
||||
X-RateLimit-Limit: 100
|
||||
X-RateLimit-Remaining: 99
|
||||
X-RateLimit-Reset: 1234567890
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 版本信息
|
||||
|
||||
### 當前版本
|
||||
|
||||
- API 版本：v1
|
||||
- 兼容：OpenAI API v1
|
||||
- 狀態：Production Ready
|
||||
|
||||
### 未來規劃
|
||||
|
||||
- [ ] 工具調用 (Function Calling)
|
||||
- [ ] 結構化輸出
|
||||
- [ ] 批處理 API
|
||||
- [ ] 文件上傳 API
|
||||
- [ ] 微調 API
|
||||
Reference in New Issue
Block a user