MarkBase Admin
ac75faa0cc
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
418 lines
7.7 KiB
Markdown
418 lines
7.7 KiB
Markdown
# MarkBase 完整 API 規劃
|
||
|
||
## 📋 API 概述
|
||
|
||
MarkBase 提供 OpenAI 兼容的 REST API，支持文本生成、多模態推理和流式輸出。
|
||
|
||
---
|
||
|
||
## 🔌 端點清單
|
||
|
||
### 基礎端點
|
||
|
||
| 方法 | 路徑 | 說明 |
|
||
|------|------|------|
|
||
| GET | `/health` | 健康檢查 |
|
||
| GET | `/v1/models` | 模型列表 |
|
||
| GET | `/v1/models/{model_id}` | 模型詳情 |
|
||
|
||
### 推理端點
|
||
|
||
| 方法 | 路徑 | 說明 |
|
||
|------|------|------|
|
||
| POST | `/v1/chat/completions` | 對話生成 |
|
||
| POST | `/v1/completions` | 文本補全 |
|
||
| POST | `/v1/embeddings` | 嵌入向量 |
|
||
|
||
### 多模態端點
|
||
|
||
| 方法 | 路徑 | 說明 |
|
||
|------|------|------|
|
||
| POST | `/v1/chat/completions` | 圖片理解 |
|
||
| POST | `/v1/chat/completions` | 音訊理解 |
|
||
|
||
---
|
||
|
||
## 📝 詳細 API 規範
|
||
|
||
### GET /health
|
||
|
||
健康檢查端點
|
||
|
||
**響應**:
|
||
```json
|
||
{
|
||
"status": "healthy",
|
||
"model": "markbase-12b",
|
||
"layers": 48,
|
||
"vocabSize": 262144,
|
||
"version": "1.0.0"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### GET /v1/models
|
||
|
||
獲取可用模型列表
|
||
|
||
**響應**:
|
||
```json
|
||
{
|
||
"object": "list",
|
||
"data": [
|
||
{
|
||
"id": "markbase-12b",
|
||
"object": "model",
|
||
"created": 1234567890,
|
||
"owned_by": "markbase",
|
||
"permissions": ["read"],
|
||
"capabilities": {
|
||
"text": true,
|
||
"vision": true,
|
||
"audio": true
|
||
}
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### POST /v1/chat/completions
|
||
|
||
對話生成（支持文本、圖片、音訊）
|
||
|
||
#### 請求體
|
||
|
||
```json
|
||
{
|
||
"model": "markbase-12b",
|
||
"messages": [
|
||
{
|
||
"role": "system",
|
||
"content": "You are a helpful assistant."
|
||
},
|
||
{
|
||
"role": "user",
|
||
"content": [
|
||
{"type": "text", "text": "Describe this image"},
|
||
{
|
||
"type": "image_url",
|
||
"image_url": {
|
||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
|
||
"detail": "high"
|
||
}
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"max_tokens": 100,
|
||
"temperature": 0.7,
|
||
"top_p": 0.95,
|
||
"top_k": 40,
|
||
"stream": false,
|
||
"stop": ["\n\n"],
|
||
"presence_penalty": 0.0,
|
||
"frequency_penalty": 0.0
|
||
}
|
||
```
|
||
|
||
#### 參數說明
|
||
|
||
| 參數 | 類型 | 默認 | 必填 | 說明 |
|
||
|------|------|------|:----:|------|
|
||
| `model` | string | - | ✅ | 模型 ID |
|
||
| `messages` | array | - | ✅ | 消息列表 |
|
||
| `max_tokens` | int | 100 | ❌ | 最大生成 token 數 (1-4096) |
|
||
| `temperature` | float | 1.0 | ❌ | 溫度 (0.0-2.0) |
|
||
| `top_p` | float | 1.0 | ❌ | 核采樣閾值 (0.0-1.0) |
|
||
| `top_k` | int | - | ❌ | Top-k 采樣 |
|
||
| `stream` | bool | false | ❌ | 是否流式輸出 |
|
||
| `stop` | array | - | ❌ | 停止序列 |
|
||
| `presence_penalty` | float | 0.0 | ❌ | 存在懲罰 (-2.0-2.0) |
|
||
| `frequency_penalty` | float | 0.0 | ❌ | 頻率懲罰 (-2.0-2.0) |
|
||
|
||
#### 響應（非流式）
|
||
|
||
```json
|
||
{
|
||
"id": "chatcmpl-abc123def456",
|
||
"object": "chat.completion",
|
||
"created": 1234567890,
|
||
"model": "markbase-12b",
|
||
"system_fingerprint": "fp_markbase",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"message": {
|
||
"role": "assistant",
|
||
"content": "This is a beautiful landscape photo...",
|
||
"tool_calls": []
|
||
},
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"usage": {
|
||
"prompt_tokens": 150,
|
||
"completion_tokens": 50,
|
||
"total_tokens": 200
|
||
}
|
||
}
|
||
```
|
||
|
||
#### 響應（流式）
|
||
|
||
```
|
||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
|
||
|
||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":"This"},"finish_reason":null}]}
|
||
|
||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{"content":" is"},"finish_reason":null}]}
|
||
|
||
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"markbase-12b","system_fingerprint":"fp_markbase","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||
|
||
data: [DONE]
|
||
```
|
||
|
||
---
|
||
|
||
### POST /v1/completions
|
||
|
||
文本補全（舊版 API，建議使用 /v1/chat/completions）
|
||
|
||
#### 請求體
|
||
|
||
```json
|
||
{
|
||
"model": "markbase-12b",
|
||
"prompt": "Hello, world!",
|
||
"max_tokens": 100,
|
||
"temperature": 0.7,
|
||
"top_p": 0.95,
|
||
"top_k": 40,
|
||
"stream": false,
|
||
"stop": ["\n\n"]
|
||
}
|
||
```
|
||
|
||
#### 響應
|
||
|
||
```json
|
||
{
|
||
"id": "cmpl-abc123",
|
||
"object": "text_completion",
|
||
"created": 1234567890,
|
||
"model": "markbase-12b",
|
||
"choices": [
|
||
{
|
||
"index": 0,
|
||
"text": " How are you today?",
|
||
"finish_reason": "stop"
|
||
}
|
||
],
|
||
"usage": {
|
||
"prompt_tokens": 10,
|
||
"completion_tokens": 20,
|
||
"total_tokens": 30
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### POST /v1/embeddings
|
||
|
||
生成嵌入向量
|
||
|
||
#### 請求體
|
||
|
||
```json
|
||
{
|
||
"model": "markbase-12b",
|
||
"input": "Hello world",
|
||
"encoding_format": "float"
|
||
}
|
||
```
|
||
|
||
#### 響應
|
||
|
||
```json
|
||
{
|
||
"object": "list",
|
||
"data": [
|
||
{
|
||
"object": "embedding",
|
||
"index": 0,
|
||
"embedding": [0.1, 0.2, 0.3, ...],
|
||
"usage": {
|
||
"prompt_tokens": 5,
|
||
"total_tokens": 5
|
||
}
|
||
}
|
||
],
|
||
"model": "markbase-12b",
|
||
"usage": {
|
||
"prompt_tokens": 5,
|
||
"total_tokens": 5
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🖼️ 多模態格式
|
||
|
||
### 圖片輸入
|
||
|
||
```json
|
||
{
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": [
|
||
{"type": "text", "text": "What's in this image?"},
|
||
{
|
||
"type": "image_url",
|
||
"image_url": {
|
||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
|
||
"detail": "high"
|
||
}
|
||
}
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**支持的格式**：
|
||
- JPEG
|
||
- PNG
|
||
- WebP
|
||
- GIF
|
||
|
||
**大小限制**：
|
||
- 最大 20MB
|
||
- 建議 1024x1024 以下
|
||
|
||
### 音訊輸入
|
||
|
||
```json
|
||
{
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": [
|
||
{"type": "text", "text": "Transcribe this audio"},
|
||
{
|
||
"type": "audio_url",
|
||
"audio_url": {
|
||
"url": "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA="
|
||
}
|
||
}
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**支持的格式**：
|
||
- WAV
|
||
- MP3
|
||
- FLAC
|
||
- OGG
|
||
|
||
**大小限制**：
|
||
- 最大 25MB
|
||
- 建議 60 秒以下
|
||
|
||
---
|
||
|
||
## ⚠️ 錯誤格式
|
||
|
||
### 標準錯誤響應
|
||
|
||
```json
|
||
{
|
||
"error": {
|
||
"message": "Invalid request: prompt cannot be empty",
|
||
"type": "invalid_request_error",
|
||
"param": "prompt",
|
||
"code": 400
|
||
}
|
||
}
|
||
```
|
||
|
||
### 錯誤代碼
|
||
|
||
| HTTP 狀態 | 錯誤類型 | 說明 |
|
||
|:---------:|----------|------|
|
||
| 400 | `invalid_request_error` | 請求參數錯誤 |
|
||
| 401 | `authentication_error` | 認證失敗 |
|
||
| 403 | `permission_error` | 權限不足 |
|
||
| 404 | `not_found_error` | 資源不存在 |
|
||
| 429 | `rate_limit_error` | 請求頻率限制 |
|
||
| 500 | `server_error` | 服務器內部錯誤 |
|
||
| 503 | `service_unavailable` | 服務不可用 |
|
||
|
||
---
|
||
|
||
## 📊 性能指標
|
||
|
||
### 預期性能
|
||
|
||
| 模型 | 速度 | 內存 |
|
||
|------|------|------|
|
||
| E4B (4B) | ~20 tok/s | ~3GB |
|
||
| 12B | ~19 tok/s | ~9GB |
|
||
|
||
### 延遲
|
||
|
||
| 指標 | E4B | 12B |
|
||
|------|:---:|:---:|
|
||
| 首字延遲 | <100ms | <150ms |
|
||
| 平均延遲 | 50ms/token | 53ms/token |
|
||
|
||
---
|
||
|
||
## 🔐 認證
|
||
|
||
### API Key 認證（未來功能）
|
||
|
||
```bash
|
||
curl http://localhost:8080/v1/chat/completions \
|
||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"messages": [{"role": "user", "content": "Hello"}]}'
|
||
```
|
||
|
||
---
|
||
|
||
## 📈 限流（未來功能）
|
||
|
||
### 限流頭部
|
||
|
||
```
|
||
X-RateLimit-Limit: 100
|
||
X-RateLimit-Remaining: 99
|
||
X-RateLimit-Reset: 1234567890
|
||
```
|
||
|
||
---
|
||
|
||
## 📝 版本信息
|
||
|
||
### 當前版本
|
||
|
||
- API 版本：v1
|
||
- 兼容：OpenAI API v1
|
||
- 狀態：Production Ready
|
||
|
||
### 未來規劃
|
||
|
||
- [ ] 工具調用 (Function Calling)
|
||
- [ ] 結構化輸出
|
||
- [ ] 批處理 API
|
||
- [ ] 文件上傳 API
|
||
- [ ] 微調 API
|