MarkBase Admin
ac75faa0cc
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
614 lines
13 KiB
Markdown
614 lines
13 KiB
Markdown
# MarkBase 功能补充路线图
|
||
|
||
## 目标定位
|
||
|
||
**MarkBase 定位**：
|
||
- Apple Silicon 专属高性能推理引擎
|
||
- Swift 生态系统集成
|
||
- 教育研究 + 原型开发平台
|
||
- iOS/macOS 应用后端集成
|
||
|
||
**不竞争**：
|
||
- 生产级多GPU服务（vLLM领域）
|
||
- 跨平台通用部署（llama.cpp领域）
|
||
- 一键易用工具（ollama领域）
|
||
|
||
---
|
||
|
||
## Phase 1: 核心功能完善（必需）
|
||
|
||
### 1.1 Tokenizer 集成
|
||
|
||
**目标**：支持文本输入，无需手动token ID
|
||
|
||
**实现方案**：
|
||
```swift
|
||
// Tokenizer protocols
|
||
public protocol Tokenizer {
|
||
func encode(text: String) -> [Int]
|
||
func decode(tokens: [Int]) -> String
|
||
var vocabSize: Int { get }
|
||
}
|
||
|
||
// SentencePiece tokenizer (Gemma使用)
|
||
public final class SentencePieceTokenizer: Tokenizer {
|
||
private let model: SentencePieceModel
|
||
private let vocab: [String: Int]
|
||
private let reverseVocab: [Int: String]
|
||
|
||
public init(modelPath: String) throws {
|
||
// Load .model or .tokenizer.json
|
||
}
|
||
|
||
public func encode(text: String) -> [Int] {
|
||
// BPE encoding algorithm
|
||
}
|
||
|
||
public func decode(tokens: [Int]) -> String {
|
||
// Token to text conversion
|
||
}
|
||
}
|
||
```
|
||
|
||
**文件结构**：
|
||
```
|
||
Sources/G12B/Tokenizer/
|
||
├── Tokenizer.swift (protocol)
|
||
├── SentencePieceTokenizer.swift
|
||
├── BPETokenizer.swift
|
||
└── TokenizerLoader.swift
|
||
```
|
||
|
||
**依赖**：
|
||
- 无外部依赖（纯Swift实现）
|
||
- 或集成 `swift-sentencepiece`（轻量库）
|
||
|
||
**时间估算**：2-3天
|
||
- Day 1: 协议定义 + SentencePiece解析
|
||
- Day 2: Encode/decode实现 + 测试
|
||
- Day 3: Gemma tokenizer适配 + 集成
|
||
|
||
**测试验证**：
|
||
```swift
|
||
let tokenizer = try SentencePieceTokenizer(modelPath: modelDir)
|
||
let tokens = tokenizer.encode("Hello world")
|
||
let text = tokenizer.decode(tokens)
|
||
XCTAssertEqual(text, "Hello world")
|
||
```
|
||
|
||
---
|
||
|
||
### 1.2 流式输出
|
||
|
||
**目标**：Token-by-token生成，实时显示
|
||
|
||
**实现方案**：
|
||
```swift
|
||
public final class StreamingGenerator {
|
||
private let model: E4BModel
|
||
private let tokenizer: Tokenizer
|
||
private let engine: MarkBaseEngine
|
||
|
||
public func generate(
|
||
prompt: String,
|
||
maxTokens: Int,
|
||
temperature: Float = 1.0
|
||
) -> AsyncStream<String> {
|
||
// AsyncStream for token-by-token output
|
||
return AsyncStream { continuation in
|
||
// Generation loop
|
||
for token in generatedTokens {
|
||
let text = tokenizer.decode([token])
|
||
continuation.yield(text)
|
||
}
|
||
continuation.finish()
|
||
}
|
||
}
|
||
}
|
||
|
||
// Usage
|
||
let generator = StreamingGenerator(model: model, tokenizer: tokenizer)
|
||
for await tokenText in generator.generate(prompt: "Hello", maxTokens: 100) {
|
||
print(tokenText) // Real-time output
|
||
}
|
||
```
|
||
|
||
**技术要点**：
|
||
- 使用 Swift `AsyncStream`（异步流）
|
||
- 每生成一个token立即输出
|
||
- 支持异步取消
|
||
|
||
**文件结构**：
|
||
```
|
||
Sources/G12B/Generator/
|
||
├── StreamingGenerator.swift
|
||
├── GenerationConfig.swift
|
||
```
|
||
|
||
**时间估算**：1天
|
||
|
||
---
|
||
|
||
### 1.3 采样策略
|
||
|
||
**目标**：支持Top-k、Top-p、Temperature等采样
|
||
|
||
**实现方案**：
|
||
```swift
|
||
public struct SamplingConfig {
|
||
public let temperature: Float // 0.0-2.0
|
||
public let topK: Int? // Top-k sampling
|
||
public let topP: Float? // Top-p (nucleus) sampling
|
||
public let repetitionPenalty: Float?
|
||
|
||
public init(temperature: Float = 1.0, topK: Int? = nil, topP: Float? = nil) {
|
||
self.temperature = temperature
|
||
self.topK = topK
|
||
self.topP = topP
|
||
}
|
||
}
|
||
|
||
public final class Sampler {
|
||
public func sample(logits: [Float], config: SamplingConfig) -> Int {
|
||
// Apply temperature
|
||
var probs = softmax(logits.map { $0 / config.temperature })
|
||
|
||
// Top-k filtering
|
||
if let k = config.topK {
|
||
probs = applyTopK(probs, k: k)
|
||
}
|
||
|
||
// Top-p filtering
|
||
if let p = config.topP {
|
||
probs = applyTopP(probs, p: p)
|
||
}
|
||
|
||
// Random sampling
|
||
return randomSample(probs)
|
||
}
|
||
|
||
private func softmax(_ values: [Float]) -> [Float]
|
||
private func applyTopK(_ probs: [Float], k: Int) -> [Float]
|
||
private func applyTopP(_ probs: [Float], p: Float) -> [Float]
|
||
}
|
||
```
|
||
|
||
**文件结构**：
|
||
```
|
||
Sources/G12B/Sampling/
|
||
├── Sampler.swift
|
||
├── SamplingConfig.swift
|
||
├── Softmax.swift (Metal kernel)
|
||
```
|
||
|
||
**时间估算**：1-2天
|
||
- Day 1: 采样算法实现 + Softmax Metal kernel
|
||
- Day 2: 测试 + 验证生成质量
|
||
|
||
---
|
||
|
||
## Phase 2: 生产功能增强（重要）
|
||
|
||
### 2.1 HTTP API服务
|
||
|
||
**目标**：提供REST API endpoint
|
||
|
||
**实现方案**：
|
||
```swift
|
||
// 使用 Vapor 或 Hummingbird (轻量)
|
||
import Hummingbird
|
||
|
||
public final class InferenceAPI {
|
||
private let generator: StreamingGenerator
|
||
|
||
public func startServer(port: Int = 8080) throws {
|
||
let app = HBApplication(port: port)
|
||
|
||
// POST /generate
|
||
app.router.post("/generate") { request, context in
|
||
let body = try request.body.decode(GenerateRequest.self)
|
||
|
||
let result = try generator.generate(
|
||
prompt: body.prompt,
|
||
maxTokens: body.maxTokens ?? 100,
|
||
config: body.config ?? SamplingConfig()
|
||
)
|
||
|
||
return GenerateResponse(tokens: result)
|
||
}
|
||
|
||
// POST /stream (WebSocket)
|
||
app.router.post("/stream") { ... }
|
||
|
||
try app.start()
|
||
}
|
||
}
|
||
|
||
struct GenerateRequest: Codable {
|
||
let prompt: String
|
||
let maxTokens: Int?
|
||
let config: SamplingConfig?
|
||
}
|
||
|
||
struct GenerateResponse: Codable {
|
||
let tokens: [Int]
|
||
let text: String
|
||
}
|
||
```
|
||
|
||
**API设计**：
|
||
- `POST /generate` - 单次生成
|
||
- `POST /stream` - 流式生成（WebSocket）
|
||
- `GET /models` - 模型列表
|
||
- `GET /health` - 健康检查
|
||
|
||
**依赖选择**：
|
||
- **Hummingbird**（推荐）：轻量、Swift原生
|
||
- **Vapor**：功能完整、但较重
|
||
|
||
**文件结构**：
|
||
```
|
||
Sources/G12B/API/
|
||
├── InferenceAPI.swift
|
||
├── APIModels.swift
|
||
├── Routes.swift
|
||
```
|
||
|
||
**时间估算**：3-4天
|
||
- Day 1: API框架搭建 + 基础endpoint
|
||
- Day 2: 请求处理 + 错误处理
|
||
- Day 3: WebSocket流式输出
|
||
- Day 4: 测试 + 文档
|
||
|
||
---
|
||
|
||
### 2.2 并发支持
|
||
|
||
**目标**：多request并发处理
|
||
|
||
**实现方案**：
|
||
```swift
|
||
public final class ConcurrentGenerator {
|
||
private let model: E4BModel
|
||
private let tokenizer: Tokenizer
|
||
private let engine: MarkBaseEngine
|
||
private let queue: DispatchQueue
|
||
|
||
// Batch processing with KV cache sharing
|
||
public func generateBatch(
|
||
prompts: [String],
|
||
maxTokens: Int
|
||
) async throws -> [String] {
|
||
return try await withThrowingTaskGroup(of: String.self) { group in
|
||
for prompt in prompts {
|
||
group.addTask {
|
||
try await generateSingle(prompt: prompt, maxTokens: maxTokens)
|
||
}
|
||
}
|
||
|
||
var results: [String] = []
|
||
for try await result in group {
|
||
results.append(result)
|
||
}
|
||
return results
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**技术要点**：
|
||
- Swift async/await并发
|
||
- DispatchQueue调度
|
||
- 批处理KV cache优化
|
||
|
||
**文件结构**：
|
||
```
|
||
Sources/G12B/Concurrent/
|
||
├── ConcurrentGenerator.swift
|
||
├── RequestQueue.swift
|
||
```
|
||
|
||
**时间估算**：2-3天
|
||
|
||
---
|
||
|
||
## Phase 3: 生态完善（可选）
|
||
|
||
### 3.1 模型自动下载
|
||
|
||
**目标**：自动从HuggingFace下载模型
|
||
|
||
```swift
|
||
public final class ModelDownloader {
|
||
public func download(
|
||
modelId: String,
|
||
cacheDir: String = "~/.cache/huggingface"
|
||
) async throws -> String {
|
||
// Download from HuggingFace Hub
|
||
// Use huggingface-cli or custom implementation
|
||
}
|
||
}
|
||
```
|
||
|
||
**时间估算**：2-3天
|
||
|
||
---
|
||
|
||
### 3.2 iOS/macOS应用集成
|
||
|
||
**目标**：提供App框架模板
|
||
|
||
```swift
|
||
// SwiftUI integration
|
||
public struct ChatView: View {
|
||
@StateObject private var chatModel = ChatModel()
|
||
|
||
var body: some View {
|
||
VStack {
|
||
// Chat UI
|
||
}
|
||
}
|
||
}
|
||
|
||
public final class ChatModel: ObservableObject {
|
||
private let generator: StreamingGenerator
|
||
@Published var messages: [Message] = []
|
||
}
|
||
```
|
||
|
||
**时间估算**：5-7天
|
||
|
||
---
|
||
|
||
## 实施优先级
|
||
|
||
### 第一阶段（必需，4-6天）
|
||
|
||
| 功能 | 时间 | 依赖 | 优先级 |
|
||
|------|------|------|--------|
|
||
| Tokenizer集成 | 2-3天 | 无 | ⭐⭐⭐⭐⭐ |
|
||
| 流式输出 | 1天 | Tokenizer | ⭐⭐⭐⭐⭐ |
|
||
| 采样策略 | 1-2天 | 无 | ⭐⭐⭐⭐ |
|
||
|
||
**完成后效果**：
|
||
- ✅ 可直接输入文本（无需手动token）
|
||
- ✅ 实时流式输出
|
||
- ✅ 灵活采样策略
|
||
- ✅ 完整文本生成体验
|
||
|
||
---
|
||
|
||
### 第二阶段（重要，5-7天）
|
||
|
||
| 功能 | 时间 | 依赖 | 优先级 |
|
||
|------|------|------|--------|
|
||
| HTTP API | 3-4天 | Tokenizer, 采样 | ⭐⭐⭐⭐ |
|
||
| 并发支持 | 2-3天 | API | ⭐⭐⭐ |
|
||
|
||
**完成后效果**：
|
||
- ✅ REST API可用
|
||
- ✅ 多request并发
|
||
- ✅ 服务级部署
|
||
|
||
---
|
||
|
||
### 第三阶段（可选，7-10天）
|
||
|
||
| 功能 | 时间 | 依赖 | 优先级 |
|
||
|------|------|------|--------|
|
||
| 模型自动下载 | 2-3天 | 无 | ⭐⭐ |
|
||
| iOS/macOS App模板 | 5-7天 | API | ⭐⭐ |
|
||
|
||
---
|
||
|
||
## 兼容性设计
|
||
|
||
### E4B和12B统一接口
|
||
|
||
```swift
|
||
// Unified generation interface
|
||
public protocol TextGenerator {
|
||
func generate(
|
||
prompt: String,
|
||
maxTokens: Int,
|
||
config: SamplingConfig
|
||
) throws -> String
|
||
|
||
func streamGenerate(
|
||
prompt: String,
|
||
maxTokens: Int,
|
||
config: SamplingConfig
|
||
) -> AsyncStream<String>
|
||
}
|
||
|
||
// E4B和12B都实现此协议
|
||
extension E4BModel: TextGenerator { ... }
|
||
extension MultimodalModel: TextGenerator { ... }
|
||
```
|
||
|
||
**设计原则**：
|
||
- E4B和12B共享相同接口
|
||
- Tokenizer统一加载
|
||
- 采样策略通用
|
||
- API统一endpoint
|
||
|
||
---
|
||
|
||
## 技术栈选择
|
||
|
||
### 依赖库（推荐）
|
||
|
||
| 功能 | 推荐库 | 原因 |
|
||
|------|--------|------|
|
||
| **HTTP框架** | Hummingbird | 轻量、Swift原生 |
|
||
| **Tokenizer** | 纯Swift实现 | 无外部依赖 |
|
||
| **异步并发** | Swift AsyncStream | 语言原生 |
|
||
| **JSON处理** | Codable | 语言原生 |
|
||
|
||
**避免依赖**：
|
||
- ❌ Vapor（太重）
|
||
- ❌ 外部tokenizer库（Swift生态少）
|
||
- ❌ Python互操作（破坏纯Swift）
|
||
|
||
---
|
||
|
||
## 测试策略
|
||
|
||
### 每阶段测试
|
||
|
||
**Phase 1测试**：
|
||
```swift
|
||
// Tokenizer测试
|
||
func testTokenizer() throws {
|
||
let tokenizer = try SentencePieceTokenizer(modelPath: modelDir)
|
||
let tokens = tokenizer.encode("Hello world")
|
||
XCTAssertEqual(tokens.count, > 0)
|
||
let decoded = tokenizer.decode(tokens)
|
||
XCTAssertEqual(decoded, "Hello world")
|
||
}
|
||
|
||
// 流式输出测试
|
||
func testStreaming() async throws {
|
||
let generator = StreamingGenerator(model: model, tokenizer: tokenizer)
|
||
var tokens: [String] = []
|
||
for await token in generator.generate(prompt: "Test", maxTokens: 10) {
|
||
tokens.append(token)
|
||
}
|
||
XCTAssertEqual(tokens.count, 10)
|
||
}
|
||
|
||
// 采样测试
|
||
func testSampling() throws {
|
||
let sampler = Sampler()
|
||
let config = SamplingConfig(temperature: 0.8, topK: 50)
|
||
let logits = model.forward(tokenId: 0, position: 0)
|
||
let token = sampler.sample(logits: logits, config: config)
|
||
XCTAssertGreaterThanOrEqual(token, 0)
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 文档更新
|
||
|
||
### 每阶段更新文档
|
||
|
||
**Phase 1完成后**：
|
||
- README.md更新（Tokenizer + Streaming示例）
|
||
- API_REFERENCE.md新增
|
||
- QUICK_START.md快速指南
|
||
|
||
**Phase 2完成后**：
|
||
- API_SERVER.md（HTTP endpoint文档）
|
||
- DEPLOYMENT.md（部署指南）
|
||
|
||
---
|
||
|
||
## 实施建议
|
||
|
||
### 方案A：快速原型（推荐）
|
||
|
||
**时间**：4-6天（Phase 1）
|
||
|
||
**目标**：
|
||
- ✅ Tokenizer集成
|
||
- ✅ 流式输出
|
||
- ✅ 采样策略
|
||
|
||
**效果**：
|
||
- 完整文本生成体验
|
||
- 媒体演示可用
|
||
- 教育价值最大化
|
||
|
||
---
|
||
|
||
### 方案B：生产级（可选）
|
||
|
||
**时间**：9-13天（Phase 1+2）
|
||
|
||
**目标**：
|
||
- ✅ Phase 1功能
|
||
- ✅ HTTP API
|
||
- ✅ 并发支持
|
||
|
||
**效果**：
|
||
- 服务级部署
|
||
- 多用户访问
|
||
- API可用
|
||
|
||
---
|
||
|
||
### 方案C：完整生态（不推荐）
|
||
|
||
**时间**：16-23天（Phase 1+2+3）
|
||
|
||
**投入产出低**：
|
||
- 不竞争ollama易用性
|
||
- 不竞争vLLM生产级
|
||
- 定位错位
|
||
|
||
---
|
||
|
||
## 关键决策
|
||
|
||
**需要回答**：
|
||
1. **目标用户是谁？**
|
||
- Swift开发者？研究者？生产用户？
|
||
|
||
2. **投入预算？**
|
||
- 4-6天？9-13天？16+天？
|
||
|
||
3. **定位策略？**
|
||
- 教育研究工具？
|
||
- iOS/macOS应用后端？
|
||
- API服务提供者？
|
||
|
||
---
|
||
|
||
## 我的推荐
|
||
|
||
**推荐方案A（快速原型）**
|
||
|
||
**理由**：
|
||
1. **投入产出最优**
|
||
- 4-6天投入
|
||
- 完整文本生成体验
|
||
- 教育演示价值最大化
|
||
|
||
2. **定位正确**
|
||
- 教育研究工具
|
||
- Swift开发者友好
|
||
- Apple Silicon专属
|
||
|
||
3. **避免竞争**
|
||
- 不与ollama竞争易用性
|
||
- 不与vLLM竞争生产级
|
||
- 保持差异化优势
|
||
|
||
**下一步行动**：
|
||
- 用户确认方案选择
|
||
- 开始Phase 1实施（Tokenizer + Streaming + Sampling）
|
||
|
||
---
|
||
|
||
## 总结
|
||
|
||
**MarkBase核心竞争力**：
|
||
- ✅ Apple Silicon性能优化
|
||
- ✅ 纯Swift原生实现
|
||
- ✅ 教育研究价值
|
||
- ✅ 完全定制能力
|
||
|
||
**功能缺口**：
|
||
- ❌ Tokenizer（必需）
|
||
- ❌ 流式输出（必需）
|
||
- ❌ 采样策略（必需）
|
||
- ⚠️ API服务（可选）
|
||
|
||
**最优策略**：
|
||
- Phase 1实施（4-6天）
|
||
- 定位为教育/研究工具
|
||
- 保持Swift生态特色
|
||
- 不竞争生产市场
|
||
|
||
是否开始Phase 1实施？ |