# MarkBase-12B Swift Metal Inference Engine

## 快速开始

### 构建
```bash
cd /Users/accusys/MarkBase12B
swift build
```

### 测试
```bash
swift test
swift test --filter E4BSimpleInferenceTest.testTokenizerEncoding  # Tokenizer测试
swift test --filter E4BSimpleInferenceTest.testMultimodalVisionInference  # Multimodal测试
```

### 运行服务器（开发中）
```bash
swift run G12BServer /path/to/model 8080 markbase-e4b
```

## API Endpoints

### 文本生成
```
POST /v1/chat/completions
{
  "model": "markbase-e4b",
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 100
}
```

### 多模态生成
```
POST /v1/multimodal/chat/completions
{
  "model": "markbase-e4b",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "Describe this image"},
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
    ]
  }],
  "max_tokens": 100
}
```

## 性能指标

| Metric | Value |
|--------|-------|
| RDMA带宽 | 5761 MB/s (Thunderbolt 5) |
| POC吞吐 | 658 tokens/s (分布式) |
| Embedding验证 | Swift = Python精确匹配 |

## 架构

```
Swift Metal Engine
├── MarkBaseEngine (Metal kernels)
├── E4BModel (42 layers)
├── BPETokenizer (sentencepiece)
├── MultimodalModel
│   ├── VisionTower (16 layers)
│   ├── AudioTower (12 layers)
│   ├── Vision preprocessing
│   ├── Pooling (196→1)
│   └── Normalization
└── MarkBaseServer (API handlers)
```

## 文件结构

```
Sources/
├── G12B/                     # 核心库
│   ├── Metal/                 # Metal kernels
│   ├── Tokenizer/             # Tokenizer
│   ├── Sampling/              # Sampling strategies
│   ├── Vision/                # Vision tower
│   ├── Audio/                 # Audio tower
│   ├── Generator/             # Streaming generator
│   └── Model.swift            # Main model
│
├── G12BServer/                # API服务器
│   ├── MarkBaseServer.swift   # Main server
│   ├── MultimodalAPI.swift    # Multimodal types
│   ├── ModelsAPI.swift        # API models
│   └── Errors.swift           # Error handling
│
└── Tests/
    └── E4BSimpleInferenceTest.swift  # 测试
```

## 限制说明

**E4B-MarkBase 是 Gemma4ForConditionalGeneration (multimodal)**
- 纯文本生成产生随机输出
- 需要 vision/audio conditioning
- 这是模型架构特性，不是bug

## 下一步

1. HTTP服务器集成 (Hummingbird)
2. Python参考验证
3. Audio预处理
4. 性能优化