# Momentry Core Processors 快速参考

**更新日期**: 2026-04-28

---

## 完成度总览

| # | 模块 | 功能 | 完成度 | Python | Rust | API | 数据库 | 备注 |
|---|------|------|--------|--------|------|-----|--------|------|
| 1 | **ASR** | 语音识别 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | WhisperX |
| 2 | **ASRX** | 说话人分离 | ⚠️ 80% | ✅ | ✅ | ✅ | ✅ | **修复中** |
| 3 | **CUT** | 场景检测 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | PySceneDetect |
| 4 | **YOLO** | 物体检测 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | YOLOv8 |
| 5 | **OCR** | 文字识别 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | PaddleOCR |
| 6 | **Face** | 人脸检测 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | InsightFace ⭐ |
| 7 | **Pose** | 姿态估计 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | MediaPipe |
| 8 | **Scene** | 场景分类 | ✅ 100% | ✅ | ✅ | ✅ | ✅ | **MIT Places365** ⭐ |
| 9 | **Caption** | 字幕生成 | ✅ 100% | ✅ | ✅ | ✅ | ⚠️ | GPT-4V (付费) |
| 10 | **Story** | 故事生成 | ✅ 100% | ✅ | ✅ | ✅ | ⚠️ | GPT-4 (付费) |

**统计**:
- ✅ 完成: 8/10 (80%)
- ⚠️ 修复中: 1/10 (10%)
- ⚠️ 待数据库: 2/10 (20%)
- 💰 付费 API: 2/10 (Caption, Story)
- ⭐ Benchmark完成: 4/10 (Face, YOLO, CUT, Scene)

---

## 快速参考

### ASR (语音识别)
```bash
# 使用
python3 scripts/asr_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/asr/384b0ff44aaaa1f14cb2cd63b3fea966

# 示例
ExaSAN: 78 segments, 15KB
Charade: 1826 segments, 198KB
```

### ASRX (说话人分离) ⚠️
```bash
# 使用 (修复后)
python3 scripts/asrx_processor_custom.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/asrx/384b0ff44aaaa1f14cb2cd63b3fea966

# 测试结果
Charade: 1118 segments, 8 speakers, 99.82% match rate
```

**问题**: 原实现 (pyannote) 不可用  
**修复**: 使用自定义 SpeechBrain 实现  
**状态**: 已测试通过，待切换

### CUT (场景检测)
```bash
# 使用
python3 scripts/cut_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/cut/384b0ff44aaaa1f14cb2cd63b3fea966

# 示例
Charade: 1331 scenes, 217KB
ExaSAN: 18 scenes, 2KB
```

### YOLO (物体检测)
```bash
# 使用
python3 scripts/yolo_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/yolo/384b0ff44aaaa1f14cb2cd63b3fea966

# 示例
Charade: 127MB, 15234 objects, 80 classes
ExaSAN: 3.2MB, 456 objects
```

### OCR (文字识别)
```bash
# 使用
python3 scripts/ocr_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/ocr/9760d0820f0cf9a7

# 示例
ExaSAN: 102 frames, 234 texts, 65KB
```

### Face (人脸检测)
```bash
# 使用
python3 scripts/face_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/face/9760d0820f0cf9a7

# 示例
ExaSAN: 49 frames, 67 faces, 12KB
Charade: 1116 faces, 8 unique persons
```

### Pose (姿态估计)
```bash
# 使用
python3 scripts/pose_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/pose/9760d0820f0cf9a7

# 示例
ExaSAN: 1853 frames, 2341 persons, 603KB
```

### Scene (场景分类) - MIT Places365
```bash
# 使用
python3 scripts/scene_classifier.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/scene/<uuid>

# 模型
ResNet18-Places365 (43MB)
365 scene categories
```

**类别示例**:
```
living_room, bedroom, kitchen, bathroom, restaurant,
gym, supermarket, basketball_court, park, beach, airport...
```

### Caption (字幕生成) 💰
```bash
# 使用
python3 scripts/caption_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/caption/<uuid>

# 配置
export OPENAI_API_KEY="sk-..."
```

### Story (故事生成) 💰
```bash
# 使用
python3 scripts/story_processor.py video.mp4 output.json

# API
curl http://localhost:3002/api/v1/story/<uuid>

# 配置
export OPENAI_API_KEY="sk-..."
```

---

## 数据库表

### PostgreSQL

```sql
-- 通用 chunks 表
chunks (id, uuid, chunk_type, start_time, end_time, text, embedding)

-- Face 相关
face_detections (id, uuid, frame_number, face_id, bbox, confidence, embedding, cluster_id)
person_identities (id, uuid, person_name, actor_name, character_name, face_embedding)
person_appearances (id, uuid, person_identity_id, face_detection_id, asrx_segment_start, asrx_segment_end, confidence)

-- Jobs
jobs (id, uuid, processor_type, status, progress, error_message, created_at, started_at, completed_at)
```

### MongoDB

```javascript
// YOLO
db.yolo_frames.insertOne({uuid, frame_number, objects})

// OCR
db.ocr_frames.insertOne({uuid, frame_number, texts})

// Pose
db.pose_frames.insertOne({uuid, frame_number, persons})

// Story
db.story_chunks.insertOne({uuid, parent_chunks, child_chunks, stats})
```

---

## 批量处理

```bash
# 处理所有模块
cargo run -- process video.mp4 --modules all

# 处理指定模块
cargo run -- process video.mp4 --modules asr,cut,yolo

# 强制重新处理
cargo run -- process video.mp4 --modules asr --force
```

---

## 性能 (Apple Silicon M4)

| 模块 | 实时倍速 | FPS | 备注 |
|------|----------|-----|------|
| ASR | 10-20x | - | WhisperX |
| ASRX | 150x | - | SpeechBrain |
| CUT | 50-100x | - | PySceneDetect |
| YOLO | 20-40x | 100-200 | YOLOv8n |
| OCR | 30-50x | - | PaddleOCR |
| Face | 15-30x | - | RetinaFace |
| Pose | 25-40x | - | MediaPipe |

---

## 待办事项

### 高优先级
- [x] Scene: 添加数据库存储 ✅ (2026-04-28)
- [ ] ASRX: 切换到自定义 SpeechBrain 实现
- [ ] Caption: 添加数据库存储
- [ ] Story: 添加数据库存储

### 已完成 (2026-04-28)
- [x] **Scene Processor**: ProcessorType + store_scene_pre_chunks_batch + Benchmark测试
- [x] **CUT Processor**: PySceneDetect Benchmark测试 (2.54秒, 19场景)
- [x] **YOLO Processor**: CPU版本 Benchmark测试 (111.81秒, 8486物体, 26类)
- [x] **Face Processor**: InsightFace Benchmark测试 (7.04秒, 112人脸, 100%检测率) ⭐

### 中优先级
- [ ] 统一 API 错误处理
- [ ] 添加批量处理接口
- [ ] 优化 GPU/MPS 内存使用

---

**详细文档**: `docs/PROCESSOR_IMPLEMENTATION_STATUS.md`
