# Pose Action Decoder 功能文档

> 创建日期: 2026-04-28
> 脚本路径: `scripts/utils/pose_action_decoder.py`

---

## 功能概述

**Pose Action Decoder** 将 `pose_trace` 解析成人类可读的动作名称：

| 动作类型 | 示例 |
|----------|------|
| **转身动作** | turn_left, turn_right, turn_full |
| **仰俯动作** | look_up, look_down, return_neutral |
| **复杂动作** | shake_head, nod_head |
| **稳定动作** | frontal_stable, profile_right_stable |

---

## Action 分类

### 1. 简单动作

| Pose 变化 | 动作名称 |
|-----------|----------|
| frontal → three_quarter | `turn_partial` |
| frontal → profile_left | `turn_left` |
| frontal → profile_right | `turn_right` |
| three_quarter → profile_left | `turn_left` |
| three_quarter → profile_right | `turn_right` |
| profile_left → profile_right | `turn_full` |
| profile_right → profile_left | `turn_full` |
| neutral → tilted_up | `look_up` |
| neutral → tilted_down | `look_down` |

---

### 2. 复杂动作⭐

| 动作名称 | Pattern | Frame Range |
|----------|---------|-------------|
| **shake_head** | profile_left → profile_right → profile_left | 5-30 frames |
| **shake_head_reverse** | profile_right → profile_left → profile_right | 5-30 frames |
| **nod_head** | tilted_up → tilted_down → tilted_up | 3-20 frames |

**检测逻辑**:
- 3 次 pose 变化在短时间内发生
- Pattern 匹配预定义序列
- Duration 在指定范围内

---

### 3. 稳定动作

| Pose 类型 | 动作名称 | 条件 |
|-----------|----------|------|
| frontal | `frontal_stable` | duration >= 10 frames |
| three_quarter | `three_quarter_stable` | duration >= 10 frames |
| profile_left | `profile_left_stable` | duration >= 10 frames |
| profile_right | `profile_right_stable` | duration >= 10 frames |

**Pitch 修饰**:
- `three_quarter_stable_pitch_tilted_up`
- `profile_right_stable_pitch_tilted_down`

---

### 4. 短暂动作

| Pose 类型 | 动作名称 | 条件 |
|-----------|----------|------|
| Any | `pose_<angle>_brief` | duration < 10 frames |

**说明**: 短暂 pose 通常是过渡状态。

---

## 输出结构

### 1. action_timeline

```json
{
  "action_timeline": [
    {
      "frame": 155,                         // 帧号
      "action": "profile_right_stable",     // 动作名称
      "duration_frames": 18,                // 持续帧数
      "description": "stable profile_right pose for 18 frames",
      "type": "stable"                      // 类型: stable/transitional/transition/complex
    },
    {
      "frame": 173,
      "action": "turn_to_three_quarter",
      "duration_frames": 1,
      "description": "transition from profile_right to three_quarter",
      "type": "transition"
    },
    ...                                     // 共 17 个
  ]
}
```

---

### 2. action_summary

```json
{
  "action_summary": {
    "total_actions": 17,                    // 总动作数
    "unique_actions": 6,                    // 唯一动作数
    "action_counts": {                      // 动作计数
      "turn_right": 4,
      "turn_to_three_quarter": 4,
      "profile_right_stable": 3,
      "pose_three_quarter_brief": 3,
      "pose_profile_right_brief": 2,
      "three_quarter_stable": 1
    },
    "action_durations_frames": {            // 动作总持续时间
      "profile_right_stable": 106,
      "three_quarter_stable": 11,
      ...
    },
    "complex_action_count": 0,              // 复杂动作数
    "stable_percentage": 23.5               // 稳定动作百分比
  }
}
```

---

### 3. complex_actions

```json
{
  "complex_actions": [
    {
      "action": "shake_head",
      "start_frame": 100,
      "end_frame": 115,
      "duration_frames": 15,
      "description": "shake head left-right-left"
    },
    {
      "action": "nod_head",
      "start_frame": 200,
      "end_frame": 210,
      "duration_frames": 10,
      "description": "nod head up-down"
    }
  ]
}
```

---

### 4. Human-readable Description

**Trace 2 示例**:
```
Stable poses: stable profile_right pose for 18 frames, stable three_quarter pose for 11 frames, stable profile_right pose for 71 frames.
Transitions: turn_to_three_quarter, turn_right, turn_to_three_quarter, turn_right, turn_to_three_quarter
```

---

## 使用方式

### 基础用法

```bash
# 解析所有 traces
python3 scripts/utils/pose_action_decoder.py \
  --face-json video.face_traced.json \
  --output-json pose_action_data.json \
  --output-plot pose_action_timeline.png

# 仅解析特定 trace
python3 scripts/utils/pose_action_decoder.py \
  --face-json video.face_traced.json \
  --trace-id 2 \
  --output-json pose_action_trace2.json
```

---

### 输出文件

| 文件 | 内容 |
|------|------|
| **JSON** | action_timeline, action_summary, complex_actions |
| **PNG** | Action timeline 可视化（色块表示不同动作） |

---

## 实测案例

### Trace 2 分析（preview.mp4）

| 指标 | 值 |
|------|-----|
| **Total Actions** | 17 |
| **Unique Actions** | 6 |
| **Stable Percentage** | 23.5% |
| **Complex Actions** | 0 |

**Action Counts**:
```
turn_right: 4               → 4 次右转
turn_to_three_quarter: 4    → 4 次转到 three_quarter
profile_right_stable: 3     → 3 段稳定右侧面
pose_three_quarter_brief: 3 → 3 段短暂 three_quarter
pose_profile_right_brief: 2 → 2 段短暂右侧面
three_quarter_stable: 1     → 1 段稳定 three_quarter
```

**Human-readable Description**:
```
Stable poses: 
  - stable profile_right pose for 18 frames (frame 155)
  - stable three_quarter pose for 11 frames (frame 177)
  - stable profile_right pose for 71 frames (frame 188) ✅ 最长稳定

Transitions: 
  - turn_to_three_quarter (4 times)
  - turn_right (4 times)
```

---

### Trace 3 分析（完全稳定）

| 指标 | 值 |
|------|-----|
| **Total Actions** | 1 |
| **Stable Percentage** | **100%** ✅ |

**Action Counts**:
```
profile_left_stable: 1      → 1 段稳定左侧面（32 frames）
```

**说明**: Trace 3 无 pose 变化，完全稳定。

---

## Action Timeline 可视化

### PNG 输出

- **色块**: 不同颜色表示不同动作类型
- **宽度**: 色块宽度 = 动作持续时间
- **标签**: stable actions (> 30 frames) 显示名称
- **虚线**: transition actions（瞬间动作）

### 颜色映射

| Action | Color |
|--------|-------|
| frontal_stable | Green |
| three_quarter_stable | Blue |
| profile_left_stable | Orange |
| profile_right_stable | Red |
| turn_left/right | Purple |
| shake_head | Yellow |
| nod_head | Cyan |

---

## 应用场景

| 场景 | 用途 |
|------|------|
| **视频摘要** | 自动生成动作描述 |
| **行为分析** | 统计转身、点头、摇头次数 |
| **质量控制** | 检测 pose 稳定性（stable_percentage） |
| **片段剪辑** | 根据 action_timeline 定位关键片段 |

---

## 与 Face Tracker 整合

### 完整流程

```bash
# 1. Face detection
python3 scripts/face_processor.py video.mp4 video.face.json --sample-interval 1

# 2. Face tracking
python3 scripts/utils/face_tracker.py \
  --face-json video.face.json \
  --output video.face_traced.json

# 3. Pose transition analysis
python3 scripts/utils/pose_transition_analyzer.py \
  --face-json video.face_traced.json \
  --output-json pose_transition_analysis.json

# 4. Pose action decoding
python3 scripts/utils/pose_action_decoder.py \
  --face-json video.face_traced.json \
  --output-json pose_action_data.json \
  --output-plot pose_action_timeline.png
```

---

## Action 数据应用

### 1. 视频摘要生成

```python
# 从 action_timeline 生成摘要
summary = f"""
视频中检测到 {total_traces} 个人物:
- Trace 2: {action_summary['total_actions']} 个动作
  主要动作: {dominant_actions}
  稳定性: {action_summary['stable_percentage']}%
"""
```

---

### 2. 关键片段定位

```python
# 定位 shake_head 片段
for action in action_timeline:
    if action['action'] == 'shake_head':
        clip_range = (action['start_frame'], action['end_frame'])
        # 提取片段进行剪辑
```

---

### 3. 行为统计

```python
# 统计转身次数
turn_count = sum(1 for a in action_timeline if a['action'].startswith('turn_'))

# 统计点头/摇头次数
nod_count = sum(1 for a in complex_actions if a['action'] == 'nod_head')
shake_count = sum(1 for a in complex_actions if a['action'] == 'shake_head')
```

---

## 未来改进

| Phase | 功能 | 优先级 |
|-------|------|--------|
| **Phase 1** | 基础 Action 解析（已完成） | ✅ |
| **Phase 2** | 添加更多复杂动作 pattern | 中 |
| **Phase 3** | Action-based video segmentation | 低 |
| **Phase 4** | Real-time action detection API | 低 |

---

## 参考文档

| 文件 | 说明 |
|------|------|
| `scripts/utils/pose_action_decoder.py` | Action 解析脚本 |
| `scripts/utils/pose_transition_analyzer.py` | Pose transition 分析 |
| `scripts/utils/face_tracker.py` | Face tracking |
| `docs_v1.0/FACE_TRACKER_DATA_STRUCTURE.md` | Trace 数据结构 |

---

## 版本信息

- 版本: 1.0
- 创建日期: 2026-04-28
- 状态: ✅ Pose Action Decoder 完成
