feat: add pose and appearance endpoints for face detail display

- Add GET /api/v1/file/:file_uuid/pose endpoint - Add GET /api/v1/file/:file_uuid/appearance endpoint - Keypoint deduplication (highest confidence) - Add face-groups endpoint for Studio integration - Add profile routes (trace-profile, file-profile) - Update API documentation
2026-07-19 20:43:35 +08:00
parent 455c6b81e6
commit d0b7bfa56b
7 changed files with 824 additions and 2 deletions
@@ -0,0 +1,300 @@
+# Pose & Appearance API POC Plan
+
+**Date**: 2026-07-19
+**Author**: Core API Team
+**Status**: Ready for Implementation
+**Target**: Studio Team
+
+---
+
+## 1. Overview
+
+This document outlines the POC implementation for pose skeleton and appearance color display in Momentry Studio's Face Detail Modal.
+
+---
+
+## 2. Endpoints
+
+### 2.1 Get Pose
+
+```
+GET /api/v1/file/:file_uuid/pose?frame=:frame_no
+```
+
+**Parameters:**
+
+| Name | Type | Required | Description |
+|------|------|----------|-------------|
+| `file_uuid` | string | Yes | File UUID (path parameter) |
+| `frame` | integer | Yes | Frame number (query parameter) |
+
+**Response (200):**
+
+```json
+{
+  "frame": 303,
+  "keypoints": [
+    {"name": "nose", "x": 315.9, "y": 364.2, "confidence": 0.616},
+    {"name": "left_eye", "x": 312.4, "y": 362.8, "confidence": 0.616},
+    {"name": "right_eye", "x": 316.2, "y": 361.4, "confidence": 0.616},
+    {"name": "left_ear", "x": 343.1, "y": 354.7, "confidence": 0.856},
+    {"name": "right_ear", "x": 0, "y": 0, "confidence": 0},
+    {"name": "left_shoulder", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_shoulder", "x": 296.0, "y": 410.8, "confidence": 0.863},
+    {"name": "left_elbow", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_elbow", "x": 0, "y": 0, "confidence": 0},
+    {"name": "left_wrist", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_wrist", "x": 0, "y": 0, "confidence": 0},
+    {"name": "left_hip", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_hip", "x": 0, "y": 0, "confidence": 0},
+    {"name": "left_knee", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_knee", "x": 0, "y": 0, "confidence": 0},
+    {"name": "left_ankle", "x": 0, "y": 0, "confidence": 0},
+    {"name": "right_ankle", "x": 0, "y": 0, "confidence": 0}
+  ],
+  "pose_class": "unknown",
+  "confidence": null
+}
+```
+
+**Fields:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `frame` | integer | Frame number |
+| `keypoints` | array | 17 COCO keypoints (deduplicated) |
+| `keypoints[].name` | string | Keypoint name |
+| `keypoints[].x` | float | X coordinate in pixels |
+| `keypoints[].y` | float | Y coordinate in pixels |
+| `keypoints[].confidence` | float | Detection confidence 0-1 (0 means not detected) |
+| `pose_class` | string | Always `"unknown"` in POC |
+| `confidence` | null | Always `null` in POC |
+
+**Error Responses:**
+
+| HTTP | When |
+|------|------|
+| 404 | Frame not found or pose.json not found |
+| 401 | Missing or invalid API key |
+
+---
+
+### 2.2 Get Appearance
+
+```
+GET /api/v1/file/:file_uuid/appearance?frame=:frame_no
+```
+
+**Parameters:**
+
+| Name | Type | Required | Description |
+|------|------|----------|-------------|
+| `file_uuid` | string | Yes | File UUID (path parameter) |
+| `frame` | integer | Yes | Frame number (query parameter) |
+
+**Response (200):**
+
+```json
+{
+  "frame": 0,
+  "dominant_colors": [],
+  "hsv_histogram": [
+    [0.59, 0, 0, ...],
+    [0.97, 0.02, 0.01, ...],
+    [0.004, 0.02, 0.03, ...]
+  ]
+}
+```
+
+**Fields:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `frame` | integer | Frame number |
+| `dominant_colors` | array | Empty array in POC (`[]`) |
+| `hsv_histogram` | array | 3x30 HSV histogram bins |
+
+**Error Responses:**
+
+| HTTP | When |
+|------|------|
+| 404 | Frame not found or appearance.json not found |
+| 401 | Missing or invalid API key |
+
+---
+
+## 3. Implementation Notes
+
+### 3.1 Data Source
+
+| Processor | File | Location |
+|-----------|------|----------|
+| Pose | `{file_uuid}.pose.json` | `{OUTPUT_DIR}/{file_uuid}/` |
+| Appearance | `{file_uuid}.appearance.json` | `{OUTPUT_DIR}/{file_uuid}/` |
+
+### 3.2 Keypoint Deduplication
+
+The pose processor may output multiple keypoints with the same name (e.g., 8 "nose" entries). Core API will:
+
+1. Group keypoints by `name`
+2. Select the one with highest `confidence`
+3. Return exactly 17 unique keypoints
+
+If a keypoint is not detected, it will have:
+```json
+{"name": "left_shoulder", "x": 0, "y": 0, "confidence": 0}
+```
+
+### 3.3 COCO-17 Keypoint Names
+
+```
+nose, left_eye, right_eye, left_ear, right_ear,
+left_shoulder, right_shoulder, left_elbow, right_elbow,
+left_wrist, right_wrist, left_hip, right_hip,
+left_knee, right_knee, left_ankle, right_ankle
+```
+
+### 3.4 Frame Alignment
+
+- Pose/appearance `frame` numbers should align with face `best_face_frame`
+- If frame not found, API returns 404
+- Frontend should use the face's `best_face_frame` or `start_frame` to query pose/appearance
+
+---
+
+## 4. Frontend Integration (Studio Team)
+
+### 4.1 API Builder
+
+**File**: `src/api/index.ts`
+
+```typescript
+case 'get_pose': {
+  const { fileUuid, frame } = a
+  return {
+    url: `/api/v1/file/${fileUuid}/pose?frame=${frame}`,
+    method: 'GET'
+  }
+}
+
+case 'get_appearance': {
+  const { fileUuid, frame } = a
+  return {
+    url: `/api/v1/file/${fileUuid}/appearance?frame=${frame}`,
+    method: 'GET'
+  }
+}
+```
+
+### 4.2 Store Functions
+
+**File**: `src/store.ts`
+
+```typescript
+export async function getPose(fileUuid: string, frame: number): Promise<PoseResponse> {
+  const response = await callApi('get_pose', { fileUuid, frame })
+  return response
+}
+
+export async function getAppearance(fileUuid: string, frame: number): Promise<AppearanceResponse> {
+  const response = await callApi('get_appearance', { fileUuid, frame })
+  return response
+}
+```
+
+### 4.3 TypeScript Types
+
+**File**: `src/api/types.ts`
+
+Already defined in Studio codebase:
+- `PoseKeypoint`
+- `PoseClass`
+- `PoseResponse`
+- `DominantColor`
+- `AppearanceResponse`
+
+### 4.4 Usage Example
+
+```typescript
+// In Face Detail Modal
+const face = selectedFace
+const frame = face.best_face_frame || face.start_frame
+
+// Fetch pose
+const pose = await getPose(fileUuid, frame)
+
+// Fetch appearance
+const appearance = await getAppearance(fileUuid, frame)
+
+// Render skeleton on canvas
+renderPoseSkeleton(canvas, pose.keypoints)
+
+// Display colors (when implemented)
+displayDominantColors(appearance.dominant_colors)
+```
+
+---
+
+## 5. POC Limitations
+
+| Feature | POC Status | Future |
+|---------|------------|--------|
+| Pose classification | `"unknown"` | ML model for standing/sitting/etc. |
+| Dominant colors | Empty array `[]` | HSV extraction algorithm |
+| Multiple persons | First person only | Match by trace_id |
+| Confidence | `null` | Aggregated confidence score |
+
+---
+
+## 6. Testing
+
+### 6.1 Test Commands
+
+```bash
+# Set variables
+API="http://localhost:3002"
+KEY="muser_demo_key_32chars_abcdef1234567890"
+FILE_UUID="477b24d3030c56b64092c8fea96676df"
+
+# Test pose endpoint
+curl -s "$API/api/v1/file/$FILE_UUID/pose?frame=303" \
+  -H "X-API-Key: $KEY" | jq '.keypoints | length'
+
+# Expected: 17
+
+# Test appearance endpoint
+curl -s "$API/api/v1/file/$FILE_UUID/appearance?frame=303" \
+  -H "X-API-Key: $KEY" | jq '.hsv_histogram | length'
+
+# Expected: 3 (H, S, V channels)
+```
+
+### 6.2 Expected Response Time
+
+- Target: < 100ms per request
+- Cacheable: Yes (pose/appearance data doesn't change)
+
+---
+
+## 7. Timeline
+
+| Phase | Team | Task | Status |
+|-------|------|------|--------|
+| 1 | Core API | Implement endpoints | ✅ Complete |
+| 2 | Core API | Test & verify | ✅ Complete |
+| 3 | Core API | Notify Studio | ✅ Complete |
+| 4 | Studio | Frontend integration | Pending |
+| 5 | Studio | UI testing | Pending |
+
+---
+
+## 8. Contact
+
+**Core API Team**: Will notify Studio when Phase 2 is complete via `M4_workspace/` document.
+
+**Studio Team**: Ready to integrate once Phase 3 notification is received.
+
+---
+
+*Document Version: 1.0*
+*Last Updated: 2026-07-19*