feat: add source prefix to search results

- Add [OCR], [ASRX], [ASRX+OCR] prefix to text_content - Add content field to SemanticSearchResult struct - Update SQL queries to include content field - Helps users distinguish the source of search results
2026-07-19 14:06:06 +08:00
parent 5e83ee7dac
commit 87aa7e0c40
10 changed files with 3947 additions and 6 deletions
@@ -0,0 +1,538 @@
+---
+title: Identity Agent V2.0 Redesign
+version: 2.0
+date: 2026-07-15
+author: OpenCode
+status: Draft
+---
+
+# Identity Agent V2.0 重新設計
+
+## 1. 概述
+
+### 目的
+將 face trace 與 seed photos 透過 Qdrant 向量比對，建立 pending people 綁定建議，由用戶確認後升級為 known people。
+
+### 核心原則
+
+| 原則 | 說明 |
+|------|------|
+| **只綁定，不確認** | Identity Agent 只建立 suggested 綁定，不升級為 confirmed |
+| **所有綁定都需確認** | 即使 matching 的是已知人物（known people），也要建立 pending 綁定 |
+| **用戶決定** | 每個新綁定都需要用戶在 UI 上點擊確認 |
+| **多重綁定** | 一個 trace 可同時匹配多個 seeds，都記錄為建議 |
+
+### 流程圖
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Identity Agent V2.0                        │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│  Data Stores:                                               │
+│    ├─ Qdrant _faces (512D, trace-level faces)               │
+│    ├─ Qdrant _seeds (512D, seed photos per file_uuid)       │
+│    ├─ PG identities (metadata: name, tmdb_id, status)       │
+│    └─ PG tkg_nodes (face_track, status + suggestions)       │
+│                                                             │
+│  Flow:                                                      │
+│    1. TMDb/Upload/Name → Seeds (_seeds collection)          │
+│    2. Identity Agent → Match _seeds vs _faces               │
+│    3. Write suggestions → TKG + Qdrant (suggested)          │
+│    4. User Confirm → status = confirmed                     │
+│    5. User Reject → status = stranger                       │
+│                                                             │
+│  Rules:                                                     │
+│    - All bindings start as "suggested"                      │
+│    - Even known people get new pending bindings             │
+│    - Multiple suggestions per trace allowed                 │
+│    - User confirms each binding individually                │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 2. 種子管理
+
+### 種子來源
+
+| 來源 | 觸發方式 | file_uuid 綁定 |
+|------|----------|---------------|
+| TMDb probe | 檔案註冊時自動探測 | 探測檔案的 uuid |
+| 用戶上傳照片 | UI 上傳 face photo | 當前查看檔案的 uuid |
+| Pending Face 命名 | 選擇 trace 並命名 | 該 face 所在檔案的 uuid |
+
+### Qdrant _seeds 結構
+
+```json
+{
+  "vector": [512 floats],
+  "payload": {
+    "identity_id": 3,
+    "identity_uuid": "69e50666-0c2d-4215-968b-d1fdd820fb7d",
+    "name": "Walter Matthau",
+    "source": "tmdb",
+    "file_uuid": "4bb70f2b0d6d9c1a2900298666189a73",
+    "tmdb_id": 3490,
+    "status": "confirmed",
+    "angles": [
+      {"frame": 100, "embedding": [...]},
+      {"frame": 150, "embedding": [...]}
+    ]
+  }
+}
+```
+
+### 種子產生流程
+
+```
+TMDb Probe
+  → 下載演員照片
+  → 提取 face embedding
+  → Push to _seeds (file_uuid 綁定)
+
+用戶上傳
+  → 上傳照片
+  → 檢測 face + 提取 embedding
+  → Push to _seeds (file_uuid 綁定)
+
+Pending Face 命名
+  → 選擇 trace
+  → 輸入名稱
+  → 從 trace 提取 representative embedding
+  → Push to _seeds (file_uuid 綁定)
+```
+
+---
+
+## 3. 匹配流程
+
+### 匹配閾值
+
+| Round | Threshold | Seed Source |
+|-------|-----------|-------------|
+| Round 1 | 0.55 | TMDb seeds |
+| Round 2 | 0.55 | Confirmed traces (propagation) |
+| Round 3+ | 0.50 | More confirmed traces |
+| Stranger | 0.40 | Unmatched traces (clustering) |
+
+### 匹配邏輯
+
+```python
+def match_faces_round_1(file_uuid: str) -> dict:
+    """
+    Returns: {trace_id: [suggestions]}
+    每個 trace 可有多個 suggestions（多重綁定）
+    """
+    traces = get_trace_representatives(file_uuid)
+    seeds = get_seeds(file_uuid=file_uuid)  # 只查該檔案的 seeds
+    
+    suggestions = {}
+    
+    for trace_id, reps in traces.items():
+        trace_suggestions = []
+        
+        for seed in seeds:
+            score = multi_angle_match(seed["embedding"], reps)
+            if score >= TH_ROUND_1:
+                trace_suggestions.append({
+                    "identity_id": seed["identity_id"],
+                    "identity_uuid": seed["identity_uuid"],
+                    "name": seed["name"],
+                    "score": round(score, 4),
+                    "source": seed["source"],
+                    "is_known": seed.get("status") == "confirmed"
+                })
+        
+        if trace_suggestions:
+            trace_suggestions.sort(key=lambda x: x["score"], reverse=True)
+            suggestions[trace_id] = trace_suggestions
+    
+    return suggestions
+```
+
+---
+
+## 4. 多重綁定
+
+### 設計原則
+
+一個 face trace 可以同時匹配多個 seeds（分數都很高），所有匹配都記錄為建議，由用戶決定選擇哪一個。
+
+### 範例
+
+```
+Trace #432 (7 frames, confidence: 0.83)
+├─ Walter Matthau (58.6%) ← is_known: true
+├─ George Kennedy (57.2%) ← is_known: true
+└─ [Not yet assigned]
+```
+
+---
+
+## 5. 資料儲存
+
+### Qdrant _faces payload（多重綁定）
+
+```json
+{
+  "file_uuid": "9f880a88e9f027255714906d84865fd4",
+  "trace_id": 432,
+  "frame": 13494,
+  "confidence": 0.829,
+  "bbox": {"x": 272, "y": 38, "width": 87, "height": 87},
+  
+  "identity_id": null,
+  "identity_uuid": null,
+  
+  "suggestions": [
+    {"identity_id": 3, "identity_uuid": "69e50666...", "name": "Walter Matthau", "score": 0.586, "source": "tmdb", "is_known": true},
+    {"identity_id": 5, "identity_uuid": "102e8ed0...", "name": "George Kennedy", "score": 0.572, "source": "tmdb", "is_known": true}
+  ],
+  "suggested_by": "tmdb"
+}
+```
+
+### TKG face_track properties（多重綁定）
+
+```json
+{
+  "trace_id": 432,
+  "frame_count": 7,
+  "start_frame": 100,
+  "end_frame": 300,
+  
+  "status": "suggested",
+  "suggestions": [
+    {"identity_id": 3, "identity_uuid": "69e50666...", "name": "Walter Matthau", "score": 0.586, "source": "tmdb", "is_known": true},
+    {"identity_id": 5, "identity_uuid": "102e8ed0...", "name": "George Kennedy", "score": 0.572, "source": "tmdb", "is_known": true}
+  ],
+  "suggested_by": "tmdb"
+}
+```
+
+### 雙寫入邏輯
+
+```python
+def write_suggestions(file_uuid: str, suggestions: Dict):
+    """同時寫入 TKG 和 Qdrant"""
+    # 1. 寫入 TKG
+    tkg_updated = batch_mark_suggestions(file_uuid, suggestions)
+    
+    # 2. 寫入 Qdrant _faces
+    qdrant_updated = update_faces_suggestions(file_uuid, suggestions)
+    
+    return {"tkg": tkg_updated, "qdrant": qdrant_updated}
+```
+
+---
+
+## 6. Pending People
+
+### 查詢邏輯
+
+**API:** `GET /api/v1/file/:file_uuid/pending-persons`
+
+**查詢：** PG tkg_nodes WHERE status = 'suggested'
+
+```sql
+SELECT 
+    jsonb_array_elements(properties->'suggestions') as suggestion,
+    COUNT(DISTINCT (properties->>'trace_id')) as trace_count,
+    MIN(created_at) as created_at
+FROM tkg_nodes
+WHERE file_uuid = $1 
+  AND node_type = 'face_track' 
+  AND properties->>'status' = 'suggested'
+  AND jsonb_array_length(properties->'suggestions') > 0
+GROUP BY suggestion
+ORDER BY trace_count DESC
+```
+
+### 回傳格式
+
+```json
+{
+  "success": true,
+  "message": "Found 5 pending persons",
+  "data": [
+    {
+      "identity_uuid": "69e50666-0c2d-...",
+      "identity_id": 3,
+      "name": "Walter Matthau",
+      "trace_count": 4,
+      "is_known": true,
+      "created_at": "2026-07-15 12:00:00"
+    }
+  ]
+}
+```
+
+---
+
+## 7. Known Person 複製邏輯
+
+### 規則
+
+當 Identity Agent 匹配到已知人物（status = 'confirmed'）時：
+1. **不自動確認** — 仍建立 suggested 綁定
+2. **標記 is_known** — 讓 UI 顯示為已知人物
+3. **用戶仍需確認** — 每個新綁定都需要用戶操作
+
+### 匹配結果結構
+
+```json
+{
+  "trace_id": 432,
+  "suggestions": [
+    {
+      "identity_id": 3,
+      "name": "Walter Matthau",
+      "score": 0.586,
+      "is_known": true  // ← 已知人物標記
+    }
+  ]
+}
+```
+
+### UI 顯示
+
+```
+Pending People (Charade)
+
+Trace #432
+├─ Walter Matthau (58.6%) ✓ Known
+│   [Confirm] [Reject]
+├─ George Kennedy (57.2%) ✓ Known
+│   [Confirm] [Reject]
+```
+
+---
+
+## 8. Speaker Binding
+
+### 改查 TKG suggested
+
+**舊邏輯：** 查 Qdrant _faces WHERE identity_id EXISTS（永遠 0 筆）
+
+**新邏輯：** 查 PG tkg_nodes WHERE status = 'suggested'
+
+### 具體實現
+
+```rust
+pub async fn bind_speakers(pool: &PgPool, file_uuid: &str) -> Result<usize> {
+    // 1. 查 TKG suggested 節點
+    let rows: Vec<(i32, i32, i64, i64, f64)> = sqlx::query_as(
+        "SELECT (properties->>'trace_id')::int,
+                (properties->'suggestions'->0->>'identity_id')::int,
+                (properties->>'start_frame')::bigint,
+                (properties->>'end_frame')::bigint,
+                COALESCE((properties->>'confidence')::float, 0.0)
+         FROM tkg_nodes
+         WHERE file_uuid = $1 AND node_type = 'face_track' 
+           AND properties->>'status' = 'suggested'
+           AND jsonb_array_length(properties->'suggestions') > 0"
+    ).bind(file_uuid).fetch_all(pool).await?;
+    
+    // 2. 讀取 ASRX segments
+    let asrx_data = load_asrx(file_uuid).await?;
+    let speakers = extract_speakers(&asrx_data);
+    
+    // 3. 計算重疊 + 寫入 identity_bindings
+    let mut bindings = 0;
+    for (trace_id, identity_id, start_frame, end_frame, confidence) in rows {
+        let best_speaker = compute_overlap(start_frame, end_frame, &speakers, fps);
+        if best_speaker.overlap_ratio > 0.3 {
+            insert_identity_binding(identity_id, &best_speaker.speaker_id, ...);
+            bindings += 1;
+        }
+    }
+    
+    Ok(bindings)
+}
+```
+
+---
+
+## 9. API 設計
+
+### 完整 API 清單
+
+| Method | Path | 說明 |
+|--------|------|------|
+| POST | `/api/v1/agents/identity/run` | 執行 Identity Agent（全部 seeds） |
+| POST | `/api/v1/agents/identity/run-for-seed` | 執行單一 seed 匹配 |
+| POST | `/api/v1/agents/identity/cluster` | 執行 trace clustering（無 seed） |
+| POST | `/api/v1/file/:file_uuid/cluster-agent` | 執行 trace clustering（檔案專用） |
+| POST | `/api/v1/agents/identity/generate-seeds` | 產生種子 embeddings |
+| GET | `/api/v1/file/:file_uuid/pending-persons` | 列出 pending people |
+| POST | `/api/v1/file/:file_uuid/pending-person` | 建立 pending person |
+| POST | `/api/v1/identity/:uuid/bind` | 綁定 identity |
+| POST | `/api/v1/identity/:uuid/unbind` | 解綁 identity |
+| POST | `/api/v1/identity/:uuid/confirm` | 確認 pending → confirmed |
+| POST | `/api/v1/identity/:uuid/reject` | 拒絕 pending → stranger |
+
+### run-for-seed API
+
+**Request:**
+```json
+{
+  "file_uuid": "9f880a88e9f027255714906d84865fd4",
+  "identity_uuid": "c3545906-c82d-4b66-aa1d-150bc02decce"
+}
+```
+
+**Response:**
+```json
+{
+  "success": true,
+  "message": "Found 19 new matches for seed",
+  "matches": 19
+}
+```
+
+---
+
+## 10. 前端 UI
+
+### Pending People 顯示
+
+```
+┌─────────────────────────────────────────────────────────┐
+│ Pending People (Charade)                                │
+├─────────────────────────────────────────────────────────┤
+│                                                         │
+│ Audrey Hepburn (6 traces) ✓ Known Person               │
+│ [Confirm All] [Review] [Skip]                          │
+│                                                         │
+│ George Kennedy (4 traces) ✓ Known Person               │
+│ [Confirm All] [Review] [Skip]                          │
+│                                                         │
+│ Walter Matthau (4 traces) ✓ Known Person               │
+│ [Confirm All] [Review] [Skip]                          │
+│                                                         │
+│ Bernard Musson (1 trace)                               │
+│ [Confirm] [Skip]                                       │
+│                                                         │
+│ James Coburn (1 trace)                                 │
+│ [Confirm] [Skip]                                       │
+│                                                         │
+└─────────────────────────────────────────────────────────┘
+```
+
+### Trace 層級多重綁定
+
+```
+┌─────────────────────────────────────────────────────────┐
+│ Trace #432 (7 frames)                                   │
+├─────────────────────────────────────────────────────────┤
+│                                                         │
+│  Suggestions:                                           │
+│  ○ Walter Matthau (58.6%) ✓ Known                       │
+│  ○ George Kennedy (57.2%) ✓ Known                       │
+│                                                         │
+│  [Select Walter Matthau]  [Select George Kennedy]       │
+│  [Skip - Not Recognized]                                │
+│                                                         │
+└─────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 11. Cluster Agent（無 Seed 分群）
+
+### 目的
+不需要 seed photos，直接對現有 face traces 進行相似度分群，找出相似臉並分組。
+
+### 與 Identity Agent 的區別
+
+| | Identity Agent | Cluster Agent |
+|---|---|---|
+| Seed | 需要 seed person | 不需要 seed |
+| 目標 | TMDB 匹配已知演員 | 用現有 face traces 找出相似臉分群 |
+| 輸出 | identity_match_round1.json | cluster_result.json |
+| 算法 | Cosine similarity vs seeds | Greedy clustering with centroids |
+
+### API
+
+**Request:**
+```json
+POST /api/v1/file/:file_uuid/cluster-agent
+{}
+```
+
+**Response:**
+```json
+{
+  "success": true,
+  "message": "Found 30 clusters from 412 traces",
+  "clusters": 30,
+  "total_traces": 412,
+  "output_path": "/Users/accusys/momentry/output/{uuid}/{uuid}.cluster_result.json",
+  "cluster_details": [
+    {"cluster_id": 1, "trace_count": 49, "trace_ids": [...], "representative_trace": 2136},
+    {"cluster_id": 2, "trace_count": 36, "trace_ids": [...], "representative_trace": 2}
+  ]
+}
+```
+
+### 輸出檔案
+
+`{output}/{uuid}/{uuid}.cluster_result.json`
+
+```json
+{
+  "file_uuid": "9f880a88e9f027255714906d84865fd4",
+  "threshold": 0.40,
+  "total_traces": 412,
+  "clusters": [
+    {"cluster_id": 1, "trace_count": 49, "trace_ids": [2136, ...], "representative_trace": 2136}
+  ]
+}
+```
+
+---
+
+## 12. 修改計畫
+
+### Python Scripts
+
+| 檔案 | 改動 |
+|------|------|
+| `identity_matcher.py` | 輸出 suggestions 陣列，支援多重綁定 |
+| `qdrant_faces.py` | 新增 `update_faces_suggestions()` |
+| `tkg_helper.py` | 修改 `batch_mark_suggestions()` 寫入 suggestions 陣列 |
+
+### Rust 後端
+
+| 檔案 | 改動 |
+|------|------|
+| `identity_agent_api.rs` | `bind_speakers()` 改查 TKG suggested |
+| `identity_binding.rs` | `list_pending_persons()` 已改為查 TKG ✓ |
+
+### 前端
+
+| 檔案 | 改動 |
+|------|------|
+| `PeopleView.vue` | 顯示多重綁定、is_known 標記、觸發按鈕 |
+| `store.ts` | `ensureFaceCandidates` 傳 fileUuid ✓ |
+| `api/index.ts` | 新增 `run_identity_for_seed` API 映射 |
+
+### Pending People Card UI
+
+每個 pending person 卡片新增觸發按鈕（右上角 refresh icon）：
+- 點擊後觸發單一 seed 匹配
+- 顯示匹配結果數量
+- 自動重新載入 pending people 列表
+
+---
+
+## 12. 版本歷史
+
+| 版本 | 日期 | 說明 |
+|------|------|------|
+| V1.0 | 2026-05-07 | 初始設計 |
+| V2.0 | 2026-07-15 | 重新設計：多重綁定、只綁定不確認、TKG 雙寫入 |
+| V2.1 | 2026-07-16 | 新增 run-for-seed API、前端觸發按鈕 |
+| V2.2 | 2026-07-16 | 新增 Cluster Agent（無 seed 分群）、cluster-agent API |