warren/momentry_core
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

fix: face group name read consistency, sync_file_status fix, cleanup ghost records, identity_agent replaced with face_dedup

Browse Source 
- get_face_groups_handler: COALESCE(tp.name, tn.label) for name consistency
- sync_file_status: compare JSON vs pre_chunks (not chunk table)
- face consistency: compare frames.len() not total_faces
- cleanup 2 ghost records with NULL file_name/file_path
- replace identity_agent with face_dedup in pipeline stages
- remove identity_agent_api.rs and all references
- update required_processors to match actual processors
- update AGENTS.md with team responsibilities
- add Studio pipeline changes documentation
This commit is contained in:
Accusys
2026-07-27 02:15:51 +08:00
parent fcdeab82e6
commit 39a2cbc65b
118 changed files with 19386 additions and 2964 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_v1.0/GUIDES/QC_FaceCluster_Support.md
+270
View File
@@ -0,0 +1,270 @@
---
title: QC FaceCluster Support Guide
version: 2.0
date: 2026-07-24
author: OpenCode
status: final
---
# QC Modal: face_cluster Support
> Companion guide for Studio team: frontend changes in `/Users/accusys/momentry_studio/src/views/LibraryView.vue` for `face_cluster` support, and backend multi-stage trace dedup upgrade.
| Scope | `/Users/accusys/momentry_studio/src/views/LibraryView.vue` |
|-------|-------------------------------------------------------------|
| Backend changes | [`/Users/accusys/momentry_core/docs_v1.0/DESIGN/AlwaysProduce_Processing_Contract.md`](../DESIGN/AlwaysProduce_Processing_Contract.md) |
| Status | ✅ Frontend done (commit `2af8ffa`). Backend upgraded to multi-stage trace dedup. |
| Version | 2.0 |
**Glossary:**
| Term | Definition |
|------|------------|
| **Always-Produce rule** | Every processor MUST write its output JSON after scanning the last frame, even for zero results. See [`/Users/accusys/momentry_core/docs_v1.0/DESIGN/AlwaysProduce_Processing_Contract.md`](../DESIGN/AlwaysProduce_Processing_Contract.md#2-always-produce-rule). |
| **Stage 2** | Second-level processors that depend on Stage 1 (`asr`, `ocr`, `face`): `asrx`, `face_cluster`, `pose`, `appearance`. |
| **QC Modal** | Pipeline Quality Control modal launched via the 🔍 button in the file context menu (advanced mode). |
| **trace_id** | Per-video integer identifier linking face detections across frames (from face tracker). Each `trace_id` represents the same person in a continuous shot. |
| **Multi-stage dedup** | Two-stage clustering: (1) strict AgglomerativeClustering on trace-level mean embeddings, (2) cross-cluster merge via trace-pair voting with temporal overlap guard. |
---
## Background
The backend at `/Users/accusys/momentry_core/scripts/fast_face_clustering_processor.py` was upgraded from single-pass face-level clustering to **multi-stage trace-based deduplication**.
### Problem: Single-pass clustering limitations
The old algorithm ran AgglomerativeClustering (cosine distance threshold 0.4) on up to 25k+ individual face embeddings, using random sampling for large datasets. This caused:
- **Fragmentation**: same person appearing in different shots/scenes got split across multiple clusters because their face embeddings exceeded the fixed threshold
- **Sampling bias**: only 5000 faces sampled for large files, minority clusters missed
- **No temporal info**: no use of `trace_id` or frame-range overlap checks
### Analysis: 12 files, 180,791 face embeddings
Analysis of all production data in Qdrant `_faces` collection showed:
| Metric | Value |
|--------|-------|
| Files analyzed | 12 |
| Total face embeddings | 180,791 |
| Worst fragmentation | `c36f35685177` — 62k faces, 5,616 traces, 46% single-face |
| Cross-trace pairs >0.7 similarity | 529,305 in worst file |
| Temporal overlap among high-sim pairs | 99.9% **non-overlapping** (safe to merge) |
| Temporal overlap for talking head | 100% **overlapping** (temporal guard prevents false merge) |
### Solution: Multi-stage trace dedup
1. **Trace aggregation**: group all Qdrant face embeddings by `trace_id`, compute confidence-weighted mean embedding per trace + frame range
2. **Stage 1 (strict)**: AgglomerativeClustering on trace-level mean vectors (cosine distance threshold 0.35)
3. **Stage 2 (merge)**: trace-pair voting across cluster boundaries — if >30% of cross-cluster trace pairs have similarity >0.70 AND overall cluster frame ranges don't overlap → merge
### Sourced from Qdrant `_faces` Collection
Collection: `_faces` (512D, Cosine distance)
Payload: `{file_uuid, frame, trace_id, bbox, confidence, identity_id, identity_uuid}`
---
## face_cluster.json Output Format (Unchanged)
From `/Users/accusys/momentry_core/scripts/fast_face_clustering_processor.py`:
```json
{
"status": "has_faces",
"file_uuid": "9781de6d...",
"clusters": [
{
"cluster_id": "Person_0",
"face_count": 12,
"representative_face": {
"face_id": "face_10_3",
"confidence": 0.95,
"frame": 123,
"bbox": { "x": 100, "y": 200, "width": 50, "height": 60 }
}
}
],
"frames": [
{
"frame": 123,
"timestamp": 5.13,
"faces": [
{ "face_id": "face_10_3", "cluster_id": "Person_0", "confidence": 0.95 }
]
}
]
}
```
On no-faces / no-data, status is `"no_faces"`, `"no_face_json"`, or `"no_embeddings"`, with `"clusters": []` and `"frames": []`.
---
## Backend: Complete
| File | Change | Status |
|------|--------|--------|
| `/Users/accusys/momentry_core/scripts/fast_face_clustering_processor.py` | **Multi-stage trace dedup**: trace aggregation + Stage 1 strict clustering (0.35) + Stage 2 trace-pair voting merge (0.70 sim, 0.30 ratio) + temporal overlap guard | ✅ Done |
| `/Users/accusys/momentry_core/scripts/fast_face_clustering_processor.py` | Always-Produce: 3 early returns write empty output + RedisPublisher progress | ✅ Done |
| `/Users/accusys/momentry_core/src/worker/job_worker.rs:186` | Worker heartbeat EXPIRE 15s after HMSET | ✅ Done |
| `/Users/accusys/momentry_core/src/api/health.rs:631-655` | `check_worker_alive()` — Redis TTL check replacing `ps aux` | ✅ Done |
| `/Users/accusys/momentry_core/src/api/processing.rs:842` | `GET /api/v1/file/{uuid}/processor-counts` — auto-includes `FaceCluster` via `ProcessorType::all()` | ✅ Already correct |
### Test Results (7 files, 155,095 face embeddings total)
| File | Type | Faces | Traces | Stage 1 | Final | Merged |
|------|------|-------|--------|---------|-------|--------|
| `c36f35685177` | Crowd | 62,298 | 5,616 | 1,113 | 826 | **287** |
| `d8acb03870f0` | Crowd | 693 | 107 | 45 | 43 | 2 |
| `84d838f260e1` | Crowd | 597 | 89 | 34 | 33 | 1 |
| `c0a9dc37cd84` | Crowd | 1,137 | 78 | 26 | 24 | 2 |
| `31a6b8212760` | Multi | 744 | 32 | 10 | 9 | 1 |
| `5e5f3de82208` | Multi | 532 | 22 | 4 | 4 | 0 |
| `bfba056f5021` | Talking head | 89,791 | 16 | 4 | 4 | **0 (correct)** |
**Key verification**: talking head file had 100% temporal overlap among all high-sim trace pairs — Stage 2 correctly merged 0 clusters (temporal guard prevented false positive).
---
## Frontend: Studio Team Changes (Already Done)
**Commit**: `2af8ffa` → Gitea (pushed by Studio team)
### Change 1: Job Output File List (P0)
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L1034
Add `'face_cluster'` to the `processors` array in `refreshSystemStatus()`:
```typescript
const processors = ['cut', 'asr', 'asrx', 'face', 'ocr', 'pose', 'appearance', 'face_cluster']
```
Expected: QC Modal → Jobs → each job's output list includes `face_cluster.json` with cluster count. Empty results show `face_cluster.json (0筆)`.
### Change 2: QC Result Query List (P0)
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L1096
Add `'face_cluster'` to the `processors` array in `runProcessorQC()`:
```typescript
const processors = ['cut', 'asr', 'asrx', 'face', 'ocr', 'pose', 'appearance', 'face_cluster']
```
Expected: Pipeline visualization and node status include `FACE_CLUSTER`.
### Change 3: `getJsonCount()` — Add `clusters` Case (P0)
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L1002
Insert `data.clusters` check before `return 0`:
```typescript
if (data.cuts) return data.cuts.length
if (data.clusters) return data.clusters.length
return 0
```
`face_cluster.json` uses `clusters` array — without this branch, count always shows 0.
### Change 4: Pipeline Stage 2 — Add FACE_CLUSTER Node (P0)
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L460
Add `FACE_CLUSTER` to the Stage 2 filter:
```html
v-for="r in qcResults.filter(p => ['ASRX','FACE_CLUSTER','APPEARANCE'].includes(p.processor))"
```
Expected pipeline:
```
[ASRX] → [FACE_CLUSTER] → [APPEARANCE] → 📄 JSON Outputs
```
### Change 5 (Optional, P1): Context Menu
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L179
Insert `face_cluster` block between ASRX and Face:
```html
<div class="ms-proc-line" :class="procStatusClass('face_cluster')">
<label class="ms-fm-check-label"><input type="checkbox" v-model="procFaceCluster"> Face Cluster</label>
<span class="ms-proc-status">{{ procStatusIcon('face_cluster') }}</span>
<span class="ms-proc-count" @click.stop="viewProcessorJson('face_cluster')">{{ procCountLabel('face_cluster', 'frame') }}</span>
<button class="ms-proc-redo" @click.stop="redoProcessor('face_cluster')" title="Re-run Face Cluster">🔄</button>
</div>
```
Add reactive ref:
```typescript
const procFaceCluster = ref(true)
```
### Change 6 (Optional, P1): `selectedProcessors()`
**File**: `/Users/accusys/momentry_studio/src/views/LibraryView.vue`
**Line**: ~L930
```typescript
if (procAsrx.value) procs.push('asrx')
if (procFaceCluster.value) procs.push('face_cluster')
if (procFace.value) procs.push('face')
```
---
## What to Expect in QC
With the multi-stage algorithm:
| Before (single-pass) | After (multi-stage trace dedup) |
|----------------------|--------------------------------|
| Many small clusters for the same person across different shots | Fewer, more accurate clusters — cross-shot fragments merged |
| Single-face traces often assigned to wrong cluster (noise) | Single-face traces remain as small clusters but don't pollute larger ones |
| Talking head: reasonable (limited impact) | Unchanged (temporal guard prevents false merge) |
| Crowd/multi-person: severe fragmentation | 26% fewer clusters in worst case (287 clusters merged in test) |
**Example**: `c36f35685177` (62k faces, crowd scene)
- Old: ~1,113+ clusters (single-pass, sampling-based)
- New: 826 clusters (trace-level, two-stage, temporal verified)
---
## Verification
1. Open QC Modal on a processed file (with or without faces)
2. Verify Jobs output list includes `face_cluster.json`
3. Verify Pipeline Stage 2 shows `FACE_CLUSTER` node with ✓ status and cluster count
4. For a no-faces file (e.g., `9781de6d...`), confirm `face_cluster.json` shows 0 clusters
---
## See Also
- [`/Users/accusys/momentry_core/docs_v1.0/DESIGN/AlwaysProduce_Processing_Contract.md`](../DESIGN/AlwaysProduce_Processing_Contract.md) — Frame-Scan model, Always-Produce rule, Redis progress spec
- [`/Users/accusys/momentry_core/docs_v1.0/DESIGN/Worker_Health_Check_Mechanism.md`](../DESIGN/Worker_Health_Check_Mechanism.md) — Worker heartbeat TTL mechanism
- [`/Users/accusys/momentry_core/docs_v1.0/DESIGN/FILE_LIFECYCLE_V1.0.md`](../DESIGN/FILE_LIFECYCLE_V1.0.md) — Processor stage definitions
- `/Users/accusys/momentry_core/src/core/db/postgres_db.rs:495` — `ProcessorType::all()` includes `FaceCluster`
- `/Users/accusys/momentry_core/src/api/processing.rs:842` — `GET /api/v1/file/{uuid}/processor-counts`
- `/Users/accusys/momentry_core/scripts/fast_face_clustering_processor.py` — Multi-stage trace dedup implementation
---
## Version History
| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 1.0 | 2026-07-24 | OpenCode | Initial — frontend changes for Always-Produce face_cluster support |
| 2.0 | 2026-07-24 | OpenCode | Backend upgraded to multi-stage trace dedup. Frontend changes completed. |