fix: face group name read consistency, sync_file_status fix, cleanup ghost records, identity_agent replaced with face_dedup

- get_face_groups_handler: COALESCE(tp.name, tn.label) for name consistency
- sync_file_status: compare JSON vs pre_chunks (not chunk table)
- face consistency: compare frames.len() not total_faces
- cleanup 2 ghost records with NULL file_name/file_path
- replace identity_agent with face_dedup in pipeline stages
- remove identity_agent_api.rs and all references
- update required_processors to match actual processors
- update AGENTS.md with team responsibilities
- add Studio pipeline changes documentation
This commit is contained in:
Accusys
2026-07-27 02:15:51 +08:00
parent fcdeab82e6
commit 39a2cbc65b
118 changed files with 19386 additions and 2964 deletions
+24 -5
View File
@@ -145,11 +145,30 @@ Get real-time processing progress for a file via Redis pub/sub. Includes per-pro
| 3 | `asrx` | asr | Speaker diarization |
| 4 | `yolo` | — | Object detection |
| 5 | `ocr` | — | Text recognition |
| 6 | `face` | — | Face detection & embedding |
| 7 | `pose` | — | Pose estimation |
| 8 | `visual_chunk` | yolo | Visual scene chunks |
| 9 | `story` | asr, asrx, cut, yolo, face | Scene summaries (template) |
| 10 | `5w1h` | story | 5W1H analysis (Gemma4 LLM) |
| 6 | `face` | — | Face detection & embedding (8Hz sampling) |
| 7 | `face_trace` | face | Face tracking (IoU + embedding, assigns trace_id) |
| 8 | `pose` | face_trace | Pose expansion from face traces, inherits trace_id |
| 9 | `appearance` | pose | Appearance expansion from pose traces, inherits trace_id |
**Key Concepts:**
- **Face** = Identity anchor (who is this person?) — requires high-quality embedding
- **Pose** = Tracking (where is this person?) — extends tracking when face is occluded
- **Appearance** = Tracking (what do they look like?) — extends tracking when pose is occluded
**Trace ID Inheritance:**
```
Face trace (identity anchor)
↓ inherits trace_id
Pose expansion (tracking continuity)
↓ inherits trace_id
Appearance expansion (tracking continuity)
```
**Frame Count Relationship:**
```
face frames ≤ pose frames ≤ appearance frames
```
(Each level expands outward from the previous level's traces)
All processors except `story` and `5w1h` run concurrently when their dependencies are met. Story and 5W1H run sequentially after their prerequisites.
@@ -1102,4 +1102,5 @@ PATCH /api/v1/identity/:identity_uuid
This **replaces** the entire `aliases` array. To add to existing aliases, include all existing entries in the request.
---
*Updated: 2026-07-21 — Fixed bind/unbind TKG update to match both trace_N and face_track_N external_id formats*
*Updated: 2026-06-20 — Added identity files, chunks, faces, status, and JSON endpoints*
@@ -65,4 +65,63 @@ curl -s -X POST "$API/api/v1/agents/identity/match-from-trace" \
```
---
*Updated: 2026-05-19 12:49:24*
### `POST /api/v1/agents/identity/confirm`
**Auth**: Required
**Scope**: file-level
Confirm identity binding for a trace. This marks the trace as confirmed in TKG, updates face_detections, adds to _seeds, and optionally triggers Round 2 propagation.
#### Request Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_uuid` | string | Yes | Video file UUID |
| `trace_id` | integer | Yes | Face trace ID to confirm |
| `identity_id` | integer | Yes | Identity internal ID |
| `identity_uuid` | string | Yes | Identity UUID |
| `name` | string | Yes | Identity name |
| `propagate` | boolean | No | Auto-trigger Round 2 matching (default: true) |
#### Example
```bash
curl -s -X POST "$API/api/v1/agents/identity/confirm" \
-H "Authorization: Bearer $JWT" \
-H "Content-Type: application/json" \
-d '{"file_uuid": "'"$FILE_UUID"'", "trace_id": 10, "identity_id": 42, "identity_uuid": "'"$IDENTITY_UUID"'", "name": "Cary Grant", "propagate": false}'
```
#### Response (200)
```json
{
"success": true,
"file_uuid": "384b0ff44aaaa1f1",
"trace_id": 10,
"identity_uuid": "a9a90105...",
"name": "Cary Grant",
"steps": {
"tkg_updated": true,
"qdrant_updated": 150,
"pg_updated": 150,
"seed_added": true
},
"propagation": {
"matched": 5,
"message": "Propagation completed"
}
}
```
#### Side Effects
1. TKG face_track node status → 'confirmed'
2. Qdrant _faces: identity_uuid added to payload
3. PG face_detections: identity_id set
4. Trace centroid added to _seeds (source='propagation')
5. Round 2 matching triggered (if propagate=true)
---
*Updated: 2026-06-26 00:30:00*
+12 -9
View File
@@ -42,6 +42,7 @@ These steps run after the 10 processors and are **required for pipeline completi
| # | Step | Triggers When | Verification |
|---|------|--------------|-------------|
| 1 | **Rule 1 Sentence Chunking** | ASR + ASRX done | `chunk` table has rows with `chunk_type = 'sentence'` |
| 1.1 | **Rule 1 OCR Chunks** | OCR done | OCR pre_chunks grouped into sentence chunks |
| 2 | **Auto-Vectorize** | Rule 1 done | `chunk.embedding` IS NOT NULL for sentence chunks |
| 3 | **Phase 1 Pack** | Rule 1 done | `release_pack.py --phase 1` executed |
| 4 | **Rule 3 Scene Chunking** | All 10 processors done + Cut + ASR | `chunk` table has rows with `chunk_type = 'cut'` |
@@ -81,15 +82,17 @@ curl "$API/api/v1/stats/ingestion-status/bd80fec9c42afb0307eb28f22c64c76a" | jq
{
"file_uuid": "bd80fec9c42afb0307eb28f22c64c76a",
"steps": [
{ "name": "rule1_sentence", "status": "pending", "detail": "0 sentence chunks" },
{ "name": "auto_vectorize", "status": "pending", "detail": "0 embedded" },
{ "name": "rule3_scene", "status": "pending", "detail": "0 scene chunks" },
{ "name": "face_trace", "status": "pending", "detail": "0 traces" },
{ "name": "trace_chunks", "status": "pending", "detail": "0 trace chunks" },
{ "name": "tkg", "status": "pending", "detail": "0 nodes, 0 edges" },
{ "name": "identity_match", "status": "pending", "detail": "0 identities" },
{ "name": "scene_metadata", "status": "pending", "detail": null },
{ "name": "5w1h", "status": "pending", "detail": "0 scenes with 5W1H" }
{ "name": "rule1_sentence", "status": "done", "detail": "35 sentence chunks" },
{ "name": "rule1_ocr", "status": "done", "detail": "30 OCR frames" },
{ "name": "rule1_ocr_chunks", "status": "done", "detail": "3 OCR-only chunks" },
{ "name": "auto_vectorize", "status": "pending", "detail": "0 embedded" },
{ "name": "rule3_scene", "status": "pending", "detail": "0 scene chunks" },
{ "name": "face_trace", "status": "pending", "detail": "0 traces" },
{ "name": "trace_chunks", "status": "pending", "detail": "0 trace chunks" },
{ "name": "tkg", "status": "pending", "detail": "0 nodes, 0 edges" },
{ "name": "identity_match", "status": "pending", "detail": "0 identities" },
{ "name": "scene_metadata", "status": "pending", "detail": null },
{ "name": "5w1h", "status": "pending", "detail": "0 scenes with 5W1H" }
]
}
```
+187 -39
View File
@@ -6,31 +6,82 @@
TKG is a time-aligned knowledge graph built from multi-processor outputs (face, yolo, ocr, pose, asrx, gaze, lip, appearance). It produces 9 node types and 14 edge types stored in `dev.tkg_nodes` and `dev.tkg_edges`.
**Node naming convention:** All trace types use `_track` suffix. Text uses `_region` (non-temporal).
**See also:** `docs_v1.0/DESIGN/TKG_FORMATION_V1.0.md` for formation phases, flow diagrams, and query examples.
### Node Types
| Node Type | Description | Key Properties |
|-----------|-------------|----------------|
| `face_trace` | A tracked face identity over time | `trace_id`, `face_count`, `avg_confidence` |
| `gaze_trace` | Gaze direction over time | `direction` (frontal/left/right/up/down + diagonals) |
| `lip_trace` | Lip movement synced with speech | `speaker_id`, `lip_area_range` |
| `text_trace` | Spoken text aligned to time | `speaker_id`, `text`, `start_time`, `end_time` |
| `appearance_trace` | Human appearance (clothing) over time | `clothing_color`, `upper_cloth`, `lower_cloth` |
| `skin_tone_trace` | Fitzpatrick skin tone classification | `fitzpatrick_type` (I–VI) |
| `accessory` | Detected accessories | `type` (glasses/hat/etc.), `confidence` |
| `object` | YOLO-detected object | `class`, `confidence`, `frame_count` |
| `speaker` | ASRX speaker segment | `speaker_id`, `segment_count`, `total_duration` |
| Node Type | External ID Format | Description | Key Properties |
|-----------|-------------------|-------------|----------------|
| `face_track` | `trace_{trace_id}` | A tracked face identity over time | `trace_id`, `frame_count`, `status`, `avg_bbox`, `avg_yaw`, `avg_pitch`, `avg_roll`, `start_frame`, `end_frame`, `pose_count` |
| `gaze_track` | `gaze_track_{id}` | Gaze direction over time | `direction` (frontal/left/right/up/down + diagonals) |
| `lip_track` | `lip_track_{id}` | Lip movement synced with speech | `speaker_id`, `lip_area_range` |
| `text_region` | `text_region_{id}` | Spoken text aligned to time | `speaker_id`, `text`, `start_time`, `end_time` |
| `appearance_trace` | `appearance_{trace_id}` | Human appearance (clothing) over time | `clothing_color`, `upper_cloth`, `lower_cloth` |
| `accessory` | `accessory_{id}` | Detected accessories | `type` (glasses/hat/etc.), `confidence` |
| `object` | `object_{class}_{id}` | YOLO-detected object | `class`, `confidence`, `frame_count` |
| `speaker` | `speaker_{speaker_id}` | ASRX speaker segment | `speaker_id`, `segment_count`, `total_duration` |
---
### Identity Agent Integration (face_track nodes)
Identity Agent marks face_track nodes with identity binding status.
#### face_track Status Values
| Status | Description | Properties |
|--------|-------------|------------|
| `pending` | No identity suggestion | Default state |
| `suggested` | Identity Agent suggested | `pending_identity_name`, `pending_identity_uuid`, `suggested_by`, `confidence` |
| `confirmed` | User confirmed binding | `identity_uuid`, `identity_id`, `identity_ref`, `identity_name` |
| `stranger` | Stranger cluster member | `stranger_id`, `stranger_ref` |
#### Suggested By Values
| Value | Description |
|-------|-------------|
| `tmdb` | TMDb seed matched |
| `propagation` | Confirmed trace propagation |
| `manual` | User manual selection |
#### Example face_track Node
```json
{
"node_type": "face_track",
"external_id": "face_track_1",
"label": "Face Track 1",
"properties": {
"trace_id": 1,
"frame_count": 45,
"start_frame": 100,
"end_frame": 300,
"avg_bbox": {"x": 100, "y": 200, "width": 80, "height": 100},
"status": "suggested",
"pending_identity_name": "Tom Hanks",
"pending_identity_uuid": "xxx-xxx",
"suggested_by": "tmdb",
"confidence": 0.91
}
}
```
---
### Edge Types
| Edge Type | Source → Target | Description |
|-----------|-----------------|-------------|
| `co_occurs` | object ↔ object | Two objects appear together in same frame |
| `speaker_face` | speaker ↔ face_trace | Speaker matched to face trace via lip sync |
| `face_face` | face_trace ↔ face_trace | Two face traces interact (mutual gaze) |
| `mutual_gaze` | gaze_trace ↔ gaze_trace | Two people looking at each other |
| `lip_sync` | lip_trace ↔ text_trace | Lip movement aligned with spoken text |
| `has_appearance` | face_trace ↔ appearance_trace | Face has specific appearance |
| `wears` | face_trace ↔ accessory | Face wears an accessory |
| Edge Type | Storage Name | Source → Target | Description |
|-----------|--------------|-----------------|-------------|
| `co_occurs` | `CO_OCCURS_WITH` | object ↔ object | Two objects appear together in same frame |
| `speaker_face` | `SPEAKS_AS` | speaker → face_track | Speaker matched to face track via lip sync |
| `face_face` | `INTERACTS_WITH` | face_track ↔ face_track | Two face tracks interact (mutual gaze) |
| `mutual_gaze` | `MUTUAL_GAZE` | gaze_track ↔ gaze_track | Two people looking at each other |
| `lip_sync` | `LIP_SYNC` | lip_track → text_region | Lip movement aligned with spoken text |
| `has_appearance` | `HAS_APPEARANCE` | face_track → appearance_trace | Face has specific appearance |
| `wears` | `WEARS` | face_track → accessory | Face wears an accessory |
| `hand_object` | `HOLDS` | hand → object | Hand holding object |
---
@@ -55,10 +106,10 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/rebuild" \
"success": true,
"file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
"result": {
"face_trace_nodes": 16,
"gaze_trace_nodes": 16,
"lip_trace_nodes": 12,
"text_trace_nodes": 24,
"face_track_nodes": 16,
"gaze_track_nodes": 16,
"lip_track_nodes": 12,
"text_region_nodes": 24,
"appearance_trace_nodes": 8,
"skin_tone_trace_nodes": 5,
"accessory_nodes": 3,
@@ -96,18 +147,18 @@ Query TKG nodes with pagination and optional type filter.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `node_type` | string | No | all | Filter by node type: `face_trace`, `gaze_trace`, `lip_trace`, `text_trace`, `appearance_trace`, `skin_tone_trace`, `accessory`, `object`, `speaker` |
| `node_type` | string | No | all | Filter by node type: `face_track`, `gaze_track`, `lip_track`, `text_region`, `appearance_trace`, `skin_tone_trace`, `accessory`, `object`, `speaker` |
| `page` | integer | No | 1 | Page number |
| `page_size` | integer | No | 100 | Items per page (max 500) |
#### Example
```bash
# Get all face_trace nodes
# Get all face_track nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"node_type": "face_trace", "page": 1, "page_size": 50}'
-d '{"node_type": "face_track", "page": 1, "page_size": 50}'
# Get all nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
@@ -128,12 +179,12 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
"nodes": [
{
"id": 1,
"node_type": "face_trace",
"external_id": "trace_0",
"label": "Face Trace 0",
"node_type": "face_track",
"external_id": "face_track_0",
"label": "Face Track 0",
"properties": {
"trace_id": 0,
"face_count": 142,
"frame_count": 142,
"avg_confidence": 0.87
}
}
@@ -177,17 +228,17 @@ Query TKG edges with pagination and optional filters.
#### Example
```bash
# Get all co_occurrence edges
# Get all co_occurs edges
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"edge_type": "co_occurs"}'
# Get edges between face_trace and speaker nodes
# Get edges between face_track and speaker nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"source_type": "speaker", "target_type": "face_trace"}'
-d '{"source_type": "speaker", "target_type": "face_track"}'
```
#### Response (200)
@@ -251,12 +302,12 @@ curl -s "$API/api/v1/file/$FILE_UUID/tkg/node/1" \
"success": true,
"node": {
"id": 1,
"node_type": "face_trace",
"external_id": "trace_0",
"label": "Face Trace 0",
"node_type": "face_track",
"external_id": "face_track_0",
"label": "Face Track 0",
"properties": {
"trace_id": 0,
"face_count": 142,
"frame_count": 142,
"avg_confidence": 0.87
}
},
@@ -375,4 +426,101 @@ curl -s "$API/api/v1/file/$FILE_UUID/processor-counts" \
---
*Updated: 2026-06-20 12:00:00*
### Trace Management
Endpoints for managing face traces: list, delete, restore, and merge.
#### `DELETE /api/v1/file/:file_uuid/trace/:trace_id`
**Auth**: Required
Soft-delete a face trace (default) or hard-delete with `{"hard_delete": true}`.
Soft delete marks Qdrant points with `status: "deleted"` and TKG nodes with `status: "deleted"` in properties. Deleted traces are excluded from the traces list.
Hard delete permanently removes Qdrant points and TKG nodes.
**Request Body** (optional):
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `hard_delete` | boolean | `false` | Permanently delete instead of marking |
**Example**:
```bash
# Soft delete
curl -X DELETE "$API/api/v1/file/$FILE_UUID/trace/8" \
-H "X-API-Key: $KEY" -H "Content-Type: application/json" -d '{}'
# Hard delete
curl -X DELETE "$API/api/v1/file/$FILE_UUID/trace/8" \
-H "X-API-Key: $KEY" -H "Content-Type: application/json" \
-d '{"hard_delete": true}'
```
**Response**:
```json
{
"success": true,
"file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
"trace_id": 8,
"hard_delete": false,
"qdrant_marked": true,
"tkg_nodes_marked": 1
}
```
---
#### `POST /api/v1/file/:file_uuid/trace/:trace_id/restore`
**Auth**: Required
Undo a soft-deleted trace. Clears `status: "deleted"` from Qdrant points and TKG node properties.
**Example**:
```bash
curl -X POST "$API/api/v1/file/$FILE_UUID/trace/8/restore" \
-H "X-API-Key: $KEY"
```
**Response**:
```json
{
"success": true,
"file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
"trace_id": 8,
"qdrant_restored": true,
"tkg_nodes_restored": 1
}
```
---
#### `POST /api/v1/file/:file_uuid/trace/:source_trace_id/merge/:target_trace_id`
**Auth**: Required
Merge all face points from source trace into target trace. Updates Qdrant `trace_id` and deletes source TKG node.
**Example**:
```bash
curl -X POST "$API/api/v1/file/$FILE_UUID/trace/16/merge/3" \
-H "X-API-Key: $KEY"
```
**Response**:
```json
{
"success": true,
"file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
"source_trace_id": 16,
"target_trace_id": 3,
"points_moved": 58,
"tkg_nodes_deleted": 1
}
```
---
*Updated: 2026-07-21 01:00:00*
+2 -73
View File
@@ -1,5 +1,5 @@
<!-- module: profile -->
<!-- description: Trace profile, face groups, and file profile management — read/update face trace names, key frames, aliases, and file paths -->
<!-- description: Trace profile and file profile management — read/update face trace names, key frames, aliases, and file paths -->
<!-- depends: 01_auth, 07_identity, 15_tkg -->
## Profile Management
@@ -158,77 +158,6 @@ curl -s -X PUT "$API/api/v1/trace-profile/group" \
---
### `GET /api/v1/file/:file_uuid/face-groups`
**Auth**: Required
**Scope**: file-level
Get face groups for a file. Groups face traces by their label (name) and returns both named groups and unassigned traces.
#### Path Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_uuid` | string | Yes | File UUID |
#### Example
```bash
curl -s "$API/api/v1/file/$FILE_UUID/face-groups" \
-H "X-API-Key: $KEY"
```
#### Response (200)
```json
{
"success": true,
"file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
"face_groups": [
{
"group_id": 1,
"name": "Cary Grant",
"trace_ids": [9, 6, 8],
"trace_count": 3,
"representative_trace": 9,
"editable": true
},
{
"group_id": 2,
"name": "Audrey Hepburn",
"trace_ids": [1, 11, 4, 10],
"trace_count": 4,
"representative_trace": 1,
"editable": true
}
],
"total_groups": 2,
"unassigned_traces": [5, 7]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Always `true` on success |
| `file_uuid` | string | File UUID |
| `face_groups` | array | List of named face groups |
| `face_groups[].group_id` | integer | Sequential group number (starts at 1) |
| `face_groups[].name` | string | Group name (from TKG `label`) |
| `face_groups[].trace_ids` | integer[] | Trace IDs in this group |
| `face_groups[].trace_count` | integer | Number of traces in group |
| `face_groups[].representative_trace` | integer | First trace ID in group |
| `face_groups[].editable` | boolean | Always `true` (groups can be renamed) |
| `total_groups` | integer | Total named groups |
| `unassigned_traces` | integer[] | Traces with default names (e.g., "Face Trace 9") |
#### Notes
- Unassigned traces have labels starting with "Face Trace " or "Trace "
- Groups are sorted by first trace_id in each group
- `group_id` is dynamically generated and may change between requests
---
### `GET /api/v1/file-profile`
**Auth**: Required
@@ -335,5 +264,5 @@ curl -s -X PUT "$API/api/v1/file-profile" \
| `aliases` | `properties->'aliases'` | Multi-language name aliases |
---
*Updated: 2026-07-19 — Added face-groups endpoint for Studio Proxy integration*
*Updated: 2026-07-21 — Fixed external_id matching (trace_N + face_track_N formats), fixed parameter ordering in UPDATE query*
*Updated: 2026-07-18 — New profile module: trace-profile (GET, PUT, PUT group) and file-profile (GET, PUT)*