- trace_vlm_caption.py: use MOMENTRY_LLM_VISION_URL and MOMENTRY_LLM_VISION_MODEL
- scene_vlm_caption.py: use MOMENTRY_LLM_VISION_URL and MOMENTRY_LLM_VISION_MODEL
- Changed from Ollama /api/generate to OpenAI-compatible /v1/chat/completions format
- Added embedding server environment variables
- get_face_groups_handler: COALESCE(tp.name, tn.label) for name consistency
- sync_file_status: compare JSON vs pre_chunks (not chunk table)
- face consistency: compare frames.len() not total_faces
- cleanup 2 ghost records with NULL file_name/file_path
- replace identity_agent with face_dedup in pipeline stages
- remove identity_agent_api.rs and all references
- update required_processors to match actual processors
- update AGENTS.md with team responsibilities
- add Studio pipeline changes documentation
- Extract full frame at best confidence frame using ffmpeg
- Crop face region using average bbox coordinates
- Save to .faces/{file_uuid}/{trace_id}/key_frame.jpg and key_face.jpg
- TKG node properties store full file paths for UI display
- Supports future extension for cut representative frames
- FacePoint now includes confidence and point_id from Qdrant
- TKG build tracks best face (highest confidence) per trace
- key_frame = frame number of best face, key_face = Qdrant point ID
- Every face trace now has representative frame and face for UI display
- tkg.rs: use trace_N as primary external_id, delete duplicate face_track_N nodes
- trace_agent_api.rs: add POST /trace/:id/restore to undo soft delete
- ensure 1:1 mapping: each trace has exactly one TKG node with full profile
- Add Qdrant count verification before TKG build
- Wait up to 30 seconds for Qdrant to have all face embeddings
- Prevents race condition where TKG reads incomplete data
- Add count_points method to QdrantDb
- Check face.json status before waiting for face_traced.json
- If status='no_faces', spawn TKG immediately (no face_track nodes)
- If status='has_faces', wait for face_traced.json
- Prevents infinite wait when video has no faces
- Add check for face_traced.json existence before spawning TKG build
- Prevents race condition where TKG reads Qdrant before trace_id is written
- Logs clearly when waiting and when proceeding
- Add POST /api/v1/file/:file_uuid/cluster-agent endpoint for on-demand face clustering
- Fix OCR chunks being labeled as ASRX: use ChunkType.as_str() instead of {:?}
- Rule 1 now deletes old chunks before re-inserting to avoid stale data
- Add fallback face_traced.json when store_traced_faces.py fails
- ingestion_complete now handles status='error' to unblock jobs
- VLM describe tool now uses trace_id with pre-extracted face crops
- Update system prompt to prioritize smart_search over find_file
- Add source prefix ([OCR], [ASRX], [ASRX+OCR]) to exec_smart_search results
- Users can now see chunk content with source indicators
- Add [OCR], [ASRX], [ASRX+OCR] prefix to text_content
- Add content field to SemanticSearchResult struct
- Update SQL queries to include content field
- Helps users distinguish the source of search results
- Fix OCR confidence threshold: 0.5 → 0.2
- Fix fetch_ocr_texts frame calculation (use start_frame from DB)
- Fix text_match filter to use text_content instead of summary
- Process all merged results (not just top 30) to include keyword results
- Add logging for keyword search debugging
Fixes issue where OCR text like 'AUDIO MONITORING' and 'thunderbolt'
could not be found via keyword search.
- agent_search.rs: text_region -> text_trace, removed skin_tone_trace
(never built, misleading LLM agents)
- tkg.rs + scan.rs: hand_object -> HAND_OBJECT for consistency with
all other edge types (CO_OCCURS_WITH, SPEAKS_AS, etc.)
- rule2_ingest.rs: added LIP_SYNC and HAND_OBJECT to edge_type
priority list so these edges generate relationship chunks
- Collect edges in Vec first, then batch insert in chunks of 100
- Uses sqlx::query_builder::QueryBuilder for efficient batch INSERT
- Reduces SQL round-trips from O(edges) to O(edges/100)
- Maintains ON CONFLICT DO UPDATE semantics
- scroll_face_points now retries up to 3 times on failure
- Exponential backoff: 1s, 2s, 4s between attempts
- Logs retry attempts for debugging
- Prevents silent failures when Qdrant is temporarily unavailable
- Create src/core/db/schema.rs with t() and table_name() functions
- Remove duplicate t() from tkg.rs and trace_agent_api.rs
- postgres_db.rs uses schema::table_name() from shared module
- Add build_node_id_map() to fetch all node IDs once into HashMap
- Pass node_id_map to all 6 edge builders
- build_co_occurrence_edges: uses map lookup instead of per-face SELECT
- build_speaker_face_edges: uses map lookup instead of per-trace SELECT
- Remove redundant Qdrant scroll in build_co_occurrence_edges
- Expected: 90%+ reduction in SQL queries during edge building
- Replaced old wrapper script with two launchd plists (server + worker)
- Both services auto-start at login via RunAtLoad + KeepAlive
- Removed MOMENTRY_FORCE_RETRY=true from .env to prevent infinite retries
- Added REDIS_URL and all required env vars to launchd plists
- Truncated 3.6GB of oversized logs (2.3G worker + 1.3G launchd stdout)
- Deleted 1214 orphaned processor_results records
- Updated run-server-3002.sh to manage via launchctl
- Removed obsolete wrapper scripts
- Created src/core/tkg/ module with service, log, and models
- Added tkg_operation_log table for tracking all TKG operations
- TkgService: build, rebuild (with force option), delete, get_operations
- Updated rebuild endpoint with force parameter
- Added GET /api/v1/file/:file_uuid/tkg for operation history
- Added DELETE /api/v1/file/:file_uuid/tkg for TKG deletion
- TKG rebuild now always triggers Rule 2 (even with 0 edges)
- Full audit trail for all TKG operations (create/update/delete/rebuild)
When a job status is 'completed', all ingestion steps are marked as 'done'
regardless of node/edge counts. This fixes the UI showing pending steps
for completed jobs with 0 nodes/edges (e.g., videos with single speaker).
1. Add Redis PipelineProgress cleanup in unregister_internal
- Deletes progress key when file is unregistered
2. Fix ingestion_complete asr_status query
- Query ASRX first, then ASR
- Filter out NULL values to avoid false negatives
3. Fix TKG 0 nodes/edges support
- TKG is considered done if face traces are complete
- TKG may create 0 nodes/edges for videos with minimal content
4. Add ASRX fallback to ASR segments
- When ASRX has 0 segments but ASR has segments, use ASR segments
- Ensures at least one ASRX output when ASR has content
When a job completes via the essential_completed branch (all essential
processors done but some non-essential failed), the PipelineProgress
was not being updated to 100%, causing a discrepancy where the status
showed 'completed' but progress showed <100%.
Now PipelineProgress.mark_completed() is called in both completion paths.
When ASRX falls back to ASR segments, end_frame may be 0.
The FPS calculation now handles this case correctly by checking
both end_frame > 0 and end_time > 0 before dividing.
This prevents division by zero and incorrect FPS values when
processing videos with ASRX fallback segments.
1. trace_done now checks for 'no_faces' status in face_traced.json
- Videos with no detected faces now complete correctly
- Previously stuck because trace_count=0 returned false
2. ASRX fallback to ASR segments includes start_frame/end_frame
- Added _convert_asr_segments_to_asrx helper function
- TKG can now process fallback segments correctly
This allows processing of:
- Videos with no audio track (ASR: no_audio_track)
- Videos with no faces (face_traced.json: no_faces)
When ASRX processing fails (error or exception), it now falls back to
ASR segments instead of returning silent_audio with 0 segments.
This ensures asrx_segments >= asr_segments always, fixing the bug where
ASRX would report 0 segments even when ASR detected speech.
When a file is re-registered, the old PipelineProgress in Redis
was not cleared, causing the Portal to show 'completed' status
even though the new job was still running.
Now PipelineProgress is deleted when a new job starts processing.
Previously, ingestion_complete only checked ASR and face traces,
causing jobs to complete before TKG was triggered. This resulted in
TKG nodes being 0 even after job completion.
Now ingestion_complete also checks if TKG nodes exist for the file.
The job stays in 'running' status until TKG completes.
The TKG guard was checking if nodes exist before spawning TKG build.
This caused a deadlock when:
1. TKG spawns (async)
2. ingestion_complete checks → false (nodes not yet created)
3. Returns Ok(false), job stays running
4. Next poll: guard checks nodes → no nodes yet (TKG still running)
5. Skips TKG → deadlock!
Since build_tkg uses ON CONFLICT (idempotent), it's safe to call
multiple times. Removed the guard to fix the deadlock.
When a job is deleted from the database (e.g., by unregister),
the worker now checks if the job still exists before processing.
If the job no longer exists, it skips it instead of getting stuck.
This fixes the issue where unregistering a file would leave the
worker stuck trying to process a non-existent job.
Identity agent now checks Qdrant _seeds collection for the file's
seed identity photos before running. If no seeds exist, the agent
is skipped to avoid unnecessary processing.
Flow:
1. Job completes (Face + ASRX done)
2. Check Qdrant _seeds for file_uuid
3. If seeds exist → run identity agent
4. If no seeds → skip identity agent
Previously Rule 1 only created chunks from ASRX segments, merging OCR
text where frame ranges overlapped. OCR text that didn't overlap with
any ASRX segment was ignored.
Now Rule 1 has two phases:
1. Process ASRX segments (merge OCR where overlapping) - existing behavior
2. Create chunks for OCR-only text (frames not covered by ASRX)
OCR-only chunks are grouped by consecutive frames (within 5 frames)
to avoid creating too many single-frame chunks.
Example: ASRX 819 + OCR-only 4 = 823 sentence chunks
Bug: ASRX handler stored pre_chunks as BOTH 'asrx' and 'asr' types.
This caused confusion because Rule 1 queries 'asrx' type, but only
'asr' type existed in the database (asrx type was deleted or never stored).
Fix: Remove the duplicate 'asr' storage (lines 530-542).
ASRX handler now only stores 'asrx' type pre_chunks to workspace SQLite.
PostgreSQL pre_chunks are stored by processor.rs with correct 'asrx' type.
This ensures Rule 1 can find ASRX pre_chunks correctly.
Accusys