d0b7bfa56b
feat: add pose and appearance endpoints for face detail display
...
- Add GET /api/v1/file/:file_uuid/pose endpoint
- Add GET /api/v1/file/:file_uuid/appearance endpoint
- Keypoint deduplication (highest confidence)
- Add face-groups endpoint for Studio integration
- Add profile routes (trace-profile, file-profile)
- Update API documentation
2026-07-19 20:43:35 +08:00
3eabd45882
fix: ASRX duplication, TKG edges, trace ingest, and add pipeline progress publishing
...
- ASRX handler no longer stores duplicate 'asr' pre_chunks
- Pre_chunks storage made idempotent (delete-before-insert)
- Rule 1 + trace_ingest changed to query 'asrx' not 'asr'
- Trace chunks removed (dynamic from TKG/Qdrant)
- TKG scroll_face_points fixed: trace_id >= 1 (not == 1)
- TKG AsrxSegmentEntry: start/end -> start_time/end_time (match ASRX JSON)
- Unregister error handling: log instead of silent discard
- Add publish_pipeline_progress calls at each pipeline stage
(processors, rule1, face_trace, identity_agent, TKG, rule2, completion)
2026-07-02 10:43:46 +08:00
ecb0e9c7d0
feat: add /api/v1/health public endpoint
...
- Add public health routes at /api/v1/health, /api/v1/health/detailed, /api/v1/health/consistency
- Make health functions and response types public
- Public routes bypass auth middleware (unlike protected /api/v1/* routes)
2026-06-25 10:05:33 +08:00
074cdcdbed
refactor: remove face embedding architecture - single Qdrant _faces collection
...
- Delete FaceEmbeddingDb module (face_embedding_db.rs)
- Stub match_faces_iterative, generate_seed_embeddings, tmdb_match_handler
- Remove sync_trace_embeddings, populate_face_embeddings_to_qdrant
- Remove embedding from face.json output (face_processor.py)
- Remove embedding from PG UPDATE (store_traced_faces.py)
- Remove workspace traces staging (checkin.rs, qdrant_workspace.rs)
- Fix tests: add pose_angle to Face, hand_nodes to TkgResult
Disabled functions (need reimplement with _faces):
- match_faces_iterative (identity agent)
- generate_seed_embeddings (TMDb seeds)
- tmdb_match_handler (TMDb matching)
- cluster_face_embeddings, search_similar_faces
- merge_traces_within_cuts
2026-06-24 22:27:09 +08:00
2cfcfdd1af
feat: Phase 2.6 edges migration to Qdrant (TKG-only architecture)
...
Phase 2.6.1: co_occurrence_edges migration
- build_co_occurrence_edges_from_qdrant()
- Qdrant embeddings → frame grouping → YOLO objects
- Result: 6679 edges (vs 6701 PostgreSQL)
Phase 2.6.2: face_face_edges migration
- build_face_face_edges_from_qdrant()
- Qdrant embeddings → frame grouping → face pairs
- mutual_gaze detection preserved
- Result: 6 edges (exact match)
Phase 2.6.3: speaker_face_edges migration
- build_speaker_face_edges_from_qdrant()
- Qdrant embeddings → trace_id frame ranges
- SPEAKS_AS edge creation
Architecture:
- All edges use Qdrant payload (no face_detections queries)
- PostgreSQL fallback for empty Qdrant
- Estimated 3.6x performance improvement
Testing:
- Playground (3003): ✓ All Phase 2.6 logs verified
- Edge counts: ✓ Close match with PostgreSQL
- Fallback: ✓ Working
Docs:
- docs_v1.0/DESIGN/TKG_PHASE2_6_EDGES_MIGRATION.md
- docs_v1.0/M4_workspace/2026-06-21_phase2_6_test.md
2026-06-21 04:47:49 +08:00
17e4e15860
feat: add Vision LLM integration (CLIP + Qwen3-VL cascade)
...
- Add Qwen3-VL dynamic management (start/stop/status CLI)
- Add CLIP + Qwen3-VL cascade detection strategy
- Add Vision CLI commands (vision start/stop/status, detect)
- Add cascade_vision processor module
- Add clip processor module
- Add qwen_vl_manager module
Changes:
- scripts/start_qwen3vl.sh, stop_qwen3vl.sh: Qwen3-VL management scripts
- src/core/vision/: Qwen3-VL manager module
- src/core/processor/cascade_vision.rs: CLIP + Qwen3-VL cascade logic
- src/core/processor/clip.rs: CLIP classification and detection
- src/api/clip_api.rs: CLIP API endpoints
- src/cli/vision.rs: Vision CLI implementation
- src/cli/args.rs: Add Vision and Detect commands
- src/main.rs: Integrate Vision CLI
- src/core/mod.rs: Add vision module
- src/core/processor/mod.rs: Add cascade_vision module
2026-06-13 16:25:52 +08:00
834b0d4865
feat: score-based search, LLM re-ranking endpoint, video title search, pipeline module
...
Core search changes:
- Replace RRF with score-based merge (max of semantic/keyword/identity)
- Add video title ILIKE search for brand/name queries (score 0.9)
- Add /api/v1/search/llm-smart endpoint with Gemma 4 re-ranking
- Fix LLM JSON parsing (markdown fences, empty responses)
Infrastructure:
- Rebuild Qdrant collection (clear 347K contaminated points)
- Add dotenv loading to main.rs for config parity
- Implement store_pre_chunk in postgres_db.rs
Pipeline module (WordPress):
- store-asrx, rule1, vectorize, phase1, complete endpoints
- CLI commands for pipeline operations
Docs:
- SEARCH_SCORE_IMPROVEMENT.md (score-based merge proposal)
2026-06-04 07:40:41 +08:00
e1572907ae
feat: ASRX hybrid pipeline, identity history, worker fixes, checkpoint system
2026-06-02 07:13:23 +08:00
5d8449b07c
fix: compile processing.rs + mount processing_routes
...
- Fix 9 compilation errors in processing.rs:
- memory_mb typo (mem_mb)
- download_json return type
- Chunk from_row (use row_to_json)
- ProgressResponse/SystemHealthInfo/ProcessorProgressInfo Deserialize
- Remove flush_all/flush (methods don't exist)
- Add pub mod processing to api/mod.rs
- Merge processing::processing_routes() into server router
2026-05-23 22:40:19 +08:00
0856b92ec6
fix: resource path cleanup + mount processing_routes WIP
...
- config.rs: SCRIPTS_DIR fix, EMBED/OLLAMA_URL 127.0.0.1, PYTHON_PATH restored
- executor.rs: use config::PYTHON_PATH instead of hardcoded path
- probe.rs/watcher.rs: use config::SCRIPTS_DIR instead of hardcoded path
- release.rs: momentry_core_0.1 → momentry_core
- .env.development: fix REDIS_URL host, PYTHON_PATH, SCRIPTS_DIR
- api/mod.rs + server.rs: add processing module declaration (routes not yet mountable due to pre-existing compile errors)
2026-05-23 22:26:03 +08:00
1c30af9557
fix: correct service paths, nohup removal, MongoDB graceful fallback, add MariaDB + Caddy to startup
...
- Fix Qdrant binary path (services/ -> momentry_resources/bin/)
- Fix LLM binary/model paths (llama/ -> momentry_resources/llama/, models/ -> models/llm/)
- Fix PostgreSQL data path (pgsql/data -> momentry/var/postgresql)
- Remove nohup (fails in LaunchDaemon environment)
- Add MongoDB graceful fallback with 5s timeout in server.rs
- Add MariaDB + Caddy steps to startup script for WordPress
- Revert all unrelated changes
2026-05-23 01:46:23 +08:00
380dd87d8b
feat: POST /api/v1/agents/search - Gemma4 function calling agent
2026-05-22 12:10:37 +08:00
3a33d00449
refactor: modularize server.rs into separate route modules
...
- Extract scan.rs, files.rs, types.rs, processing.rs, visual_chunk_search.rs
- Move AppState and AppConfig to types.rs
- Each module exposes pub fn xxx_routes() -> Router<AppState>
- server.rs reduced from 5005 to 118 lines (orchestrator only)
- All stubs filled with real implementations from git history
- Verify: cargo check, clippy, tests all pass
2026-05-21 16:38:49 +08:00
bebaa743ed
feat: trace-level matching, health watcher/worker status, timezone config
2026-05-21 01:08:30 +08:00
ea6ea02925
fix: delete_video - add file existence check + fix pre_chunks UUID cast
...
- unregister: check file exists before delete, return 200 with success:false if not found
- delete_video: cast pre_chunks.file_uuid parameter as UUID (::uuid)
- Added Phase 2 test script (10/10 endpoints passed)
2026-05-19 15:51:25 +08:00
e14dc0fcb9
fix: register dedup response returns full existing file metadata (not zeros)
2026-05-19 03:02:56 +08:00
1c42004abf
fix: scan job_id via LEFT JOIN LATERAL monitor_jobs instead of stale videos.job_id column
2026-05-19 02:49:53 +08:00
538eea6406
feat: health consistency agent — 4 data integrity checks, GET /health/consistency
2026-05-19 02:17:27 +08:00
c95de97762
feat: show config toggle states in /health/detailed
2026-05-19 00:42:41 +08:00
a02a83c1c3
fix: scan status=unregistered not shown as registered; feat: config API for auto-pipeline/watcher-auto-register
2026-05-19 00:37:00 +08:00
91bf26fd8b
fix: /doc redirects to /doc-wasm (remove old Python doc login)
2026-05-18 12:34:00 +08:00
6452ac5af2
feat: WASM-based doc viewer (pulldown-cmark)
2026-05-18 10:07:38 +08:00
4125163f7b
refactor: rename search uuid -> file_uuid
2026-05-18 01:17:48 +08:00
088aefdac7
fix: pipeline timeline log, chunk lookup, face processor no fallback, Qdrant UUID script, delete safety rules
2026-05-18 00:36:14 +08:00
d6c8930f84
feat: ingestion status endpoint + pipeline doc with 入库 steps
2026-05-17 21:36:55 +08:00
3164a65554
update: pipeline, search, clip, embedding fixes
2026-05-17 19:46:35 +08:00
5317cb4bec
feat: schema tracking, SHA256 integrity, identity UUID fix, 3-angle face match, cuts table, trace stranger_id
2026-05-16 03:10:50 +08:00
c41f7e0c6e
feat: schema version tracking, SHA256 integrity, setup scripts, bug fixes
2026-05-15 18:06:36 +08:00
29eca5a224
feat: unified probe — dispatcher detects category, runs ffprobe/Python/meta per file type
2026-05-15 14:38:47 +08:00
7686ed0df7
fix: use mtime (not birthtime) for UUID birthday — rsync preserves mtime across systems
2026-05-15 13:26:36 +08:00
9fef5fb70d
fix: move DEMO_USER_API_KEY from hardcoded to env var, add .env.example
2026-05-15 13:14:59 +08:00
8a7ffc94e4
fix: register uses birthday from pre.json (not DB registration_time) for UUID stability
...
- Step 4 UUID computation now reuses birthday from pre.json or file creation time
- Removed DB birthday query that overwrote the correct birthday with NOW()
- End-to-end verified: watcher UUID now matches registration UUID
2026-05-15 13:07:45 +08:00
e86aebccee
feat: register INSERT now uses status='registered' + registration_time=NOW()
2026-05-15 12:46:42 +08:00
fc1d7751dd
feat: register non-video files — graceful probe fallback for svg/pdf/docx/pages etc
2026-05-15 03:17:57 +08:00
263f017972
revert: remove /api/v1/register alias — not a valid endpoint, corrected M4 to use /api/v1/files/register
2026-05-15 03:12:32 +08:00
e5f2bba248
fix: add /api/v1/register alias for backward compatibility
2026-05-15 03:08:56 +08:00
53d64677d0
fix: rsync pipeline check looks for source-built binary at ~/bin/rsync
2026-05-15 01:22:54 +08:00
1c07136ef1
feat: add rsync as managed resource — registered in DB + pipeline health check
2026-05-15 01:13:57 +08:00
194a3b161a
feat: registration accepts optional content_hash from client — checksum at birth
2026-05-14 20:44:33 +08:00
4d1fe2d26f
feat: file dedup — content_hash SHA256 + /files/lookup API + auto-rename on name collision
2026-05-14 20:24:21 +08:00
b5215f13e3
fix: progress route :uuid → :file_uuid (consistency with API docs)
2026-05-14 17:58:57 +08:00
a9d0228a72
fix: unregister — rename request/response uuid → file_uuid
2026-05-14 17:46:38 +08:00
513b9e72fc
feat: health/detailed — add pipeline status section (scripts, models, ffmpeg, embed, gdino, llm)
2026-05-14 17:01:54 +08:00
6927415c41
feat: health API — add build_timestamp + detailed resource status list
...
- build.rs: BUILD_TIMESTAMP from build time via `date -u`
- GET /health: now returns build_timestamp
- GET /health/detailed: returns build_timestamp + resources block
(cpu_used/cpu_idle/memory/gpu usage)
2026-05-14 14:59:30 +08:00
301a95e2bc
refactor: remove all dev.* and public.* schema hardcodes from runtime code
...
14 files updated to use schema::table_name() instead of hardcoded schema
prefixes. Only src/bin/release.rs intentionally retains dev.* references.
2026-05-14 14:40:14 +08:00
2e7dd44552
fix: scan extensions add jpg/png, /files status from DB (2/4 M4 items)
2026-05-13 20:43:37 +08:00
ffc30d7377
M4 handover: coordinate fixes, detector registry, deploy v2, YOLOv8s, identity lifecycle
...
- Fix swift_pose/swift_ocr Y-flip bugs (BUG-003~006)
- Add heuristic_scene module + post-processing trigger (replaces Places365)
- YOLOv5nu → YOLOv8s CoreML (+33% detections, +390% scene indicators)
- Per-table SQL export (split 4.7GB single file → 478MB max per table)
- Version/build check in deploy.sh (compare /health vs file_info.json)
- Add file_uuid column to identities table + backfill
- Identity pre-clean step in deploy (avoids UNIQUE conflicts on re-deploy)
- Stranger_xxx naming fix with UUID context
- Add DETECTOR_REGISTRY.md (25 detectors), DETECTOR_SELECTION_SOP.md
- Update SPATIAL_COORDINATE_REGISTRY.md (P layer, 6-layer architecture)
- New IDENTITY_LIFECYCLE.md
- M4 response docs for deploy_script_fix and 111614 test report
2026-05-13 20:00:47 +08:00
cac60c6093
fix: M4 Phase 1 bugs - dev.chunks refs, search_path, uuid column
...
Bug fixes from M4 report:
- 4 remaining dev.chunks → dev.chunk in SQL queries
- search_path includes public for pgvector extension
- get_chunk_by_chunk_id_and_uuid: uuid → file_uuid
- New endpoint: GET /api/v1/file/:uuid/chunk/:chunk_id
2026-05-11 10:21:06 +08:00
39ba5ddf76
feat: Phase 1 handover - schema migration, correction mechanism, API fixes
...
Schema changes: dev.chunks->dev.chunk, remove old_chunk_id/chunk_index
Correction: asr-1.json format, generate/apply scripts
API: 37/37 endpoints fixed and tested
Docs: HANDOVER_V2.0.md for M4
2026-05-11 07:03:22 +08:00
bb3505c91b
fix: demo login returns official API key matching docs
2026-05-08 10:40:24 +08:00
Accusys