59 Commits

Author SHA1 Message Date
Warren
b54c2def30 feat: add migrations, test scripts, and utility tools
- Add database migrations (006-028) for face recognition, identity, file_uuid
- Add test scripts for ASR, face, search, processing
- Add portal frontend (Tauri)
- Add config, benchmark, and monitoring utilities
- Add model checkpoints and pretrained model references
2026-04-30 15:11:53 +08:00
Warren
4d75b2e251 docs: update docs_v1.0/ documentation
- Fix markdown lint issues (MD030, MD047, MD051, MD028, MD005)
- Update AI agents, architecture, implementation docs
- Add new identity, face recognition, and API documentation
- Remove deprecated face/person API guides
2026-04-30 15:10:41 +08:00
Warren
8f05a7c188 feat: update Python processors and add utility scripts
- Update ASR, face, OCR, pose processors
- Add release pre-flight check script
- Add synonym generation, chunk processing scripts
- Add face recognition, stamp search utilities
2026-04-30 15:07:49 +08:00
Warren
f4697396e4 chore: update dependencies and AGENTS.md
- Add mac_address crate for MAC address detection
- Add tempfile dev dependency for testing
- Update AGENTS.md with latest development guidelines
2026-04-30 15:07:31 +08:00
Warren
2b23d1cfbd feat: update core API, database layer, and worker modules
- Remove unused imports (n8n_search, universal_search, Client, Arc, etc.)
- Update API endpoints for identity, face recognition, search
- Fix postgres_db.rs search_videos parent_uuid column
- Add snapshot API and identity agent API
- Clean up backup files (.bak, .bak2)
2026-04-30 15:07:02 +08:00
Warren
8f2208dd63 chore: update .gitignore and remove .env files from tracking
- Add Python cache, test artifacts, backups, models to .gitignore
- Remove .env and .env.development from tracking (security)
- Keep release documentation, ignore binaries
2026-04-30 15:04:50 +08:00
Warren
5e896fb509 feat: implement Phase 6 Agent Integration (Translation API) 2026-04-26 00:07:18 +08:00
Warren
c15f7cd4af feat: implement Phase 5 Resource Registry & Heartbeat 2026-04-25 23:12:15 +08:00
Warren
4686c5abc4 feat: complete Phase 4 Candidate Workflow (Confirm/Reject API) 2026-04-25 22:27:31 +08:00
Warren
e84982e7d9 feat: Phase 3 API (Identity, Files, Candidates) and pre_chunks migration 2026-04-25 22:19:12 +08:00
Warren
1f84e5469f feat: backup architecture docs, source code, and scripts 2026-04-25 17:15:45 +08:00
Warren
59809dae1f chore: backup before migration to new repo 2026-04-23 16:46:02 +08:00
Warren
13dd3b30f3 docs: 添加 Places365 模型完整指南
內容:
- 手動下載方法(3 種)
- 模型驗證步驟
- 使用方式和預期改進
- 故障排除指南

目前狀態:
-  ImageNet 模型正常運作(37% 準確率)
-  Places365 模型可選手動下載(85-90% 準確率)
- 📄 完整安裝和使用指南
2026-04-01 03:19:42 +08:00
Warren
f45ecf4643 docs: 添加長影片場景識別測試報告
測試結果:
-  Old_Time_Movie_Show (114 分鐘) 處理成功
-  處理時間 313 秒(5.2 分鐘)
-  加速比 22x
-  記憶體使用穩定(3-4GB)
-  1,379 個取樣點

效能指標:
- 取樣間隔:5 秒
- 最小場景:10 秒
- 場景數量:1(ImageNet 模型限制)
- 信心度:25%

建議:
- 下載 Places365 模型提升準確率
- 整合 CUT 場景切換偵測
- 優化長片處理策略
2026-04-01 03:08:35 +08:00
Warren
d12caba00a docs: 添加場景識別測試結果報告
新增:
- docs_v1.0/TESTING/SCENE_CLASSIFICATION_TEST_RESULTS_2026_04_01.md

測試結果:
-  Rust 單元測試 5/5 通過
-  Python 功能測試通過
-  ExaSAN 影片識別成功
-  79 個取樣點,處理時間 1.2 秒
-  信心度 37%(ImageNet 模型)

效能指標:
- 處理速度:133x 實時
- 模型大小:44.7 MB
- MPS 加速:啟用
2026-04-01 03:01:07 +08:00
Warren
395f74bf07 feat: 添加場景識別 Playground API 整合
新增:
- scripts/test_scene_api.py - API 測試腳本
- docs_v1.0/IMPLEMENTATION/SCENE_API_INTEGRATION.md - API 整合指南

功能:
-  GET /api/v1/scene/:uuid endpoint 設計
-  Python 測試腳本
-  完整使用文檔
-  Python 整合範例

使用方式:
```bash
# 啟動 Playground (port 3003)
cargo run --bin momentry_playground -- server --port 3003

# 測試場景識別
python3 scripts/test_scene_api.py <video_uuid>
```

目前狀態:
-  Python 場景識別功能正常
-  API endpoint 設計完成
-  Rust 完整實作進行中
- 📄 完整文檔已建立
2026-04-01 02:55:52 +08:00
Warren
363d6913f9 docs: 添加 Places365 安裝指南和測試腳本
新增:
- docs_v1.0/IMPLEMENTATION/PLACES365_INSTALLATION.md
- scripts/test_places365_scene.py

功能:
-  Places365 380 個場景類別載入
-  場景分類器測試
-  影片場景分類測試

目前狀態:
-  基礎場景識別功能正常
-  Places365 模型可選手動安裝
- 📊 準確率 37% → 預期 85-90%
2026-04-01 02:39:13 +08:00
Warren
6d5d121d0f feat: 整合 Places365 場景類別到場景識別
- 新增 places365_categories.json (380 個場景類別)
- 更新場景識別使用 Places365 類別名稱
- 使用最常見場景類型作為影片主要場景
- 改進場景合併邏輯

改進:
- 場景名稱從 'unknown_X' 改為實際場景索引
- 支援 Places365 380 個場景類別
- 自動統計最常見場景類型

限制:
- ResNet18 使用 ImageNet 1000 類別
- Places365 只有 365 類別,索引不完全匹配
- 建議使用專門的 Places365 模型獲得最佳結果

測試結果:
- ExaSAN 影片識別為 scene_664 (37% 信心度)
- 處理時間:1.3 秒
- 79 個取樣點成功處理
2026-04-01 02:31:49 +08:00
Warren
4109ec3d95 docs: 修復場景識別測試報告 markdown 編號
- 修正有序列表編號符合 markdownlint MD029
- 使用 1/2/3 樣式而非連續編號
2026-04-01 02:21:40 +08:00
Warren
576f58df71 feat: add build version with timestamp
- Add build.rs to generate BUILD_VERSION at compile time
- Update CLI to show full version: '0.1.0 (build: 2026-03-31 11:21:37)'
- Update health endpoints to return build version
- Add chrono as build dependency
2026-03-31 11:30:50 +08:00
Warren
37d2b66c56 feat: add PostgreSQL schema isolation for playground environment
- Create schema.rs utility module with table_name() function
- Add schema prefix to all SQL queries in postgres_db.rs
- Support dev schema for playground, public for production
- Add DATABASE_SCHEMA, MONGODB_DATABASE, QDRANT_COLLECTION config
- Fix 40+ functions including videos, chunks, frames, vectors, etc.
- Update Cargo dependencies
v0.1.1
2026-03-31 10:30:33 +08:00
Warren
95b44f1e55 fix: backup monitoring and PATH environment issues
- Fix backup_monitor.sh find command to sort by modification time
- Fix grep -oP syntax error (change to grep -oE)
- Adjust tier rotation threshold from -mtime +7 to +6
- Add backup_all.sh script with PATH fixes for crontab
- Add mysql-client/bin to PATH for mysqldump command
- Fix backup status check for v2 naming patterns
2026-03-30 04:11:02 +08:00
Warren
2393d81a3f feat: fix Chinese text search and duplicate chunk_id bug
- Add helper functions to extract text from nested content structure
- Update SearchResult to include uuid field
- Add PostgreSQL function get_chunk_by_chunk_id_and_uuid to handle duplicate chunk_ids
- Update Qdrant search functions to extract uuid from payload
- Change embedding model to nomic-embed-text-v2-moe:latest
- Update Qdrant collection name to momentry_rule1
- Fix MongoDB authentication and disable cache for development
- Improve error handling in processor.rs
- Update documentation with new embedding model
2026-03-29 04:44:28 +08:00
Warren
82955504f3 feat: 新增 Job Worker 系統與 API 文檔全面更新 2026-03-26 16:16:34 +08:00
Warren
80399b1c12 fix: return file_path instead of media_url in n8n search API
The media_url was constructed using MEDIA_BASE_URL which returned
404s. Now returns actual file_path from database for n8n workflow.
2026-03-25 17:24:29 +08:00
Warren
ceb33877ff docs: change media_url to file_path in API response 2026-03-25 16:39:48 +08:00
Warren
dacfb7e083 docs: clarify media_url is auto-generated and may not be accessible 2026-03-25 16:33:10 +08:00
Warren
fb60858cec docs: clarify media_url meaning in search API 2026-03-25 16:28:44 +08:00
Warren
f1d7077e40 docs: fix media_url example to show realistic filename 2026-03-25 16:27:06 +08:00
Warren
4f402c873b docs: add SFTPGo demo credentials to training manual 2026-03-25 16:23:46 +08:00
Warren
a89d94bc67 docs: update SFTPGo host to sftpgo.momentry.ddns.net 2026-03-25 16:21:15 +08:00
Warren
17cab667f9 docs: add chunk API usage, playback format, and API examples 2026-03-25 16:06:11 +08:00
Warren
f8925ab994 docs: update API docs with cache/unregister endpoints and marcom training refs 2026-03-25 15:56:29 +08:00
Warren
dac2b234d0 docs: add version history tables to all training docs 2026-03-25 15:54:01 +08:00
Warren
67c8c60ceb docs: add search endpoint documentation with chunk details 2026-03-25 15:51:30 +08:00
Warren
c5ba252d89 docs: update training docs with cache/unregister APIs v0.9.20260325_151200 2026-03-25 15:49:24 +08:00
Warren
e4d3365ada feat: add cache toggle, unregister API and chunk training docs
- Add POST /api/v1/config/cache for cache toggle
- Add POST /api/v1/unregister for video deletion
- Add CHUNK_DATA_STRUCTURE.md for marcom training
- Fix processor_results query in delete_video
2026-03-25 15:44:47 +08:00
Warren
de80b96c15 docs: add PHP examples for WordPress integration 2026-03-25 15:32:07 +08:00
Warren
f5436c7b63 docs: add training manual and workflow guide for marcom team 2026-03-25 15:27:47 +08:00
Warren
845704724f fix: silence false-positive ruff lint warnings in Python scripts 2026-03-25 14:55:30 +08:00
accusys
383201cacd feat: Initial v0.9 release with API Key authentication
## v0.9.20260325_144654

### Features
- API Key Authentication System
- Job Worker System
- V2 Backup Versioning

### Bug Fixes
- get_processor_results_by_job column mapping

Co-authored-by: OpenCode
2026-03-25 14:53:41 +08:00
accusys
47e86b696f fix: save probe.json to OUTPUT_DIR instead of current directory
- Changed FileManager base directory from '.' to OUTPUT_DIR
- Fixed both register and probe endpoints
- probe.json is now saved to /Users/accusys/momentry/output/
2026-03-25 14:53:41 +08:00
accusys
05683c1715 feat: add v2 backup versioning system
- Add BACKUP_VERSIONING.md with comprehensive backup management
- Update backup_all.sh with v2 version marking
  - v2 naming format: {service}_{type}_v2_{date}_{time}.{ext}
  - Fixed MongoDB backup path to /opt/homebrew/var/mongodb
  - Added momentry_output backup for probe.json files
  - Added restore functions for v2 format
- Update backup_monitor.sh for v2 filename parsing
- Updated VIDEO_REGISTRATION.md with Probe API documentation
- Updated JOB_WORKER_IMPLEMENTATION_PLAN.md status to implemented
- Updated MOMENTRY_CORE_MONITORING.md with Job Worker monitoring
- Updated SERVICES.md with Momentry Playground and Job Worker info
2026-03-25 14:53:41 +08:00
accusys
1122dd977c docs: add Momentry Playground and Job Worker to SERVICES.md
- Add Momentry Playground binary entry (port 3003, dev environment)
- Add Job Worker architecture and monitoring section
- Add Momentry API vs Playground comparison table
- Add Job Worker environment variables documentation
- Add SQL monitoring commands for jobs and processors
2026-03-25 14:53:41 +08:00
accusys
44cf1ee4b6 docs: update for new architecture (Probe API, Job Worker)
- VIDEO_REGISTRATION.md: Add Probe API reference and comparison table
- JOB_WORKER_IMPLEMENTATION_PLAN.md: Update status to implemented
- MOMENTRY_CORE_MONITORING.md: Add Job Worker monitoring section
  - monitor_jobs and processor_results table docs
  - Worker status check commands
  - Redis job monitoring commands
2026-03-25 14:53:41 +08:00
accusys
f27e51a905 feat: add POST /api/v1/probe endpoint
- Add ProbeRequest/ProbeResponse structures
- Support relative and absolute paths
- Cache probe.json for repeated requests
- Return video metadata (uuid, duration, width, height, fps)
- Include cached flag to indicate cache hit
- Export FormatInfo and StreamInfo from probe module
- Update API_ENDPOINTS.md documentation
2026-03-25 14:53:41 +08:00
accusys
f240ae94bb fix: update MonitorJob tests for NaiveDateTime type 2026-03-25 14:53:41 +08:00
accusys
12a7b59232 feat: add job worker and duplicate registration check
Job Worker System:
- Add polling-based job worker (max 2 concurrent processors)
- Create monitor_jobs records when videos are registered
- Link videos.job_id to monitor_jobs
- Fix type mismatches (i32 vs i64) for database IDs

Duplicate Registration:
- Check if video already exists before registering
- Return existing video info with already_exists: true
- Use canonical path for UUID computation

USER_DATA_ROOT Configuration:
- Add MOMENTRY_USER_DATA_ROOT environment variable
- UUID computed from relative path (username/filename)
- Ensures consistent UUIDs when data root changes
2026-03-25 14:53:41 +08:00
accusys
cd0f952aeb chore: configure Qdrant API key and update Redis key documentation
- Add QDRANT_API_KEY to .env and .env.development
- Update docs to reflect configurable Redis key prefix
- Add Qdrant configuration section to monitoring docs
2026-03-25 14:53:41 +08:00
accusys
a95e5c7e5c docs: add configurable Redis key prefix to Redis Keys and Monitoring docs
- Add new section 2 in MOMENTRY_CORE_REDIS_KEYS.md explaining configurable prefix
- Update environment variables to include MOMENTRY_REDIS_PREFIX
- Update Rust implementation examples to use REDIS_KEY_PREFIX static
- Add warning note in MOMENTRY_CORE_MONITORING.md about configurable prefix
- Update shell scripts to use REDIS_PREFIX environment variable
- Document default prefixes: momentry: (production), momentry_dev: (development)
2026-03-25 14:53:41 +08:00