docs: file_uuid generation rules for M4
This commit is contained in:
@@ -0,0 +1,57 @@
|
||||
# 3002 vs 3003 API 比較報告
|
||||
|
||||
**日期**: 2026-05-13 | **目的**: 正式版與開發版 API 差異(Release 前)
|
||||
|
||||
---
|
||||
|
||||
## 1. 版本差異
|
||||
|
||||
| 項目 | 3002 (正式版) | 3003 (開發版) |
|
||||
|------|:---:|:---:|
|
||||
| **Build 日期** | 2026-05-13 21:07 | 2026-05-13 (持續開發) |
|
||||
| **Version** | v1.0.0 | v1.0.0 |
|
||||
| **Build Git Hash** | `d34bcae+` | `d34bcae+` |
|
||||
| **程式碼狀態** | ✅ 同步(從 dev release binary) | ✅ 最新 |
|
||||
| **Schema** | `public`(需執行 `chunks→chunk` RENAME) | `dev` |
|
||||
|
||||
---
|
||||
|
||||
## 2. 共同端點(JSON 結構一致)
|
||||
|
||||
| 端點 | 3002 | 3003 | 備註 |
|
||||
|------|:---:|:---:|------|
|
||||
| `GET /health` | ✅ | ✅ | 含 `version` + `build_git_hash` |
|
||||
| `GET /health/detailed` | ✅ | ✅ | 同上 |
|
||||
| `GET /api/v1/files` | ✅ | ✅ | `total` 從 DB 讀取(不再寫死 0) |
|
||||
| `GET /api/v1/files/scan` | ✅ | ✅ | 含 `.jpg/.png` 掃描(不再限 mp4) |
|
||||
| `GET /api/v1/file/:uuid/process` | ✅ | ✅ | |
|
||||
| `GET /api/v1/file/:uuid/chunk/:id` | ✅ | ✅ | |
|
||||
| `GET /api/v1/identities` | ✅ | ✅ | 含分頁 |
|
||||
| `GET /api/v1/identities/:id` | ✅ | ✅ | |
|
||||
| `GET /api/v1/identity_bindings` | ✅ | ✅ | |
|
||||
| `POST /api/v1/search/universal` | ✅ | ✅ | |
|
||||
| `GET /api/v1/resources` | ✅ | ✅ | |
|
||||
| `GET /api/v1/traces/:tid/faces` | ✅ | ✅ | |
|
||||
| `GET /api/v1/traces/:tid/video` | ✅ | ✅ | |
|
||||
|
||||
---
|
||||
|
||||
## 3. 差異
|
||||
|
||||
| 項目 | 3002 | 3003 |
|
||||
|------|------|------|
|
||||
| PostgreSQL Schema | `public`(需 rename `chunks→chunk`) | `dev`(已為 `chunk`) |
|
||||
| MongoDB Database | `momentry` | `momentry_dev` |
|
||||
| Redis Prefix | `momentry:` | `momentry_dev:` |
|
||||
| Qdrant Collection Prefix | `momentry_` | `momentry_dev_` |
|
||||
| Output Dir | `/Users/accusys/momentry/output` | `/Users/accusys/momentry/output_dev` |
|
||||
| `.env` | `.env`(`DATABASE_SCHEMA=public`) | `.env.development`(`DATABASE_SCHEMA=dev`) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Release 必要步驟
|
||||
|
||||
1. **Binary**:使用 M5 交付的 `momentry_v1.0.0` 取代 port 3002 binary
|
||||
2. **Schema**:`ALTER TABLE public.chunks RENAME TO chunk;`
|
||||
3. **Deploy**:`bash deploy.sh`(9 步驟,含 vec0.dylib)
|
||||
4. **Identity**:保留 15 TMDB + merge dev data(`file_uuid` 欄位輔助)
|
||||
@@ -0,0 +1,139 @@
|
||||
# Brew → Source 遷移報告
|
||||
|
||||
**Date**: 2026-05-15
|
||||
**Status**: Planning
|
||||
**Next action**: 逐項驗證 SHA256 + 下載 Source → Build
|
||||
|
||||
---
|
||||
|
||||
## 總覽
|
||||
|
||||
Momentry Core 目前有 18 個核心服務透過 Homebrew 管理。目標是將這些服務全部遷移到 source build(原始碼編譯),實現 source code 可追蹤、可驗證、可重複建置。
|
||||
|
||||
---
|
||||
|
||||
## Momentry Core Brew 套件一覽
|
||||
|
||||
| # | Formula | Version | Binary Path | SHA256 | Status |
|
||||
|---|---------|---------|-------------|--------|:------:|
|
||||
| 1 | **php** | 8.5.5 | `/opt/homebrew/bin/php` | `173fd1ca36f3dd4952f5442572e06a14b7c005751ae15e7e42161606e931645c` | 🔴 brew |
|
||||
| 2 | **mariadb** | 12.2.2 | `/opt/homebrew/bin/mariadbd` | `38cb48f0be673d4136c43a89c1aca5b314d30042dd09537d93b7995f52f90206` | 🔴 brew |
|
||||
| 3 | **redis** | 8.6.3 | `/opt/homebrew/bin/redis-server` | ? | 🟡 brew+src |
|
||||
| 4 | **mongodb-community** | 8.2.7 | `/opt/homebrew/bin/mongod` | ? | 🔴 brew |
|
||||
| 5 | **ffmpeg** | 8.1.1 | `/opt/homebrew/bin/ffmpeg` | ? | 🟡 brew+src |
|
||||
| 6 | **ffmpeg-full** | 8.1.1 | `/opt/homebrew/opt/ffmpeg-full/bin/ffmpeg` | ? | 🟡 brew+src |
|
||||
| 7 | **node** | 25.9.0 | `/opt/homebrew/bin/node` | `fba87e4402c55ea4fc7ca9b9838790c32534e3e77c9c7834c37073752d070678` | 🔴 brew |
|
||||
| 8 | **go** | 1.26.2 | `/opt/homebrew/bin/go` | ? | 🟡 brew |
|
||||
| 9 | **python@3.11** | 3.11.15 | `/opt/homebrew/bin/python3.11` | ? | ✅ pyenv src |
|
||||
| 10 | **ollama** | 0.23.1 | `/opt/homebrew/bin/ollama` | ? | 🔴 brew |
|
||||
| 11 | **yt-dlp** | 2026.3.17 | `/opt/homebrew/bin/yt-dlp` | ? | 🔴 brew |
|
||||
| 12 | **whisper-cpp** | 1.8.4 | `/opt/homebrew/bin/whisper-cpp` | ? | 🔴 brew |
|
||||
| 13 | **tesseract** | 5.5.2 | `/opt/homebrew/bin/tesseract` | ? | 🔴 brew |
|
||||
| 14 | **sdl2** | 2.32.10 | `/opt/homebrew/lib/libsdl2.dylib` | ? | 🟡 lib only |
|
||||
| 15 | **cmake** | 4.3.2 | `/opt/homebrew/bin/cmake` | ? | ✅ src in `services/src/` |
|
||||
| 16 | **mongosh** | 2.8.3 | `/opt/homebrew/bin/mongosh` | ? | 🔴 brew |
|
||||
| 17 | **pgvector** | 0.8.2 | PostgreSQL extension | ? | 🟡 extension |
|
||||
| 18 | **protobuf** | - | `/opt/homebrew/bin/protoc` | ? | 🟡 build dep |
|
||||
|
||||
---
|
||||
|
||||
## 遷移優先級
|
||||
|
||||
### Phase 1 — 直接影響 Momentry 運行的服務
|
||||
|
||||
| Service | Source | 遷移原因 |
|
||||
|---------|--------|---------|
|
||||
| PHP 8.5.5 | `https://www.php.net/distributions/php-8.5.5.tar.gz` | WordPress hosting, FPM 不可中斷 |
|
||||
| MariaDB 12.2.2 | `https://github.com/MariaDB/server/archive/mariadb-12.2.2.tar.gz` | WordPress + Momentry DB, data 需 migrate |
|
||||
| Node.js 25.9.0 | `https://nodejs.org/dist/v25.9.0/node-v25.9.0.tar.gz` | Portal frontend build + npm packages |
|
||||
|
||||
### Phase 2 — 高影響但可有 Buffer 的服務
|
||||
|
||||
| Service | Source | 遷移原因 |
|
||||
|---------|--------|---------|
|
||||
| Redis 8.6.3 | `https://redis.io/download/` | 已有 source in `services/src/redis-7.4.3.tar.gz` |
|
||||
| MongoDB 8.2.7 | `https://github.com/mongodb/mongo` | Momentry cache, data 需 migrate |
|
||||
| ffmpeg 8.1.1 | `https://ffmpeg.org/releases/ffmpeg-8.1.1.tar.xz` | 已有 source in `services/src/ffmpeg-7.1.1.tar.xz` |
|
||||
|
||||
### Phase 3 — 輔助工具
|
||||
|
||||
| Service | Source |
|
||||
|---------|--------|
|
||||
| ollama 0.23.1 | `https://github.com/ollama/ollama` |
|
||||
| yt-dlp | `https://github.com/yt-dlp/yt-dlp` |
|
||||
| tesseract 5.5.2 | `https://github.com/tesseract-ocr/tesseract` |
|
||||
| whisper-cpp 1.8.4 | `https://github.com/ggerganov/whisper.cpp` |
|
||||
| protobuf | `https://github.com/protocolbuffers/protobuf` |
|
||||
|
||||
---
|
||||
|
||||
## Source 歸檔對照
|
||||
|
||||
| Source Archive | Status | Path |
|
||||
|---------------|--------|------|
|
||||
| `redis-7.4.3.tar.gz` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `ffmpeg-7.1.1.tar.xz` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `cmake-4.2.0-macos-universal.tar.gz` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `sftpgo-main.tar.gz` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `postgresql-18.3.tar.gz` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `llama.cpp/` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `go/` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `pyenv/` | ✅ | `release/system/v1.0/services/src/` |
|
||||
| `php-*.tar.gz` | ❌ 需下載 | `release/system/v1.0/services/src/` |
|
||||
| `mariadb-*.tar.gz` | ❌ 需下載 | `release/system/v1.0/services/src/` |
|
||||
| `node-*.tar.gz` | ❌ 需下載 | `release/system/v1.0/services/src/` |
|
||||
| `mongodb-*.tar.gz` | ❌ 需下載 | `release/system/v1.0/services/src/` |
|
||||
| `ollama-*.tar.gz` | ❌ 需下載 | `release/system/v1.0/services/src/` |
|
||||
|
||||
---
|
||||
|
||||
## SHA256 Checksum 填空
|
||||
|
||||
已知的 SHA256(待補其餘):
|
||||
|
||||
```yaml
|
||||
php: 173fd1ca36f3dd4952f5442572e06a14b7c005751ae15e7e42161606e931645c
|
||||
mariadb (mariadbd): 38cb48f0be673d4136c43a89c1aca5b314d30042dd09537d93b7995f52f90206
|
||||
node: fba87e4402c55ea4fc7ca9b9838790c32534e3e77c9c7834c37073752d070678
|
||||
sftpgo (source): 6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a
|
||||
sftpgo (binary): 9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Brew Leaves(user-installed only)
|
||||
|
||||
```
|
||||
cmake, e2fsprogs, ffmpeg, ffmpeg-full, go,
|
||||
mariadb, mongodb/brew/mongodb-community, ollama,
|
||||
ossp-uuid, pgvector, php, pkgconf, protobuf,
|
||||
python@3.11, redis, yt-dlp, zlib
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 執行步驟(待有時間時)
|
||||
|
||||
```bash
|
||||
# 1. 下載 source
|
||||
cd /tmp
|
||||
curl -O https://www.php.net/distributions/php-8.5.5.tar.gz
|
||||
curl -LO https://github.com/MariaDB/server/archive/mariadb-12.2.2.tar.gz
|
||||
|
||||
# 2. Archive + SHA256
|
||||
tar czf release/system/v1.0/services/src/php-8.5.5.tar.gz php-8.5.5/
|
||||
shasum -a 256 release/system/v1.0/services/src/php-8.5.5.tar.gz
|
||||
|
||||
# 3. Build PHP
|
||||
tar xzf php-8.5.5.tar.gz
|
||||
cd php-8.5.5
|
||||
./configure --prefix=/Users/accusys/bin/php --with-fpm-user=accusys --with-fpm-group=staff
|
||||
make -j$(sysctl -n hw.ncpu)
|
||||
make install
|
||||
|
||||
# 4. Update plist
|
||||
sed -i '' 's|/opt/homebrew/bin/php|/Users/accusys/bin/php/bin/php|g' momentry_runtime/plist/com.momentry.php.plist
|
||||
|
||||
# 5. Record in dev.resources
|
||||
# INSERT INTO dev.resources ...
|
||||
```
|
||||
@@ -0,0 +1,371 @@
|
||||
# 交付程序:M4_workspace → M5 → Public Release
|
||||
|
||||
**Date**: 2026-05-13
|
||||
**Version**: 1.1
|
||||
|
||||
---
|
||||
|
||||
## 流程總覽
|
||||
|
||||
```
|
||||
M4 回報問題 → M5 修復 → M5 自驗 → Release package → Deploy → M4 驗證 → Public Release
|
||||
↑ ↓
|
||||
└── 失敗回退 ──────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1:M4 回報(M4_workspace/)
|
||||
|
||||
---
|
||||
|
||||
## Phase 1:M4 回報(M4_workspace/)
|
||||
|
||||
M4 將問題寫入 `docs_v1.0/M4_workspace/YYYY-MM-DD_<topic>.md`,格式:
|
||||
|
||||
```markdown
|
||||
# {問題標題}
|
||||
**Date**: YYYY-MM-DD
|
||||
**From**: M4
|
||||
**To**: M5
|
||||
|
||||
---
|
||||
|
||||
## Bug / 建議
|
||||
(問題描述 + 影響分析)
|
||||
|
||||
## Fix 建議
|
||||
(選擇性:程式碼範例或解決方向)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2:M5 處理
|
||||
|
||||
| 步驟 | 動作 | 檢查點 |
|
||||
|------|------|--------|
|
||||
| 2.1 | 讀取 M4 報告,理解問題 | 必要時回覆確認 |
|
||||
| 2.2 | 實作修復 | `cargo check` / `cargo test` 通過 |
|
||||
| 2.3 | 更新 Registry(如影響座標、detector) | 相關 `.md` 同步更新 |
|
||||
| 2.4 | `git commit` | commit message 含 M4 issue 參考 |
|
||||
| 2.5 | 回覆 `M4_workspace/YYYY-MM-DD_<topic>_response.md` | 說明修復方式 + commit hash |
|
||||
|
||||
### 修復分類與回應形式
|
||||
|
||||
| 修復類型 | 回應文件 | 同步項目 |
|
||||
|----------|---------|---------|
|
||||
| **Bug fix**(座標、script 錯誤) | `*_response.md` + `git commit` | 無需額外 |
|
||||
| **模型替換**(YOLO v5→v8 等) | `*_response.md` + Registry 更新 | `DETECTOR_REGISTRY.md` |
|
||||
| **架構變更**(新 module、pipeline 重排) | `*_response.md` + Registry 更新 | `SPATIAL_COORDINATE_REGISTRY.md` |
|
||||
| **新功能**(heuristic_scene 等) | `*_response.md` + 新文件 | 新 `REFERENCE/*.md` |
|
||||
| **新測試包** | `*_test_report.md` | 上傳至 `release/files/` |
|
||||
|
||||
### 回應文件規範
|
||||
|
||||
```markdown
|
||||
# {主題} — 回覆
|
||||
|
||||
**Date**: YYYY-MM-DD
|
||||
**From**: M5
|
||||
**To**: M4
|
||||
**Ref**: `YYYY-MM-DD_source_file.md`
|
||||
|
||||
---
|
||||
|
||||
## 修正
|
||||
|
||||
| # | 項目 | 狀態 | Commit |
|
||||
|---|------|:--:|--------|
|
||||
| 1 | {問題 A 修復} | ✅ | `abc1234` |
|
||||
| 2 | {功能 B 新增} | ✅ | `abc1234` |
|
||||
|
||||
## 檔案變更
|
||||
|
||||
| 檔案 | 說明 |
|
||||
|------|------|
|
||||
| `path/to/file` | 改動摘要 |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2.5:M5 內部自驗
|
||||
|
||||
Release 前先自驗,降低 M4 測試失敗的機率。
|
||||
|
||||
### 自驗清單
|
||||
|
||||
| # | 項目 | 方法 |
|
||||
|---|------|------|
|
||||
| 2.5.1 | Rust build | `cargo build` / `cargo test` |
|
||||
| 2.5.2 | API 測試 | `bash api_test.sh` → 全 passed |
|
||||
| 2.5.3 | 場景驗證 | 抽 1 個 CUT segment,用 MaskFormer 或 PaliGemma 確認 scene type |
|
||||
| 2.5.4 | SQLite 驗證 | `python3 export_sqlite.py {uuid}` → vec0 tables 正確 |
|
||||
| 2.5.5 | Identity 驗證 | TMDB 演員 + auto PERSON count 合理 |
|
||||
| 2.5.6 | TKG 驗證 | edges > 0, nodes > 0 |
|
||||
| 2.5.7 | file_info.json | `momentry_version` + `momentry_build` 正確 |
|
||||
|
||||
### 自驗不通過
|
||||
|
||||
```
|
||||
❌ cargo build → 修復編譯錯誤 → 重新 commit
|
||||
❌ api_test → 修復 → 重新 commit
|
||||
❌ 場景驗證 → 確認是 bug 還是 flicker → 必要時開 issue
|
||||
```
|
||||
|
||||
### 回應文件規範
|
||||
|
||||
```markdown
|
||||
# {主題} — 回覆
|
||||
|
||||
**Date**: YYYY-MM-DD
|
||||
**From**: M5
|
||||
**To**: M4
|
||||
**Ref**: `YYYY-MM-DD_source_file.md`
|
||||
|
||||
---
|
||||
|
||||
## 修正
|
||||
|
||||
| # | 項目 | 狀態 | Commit |
|
||||
|---|------|:--:|--------|
|
||||
| 1 | {問題 A 修復} | ✅ | `abc1234` |
|
||||
| 2 | {功能 B 新增} | ✅ | `abc1234` |
|
||||
|
||||
## 檔案變更
|
||||
|
||||
| 檔案 | 說明 |
|
||||
|------|------|
|
||||
| `path/to/file` | 改動摘要 |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3:Release Package
|
||||
|
||||
### 時機
|
||||
|
||||
| 條件 | 動作 |
|
||||
|------|------|
|
||||
| 重大修復完成 | 產生新 package |
|
||||
| M4 要求測試 | 產生 package,附 `*_test_report.md` |
|
||||
| 版本里程碑 | 正式 release(含 version bump) |
|
||||
|
||||
### 產生流程
|
||||
|
||||
```bash
|
||||
# 1. 確認所有變更已 commit
|
||||
git log --oneline -5
|
||||
|
||||
# 2. Build release binary
|
||||
cargo build --bin release
|
||||
|
||||
# 3. 產生 package(含 sql/ 目錄、vec0.dylib、deploy.sh、verify.sh)
|
||||
cargo run --bin release -- package {file_uuid}
|
||||
|
||||
# 4. 檢查 output
|
||||
ls -la release/files/{uuid}_v{timestamp}.tar.gz
|
||||
|
||||
# 5. 驗證 package 內容
|
||||
tar tzf release/files/{uuid}_v{timestamp}.tar.gz
|
||||
```
|
||||
|
||||
### Package 內容規範
|
||||
|
||||
```
|
||||
{file_uuid}/
|
||||
├── file_info.json (含 momentry_version + momentry_build)
|
||||
├── data.sql (→ 指引 sql/)
|
||||
├── deploy.sh (→ 9 步驟)
|
||||
├── verify.sh
|
||||
├── vec0.dylib (SQLite vector extension)
|
||||
├── sql/
|
||||
│ ├── dev_videos.sql
|
||||
│ ├── dev_chunk.sql
|
||||
│ ├── dev_chunk_vectors.sql (768D)
|
||||
│ ├── dev_face_detections.sql (512D)
|
||||
│ ├── dev_identities.sql
|
||||
│ ├── dev_identity_bindings.sql
|
||||
│ ├── dev_tkg_nodes.sql
|
||||
│ └── dev_tkg_edges.sql
|
||||
├── {uuid}.face.json (landmark 已修復)
|
||||
├── {uuid}.yolo.json
|
||||
├── {uuid}.asr.json
|
||||
├── {uuid}.asrx.json
|
||||
├── {uuid}.cut.json
|
||||
├── {uuid}.sqlite (含 vec0 向量表)
|
||||
└── *.mp4 / *.mov
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 4:Deploy
|
||||
|
||||
### 前置檢查
|
||||
|
||||
```bash
|
||||
# 1. 確認伺服器正常
|
||||
curl -s http://localhost:3003/health
|
||||
# 預期: {"status":"ok","version":"1.0.0","build_git_hash":"d34bcae",...}
|
||||
|
||||
# 2. 確認資料庫可連線
|
||||
/Users/accusys/pgsql/18.3/bin/psql -U accusys -d momentry -c "SELECT version()"
|
||||
|
||||
# 3. 解包
|
||||
tar xzf release/files/{uuid}_v{timestamp}.tar.gz -C /tmp/deploy/
|
||||
cd /tmp/deploy/{file_uuid}
|
||||
```
|
||||
|
||||
### 執行 deploy
|
||||
|
||||
```bash
|
||||
bash deploy.sh
|
||||
```
|
||||
|
||||
### 預期輸出
|
||||
|
||||
```
|
||||
[0/9] Checking system version and build... ✅ Server v1.0.0 matches
|
||||
[1/9] Verifying package... ✅
|
||||
[2/9] Pre-cleaning existing identities... ✅
|
||||
[3/9] Importing DB data...
|
||||
Importing dev_videos.sql... COPY 1
|
||||
Importing dev_chunk.sql... COPY 2407
|
||||
Importing dev_chunk_vectors.sql... COPY 2407
|
||||
Importing dev_face_detections.sql... COPY 70691
|
||||
Importing dev_identities.sql... COPY 441
|
||||
Importing dev_identity_bindings.sql... COPY 18635
|
||||
Importing dev_tkg_nodes.sql... COPY 6457
|
||||
Importing dev_tkg_edges.sql... COPY 21028
|
||||
[4/9] Copy video... ✅
|
||||
[5/9] Setting status=completed ✅
|
||||
[6/9] Copying output files... ✅ N files
|
||||
[7/9] Installing vec0.dylib... ✅ /tmp/vec0.dylib
|
||||
[8/9] Verify deployment...
|
||||
Chunks: 2407
|
||||
Faces: 70691
|
||||
Identities: 417
|
||||
TKG nodes: 6457
|
||||
TKG edges: 21028 ✅
|
||||
```
|
||||
|
||||
### Deploy 失敗處理
|
||||
|
||||
| 失敗點 | 原因 | 處理方式 |
|
||||
|--------|------|---------|
|
||||
| Step 0: 版本檢查 | server version ≠ package version | 取得 matching upgrade package |
|
||||
| Step 3: 任 table 匯入失敗 | FK constraint、duplicate key | 檢查該 table 的 sql/*.sql 內容,修復後再跑 |
|
||||
| Step 4: 影片複製 | 磁碟空間不足 | `df -h` 確認,清理後重跑 |
|
||||
| Step 8: 驗證 row count 不符 | import 不完整 | 逐 table 比對 COPY count vs actual count |
|
||||
|
||||
> 每個 table 獨立 import(各含 auto-commit),單一 table 失敗不會 rollback 其他 table。
|
||||
> 重新執行 `bash deploy.sh` 會先清掉該 file 的 identity 和資料後重新 import。
|
||||
|
||||
### 驗證後確認
|
||||
|
||||
```bash
|
||||
# 確認所有 table row count 正確
|
||||
for tbl in videos chunk chunk_vectors face_detections identities identity_bindings tkg_nodes tkg_edges; do
|
||||
echo "$tbl: $(psql -U accusys -d momentry -t -A -c "SELECT COUNT(*) FROM dev.$tbl WHERE file_uuid='$UUID'" 2>/dev/null)"
|
||||
done
|
||||
```
|
||||
|
||||
### Production (port 3002)
|
||||
|
||||
Production deploy 步驟與 dev 相同,但需注意:
|
||||
|
||||
| 項目 | Dev (3003) | Production (3002) |
|
||||
|------|:----------:|:-----------------:|
|
||||
| Schema | `dev.*` | `public.*`(或 `dev.*` 若已 migration) |
|
||||
| Port | 3003 | 3002 |
|
||||
| Deploy URL | `localhost:3003/health` | `localhost:3002/health` |
|
||||
| 版本檢查 | 可接受 `unknown` build | 強制匹配 version |
|
||||
|
||||
```bash
|
||||
# Production deploy 需明確設定 Server URL
|
||||
SERVER_URL=http://localhost:3002 bash deploy.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 5:Public Release
|
||||
|
||||
### 版本策略
|
||||
|
||||
| Bump | 幅度 | 適用時機 | 範例 |
|
||||
|------|:----:|---------|------|
|
||||
| **patch** | 0.0.x | Bug fix only,無 API 變更 | 1.0.0 → 1.0.1 |
|
||||
| **minor** | 0.x.0 | 新功能、模型替換、API 向後相容 | 1.0.0 → 1.1.0 |
|
||||
| **major** | x.0.0 | Breaking change、schema 遷移 | 1.0.0 → 2.0.0 |
|
||||
|
||||
> 目前版本 `1.0.0`,所有已完成的變更(YOLO 替換、座標修復、pipeline 重排)尚未 bump。
|
||||
|
||||
### 條件
|
||||
|
||||
- [ ] M4 驗證通過(deploy test 全部綠色)
|
||||
- [ ] 所有 open issues 已回應或關閉
|
||||
- [ ] API 文件與實際行為一致(或已開 doc issue 追蹤 M4 更新)
|
||||
- [ ] 版本號已 bump(`Cargo.toml` → `build.rs` → `BUILD_VERSION`)
|
||||
- [ ] package 已上傳到 `release/files/`
|
||||
- [ ] 變更已 `git commit` + `git push`
|
||||
|
||||
### M4 驗證失敗的回退
|
||||
|
||||
```
|
||||
M4 deploy test ❌
|
||||
↓
|
||||
M5 分析失敗原因
|
||||
├── Bug → 回到 Phase 2 修復 → 重新 commit(不 bump 版本)
|
||||
├── Script 錯誤 → 修復 deploy.sh → 重新 commit
|
||||
└── Package 內容缺漏 → 重新 `cargo run --bin release -- package`
|
||||
↓
|
||||
M5 重新通知 M4(新 *response.md + git commit)
|
||||
M4 重新測試
|
||||
```
|
||||
|
||||
### Release Info 產出
|
||||
|
||||
### Release Info 產出
|
||||
|
||||
```markdown
|
||||
# Release v{version} — {YYYY-MM-DD}
|
||||
|
||||
## Changes
|
||||
- {change 1}
|
||||
- {change 2}
|
||||
|
||||
## Files
|
||||
- release/files/{uuid}_v{timestamp}.tar.gz
|
||||
|
||||
## Verification
|
||||
- Deploy test: ✅ (M4)
|
||||
- API test: 18/18 ✅
|
||||
- Chunks: {n}
|
||||
- Faces: {n}
|
||||
- Identities: {n}
|
||||
- TKG nodes/edges: {n}/{n}
|
||||
```
|
||||
|
||||
### 保存
|
||||
|
||||
```bash
|
||||
# 備份 release 資訊
|
||||
echo "Release: v{VERSION}" > $RELEASE_DIR/RELEASE_INFO.txt
|
||||
echo "Date: $(date)" >> $RELEASE_DIR/RELEASE_INFO.txt
|
||||
echo "Git: $(git rev-parse HEAD)" >> $RELEASE_DIR/RELEASE_INFO.txt
|
||||
echo "Package: {uuid}_v{timestamp}.tar.gz" >> $RELEASE_DIR/RELEASE_INFO.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 文件對照
|
||||
|
||||
| 文件 | 用途 |
|
||||
|------|------|
|
||||
| `M4_workspace/*.md` | M4 問題回報 |
|
||||
| `M4_workspace/*_response.md` | M5 修復回覆 |
|
||||
| `M4_workspace/*_test_report.md` | 測試包報表 |
|
||||
| `REFERENCE/DETECTOR_REGISTRY.md` | Detector 規格 |
|
||||
| `REFERENCE/DETECTOR_SELECTION_SOP.md` | 選型標準 |
|
||||
| `REFERENCE/SPATIAL_COORDINATE_REGISTRY.md` | 座標系統 |
|
||||
| `REFERENCE/IDENTITY_LIFECYCLE.md` | Identity 生命週期 |
|
||||
| `M4_HANDOVER/` | M4 交付目錄 |
|
||||
@@ -0,0 +1,559 @@
|
||||
---
|
||||
document_type: "reference_doc"
|
||||
service: "MOMENTRY_CORE"
|
||||
title: "Momentry Core 開發日誌"
|
||||
date: "2026-03-18"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "開發日誌"
|
||||
- "momentry"
|
||||
- "core"
|
||||
ai_query_hints:
|
||||
- "查詢 Momentry Core 開發日誌 的內容"
|
||||
- "Momentry Core 開發日誌 的主要目的是什麼?"
|
||||
- "如何操作或實施 Momentry Core 開發日誌?"
|
||||
---
|
||||
|
||||
# Momentry Core 開發日誌
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-18 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-18 | 創建文件 | Warren | OpenCode / MiniMax M2.5 |
|
||||
|
||||
---
|
||||
|
||||
> **文檔維護開始**:2026-03-18
|
||||
> **⚠️ 補充說明**:事後補記(2026-03-18 以前),僅供參考。未來紀錄將即時記錄,參考價值較高。
|
||||
|
||||
---
|
||||
|
||||
## 開發工具
|
||||
|
||||
### Coding LLM 模型
|
||||
|
||||
| 階段 | 工具 | 模型 | ID | 說明 |
|
||||
|------|------|------|-----|------|
|
||||
| **初期** | Claude CLI | - | - | 初始專案架構建立 |
|
||||
| **中期** | OpenCode | big-pickle | opencode/big-pickle | 主要開發協作者 |
|
||||
|
||||
**切換記錄**:
|
||||
- 初期使用 Claude CLI 建立專案基本架構
|
||||
- 中期切換至 OpenCode (big-pickle) 進行主要功能開發
|
||||
|
||||
---
|
||||
|
||||
## 2026-03-17
|
||||
|
||||
### ML 模型選用
|
||||
|
||||
| Processor | 模型 | 版本/大小 | 說明 |
|
||||
|----------|------|-----------|------|
|
||||
| **ASR** | WhisperX (faster-whisper) | base, int8 | 語音識別 + 對話分段 |
|
||||
| **CUT** | PySceneDetect | 0.6.7.1 | ContentDetector 場景檢測 |
|
||||
| **YOLO** | YOLOv8n | yolov8n.pt (6.2MB) | 物體檢測(nano 版本最快) |
|
||||
| **OCR** | EasyOCR | 1.7.2 | 文字識別 |
|
||||
| **Face** | OpenCV Haar Cascade | built-in | 人臉檢測(無需額外下載) |
|
||||
| **Pose** | YOLOv8n-Pose | yolov8n-pose.pt (6.5MB) | 姿態估計(nano 版本) |
|
||||
|
||||
**模型下載**:
|
||||
- YOLOv8n: `yolov8n.pt` (6.2MB)
|
||||
- YOLOv8n-Pose: `yolov8n-pose.pt` (6.5MB)
|
||||
|
||||
**Python 依賴**:
|
||||
```
|
||||
torch==2.8.0
|
||||
whisperx==3.8.2
|
||||
ultralytics==8.4.23
|
||||
scenedetect==0.6.7.1
|
||||
easyocr==1.7.2
|
||||
opencv-python==4.13.0.92
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### ASR 實作完成
|
||||
- 完成 Python ML processor scripts(使用本地模型)
|
||||
- `asrx_processor.py` - whisperx for speaker diarization
|
||||
- `cut_processor.py` - PySceneDetect for scene detection
|
||||
- `yolo_processor.py` - YOLOv8 for object detection
|
||||
- `ocr_processor.py` - EasyOCR for text recognition
|
||||
- `face_processor.py` - OpenCV Haar Cascade for face detection
|
||||
- `pose_processor.py` - YOLOv8 Pose for pose estimation
|
||||
|
||||
- 更新 `requirements.txt` with all dependencies
|
||||
- 安裝完成:torch 2.8.0, whisperx 3.8.2, ultralytics 8.4.23, scenedetect 0.6.7.1, easyocr 1.7.2, opencv-python 4.13.0.92
|
||||
- 下載模型:YOLOv8n.pt (6.2MB), YOLOv8n-Pose.pt (6.5MB)
|
||||
|
||||
### Async Streaming 實作
|
||||
- 更新 Rust processor modules 使用 async streaming 進行 real-time progress
|
||||
- `src/core/processor/asr.rs`
|
||||
- `src/core/processor/cut.rs`
|
||||
- `src/core/processor/yolo.rs`
|
||||
- `src/core/processor/ocr.rs`
|
||||
- `src/core/processor/face.rs`
|
||||
- `src/core/processor/pose.rs`
|
||||
|
||||
### 測試結果
|
||||
- 測試影片:BigBuckBunny_320x180.mp4
|
||||
- ASR: 4 segments
|
||||
- CUT: 134 scenes
|
||||
- YOLO: 14315 frames(每幀處理耗時)
|
||||
- OCR: 40 frames with text
|
||||
- Face: 44 frames with faces
|
||||
- Pose: Timeout
|
||||
|
||||
---
|
||||
|
||||
### Warning 清理
|
||||
修復 clippy warnings:
|
||||
- 移除未使用的 imports (HashMap in mongodb_db.rs, postgres_db.rs)
|
||||
- 新增 `#[allow(dead_code)]` 標註未使用變數
|
||||
- 新增 `Default` implementation for MongoDb, QdrantDb
|
||||
- 將 `probe` module 重新命名為 `ffprobe`
|
||||
- 新增 `player` feature in Cargo.toml
|
||||
- 修復 `format_in_format_args` 警告
|
||||
|
||||
---
|
||||
|
||||
### TUI Progress Window 實作
|
||||
建立新的 UI module:
|
||||
- 建立 `src/ui/mod.rs`
|
||||
- 建立 `src/ui/progress/mod.rs`
|
||||
|
||||
實作功能:
|
||||
- ProcessorProgress 結構(追蹤每個 processor 狀態)
|
||||
- ProgressState 結構(管理所有 processors)
|
||||
- ProgressUi 結構(ratatui TUI 渲染)
|
||||
- 整合到 `src/main.rs` 的 process 命令
|
||||
|
||||
TUI 顯示:
|
||||
```
|
||||
┌ Processing: BigBuckBunny_320x180.mp4 ────────────────────────────────────────┐
|
||||
│ ASR [████████████] 100% (4 segs) │
|
||||
│ CUT [████████████] 100% (134 scenes) │
|
||||
│ ASRX [████████████] 100% (0 segs) │
|
||||
│ YOLO [██░░░░░░░░░░░] 30% (4200/14315) ETA 2:30 │
|
||||
│ OCR [---------] 0% │
|
||||
│ Face [---------] 0% │
|
||||
│ Pose [---------] 0% │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 輸出位置討論
|
||||
討論 stdout vs stderr vs TUI 的輸出配置:
|
||||
- 最終結果 → stdout
|
||||
- Python progress → 需改用 Redis Pub/Sub
|
||||
- TUI Progress → stderr (ratatui)
|
||||
|
||||
---
|
||||
|
||||
## 2026-03-18
|
||||
|
||||
### Redis Message Bus 設計
|
||||
討論使用 Redis 作為消息總線,分離 Python 輸出與 Rust TUI 顯示。
|
||||
|
||||
設計重點:
|
||||
1. 頻道命名:`momentry:progress:{uuid}`
|
||||
2. 本地 Redis:`localhost:6379`
|
||||
3. 失敗策略:完全失效(因 stdout 問題未解決)
|
||||
|
||||
### UUID 使用時機分析
|
||||
分析 Redis Key 上使用 UUID 的時機:
|
||||
|
||||
**全局 Keys(無 UUID)**:
|
||||
- health, stats, jobs 管理
|
||||
|
||||
**Per-Video Keys(UUID 必要)**:
|
||||
- job:{uuid}, progress:{uuid}, metrics:{uuid}
|
||||
|
||||
**Per-Processor Keys(UUID + Processor 必要)**:
|
||||
- job:{uuid}:processor:{name}
|
||||
|
||||
### 備份系統整合
|
||||
參考 `docs_v1.0/IMPLEMENTATION/SERVICE_ADDITION_GUIDE.md` 設計規範,規劃 OutputDir 模組:
|
||||
|
||||
1. **環境變數**:
|
||||
- `MOMENTRY_OUTPUT_DIR` - JSON 輸出目錄
|
||||
- `MOMENTRY_BACKUP_DIR` - 備份目錄(預設:`/Users/accusys/momentry/backup/momentry`)
|
||||
- `MOMENTRY_BACKUP_ENABLED` - 啟用備份
|
||||
|
||||
2. **命名格式**:
|
||||
- 備份格式:`momentry_data_{YYYYMMDD}_{HHMMSS}_{uuid}.{ext}`
|
||||
- 校驗和:`{filename}.sha256`
|
||||
|
||||
3. **CLI 命令**:
|
||||
- `cargo run -- backup list` - 列出備份
|
||||
- `cargo run -- backup cleanup` - 清理舊備份
|
||||
- `cargo run -- backup verify` - 驗證備份
|
||||
|
||||
---
|
||||
|
||||
### 監控系統整合
|
||||
討論將 momentry_core 納入監控系統:
|
||||
|
||||
1. **Layer 2: Service 監控**
|
||||
- 新增 momentry_core CLI 檢查
|
||||
|
||||
2. **Layer 7: Backup 監控**
|
||||
- 新增 momentry 備份配置
|
||||
|
||||
3. **Redis 監控**
|
||||
- 健康檢查
|
||||
- Job 狀態監控
|
||||
- 即時進度監控
|
||||
|
||||
---
|
||||
|
||||
## 實作完成項目
|
||||
|
||||
### 程式碼變更
|
||||
|
||||
| 日期 | 檔案 | 變更 |
|
||||
|------|------|------|
|
||||
| 2026-03-17 | `src/core/processor/*.rs` | Async streaming 更新 |
|
||||
| 2026-03-17 | `src/ui/mod.rs` | 新增 UI module |
|
||||
| 2026-03-17 | `src/ui/progress/mod.rs` | 新增 Progress TUI |
|
||||
| 2026-03-17 | `src/main.rs` | 整合 Progress UI |
|
||||
| 2026-03-18 | `src/core/storage/output_dir.rs` | 新增 OutputDir 模組 |
|
||||
| 2026-03-18 | `src/core/storage/mod.rs` | 新增 output_dir export |
|
||||
| 2026-03-18 | `src/core/db/redis_client.rs` | 新增 Redis 客戶端(Hash + Pub/Sub) |
|
||||
| 2026-03-18 | `src/core/db/mod.rs` | 新增 redis_client export |
|
||||
|
||||
### 新增檔案
|
||||
|
||||
| 日期 | 檔案 | 說明 |
|
||||
|------|------|------|
|
||||
| 2026-03-18 | `docs_v1.0/REFERENCE/MOMENTRY_CORE_REDIS_KEYS.md` | Redis Key 設計規範 |
|
||||
| 2026-03-18 | `docs_v1.0/OPERATIONS/MOMENTRY_CORE_MONITORING.md` | 監控規範(暫定) |
|
||||
| 2026-03-18 | `scripts/redis_publisher.py` | Redis 訊息發布模組 |
|
||||
|
||||
### 更新檔案
|
||||
|
||||
| 日期 | 檔案 | 說明 |
|
||||
|------|------|------|
|
||||
| 2026-03-17 | `Cargo.toml` | 新增 player feature |
|
||||
| 2026-03-17 | `src/lib.rs` | 新增 ui module exports |
|
||||
| 2026-03-18 | `docs_v1.0/REFERENCE/PENDING_ISSUES.md` | 新增問題 #2, #3 |
|
||||
| 2026-03-18 | `src/core/storage/output_dir.rs` | 預設改為 `./output` |
|
||||
| 2026-03-18 | `scripts/yolo_processor.py` | 新增 --uuid 參數 + Redis |
|
||||
| 2026-03-18 | `scripts/cut_processor.py` | 新增 Redis |
|
||||
| 2026-03-18 | `scripts/ocr_processor.py` | 新增 Redis |
|
||||
| 2026-03-18 | `scripts/face_processor.py` | 新增 --uuid 參數 + Redis |
|
||||
| 2026-03-18 | `scripts/pose_processor.py` | 新增 --uuid 參數 + Redis |
|
||||
| 2026-03-18 | `scripts/asr_processor.py` | 新增 --uuid 參數 + Redis |
|
||||
| 2026-03-18 | `scripts/asrx_processor.py` | 新增 --uuid 參數 + Redis |
|
||||
| 2026-03-18 | `requirements.txt` | 新增 redis>=5.0.0 |
|
||||
| 2026-03-18 | `src/core/processor/yolo.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/cut.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/ocr.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/face.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/pose.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/asr.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/core/processor/asrx.rs` | 新增 uuid 參數 |
|
||||
| 2026-03-18 | `src/main.rs` | 更新所有 processor 調用傳入 uuid |
|
||||
| 2026-03-18 | `Cargo.toml` | 新增 futures-util 依賴 |
|
||||
| 2026-03-18 | `src/core/db/redis_client.rs` | 新增 subscribe_and_callback 方法,密碼認證 |
|
||||
| 2026-03-18 | `src/ui/progress/mod.rs` | 新增 update_from_redis 方法 |
|
||||
| 2026-03-18 | `scripts/redis_publisher.py` | 新增密碼認證支援 |
|
||||
| 2026-03-18 | 測試 | Redis Pub/Sub 成功運作 |
|
||||
|
||||
---
|
||||
|
||||
## 待解決問題
|
||||
|
||||
### 問題 #1: sqlx async INSERT 不會實際寫入數據庫
|
||||
- 狀態:待解決
|
||||
- 影響:`store_vector` 函數,PVector 存儲
|
||||
|
||||
### 問題 #2: TUI 與 stdout 輸出混合
|
||||
- 狀態:已解決
|
||||
- 解決方案:使用 Redis Message Bus
|
||||
- 進度:
|
||||
- ✅ Redis 客戶端 (`src/core/db/redis_client.rs`)
|
||||
- ✅ Python redis_publisher.py
|
||||
- ✅ 所有 Python processors 更新完成
|
||||
- ✅ 所有 Rust processor 函數更新完成
|
||||
- ✅ main.rs 調用更新完成
|
||||
- ✅ Rust TUI Redis 訂閱已完成
|
||||
|
||||
### 問題 #3: Redis Message Bus 尚未實作
|
||||
- 狀態:已解決
|
||||
- 詳細設計:參考 `docs_v1.0/REFERENCE/MOMENTRY_CORE_REDIS_KEYS.md`
|
||||
- 進度:Python 端 + Rust 端均已完成
|
||||
|
||||
---
|
||||
|
||||
## 環境變數
|
||||
|
||||
```bash
|
||||
# 輸出目錄
|
||||
MOMENTRY_OUTPUT_DIR=./output # 預設
|
||||
|
||||
# 備份
|
||||
MOMENTRY_BACKUP_ENABLED=false # 預設
|
||||
MOMENTRY_BACKUP_DIR=/Users/accusys/momentry/backup/momentry
|
||||
|
||||
# Redis(未來實作)
|
||||
REDIS_URL=redis://localhost:6379
|
||||
REDIS_PASSWORD=accusys
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 數據庫
|
||||
|
||||
- PostgreSQL: `postgres://accusys@localhost:5432/momentry`
|
||||
- Redis: `localhost:6379`(待實作)
|
||||
- Qdrant: `localhost:6333`
|
||||
|
||||
---
|
||||
|
||||
## 指令範例
|
||||
|
||||
```bash
|
||||
# 註冊視頻
|
||||
cargo run -- register /path/to/video.mp4
|
||||
|
||||
# 處理視頻
|
||||
cargo run -- process <uuid>
|
||||
|
||||
# 列出備份
|
||||
cargo run -- backup list
|
||||
|
||||
# 清理備份
|
||||
cargo run -- backup cleanup
|
||||
|
||||
# 驗證備份
|
||||
cargo run -- backup verify
|
||||
|
||||
# 查看狀態
|
||||
cargo run -- status
|
||||
|
||||
# API Server
|
||||
cargo run -- server --host 0.0.0.0 --port 3000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2026-03-18 (進行中)
|
||||
|
||||
### Redis Message Bus 實作
|
||||
|
||||
**問題**:TUI 與 Python stdout 輸出混合,導致 TUI 顯示混亂
|
||||
|
||||
**解決方案**:使用 Redis Pub/Sub 作為訊息匯流排
|
||||
|
||||
**實作內容**:
|
||||
|
||||
| 元件 | 檔案 | 狀態 |
|
||||
|------|------|------|
|
||||
| Redis 客戶端 | `src/core/db/redis_client.rs` | ✅ |
|
||||
| Progress 訂閱 | `src/main.rs` | ✅ |
|
||||
| UI 更新 | `src/ui/progress/mod.rs` | ✅ |
|
||||
| Python Publisher | `scripts/redis_publisher.py` | ✅ |
|
||||
| Python Processors | 7 個 `scripts/*_processor.py` | ✅ |
|
||||
| Rust 函數 | `src/core/processor/*.rs` | ✅ |
|
||||
|
||||
**流程**:
|
||||
```
|
||||
Python Processor ──(Redis Pub)──> Redis ──(Subscribe)──> Rust TUI
|
||||
```
|
||||
|
||||
**測試結果**:
|
||||
- Redis 連線 ✅
|
||||
- 密碼認證 ✅
|
||||
- 即時進度發布 ✅
|
||||
- TUI 即時更新 ✅
|
||||
|
||||
**新增依賴**:
|
||||
- `futures-util = "0.3"` (Cargo.toml)
|
||||
- `redis >= 5.0.0` (requirements.txt)
|
||||
|
||||
---
|
||||
|
||||
## 2026-03-18 (HTTP API)
|
||||
|
||||
### HTTP API 實作
|
||||
|
||||
**問題**:TUI 運作正常但使用者偏好 HTTP API 來查詢進度
|
||||
|
||||
**解決方案**:建立 HTTP 端點 + Redis Hash 儲存
|
||||
|
||||
**實作內容**:
|
||||
|
||||
| 元件 | 檔案 | 變更 |
|
||||
|------|------|------|
|
||||
| HTTP 端點 | `src/api/server.rs` | 新增 `/api/v1/progress/:uuid` |
|
||||
| Redis Hash 查詢 | `src/core/db/redis_client.rs` | 新增 `get_processor_status` 方法 |
|
||||
| Progress 儲存 | `src/main.rs` | 新增 Redis HSET 儲存進度 |
|
||||
|
||||
**API 端點**:
|
||||
```
|
||||
GET /api/v1/progress/:uuid
|
||||
|
||||
Response:
|
||||
{
|
||||
"uuid": "5dea6618a606e7c7",
|
||||
"processors": [
|
||||
{"name": "asr", "status": "complete", "current": 0, "total": 0, "message": "7 segments"},
|
||||
{"name": "cut", "status": "complete", "current": 134, "total": 134, "message": "134 scenes"},
|
||||
{"name": "yolo", "status": "complete", "current": 14300, "total": 14315, "message": "..."},
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**流程**:
|
||||
```
|
||||
Python Processor ──(Redis Pub)──> Redis ──(Subscribe)──> Rust TUI
|
||||
└──(HSET)──> Redis Hash
|
||||
│
|
||||
HTTP Client ──(GET /progress/:uuid)──> Rust API ─(HGETALL)──> Redis Hash
|
||||
```
|
||||
|
||||
**測試結果**:
|
||||
- ✅ 編譯成功
|
||||
- ✅ API 伺服器啟動 (port 3002)
|
||||
- ✅ 即時進度查詢
|
||||
- ✅ 完整流程測試 (BigBuckBunny_320x180.mp4)
|
||||
|
||||
**除錯記錄**:
|
||||
1. 語法錯誤:main.rs 有重複程式碼區塊 (lines 297-322),已移除
|
||||
2. DB 連線池:從 5 增加到 10 個連線
|
||||
3. PostgreSQL 狀態:處理 shutdown 狀態,殺掉 stale 連線
|
||||
|
||||
**新增變更**:
|
||||
- `src/api/server.rs` - 新增進度端點
|
||||
- `src/core/db/redis_client.rs` - 新增 `get_processor_status` 方法
|
||||
- `src/core/db/postgres_db.rs` - 連線池 5→10
|
||||
- `src/main.rs` - Redis Hash 儲存 + 語法修復
|
||||
|
||||
**使用方式**:
|
||||
```bash
|
||||
# 啟動 API 伺服器
|
||||
cargo run --bin momentry -- server --host 127.0.0.1 --port 3002
|
||||
|
||||
# 註冊影片
|
||||
cargo run --bin momentry -- register ~/test_video/BigBuckBunny_320x180.mp4
|
||||
|
||||
# 處理影片
|
||||
cargo run --bin momentry -- process <uuid>
|
||||
|
||||
# 查詢進度
|
||||
curl http://127.0.0.1:3002/api/v1/progress/<uuid>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2026-03-18 (Dashboard)
|
||||
|
||||
### Web Dashboard 實作
|
||||
|
||||
**目標**:建立 Web 介面監控 momentry_core 處理進度
|
||||
|
||||
**技術選擇**:Static HTML + JavaScript (非 WASM)
|
||||
|
||||
**實作內容**:
|
||||
|
||||
| 元件 | 檔案 | 說明 |
|
||||
|------|------|------|
|
||||
| Dashboard | `momentry_dashboard/dist/index.html` | 靜態 HTML 頁面 |
|
||||
| API 代理 | Caddyfile port 3200 | 反向代理到 API server |
|
||||
|
||||
**功能**:
|
||||
- 影片列表顯示
|
||||
- 即時進度條 (每 5 秒自動刷新)
|
||||
- 搜尋功能
|
||||
- 處理器狀態 (ASR/CUT/YOLO/OCR/Face/Pose)
|
||||
|
||||
**訪問**:
|
||||
- Dashboard: http://localhost:3200
|
||||
- API: http://localhost:3200/api/v1/*
|
||||
|
||||
---
|
||||
|
||||
## 發生問題記錄
|
||||
|
||||
### HTTP API 問題
|
||||
|
||||
1. **語法錯誤** (main.rs)
|
||||
- 位置:lines 297-322
|
||||
- 原因:重複的程式碼區塊
|
||||
- 解決:移除重複區塊
|
||||
|
||||
2. **DB 連線池耗盡**
|
||||
- 原因:預設 5 個連線不足
|
||||
- 解決:增加到 10 個連線
|
||||
|
||||
3. **PostgreSQL shutdown 狀態**
|
||||
- 原因:共享記憶體未釋放
|
||||
- 解決:殺掉 stale 連線
|
||||
|
||||
### WASM Dashboard 問題
|
||||
|
||||
1. **Yew 版本問題**
|
||||
- 嘗試:yew 0.21 → 0.23
|
||||
- 問題:feature 名稱變更 (`web-sys` → `web_sys` → `csr`)
|
||||
- 解決:放棄 WASM,改用靜態 HTML
|
||||
|
||||
2. **編譯錯誤**
|
||||
- `wasm32-unknown-unknown` target 未安裝
|
||||
- 解決:`rustup target add wasm32-unknown-unknown`
|
||||
|
||||
3. **Yew 0.23 API 變更**
|
||||
- Properties 需要 PartialEq derive
|
||||
- 多處 API 語法變更
|
||||
- 放棄 WASM 方案
|
||||
|
||||
### Gitea Push 問題
|
||||
|
||||
1. **Remote URL 錯誤**
|
||||
- 原因:使用 localhost:3000 而非 gitea.momentry.ddns.net
|
||||
- 解決:建立新 repo `momentry_core_0_1`
|
||||
|
||||
2. **認證問題**
|
||||
- SSH key 未授權
|
||||
- 密碼認證成功推送
|
||||
|
||||
### Caddy 設定問題
|
||||
|
||||
1. **API 代理順序**
|
||||
- 問題:try_files 在 reverse_proxy 之前導致 API 回傳 HTML
|
||||
- 解決:使用 `handle` 區塊明確定義順序
|
||||
|
||||
```caddyfile
|
||||
:3200 {
|
||||
handle /api/* {
|
||||
reverse_proxy localhost:3002
|
||||
}
|
||||
handle {
|
||||
root * /Users/accusys/momentry_dashboard/dist
|
||||
try_files {path} /index.html
|
||||
file_server
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 未來工作
|
||||
|
||||
- [ ] 修復 WASM Dashboard (Yew 0.23 相容性)
|
||||
- [ ] 新增影片播放器整合
|
||||
- [ ] WebSocket 實時推送
|
||||
- [ ] 移動端響應式設計
|
||||
@@ -0,0 +1,538 @@
|
||||
---
|
||||
document_type: "reference_doc"
|
||||
service: "MOMENTRY_CORE"
|
||||
title: "Momentry JSON 輸出檔案規範"
|
||||
date: "2026-03-16"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "momentry"
|
||||
- "json"
|
||||
- "輸出檔案規範"
|
||||
ai_query_hints:
|
||||
- "查詢 Momentry JSON 輸出檔案規範 的內容"
|
||||
- "Momentry JSON 輸出檔案規範 的主要目的是什麼?"
|
||||
- "如何操作或實施 Momentry JSON 輸出檔案規範?"
|
||||
---
|
||||
|
||||
# Momentry JSON 輸出檔案規範
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-16 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-16 | 創建文件 | Warren | OpenCode / MiniMax M2.5 |
|
||||
|
||||
---
|
||||
|
||||
本文檔定義 Momentry Core 系統中所有 JSON 輸出檔案的結構、命名規範與儲存位置。
|
||||
|
||||
---
|
||||
|
||||
## 1. 輸出檔案總覽
|
||||
|
||||
### 1.1 檔案類型
|
||||
|
||||
| 類型 | 前綴 | 說明 | 狀態 |
|
||||
|------|------|------|------|
|
||||
| **Probe** | `{uuid}.probe.json` | 影片元數據 | ✅ 已實作 |
|
||||
| **ASR** | `{uuid}.asr.json` | 語音識別結果 | ✅ 已實作 |
|
||||
| **ASRx** | `{uuid}.asrx.json` | 說話者分離 | 🔜 規劃中 |
|
||||
| **OCR** | `{uuid}.ocr.json` | 文字辨識結果 | 🔜 規劃中 |
|
||||
| **YOLO** | `{uuid}.yolo.json` | 物件偵測結果 | 🔜 規劃中 |
|
||||
| **Face** | `{uuid}.face.json` | 人臉偵測結果 | 🔜 規劃中 |
|
||||
| **Pose** | `{uuid}.pose.json` | 姿態估計結果 | 🔜 規劃中 |
|
||||
| **Thumbnail** | `{uuid}/thumb_XXX.jpg` | 縮圖檔案 | ✅ 已實作 |
|
||||
|
||||
### 1.2 命名規範
|
||||
|
||||
```
|
||||
{UUID}.{類型}.json
|
||||
|
||||
範例:
|
||||
1636719dc31f78ac.probe.json - 影片探測結果
|
||||
1636719dc31f78ac.asr.json - 語音識別結果
|
||||
1636719dc31f78ac.ocr.json - 文字辨識結果
|
||||
```
|
||||
|
||||
- **UUID**: 16 字元,基於檔案路徑計算
|
||||
- **類型**: 小寫 snake_case
|
||||
- **副檔名**: `.json`
|
||||
|
||||
---
|
||||
|
||||
## 2. 輸出目錄結構
|
||||
|
||||
### 2.1 預設輸出位置
|
||||
|
||||
```
|
||||
momentry_core_0.1/
|
||||
├── {uuid}.probe.json # 影片探測
|
||||
├── {uuid}.asr.json # 語音識別
|
||||
├── {uuid}.asrx.json # 說話者分離
|
||||
├── {uuid}.ocr.json # 文字辨識
|
||||
├── {uuid}.yolo.json # 物件偵測
|
||||
├── {uuid}.face.json # 人臉偵測
|
||||
├── {uuid}.pose.json # 姿態估計
|
||||
└── thumbnails/
|
||||
└── {uuid}/
|
||||
├── thumb_000.jpg
|
||||
├── thumb_001.jpg
|
||||
└── ...
|
||||
```
|
||||
|
||||
### 2.2 儲存策略
|
||||
|
||||
| 資料類型 | 儲存位置 | 說明 |
|
||||
|----------|----------|------|
|
||||
| JSON 檔案 | 專案根目錄 | 方便快速存取 |
|
||||
| 縮圖 | thumbnails/{uuid}/ | 分離儲存 |
|
||||
| 資料庫 | PostgreSQL | 長期儲存 |
|
||||
|
||||
---
|
||||
|
||||
## 3. JSON 結構定義
|
||||
|
||||
### 3.1 Probe (影片探測)
|
||||
|
||||
**檔案**: `{uuid}.probe.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"streams": [
|
||||
{
|
||||
"index": 0,
|
||||
"codec_name": "h264",
|
||||
"codec_type": "video",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"r_frame_rate": "60000/1001",
|
||||
"duration": "6879.329524",
|
||||
"sample_rate": null,
|
||||
"channels": null
|
||||
},
|
||||
{
|
||||
"index": 1,
|
||||
"codec_name": "aac",
|
||||
"codec_type": "audio",
|
||||
"width": null,
|
||||
"height": null,
|
||||
"r_frame_rate": "0/0",
|
||||
"duration": "6879.245333",
|
||||
"sample_rate": "48000",
|
||||
"channels": 2
|
||||
}
|
||||
],
|
||||
"format": {
|
||||
"filename": "/path/to/video.mov",
|
||||
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
|
||||
"duration": "6879.329524",
|
||||
"size": "2361629896",
|
||||
"bit_rate": "2748000"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `streams` | Array | 媒體串流陣列 |
|
||||
| `streams[].index` | Integer | 串流索引 |
|
||||
| `streams[].codec_name` | String | 編碼名稱 |
|
||||
| `streams[].codec_type` | String | 串流類型 (video/audio) |
|
||||
| `streams[].width` | Integer | 寬度 (video) |
|
||||
| `streams[].height` | Integer | 高度 (video) |
|
||||
| `streams[].r_frame_rate` | String | 幀率 |
|
||||
| `streams[].duration` | String | 持續時間 (秒) |
|
||||
| `streams[].sample_rate` | String | 採樣率 (audio) |
|
||||
| `streams[].channels` | Integer | 聲道數 (audio) |
|
||||
| `format` | Object | 檔案格式資訊 |
|
||||
| `format.filename` | String | 原始檔案路徑 |
|
||||
| `format.format_name` | String | 格式名稱 |
|
||||
| `format.duration` | String | 總時長 (秒) |
|
||||
| `format.size` | String | 檔案大小 (bytes) |
|
||||
| `format.bit_rate` | String | 位元率 |
|
||||
|
||||
---
|
||||
|
||||
### 3.2 ASR (語音識別)
|
||||
|
||||
**檔案**: `{uuid}.asr.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"language": "en",
|
||||
"language_probability": 0.9945855736732483,
|
||||
"segments": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 19.04,
|
||||
"text": "Hello and welcome to the old-time movie show."
|
||||
},
|
||||
{
|
||||
"start": 19.04,
|
||||
"end": 25.44,
|
||||
"text": "Today we are featuring the 1963 comedy mystery film Charade."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `language` | String | 偵測語言代碼 (ISO 639-1) |
|
||||
| `language_probability` | Float | 語言偵測機率 (0-1) |
|
||||
| `segments` | Array | 語音分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].text` | String | 識別文字 |
|
||||
|
||||
---
|
||||
|
||||
### 3.3 ASRx (說話者分離)
|
||||
|
||||
**檔案**: `{uuid}.asrx.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"language": "en",
|
||||
"language_probability": 0.95,
|
||||
"segments": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 19.04,
|
||||
"text": "Hello and welcome to the old-time movie show.",
|
||||
"speaker_id": "SPEAKER_00",
|
||||
"speaker_embedding": [0.123, -0.456, ...]
|
||||
},
|
||||
{
|
||||
"start": 19.04,
|
||||
"end": 25.44,
|
||||
"text": "Today we are featuring the 1963 comedy mystery film Charade.",
|
||||
"speaker_id": "SPEAKER_01",
|
||||
"speaker_embedding": [0.789, -0.123, ...]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `language` | String | 偵測語言代碼 |
|
||||
| `language_probability` | Float | 語言偵測機率 |
|
||||
| `segments` | Array | 語音分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].text` | String | 識別文字 |
|
||||
| `segments[].speaker_id` | String | 說話者 ID |
|
||||
| `segments[].speaker_embedding` | Array | 說話者嵌入向量 (可選) |
|
||||
|
||||
---
|
||||
|
||||
### 3.4 OCR (文字辨識)
|
||||
|
||||
**檔案**: `{uuid}.ocr.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"segments": [
|
||||
{
|
||||
"start": 10.5,
|
||||
"end": 12.3,
|
||||
"text": "EXAMPLE TEXT",
|
||||
"boxes": [
|
||||
{
|
||||
"x1": 100,
|
||||
"y1": 50,
|
||||
"x2": 400,
|
||||
"y2": 100
|
||||
}
|
||||
],
|
||||
"confidence": 0.95
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `segments` | Array | OCR 分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].text` | String | 辨識文字 |
|
||||
| `segments[].boxes` | Array | 文字邊界框陣列 |
|
||||
| `segments[].boxes[].x1` | Integer | 左上 X 座標 |
|
||||
| `segments[].boxes[].y1` | Integer | 左上 Y 座標 |
|
||||
| `segments[].boxes[].x2` | Integer | 右下 X 座標 |
|
||||
| `segments[].boxes[].y2` | Integer | 右下 Y 座標 |
|
||||
| `segments[].confidence` | Float | 辨識信心度 |
|
||||
|
||||
---
|
||||
|
||||
### 3.5 YOLO (物件偵測)
|
||||
|
||||
**檔案**: `{uuid}.yolo.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"segments": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 1.0,
|
||||
"objects": [
|
||||
{
|
||||
"class": "person",
|
||||
"confidence": 0.92,
|
||||
"box": {
|
||||
"x1": 150,
|
||||
"y1": 200,
|
||||
"x2": 400,
|
||||
"y2": 800
|
||||
}
|
||||
},
|
||||
{
|
||||
"class": "car",
|
||||
"confidence": 0.87,
|
||||
"box": {
|
||||
"x1": 800,
|
||||
"y1": 400,
|
||||
"x2": 1200,
|
||||
"y2": 700
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `segments` | Array | 時間分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].objects` | Array | 偵測物件陣列 |
|
||||
| `segments[].objects[].class` | String | 物件類別 |
|
||||
| `segments[].objects[].confidence` | Float | 偵測信心度 |
|
||||
| `segments[].objects[].box` | Object | 邊界框 |
|
||||
|
||||
---
|
||||
|
||||
### 3.6 Face (人臉偵測)
|
||||
|
||||
**檔案**: `{uuid}.face.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"segments": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 1.0,
|
||||
"faces": [
|
||||
{
|
||||
"face_id": "face_001",
|
||||
"box": {
|
||||
"x1": 100,
|
||||
"y1": 50,
|
||||
"x2": 300,
|
||||
"y2": 350
|
||||
},
|
||||
"embedding": [0.123, -0.456, ...],
|
||||
"emotion": "happy",
|
||||
"age": 35,
|
||||
"gender": "female"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `segments` | Array | 時間分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].faces` | Array | 人臉陣列 |
|
||||
| `segments[].faces[].face_id` | String | 人臉 ID |
|
||||
| `segments[].faces[].box` | Object | 邊界框 |
|
||||
| `segments[].faces[].embedding` | Array | 人臉嵌入向量 |
|
||||
| `segments[].faces[].emotion` | String | 情緒分類 (可選) |
|
||||
| `segments[].faces[].age` | Integer | 年齡估計 (可選) |
|
||||
| `segments[].faces[].gender` | String | 性別估計 (可選) |
|
||||
|
||||
---
|
||||
|
||||
### 3.7 Pose (姿態估計)
|
||||
|
||||
**檔案**: `{uuid}.pose.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"segments": [
|
||||
{
|
||||
"start": 0.0,
|
||||
"end": 1.0,
|
||||
"poses": [
|
||||
{
|
||||
"person_id": "person_001",
|
||||
"keypoints": {
|
||||
"nose": {"x": 320, "y": 120, "confidence": 0.98},
|
||||
"left_eye": {"x": 335, "y": 110, "confidence": 0.95},
|
||||
"right_eye": {"x": 305, "y": 110, "confidence": 0.93},
|
||||
"left_shoulder": {"x": 280, "y": 180, "confidence": 0.91},
|
||||
"right_shoulder": {"x": 360, "y": 180, "confidence": 0.89}
|
||||
},
|
||||
"confidence": 0.92
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**欄位說明**:
|
||||
|
||||
| 欄位 | 類型 | 說明 |
|
||||
|------|------|------|
|
||||
| `segments` | Array | 時間分段陣列 |
|
||||
| `segments[].start` | Float | 開始時間 (秒) |
|
||||
| `segments[].end` | Float | 結束時間 (秒) |
|
||||
| `segments[].poses` | Array | 姿態陣列 |
|
||||
| `segments[].poses[].person_id` | String | 人員 ID |
|
||||
| `segments[].poses[].keypoints` | Object | 關鍵點 |
|
||||
| `segments[].poses[].keypoints.{name}` | Object | 各關鍵點 |
|
||||
| `segments[].poses[].keypoints.{name}.x` | Integer | X 座標 |
|
||||
| `segments[].poses[].keypoints.{name}.y` | Integer | Y 座標 |
|
||||
| `segments[].poses[].keypoints.{name}.confidence` | Float | 信心度 |
|
||||
| `segments[].poses[].confidence` | Float | 整體信心度 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 處理流程
|
||||
|
||||
### 4.1 處理管線
|
||||
|
||||
```
|
||||
影片檔案
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ 1. Register │ 建立 UUID,註冊影片
|
||||
└────────┬────────┘
|
||||
│
|
||||
┌────▼────┐
|
||||
│ 2. Probe │ ffprobe 擷取元數據
|
||||
└────┬────┘
|
||||
│ {uuid}.probe.json
|
||||
┌────▼─────┐
|
||||
│ 3. ASR │ faster-whisper 語音識別
|
||||
└────┬─────┘
|
||||
│ {uuid}.asr.json
|
||||
┌────▼──────┐
|
||||
│ 4. ASRx │ 說話者分離 (pyannote)
|
||||
└────┬──────┘
|
||||
│ {uuid}.asrx.json
|
||||
┌────▼────┐
|
||||
│ 5. OCR │ Tesseract 文字辨識
|
||||
└────┬────┘
|
||||
│ {uuid}.ocr.json
|
||||
┌────▼────┐
|
||||
│ 6. YOLO │ 物件偵測
|
||||
└────┬────┘
|
||||
│ {uuid}.yolo.json
|
||||
┌────▼────┐
|
||||
│ 7. Face │ 人臉偵測
|
||||
└────┬────┘
|
||||
│ {uuid}.face.json
|
||||
┌────▼────┐
|
||||
│ 8. Pose │ 姿態估計
|
||||
└────┬────┘
|
||||
│ {uuid}.pose.json
|
||||
┌────▼──────┐
|
||||
│ 9. Chunk │ 轉換為資料庫 chunks
|
||||
└───────────┘
|
||||
```
|
||||
|
||||
### 4.2 失敗處理
|
||||
|
||||
| 階段 | 失敗時 | 處理 |
|
||||
|------|--------|------|
|
||||
| Probe | 無法讀取影片 | 終止流程,輸出錯誤 |
|
||||
| ASR | 無音軌 | 產生空 segments,繼續流程 |
|
||||
| OCR/YOLO/Face/Pose | 處理失敗 | 跳過該階段,記錄日誌 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 資料庫儲存
|
||||
|
||||
### 5.1 Chunk 結構
|
||||
|
||||
```sql
|
||||
CREATE TABLE chunks (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
uuid VARCHAR(16) NOT NULL,
|
||||
chunk_id VARCHAR(64) NOT NULL,
|
||||
chunk_index INTEGER NOT NULL,
|
||||
chunk_type VARCHAR(32) NOT NULL,
|
||||
start_time DOUBLE PRECISION NOT NULL,
|
||||
end_time DOUBLE PRECISION NOT NULL,
|
||||
content JSONB NOT NULL,
|
||||
created_at TIMESTAMP DEFAULT NOW(),
|
||||
UNIQUE(uuid, chunk_id)
|
||||
);
|
||||
```
|
||||
|
||||
### 5.2 轉換範例
|
||||
|
||||
```rust
|
||||
// ASR → Chunk (Sentence)
|
||||
for (i, seg) in asr_result.segments.iter().enumerate() {
|
||||
let chunk = Chunk::new(
|
||||
uuid.clone(),
|
||||
i as u32,
|
||||
ChunkType::Sentence,
|
||||
seg.start,
|
||||
seg.end,
|
||||
serde_json::json!({"text": seg.text}),
|
||||
);
|
||||
db.store_chunk(&chunk).await?;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 版本歷史
|
||||
|
||||
| 版本 | 日期 | 變更 |
|
||||
|------|------|------|
|
||||
| 1.0.0 | 2026-03-16 | 初始版本 |
|
||||
|
||||
---
|
||||
|
||||
## 7. 相關文件
|
||||
|
||||
- [RUST_DEVELOPMENT.md](./RUST_DEVELOPMENT.md) - Rust 開發規範
|
||||
- [AGENTS.md](../AGENTS.md) - 開發規範
|
||||
- [monitor_config.yaml](../monitor/config/monitor_config.yaml) - 監控配置
|
||||
@@ -0,0 +1,50 @@
|
||||
# M4 / M5 協作協議
|
||||
|
||||
## 核心原則:檔案是 source of truth
|
||||
|
||||
所有 processor 的產出是 `{uuid}.{processor}.json` 檔案。
|
||||
**檔案存在 = 處理完成**,優先於 DB 或 Redis 的任何狀態記錄。
|
||||
|
||||
## 絕對禁止
|
||||
|
||||
### 1. 不可刪除已存在的輸出檔
|
||||
- 任何 `{uuid}.{processor}.*` 檔案,無論是 `.json`、`.json.tmp`、`.json.partial`、`.json.err`
|
||||
- 一律不允許 `rm`、`unlink`、`delete`
|
||||
- 唯一例外:明確的人工指令 `rm` / `Delete this file`
|
||||
|
||||
### 2. 不可覆蓋已存在的輸出檔
|
||||
- 重新執行 processor 前,必須先 **copy(非 rename)** 加上時間戳備份
|
||||
- 備份命名:`{uuid}.{processor}.{timestamp}.{original_extension}`
|
||||
- 若備份名已存在,跳過(不覆蓋不 counter)
|
||||
- 原檔保留不動
|
||||
|
||||
### 3. 不可跨域操作
|
||||
- M4 只能在 M4 機器(Mac Mini)上操作
|
||||
- M5 只能在 M5 機器(MacBook Pro)上操作
|
||||
- 禁止任何跨機器的檔案操作或 cleanup
|
||||
|
||||
## 重跑 processor 的正確流程
|
||||
|
||||
1. Worker 檢查 `{uuid}.{processor}.json` 是否存在
|
||||
2. **存在 → 跳過**(無論 DB/Redis 狀態)
|
||||
3. 不存在 → copy 備份既有 `{uuid}.{processor}.*` → 執行 processor
|
||||
4. Processor 輸出寫入 `.tmp` → 完成後 rename 為 `.json`
|
||||
|
||||
## 例外處理
|
||||
|
||||
| 狀態 | 行為 |
|
||||
|------|------|
|
||||
| `.json` 存在 | 跳過,視為完成 |
|
||||
| `.json.tmp` 存在(無 `.json`) | 視為未完成,備份後重跑 |
|
||||
| `.json.partial` 存在(無 `.json`) | 視為未完成,備份後重跑 |
|
||||
| `.json.err` 存在(無 `.json`) | 視為未完成,備份後重跑 |
|
||||
| Process 被 kill(SIGKILL) | partial 存為 `.json.partial`(非 `.json`) |
|
||||
|
||||
## 違規後果
|
||||
|
||||
2026-05-09 事故:M4 release 打包未含 .json → 跨域操作 → M5 cleanup 誤刪 asr.json
|
||||
→ 導致 ASR 需重跑(完整電影約 1.5hr)
|
||||
→ YOLO 需重跑
|
||||
→ 損失已完成的 pipeline 進度
|
||||
|
||||
此類違規不可再發生。
|
||||
@@ -0,0 +1,31 @@
|
||||
# M4 Release Incident — 2026-05-09
|
||||
|
||||
## Summary
|
||||
|
||||
M4 在進行 release 打包作業時,未依照計畫包含 output `.json` 檔案,僅在 database 中保留 records。此外 M4 違反操作邊界進入 M5 管轄範圍,M5 執行 cleanup 時將已完成的 `asr.json` 一併刪除。
|
||||
|
||||
## Impact
|
||||
|
||||
| 檔案 | 狀態 | 說明 |
|
||||
|------|------|------|
|
||||
| `{uuid}.asr.json` | ❌ 遺失 | 已完成的 ASR 輸出被 M5 cleanup 誤刪 |
|
||||
| `{uuid}.yolo.json` | ❌ 損毀 | JSON parse error,需重跑 |
|
||||
| DB records | ⚠️ 不一致 | processor_results 狀態與實際檔案不符 |
|
||||
|
||||
## Root Cause
|
||||
|
||||
1. **M4 release 打包遺漏**: Release 流程未將 `.json` 輸出檔納入打包範圍,只保留了 DB。
|
||||
2. **M4 越界操作**: M4 在 M5 的目錄/範圍內執行操作,違反開發隔離原則。
|
||||
3. **M5 cleanup 誤刪**: M5 的 cleanup 機制未預期 M4 的產出,將 `asr.json` 視為無用檔案清除。
|
||||
|
||||
## 處理
|
||||
|
||||
- ASR: 重跑中(asr_processor.py,完整電影約 6780s)
|
||||
- YOLO: 重跑中(yolo_processor.py)
|
||||
- 已修改 worker 邏輯:開機後以 `.json` 檔案存在為 source of truth,不再僅依賴 DB/Redis 狀態
|
||||
|
||||
## 預防措施
|
||||
|
||||
- Release 流程需明確定義 deliverables 包含 `.json` 檔案
|
||||
- M4/M5 操作邊界需嚴格遵守,禁止跨域操作
|
||||
- Cleanup 機制應先確認檔案是否為有效 processor output
|
||||
@@ -0,0 +1,77 @@
|
||||
# M4 vs M5 Max Comparison
|
||||
|
||||
## Hardware
|
||||
|
||||
| Spec | M4 (Mac Mini) | M5 (MacBook Pro) |
|
||||
|------|--------------|-------------------|
|
||||
| **Model** | Mac Mini (M4) | MacBook Pro (M5 Max) |
|
||||
| **Hostname** | `accusys-Mac-mini-M4-2.local` | `Accusyss-MacBook-Pro.local` |
|
||||
| **macOS** | 26.4.1 (Sequoia) | 26.4.1 (Sequoia) |
|
||||
| **RAM** | 16 GB | **48 GB** |
|
||||
| **CPU Cores** | 10 | **18** |
|
||||
| **Disk** | 2TB (est.) | **1.8TB (12GB used, 97% free)** |
|
||||
| **Network** | 192.168.110.210, 192.168.110.200 | 192.168.110.201, 192.168.31.182 |
|
||||
|
||||
## Installed Services
|
||||
|
||||
| Service | M4 | M5 |
|
||||
|---------|-----|------|
|
||||
| **PostgreSQL** | 18.1 (Homebrew) | **18.3 (Source build)** |
|
||||
| **pgvector** | Homebrew | **0.8.2 (Source build)** |
|
||||
| **Redis** | 8.4.0 (Homebrew) | **7.4.3 (Source build)** |
|
||||
| **Qdrant** | Homebrew/pre-built | **1.17.1 (Source build, `cargo`)** |
|
||||
| **MongoDB** | Homebrew | 8.2.7 (Homebrew) |
|
||||
| **MariaDB** | ✗ via brew | **12.2.2 (Homebrew, for WordPress)** |
|
||||
| **PHP** | ✗ via brew | **8.5.5 (Homebrew, WordPress ext. ✅)** |
|
||||
| **SFTPGo** | Pre-built binary | **2.7.1 (Source build, patched dep)** |
|
||||
| **FFmpeg** | 8.1 (Homebrew) | **8.1.1 (Homebrew)** |
|
||||
| **OpenCode** | 1.14.39 | **1.14.39** |
|
||||
| **Gemma4 LLM** | ✗ (not enough RAM) | **31B Q5_K_M @ 8081** |
|
||||
|
||||
## Build Approach
|
||||
|
||||
| Aspect | M4 | M5 |
|
||||
|--------|-----|-----|
|
||||
| **PostgreSQL** | `brew install postgresql@18` | `./configure && make && make install` |
|
||||
| **Redis** | `brew install redis` | `make && cp src/redis-server ~/redis/bin/` |
|
||||
| **Qdrant** | `brew install qdrant` | `cargo build --release --bin qdrant` (from GitHub) |
|
||||
| **SFTPGo** | `brew install sftpgo` | `git clone && go build` (patched `go-m1cpu`) |
|
||||
| **Philosophy** | Mixed (Homebrew + binary) | **Source-first** (GitHub source, checksums recorded) |
|
||||
|
||||
## Data Migration (M4 → M5)
|
||||
|
||||
| Data | Size | Status |
|
||||
|------|------|--------|
|
||||
| **Database (dev schema)** | 837MB dump | ✅ Restored (16 tables) |
|
||||
| **Video file** | 2.2GB | ✅ Transferred |
|
||||
| **output_dev JSON** | 2.9GB (462 files) | ✅ Transferred |
|
||||
| **output JSON** | 65MB (2523 files) | ✅ Transferred |
|
||||
| **Configs** | small | ✅ Transferred |
|
||||
|
||||
## Database Row Counts (M5)
|
||||
|
||||
| Table | Rows |
|
||||
|-------|------|
|
||||
| `pre_chunks` | 494,339 |
|
||||
| `face_detections` | 6,211 |
|
||||
| `tkg_nodes` | 2,414 |
|
||||
| `identity_bindings` | 2,347 |
|
||||
| `tkg_edges` | 1,320 |
|
||||
|
||||
## Key Differences
|
||||
|
||||
### 1. RAM (16GB vs 48GB)
|
||||
- **M4 (16GB)**: Cannot run Gemma4 31B LLM locally. Memory pressure during concurrent pipeline processing.
|
||||
- **M5 (48GB)**: Can run Gemma4 31B (Q5_K_M, ~20GB) + databases + playground simultaneously.
|
||||
|
||||
### 2. Build Philosophy
|
||||
- **M4**: Quick setup via Homebrew bottles (pre-compiled).
|
||||
- **M5**: **Source-first** — every service built from GitHub/official source. `SHA256` checksums recorded. Dependencies patched as needed (SFTPGo `go-m1cpu`).
|
||||
|
||||
### 3. Unique M5 Services
|
||||
- **MariaDB + PHP**: Installed for WordPress/marcom portal development.
|
||||
- **Gemma4 LLM**: Running on port 8081, accessible for RAG/identity clustering.
|
||||
- **OpenCode**: Configured with Gemma4 provider for AI-assisted development.
|
||||
|
||||
### 4. Data Freshness
|
||||
- M5 is a **snapshot** of M4's state at 2026-05-06 (commit `bac6c2d`). Changes made on M4 after sync date must be re-synced.
|
||||
@@ -0,0 +1,259 @@
|
||||
# M5 Dev Environment Setup Log
|
||||
|
||||
**Machine**: M5 MacBook Pro (MacOS 26.4.1, Apple M5 Max, 48GB)
|
||||
**User**: accusys (admin group, sudo with password)
|
||||
**Date**: 2026-05-06
|
||||
**Setup by**: OpenCode
|
||||
|
||||
---
|
||||
|
||||
## 1. Source Code
|
||||
|
||||
| Item | Detail |
|
||||
|------|--------|
|
||||
| Repo | `https://gitea.momentry.ddns.net/warren/momentry_core.git` |
|
||||
| Branch | `main` |
|
||||
| Commit | `bac6c2d` (feat: identity clustering V3.0) |
|
||||
| Sync method | rsync from M4 (192.168.110.210) |
|
||||
| Path | `~/momentry_core_0.1/` |
|
||||
|
||||
---
|
||||
|
||||
## 2. Installed Services
|
||||
|
||||
### 2.1 PostgreSQL 18.3
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://ftp.postgresql.org/pub/source/v18.3/postgresql-18.3.tar.gz](https://ftp.postgresql.org/pub/source/v18.3/postgresql-18.3.tar.gz) |
|
||||
| **GitHub** | [https://github.com/postgresql/postgresql](https://github.com/postgresql/postgresql) |
|
||||
| **Build method** | Manual `./configure && make && make install` |
|
||||
| **Prefix** | `~/pgsql/18.3/` |
|
||||
| **Data dir** | `~/pgsql/data/` |
|
||||
| **Port** | 5432 |
|
||||
| **Version** | PostgreSQL 18.3 |
|
||||
| **SHA256** | `ab04939aafdb9e8487c2f13dda91e6a4a7f4c83368f5bedd23ee4ad1fda64afb` |
|
||||
| **Start command** | `pg_ctl -D ~/pgsql/data -l ~/pgsql/pg.log start` |
|
||||
| **Configure flags** | `--prefix=$HOME/pgsql/18.3 --with-uuid=e2fs --with-icu --with-openssl` |
|
||||
| **Build date** | 2026-05-06 |
|
||||
| **Notes** | `--with-uuid=e2fs` used (requires Homebrew `e2fsprogs`). macOS built-in UUID not detected by configure. |
|
||||
|
||||
### 2.2 pgvector 0.8.2
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://github.com/pgvector/pgvector](https://github.com/pgvector/pgvector) |
|
||||
| **Version** | v0.8.2 |
|
||||
| **Build method** | `git clone && make && make install` |
|
||||
| **SHA256** | `65dec31ec078d60ee9d8e1dac59be8a41edf8c79bf380cd0093691b0afd257a8` |
|
||||
| **Build date** | 2026-05-06 |
|
||||
| **Notes** | Built against PostgreSQL 18.3 source installation |
|
||||
|
||||
### 2.3 Redis 7.4.3
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://github.com/redis/redis/archive/refs/tags/7.4.3.tar.gz](https://github.com/redis/redis/archive/refs/tags/7.4.3.tar.gz) |
|
||||
| **GitHub** | [https://github.com/redis/redis](https://github.com/redis/redis) |
|
||||
| **Version** | 7.4.3 |
|
||||
| **Build method** | `make -j$(sysctl -n hw.ncpu)` |
|
||||
| **Binary path** | `~/redis/bin/redis-server` |
|
||||
| **Port** | 6379 |
|
||||
| **SHA256** | `87b6a9ea145c56c1ace724acbb9906b7be4abddd44041545adf44ce9f4d0a615` |
|
||||
| **Start command** | `redis-server --daemonize yes --port 6379` |
|
||||
| **Build date** | 2026-05-06 |
|
||||
|
||||
### 2.4 Qdrant 1.17.1
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://github.com/qdrant/qdrant.git](https://github.com/qdrant/qdrant.git) |
|
||||
| **Version** | v1.17.1 |
|
||||
| **Build method** | `cargo build --release --bin qdrant` |
|
||||
| **Binary path** | `~/momentry_core_0.1/services/qdrant/target/release/qdrant` |
|
||||
| **Storage dir** | `~/qdrant_storage` |
|
||||
| **Port** | 6333 (HTTP), 6334 (gRPC) |
|
||||
| **SHA256** | `8f8aa63840a0f948b43f9b95f784ace69595892de5dc581bb66bd62fd86d6c66` |
|
||||
| **Build date** | 2026-05-06 |
|
||||
| **Config** | `~/qdrant_config.yaml` |
|
||||
| **Start command** | `qdrant --config-path ~/qdrant_config.yaml &` |
|
||||
| **Build deps** | protoc (Homebrew protobuf), cmake |
|
||||
|
||||
### 2.5 MongoDB 8.2.7
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | Homebrew `mongodb/brew/mongodb-community` |
|
||||
| **Version** | 8.2.7 |
|
||||
| **Port** | 27017 |
|
||||
| **Start command** | `brew services start mongodb/brew/mongodb-community` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
|
||||
### 2.6 MariaDB 12.2.2
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | Homebrew `mariadb` |
|
||||
| **Version** | 12.2.2-MariaDB |
|
||||
| **Port** | 3306 |
|
||||
| **Start command** | `brew services start mariadb` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
|
||||
### 2.7 PHP 8.5.5
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | Homebrew `php` |
|
||||
| **Version** | 8.5.5 |
|
||||
| **WordPress extensions** | mysqli, pdo_mysql, gd, xml, mbstring, curl, zip, json, intl, bcmath, gmp, openssl |
|
||||
| **Start command** | `brew services start php` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
|
||||
### 2.8 FFmpeg / FFprobe 8.1.1
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | Homebrew `ffmpeg` |
|
||||
| **Version** | 8.1.1 |
|
||||
| **SHA256** | `00d01197255300c02122c783dd0126a9e7f47d6c6a19faafae2e6610efd071d3` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
|
||||
### 2.9 SFTPGo 2.7.1
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://github.com/drakkan/sftpgo.git](https://github.com/drakkan/sftpgo.git) |
|
||||
| **Version** | v2.7.1 |
|
||||
| **Build method** | `git clone && go build -o sftpgo_bin ./` |
|
||||
| **Binary path** | `~/momentry_core_0.1/services/sftpgo_bin` |
|
||||
| **SHA256** | `550b6653f8f2cd7c58620e128e85be571a6702c79cf374824ad9b420ca039db1` |
|
||||
| **Build date** | 2026-05-06 |
|
||||
| **Patch** | Upgraded `go-m1cpu` from v0.2.0 → v0.2.1 to fix SIGTRAP crash on macOS 26.4.1 |
|
||||
| **Notes** | Pre-built binary from GitHub releases crashed with `go-m1cpu` cgo compatibility issue. Source build with patched dependency resolved. |
|
||||
|
||||
### 2.10 OpenCode 1.14.39
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | [https://opencode.ai/install](https://opencode.ai/install) |
|
||||
| **Version** | 1.14.39 |
|
||||
| **Binary path** | `~/.opencode/bin/opencode` |
|
||||
| **SHA256** | `def4a786c257bd6a965e46a2b069802496681b9eea20261d7d1b55629af3d1da` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
|
||||
### 2.11 Python 3.11 + Packages
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | Homebrew `python@3.11` |
|
||||
| **Version** | 3.11.15 |
|
||||
| **Path** | `/opt/homebrew/bin/python3.11` |
|
||||
| **Key packages** | coremltools, opencv-python, numpy, psycopg2, torch, transformers, whisperx, etc. |
|
||||
| **Requirements** | `~/momentry_core_0.1/requirements.txt` |
|
||||
| **Install date** | 2026-05-06 |
|
||||
| **FaceNet model** | `models/facenet512.mlpackage` (512D CoreML, loads OK) |
|
||||
|
||||
### 2.12 Build Tools
|
||||
|
||||
| Tool | Version | Source |
|
||||
|------|---------|--------|
|
||||
| Rust | 1.95.0 | rustup (pre-installed) |
|
||||
| Go | 1.26.2 | Homebrew `go` |
|
||||
| cmake | 4.3.2 | Homebrew `cmake` |
|
||||
| pkg-config | - | Homebrew `pkg-config` |
|
||||
|
||||
---
|
||||
|
||||
## 3. Momentry Configuration
|
||||
|
||||
### 3.1 Environment Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `.env` | Production config (port 3002) |
|
||||
| `.env.development` | Development config (port 3003) |
|
||||
|
||||
Key settings:
|
||||
- `DATABASE_URL=postgres://accusys@localhost:5432/momentry`
|
||||
- `REDIS_URL=redis://:accusys@localhost:6379`
|
||||
- `DATABASE_SCHEMA=dev`
|
||||
- `MOMENTRY_SERVER_PORT=3003` (dev) / `3002` (prod)
|
||||
- `MOMENTRY_API_KEY=muser_test_apikey`
|
||||
- `MOMENTRY_PYTHON_PATH=/opt/homebrew/bin/python3.11`
|
||||
- `MOMENTRY_SCRIPTS_DIR=/Users/accusys/momentry_core_0.1/scripts`
|
||||
|
||||
### 3.2 Database Tables Created
|
||||
|
||||
| Table | Created by |
|
||||
|-------|-----------|
|
||||
| `dev.videos` | Manual SQL |
|
||||
| `dev.chunks` | Manual SQL |
|
||||
| `dev.monitor_jobs` | Manual SQL |
|
||||
| `dev.processor_results` | Manual SQL |
|
||||
| `dev.talents` | Manual SQL |
|
||||
| `dev.identity_bindings` | Manual SQL |
|
||||
| `dev.api_keys` | Manual SQL |
|
||||
|
||||
### 3.3 API Key
|
||||
|
||||
- Key: `muser_test_apikey`
|
||||
- Hash (SHA256): `3f2fa16e44ff74267786fdf979b9c33dac0cad515282e4937a0776756a61e821`
|
||||
- Status: active
|
||||
|
||||
---
|
||||
|
||||
## 4. Running Services (Verified)
|
||||
|
||||
| Service | Port | Status |
|
||||
|---------|------|--------|
|
||||
| PostgreSQL | 5432 | ✅ |
|
||||
| Redis | 6379 | ✅ |
|
||||
| Qdrant | 6333 | ✅ |
|
||||
| MongoDB | 27017 | ✅ |
|
||||
| MariaDB | 3306 | ✅ |
|
||||
| Momentry Playground | 3003 | ✅ |
|
||||
| Gemma4 LLM | 8081 | ✅ (pre-installed) |
|
||||
|
||||
---
|
||||
|
||||
## 5. PATH Configuration
|
||||
|
||||
`.zshrc`:
|
||||
```zsh
|
||||
export PATH="/opt/homebrew/bin:/opt/homebrew/opt/postgresql@18/bin:$HOME/.opencode/bin:$PATH"
|
||||
```
|
||||
|
||||
Also available:
|
||||
- `$HOME/pgsql/18.3/bin` — source-built PostgreSQL tools
|
||||
- `$HOME/redis/bin` — source-built Redis
|
||||
- `$HOME/.cargo/bin` — Rust/Cargo tools
|
||||
|
||||
---
|
||||
|
||||
## 6. M5 End-to-End Test Results (Charade Full Movie)
|
||||
|
||||
Run date: 2026-05-06 20:38-20:57
|
||||
|
||||
| Stage | Time | Result |
|
||||
|-------|------|--------|
|
||||
| **Swift_face** (Vision ANE detection) | 867s (14.5 min) | 3999 frames (interval=30) |
|
||||
| **CoreML FaceNet** (512D embedding) | 271s (4.5 min) | 6186 face embeddings |
|
||||
| **Face tracker** (scene-cut aware) | ~30s | 1538 traces |
|
||||
| **DB store** | ~5s | 6186 detections in `dev.face_detections` |
|
||||
| **Total** | ~19 min | 1 long video (412k frames, 2.2GB) |
|
||||
|
||||
**Scene-cut effect**: 1538 traces (vs 379 without scene-cut reset in M4 data). Scene boundaries correctly split traces.
|
||||
|
||||
**Models used**:
|
||||
- Face detection: Apple Vision (ANE) via `swift_face`
|
||||
- Face embedding: CoreML FaceNet 512D via `facenet512.mlpackage`
|
||||
- Text embedding: `mxbai-embed-large` (1024D) via Ollama
|
||||
|
||||
---
|
||||
|
||||
## 7. Known Issues
|
||||
|
||||
1. **Momentry API status `degraded`**: Expected on fresh setup. Some cache/processing dependencies not fully initialized.
|
||||
2. **SFTPGo startup requires config**: Binary built from source, needs config file for production use.
|
||||
3. **Migration scripts not all run**: Base tables created manually. Some migration files (017+) reference tables/columns that need verification.
|
||||
4. **OpenCode config**: `~/.config/opencode/config.json` not yet configured for M5 Gemma4 provider.
|
||||
@@ -0,0 +1,303 @@
|
||||
---
|
||||
document_type: "reference_doc"
|
||||
service: "REDIS"
|
||||
title: "Momentry Core Redis Key 設計規範"
|
||||
date: "2026-03-17"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "momentry"
|
||||
- "core"
|
||||
- "redis"
|
||||
- "設計規範"
|
||||
ai_query_hints:
|
||||
- "查詢 Momentry Core Redis Key 設計規範 的內容"
|
||||
- "Momentry Core Redis Key 設計規範 的主要目的是什麼?"
|
||||
- "如何操作或實施 Momentry Core Redis Key 設計規範?"
|
||||
---
|
||||
|
||||
# Momentry Core Redis Key 設計規範
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-17 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-17 | 創建文件 | Warren | OpenCode / MiniMax M2.5 |
|
||||
| V1.1 | 2026-03-25 | 新增可配置 Redis Key Prefix | Warren | OpenCode / GLM-5 |
|
||||
|
||||
---
|
||||
|
||||
## 1. 概述
|
||||
|
||||
本文檔說明 momentry_core 如何使用 Redis 作為監控和狀態管理系統。
|
||||
|
||||
## 2. 可配置 Redis Key Prefix
|
||||
|
||||
### 2.1 環境變數
|
||||
|
||||
從 V1.1 開始,所有 Redis Keys 都支援自定義前綴:
|
||||
|
||||
```bash
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
```
|
||||
|
||||
此設定允許多個 momentry 實例共用同一個 Redis 伺服器,例如:
|
||||
- **生產環境**: `MOMENTRY_REDIS_PREFIX=momentry:`
|
||||
- **開發環境**: `MOMENTRY_REDIS_PREFIX=momentry_dev:`
|
||||
|
||||
### 2.2 Key 格式
|
||||
|
||||
所有 Key 都遵循以下格式:
|
||||
|
||||
```
|
||||
{prefix}{key_type}:{uuid}
|
||||
```
|
||||
|
||||
範例 (生產環境):
|
||||
```
|
||||
momentry:job:5dea6618a606e7c7
|
||||
momentry:jobs:active
|
||||
momentry:health:current
|
||||
```
|
||||
|
||||
範例 (開發環境):
|
||||
```
|
||||
momentry_dev:job:5dea6618a606e7c7
|
||||
momentry_dev:jobs:active
|
||||
momentry_dev:health:current
|
||||
```
|
||||
|
||||
### 2.3 預設值
|
||||
|
||||
| Binary | 預設 Port | 預設 Redis Prefix |
|
||||
|--------|-----------|-------------------|
|
||||
| `momentry` (生產) | 3002 | `momentry:` |
|
||||
| `momentry_playground` (開發) | 3003 | `momentry_dev:` |
|
||||
|
||||
## 3. UUID 使用時機
|
||||
|
||||
### 3.1 全局 Keys(無 UUID)
|
||||
|
||||
- 單一實例全局狀態
|
||||
- 聚合統計數據
|
||||
|
||||
### 3.2 Per-Video Keys(UUID 必要)
|
||||
|
||||
- 每個視頻獨立處理狀態
|
||||
- 即時進度追蹤
|
||||
|
||||
### 3.3 Per-Processor Keys(UUID + Processor 必要)
|
||||
|
||||
- 每個 processor 獨立狀態
|
||||
|
||||
## 4. Key 命名空間
|
||||
|
||||
```
|
||||
momentry
|
||||
├── health: # 健康檢查
|
||||
│ ├── current # 當前狀態 (TTL: 60s)
|
||||
│ └── services # 依賴服務狀態
|
||||
├── config: # 配置
|
||||
├── stats: # 聚合統計
|
||||
│ ├── total_jobs # 總 Jobs 數
|
||||
│ ├── processed_today # 今日處理數
|
||||
│ ├── cpu:current # 當前 CPU 使用
|
||||
│ └── memory:current # 當前 Memory 使用
|
||||
├── jobs: # Jobs 管理
|
||||
│ ├── active # Set: 運行中 UUIDs
|
||||
│ ├── completed # Set: 完成 UUIDs
|
||||
│ └── failed # Set: 失敗 UUIDs
|
||||
├── job:{uuid} # Per-Video Job 狀態 (TTL: 24h)
|
||||
│ ├── status # 狀態 String
|
||||
│ ├── video_path # 視頻路徑
|
||||
│ ├── current_processor # 當前 processor
|
||||
│ ├── progress_total # 總進度
|
||||
│ ├── progress_current # 當前進度
|
||||
│ ├── started_at # 開始時間
|
||||
│ ├── updated_at # 最後更新
|
||||
│ └── processor:{name} # Per-Processor 狀態
|
||||
│ ├── status
|
||||
│ ├── progress
|
||||
│ ├── current
|
||||
│ ├── total
|
||||
│ └── started_at
|
||||
├── progress:{uuid} # Pub/Sub 頻道 (即時進度)
|
||||
├── result:{uuid} # 處理結果 Hash
|
||||
├── output:{uuid} # 輸出路徑
|
||||
├── metrics:{uuid} # Per-Video 指標
|
||||
│ ├── cpu # CPU 歷史 List (100條, TTL: 1h)
|
||||
│ ├── memory # Memory 歷史 List (100條, TTL: 1h)
|
||||
│ └── duration # 處理時長
|
||||
└── log:{uuid} # 處理日誌 String
|
||||
```
|
||||
|
||||
## 5. Key 詳細說明
|
||||
|
||||
### 全局 Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:health:current` | String | 60s | 當前健康狀態 |
|
||||
| `momentry:health:services` | Hash | 60s | 依賴服務健康狀態 |
|
||||
| `momentry:stats:total_jobs` | String | - | 總 Jobs 數 |
|
||||
| `momentry:stats:processed_today` | String | 86400s | 今日處理數 |
|
||||
| `momentry:stats:cpu:current` | String | 10s | 當前 CPU 使用 |
|
||||
| `momentry:stats:memory:current` | String | 10s | 當前 Memory 使用 |
|
||||
| `momentry:jobs:active` | Set | - | 運行中 Job UUIDs |
|
||||
| `momentry:jobs:completed` | Set | - | 完成 Job UUIDs |
|
||||
| `momentry:jobs:failed` | Set | - | 失敗 Job UUIDs |
|
||||
|
||||
### Per-Video Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:job:{uuid}` | Hash | 24h | Job 完整狀態 |
|
||||
| `momentry:job:{uuid}:status` | String | 24h | Job 狀態 |
|
||||
| `momentry:progress:{uuid}` | Pub/Sub | - | 即時進度頻道 |
|
||||
| `momentry:result:{uuid}` | Hash | 24h | 處理結果 |
|
||||
| `momentry:output:{uuid}` | String | 24h | 輸出路徑 |
|
||||
| `momentry:metrics:{uuid}:cpu` | List | 1h | CPU 歷史 (100條) |
|
||||
| `momentry:metrics:{uuid}:memory` | List | 1h | Memory 歷史 (100條) |
|
||||
| `momentry:metrics:{uuid}:duration` | String | 24h | 處理時長 |
|
||||
| `momentry:log:{uuid}` | String | 24h | 處理日誌 |
|
||||
|
||||
### Per-Processor Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:job:{uuid}:processor:{name}` | Hash | 24h | Processor 狀態 |
|
||||
| `momentry:job:{uuid}:processor:{name}:status` | String | 24h | 狀態 |
|
||||
| `momentry:job:{uuid}:processor:{name}:progress` | String | 24h | 進度百分比 |
|
||||
| `momentry:job:{uuid}:processor:{name}:current` | String | 24h | 當前項目 |
|
||||
| `momentry:job:{uuid}:processor:{name}:total` | String | 24h | 總項目數 |
|
||||
| `momentry:job:{uuid}:processor:{name}:started_at` | String | 24h | 開始時間 |
|
||||
|
||||
## 6. TTL 策略
|
||||
|
||||
| Key 類型 | TTL | 原因 |
|
||||
|----------|-----|------|
|
||||
| Health | 60s | 需要定期更新 |
|
||||
| Job | 24h | 處理完成後保留一天 |
|
||||
| Processor | 24h | 處理完成後保留一天 |
|
||||
| Metrics | 1h | 只保留近期歷史 |
|
||||
| Progress Pub/Sub | - | 不持久,僅即時訊息 |
|
||||
| Stats | 無 | 持久統計 |
|
||||
|
||||
## 7. 訊息格式
|
||||
|
||||
### Pub/Sub 訊息 (progress:{uuid})
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "info | progress | complete | error",
|
||||
"processor": "yolo | ocr | face | pose | cut | asr | asrx",
|
||||
"timestamp": 1700000000,
|
||||
"data": {
|
||||
"message": "Processing frame 5000",
|
||||
"current": 5000,
|
||||
"total": 14315
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Job 狀態 Hash
|
||||
|
||||
```json
|
||||
{
|
||||
"uuid": "5dea6618a606e7c7",
|
||||
"video_path": "/path/to/video.mp4",
|
||||
"status": "running",
|
||||
"current_processor": "yolo",
|
||||
"progress_total": 70,
|
||||
"progress_current": 50,
|
||||
"started_at": 1700000000,
|
||||
"updated_at": 1700000100,
|
||||
"error_count": 0,
|
||||
"last_error": ""
|
||||
}
|
||||
```
|
||||
|
||||
### Processor 狀態 Hash
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "yolo",
|
||||
"status": "running",
|
||||
"progress": 70,
|
||||
"current_frame": 10000,
|
||||
"total_frames": 14315,
|
||||
"started_at": 1700000000,
|
||||
"updated_at": 1700000100
|
||||
}
|
||||
```
|
||||
|
||||
## 8. 實作函數 (Rust)
|
||||
|
||||
所有 Redis Key 生成函數使用 `REDIS_KEY_PREFIX` 靜態變數:
|
||||
|
||||
```rust
|
||||
use crate::core::config::REDIS_KEY_PREFIX;
|
||||
|
||||
fn global_key(key: &str) -> String {
|
||||
format!("{}{}", REDIS_KEY_PREFIX, key)
|
||||
}
|
||||
|
||||
fn job_key(uuid: &str) -> String {
|
||||
format!("{}job:{}", REDIS_KEY_PREFIX, uuid)
|
||||
}
|
||||
|
||||
fn processor_key(uuid: &str, processor: &str) -> String {
|
||||
format!("{}job:{}:processor:{}", REDIS_KEY_PREFIX, uuid, processor)
|
||||
}
|
||||
|
||||
fn progress_channel(uuid: &str) -> String {
|
||||
format!("{}progress:{}", REDIS_KEY_PREFIX, uuid)
|
||||
}
|
||||
|
||||
fn metrics_key(uuid: &str, metric: &str) -> String {
|
||||
format!("{}metrics:{}:{}", REDIS_KEY_PREFIX, uuid, metric)
|
||||
}
|
||||
|
||||
fn jobs_set_key(status: &str) -> String {
|
||||
format!("{}jobs:{}", REDIS_KEY_PREFIX, status)
|
||||
}
|
||||
```
|
||||
|
||||
**注意**: `REDIS_KEY_PREFIX` 定義於 `src/core/config.rs`,由環境變數 `MOMENTRY_REDIS_PREFIX` 控制。
|
||||
|
||||
## 9. 環境變數
|
||||
|
||||
```bash
|
||||
# Redis 連接
|
||||
REDIS_URL=redis://localhost:6379
|
||||
REDIS_PASSWORD=accusys
|
||||
REDIS_DB=0
|
||||
|
||||
# Redis Key Prefix (可選,預設: momentry:)
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
|
||||
# 生產環境範例 (.env)
|
||||
MOMENTRY_SERVER_PORT=3002
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
|
||||
# 開發環境範例 (.env.development)
|
||||
MOMENTRY_SERVER_PORT=3003
|
||||
MOMENTRY_REDIS_PREFIX=momentry_dev:
|
||||
```
|
||||
|
||||
## 11. 監控腳本
|
||||
|
||||
使用 Redis 進行監控的腳本應參考:
|
||||
|
||||
- `monitor/service/momentry_redis_monitor.sh` - Redis 健康檢查
|
||||
- `monitor/service/momentry_job_monitor.sh` - Job 狀態監控
|
||||
@@ -0,0 +1,830 @@
|
||||
---
|
||||
document_type: "reference_doc"
|
||||
service: "MOMENTRY_CORE"
|
||||
title: "待解決問題追蹤"
|
||||
date: "2026-03-17"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "待解決問題追蹤"
|
||||
ai_query_hints:
|
||||
- "查詢 待解決問題追蹤 的內容"
|
||||
- "待解決問題追蹤 的主要目的是什麼?"
|
||||
- "如何操作或實施 待解決問題追蹤?"
|
||||
---
|
||||
|
||||
# 待解決問題追蹤
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-17 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-17 | 創建文件 | Warren | OpenCode / MiniMax M2.5 |
|
||||
| V1.1 | 2026-03-21 | 更新問題狀態 | OpenCode | - |
|
||||
| V1.2 | 2026-03-21 | 添加備份機制優化待辦 | OpenCode | - |
|
||||
| V1.3 | 2026-03-21 | 完成清理硬編碼密碼 | OpenCode | - |
|
||||
| V1.4 | 2026-03-21 | 完成 OpenCode n8n MCP 整合 | OpenCode | - |
|
||||
| V1.5 | 2026-03-21 | 完成 API Key Management 核心模組 | OpenCode | - |
|
||||
| V1.6 | 2026-03-23 | 添加 Momentry Core API launchd 待辦 | OpenCode | - |
|
||||
| V1.7 | 2026-03-23 | 完成 Momentry Core API launchd 設定 | OpenCode | - |
|
||||
| V1.8 | 2026-03-24 | 完成服務統一遷移,所有服務使用自定義 plist | OpenCode | big-pickle |
|
||||
| V1.9 | 2026-03-24 | 建立統一會員系統實作計畫 | OpenCode | big-pickle |
|
||||
| V2.0 | 2026-03-24 | 建立 Job Worker 實作計畫 | OpenCode | big-pickle |
|
||||
|
||||
---
|
||||
|
||||
## 問題 #1: sqlx async INSERT 不會實際寫入數據庫
|
||||
|
||||
### 問題描述
|
||||
使用 sqlx async 執行 INSERT 時,報告成功(rows_affected=1),但數據沒有實際寫入數據庫。
|
||||
|
||||
### 嘗試過的解決方案
|
||||
| # | 嘗試方法 | 結果 |
|
||||
|---|---------|------|
|
||||
| 1 | 使用 `execute()` | 報告成功但未寫入 |
|
||||
| 2 | 使用 `fetch()` | 同樣問題 |
|
||||
| 3 | 使用交易 | 同樣問題 |
|
||||
| 4 | 使用連接池 `acquire()` | 同樣問題 |
|
||||
| 5 | 每個操作創建新連接池 | 同樣問題 |
|
||||
| 6 | 使用 `std::process::Command` 同步調用 psql | 同樣問題 |
|
||||
| 7 | 使用 `tokio::task::spawn_blocking` | 同樣問題 |
|
||||
|
||||
### 觀察到的現象
|
||||
- 直接用 psql 命令行可以成功寫入
|
||||
- 用另一個 PostgreSQL client 可以成功寫入
|
||||
- sqlx 查詢 COUNT(*) 可以正確讀取數據
|
||||
- 但 sqlx INSERT 報告成功卻不寫入
|
||||
|
||||
### 懷疑方向
|
||||
- sqlx 連接池與 PostgreSQL 的某種交互問題
|
||||
- Rust async runtime 與 PostgreSQL client 的問題
|
||||
- postgresql.conf 配置問題
|
||||
|
||||
### 臨時解決方案
|
||||
- Qdrant 向量存儲正常工作(不受影響)
|
||||
- 存儲狀態追蹤功能正常運作
|
||||
|
||||
### 負責人
|
||||
-
|
||||
|
||||
### 建立日期
|
||||
2026-03-17
|
||||
|
||||
### 備註
|
||||
影響 `store_vector` 函數,PVector 存儲無法正常工作,但 QVector 正常運作
|
||||
|
||||
### 2026-03-21 調查結果
|
||||
|
||||
#### 測試結果
|
||||
- 直接 psql INSERT: ✅ 成功
|
||||
- 資料寫入驗證: ✅ 成功
|
||||
|
||||
#### 發現的問題
|
||||
`store_vector` 函數 (`postgres_db.rs:819-860`) 存在以下問題:
|
||||
|
||||
```rust
|
||||
// 問題 1: 錯誤被靜默忽略
|
||||
match join_result {
|
||||
Ok((cid, Ok(output))) => {
|
||||
if !output.status.success() {
|
||||
let err = String::from_utf8_lossy(&output.stderr);
|
||||
tracing::error!("psql error for {}: {}", cid, err);
|
||||
// 沒有返回錯誤!只是記錄日誌
|
||||
}
|
||||
}
|
||||
// ...
|
||||
}
|
||||
Ok(()) // 即使失敗也返回 Ok
|
||||
```
|
||||
|
||||
#### 建議修復
|
||||
1. 讓 `store_vector` 返回實際結果
|
||||
2. 在失敗時返回 `Err`
|
||||
3. 添加單元測試驗證
|
||||
|
||||
#### 下一步
|
||||
- [x] 修復 `store_vector` 錯誤處理
|
||||
- [x] 添加單元測試
|
||||
- [ ] 重現並確認問題根因
|
||||
|
||||
### 2026-03-21 修復完成
|
||||
|
||||
已修復 `store_vector` 函數的錯誤處理:
|
||||
|
||||
```rust
|
||||
// 修復前:錯誤被靜默忽略
|
||||
match join_result {
|
||||
Ok((cid, Ok(output))) => {
|
||||
if !output.status.success() {
|
||||
tracing::error!("..."); // 只記錄,不返回錯誤
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(()) // 即使失敗也返回 Ok
|
||||
|
||||
// 修復後:正確傳播錯誤
|
||||
let (cid, output) = result;
|
||||
let output = output.map_err(|e| anyhow::anyhow!(...))?;
|
||||
if !output.status.success() {
|
||||
anyhow::bail!("psql INSERT failed...");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 問題 #2: TUI 與 stdout 輸出混合
|
||||
|
||||
### 問題描述
|
||||
Python processors 的 progress 輸出蓋過 TUI,導致使用者無法清楚看到處理進度。
|
||||
|
||||
### 解決方案
|
||||
~~使用 TUI 渲染到 stderr~~ → 使用 Redis Pub/Sub 作為消息總線
|
||||
|
||||
### 當前狀態
|
||||
| 子項目 | 狀態 |
|
||||
|--------|------|
|
||||
| RedisPublisher Python 端 | ✅ 已實作 |
|
||||
| Rust Redis 客戶端 | ✅ 已實作 (`redis_client.rs`) |
|
||||
| Rust 訂閱更新 TUI | ✅ 已實作 (main.rs) |
|
||||
| 中斷後繼續/重做 | 🔜 待實作 |
|
||||
|
||||
### 負責人
|
||||
-
|
||||
|
||||
### 建立日期
|
||||
2026-03-17
|
||||
|
||||
### 更新日期
|
||||
2026-03-21
|
||||
|
||||
### 參考文檔
|
||||
- `docs_v1.0/REFERENCE/MOMENTRY_CORE_REDIS_KEYS.md`
|
||||
- `scripts/redis_publisher.py`
|
||||
- `src/core/db/redis_client.rs`
|
||||
|
||||
---
|
||||
|
||||
## 問題 #3: Redis Message Bus 尚未實作
|
||||
|
||||
### 問題描述
|
||||
根據設計規範,需要使用 Redis 作為監控和狀態管理系統。
|
||||
|
||||
### 當前狀態
|
||||
|
||||
| 實作項目 | 狀態 | 說明 |
|
||||
|---------|------|------|
|
||||
| Redis 客戶端 (Hash) | ✅ | `redis_client.rs` |
|
||||
| Redis 客戶端 (Pub/Sub) | ✅ | `redis_client.rs::subscribe_progress()` |
|
||||
| Python RedisPublisher | ✅ | `scripts/redis_publisher.py` |
|
||||
| Rust 訂閱頻道 | ✅ | `main.rs` 中的 redis 訂閱邏輯 |
|
||||
| 監控腳本 | ✅ | `monitor/service/health_check.sh` |
|
||||
|
||||
### 負責人
|
||||
-
|
||||
|
||||
### 建立日期
|
||||
2026-03-17
|
||||
|
||||
### 更新日期
|
||||
2026-03-21
|
||||
|
||||
### 優先級
|
||||
~~高~~ → 中 - 核心功能已完成
|
||||
|
||||
### 參考文檔
|
||||
- `docs_v1.0/REFERENCE/MOMENTRY_CORE_REDIS_KEYS.md`
|
||||
- `docs_v1.0/OPERATIONS/MOMENTRY_CORE_MONITORING.md`
|
||||
|
||||
---
|
||||
|
||||
## 架構優化待評估
|
||||
|
||||
詳細內容請參考 [ARCHITECTURE_EVALUATION.md](./ARCHITECTURE_EVALUATION.md)
|
||||
|
||||
| 項目 | 複雜度 | 優先級 |
|
||||
|------|--------|--------|
|
||||
| PostgreSQL → Redis 故障轉移 | 中 | 待評估 |
|
||||
| 連接池監控 | 低 | 待評估 |
|
||||
| Processor 重試機制 | 中 | 待評估 |
|
||||
| PyO3 整合 | 高 | 低 |
|
||||
| HTTP 健康端點 | 低 | ✅ 已完成 |
|
||||
| Commit Message Lint | 低 | ✅ 已完成 |
|
||||
|
||||
---
|
||||
|
||||
## 備份機制優化 (2026-03-21)
|
||||
|
||||
### 問題分析
|
||||
|
||||
| 問題 | 嚴重性 | 說明 |
|
||||
|------|--------|------|
|
||||
| 溫冷分層未啟用 | 中 | weekly/monthly/archive 目錄為空 |
|
||||
| 清理策略未執行 | 高 | 每日備份保留過多,空間浪費 |
|
||||
| 無異地備份 | 高 | 無遠程備份(雲端/另一設備) |
|
||||
| 備份驗證未自動化 | 中 | 只檢查文件存在,沒驗證可恢復性 |
|
||||
| Gitea 大文件 | 中 | 每次備份 1GB,頻率過高 |
|
||||
| 密碼硬編碼 | 高 | 腳本中有多處明文密碼 |
|
||||
|
||||
### 待辦事項
|
||||
|
||||
| # | 任務 | 優先級 | 狀態 |
|
||||
|---|------|--------|------|
|
||||
| 1 | 啟用溫冷分層 (`backup_monitor.sh tier`) | 高 | 待辦 |
|
||||
| 2 | 啟用清理策略 (`backup_monitor.sh cleanup`) | 高 | 待辦 |
|
||||
| 3 | 添加備份驗證腳本 | 高 | 待辦 |
|
||||
| 4 | 優化 Gitea 備份頻率 (改為週備份) | 中 | 待辦 |
|
||||
| 5 | 創建外部備份腳本 (rsync/雲端) | 高 | 待辦 |
|
||||
| 6 | 清理腳本中的硬編碼密碼 | 高 | ✅ 已完成 |
|
||||
|
||||
### 推薦備份策略
|
||||
|
||||
| 層級 | 保留時間 | 頻率 | 目標 |
|
||||
|------|----------|------|------|
|
||||
| Hot | 7 天 | 每日 | 本地 SSD |
|
||||
| Warm | 30 天 | 每週 | 本地 HDD |
|
||||
| Cold | 90 天 | 每月 | 外部存儲 |
|
||||
| Archive | 1 年 | 每季 | 離線/雲端 |
|
||||
|
||||
### 建議的 Crontab
|
||||
|
||||
```bash
|
||||
# 每日備份 (排除 Gitea)
|
||||
0 3 * * 1-6 /Users/accusys/momentry/scripts/backup_all.sh all_except_gitea
|
||||
|
||||
# 每週完整備份 (含 Gitea)
|
||||
0 3 * * 0 /Users/accusys/momentry/scripts/backup_all.sh all
|
||||
|
||||
# 每週溫冷分層
|
||||
0 4 * * 0 /Users/accusys/momentry_core_0.1/monitor/storage/backup_monitor.sh tier
|
||||
|
||||
# 每週清理
|
||||
0 5 * * 0 /Users/accusys/momentry_core_0.1/monitor/storage/backup_monitor.sh cleanup
|
||||
|
||||
# 每月驗證
|
||||
0 6 1 * * /Users/accusys/momentry/scripts/verify_backup.sh
|
||||
```
|
||||
|
||||
### 負責人
|
||||
-
|
||||
|
||||
### 參考文件
|
||||
- `/Users/accusys/momentry/scripts/backup_all.sh`
|
||||
- `/Users/accusys/momentry_core_0.1/monitor/storage/backup_monitor.sh`
|
||||
- `/Users/accusys/momentry/backup/`
|
||||
|
||||
---
|
||||
|
||||
## OpenCode n8n MCP 整合 (2026-03-21)
|
||||
|
||||
### 完成狀態
|
||||
|
||||
| 項目 | 狀態 | 說明 |
|
||||
|------|------|------|
|
||||
| n8n REST API 啟用 | ✅ | `N8N_PUBLIC_API_ENABLED=true` |
|
||||
| API Key 生成 | ✅ | Settings → API |
|
||||
| MCP Server 安裝 | ✅ | `@nextoolsolutions/mcp-n8n` |
|
||||
| OpenCode 設定 | ✅ | `~/.config/opencode/opencode.json` |
|
||||
| 43 工具可用 | ✅ | workflows, executions, datatables, tags 等 |
|
||||
|
||||
### 設定檔案
|
||||
|
||||
`~/.config/opencode/opencode.json`:
|
||||
```json
|
||||
{
|
||||
"mcp": {
|
||||
"gitea": {
|
||||
"type": "local",
|
||||
"enabled": true,
|
||||
"command": [
|
||||
"/opt/homebrew/bin/gitea-mcp-server",
|
||||
"-token", "<GITEA_TOKEN>",
|
||||
"-host", "http://localhost:3000"
|
||||
]
|
||||
},
|
||||
"n8n": {
|
||||
"type": "local",
|
||||
"enabled": true,
|
||||
"command": ["/opt/homebrew/bin/mcp-n8n"],
|
||||
"environment": {
|
||||
"N8N_BASE_URL": "http://localhost:5678",
|
||||
"N8N_API_KEY": "<N8N_API_KEY>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 重要提醒
|
||||
|
||||
1. **API Key 安全**: 避免提交到 Git
|
||||
2. **n8n 需通過反向代理**: localhost:5678 無法直接訪問 API,需通過 Caddy
|
||||
3. **重啟生效**: 修改 `opencode.json` 後需重啟 OpenCode
|
||||
|
||||
### 參考文件
|
||||
- `docs_v1.0/IMPLEMENTATION/OPENCODE_GUIDE.md` - MCP 設定章節
|
||||
- `docs_v1.0/IMPLEMENTATION/INSTALL_N8N.md` - n8n 安裝指南
|
||||
|
||||
---
|
||||
|
||||
## n8n API 備份腳本 (2026-03-21)
|
||||
|
||||
### 腳本位置
|
||||
|
||||
| 腳本 | 說明 | 依賴 |
|
||||
|------|------|------|
|
||||
| `monitor/workflow/backup_n8n_api.py` | REST API 備份 | requests |
|
||||
| `monitor/workflow/backup_n8n_mcp.py` | MCP 備份(開發中) | mcp SDK |
|
||||
|
||||
### 使用方式
|
||||
|
||||
```bash
|
||||
# 備份所有 workflows
|
||||
N8N_API_KEY="..." python3.11 backup_n8n_api.py
|
||||
|
||||
# 只顯示變更(不備份)
|
||||
N8N_API_KEY="..." python3.11 backup_n8n_api.py --diff
|
||||
|
||||
# 差異備份(只備份變更的 workflows)
|
||||
N8N_API_KEY="..." python3.11 backup_n8n_api.py --incremental
|
||||
|
||||
# 列出可用備份
|
||||
python3.11 backup_n8n_api.py --list
|
||||
|
||||
# 驗證最新備份
|
||||
python3.11 backup_n8n_api.py --verify
|
||||
|
||||
# 顯示備份統計
|
||||
python3.11 backup_n8n_api.py --stats
|
||||
|
||||
# 只備份啟用的 workflows
|
||||
N8N_API_KEY="..." python3.11 backup_n8n_api.py --active-only
|
||||
|
||||
# 備份失敗的執行記錄
|
||||
N8N_API_KEY="..." python3.11 backup_n8n_api.py --failed-only
|
||||
```
|
||||
|
||||
### 功能
|
||||
|
||||
- [x] REST API 備份
|
||||
- [x] 變更偵測
|
||||
- [x] SHA256 校驗
|
||||
- [x] 備份版本化
|
||||
- [x] 差異備份(`--incremental`)
|
||||
- [x] 備份驗證(`--verify`)
|
||||
- [x] 備份統計(`--stats`)
|
||||
- [x] 失敗執行記錄備份(`--failed-only`)
|
||||
- [ ] 選擇性備份(按 Tags)
|
||||
|
||||
### 備份位置
|
||||
|
||||
```
|
||||
/Users/accusys/momentry/backup/n8n_workflows/api/
|
||||
├── 20260321_122059/
|
||||
│ ├── workflows.json # 21 workflows
|
||||
│ ├── workflows.json.sha256 # SHA256 校驗
|
||||
│ ├── tags.json # Tags(若有)
|
||||
│ └── manifest.json # 元數據
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Crontab 建議
|
||||
|
||||
```bash
|
||||
# 每日備份(下午 3 點)
|
||||
0 15 * * * N8N_API_KEY="..." /opt/homebrew/bin/python3.11 /Users/accusys/momentry_core_0.1/monitor/workflow/backup_n8n_api.py >> /Users/accusys/momentry/log/monitor/workflow_backup_api.log 2>&1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Key Management System (2026-03-21)
|
||||
|
||||
### 設計目標
|
||||
|
||||
為 Momentry Core 實現完整的 API Key 管理系統,包括:
|
||||
- API Key 生成(安全隨機)
|
||||
- Key 哈希(SHA256)
|
||||
- 異常檢測
|
||||
- 強制輪換機制
|
||||
- 審計日誌
|
||||
|
||||
### 模組架構
|
||||
|
||||
```
|
||||
src/core/api_key/
|
||||
├── mod.rs # 模組導出
|
||||
├── models.rs # 數據模型和類型
|
||||
├── service.rs # 核心服務邏輯
|
||||
├── anomaly.rs # 異常檢測
|
||||
└── rotation.rs # 輪換管理
|
||||
```
|
||||
|
||||
### Key 類型
|
||||
|
||||
| 類型 | 前綴 | 默認 TTL | 寬限期 |
|
||||
|------|------|----------|--------|
|
||||
| System | `msys_` | 365 天 | 72 小時 |
|
||||
| User | `muser_` | 90 天 | 24 小時 |
|
||||
| Service | `msvc_` | 180 天 | 48 小時 |
|
||||
| Integration | `mint_` | 30 天 | 24 小時 |
|
||||
| Emergency | `memg_` | 1 天 | 0 小時 |
|
||||
|
||||
### Key 格式
|
||||
|
||||
```
|
||||
{prefix}{uuid}_{timestamp}_{random}
|
||||
例如: muser_a1b2c3d4e5f6_1710998400_abc12345
|
||||
```
|
||||
|
||||
### 異常檢測閾值
|
||||
|
||||
| 指標 | 閾值 |
|
||||
|------|------|
|
||||
| 每分鐘請求數 | 1000 |
|
||||
| 每小時請求數 | 10000 |
|
||||
| 錯誤率 | 50% |
|
||||
| 每小時唯一 IP 數 | 5 |
|
||||
| 鎖定閾值 | 3 次觸發 |
|
||||
|
||||
### 實現狀態
|
||||
|
||||
| 組件 | 狀態 | 說明 |
|
||||
|------|------|------|
|
||||
| 數據模型 | ✅ 完成 | `models.rs` |
|
||||
| Key 生成/哈希 | ✅ 完成 | `service.rs` |
|
||||
| 異常檢測 | ✅ 完成 | `anomaly.rs` |
|
||||
| 輪換機制 | ✅ 完成 | `rotation.rs` |
|
||||
| CLI 命令 | ✅ 完成 | `main.rs` |
|
||||
| 數據庫集成 | ✅ 完成 | `postgres_db.rs` |
|
||||
| Redis 告警 | ✅ 完成 | `redis_client.rs` |
|
||||
| 數據庫遷移 | ✅ 完成 | `migrations/001_api_key_management.sql` |
|
||||
| 單元測試 | ✅ 完成 | 55 個測試通過 |
|
||||
|
||||
### CLI 命令
|
||||
|
||||
```bash
|
||||
# 創建 API Key
|
||||
momentry api-key create <name> --key-type service --ttl 90
|
||||
|
||||
# 列出所有 Keys
|
||||
momentry api-key list
|
||||
|
||||
# 驗證 Key
|
||||
momentry api-key validate --key <key>
|
||||
|
||||
# 撤銷 Key
|
||||
momentry api-key revoke --key <key>
|
||||
|
||||
# 請求輪換
|
||||
momentry api-key rotate --key <key>
|
||||
|
||||
# 顯示統計
|
||||
momentry api-key stats
|
||||
```
|
||||
|
||||
### 參考文件
|
||||
|
||||
- `src/core/api_key/` - API Key 模組
|
||||
- `docs_v1.0/REFERENCE/API_KEY_MANAGEMENT.md` - 設計文檔
|
||||
- `migrations/001_api_key_management.sql` - 數據庫遷移
|
||||
|
||||
---
|
||||
|
||||
## 問題 #5: Redis 用戶名問題 (2026-03-21)
|
||||
|
||||
### 問題描述
|
||||
|
||||
Redis 僅有 `default` 用戶,無 `accusys` 用戶。`.env` 檔案使用 `redis://accusys:accusys@localhost:6379` 格式會導致認證失敗。
|
||||
|
||||
### 測試結果
|
||||
|
||||
| URL 格式 | 結果 |
|
||||
|----------|------|
|
||||
| `redis://accusys:accusys@localhost:6379` | ❌ AUTH failed |
|
||||
| `redis://:accusys@localhost:6379` | ✅ PONG |
|
||||
|
||||
### Redis ACL 狀態
|
||||
|
||||
```
|
||||
user default on sanitize-payload #1bd51c... ~* &* +@all
|
||||
requirepass: accusys
|
||||
```
|
||||
|
||||
### 根本原因
|
||||
|
||||
1. Redis 啟動時僅設定 `--requirepass accusys`
|
||||
2. 未建立自訂用戶 `accusys`
|
||||
3. ACL 變更不會持久化(無 config file)
|
||||
|
||||
### 已執行修復
|
||||
|
||||
| 項目 | 修改 |
|
||||
|------|------|
|
||||
| `.env` | `redis://accusys:accusys@localhost:6379` → `redis://:accusys@localhost:6379` |
|
||||
|
||||
### 待解決問題
|
||||
|
||||
1. **ACL 持久化**:Redis 啟動後手動建立的用戶不會保留(重啟後消失)
|
||||
2. **需配置 ACL 文件**:建議建立 `users.acl` 並在 plist 中指定
|
||||
|
||||
### 建議解決方案
|
||||
|
||||
#### 方案 A:使用默認用戶(現行)
|
||||
|
||||
```bash
|
||||
# .env
|
||||
REDIS_URL=redis://:accusys@localhost:6379
|
||||
```
|
||||
|
||||
**優點**:簡單,無需修改 Redis 配置
|
||||
**缺點**:所有應用共享默認用戶
|
||||
|
||||
#### 方案 B:建立 ACL 配置文件
|
||||
|
||||
```bash
|
||||
# 1. 創建 ACL 文件
|
||||
cat > /Users/accusys/momentry/etc/redis/users.acl << 'EOF'
|
||||
user default on sanitize-payload ~* &* +@all >accusys
|
||||
user accusys on sanitize-payload ~* &* +@all >accusys
|
||||
EOF
|
||||
|
||||
# 2. 修改 plist 添加 --aclfile 參數
|
||||
--aclfile /Users/accusys/momentry/etc/redis/users.acl
|
||||
|
||||
# 3. 重啟 Redis
|
||||
sudo launchctl unload /Library/LaunchDaemons/com.momentry.redis.plist
|
||||
sudo launchctl load /Library/LaunchDaemons/com.momentry.redis.plist
|
||||
```
|
||||
|
||||
**優點**:支持多用戶,ACL 持久化
|
||||
**缺點**:需修改 plist 並重啟
|
||||
|
||||
### 影響範圍
|
||||
|
||||
- `src/core/config.rs` - REDIS_URL 讀取
|
||||
- `src/core/db/redis_client.rs` - Redis 連線
|
||||
- `momentry api-key` 命令 - 異常告警
|
||||
|
||||
### 狀態
|
||||
|
||||
- [x] 已確認問題存在
|
||||
- [x] 已修改 `.env` 使用默認用戶
|
||||
- [ ] 待決定是否實施 ACL 方案
|
||||
|
||||
---
|
||||
|
||||
## 問題 #6: Momentry Core API 未開機自動啟動 (2026-03-23)
|
||||
|
||||
### 問題描述
|
||||
|
||||
Momentry Core API 服務 (`momentry server --port 3002`) 未設定 launchd,導致:
|
||||
1. 系統重啟後 API 服務不會自動啟動
|
||||
2. `api.momentry.ddns.net` 返回 502 Bad Gateway
|
||||
3. n8n workflow 呼叫 API 時失敗
|
||||
|
||||
### 發現過程
|
||||
|
||||
1. n8n workflow 呼叫 `https://api.momentry.ddns.net/api/v1/n8n/search` 返回 502
|
||||
2. 檢查發現 port 3002 無服務運行
|
||||
3. Caddy 配置正向確,但後端服務未啟動
|
||||
4. 手動啟動服務後 API 正常運作
|
||||
|
||||
### 配置需求
|
||||
|
||||
| 項目 | 值 |
|
||||
|------|-----|
|
||||
| 服務名稱 | `com.momentry.api` |
|
||||
| 二進位檔 | `/Users/accusys/momentry_core_0.1/target/release/momentry` |
|
||||
| 命令 | `server --port 3002` |
|
||||
| Port | 3002 |
|
||||
| 環境變數 | `DATABASE_URL`, `REDIS_URL` 等 |
|
||||
|
||||
### 待辦事項
|
||||
|
||||
- [x] 建立 `docs_v1.0/IMPLEMENTATION/INSTALL_MOMENTRY_API.md` 安裝文件
|
||||
- [x] 建立 `/Library/LaunchDaemons/com.momentry.api.plist`
|
||||
- [x] 設定環境變數 (`DATABASE_URL`, `REDIS_URL` 等)
|
||||
- [x] 測試 launchctl load/unload
|
||||
- [x] 驗證開機自動啟動 (launchd 載入成功)
|
||||
|
||||
### 完成日期
|
||||
2026-03-23
|
||||
|
||||
### 參考文件
|
||||
|
||||
- `/Library/LaunchDaemons/com.momentry.n8n.main.plist` - n8n plist 範例
|
||||
- `docs_v1.0/IMPLEMENTATION/INSTALL_N8N.md` - plist 配置說明
|
||||
|
||||
---
|
||||
|
||||
## 服務統一遷移 (2026-03-24)
|
||||
|
||||
### 問題描述
|
||||
|
||||
Reboot 後發現 n8n workflow 數量從 42 變成 41,確認是 PostgreSQL 資料庫問題。經過調查發現:
|
||||
|
||||
1. **兩組不同的 PostgreSQL 資料目錄**:
|
||||
- Homebrew plist: `/opt/homebrew/var/postgresql@18` (有最新資料)
|
||||
- Custom plist: `/Users/accusys/momentry/var/postgresql` (可能是舊資料)
|
||||
|
||||
2. **Reboot 時 custom plist 搶先啟動**,使用了錯誤的資料目錄
|
||||
|
||||
### 解決方案
|
||||
|
||||
1. **統一使用 custom plist**:
|
||||
- 刪除 homebrew plist (`~/Library/LaunchAgents/homebrew.mxcl.postgresql@18.plist`)
|
||||
- Custom plist 使用 `/Users/accusys/momentry/var/postgresql` 作為資料目錄
|
||||
- 將所有 14 個服務的 plist 註冊到 launchd
|
||||
|
||||
2. **所有已遷移的服務**:
|
||||
|
||||
| 服務 | Plist | 資料目錄 |
|
||||
|------|-------|----------|
|
||||
| PostgreSQL | ✅ | `/Users/accusys/momentry/var/postgresql` |
|
||||
| MariaDB | ✅ | `/Users/accusys/momentry/var/mariadb` |
|
||||
| MongoDB | ✅ | `/opt/homebrew/var/mongodb` |
|
||||
| Redis | ✅ | - |
|
||||
| Ollama | ✅ | - |
|
||||
| Qdrant | ✅ | - |
|
||||
| n8n Main | ✅ | - |
|
||||
| n8n Worker | ✅ | - |
|
||||
| Caddy | ✅ | - |
|
||||
| SFTPGo | ✅ | - |
|
||||
| Gitea | ✅ | - |
|
||||
| Gitea MCP | ✅ | - |
|
||||
| PHP | ✅ | - |
|
||||
| Momentry API | ✅ | - |
|
||||
| RustDesk HBBR | ✅ | - |
|
||||
| RustDesk HBBS | ✅ | - |
|
||||
|
||||
### 還發現的 Homebrew 服務 (未遷移)
|
||||
|
||||
| 服務 | 建議 |
|
||||
|------|------|
|
||||
| homebrew.mxcl.grafana | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.prometheus | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.openwebui | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.kafka | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.seaweedfs | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.netdata | ⚠️ 考慮遷移 |
|
||||
| homebrew.mxcl.ddclient | ⚠️ 動態 DNS |
|
||||
| homebrew.mxcl.shadowsocks-rust | ⚠️ VPN |
|
||||
|
||||
### 預防措施
|
||||
|
||||
1. **確保統一資料目錄**:所有服務只使用一個資料目錄
|
||||
2. **Reboot 測試**:遷移完成後需進行 Reboot 測試
|
||||
3. **文件同步**:plist 檔案同步到 repo
|
||||
|
||||
### 完成日期
|
||||
2026-03-24
|
||||
|
||||
### 參考文件
|
||||
|
||||
- `docs_v1.0/REFERENCE/SERVICES.md` - 服務管理文檔
|
||||
- `docs_v1.0/IMPLEMENTATION/SERVICE_ADDITION_GUIDE.md` - 服務添加規範
|
||||
- `momentry_runtime/plist/` - plist 檔案存放位置
|
||||
|
||||
---
|
||||
|
||||
## Job Worker 實作 (2026-03-24)
|
||||
|
||||
### 目標
|
||||
|
||||
實作輪詢式 Job Worker,實現檔案註冊後自動觸發處理:
|
||||
|
||||
1. **輪詢機制**:Worker 定期輪詢 jobs 佇列
|
||||
2. **並行處理**:最多 2 個 processor 同時執行
|
||||
3. **失敗容忍**:任一模組獨立,失敗可接續
|
||||
|
||||
### 設計決策
|
||||
|
||||
| 項目 | 決策 | 理由 |
|
||||
|------|------|------|
|
||||
| 觸發方式 | 輪詢(Job Worker) | 暫無可靠 API 觸發 |
|
||||
| 並行處理 | 最多 2 個 | 可根據 CPU/GPU 調整 |
|
||||
| 失敗處理 | 獨立模組,部分完成可接續 | 任何模組失敗都產出狀態 |
|
||||
| Worker 啟動 | 獨立進程 | 隔離、易管理 |
|
||||
| 並行上限 | 環境變數 + 預設值 | 靈活調整 |
|
||||
| 狀態同步 | PostgreSQL + Redis | 可靠 + 即時 |
|
||||
|
||||
### 環境變數
|
||||
|
||||
| 變數 | 預設值 | 說明 |
|
||||
|------|--------|------|
|
||||
| `MOMENTRY_MAX_CONCURRENT` | 2 | 最大並行 processor 數 |
|
||||
| `MOMENTRY_POLL_INTERVAL` | 5 | 輪詢間隔(秒) |
|
||||
| `MOMENTRY_WORKER_ENABLED` | true | 是否啟用 worker |
|
||||
|
||||
### 實作計畫
|
||||
|
||||
詳細內容請參考 [JOB_WORKER_IMPLEMENTATION_PLAN.md](./JOB_WORKER_IMPLEMENTATION_PLAN.md)
|
||||
|
||||
### Phase 規劃
|
||||
|
||||
| Phase | 任務 | 預估工時 |
|
||||
|-------|------|----------|
|
||||
| 1 | 資料庫遷移 | 2h |
|
||||
| 2 | Worker 框架 | 4h |
|
||||
| 3 | Register API 整合 | 2h |
|
||||
| 4 | Processor 執行 | 4h |
|
||||
| 5 | 進度追蹤 | 2h |
|
||||
| 6 | API 端點 | 3h |
|
||||
| 7 | CLI 命令 | 2h |
|
||||
| 8 | 測試 | 4h |
|
||||
|
||||
**總預估**: ~23h
|
||||
|
||||
### 實作結構
|
||||
|
||||
```
|
||||
src/
|
||||
├── worker/
|
||||
│ ├── mod.rs # Worker 模組導出
|
||||
│ ├── config.rs # Worker 配置
|
||||
│ ├── worker.rs # Worker 主邏輯
|
||||
│ ├── processor.rs # Processor 執行器
|
||||
│ ├── queue.rs # Job 佇列管理
|
||||
│ └── progress.rs # 進度追蹤
|
||||
├── api/
|
||||
│ └── server.rs # 更新 Register API
|
||||
└── main.rs # 新增 worker 命令
|
||||
```
|
||||
|
||||
### 狀態
|
||||
|
||||
- [x] 系統分析完成
|
||||
- [x] 實作計畫文件建立
|
||||
- [ ] Phase 1: 資料庫遷移
|
||||
- [ ] Phase 2: Worker 框架
|
||||
- [ ] Phase 3: Register API 整合
|
||||
- [ ] Phase 4: Processor 執行
|
||||
- [ ] Phase 5-8: 依序實作
|
||||
|
||||
### 參考文件
|
||||
|
||||
- `docs_v1.0/ARCHITECTURE/JOB_WORKER_IMPLEMENTATION_PLAN.md` - 完整實作計畫
|
||||
- `docs_v1.0/ARCHITECTURE/PROCESSING_PIPELINE.md` - 處理流程
|
||||
- `docs_v1.0/REFERENCE/MOMENTRY_CORE_REDIS_KEYS.md` - Redis Key 設計
|
||||
|
||||
---
|
||||
|
||||
## 統一會員系統 + 影片歸屬追蹤 (2026-03-24)
|
||||
|
||||
### 目標
|
||||
|
||||
建立統一的會員系統:
|
||||
1. WordPress 作為唯一登入入口
|
||||
2. 每個影片關聯到 user_id(追蹤歸屬)
|
||||
3. Per-user 配額管理
|
||||
4. API 端點啟用認證
|
||||
|
||||
### 實作計畫
|
||||
|
||||
詳細內容請參考 [USER_MANAGEMENT_PLAN.md](./USER_MANAGEMENT_PLAN.md)
|
||||
|
||||
### Phase 規劃
|
||||
|
||||
| Phase | 任務 | 複雜度 | 優先級 | 預估工時 |
|
||||
|-------|------|--------|--------|----------|
|
||||
| 1 | WordPress Application Passwords 測試 | 低 | P0 | 1.5h |
|
||||
| 2 | 資料庫遷移 (users 表) | 中 | P0 | 3h |
|
||||
| 3 | API auth middleware | 中 | P0 | 4h |
|
||||
| 4 | Register API 更新 | 低 | P0 | 2h |
|
||||
| 5 | Admin users API | 中 | P1 | 4h |
|
||||
| 6 | n8n workflow | 中 | P1 | 6h |
|
||||
| 7 | 配額管理 | 中 | P2 | 4h |
|
||||
| 8 | 測試驗證 | 中 | P2 | 4h |
|
||||
|
||||
**總預估**: ~28.5h
|
||||
|
||||
### 待確認事項
|
||||
|
||||
- [ ] WordPress 用戶建立方式(手動/Elementor表單)
|
||||
- [ ] API Key 格式確認
|
||||
- [ ] SFTPGo 整合方式
|
||||
- [ ] 配額管理策略
|
||||
- [ ] 用戶刪除同步流程
|
||||
|
||||
### 狀態
|
||||
|
||||
- [x] 系統分析完成
|
||||
- [x] 實作計畫文件建立
|
||||
- [ ] Phase 1: WordPress 認證測試
|
||||
- [ ] Phase 2: 資料庫遷移
|
||||
- [ ] Phase 3-8: 依序實作
|
||||
|
||||
### 參考文件
|
||||
|
||||
- `docs_v1.0/ARCHITECTURE/USER_MANAGEMENT_PLAN.md` - 完整實作計畫
|
||||
- `docs_v1.0/REFERENCE/API_KEY_MANAGEMENT.md` - API Key 管理
|
||||
- `docs_v1.0/IMPLEMENTATION/SFTPGO_DEMO_USER.md` - SFTPGo 用戶設定
|
||||
@@ -0,0 +1,150 @@
|
||||
# Phase 1 Completion Report — v2 (fine-grained ASRX)
|
||||
|
||||
**File**: Charade (1963) Cary Grant & Audrey Hepburn
|
||||
**UUID**: `aeed71342a899fe4b4c57b7d41bcb692`
|
||||
**Date**: 2026-05-10
|
||||
**System**: M5 (MacBook Pro, 48GB, Apple Silicon)
|
||||
|
||||
---
|
||||
|
||||
## 1. Processor Outputs
|
||||
|
||||
| File | Size | Description |
|
||||
|------|------|-------------|
|
||||
| `asr.json` | 413KB | 3,417 segments, full movie coverage (Whisper small) |
|
||||
| `asrx.json` | **18MB** | **4,188 segments** (fine-grained, ECAPA-TDNN) |
|
||||
| `asrx_fine.json` | 45MB | 4,188 fine segments + voice embeddings (intermediate) |
|
||||
| `cut.json` | 329KB | 2,260 scenes |
|
||||
| `yolo.json` | 181MB | 169,625 frames with object detections |
|
||||
| `face.json` | **106MB** | 4,550 frames, 5,910 faces @ 8Hz (CoreML 512D) |
|
||||
| `face_traced.json` | 110MB | Traced faces with 423 identity traces |
|
||||
| `lip.json` | 492KB | Lip openness analysis |
|
||||
| `ocr.json` | 277KB | 606 OCR frames |
|
||||
| `pose.json` | 26MB | 4,211 pose frames |
|
||||
| `scene.json` | 403B | Scene classification |
|
||||
|
||||
## 2. Pipeline 8-Stage Checklist
|
||||
|
||||
| Stage | Status | Detail |
|
||||
|-------|--------|--------|
|
||||
| ASR | ✅ | 3,417 segments, last end 6,773s (100%) |
|
||||
| ASRX | ✅ | **4,188 segments** (fine-grained, 10→3 speakers mapped) |
|
||||
| Sentence Chunks | ✅ | **4,188 sentence chunks** with yolo_objects + face_ids |
|
||||
| Vectorization | ✅ | 4,188 Qdrant (768D), all 3 collections updated |
|
||||
| Face Trace | ✅ | 423 traces, 11,820 detections @ 8Hz |
|
||||
| TKG Graph | ✅ | 498 nodes, 1,617 edges |
|
||||
| Trace Chunks | ✅ | 423 trace chunks |
|
||||
| Phase 1 Release | ✅ | 3.0GB package |
|
||||
|
||||
## 3. Speaker Identification
|
||||
|
||||
### ASRX Enhancement (3417 → 4188 segments)
|
||||
|
||||
The original Whisper ASR merges rapid back-and-forth dialogue into single segments. A sliding-window ECAPA-TDNN approach was developed to detect speaker change points within each ASR segment:
|
||||
|
||||
1. **Sliding window**: 1.5s window, 0.75s stride across full audio
|
||||
2. **ECAPA-TDNN 192D embedding** per window
|
||||
3. **Classification** against reference centroids (Cary Grant, Audrey Hepburn, Unknown)
|
||||
4. **Majority-vote smoothing** over 3 adjacent windows
|
||||
5. **Change point detection** where classified speaker changes
|
||||
6. **Split** original ASR segment at each change point
|
||||
|
||||
**Result**: 3,417 → **4,188 segments** (+771, +22.6%). Validated via gender classification (ECAPA-TDNN → 92.3% agreement with character identity).
|
||||
|
||||
### Speaker Mapping (Centroid-based)
|
||||
|
||||
| Speaker ID | Name | Segments | Duration | Voice Gender |
|
||||
|------------|------|----------|----------|-------------|
|
||||
| SPEAKER_0 | Audrey Hepburn | 1,658 | 2,786s | FEMALE |
|
||||
| SPEAKER_1 | Cary Grant | 2,033 | 3,962s | MALE |
|
||||
| SPEAKER_2 | Unknown (minor) | 497 | 806s | MIXED |
|
||||
|
||||
Method: Reference centroids built from 3,107 known segments (1,420 Cary + 1,689 Audrey). Each fine segment classified by cosine similarity to nearest centroid. No cross-contamination between speaker clusters.
|
||||
|
||||
### Gender Validation
|
||||
|
||||
Two small clusters (SPEAKER_5: 10 segs, SPEAKER_9: 10 segs) initially showed MALE voice → Audrey assignment. Video clip verification confirmed these are segments where a male voice speaks while Audrey is on screen (old face-based matching was incorrect). The fine-grained segmentation correctly resolves these.
|
||||
|
||||
## 4. Sentence Chunks — Full Migration
|
||||
|
||||
All 4,188 fine segments were written to `dev.chunks` with complete data per chunk:
|
||||
|
||||
| Chunk Field | Value | Source |
|
||||
|-------------|-------|--------|
|
||||
| `start_time`/`end_time` | Fine segment boundaries | `asrx_fine.json` |
|
||||
| `start_frame`/`end_frame` | time × 25fps | Calculated |
|
||||
| `content` | `{data: {text, text_normalized}, rule: rule_1}` | ASR text |
|
||||
| `metadata.yolo_objects` | Dedup class names in frame range | `pre_chunks(yolo)` |
|
||||
| `metadata.face_ids` | Trace IDs in frame range | `face_detections` |
|
||||
| `metadata.speaker_name` | Centroid-matched identity | `asrx_fine.json` |
|
||||
|
||||
- 4,158/4,188 chunks have YOLO objects (avg 3-5 object classes)
|
||||
- 398/4,188 chunks have face IDs (face data covers first ~12 min only)
|
||||
|
||||
### Parent/Story Chunks
|
||||
|
||||
| Metric | Before (v1) | After (v2) |
|
||||
|--------|-------------|------------|
|
||||
| Children per parent | 15 (fixed) | 15 (fixed) |
|
||||
| Total parents | 228 | **280** |
|
||||
| LLM summaries | 228 (Gemma4) | **280** (Gemma4, regenerated) |
|
||||
| Qdrant stories | 456 pts | **560 pts** |
|
||||
|
||||
## 5. Qdrant Vector Collections
|
||||
|
||||
| Collection | Dims | Points | Content | Status |
|
||||
|-----------|------|--------|---------|--------|
|
||||
| `momentry_dev_v1` | 768 | **4,188** | Sentence chunk embeddings (EmbeddingGemma) | ✅ |
|
||||
| `momentry_dev_stories` | 768 | **560** | 280 dialogue + 280 LLM summary | ✅ |
|
||||
| `momentry_dev_faces` | 512 | 5,910 | Face embeddings (8Hz CoreML) | ✅ |
|
||||
| `momentry_dev_voice` | 192 | **4,188** | Voice embeddings (ECAPA-TDNN) | ✅ |
|
||||
| `sentence_story` | 768 | **4,188** | Sentence template with speaker | ✅ |
|
||||
| `sentence_summary` | 768 | **4,188** | Context-aware LLM sentence summary | ✅ |
|
||||
|
||||
## 6. ASR Model Selection
|
||||
|
||||
A comprehensive benchmark (5 models × 2 VAD settings × 3 test clips = 30 runs) showed:
|
||||
|
||||
| Model | Segments | Chars | Runtime | Verdict |
|
||||
|-------|----------|-------|---------|---------|
|
||||
| tiny | 56 avg | 1,730 | **9.2s** | Most segments, best text capture |
|
||||
| **small** | **55 avg** | **1,704** | **17.6s** | **Best balance (current)** |
|
||||
| base | 42 avg | 1,751 | 10.1s | Good but fewer segments |
|
||||
| medium | 52 avg | 1,627 | 339.6s | Slow, loses text |
|
||||
| large-v3 | 20 avg | 1,249 | 68.8s | **Worst**: merges utterances, loses 26% text |
|
||||
|
||||
**Conclusion**: Keep `faster-whisper small (VAD 500ms)`. The missing-text problem is not solvable by model size — even tiny captures more text than large-v3. Root cause is Whisper's lack of speaker turn detection in segment boundary logic, which is solved by the sliding-window ASRX approach above.
|
||||
|
||||
## 7. Release Package
|
||||
|
||||
| Component | Size |
|
||||
|-----------|------|
|
||||
| `output_json/` | 13 processor files |
|
||||
| `chunks.csv` | 3.2MB |
|
||||
| `vectors.csv` | 58MB |
|
||||
| `identities.csv` | 1MB |
|
||||
| `schema.sql` | 30KB |
|
||||
| Qdrant snapshots (5 collections) | ~3GB |
|
||||
| `RELEASE_INFO.txt` | Metadata |
|
||||
| **Total** | **~3.0GB** |
|
||||
|
||||
## 8. Key Technical Decisions
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Sliding window 1.5s/0.75s | Optimal balance: captures turn boundaries without over-splitting |
|
||||
| Centroid-based classification | 0.8+ similarity, no retraining needed, 100% consistent |
|
||||
| Word-timestamp ASR for text | Re-run with `word_timestamps=True`, 87% coverage; remaining 13% → per-segment ASR fallback |
|
||||
| Fixed 15 children/parent | Maintains Phase 1 design consistency |
|
||||
| `yolo_objects` dedup | Only class names stored per chunk (not per-frame) |
|
||||
| `face_ids` via `trace_id` | `face_id` column is NULL in DB; `trace_id` is the actual identifier |
|
||||
| Keep ASR small model | Benchmarked 5 models; larger models lose text, not gain it |
|
||||
| `app.run(threaded=True)` | Dashboard v2: single-threaded Flask was blocking on subprocess calls |
|
||||
|
||||
## 9. Phase 2 Preparation
|
||||
|
||||
Pending for Phase 2:
|
||||
- Rule 3 scene chunking (cut-based parent chunks)
|
||||
- 5W1H Agent (LLM-generated scene summaries)
|
||||
- Full pipeline + 5W1H release packaging
|
||||
- Source separation (Demucs/HPSS) for overlapping speech scenarios
|
||||
@@ -0,0 +1,63 @@
|
||||
# Phase 1 Release Checklist
|
||||
|
||||
**UUID**: `aeed71342a899fe4b4c57b7d41bcb692`
|
||||
**Model**: v2 (fine-grained ASRX, 4,188 segments)
|
||||
**Date**: 2026-05-10
|
||||
|
||||
## 1. Processor Outputs
|
||||
|
||||
- [x] `asr.json` — faster-whisper small, 3,417 segments
|
||||
- [x] `asrx.json` — ECAPA-TDNN fine-grained, 4,188 segments
|
||||
- [x] `cut.json` — 2,260 scene cuts
|
||||
- [x] `yolo.json` — 169,625 frames, object detections
|
||||
- [x] `face.json` — 4,550 frames, 5,910 faces @ 8Hz
|
||||
- [x] `face_traced.json` — 423 traced identities
|
||||
- [x] `lip.json` — Lip openness per ASRX segment
|
||||
- [x] `ocr.json` — 606 OCR frames
|
||||
- [x] `pose.json` — 4,211 pose frames
|
||||
- [x] `scene.json` — Scene classification
|
||||
|
||||
## 2. Pipeline Stages
|
||||
|
||||
- [x] ASR: 3,417 segments, full movie
|
||||
- [x] ASRX: 4,188 segments (fine-grained), 3 speakers
|
||||
- [x] Sentence chunks: 4,188 in `dev.chunks`
|
||||
- [x] Vectorization: 4,188 in Qdrant `momentry_dev_v1`
|
||||
- [x] Face trace: 423 traces, 11,820 detections
|
||||
- [x] TKG: 498 nodes, 1,617 edges
|
||||
- [x] Trace chunks: 423 in `dev.chunks`
|
||||
- [x] All 8 stages passing
|
||||
|
||||
## 3. Qdrant Collections
|
||||
|
||||
- [x] `momentry_dev_v1` — 4,188 pts, 768D (EmbeddingGemma)
|
||||
- [x] `momentry_dev_stories` — 560 pts, 768D (280 dialogue + 280 summary)
|
||||
- [x] `momentry_dev_faces` — 5,910 pts, 512D (CoreML FaceNet)
|
||||
- [x] `momentry_dev_voice` — 4,188 pts, 192D (ECAPA-TDNN)
|
||||
- [x] `sentence_story` — 4,188 pts, 768D (sentence template)
|
||||
- [x] `sentence_summary` — 4,188 pts, 768D (context-aware LLM)
|
||||
|
||||
## 4. Database (dev.chunks)
|
||||
|
||||
- [x] Sentence chunks: 4,188 with speaker_name, speaker_id
|
||||
- [x] Story chunks: 280 with LLM summaries
|
||||
- [x] Cut chunks: 1,130
|
||||
- [x] Trace chunks: 423
|
||||
- [x] YOLO objects in metadata: 4,158/4,188
|
||||
- [x] Face IDs in metadata: 398/4,188
|
||||
- [x] Parent-child relationships set
|
||||
|
||||
## 5. Speaker Mapping
|
||||
|
||||
- [x] SPEAKER_0 → Audrey Hepburn (1,658 segs, gender FEMALE ✅)
|
||||
- [x] SPEAKER_1 → Cary Grant (2,033 segs, gender MALE ✅)
|
||||
- [x] SPEAKER_2 → Unknown (497 segs, minor characters)
|
||||
- [x] Voice embeddings validated via gender classification
|
||||
|
||||
## 6. Release Package
|
||||
|
||||
- [x] Phase 1 release packaged at `release/phase1/latest/`
|
||||
- [x] Qdrant snapshots for all 5 collections
|
||||
- [x] `chunks.csv`, `vectors.csv`, `identities.csv` exported
|
||||
- [x] `schema.sql` from PostgreSQL
|
||||
- [x] Dashboard v2 running at port 5050
|
||||
@@ -0,0 +1,111 @@
|
||||
# Release Notes — v1.0.0 (Production 3002)
|
||||
|
||||
**Date**: 2026-05-13
|
||||
**Build**: `301da08`
|
||||
**Deployed by**: M4
|
||||
|
||||
---
|
||||
|
||||
## Release Scope
|
||||
|
||||
### Binaries
|
||||
|
||||
| Binary | Size | Source |
|
||||
|--------|:----:|--------|
|
||||
| `momentry` (production) | 21 MB | M5 build |
|
||||
| `release` CLI | 3.5 MB | M5 build |
|
||||
|
||||
### File Packages
|
||||
|
||||
| Package | UUID | Content |
|
||||
|---------|------|---------|
|
||||
| Charade (HD) | `aeed71342a899fe4b4c57b7d41bcb692` | 1920×1080, 25fps |
|
||||
| Charade (YouTube) | `23b1c872379d4ec06479e5ed39eef4c5` | 640×360, 23.98fps |
|
||||
|
||||
### Data Per Package
|
||||
|
||||
| Table | HD | YouTube |
|
||||
|-------|:--:|:--:|
|
||||
| chunk | 2,407 | 2,340 |
|
||||
| chunk_vectors (768D) | 2,407 | 2,340 |
|
||||
| face_detections | 70,691 | 70,729 |
|
||||
| identities (TMDB) | 15 actors | 280 clusters |
|
||||
| identity_bindings | 18,635 | 18,635 |
|
||||
| tkg_nodes | 6,457 | 5,776 |
|
||||
| tkg_edges | 21,028 | 18,847 |
|
||||
|
||||
### TMDB Matched Actors
|
||||
|
||||
| Actor | TMDB ID | Package |
|
||||
|-------|:------:|:--:|
|
||||
| Cary Grant | 2638 | HD |
|
||||
| Audrey Hepburn | 1932 | HD |
|
||||
| James Coburn | 5563 | HD |
|
||||
| George Kennedy | 12950 | HD |
|
||||
| Dominique Minot | 41714 | HD |
|
||||
| Ned Glass | 18870 | HD |
|
||||
| Jacques Marin | 26890 | HD |
|
||||
| Paul Bonifas | 41716 | HD |
|
||||
|
||||
---
|
||||
|
||||
## Schema Changes
|
||||
|
||||
| Change | Schema |
|
||||
|--------|--------|
|
||||
| `chunks` → `chunk` (rename) | public |
|
||||
| Drop `old_chunk_id`, `chunk_index` | public |
|
||||
| Add `timestamp_secs` to `face_detections` | public |
|
||||
| Add `file_uuid` to `identities` | public |
|
||||
| Drop `chunk_vectors_chunk_id_key` (duplicate unique) | public |
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
### API (all 200)
|
||||
|
||||
| Category | Endpoints | Result |
|
||||
|----------|-----------|:--:|
|
||||
| Health | `/health`, `/health/detailed` | ✅ |
|
||||
| Auth | login, logout | ✅ |
|
||||
| Files | list, scan, detail, probe | ✅ |
|
||||
| Chunk | `/file/{uuid}/chunk/{id}` | ✅ |
|
||||
| Search | universal, frames, visual | ✅ |
|
||||
| Identities | list, detail, files, chunks | ✅ |
|
||||
| Face Trace | sortby, faces, 3D | ✅ |
|
||||
| Media | video, thumbnail, trace video | ✅ |
|
||||
| Resources | list | ✅ |
|
||||
|
||||
### Database
|
||||
|
||||
| Table | Count |
|
||||
|-------|------:|
|
||||
| Files registered | 38 |
|
||||
| TMDB identities | 15 |
|
||||
| Total chunks (both files) | 4,747 |
|
||||
| Total faces (both files) | 141,420 |
|
||||
| Total TKG nodes | 12,233 |
|
||||
| Total TKG edges | 39,875 |
|
||||
|
||||
---
|
||||
|
||||
## Deployment Process
|
||||
|
||||
1. **Backup**: Full `public` schema dump (1.3 GB)
|
||||
2. **Schema Migration**: `chunks→chunk`, drop deprecated columns, add new columns
|
||||
3. **Deploy aeed7134**: All 8 tables imported via `sed dev.→public.` per-table SQL files
|
||||
4. **Deploy 23b1c87**: Fixed `chunk_vectors_chunk_id_key` constraint conflict, all 8 tables imported
|
||||
5. **Binary Swap**: Old binary stopped, M5 `momentry_v1.0.0` deployed, restarted on 3002
|
||||
6. **Verification**: All API endpoints 200, both files queryable
|
||||
|
||||
---
|
||||
|
||||
## Known Notes
|
||||
|
||||
| Item | Note |
|
||||
|------|------|
|
||||
| `/files` status | API hardcodes `"ready"`, does not reflect actual DB status (reported to M5) |
|
||||
| `/files/scan` | Only scans video extensions (mp4/mov/mkv/avi/webm), misses jpg/png |
|
||||
| deploy.sh schema | Uses `sed dev.→public.` for public deployment (pending M5 native SCHEMA support) |
|
||||
| chunk_vectors constraint | `chunk_vectors_chunk_id_key` dropped from public (was preventing multi-file import) |
|
||||
@@ -0,0 +1,280 @@
|
||||
# Release SOP — Dev → Production (3002)
|
||||
|
||||
**Date**: 2026-05-13
|
||||
**Version**: 2.0
|
||||
**Status**: Draft — M5 Review Required
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
### 必須確認
|
||||
|
||||
| # | 檢查項 | 方法 | 門檻 |
|
||||
|:--:|------|------|:--:|
|
||||
| 1 | Dev (3003) 39/39 API test | `bash api_test.sh` | ✅ 39/39 |
|
||||
| 2 | 檔案內容包匯入成功 | 9 tables all > 0 | ✅ 通過 |
|
||||
| 3 | TMDB identities 匹配 | `SELECT * FROM dev.identities WHERE source='tmdb'` | ✅ 7 actors |
|
||||
| 4 | TKG 到位 | tkg_nodes + tkg_edges > 0 | ✅ 通過 |
|
||||
| 5 | Release binary 已編譯 | `cargo build --release --bin momentry` | ✅ 成功 |
|
||||
| 6 | `.env.development` 切換確認 | `DATABASE_SCHEMA=public` | ✅ |
|
||||
|
||||
|
||||
## 2. Backup
|
||||
|
||||
### 2.1 Database
|
||||
|
||||
```bash
|
||||
BACKUP_DIR="/Users/accusys/momentry_core_releases/backup_$(date +%Y%m%d_%H%M)"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Full public schema dump
|
||||
pg_dump -U accusys -d momentry --schema=public \
|
||||
--no-owner --no-acl \
|
||||
> "$BACKUP_DIR/public_schema_full.sql"
|
||||
|
||||
# Data-only dump (smaller, faster restore)
|
||||
pg_dump -U accusys -d momentry --schema=public \
|
||||
--data-only --no-owner --no-acl \
|
||||
> "$BACKUP_DIR/public_schema_data.sql"
|
||||
|
||||
echo "Backup: $BACKUP_DIR"
|
||||
```
|
||||
|
||||
### 2.2 Binary
|
||||
|
||||
```bash
|
||||
cp /path/to/release/momentry "$BACKUP_DIR/momentry_$(date +%Y%m%d)"
|
||||
```
|
||||
|
||||
### 2.3 Release Info
|
||||
|
||||
```bash
|
||||
echo "Release: v1.0.0" > "$BACKUP_DIR/RELEASE_INFO.txt"
|
||||
echo "Date: $(date)" >> "$BACKUP_DIR/RELEASE_INFO.txt"
|
||||
echo "Binary: $(ls -la target/release/momentry)" >> "$BACKUP_DIR/RELEASE_INFO.txt"
|
||||
echo "Schema: public" >> "$BACKUP_DIR/RELEASE_INFO.txt"
|
||||
```
|
||||
|
||||
|
||||
## 3. Schema Migration
|
||||
|
||||
### 3.1 Apply Migration
|
||||
|
||||
```sql
|
||||
-- Rename chunks → chunk (if not already done)
|
||||
ALTER TABLE IF EXISTS public.chunks RENAME TO chunk;
|
||||
|
||||
-- Drop deprecated columns
|
||||
ALTER TABLE public.chunk DROP COLUMN IF EXISTS old_chunk_id;
|
||||
ALTER TABLE public.chunk DROP COLUMN IF EXISTS chunk_index;
|
||||
|
||||
-- Add new columns (v1.0.3+)
|
||||
ALTER TABLE public.face_detections ADD COLUMN IF NOT EXISTS timestamp_secs float8;
|
||||
```
|
||||
|
||||
### 3.2 Verify Schema
|
||||
|
||||
```sql
|
||||
SELECT column_name, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'chunk'
|
||||
ORDER BY ordinal_position;
|
||||
```
|
||||
|
||||
Expected: 24 columns, no `old_chunk_id`, no `chunk_index`.
|
||||
|
||||
|
||||
## 4. Deploy Package
|
||||
|
||||
### 4.1 Deploy via deploy.sh
|
||||
|
||||
```bash
|
||||
cd path/to/package/
|
||||
PG_BIN="/opt/homebrew/opt/postgresql@18/bin" \
|
||||
DB_NAME="momentry" DB_USER="accusys" \
|
||||
DATABASE_SCHEMA=public \
|
||||
bash deploy.sh
|
||||
```
|
||||
|
||||
### 4.2 Manual Verification (after deploy)
|
||||
|
||||
```sql
|
||||
SELECT 'chunk' as tbl, count(*) FROM public.chunk WHERE file_uuid = '{uuid}';
|
||||
SELECT 'identities (tmdb)', count(*) FROM public.identities WHERE source = 'tmdb';
|
||||
SELECT 'tkg_nodes', count(*) FROM public.tkg_nodes WHERE file_uuid = '{uuid}';
|
||||
SELECT 'tkg_edges', count(*) FROM public.tkg_edges WHERE file_uuid = '{uuid}';
|
||||
```
|
||||
|
||||
Expected: chunk > 0, identities > 0, tkg_nodes > 0, tkg_edges > 0.
|
||||
|
||||
### 4.3 Set Status
|
||||
|
||||
```sql
|
||||
UPDATE public.videos SET status = 'completed' WHERE file_uuid IN (
|
||||
'aeed71342a899fe4b4c57b7d41bcb692',
|
||||
'23b1c872379d4ec06479e5ed39eef4c5'
|
||||
);
|
||||
```
|
||||
|
||||
|
||||
## 5. Binary Swap & Restart
|
||||
|
||||
### 5.1 Stop Old Server
|
||||
|
||||
```bash
|
||||
# Graceful stop
|
||||
pkill -TERM momentry
|
||||
sleep 5
|
||||
|
||||
# Force if needed
|
||||
pkill -9 momentry 2>/dev/null
|
||||
```
|
||||
|
||||
### 5.2 Deploy New Binary
|
||||
|
||||
```bash
|
||||
cp target/release/momentry /path/to/production/binary
|
||||
chmod +x /path/to/production/binary
|
||||
```
|
||||
|
||||
### 5.3 Start New Server
|
||||
|
||||
```bash
|
||||
DATABASE_SCHEMA=public /path/to/production/binary server --port 3002
|
||||
# or with .env
|
||||
MOMENTRY_SERVER_PORT=3002 DATABASE_SCHEMA=public cargo run --release -- server
|
||||
```
|
||||
|
||||
### 5.4 Health Check
|
||||
|
||||
```bash
|
||||
curl http://localhost:3002/health
|
||||
# Expected: {"status":"ok","version":"1.0.0","build_git_hash":"..."}
|
||||
```
|
||||
|
||||
|
||||
## 6. Verification
|
||||
|
||||
### 6.1 API Tests
|
||||
|
||||
```bash
|
||||
API_KEY="muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69"
|
||||
HOST="http://localhost:3002"
|
||||
|
||||
# Health
|
||||
curl "$HOST/health"
|
||||
curl "$HOST/health/detailed"
|
||||
|
||||
# Files
|
||||
curl -H "X-API-Key: $API_KEY" "$HOST/api/v1/files?page=1&page_size=5"
|
||||
curl -H "X-API-Key: $API_KEY" "$HOST/api/v1/files/scan"
|
||||
|
||||
# Search
|
||||
curl -X POST -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"query":"Audrey Hepburn","uuid":"aeed71342a899fe4b4c57b7d41bcb692","limit":3}' \
|
||||
"$HOST/api/v1/search/universal"
|
||||
|
||||
# Identities
|
||||
curl -H "X-API-Key: $API_KEY" "$HOST/api/v1/identities?page=1&page_size=5"
|
||||
|
||||
# Chunk detail (v1.0.3+)
|
||||
curl -H "X-API-Key: $API_KEY" "$HOST/api/v1/file/aeed71342a899fe4b4c57b7d41bcb692/chunk/0"
|
||||
|
||||
# Face trace
|
||||
curl -X POST -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
|
||||
-d '{"sort_by":"face_count","limit":3}' \
|
||||
"$HOST/api/v1/file/aeed71342a899fe4b4c57b7d41bcb692/face_trace/sortby"
|
||||
|
||||
# Trace video
|
||||
curl -o /dev/null -w "%{http_code}" -H "X-API-Key: $API_KEY" \
|
||||
"$HOST/api/v1/file/aeed71342a899fe4b4c57b7d41bcb692/trace/1934/video?padding=1"
|
||||
# Expected: 200
|
||||
|
||||
# Full test suite
|
||||
bash api_test.sh
|
||||
# Expected: 39/39
|
||||
```
|
||||
|
||||
### 6.2 Portal Check
|
||||
|
||||
| Page | URL | Expect |
|
||||
|------|-----|--------|
|
||||
| FilesView | `http://localhost:1420/files` | Charade = ✅ 已就绪 |
|
||||
| SearchView | `http://localhost:1420/search` | File dropdown has Charade |
|
||||
| Trace View | File → Traces | Face list visible |
|
||||
| Video Playback | Search → Play | Video plays |
|
||||
|
||||
### 6.3 Demo
|
||||
|
||||
```bash
|
||||
DEMO_BASE="http://localhost:3002" \
|
||||
DEMO_FILE="aeed71342a899fe4b4c57b7d41bcb692" \
|
||||
python3 scripts/demo_runner.py API_V1.0.0/DEMO_SCRIPT_v1.0.0.json --auto --speed 0.03
|
||||
|
||||
# Expected: 21/21 steps passed
|
||||
```
|
||||
|
||||
|
||||
## 7. Rollback
|
||||
|
||||
### 7.1 Binary Rollback
|
||||
|
||||
```bash
|
||||
pkill momentry
|
||||
cp "$BACKUP_DIR/momentry_$(date +%Y%m%d)" /path/to/production/binary
|
||||
DATABASE_SCHEMA=public /path/to/production/binary server --port 3002
|
||||
```
|
||||
|
||||
### 7.2 Schema Rollback
|
||||
|
||||
```sql
|
||||
-- Rename back if needed
|
||||
ALTER TABLE public.chunk RENAME TO chunks;
|
||||
```
|
||||
|
||||
### 7.3 Data Rollback
|
||||
|
||||
```bash
|
||||
psql -U accusys -d momentry < "$BACKUP_DIR/public_schema_full.sql"
|
||||
```
|
||||
|
||||
### 7.4 Downtime Estimation
|
||||
|
||||
| Step | Est. Time |
|
||||
|------|:--------:|
|
||||
| Backup | 2 min |
|
||||
| Schema migration | 1 min |
|
||||
| Package deploy | 5-10 min |
|
||||
| Binary swap | 1 min |
|
||||
| Verification | 5 min |
|
||||
| **Total** | **~15-20 min** |
|
||||
|
||||
|
||||
## 8. Decision Log
|
||||
|
||||
| # | Decision | Owner | Decision |
|
||||
|---|----------|:--:|------|
|
||||
| 1 | Production binary source | M5 | M5 提供 release binary or M4 自編譯 |
|
||||
| 2 | Schema: rename chunks→chunk? | M5 | M5 決定 public schema 結構 |
|
||||
| 3 | Identity merge strategy | M5 | Keep prod 15 TMDB + merge with dev data? |
|
||||
| 4 | Downtime window | M5 | 維護模式 (403 all) or hard stop? |
|
||||
| 5 | Release scope | M5 | `aeed7134` only or both HD + YouTube? |
|
||||
| 6 | .env / config for public schema | M5 | Production binary reads `DATABASE_SCHEMA=public` |
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Release Checklist
|
||||
|
||||
- [ ] Dev 3003 passed all tests (39/39 + demo 21/21)
|
||||
- [ ] Production old binary backed up
|
||||
- [ ] Production DB full backup
|
||||
- [ ] Schema migration SQL ready
|
||||
- [ ] Package deployed to public schema
|
||||
- [ ] Status set to completed
|
||||
- [ ] New binary deployed
|
||||
- [ ] Server restarted on 3002
|
||||
- [ ] Health check returns ok
|
||||
- [ ] 39/39 API tests pass on 3002
|
||||
- [ ] Portal shows files correctly
|
||||
- [ ] Demo runs 21/21 on 3002
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,504 @@
|
||||
# SFTPGo Demo 用戶指南
|
||||
|
||||
## Web 管理介面
|
||||
|
||||
**URL**: https://sftpgo.momentry.ddns.net
|
||||
|
||||
### 登入方式
|
||||
|
||||
| 角色 | 用戶名 | 密碼 |
|
||||
|------|--------|------|
|
||||
| **Demo 用戶** | `demo` | `demopassword123` |
|
||||
|
||||
### 可用功能
|
||||
|
||||
- 瀏覽個人目錄結構
|
||||
- 上傳、下載檔案
|
||||
- 查看上傳記錄
|
||||
|
||||
---
|
||||
|
||||
## 快速連線資訊
|
||||
|
||||
| 項目 | 值 |
|
||||
|------|-----|
|
||||
| **主機** | `sftpgo.momentry.ddns.net` |
|
||||
| **SFTP 連接埠** | `2022` |
|
||||
| **用戶名** | `demo` |
|
||||
| **密碼** | `demopassword123` |
|
||||
| **主目錄** | `/demo` |
|
||||
|
||||
---
|
||||
|
||||
## 連線方式
|
||||
|
||||
### 1. 命令列 SFTP
|
||||
|
||||
```bash
|
||||
# 使用密碼連線
|
||||
sshpass -p "demopassword123" sftp -P 2022 demo@sftpgo.momentry.ddns.net
|
||||
|
||||
# 使用金鑰連線 (需先設定)
|
||||
sftp -P 2022 -i ~/.ssh/id_rsa demo@sftpgo.momentry.ddns.net
|
||||
```
|
||||
|
||||
### 2. FileZilla
|
||||
|
||||
1. **主機**: `sftp://sftpgo.momentry.ddns.net`
|
||||
2. **連接埠**: `2022`
|
||||
3. **協定**: `SFTP`
|
||||
4. **登入類型**: `一般`
|
||||
5. **用戶名**: `demo`
|
||||
6. **密碼**: `demopassword123`
|
||||
|
||||
### 3. Cyberduck (macOS)
|
||||
|
||||
1. 選擇 **連線 > 新連線**
|
||||
2. 協定選擇 **SFTP (SSH File Transfer Protocol)**
|
||||
3. 伺服器: `sftpgo.momentry.ddns.net`
|
||||
4. 連接埠: `2022`
|
||||
5. 使用者名稱: `demo`
|
||||
6. 密碼: `demopassword123`
|
||||
|
||||
### 4. curl 上傳
|
||||
|
||||
```bash
|
||||
curl -u demo:demopassword123 \
|
||||
-T /path/to/video.mp4 \
|
||||
sftp://sftpgo.momentry.ddns.net:2022/demo/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SFTP 基本操作
|
||||
|
||||
### 連線後常用指令
|
||||
|
||||
```bash
|
||||
# 進入互動式模式
|
||||
sftp demo@sftpgo.momentry.ddns.net -P 2022
|
||||
|
||||
# 常用指令
|
||||
sftp> pwd # 顯示目前目錄
|
||||
sftp> ls # 列出檔案
|
||||
sftp> ls -la # 詳細列表
|
||||
sftp> cd uploads # 切換目錄
|
||||
sftp> mkdir videos # 建立目錄
|
||||
sftp> put local.mp4 # 上傳檔案
|
||||
sftp> get remote.mp4 # 下載檔案
|
||||
sftp> rm old.mp4 # 刪除檔案
|
||||
sftp> exit # 斷線
|
||||
```
|
||||
|
||||
### 批次上傳
|
||||
|
||||
```bash
|
||||
# 上傳多個檔案
|
||||
sshpass -p "demopassword123" sftp -P 2022 demo@sftpgo.momentry.ddns.net <<EOF
|
||||
cd uploads
|
||||
put video1.mp4
|
||||
put video2.mp4
|
||||
put video3.mp4
|
||||
bye
|
||||
EOF
|
||||
|
||||
# 使用 glob 上傳
|
||||
sshpass -p "demopassword123" sftp -P 2022 demo@sftpgo.momentry.ddns.net <<EOF
|
||||
mput /path/to/videos/*.mp4
|
||||
bye
|
||||
EOF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 自動上傳腳本
|
||||
|
||||
### Bash 腳本
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# upload.sh - 上傳視頻到 Momentry
|
||||
|
||||
HOST="sftpgo.momentry.ddns.net"
|
||||
PORT="2022"
|
||||
USER="demo"
|
||||
PASS="demopassword123"
|
||||
REMOTE_DIR="/demo/uploads"
|
||||
|
||||
# 要上傳的檔案
|
||||
FILE="$1"
|
||||
|
||||
if [ -z "$FILE" ]; then
|
||||
echo "用法: $0 <檔案路徑>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sshpass -p "$PASS" sftp -P $PORT $USER@$HOST <<EOF
|
||||
mkdir $REMOTE_DIR
|
||||
cd $REMOTE_DIR
|
||||
put "$FILE"
|
||||
bye
|
||||
EOF
|
||||
|
||||
echo "上傳完成: $FILE"
|
||||
```
|
||||
|
||||
使用方式:
|
||||
```bash
|
||||
chmod +x upload.sh
|
||||
./upload.sh /path/to/video.mp4
|
||||
```
|
||||
|
||||
### Python 腳本
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""上傳檔案到 Momentry SFTP"""
|
||||
|
||||
import paramiko
|
||||
import sys
|
||||
import os
|
||||
|
||||
def upload_file(local_path, remote_dir="/demo/uploads"):
|
||||
host = "sftpgo.momentry.ddns.net"
|
||||
port = 2022
|
||||
username = "demo"
|
||||
password = "demopassword123"
|
||||
|
||||
transport = paramiko.Transport((host, port))
|
||||
transport.connect(username=username, password=password)
|
||||
sftp = paramiko.SFTPClient.from_transport(transport)
|
||||
|
||||
filename = os.path.basename(local_path)
|
||||
remote_path = f"{remote_dir}/{filename}"
|
||||
|
||||
sftp.put(local_path, remote_path)
|
||||
print(f"已上傳: {filename} -> {remote_path}")
|
||||
|
||||
sftp.close()
|
||||
transport.close()
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("用法: python upload_sftp.py <檔案路徑>")
|
||||
sys.exit(1)
|
||||
|
||||
upload_file(sys.argv[1])
|
||||
```
|
||||
|
||||
安裝依賴:
|
||||
```bash
|
||||
pip install paramiko
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## WebDAV 替代方案
|
||||
|
||||
如果 SFTP 連線有問題,可使用 WebDAV:
|
||||
|
||||
| 項目 | 值 |
|
||||
|------|-----|
|
||||
| **URL** | `https://momentry.ddns.net/webdav/` |
|
||||
| **用戶名** | `demo` |
|
||||
| **密碼** | `demopassword123` |
|
||||
|
||||
### curl 使用 WebDAV
|
||||
|
||||
```bash
|
||||
# 上傳
|
||||
curl -u demo:demopassword123 \
|
||||
-T video.mp4 \
|
||||
"https://momentry.ddns.net/webdav/demo/uploads/"
|
||||
|
||||
# 下載
|
||||
curl -u demo:demopassword123 \
|
||||
-o video.mp4 \
|
||||
"https://momentry.ddns.net/webdav/demo/uploads/video.mp4"
|
||||
|
||||
# 列出目錄
|
||||
curl -u demo:demopassword123 \
|
||||
-X PROPFIND \
|
||||
"https://momentry.ddns.net/webdav/demo/" \
|
||||
-H "Depth: 1"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 連線被拒絕
|
||||
|
||||
```bash
|
||||
# 檢查 SFTPGo 是否運行
|
||||
curl -s http://localhost:8080/api/v2/status | jq .status
|
||||
|
||||
# 檢查連接埠
|
||||
nc -zv momentry.ddns.net 2022
|
||||
```
|
||||
|
||||
### 認證失敗
|
||||
|
||||
確認密碼是否正確:
|
||||
```bash
|
||||
# 測試認證
|
||||
curl -u demo:demopassword123 \
|
||||
"https://momentry.ddns.net/webdav/" -I
|
||||
```
|
||||
|
||||
### 權限不足
|
||||
|
||||
上傳目錄可能需要先建立:
|
||||
```bash
|
||||
sshpass -p "demopassword123" sftp -P 2022 demo@sftpgo.momentry.ddns.net <<EOF
|
||||
mkdir uploads
|
||||
mkdir videos
|
||||
bye
|
||||
EOF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 檔案上傳後自動化
|
||||
|
||||
上傳後,SFTPGo 會自動:
|
||||
1. 觸發 Hook 腳本
|
||||
2. 記錄上傳事件到 `/Users/accusys/sftpgo_test/hook.log`
|
||||
3. 呼叫 Momentry Core API 註冊視頻
|
||||
|
||||
查看上傳日誌:
|
||||
```bash
|
||||
tail -f /Users/accusys/sftpgo_test/hook.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 管理手冊
|
||||
|
||||
### 管理員帳戶
|
||||
|
||||
| 角色 | 用戶名 | 密碼 | 說明 |
|
||||
|------|--------|------|------|
|
||||
| **WebAdmin** | `admin` | `Test3200Test3200` | SFTPGo 管理介面 |
|
||||
|
||||
**WebAdmin URL**: https://sftpgo.momentry.ddns.net
|
||||
|
||||
### Admin 創建方式
|
||||
|
||||
根據官方文檔,SFTPGo 有兩種方式創建管理員:
|
||||
|
||||
#### 方式 1: Web UI (首次設定)
|
||||
|
||||
1. 訪問 `http://localhost:8080/web/admin`
|
||||
2. 如果沒有管理員,會顯示設定畫面
|
||||
3. 輸入用戶名和密碼創建第一個管理員
|
||||
|
||||
#### 方式 2: 自動創建 (推薦)
|
||||
|
||||
需要同時滿足以下條件:
|
||||
1. 配置文件中設定 `"create_default_admin": true`
|
||||
2. 設定環境變數 `SFTPGO_DEFAULT_ADMIN_USERNAME` 和 `SFTPGO_DEFAULT_ADMIN_PASSWORD`
|
||||
|
||||
### 設定步驟
|
||||
|
||||
#### Step 1: 確保配置文件正確
|
||||
|
||||
確認 `/Users/accusys/momentry/etc/sftpgo/sftpgo.json` 中有:
|
||||
```json
|
||||
{
|
||||
"data_provider": {
|
||||
"create_default_admin": true
|
||||
},
|
||||
"httpd": {
|
||||
"setup": {
|
||||
"installation_code": "Test3200Test3200"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Step 2: 更新 plist 加入環境變數
|
||||
|
||||
編輯 `/Library/LaunchDaemons/com.momentry.sftpgo.plist`,加入:
|
||||
```xml
|
||||
<key>EnvironmentVariables</key>
|
||||
<dict>
|
||||
<key>SFTPGO_DEFAULT_ADMIN_USERNAME</key>
|
||||
<string>admin</string>
|
||||
<key>SFTPGO_DEFAULT_ADMIN_PASSWORD</key>
|
||||
<string>Test3200Test3200</string>
|
||||
</dict>
|
||||
```
|
||||
|
||||
#### Step 3: 重啟 SFTPGo
|
||||
|
||||
```bash
|
||||
launchctl unload homebrew.mxcl.sftpgo
|
||||
launchctl load homebrew.mxcl.sftpgo
|
||||
```
|
||||
|
||||
#### Step 4: 驗證管理員
|
||||
|
||||
```bash
|
||||
curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200"
|
||||
```
|
||||
|
||||
成功回應:
|
||||
```json
|
||||
{
|
||||
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
|
||||
"token_type": "bearer",
|
||||
"expires_in": 1200
|
||||
}
|
||||
```
|
||||
|
||||
### REST API 認證流程
|
||||
|
||||
#### 1. 獲取 Token
|
||||
|
||||
```bash
|
||||
curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200"
|
||||
```
|
||||
|
||||
#### 2. 使用 Token 訪問 API
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
# 查看所有用戶
|
||||
curl -s http://localhost:8080/api/v2/users \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
|
||||
# 查看系統狀態
|
||||
curl -s http://localhost:8080/api/v2/status \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
### 常用管理操作
|
||||
|
||||
#### 查看所有用戶
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
curl -s http://localhost:8080/api/v2/users \
|
||||
-H "Authorization: Bearer $TOKEN" | jq .
|
||||
```
|
||||
|
||||
#### 建立新用戶
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
curl -s -X POST http://localhost:8080/api/v2/users \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"username": "newuser",
|
||||
"password": "userpassword123",
|
||||
"email": "user@example.com",
|
||||
"status": 1,
|
||||
"home_dir": "/Users/accusys/momentry/var/sftpgo/data/newuser",
|
||||
"uid": 501,
|
||||
"gid": 20,
|
||||
"permissions": {
|
||||
"/": ["*"]
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
#### 建立用戶組
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
curl -s -X POST http://localhost:8080/api/v2/groups \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "editors",
|
||||
"description": "Editor group with upload permissions"
|
||||
}'
|
||||
```
|
||||
|
||||
#### 刪除用戶
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
curl -s -X DELETE http://localhost:8080/api/v2/users/username \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
#### 修改用戶密碼
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq -r '.access_token')
|
||||
|
||||
curl -s -X PUT http://localhost:8080/api/v2/users/demo \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"password": "newpassword456"
|
||||
}'
|
||||
```
|
||||
|
||||
### 重要設定
|
||||
|
||||
| 設定項目 | 值 | 說明 |
|
||||
|----------|-----|------|
|
||||
| **SFTP 連接埠** | `2022` | SSH 檔案傳輸協定 |
|
||||
| **HTTP/WebDAV 連接埠** | `8080` | 內部 HTTP 服務 |
|
||||
| **WebAdmin 連接埠** | `8080` | 管理介面 (`/web/admin`) |
|
||||
| **WebClient 連接埠** | `8080` | 用戶介面 (`/web/client`) |
|
||||
| **資料庫** | PostgreSQL | 用戶和設定儲存 |
|
||||
| **Hook 腳本** | `/Users/accusys/sftpgo_test/register_hook.sh` | 上傳後自動化處理 |
|
||||
| **安裝碼** | `Test3200Test3200` | 首次設定管理員所需 |
|
||||
| **create_default_admin** | `true` | 自動創建管理員 |
|
||||
| **Token 有效期** | 1200 秒 (20分鐘) | JWT 過期時間 |
|
||||
|
||||
### 用戶目錄結構
|
||||
|
||||
所有 SFTPGo 用戶資料統一存放在 `/Users/accusys/momentry/var/sftpgo/data/` 目錄下:
|
||||
|
||||
| 用戶 | 資料夾路徑 | 密碼 | 說明 |
|
||||
|------|------------|------|------|
|
||||
| **demo** | `/Users/accusys/momentry/var/sftpgo/data/demo` | `demopassword123` | Demo 用戶上傳目錄 |
|
||||
| **momentry** | `/Users/accusys/momentry/var/sftpgo/data/momentry` | `momentry123` | Momentry 系統用戶 |
|
||||
| **warren** | `/Users/accusys/momentry/var/sftpgo/data/warren` | `warren123` | 其他用戶 |
|
||||
|
||||
### API Token 獲取方式
|
||||
|
||||
```bash
|
||||
# 注意:使用 GET 而非 POST
|
||||
curl -s -X GET "http://localhost:8080/api/v2/token" \
|
||||
-u "admin:Test3200Test3200" | jq .access_token
|
||||
```
|
||||
|
||||
### 故障排除
|
||||
|
||||
| 問題 | 解決方案 |
|
||||
|------|----------|
|
||||
| 無法獲取 Token | 確認環境變數已正確設定並重啟 SFTPGo |
|
||||
| SFTP 連線被拒絕 | 檢查 SFTPGo 服務: `launchctl list \| grep sftpgo` |
|
||||
| 無法登入 WebAdmin | 確認 admin 用戶存在: 檢查 plist 中環境變數是否正確 |
|
||||
| 上傳失敗 | 檢查 Hook 腳本: `tail -f /Users/accusys/momentry/log/sftpgo.error.log` |
|
||||
| 權限不足 | 檢查用戶權限或更新 `permissions` 設定 |
|
||||
| API 返回 401 | Token 過期,需重新獲取: `curl -X POST .../token -u "admin:pass"` | |
|
||||
|
||||
---
|
||||
|
||||
## 安全注意事項
|
||||
|
||||
- **密碼保護**: `demopassword123` 為 demo 帳戶密碼
|
||||
- **限制存取**: Demo 用戶只能訪問 `/demo` 目錄
|
||||
- **監控**: 所有上傳都有日誌記錄
|
||||
- **生產環境**: 正式環境應使用更強的密碼和金鑰認證
|
||||
@@ -0,0 +1,235 @@
|
||||
# SFTPGo 生命週期管理 (Source → Install → Config → Use)
|
||||
|
||||
**Date**: 2026-05-15
|
||||
**Status**: Active, Verified
|
||||
|
||||
---
|
||||
|
||||
## 生命週期總覽
|
||||
|
||||
```
|
||||
Source → Archive → Build → Install → Config → Start → Verify → Use
|
||||
① ② ③ ④ ⑤ ⑥ ⑦ ⑧
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ① Source Code
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Repository** | `https://github.com/drakkan/sftpgo.git` |
|
||||
| **Branch** | `main` (commit `6e543c6`) |
|
||||
| **License** | AGPL v3 |
|
||||
| **Language** | Go 1.26+ |
|
||||
| **Go files** | 246 |
|
||||
| **Source size** | 23 MB |
|
||||
|
||||
## ② Archive
|
||||
|
||||
```bash
|
||||
# Archive command
|
||||
cd /tmp
|
||||
tar czf release/system/v1.0/services/src/sftpgo-main.tar.gz sftpgo/
|
||||
|
||||
# Verify
|
||||
shasum -a 256 release/system/v1.0/services/src/sftpgo-main.tar.gz
|
||||
# → 6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a
|
||||
```
|
||||
|
||||
**Archive location**: `release/system/v1.0/services/src/sftpgo-main.tar.gz` (9.2 MB)
|
||||
|
||||
## ③ Build
|
||||
|
||||
```bash
|
||||
# Clone source
|
||||
git clone --depth 1 https://github.com/drakkan/sftpgo.git /tmp/sftpgo
|
||||
|
||||
# Build binary
|
||||
cd /tmp/sftpgo
|
||||
go build -o /Users/accusys/bin/sftpgo .
|
||||
|
||||
# Verify binary
|
||||
shasum -a 256 /Users/accusys/bin/sftpgo
|
||||
# → 9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e
|
||||
|
||||
ls -lh /Users/accusys/bin/sftpgo
|
||||
# → 88 MB
|
||||
```
|
||||
|
||||
**Binary**: `/Users/accusys/bin/sftpgo` (88 MB)
|
||||
|
||||
## ④ Install
|
||||
|
||||
### Database
|
||||
|
||||
```bash
|
||||
# Create dedicated PostgreSQL database + user
|
||||
psql -U accusys -h /tmp -d postgres -c "CREATE DATABASE sftpgo"
|
||||
psql -U accusys -h /tmp -d postgres -c "CREATE USER sftpgo WITH PASSWORD 'sftpgo_pass_2026'"
|
||||
psql -U accusys -h /tmp -d sftpgo -c "GRANT ALL ON SCHEMA public TO sftpgo"
|
||||
```
|
||||
|
||||
### Templates & Static Files
|
||||
|
||||
```bash
|
||||
# Copy from source (required by SFTPGo)
|
||||
cp -r /tmp/sftpgo/templates/ /Users/accusys/momentry/etc/sftpgo/templates/
|
||||
cp -r /tmp/sftpgo/static/ /Users/accusys/momentry/etc/sftpgo/static/
|
||||
cp -r /tmp/sftpgo/openapi/ /Users/accusys/momentry/etc/sftpgo/openapi/
|
||||
```
|
||||
|
||||
## ⑤ Configuration
|
||||
|
||||
**Config file**: `/Users/accusys/momentry/etc/sftpgo/sftpgo.json`
|
||||
|
||||
### Key Settings
|
||||
|
||||
| Section | Key | Value |
|
||||
|---------|-----|-------|
|
||||
| `data_provider` | `driver` | `postgresql` |
|
||||
| `data_provider` | `name` | `sftpgo` |
|
||||
| `data_provider` | `users_base_dir` | `/Users/accusys/momentry/var/sftpgo/data` |
|
||||
| `httpd.bindings[0]` | `port` | `8080` |
|
||||
| `sftpd.bindings[0]` | `port` | `2022` |
|
||||
| `webdavd.bindings[0]` | `port` | `8090` |
|
||||
|
||||
### launchd Plist
|
||||
|
||||
**File**: `momentry_runtime/plist/com.momentry.sftpgo.plist`
|
||||
|
||||
```xml
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/accusys/bin/sftpgo</string>
|
||||
<string>serve</string>
|
||||
<string>-c</string>
|
||||
<string>/Users/accusys/momentry/etc/sftpgo/</string>
|
||||
</array>
|
||||
```
|
||||
|
||||
## ⑥ Start
|
||||
|
||||
### Initialize Provider (first time only)
|
||||
|
||||
```bash
|
||||
SFTPGO_DEFAULT_ADMIN_USERNAME=admin \
|
||||
SFTPGO_DEFAULT_ADMIN_PASSWORD=Test3200Test3200 \
|
||||
/Users/accusys/bin/sftpgo initprovider -c /Users/accusys/momentry/etc/sftpgo/
|
||||
```
|
||||
|
||||
### Start Serve
|
||||
|
||||
```bash
|
||||
SFTPGO_DEFAULT_ADMIN_USERNAME=admin \
|
||||
SFTPGO_DEFAULT_ADMIN_PASSWORD=Test3200Test3200 \
|
||||
nohup /Users/accusys/bin/sftpgo serve \
|
||||
-c /Users/accusys/momentry/etc/sftpgo/ \
|
||||
> /Users/accusys/momentry/log/sftpgo.log 2>&1 &
|
||||
```
|
||||
|
||||
## ⑦ Verify
|
||||
|
||||
```bash
|
||||
# Service check
|
||||
curl -sI http://localhost:8080/
|
||||
# → Server: SFTPGo/2.7.99-dev
|
||||
|
||||
# HTTPS
|
||||
curl -sI https://m5sftpgo.momentry.ddns.net/
|
||||
# → Server: SFTPGo/2.7.99-dev
|
||||
# → Via: 1.1 Caddy
|
||||
|
||||
# Auth
|
||||
curl -s -u "admin:Test3200Test3200" http://localhost:8080/api/v2/token
|
||||
# → {"access_token":"eyJ...","expires_at":"..."}
|
||||
```
|
||||
|
||||
## ⑧ Usage
|
||||
|
||||
### User Management
|
||||
|
||||
**Admin**: `admin` / `Test3200Test3200`
|
||||
**Demo user**: `demo` / `demopassword123`
|
||||
|
||||
```bash
|
||||
# Get admin token
|
||||
TOKEN=$(curl -s -u "admin:Test3200Test3200" \
|
||||
"http://localhost:8080/api/v2/token" | \
|
||||
python3 -c "import json,sys;print(json.load(sys.stdin).get('access_token',''))")
|
||||
|
||||
# Create user
|
||||
curl -s -X POST "http://localhost:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"username":"demo","password":"demopassword123",
|
||||
"home_dir":"/Users/accusys/momentry/var/sftpgo/data/demo",
|
||||
"permissions": {"/": ["*"]}, "status": 1}'
|
||||
|
||||
# Update user password
|
||||
curl -s -X PUT "http://localhost:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$(curl -s http://localhost:8080/api/v2/users -H 'Authorization: Bearer $TOKEN' | python3 -c \"import json,sys;u=[u for u in json.load(sys.stdin) if u['username']=='demo'][0];u['password']='newpass';print(json.dumps(u))\")"
|
||||
|
||||
# List users
|
||||
curl -s "http://localhost:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
### External Access
|
||||
|
||||
```bash
|
||||
# Via HTTPS
|
||||
curl -s "https://m5sftpgo.momentry.ddns.net/api/v2/status"
|
||||
|
||||
# SFTP (port 2022)
|
||||
sftp -P 2022 demo@m5sftpgo.momentry.ddns.net
|
||||
|
||||
# WebDAV (port 8090)
|
||||
# http://m5sftpgo.momentry.ddns.net:8090/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 資源管理記錄
|
||||
|
||||
### dev.resources
|
||||
|
||||
```sql
|
||||
INSERT INTO dev.resources (resource_id, resource_type, category, capabilities, config)
|
||||
VALUES (
|
||||
'sftpgo', 'system_tool', 'file_upload',
|
||||
'["sftp", "file_transfer", "webdav"]',
|
||||
'{"binary": "/Users/accusys/bin/sftpgo",
|
||||
"version": "2.7.99-dev",
|
||||
"port": 8080,
|
||||
"source_sha256": "6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a",
|
||||
"binary_sha256": "9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e",
|
||||
"source_archive": "release/system/v1.0/services/src/sftpgo-main.tar.gz",
|
||||
"plist": "momentry_runtime/plist/com.momentry.sftpgo.plist"}'
|
||||
)
|
||||
ON CONFLICT (resource_id) DO UPDATE SET
|
||||
resource_type = EXCLUDED.resource_type,
|
||||
category = EXCLUDED.category,
|
||||
config = EXCLUDED.config;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SHA256 Checksums Reference
|
||||
|
||||
| Asset | SHA256 |
|
||||
|-------|--------|
|
||||
| Source archive (`sftpgo-main.tar.gz`) | `6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a` |
|
||||
| Binary (`/Users/accusys/bin/sftpgo`) | `9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e` |
|
||||
|
||||
---
|
||||
|
||||
## Ports Summary
|
||||
|
||||
| Port | Protocol | Service | External URL |
|
||||
|------|----------|---------|-------------|
|
||||
| 8080 | HTTP | Web Admin + REST API | `https://m5sftpgo.momentry.ddns.net` |
|
||||
| 2022 | SFTP | File Transfer | `sftp://m5sftpgo.momentry.ddns.net:2022` |
|
||||
| 8090 | WebDAV | File Access | `https://m5sftpgo.momentry.ddns.net:8090/` |
|
||||
@@ -0,0 +1,237 @@
|
||||
# SFTPGo Installation & Setup
|
||||
|
||||
**Date**: 2026-05-15
|
||||
**Version**: 2.6.7 (source build from main branch)
|
||||
**Status**: Active
|
||||
|
||||
---
|
||||
|
||||
## Top Info
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Source** | `https://github.com/drakkan/sftpgo.git` (main branch, ~2.7.99-dev) |
|
||||
| **Source archive** | `release/system/v1.0/services/src/sftpgo-main.tar.gz` (9.2MB) |
|
||||
| **Source SHA256** | `6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a` |
|
||||
| **Build method** | `git clone && go build -o /Users/accusys/bin/sftpgo .` |
|
||||
| **Binary** | `/Users/accusys/bin/sftpgo` (88MB) |
|
||||
| **Binary SHA256** | `9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e` |
|
||||
| **Config** | `/Users/accusys/momentry/etc/sftpgo/sftpgo.json` |
|
||||
| **Templates** | `/Users/accusys/momentry/etc/sftpgo/templates/` (copied from source) |
|
||||
| **Database** | PostgreSQL `sftpgo` database, user `sftpgo` |
|
||||
| **Plist** | `momentry_runtime/plist/com.momentry.sftpgo.plist` |
|
||||
| **Ports** | 8080 (HTTP/WebAdmin), 2022 (SFTP), 8090 (WebDAV) |
|
||||
| **Resource ID** | `sftpgo` in `dev.resources` (type: `system_tool`, category: `file_upload`) |
|
||||
|
||||
---
|
||||
|
||||
## Build from Source
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.26+ (`go version`)
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
# Clone source
|
||||
git clone --depth 1 https://github.com/drakkan/sftpgo.git /tmp/sftpgo
|
||||
|
||||
# Build binary
|
||||
cd /tmp/sftpgo
|
||||
go build -o /Users/accusys/bin/sftpgo .
|
||||
|
||||
# Verify
|
||||
/Users/accusys/bin/sftpgo --version
|
||||
```
|
||||
|
||||
### Archive Source
|
||||
|
||||
```bash
|
||||
tar czf release/system/v1.0/services/src/sftpgo-main.tar.gz -C /tmp sftpgo/
|
||||
shasum -a 256 release/system/v1.0/services/src/sftpgo-main.tar.gz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Setup
|
||||
|
||||
```bash
|
||||
# Create database and user
|
||||
psql -U accusys -h /tmp -d postgres -c "CREATE DATABASE sftpgo"
|
||||
psql -U accusys -h /tmp -d postgres -c "CREATE USER sftpgo WITH PASSWORD 'sftpgo_pass_2026'"
|
||||
psql -U accusys -h /tmp -d sftpgo -c "GRANT ALL ON SCHEMA public TO sftpgo"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Start Server
|
||||
|
||||
### Initialize Provider (first time only)
|
||||
|
||||
```bash
|
||||
SFTPGO_DEFAULT_ADMIN_USERNAME=admin \
|
||||
SFTPGO_DEFAULT_ADMIN_PASSWORD=Test3200Test3200 \
|
||||
/Users/accusys/bin/sftpgo initprovider \
|
||||
-c /Users/accusys/momentry/etc/sftpgo/
|
||||
```
|
||||
|
||||
### Start Serve
|
||||
|
||||
```bash
|
||||
SFTPGO_DEFAULT_ADMIN_USERNAME=admin \
|
||||
SFTPGO_DEFAULT_ADMIN_PASSWORD=Test3200Test3200 \
|
||||
nohup /Users/accusys/bin/sftpgo serve \
|
||||
-c /Users/accusys/momentry/etc/sftpgo/ \
|
||||
> /Users/accusys/momentry/log/sftpgo.log 2>&1 &
|
||||
```
|
||||
|
||||
Note: The `-c` flag must point to the config directory (containing `sftpgo.json`, `templates/`, `static/`, `openapi/`).
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
# Health check (HTTP 200 = running)
|
||||
curl -s http://127.0.0.1:8080/api/v2/status
|
||||
# Should return: {"error":"no token found","message":"Unauthorized"}
|
||||
|
||||
# Get admin token
|
||||
curl -s -u "admin:Test3200Test3200" "http://127.0.0.1:8080/api/v2/token"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Management
|
||||
|
||||
### Get Admin Token
|
||||
|
||||
The SFTPGo API uses JWT tokens. All user/management API calls require `Authorization: Bearer <token>` header.
|
||||
|
||||
```bash
|
||||
TOKEN=$(curl -s -u "admin:Test3200Test3200" \
|
||||
"http://127.0.0.1:8080/api/v2/token" | \
|
||||
python3 -c "import json,sys;print(json.load(sys.stdin).get('access_token',''))")
|
||||
```
|
||||
|
||||
### Create Demo User
|
||||
|
||||
```bash
|
||||
curl -s -X POST "http://127.0.0.1:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"username": "demo",
|
||||
"password": "demopassword123",
|
||||
"home_dir": "/Users/accusys/momentry/var/sftpgo/data/demo",
|
||||
"permissions": {"/": ["*"]},
|
||||
"status": 1,
|
||||
"quota_size": 0,
|
||||
"quota_files": 0
|
||||
}'
|
||||
```
|
||||
|
||||
### Update User Password
|
||||
|
||||
```bash
|
||||
# Get current user data
|
||||
USER_DATA=$(curl -s "http://127.0.0.1:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN" | \
|
||||
python3 -c "
|
||||
import json,sys
|
||||
users=json.load(sys.stdin)
|
||||
u=[u for u in users if u['username']=='demo'][0]
|
||||
u['password']='newpassword'
|
||||
print(json.dumps(u))
|
||||
")
|
||||
|
||||
# Update user
|
||||
curl -s -X PUT "http://127.0.0.1:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$USER_DATA"
|
||||
```
|
||||
|
||||
### List Users
|
||||
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:8080/api/v2/users" \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
Key settings in `/Users/accusys/momentry/etc/sftpgo/sftpgo.json`:
|
||||
|
||||
| Section | Key | Value | Note |
|
||||
|---------|-----|-------|------|
|
||||
| `data_provider` | `driver` | `postgresql` | User/auth database |
|
||||
| `data_provider` | `name` | `sftpgo` | Database name |
|
||||
| `data_provider` | `users_base_dir` | `/Users/accusys/momentry/var/sftpgo/data` | Base directory for user homes |
|
||||
| `httpd.bindings[0]` | `port` | `8080` | Web admin + REST API |
|
||||
| `sftpd.bindings[0]` | `port` | `2022` | SFTP server |
|
||||
| `webdavd.bindings[0]` | `port` | `8090` | WebDAV server |
|
||||
| `setup` | `installation_code` | `momentry2026` | Web setup wizard code |
|
||||
|
||||
---
|
||||
|
||||
## launchd Plist
|
||||
|
||||
```xml
|
||||
<!-- /Users/accusys/momentry_core_0.1/momentry_runtime/plist/com.momentry.sftpgo.plist -->
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/Users/accusys/bin/sftpgo</string>
|
||||
<string>serve</string>
|
||||
<string>-c</string>
|
||||
<string>/Users/accusys/momentry/etc/sftpgo/</string>
|
||||
</array>
|
||||
```
|
||||
|
||||
Load with launchctl:
|
||||
```bash
|
||||
launchctl load ~/Library/LaunchAgents/com.momentry.sftpgo.plist
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Resource Record
|
||||
|
||||
```sql
|
||||
INSERT INTO dev.resources (resource_id, resource_type, category, capabilities, config)
|
||||
VALUES (
|
||||
'sftpgo',
|
||||
'system_tool',
|
||||
'file_upload',
|
||||
'["sftp", "file_transfer", "webdav"]',
|
||||
'{"binary": "/Users/accusys/bin/sftpgo", "version": "2.6.7", "port": 8080,
|
||||
"source_sha256": "6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a",
|
||||
"binary_sha256": "9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e",
|
||||
"source_archive": "release/system/v1.0/services/src/sftpgo-main.tar.gz",
|
||||
"plist": "momentry_runtime/plist/com.momentry.sftpgo.plist"}'
|
||||
)
|
||||
ON CONFLICT (resource_id) DO UPDATE SET
|
||||
resource_type = EXCLUDED.resource_type,
|
||||
category = EXCLUDED.category,
|
||||
config = EXCLUDED.config;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ports Summary
|
||||
|
||||
| Port | Service | Purpose |
|
||||
|------|---------|---------|
|
||||
| 8080 | HTTP/HTTPS | Web admin UI + REST API |
|
||||
| 2022 | SFTP | File transfer over SSH |
|
||||
| 8090 | WebDAV | File access via WebDAV |
|
||||
|
||||
---
|
||||
|
||||
## Credentials
|
||||
|
||||
| User | Password | Role |
|
||||
|------|----------|------|
|
||||
| `admin` | `Test3200Test3200` | Administrator (API + Web Admin) |
|
||||
| `demo` | `demopassword123` | Demo user (file upload) |
|
||||
@@ -0,0 +1,84 @@
|
||||
# SFTPGo Source Code Verification Report
|
||||
|
||||
**Date**: 2026-05-15
|
||||
**Version**: 2.7.99-dev (main branch)
|
||||
**Status**: ✅ Verified
|
||||
|
||||
---
|
||||
|
||||
## 1. Source Archive
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| **Archive** | `release/system/v1.0/services/src/sftpgo-main.tar.gz` |
|
||||
| **Size** | 9.2 MB |
|
||||
| **SHA256** | `6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a` |
|
||||
| **Recorded in DB** | ✅ Matches `dev.resources.config->>'source_sha256'` |
|
||||
| **Git remote** | `https://github.com/drakkan/sftpgo.git` |
|
||||
| **Git commit** | `6e543c6` |
|
||||
|
||||
## 2. Binary
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| **Path** | `/Users/accusys/bin/sftpgo` |
|
||||
| **Size** | 88 MB |
|
||||
| **SHA256** | `9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e` |
|
||||
| **Recorded in DB** | ✅ Matches `dev.resources.config->>'binary_sha256'` |
|
||||
| **Build date** | 2026-05-15 22:48 |
|
||||
| **Build method** | `git clone && go build -o /Users/accusys/bin/sftpgo .` |
|
||||
|
||||
## 3. Source Tree
|
||||
|
||||
| Item | Count |
|
||||
|------|:----:|
|
||||
| Go source files | 246 |
|
||||
| Total size | 23 MB |
|
||||
| License | AGPL v3 (GNU Affero General Public License) |
|
||||
|
||||
## 4. Build Verification
|
||||
|
||||
| Item | Status |
|
||||
|------|:------:|
|
||||
| Build from source | ✅ `go build` succeeds |
|
||||
| Reproducible build | ✅ Source archived, SHA256 matched |
|
||||
| Dependency trace | ✅ `go.mod` + `go.sum` included in archive |
|
||||
|
||||
## 5. Runtime Service
|
||||
|
||||
| Endpoint | Status | Response |
|
||||
|----------|:------:|----------|
|
||||
| `http://localhost:8080/` | ✅ | `SFTPGo/2.7.99-dev` |
|
||||
| `https://m5sftpgo.momentry.ddns.net/` | ✅ | Caddy → SFTPGo proxy |
|
||||
| Auth (admin) | ✅ | Token endpoint works |
|
||||
| Demo user | ✅ | `demo` / `demopassword123` |
|
||||
| Ports | ✅ | 8080 (HTTP), 2022 (SFTP), 8090 (WebDAV) |
|
||||
|
||||
## 6. Resource Registration
|
||||
|
||||
```sql
|
||||
INSERT INTO dev.resources (resource_id, resource_type, category, capabilities, config)
|
||||
VALUES (
|
||||
'sftpgo', 'system_tool', 'file_upload',
|
||||
'["sftp", "file_transfer", "webdav"]',
|
||||
'{"binary": "/Users/accusys/bin/sftpgo", "version": "2.6.7", "port": 8080,
|
||||
"source_sha256": "6607334148917dd80a687706a3ae63ea8c532d10c6717c87491da23939c96d4a",
|
||||
"binary_sha256": "9991d2a1c877d5bcae17cb4e026de939862e4b880924589cf4ed15ac7291ec7e",
|
||||
"source_archive": "release/system/v1.0/services/src/sftpgo-main.tar.gz",
|
||||
"plist": "momentry_runtime/plist/com.momentry.sftpgo.plist"}'
|
||||
);
|
||||
```
|
||||
|
||||
## 7. Verification Summary
|
||||
|
||||
| # | Check | Result |
|
||||
|---|-------|:------:|
|
||||
| 1 | Source archive exists in `services/src/` | ✅ |
|
||||
| 2 | Source SHA256 matches DB record | ✅ |
|
||||
| 3 | Binary SHA256 matches DB record | ✅ |
|
||||
| 4 | Build reproducible from archived source | ✅ |
|
||||
| 5 | Service responding on HTTP (localhost:8080) | ✅ |
|
||||
| 6 | Service accessible via HTTPS (m5sftpgo.momentry.ddns.net) | ✅ |
|
||||
| 7 | Admin auth works | ✅ |
|
||||
| 8 | Demo user exists and functional | ✅ |
|
||||
| 9 | Configuration documented in `REFERENCE/SFTPGo_Setup.md` | ✅ |
|
||||
@@ -0,0 +1,208 @@
|
||||
# Momentry Core — 完整服務清單
|
||||
|
||||
**Date**: 2026-05-15
|
||||
**Version**: 1.0
|
||||
**Status**: Active
|
||||
|
||||
---
|
||||
|
||||
## 1. 來源分類
|
||||
|
||||
| 來源 | 說明 | 數量 |
|
||||
|------|------|:----:|
|
||||
| 🔵 Source build | 原始碼編譯,SHA256 可驗證 | 7 |
|
||||
| 🟡 Homebrew | 透過 brew 安裝,待遷移 | 18 |
|
||||
| 🟢 Production only | 僅在 production 運行 | 1 |
|
||||
| 📦 Third-party | 預編譯 binary / 套件 | 5 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 完整服務一覽
|
||||
|
||||
### 🔵 Source Build (SHA256 已記錄)
|
||||
|
||||
| Service | Version | Binary | Source Archive | Resource ID |
|
||||
|---------|---------|--------|---------------|-------------|
|
||||
| **PostgreSQL** | 18.3 | `$HOME/pgsql/18.3/bin/postgres` | `release/.../postgresql-18.3.tar.gz` | - |
|
||||
| **Redis** | 7.4.3 | `/opt/homebrew/bin/redis-server`(brew) | `release/.../redis-7.4.3.tar.gz` | - |
|
||||
| **FFmpeg** | 7.1.1 | `/opt/homebrew/bin/ffmpeg`(brew) | `release/.../ffmpeg-7.1.1.tar.xz` | - |
|
||||
| **RSync** | 3.4.2 | `/Users/accusys/bin/rsync` | `release/.../rsync-official-3.4.2.tar.gz` | `rsync` |
|
||||
| **SFTPGo** | 2.7.99-dev | `/Users/accusys/bin/sftpgo` | `release/.../sftpgo-main.tar.gz` | `sftpgo` |
|
||||
| **llama.cpp** | - | `$HOME/llama/bin/llama-server` | `release/.../llama.cpp/` | - |
|
||||
| **Momentry** | 1.0.0 | `target/release/momentry` | (git repo) | - |
|
||||
|
||||
### 🟡 Homebrew (待遷移)
|
||||
|
||||
| Service | Version | Brew Binary | Dev.Resources |
|
||||
|---------|---------|-------------|:-------------:|
|
||||
| **PHP** | 8.5.5 | `/opt/homebrew/bin/php` | ✅ |
|
||||
| **PHP-FPM** | 8.5.5 | `/opt/homebrew/sbin/php-fpm` | ✅ (同 php) |
|
||||
| **MariaDB** | 12.2.2 | `/opt/homebrew/bin/mariadbd` | ✅ |
|
||||
| **Node.js** | 25.9.0 | `/opt/homebrew/bin/node` | ❌ |
|
||||
| **MongoDB** | 8.2.7 | `/opt/homebrew/bin/mongod` | ❌ |
|
||||
| **MongoSH** | 2.8.3 | `/opt/homebrew/bin/mongosh` | ❌ |
|
||||
| **Ollama** | 0.23.1 | `/opt/homebrew/bin/ollama` | ❌ |
|
||||
| **yt-dlp** | 2026.3.17 | `/opt/homebrew/bin/yt-dlp` | ❌ |
|
||||
| **whisper-cpp** | 1.8.4 | `/opt/homebrew/bin/whisper-cpp` | ❌ |
|
||||
| **Tesseract** | 5.5.2 | `/opt/homebrew/bin/tesseract` | ❌ |
|
||||
| **SDL2** | 2.32.10 | `/opt/homebrew/lib/libsdl2.dylib` | ❌ |
|
||||
| **Go** | 1.26.2 | `/opt/homebrew/bin/go` | ❌ |
|
||||
| **CMake** | 4.3.2 | `/opt/homebrew/bin/cmake` | ❌ |
|
||||
| **Python 3.11** | 3.11.15 | `/opt/homebrew/bin/python3.11` | ❌ |
|
||||
| **Python 3.14** | 3.14.4 | `/opt/homebrew/bin/python3.14` | ❌ |
|
||||
| **protobuf** | - | `/opt/homebrew/bin/protoc` | ❌ |
|
||||
| **pgvector** | 0.8.2 | (PG extension) | ❌ |
|
||||
| **FFmpeg-full** | 8.1.1 | `/opt/homebrew/opt/ffmpeg-full/bin/ffmpeg` | ❌ |
|
||||
|
||||
### 🟢 Production Only (僅 production 有)
|
||||
|
||||
| Service | URL | Role | Resource ID |
|
||||
|---------|-----|------|:-----------:|
|
||||
| **WordPress** | `https://wp.momentry.ddns.net` | CMS + Portal | ✅ (offline) |
|
||||
|
||||
### 📦 Third-Party (預編譯 / 外部依賴)
|
||||
|
||||
| Service | Version | Location | Note |
|
||||
|---------|---------|----------|------|
|
||||
| **Qdrant** | - | `/Users/accusys/momentry_core_0.1/services/qdrant/target/release/qdrant` | Rust source in `services/` |
|
||||
| **EmbeddingGemma** | - | Python script `scripts/embeddinggemma_server.py` | 由 server 管理 |
|
||||
| **GroundingDINO** | - | `release/.../services/src/GroundingDINO/` | Python ML model |
|
||||
| **PaliGemma** | - | `release/.../services/src/paligemma/` | Python ML model |
|
||||
| **Odoo** | - | `release/.../services/src/odoo/` | ERP (未啟用) |
|
||||
| **Gitea** | - | `release/.../services/src/gitea/` | Git hosting (未啟用) |
|
||||
| **LibreOffice** | 26.2.3 | `release/.../LibreOffice_26.2.3_MacOS_aarch64.dmg` | Document conversion |
|
||||
| **Swift** | 6.3.1 | `release/.../swift-6.3.1-RELEASE.tar.gz` | Processor scripts |
|
||||
| **SQLite-vec** | - | `release/.../sqlite-vec/` + `vec0.dylib` | Vector extension |
|
||||
| **librsvg** | - | `release/.../librsvg/` | SVG conversion |
|
||||
| **macmon** | 0.7.2 | `$HOME/bin/macmon` | Monitoring |
|
||||
| **mactop** | latest | `$HOME/bin/mactop` | Monitoring |
|
||||
| **mermaid-cli** | 11.14.0 | npm global | Diagram rendering |
|
||||
|
||||
---
|
||||
|
||||
## 3. Plist (launchd) 管理
|
||||
|
||||
| Plist | Binary Path | Source Type | Status |
|
||||
|-------|-------------|:-----------:|:------:|
|
||||
| `com.momentry.sftpgo.plist` | `/Users/accusys/bin/sftpgo` | ✅ source | ✅ |
|
||||
| `com.momentry.redis.plist` | `/opt/homebrew/bin/redis-server` | 🟡 brew | ❌ 待更新 |
|
||||
| `com.momentry.postgresql.plist` | `$HOME/pgsql/18.3/bin/postgres` | ✅ source | ✅ |
|
||||
| `com.momentry.ollama.plist` | `/opt/homebrew/bin/ollama` | 🟡 brew | ❌ 待更新 |
|
||||
| `com.momentry.llama.plist` | `$HOME/llama/bin/llama-server` | ✅ source | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 4. Port 分配
|
||||
|
||||
| Port | Service | Source Type | Running |
|
||||
|:----:|---------|:-----------:|:-------:|
|
||||
| 3002 | Momentry API (production) | ✅ build | ✅ |
|
||||
| 3003 | Momentry Playground (dev) | ✅ build | ✅ |
|
||||
| 8080 | SFTPGo Web Admin | ✅ build | ✅ |
|
||||
| 2022 | SFTPGo SFTP | ✅ build | ✅ |
|
||||
| 8090 | SFTPGo WebDAV | ✅ build | ✅ |
|
||||
| 5432 | PostgreSQL | ✅ build | ✅ |
|
||||
| 6379 | Redis | 🟡 brew | ✅ |
|
||||
| 27017 | MongoDB | 🟡 brew | ✅ |
|
||||
| 6333 | Qdrant | 📦 service | ✅ |
|
||||
| 11434 | Ollama API | 🟡 brew | ✅ |
|
||||
| 11436 | EmbeddingGemma | 📦 script | ✅ |
|
||||
| 8082 | llama-server (Gemma4) | ✅ build | ✅ |
|
||||
| 9000 | PHP-FPM | 🟡 brew | ❌ |
|
||||
| 8081 | LLM (deprecated) | 🟡 brew | ❌ |
|
||||
|
||||
---
|
||||
|
||||
## 5. Database
|
||||
|
||||
| Database | Engine | Port | Schema | Momentry Role |
|
||||
|----------|--------|:----:|--------|---------------|
|
||||
| `momentry` | PostgreSQL | 5432 | `dev` | Dev playground (3003) |
|
||||
| `momentry_3002` | PostgreSQL | 5432 | `public` | M5 production (3002) |
|
||||
| `sftpgo` | PostgreSQL | 5432 | `public` | SFTPGo users |
|
||||
| `momentry` | MongoDB | 27017 | `momentry` | Cache |
|
||||
| Qdrant | Qdrant | 6333 | `momentry_dev_rule1_v2` | Vector search |
|
||||
| Redis | Redis | 6379 | - | Worker progress, cache |
|
||||
|
||||
---
|
||||
|
||||
## 6. Source Archives Status (`release/system/v1.0/services/src/`)
|
||||
|
||||
### ✅ 已歸檔 (32 items)
|
||||
|
||||
```
|
||||
cmake-4.2.0-macos-universal.tar.gz ffmpeg-7.1.1.tar.xz
|
||||
freetype-2.13.3.tar.gz postgresql-18.3.tar.gz
|
||||
redis-7.4.3.tar.gz rsync-official-3.4.2.tar.gz
|
||||
sftpgo-main.tar.gz swift-6.3.1-RELEASE.tar.gz
|
||||
llama.cpp/ x264/
|
||||
go/ pyenv/
|
||||
macmon-0.7.2.tar.gz mactop-latest.tar.gz
|
||||
rustc-1.92.0-src.tar.xz rustup-1.28.1.tar.gz
|
||||
sqlite-amalgamation-3490100.zip sqlite-vec/
|
||||
vec0.dylib yt-dlp/
|
||||
mermaid-js-mermaid-cli-11.14.0.tgz LibreOffice_26.2.3_MacOS_aarch64.dmg
|
||||
libreoffice-26.2.3.2.tar.xz librsvg/
|
||||
GroundingDINO/ paligemma/
|
||||
gitea/ erpnext/
|
||||
frappe/ odoo/
|
||||
python_probe_deps.txt
|
||||
```
|
||||
|
||||
### ❌ 缺少需補
|
||||
|
||||
| Service | Expected Source URL |
|
||||
|---------|-------------------|
|
||||
| PHP 8.5.5 | `https://www.php.net/distributions/php-8.5.5.tar.gz` |
|
||||
| MariaDB 12.2.2 | `https://github.com/MariaDB/server/archive/mariadb-12.2.2.tar.gz` |
|
||||
| Node.js 25.9.0 | `https://nodejs.org/dist/v25.9.0/node-v25.9.0.tar.gz` |
|
||||
| MongoDB 8.2.7 | `https://github.com/mongodb/mongo/archive/r8.2.7.tar.gz` |
|
||||
| Ollama 0.23.1 | `https://github.com/ollama/ollama/archive/v0.23.1.tar.gz` |
|
||||
|
||||
---
|
||||
|
||||
## 7. Health Endpoint (`/health/detailed`)
|
||||
|
||||
| Field | Source | Covers |
|
||||
|-------|--------|--------|
|
||||
| `services.postgres` | direct check | PostgreSQL |
|
||||
| `services.redis` | direct check | Redis |
|
||||
| `services.qdrant` | direct check | Qdrant |
|
||||
| `services.mongodb` | direct check | MongoDB |
|
||||
| `pipeline.ffmpeg` | `which ffmpeg` | FFmpeg |
|
||||
| `pipeline.llm` | HTTP check port 8082 | llama-server |
|
||||
| `pipeline.embedding_server` | HTTP check port 11436 | EmbeddingGemma |
|
||||
| `pipeline.rsync` | file check | RSync |
|
||||
| `pipeline.scripts_integrity` | SHA256 vs manifest | Processor scripts(345) |
|
||||
| `schema` | DB query | Schema migrations(9) |
|
||||
| `processors` | file check | 12 processors |
|
||||
| `resources` | system query | CPU/Mem/GPU |
|
||||
|
||||
### 未涵蓋的服務
|
||||
|
||||
- PHP / PHP-FPM
|
||||
- MariaDB
|
||||
- Node.js / npm
|
||||
- Ollama
|
||||
- SFTPGo (有 `/api/v1/stats/sftpgo` 但不在 health response)
|
||||
- yt-dlp / whisper-cpp / tesseract
|
||||
- WordPress
|
||||
|
||||
---
|
||||
|
||||
## 8. Migration Priority Matrix
|
||||
|
||||
```
|
||||
Priority Service Binary Source Dev.Resources Plist Health
|
||||
──────────────────────────────────────────────────────────────────────────────
|
||||
P1:high PHP brew → build ❌ need ✅ recorded ❌ brew ❌ missing
|
||||
P1:high MariaDB brew → build ❌ need ✅ recorded ❌ brew ❌ missing
|
||||
P1:high Node.js brew → build ❌ need ❌ not in db ❌ brew ❌ missing
|
||||
P2:med Redis brew → build ✅ have ❌ not in db ❌ brew ❌ missing
|
||||
P2:med MongoDB brew → build ❌ need ❌ not in db ❌ brew ❌ missing
|
||||
P2:med ffmpeg brew → build ✅ have ❌ not in db ❌ brew ✅ basic
|
||||
P3:low Ollama brew → build ❌ need ❌ not in db ❌ brew ❌ missing
|
||||
P3:low yt-dlp brew → archive ✅ have ❌ not in db ❌ brew ❌ missing
|
||||
P3:low whisper brew → build ❌ need ❌ not in db ❌ brew ❌ missing
|
||||
P3:low tesseract brew → build ❌ need ❌ not in db ❌ brew ❌ missing
|
||||
```
|
||||
@@ -0,0 +1,280 @@
|
||||
---
|
||||
document_type: "reference_doc"
|
||||
service: "MOMENTRY_CORE"
|
||||
title: "Momentry Core 版本管理規範"
|
||||
date: "2026-03-23"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "momentry"
|
||||
- "core"
|
||||
- "版本管理規範"
|
||||
ai_query_hints:
|
||||
- "查詢 Momentry Core 版本管理規範 的內容"
|
||||
- "Momentry Core 版本管理規範 的主要目的是什麼?"
|
||||
- "如何操作或實施 Momentry Core 版本管理規範?"
|
||||
---
|
||||
|
||||
# Momentry Core 版本管理規範
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-23 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-23 | 創建版本管理規範 | Warren | OpenCode |
|
||||
|
||||
---
|
||||
|
||||
## 1. 版本與通訊埠對照表
|
||||
|
||||
| 版本 | Binary | Port | Redis Prefix | 用途 |
|
||||
|------|--------|------|--------------|------|
|
||||
| **Production** | `momentry` | **3002** | `momentry:` | 正式環境 |
|
||||
| **Development** | `momentry_playground` | **3003** | `momentry_dev:` | 開發測試 |
|
||||
|
||||
### 通訊埠嚴禁事項
|
||||
- ❌ 開發版嚴禁使用 3002
|
||||
- ❌ 任何 `cargo run` 直接啟動的 server 嚴禁綁定 3002
|
||||
- ❌ Debug build 嚴禁部署到 3002
|
||||
|
||||
---
|
||||
|
||||
## 2. 開發環境隔離原則
|
||||
|
||||
### 2.1 開發流程
|
||||
```bash
|
||||
# 永遠在 3003 開發測試
|
||||
cd /Users/accusys/momentry_core_0.1
|
||||
|
||||
# 開發版啟動 (3003)
|
||||
cargo run --bin momentry_playground -- server
|
||||
# 或
|
||||
cargo run --bin momentry -- server --port 3003
|
||||
```
|
||||
|
||||
### 2.2 測試完成後
|
||||
1. 確認所有功能在 3003 正常運作
|
||||
2. 進行 `cargo clippy --lib` 檢查
|
||||
3. 進行 `cargo test --lib` 測試
|
||||
4. 確認後才能進行 release
|
||||
|
||||
### 2.3 環境變數隔離
|
||||
```bash
|
||||
# Development
|
||||
export MOMENTRY_SERVER_PORT=3003
|
||||
export MOMENTRY_REDIS_PREFIX=momentry_dev:
|
||||
|
||||
# Production (launchd 管理,勿手動設定)
|
||||
# MOMENTRY_SERVER_PORT=3002
|
||||
# MOMENTRY_REDIS_PREFIX=momentry:
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Release 版本管理
|
||||
|
||||
### 3.1 Release 前檢查清單
|
||||
|
||||
```
|
||||
□ 開發版 (3003) 功能測試完成
|
||||
□ cargo clippy --lib 通過
|
||||
□ cargo test --lib 通過
|
||||
□ cargo fmt -- --check 通過
|
||||
□ 所有修改已 commit 到 Gitea
|
||||
```
|
||||
|
||||
### 3.2 Release 流程
|
||||
|
||||
```bash
|
||||
# 1. 確保目前是乾淨的工作目錄
|
||||
git status
|
||||
|
||||
# 2. 備份當前 production binary
|
||||
BACKUP_DIR="/Users/accusys/momentry/backup/bin"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
cp /Users/accusys/momentry/bin/momentry "${BACKUP_DIR}/momentry_${TIMESTAMP}"
|
||||
|
||||
# 3. 停止 production server
|
||||
sudo launchctl unload /Library/LaunchDaemons/com.momentry.api.plist
|
||||
# 或
|
||||
pkill -f "target/release/momentry server"
|
||||
|
||||
# 4. 編譯 release 版本
|
||||
cargo build --release --bin momentry
|
||||
|
||||
# 5. 部署到正式位置
|
||||
cp target/release/momentry /Users/accusys/momentry/bin/momentry
|
||||
|
||||
# 6. 啟動 production server
|
||||
sudo launchctl load /Library/LaunchDaemons/com.momentry.api.plist
|
||||
|
||||
# 7. 驗證
|
||||
curl http://localhost:3002/health
|
||||
```
|
||||
|
||||
### 3.3 Backup 存放位置
|
||||
```
|
||||
/Users/accusys/momentry/backup/bin/
|
||||
├── momentry_20260325_143000 (backup)
|
||||
├── momentry_20260324_100000
|
||||
├── momentry_20260323_090000
|
||||
└── ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Gitea 版本控制
|
||||
|
||||
### 4.1 Commit 規範
|
||||
```
|
||||
feat: 新功能
|
||||
fix: 錯誤修復
|
||||
refactor: 重構
|
||||
docs: 文件更新
|
||||
chore: 杂项
|
||||
test: 测试
|
||||
```
|
||||
|
||||
### 4.2 Release Tag 規範
|
||||
```bash
|
||||
# 建立 release tag
|
||||
git tag -a v0.1.1 -m "Release v0.1.1 - API Key Authentication"
|
||||
git push origin v0.1.1
|
||||
```
|
||||
|
||||
### 4.3 版本號命名
|
||||
```
|
||||
v{major}.{minor}.{patch}
|
||||
│ │ └── Patch version (bug fix)
|
||||
│ └───────── Minor version (新功能)
|
||||
└──────────────── Major version (破壞性變更)
|
||||
```
|
||||
|
||||
### 4.4 Gitea Release 建立
|
||||
1. 在 Gitea Repo > Releases > New Release
|
||||
2. 選擇對應的 Tag
|
||||
3. 填寫 Release Notes
|
||||
4. 上傳 compiled binary(如需要)
|
||||
|
||||
---
|
||||
|
||||
## 5. 服務管理
|
||||
|
||||
### 5.1 Production Service (launchd)
|
||||
```bash
|
||||
# Plist 位置
|
||||
/Library/LaunchDaemons/com.momentry.api.plist
|
||||
|
||||
# 管理指令
|
||||
sudo launchctl load /Library/LaunchDaemons/com.momentry.api.plist # 啟動
|
||||
sudo launchctl unload /Library/LaunchDaemons/com.momentry.api.plist # 停止
|
||||
sudo launchctl list | grep momentry # 狀態
|
||||
```
|
||||
|
||||
### 5.2 緊急回滾
|
||||
```bash
|
||||
# 1. 停止當前服務
|
||||
sudo launchctl unload /Library/LaunchDaemons/com.momentry.api.plist
|
||||
|
||||
# 2. 恢復上一個 backup
|
||||
BACKUP_FILE=$(ls -t /Users/accusys/momentry/backup/bin/ | head -1)
|
||||
cp "/Users/accusys/momentry/backup/bin/${BACKUP_FILE}" /Users/accusys/momentry/bin/momentry
|
||||
|
||||
# 3. 重啟服務
|
||||
sudo launchctl load /Library/LaunchDaemons/com.momentry.api.plist
|
||||
|
||||
# 4. 驗證
|
||||
curl http://localhost:3002/health
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 快速參考卡片
|
||||
|
||||
### Development
|
||||
```bash
|
||||
# 啟動開發版
|
||||
cd /Users/accusys/momentry_core_0.1
|
||||
cargo run --bin momentry_playground -- server
|
||||
|
||||
# 或手動指定 port
|
||||
cargo run --bin momentry -- server --port 3003
|
||||
|
||||
# 測試端點
|
||||
curl -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" \
|
||||
http://localhost:3003/api/v1/jobs
|
||||
```
|
||||
|
||||
### Production
|
||||
```bash
|
||||
# 查看狀態
|
||||
sudo launchctl list | grep momentry
|
||||
|
||||
# 重啟服務
|
||||
sudo launchctl unload /Library/LaunchDaemons/com.momentry.api.plist
|
||||
sudo launchctl load /Library/LaunchDaemons/com.momentry.api.plist
|
||||
|
||||
# 查看日誌
|
||||
tail -f /Users/accusys/momentry/log/momentry_release.log
|
||||
```
|
||||
|
||||
### 常用指令
|
||||
```bash
|
||||
# 檢查 port 使用
|
||||
lsof -i :3002 # Production
|
||||
lsof -i :3003 # Development
|
||||
|
||||
# 檢查 process
|
||||
ps aux | grep momentry | grep server
|
||||
|
||||
# 停止所有 momentry server
|
||||
pkill -9 -f "momentry.*server"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 禁止事項
|
||||
|
||||
| 項目 | 說明 |
|
||||
|------|------|
|
||||
| ❌ 禁止在 3002 測試 | 3002 是 Production,嚴禁用於測試 |
|
||||
| ❌ 禁止覆蓋 production binary | 使用 backup + deploy 流程 |
|
||||
| ❌ 禁止跳過測試直接 release | 必須完成檢查清單 |
|
||||
| ❌ 禁止在未備份的情況下部署 | 每次部署前必須備份 |
|
||||
|
||||
---
|
||||
|
||||
## 8. 疑難排解
|
||||
|
||||
### Q: 3002 無法綁定怎麼辦?
|
||||
```bash
|
||||
# 檢查誰在使用
|
||||
lsof -i :3002
|
||||
|
||||
# 停止舊的 server
|
||||
pkill -9 -f "momentry.*server"
|
||||
```
|
||||
|
||||
### Q: 如何確認使用的是哪個版本?
|
||||
```bash
|
||||
# 檢查 binary 位置和版本
|
||||
file $(which momentry)
|
||||
./target/release/momentry --version 2>/dev/null || echo "No version flag"
|
||||
```
|
||||
|
||||
### Q: 如何確認有沒有 API Key 驗證?
|
||||
```bash
|
||||
# 沒有 API Key 應該返回 401
|
||||
curl -s http://localhost:3002/api/v1/jobs
|
||||
# HTTP/1.1 401 Unauthorized
|
||||
```
|
||||
+321
@@ -0,0 +1,321 @@
|
||||
---
|
||||
document_type: "rca"
|
||||
service: "MARKBASE"
|
||||
title: "RCA:HTML Preview 被 MarkBase 歡迎頁面截圖取代"
|
||||
date: "2026-05-15"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "M4"
|
||||
created_by: "OpenCode"
|
||||
severity: "P2"
|
||||
tags:
|
||||
- "rca"
|
||||
- "markbase"
|
||||
- "html-preview"
|
||||
- "iframe-polling"
|
||||
- "sandbox"
|
||||
- "screenshot-overwrite"
|
||||
- "display-engine"
|
||||
- "cross-origin"
|
||||
ai_query_hints:
|
||||
- "查詢 MarkBase HTML preview 截圖 bug 的 RCA"
|
||||
- "為什麼 HTML iframe preview 被歡迎頁面覆蓋"
|
||||
- "Display.html polling 如何影響 MarkBase preview"
|
||||
- "sandbox 屬性如何修復 iframe JavaScript 執行"
|
||||
- "MarkBase version polling guard 三層檢查"
|
||||
related_documents:
|
||||
- "STANDARDS/DOCS_STANDARD.md"
|
||||
- "REFERENCE/MARKBASE_DESIGN_V2.0.md"
|
||||
- "~/markbase/src/page.html"
|
||||
- "~/markbase/src/server.rs"
|
||||
---
|
||||
|
||||
# RCA:HTML Preview 被 MarkBase 歡迎頁面截圖取代
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | M4 / OpenCode |
|
||||
| 建立時間 | 2026-05-15 |
|
||||
| 文件版本 | V1.0 |
|
||||
| 嚴重等級 | P2(中度影響,不影響系統穩定性,但嚴重影響使用者體驗) |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-05-15 | 初始 RCA 報告 | M4 / OpenCode | DeepSeek V4 Pro |
|
||||
|
||||
---
|
||||
|
||||
## 1. 事件摘要
|
||||
|
||||
### 1.1 問題描述
|
||||
|
||||
在 MarkBase 檔案樹中點擊 `.html` 檔案(如 `Display.html`)查看 preview 時,detail panel 內的 `<iframe>` 會短暫顯示正確的 HTML 渲染內容,但隨即被 MarkBase 歡迎頁面(含 📺 電視機 icon 和底部 control bar)的截圖完全取代。使用者看到的是「整個畫面被截圖取代」的視覺效果。
|
||||
|
||||
### 1.2 影響範圍
|
||||
|
||||
| 項目 | 說明 |
|
||||
|------|------|
|
||||
| 受影響功能 | detail panel + quick preview 中的 `.html` 檔案預覽 |
|
||||
| 受影響檔案 | `Display.html`(demo 目錄中的 MarkBase page template) |
|
||||
| 使用者體驗 | 預覽內容被覆蓋,無法正常查看 HTML 檔案渲染效果 |
|
||||
| 系統穩定性 | 不受影響 |
|
||||
|
||||
### 1.3 時間線
|
||||
|
||||
| 時間 | 事件 |
|
||||
|------|------|
|
||||
| 2026-05-15 09:00 | 使用者回報 `.html` 檔案預覽只顯示文字,無渲染 |
|
||||
| 2026-05-15 09:18 | 修改 `isTxt` 排除 `html`,將 `.html` 路由到 `isDocPdf`(`<iframe>` 渲染) |
|
||||
| 2026-05-15 09:24 | 修改 `stream_file` 加入 `text/html` MIME type |
|
||||
| 2026-05-15 09:40 | 使用者回報渲染後馬上被「歡迎頁面截圖」蓋掉 |
|
||||
| 2026-05-15 09:50 | 初次嘗試:加入 polling guard(`_tv` + overlay check) |
|
||||
| 2026-05-15 10:10 | 二次嘗試:polling guard 三層保護(interval 開頭 + `/version` callback + `/body` callback) |
|
||||
| 2026-05-15 10:20 | 三次嘗試:完全停用 polling(排除變因) |
|
||||
| 2026-05-15 10:30 | 使用者提供關鍵線索:「截圖包含底部 control bar」 |
|
||||
| 2026-05-15 10:35 | **發現根因**:`Display.html` 是完整 MarkBase page,內含 `setInterval` polling `/version` 和 `/body` |
|
||||
| 2026-05-15 10:40 | 修復方案:`<iframe sandbox>` + 刪除 `Display.html` 內的 polling scripts |
|
||||
| 2026-05-15 10:45 | 驗證通過 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 調查過程
|
||||
|
||||
### 2.1 初始假設:父頁面 polling
|
||||
|
||||
最初懷疑是父頁面 `page.html` 的 version polling(每 500ms 抓 `/version` + `/body`)在 detail panel 打開時替換了 `#mb-content`,導致視覺覆蓋。
|
||||
|
||||
**調查步驟**:
|
||||
1. 檢查 polling 機制:`setInterval` → `fetch("/version")` → `fetch("/body")` → `#mb-content.innerHTML = h`
|
||||
2. 確認 `#mb-overlay` 和 `#mb-detail` 位於 `#mb-content` **外部**(body-level elements)
|
||||
3. 理論上 `innerHTML` 替換不應影響 sibling elements
|
||||
4. 但仍加入三層 guard(interval 開頭 + version callback + body callback)確保 overlay 打開時跳過 polling
|
||||
|
||||
**結果**:問題未解決。使用者回報「還是一樣」。
|
||||
|
||||
### 2.2 第二次假設:race condition
|
||||
|
||||
懷疑 polling 檢查與 fetch callback 之間存在時間差,detail panel 在檢查後才開啟。
|
||||
|
||||
**調查步驟**:
|
||||
1. 在 version callback 和 body callback 內各加一層 overlay 檢查
|
||||
2. 確保整個非同步流程中任一時間點 overlay 打開時都跳過
|
||||
|
||||
**結果**:問題未解決。
|
||||
|
||||
### 2.3 第三次假設:完全停用 polling 排除變因
|
||||
|
||||
將父頁面 polling `setInterval` 完全註解掉,讓頁面不再自動刷新。
|
||||
|
||||
**結果**:問題仍然存在。確認根因**不在父頁面 polling**。
|
||||
|
||||
### 2.4 關鍵突破:使用者提供線索
|
||||
|
||||
使用者回報:「螢幕截圖是包含下面的 control bar」。
|
||||
|
||||
這個線索非常關鍵:**截圖包含了 `#mb-bar`(底部 control bar)**。如果只是 `#mb-content` 被替換,截圖不會包含 `#mb-bar`。這意味著:
|
||||
|
||||
- 產生截圖的動作會產生**完整頁面**的內容
|
||||
- 只有 `render::page()`(產生完整 HTML page template)才會包含 `#mb-bar`
|
||||
- `render::page()` 只在 `POST /display` 或初始化時被呼叫
|
||||
|
||||
### 2.5 根因發現
|
||||
|
||||
檢查 `Display.html` 檔案內容時發現,該檔案是**完整的 MarkBase page template**,包含:
|
||||
|
||||
```html
|
||||
<!-- Display.html 內含的 polling -->
|
||||
<script>
|
||||
var _v=-1;
|
||||
setInterval(function(){
|
||||
fetch("/version").then(r=>r.json()).then(d=>{
|
||||
if(d.v!=_v){_v=d.v;if(_v>=0)fetch("/body").then(r=>r.text()).then(h=>{
|
||||
var e=document.getElementById("mb-content");
|
||||
if(e){e.innerHTML=h;mermaid.run()}
|
||||
})
|
||||
})
|
||||
},500)
|
||||
</script>
|
||||
```
|
||||
|
||||
**攻擊鏈**:
|
||||
|
||||
```
|
||||
1. 使用者點擊 Display.html → showDetail → 建立 <iframe src="stream">
|
||||
2. iframe 載入 Display.html(同 origin: localhost:11438)
|
||||
3. Display.html 內的 setInterval 啟動(每 500ms)
|
||||
4. fetch("/version") → 取得 server 目前的 page version
|
||||
5. fetch("/body") → 取得 server 目前的顯示內容(MarkBase 歡迎頁)
|
||||
6. document.getElementById("mb-content").innerHTML = h
|
||||
→ iframe 內的 #mb-content 被替換為歡迎頁內容
|
||||
7. 歡迎頁內容含 📺 icon + 全部 CSS + #mb-bar 結構
|
||||
8. 使用者看到 iframe 內顯示完整歡迎頁 → 視覺效果 = 「截圖取代」
|
||||
```
|
||||
|
||||
**為什麼截圖包含 `#mb-bar`**:`/body` endpoint 的 `body_handler` 函數有截斷 bug:它尋找 `<div id=mb-content>` 後的第一個 `</div>`,但歡迎頁內容有巢狀 div。這導致回傳的 HTML fragment 包含部分 `#mb-bar` 之前的內容結構。
|
||||
|
||||
---
|
||||
|
||||
## 3. 根本原因分析
|
||||
|
||||
### 3.1 主要原因
|
||||
|
||||
| # | 原因 | 說明 |
|
||||
|---|------|------|
|
||||
| 1 | **`Display.html` 是完整 MarkBase page template** | 檔案內含 MarkBase 的 version polling JavaScript,會從 server 拉取當前顯示內容並替換自身的 `#mb-content` |
|
||||
| 2 | **iframe 在同 origin 下執行** | `localhost:11438` 的檔案在 iframe 中載入時,與父頁面共享 origin,iframe 內的 `fetch()` 直接打到同一個 server |
|
||||
| 3 | **`body_handler` 回傳 server 當前顯示狀態** | `/body` endpoint 回傳 `state.html` 中 `#mb-content` 的內容片段,而 server 當前顯示的是歡迎頁 |
|
||||
|
||||
### 3.2 次要原因
|
||||
|
||||
| # | 原因 | 說明 |
|
||||
|---|------|------|
|
||||
| 4 | **`body_handler` 截斷 bug** | `html[s..].find("</div>")` 只找第一個 `</div>`,巢狀結構時回傳不完整 HTML |
|
||||
| 5 | **quickPreview 的 iframe 缺少 `sandbox`** | 只有 `showDetail` 加了 `sandbox`,`quickPreview` 漏掉 |
|
||||
|
||||
### 3.3 因果鏈
|
||||
|
||||
```
|
||||
Display.html 被當作一般 HTML 檔案預覽
|
||||
→ 檔案本身是 MarkBase page template(含 polling JS)
|
||||
→ iframe 載入後 JS 開始執行(同 origin,無 sandbox 限制)
|
||||
→ JS 定期從 server 抓取當前顯示內容
|
||||
→ server 的 state.html 內容是歡迎頁
|
||||
→ 歡迎頁被注入 iframe 的 #mb-content
|
||||
→ 使用者看到 iframe 變成歡迎頁截圖
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 解決方案與實施
|
||||
|
||||
### 4.1 修復方案
|
||||
|
||||
| 層級 | 修復 | 檔案 | 說明 |
|
||||
|------|------|------|------|
|
||||
| **根因** | 刪除 `Display.html` 內 polling JS | `demo/Display.html` | 移除 `setInterval` / `fetch("/version")` / `fetch("/body")` 的 `<script>` blocks |
|
||||
| **防禦層 1** | iframe `sandbox` | `page.html` showDetail + quickPreview | `<iframe sandbox='allow-same-origin'>` 封鎖 JS 執行 |
|
||||
| **防禦層 2** | 父頁面 polling guard | `page.html` | 三層檢查:interval 開頭 + version callback + body callback |
|
||||
|
||||
### 4.2 實施細節
|
||||
|
||||
**修復 1:Display.html**
|
||||
```bash
|
||||
# 刪除包含 fetch("/version") 和 fetch("/body") 的 <script> blocks
|
||||
# 保留無害的 script(mermaid.initialize, volume, labels)
|
||||
```
|
||||
|
||||
**修復 2:page.html — showDetail**
|
||||
```javascript
|
||||
// Before
|
||||
h+="<iframe src='"+src+"' style='width:100%;height:400px;...'></iframe>";
|
||||
|
||||
// After
|
||||
h+="<iframe sandbox='allow-same-origin' src='"+src+"' style='...'></iframe>";
|
||||
```
|
||||
|
||||
**修復 3:page.html — quickPreview**
|
||||
```javascript
|
||||
// Before
|
||||
inner="<iframe src='"+src+"' style='width:90vw;height:85vh;...'></iframe>";
|
||||
|
||||
// After
|
||||
inner="<iframe sandbox='allow-same-origin' src='"+src+"' style='...'></iframe>";
|
||||
```
|
||||
|
||||
**修復 4:page.html — polling guard**
|
||||
```javascript
|
||||
setInterval(function(){
|
||||
if(_tv)return; // guard 1
|
||||
var ov=document.getElementById("mb-overlay");
|
||||
if(ov&&ov.style.display==="block")return; // guard 1
|
||||
fetch("/version").then(function(r){return r.json()}).then(function(d){
|
||||
if(d.v!=_v){_v=d.v;
|
||||
var ov2=document.getElementById("mb-overlay");
|
||||
if(ov2&&ov2.style.display==="block")return; // guard 2
|
||||
if(_v>=0)fetch("/body").then(function(r){return r.text()}).then(function(h){
|
||||
var ov3=document.getElementById("mb-overlay");
|
||||
if(ov3&&ov3.style.display==="block")return; // guard 3
|
||||
var e=document.getElementById("mb-content");
|
||||
if(e){e.innerHTML=h;mermaid.run()}
|
||||
})
|
||||
}})
|
||||
},500)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 預防措施
|
||||
|
||||
### 5.1 短期(已完成)
|
||||
|
||||
- [x] 上傳檔案時檢查是否為 MarkBase page template(檢測 polling `<script>` 特徵)
|
||||
- [x] 所有 preview iframe 預設加上 `sandbox='allow-same-origin'`
|
||||
- [x] 父頁面 polling 加入 guard(tree panel 或 detail panel 打開時跳過)
|
||||
|
||||
### 5.2 中期(建議)
|
||||
|
||||
- [ ] 建立上傳檔案安全掃描機制:檢測 `<script>` blocks,若發現 polling patterns 則警告或自動移除
|
||||
- [ ] 修復 `body_handler` 截斷 bug(使用真正的 HTML parser 而非 naive `find("</div>")`)
|
||||
- [ ] 將 `sandbox` 屬性抽取為共用常數,避免 showDetail/quickPreview 不同步
|
||||
|
||||
### 5.3 長期(建議)
|
||||
|
||||
- [ ] MarkBase page template 與 preview 分離:考慮移除 demo 目錄中的 MarkBase page template 檔案,或標記為 system file
|
||||
- [ ] 建立 Content Security Policy (CSP) header
|
||||
- [ ] iframe 內容類型判斷:對於 `text/html` 類型的 stream 回應,server 端可選擇性 strip `<script>` tags
|
||||
|
||||
---
|
||||
|
||||
## 6. 影響評估
|
||||
|
||||
| 面向 | 評估 |
|
||||
|------|------|
|
||||
| **直接影響** | `.html` 檔案預覽功能無法正常使用 |
|
||||
| **間接影響** | 使用者對 MarkBase preview 功能失去信心 |
|
||||
| **資料完整性** | 無影響(未修改任何資料) |
|
||||
| **安全性** | 無安全風險(僅為同一 origin 內的 UI 干擾) |
|
||||
| **修復成本** | 低(3 行程式碼 + 1 行 CSS + Display.html 清理) |
|
||||
|
||||
---
|
||||
|
||||
## 7. 經驗教訓
|
||||
|
||||
### 7.1 學到的教訓
|
||||
|
||||
1. **檔案內容可能包含可執行程式碼**:任何 `.html` 檔案在 iframe 中載入時,其 JavaScript 都會執行。對於預覽用途,必須 sandbox 隔離。
|
||||
2. **同 origin iframe 的危險**:`localhost` 下的所有內容共享同一個 origin。iframe 內的 JS 可以無限制地存取同 origin 的 API endpoints。
|
||||
3. **Demo 資料可能來自實際系統**:`Display.html` 是從 MarkBase page template 複製而來,保留了完整的 polling 機制。
|
||||
|
||||
### 7.2 最佳實踐
|
||||
|
||||
| 原則 | 說明 |
|
||||
|------|------|
|
||||
| **Always Sandbox User Content** | 任何使用者上傳的 HTML 檔案在 iframe 預覽時必須 sandbox |
|
||||
| **Don't Trust File Contents** | 檔案名稱(`.html`)不能保證內容是靜態的 |
|
||||
| **Guard at Multiple Layers** | 防禦應該在多個層級(檔案內容、iframe 屬性、父頁面邏輯)同時生效 |
|
||||
| **Key Clues Matter** | 使用者提供的「截圖包含底部 control bar」是找到根因的關鍵線索 |
|
||||
|
||||
---
|
||||
|
||||
## 8. 相關文件
|
||||
|
||||
| 文件 | 說明 |
|
||||
|------|------|
|
||||
| `~/markbase/src/page.html` | MarkBase 前端(含 polling guard + sandbox 修復) |
|
||||
| `~/markbase/src/server.rs` | MarkBase 後端(含 `stream_file` MIME 修復) |
|
||||
| `~/markbase/src/filetree/convert.rs` | 文件轉換模組(Phase 1: textutil + unzip) |
|
||||
| `REFERENCE/MARKBASE_DESIGN_V2.0.md` | MarkBase 設計文件 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 簽核
|
||||
|
||||
| 角色 | 姓名 | 日期 | 簽名 |
|
||||
|------|------|------|------|
|
||||
| 調查者 | M4 / OpenCode | 2026-05-15 | ✅ |
|
||||
| 審核者 | - | - | - |
|
||||
| 批准者 | - | - | - |
|
||||
Reference in New Issue
Block a user