warren/momentry_core
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 2 Wiki Activity

refactor: remove face embedding architecture - single Qdrant _faces collection

Browse Source 
- Delete FaceEmbeddingDb module (face_embedding_db.rs)
- Stub match_faces_iterative, generate_seed_embeddings, tmdb_match_handler
- Remove sync_trace_embeddings, populate_face_embeddings_to_qdrant
- Remove embedding from face.json output (face_processor.py)
- Remove embedding from PG UPDATE (store_traced_faces.py)
- Remove workspace traces staging (checkin.rs, qdrant_workspace.rs)
- Fix tests: add pose_angle to Face, hand_nodes to TkgResult

Disabled functions (need reimplement with _faces):
- match_faces_iterative (identity agent)
- generate_seed_embeddings (TMDb seeds)
- tmdb_match_handler (TMDb matching)
- cluster_face_embeddings, search_similar_faces
- merge_traces_within_cuts
This commit is contained in:
Accusys
2026-06-24 22:27:09 +08:00
parent 360cb991e1
commit 074cdcdbed
60 changed files with 657 additions and 9454 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_v1.0/API_WORKSPACE/modules/05_process.md
+102 -3
View File
@@ -51,8 +51,8 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
| `success` | boolean | Always true on 200 |
| `job_id` | integer | Monitor job ID (for job tracking) |
| `file_uuid` | string | 32-char hex UUID of the file |
| `status` | string | `"processing"` |
| `pids` | integer[] | Process IDs of started processors |
| `status` | string | `"queued"` — file enters the FIFO queue |
| `pids` | integer[] | Process IDs of started processors (empty for queued) |
| `message` | string | Human-readable status |
#### Error Responses
@@ -237,6 +237,105 @@ curl -s "$API/api/v1/jobs" -H "X-API-Key: $KEY" | jq '{count, jobs: [.jobs[] | {
| `page` | integer | Current page number |
| `page_size` | integer | Jobs per page |
### `GET /api/v1/job/:uuid`
**Auth**: Required
**Scope**: file-level
Get detailed information about a specific processing job, including its queue position.
#### Response (200)
```json
{
"id": 51,
"uuid": "c36f35685177c981aa139b66bbbccc5b",
"status": "queued",
"current_processor": null,
"progress_current": 0,
"progress_total": 0,
"processors": [],
"created_at": "2026-06-22 23:08:48.497018",
"started_at": null,
"updated_at": null,
"queue_position": 3
}
```
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Monitor job ID |
| `uuid` | string | File UUID |
| `status` | string | `"pending"`, `"queued"`, `"running"`, `"completed"`, `"failed"` |
| `current_processor` | string | Currently active processor, or null |
| `progress_current` | integer | Current progress count |
| `progress_total` | integer | Total progress count |
| `processors` | array | Processor list |
| `created_at` | string | Job creation timestamp |
| `started_at` | string | Processing start timestamp, or null |
| `updated_at` | string | Last update timestamp, or null |
| `queue_position` | integer | Position in FIFO queue (null if not pending/queued) |
---
### Status Lifecycle
```
register ──→ pending
│
trigger (POST /process)
│
queued ←── queue_position counts jobs ahead
│
worker picks up
│
processing
│
┌────────┴────────┐
▼ ▼
completed failed
│
checkin ──→ indexed
checkout ──→ checked_out
```
| Status | Meaning |
|--------|---------|
| `pending` | File registered, not yet triggered |
| `queued` | Triggered, waiting for worker in FIFO queue |
| `processing` | Worker actively processing |
| `completed` | All processors finished successfully |
| `failed` | One or more essential processors failed |
| `indexed` | Post-processing checkin complete |
| `checked_out` | User checked out the file |
Queue order is FIFO (`created_at ASC`). The `GET /api/v1/job/:uuid` endpoint returns `queue_position` showing how many jobs are ahead.
### Frontend Status Mapping
When displaying file status in the frontend list (e.g. after `GET /api/v1/files/scan`), map the `status` field as follows:
| DB Status | Status Label | Filter: 待處理 | Filter: 處理中 | Count: pendingCount | Count: processingCount |
|-----------|-------------|----------------|----------------|---------------------|-----------------------|
| `unregistered` | 未註冊 | No | No | No | No |
| `registered` | 待處理 | **Yes** | No | **Yes** | No |
| `pending` | 待處理 | **Yes** | No | **Yes** | No |
| `queued` | 排隊中 | **Yes** | **Yes** | **Yes** | **Yes** |
| `processing` | 處理中 | No | **Yes** | No | **Yes** |
| `completed` | 已完成 | No | No | No | No |
| `failed` | 處理失敗 | No | No | No | No |
| `indexed` | 已入庫 | No | No | No | No |
**`queued` 的特殊處理**：
- `statusLabel` → 顯示「排隊中」，加 `ms-badge-warn` 樣式（黃色）
- `filterPending` → 應包含 `queued`，讓它在「待處理」filter 可見
- `pendingCount` + `processingCount` → 兩者都應包含 `queued`，因它既是「待處理」也是「正在排隊」
- 在 `refreshAllStatus` / `loadFiles` 中，如果檔案狀態是 `queued`，應顯示簡單的排隊訊息（無需 polling progress）
- 當 worker pickup 後，狀態會變為 `processing`，此時 `refreshAllStatus` 會自動偵測到並開始 polling progress
- 也可以提供一個「queue_position」顯示：呼叫 `GET /api/v1/job/:uuid` 取得排在第幾位
---
### `GET /api/v1/file/:file_uuid/processor-counts`
**Auth**: Required
@@ -407,4 +506,4 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/complete" \
Phase 1 (`/phase1`) combines store-asrx + rule1 + vectorize into one call.
---
*Updated: 2026-06-20 12:00:00*
*Updated: 2026-06-23 — Added queued status, FIFO queue order, queue_position in job detail, frontend status mapping table*
docs_v1.0/doc/05_process.html
+236 -3
View File
@@ -119,12 +119,12 @@ curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>
<tr>
<td><code>status</code></td>
<td>string</td>
<td><code>"processing"</code></td>
<td><code>"queued"</code> — file enters the FIFO queue</td>
</tr>
<tr>
<td><code>pids</code></td>
<td>integer[]</td>
<td>Process IDs of started processors</td>
<td>Process IDs of started processors (empty for queued)</td>
</tr>
<tr>
<td><code>message</code></td>
@@ -507,6 +507,239 @@ curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>
</tr>
</tbody>
</table>
<h3><code>GET /api/v1/job/:uuid</code></h3>
<p><strong>Auth</strong>: Required
<strong>Scope</strong>: file-level</p>
<p>Get detailed information about a specific processing job, including its queue position.</p>
<h4>Response (200)</h4>
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
<span class="w"> </span><span class="nt">&quot;id&quot;</span><span class="p">:</span><span class="w"> </span><span class="mi">51</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;uuid&quot;</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;c36f35685177c981aa139b66bbbccc5b&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;status&quot;</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;queued&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;current_processor&quot;</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;progress_current&quot;</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;progress_total&quot;</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;processors&quot;</span><span class="p">:</span><span class="w"> </span><span class="p">[],</span>
<span class="w"> </span><span class="nt">&quot;created_at&quot;</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;2026-06-22 23:08:48.497018&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;started_at&quot;</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;updated_at&quot;</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span>
<span class="w"> </span><span class="nt">&quot;queue_position&quot;</span><span class="p">:</span><span class="w"> </span><span class="mi">3</span>
<span class="p">}</span>
</code></pre></div>
<table class="table">
<thead>
<tr>
<th>Field</th>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>id</code></td>
<td>integer</td>
<td>Monitor job ID</td>
</tr>
<tr>
<td><code>uuid</code></td>
<td>string</td>
<td>File UUID</td>
</tr>
<tr>
<td><code>status</code></td>
<td>string</td>
<td><code>"pending"</code>, <code>"queued"</code>, <code>"running"</code>, <code>"completed"</code>, <code>"failed"</code></td>
</tr>
<tr>
<td><code>current_processor</code></td>
<td>string</td>
<td>Currently active processor, or null</td>
</tr>
<tr>
<td><code>progress_current</code></td>
<td>integer</td>
<td>Current progress count</td>
</tr>
<tr>
<td><code>progress_total</code></td>
<td>integer</td>
<td>Total progress count</td>
</tr>
<tr>
<td><code>processors</code></td>
<td>array</td>
<td>Processor list</td>
</tr>
<tr>
<td><code>created_at</code></td>
<td>string</td>
<td>Job creation timestamp</td>
</tr>
<tr>
<td><code>started_at</code></td>
<td>string</td>
<td>Processing start timestamp, or null</td>
</tr>
<tr>
<td><code>updated_at</code></td>
<td>string</td>
<td>Last update timestamp, or null</td>
</tr>
<tr>
<td><code>queue_position</code></td>
<td>integer</td>
<td>Position in FIFO queue (null if not pending/queued)</td>
</tr>
</tbody>
</table>
<hr />
<h3>Status Lifecycle</h3>
<div class="codehilite"><pre><span></span><code><span class="n">register</span><span class="w"> </span><span class="err">──→</span><span class="w"> </span><span class="n">pending</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="n">trigger</span><span class="w"> </span><span class="p">(</span><span class="n">POST</span><span class="w"> </span><span class="o">/</span><span class="n">process</span><span class="p">)</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="n">queued</span><span class="w"> </span><span class="err">←──</span><span class="w"> </span><span class="n">queue_position</span><span class="w"> </span><span class="n">counts</span><span class="w"> </span><span class="n">jobs</span><span class="w"> </span><span class="n">ahead</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="n">worker</span><span class="w"> </span><span class="n">picks</span><span class="w"> </span><span class="n">up</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="n">processing</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="err">┌────────┴────────┐</span>
<span class="w"> </span><span class="err">▼</span><span class="w"> </span><span class="err">▼</span>
<span class="w"> </span><span class="n">completed</span><span class="w"> </span><span class="n">failed</span>
<span class="w"> </span><span class="err">│</span>
<span class="w"> </span><span class="n">checkin</span><span class="w"> </span><span class="err">──→</span><span class="w"> </span><span class="n">indexed</span>
<span class="w"> </span><span class="n">checkout</span><span class="w"> </span><span class="err">──→</span><span class="w"> </span><span class="n">checked_out</span>
</code></pre></div>
<table class="table">
<thead>
<tr>
<th>Status</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>pending</code></td>
<td>File registered, not yet triggered</td>
</tr>
<tr>
<td><code>queued</code></td>
<td>Triggered, waiting for worker in FIFO queue</td>
</tr>
<tr>
<td><code>processing</code></td>
<td>Worker actively processing</td>
</tr>
<tr>
<td><code>completed</code></td>
<td>All processors finished successfully</td>
</tr>
<tr>
<td><code>failed</code></td>
<td>One or more essential processors failed</td>
</tr>
<tr>
<td><code>indexed</code></td>
<td>Post-processing checkin complete</td>
</tr>
<tr>
<td><code>checked_out</code></td>
<td>User checked out the file</td>
</tr>
</tbody>
</table>
<p>Queue order is FIFO (<code>created_at ASC</code>). The <code>GET /api/v1/job/:uuid</code> endpoint returns <code>queue_position</code> showing how many jobs are ahead.</p>
<h3>Frontend Status Mapping</h3>
<p>When displaying file status in the frontend list (e.g. after <code>GET /api/v1/files/scan</code>), map the <code>status</code> field as follows:</p>
<table class="table">
<thead>
<tr>
<th>DB Status</th>
<th>Status Label</th>
<th>Filter: 待處理</th>
<th>Filter: 處理中</th>
<th>Count: pendingCount</th>
<th>Count: processingCount</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>unregistered</code></td>
<td>未註冊</td>
<td>No</td>
<td>No</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td><code>registered</code></td>
<td>待處理</td>
<td><strong>Yes</strong></td>
<td>No</td>
<td><strong>Yes</strong></td>
<td>No</td>
</tr>
<tr>
<td><code>pending</code></td>
<td>待處理</td>
<td><strong>Yes</strong></td>
<td>No</td>
<td><strong>Yes</strong></td>
<td>No</td>
</tr>
<tr>
<td><code>queued</code></td>
<td>排隊中</td>
<td><strong>Yes</strong></td>
<td><strong>Yes</strong></td>
<td><strong>Yes</strong></td>
<td><strong>Yes</strong></td>
</tr>
<tr>
<td><code>processing</code></td>
<td>處理中</td>
<td>No</td>
<td><strong>Yes</strong></td>
<td>No</td>
<td><strong>Yes</strong></td>
</tr>
<tr>
<td><code>completed</code></td>
<td>已完成</td>
<td>No</td>
<td>No</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td><code>failed</code></td>
<td>處理失敗</td>
<td>No</td>
<td>No</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td><code>indexed</code></td>
<td>已入庫</td>
<td>No</td>
<td>No</td>
<td>No</td>
<td>No</td>
</tr>
</tbody>
</table>
<p><strong><code>queued</code> 的特殊處理</strong>：
- <code>statusLabel</code> → 顯示「排隊中」，加 <code>ms-badge-warn</code> 樣式（黃色）
- <code>filterPending</code> → 應包含 <code>queued</code>，讓它在「待處理」filter 可見
- <code>pendingCount</code> + <code>processingCount</code> → 兩者都應包含 <code>queued</code>，因它既是「待處理」也是「正在排隊」
- 在 <code>refreshAllStatus</code> / <code>loadFiles</code> 中，如果檔案狀態是 <code>queued</code>，應顯示簡單的排隊訊息（無需 polling progress）
- 當 worker pickup 後，狀態會變為 <code>processing</code>，此時 <code>refreshAllStatus</code> 會自動偵測到並開始 polling progress
- 也可以提供一個「queue_position」顯示：呼叫 <code>GET /api/v1/job/:uuid</code> 取得排在第幾位</p>
<hr />
<h3><code>GET /api/v1/file/:file_uuid/processor-counts</code></h3>
<p><strong>Auth</strong>: Required
<strong>Scope</strong>: file-level</p>
@@ -652,7 +885,7 @@ curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>
<p>Phase 1 (<code>/phase1</code>) combines store-asrx + rule1 + vectorize into one call.</p>
<hr />
<p><em>Updated: 2026-06-20 12:00:00</em></p>
<p><em>Updated: 2026-06-23 — Added queued status, FIFO queue order, queue_position in job detail, frontend status mapping table</em></p>
</div>
</body>
</html>
docs_v1.0/doc_wasm/modules/05_process.md
+102 -3
View File
@@ -51,8 +51,8 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
| `success` | boolean | Always true on 200 |
| `job_id` | integer | Monitor job ID (for job tracking) |
| `file_uuid` | string | 32-char hex UUID of the file |
| `status` | string | `"processing"` |
| `pids` | integer[] | Process IDs of started processors |
| `status` | string | `"queued"` — file enters the FIFO queue |
| `pids` | integer[] | Process IDs of started processors (empty for queued) |
| `message` | string | Human-readable status |
#### Error Responses
@@ -237,6 +237,105 @@ curl -s "$API/api/v1/jobs" -H "X-API-Key: $KEY" | jq '{count, jobs: [.jobs[] | {
| `page` | integer | Current page number |
| `page_size` | integer | Jobs per page |
### `GET /api/v1/job/:uuid`
**Auth**: Required
**Scope**: file-level
Get detailed information about a specific processing job, including its queue position.
#### Response (200)
```json
{
"id": 51,
"uuid": "c36f35685177c981aa139b66bbbccc5b",
"status": "queued",
"current_processor": null,
"progress_current": 0,
"progress_total": 0,
"processors": [],
"created_at": "2026-06-22 23:08:48.497018",
"started_at": null,
"updated_at": null,
"queue_position": 3
}
```
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Monitor job ID |
| `uuid` | string | File UUID |
| `status` | string | `"pending"`, `"queued"`, `"running"`, `"completed"`, `"failed"` |
| `current_processor` | string | Currently active processor, or null |
| `progress_current` | integer | Current progress count |
| `progress_total` | integer | Total progress count |
| `processors` | array | Processor list |
| `created_at` | string | Job creation timestamp |
| `started_at` | string | Processing start timestamp, or null |
| `updated_at` | string | Last update timestamp, or null |
| `queue_position` | integer | Position in FIFO queue (null if not pending/queued) |
---
### Status Lifecycle
```
register ──→ pending
│
trigger (POST /process)
│
queued ←── queue_position counts jobs ahead
│
worker picks up
│
processing
│
┌────────┴────────┐
▼ ▼
completed failed
│
checkin ──→ indexed
checkout ──→ checked_out
```
| Status | Meaning |
|--------|---------|
| `pending` | File registered, not yet triggered |
| `queued` | Triggered, waiting for worker in FIFO queue |
| `processing` | Worker actively processing |
| `completed` | All processors finished successfully |
| `failed` | One or more essential processors failed |
| `indexed` | Post-processing checkin complete |
| `checked_out` | User checked out the file |
Queue order is FIFO (`created_at ASC`). The `GET /api/v1/job/:uuid` endpoint returns `queue_position` showing how many jobs are ahead.
### Frontend Status Mapping
When displaying file status in the frontend list (e.g. after `GET /api/v1/files/scan`), map the `status` field as follows:
| DB Status | Status Label | Filter: 待處理 | Filter: 處理中 | Count: pendingCount | Count: processingCount |
|-----------|-------------|----------------|----------------|---------------------|-----------------------|
| `unregistered` | 未註冊 | No | No | No | No |
| `registered` | 待處理 | **Yes** | No | **Yes** | No |
| `pending` | 待處理 | **Yes** | No | **Yes** | No |
| `queued` | 排隊中 | **Yes** | **Yes** | **Yes** | **Yes** |
| `processing` | 處理中 | No | **Yes** | No | **Yes** |
| `completed` | 已完成 | No | No | No | No |
| `failed` | 處理失敗 | No | No | No | No |
| `indexed` | 已入庫 | No | No | No | No |
**`queued` 的特殊處理**：
- `statusLabel` → 顯示「排隊中」，加 `ms-badge-warn` 樣式（黃色）
- `filterPending` → 應包含 `queued`，讓它在「待處理」filter 可見
- `pendingCount` + `processingCount` → 兩者都應包含 `queued`，因它既是「待處理」也是「正在排隊」
- 在 `refreshAllStatus` / `loadFiles` 中，如果檔案狀態是 `queued`，應顯示簡單的排隊訊息（無需 polling progress）
- 當 worker pickup 後，狀態會變為 `processing`，此時 `refreshAllStatus` 會自動偵測到並開始 polling progress
- 也可以提供一個「queue_position」顯示：呼叫 `GET /api/v1/job/:uuid` 取得排在第幾位
---
### `GET /api/v1/file/:file_uuid/processor-counts`
**Auth**: Required
@@ -407,4 +506,4 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/complete" \
Phase 1 (`/phase1`) combines store-asrx + rule1 + vectorize into one call.
---
*Updated: 2026-06-20 12:00:00*
*Updated: 2026-06-23 — Added queued status, FIFO queue order, queue_position in job detail, frontend status mapping table*