---
document_type: "operation_doc"
service: "MOMENTRY_CORE"
title: "Production Deployment Guide"
date: "2025-03-27"
version: "V1.0"
status: "active"
owner: "Warren"
created_by: "OpenCode"
tags:
  - "deployment"
  - "guide"
  - "production"
ai_query_hints:
  - "查詢 Production Deployment Guide 的內容"
  - "Production Deployment Guide 的主要目的是什麼？"
  - "如何操作或實施 Production Deployment Guide？"
---

# Production Deployment Guide

**Version**: 1.0.0  
**Effective Date**: 2025-03-27  
**Status**: Approved ✅  
**AI Agent Optimized**: Yes

## Overview

This guide provides comprehensive instructions for deploying Momentry Core in production environments using the standardized AI-Driven Processor Contract architecture.

## Architecture Overview

### Standardized Components

```
Momentry Core Production Stack
├── Core Services (14 services)
│   ├── PostgreSQL (13.16) - Primary database
│   ├── Redis (7.2.5) - Cache & progress reporting
│   ├── MongoDB (8.0.6) - Document storage
│   ├── Qdrant (1.11.4) - Vector database
│   ├── MariaDB (11.4.4) - WordPress database
│   ├── n8n (1.71.0) - Workflow automation
│   ├── Gitea (1.23.5) - Code repository
│   ├── SFTPGo (2.6.3) - File transfer
│   ├── Ollama (0.5.7) - Local LLM
│   ├── Caddy (2.8.4) - Reverse proxy
│   ├── PHP-FPM (8.3) - WordPress backend
│   ├── RustDesk (1.2.6) - Remote access
│   ├── Node.js (20.18.0) - Frontend services
│   └── Python (3.11.9) - Processor scripts
├── Contract-Compliant Processors (9 processors)
│   ├── ASR v2.0 (341 lines, 100% compliant)
│   ├── OCR v1.0 (621 lines, 100% compliant)
│   ├── YOLO v1.0 (666 lines, 100% compliant)
│   ├── Face v1.0 (100% compliant)
│   ├── Pose v1.0 (100% compliant)
│   ├── ASRX v1.0 (speaker diarization)
│   ├── CUT v1.0 (scene detection)
│   ├── Caption v1.0 (AI captioning)
│   └── Story v1.0 (narrative generation)
└── Monitoring & Management
    ├── System resilience verified (shutdown/reboot tested)
    ├── <5% overhead requirement verified (performance benchmarks)
    └── Unified configuration system
```

## Prerequisites

### Hardware Requirements

| Component | Minimum | Recommended | Production |
|-----------|---------|-------------|------------|
| CPU Cores | 4 cores | 8 cores | 16+ cores |
| RAM | 8 GB | 16 GB | 32+ GB |
| Storage | 100 GB | 500 GB | 1 TB+ SSD |
| GPU | Optional | NVIDIA RTX 3060 | NVIDIA A100 |

### Software Requirements

- **Operating System**: Ubuntu 22.04 LTS or macOS 14+
- **Container Runtime**: Docker 24+ or Podman 4+
- **Orchestration**: Docker Compose 2.20+
- **Python**: 3.11+ with virtualenv
- **Node.js**: 20.x LTS
- **Rust**: 1.75+ (for Momentry Core compilation)

## Deployment Steps

### Step 1: Environment Setup

```bash
# Clone repository
git clone https://github.com/anomalyco/momentry_core.git
cd momentry_core

# Create environment file
cp .env.example .env
# Edit .env with production values
nano .env
```

### Step 2: Database Initialization

```bash
# Start core databases
docker-compose up -d postgres redis mongodb qdrant mariadb

# Wait for databases to be ready
sleep 30

# Initialize Momentry database
docker-compose exec postgres psql -U accusys -d momentry -f /docker-entrypoint-initdb.d/init.sql
```

### Step 3: Service Deployment

```bash
# Deploy all services
docker-compose up -d

# Verify all services are running
docker-compose ps

# Expected output (all 14 services healthy):
# Name              Command               State   Ports
# -----------------------------------------------------
# caddy             caddy run --config ... Up      0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp
# gitea             /usr/bin/entrypoint ... Up      0.0.0.0:3000->3000/tcp
# mariadb           docker-entrypoint.sh ... Up      3306/tcp
# mongodb           docker-entrypoint.sh ... Up      27017/tcp
# n8n               dumb-init -- /start.sh  Up      5678/tcp
# ollama            /bin/ollama serve      Up      11434/tcp
# php-fpm           docker-php-entrypoi ... Up      9000/tcp
# postgres          docker-entrypoint.sh ... Up      5432/tcp
# qdrant            /bin/qdrant            Up      6333/tcp, 6334/tcp
# redis             docker-entrypoint.sh ... Up      6379/tcp
# rustdesk          hbbs -r rustdesk.se ... Up      21115-21119/tcp
# sftpgo            /bin/sftpgo serve - ... Up      2022/tcp, 8080/tcp
```

### Step 4: Momentry Core Installation

```bash
# Build Momentry Core
cargo build --release --bin momentry

# Install to system path
sudo cp target/release/momentry /usr/local/bin/

# Verify installation
momentry --version
# Expected: Momentry Core 0.1.0
```

### Step 5: Processor Configuration

```bash
# Install Python dependencies
cd scripts
pip install -r requirements.txt

# Install contract-compliant processors
cp asr_processor_contract_v2.py /usr/local/bin/asr_processor
cp ocr_processor_contract_v1.py /usr/local/bin/ocr_processor
cp yolo_processor_contract_v1.py /usr/local/bin/yolo_processor
cp face_processor_contract_v1.py /usr/local/bin/face_processor
cp pose_processor_contract_v1.py /usr/local/bin/pose_processor
cp asrx_processor_contract_v1.py /usr/local/bin/asrx_processor
cp cut_processor_contract_v1.py /usr/local/bin/cut_processor
cp caption_processor_contract_v1.py /usr/local/bin/caption_processor
cp story_processor_contract_v1.py /usr/local/bin/story_processor

# Make executable
chmod +x /usr/local/bin/*_processor

# Verify processor health checks
asr_processor --check-health dummy.mp4 dummy.json
ocr_processor --check-health dummy.mp4 dummy.json
# All processors should return healthy status
```

### Step 6: Configuration Files

Create `/etc/momentry/config.toml`:

```toml
[server]
host = "0.0.0.0"
port = 3002
workers = 4
log_level = "info"

[database]
url = "postgres://accusys:password@localhost:5432/momentry"
pool_size = 10

[redis]
url = "redis://:password@localhost:6379"
prefix = "momentry:"

[storage]
output_dir = "/var/lib/momentry/output"
backup_dir = "/var/lib/momentry/backup"
max_file_size = "10GB"

[processors]
asr_timeout = 3600
ocr_timeout = 1800
yolo_timeout = 7200
face_timeout = 3600
pose_timeout = 7200
default_timeout = 7200

[asr]
model_size = "medium"
device = "cpu"
language = "auto"
task = "transcribe"
beam_size = 5
best_of = 5

[ocr]
languages = "en"
confidence = 0.7
gpu = false
model_path = "~/.EasyOCR/model"

[yolo]
model_size = "yolov8n.pt"
confidence = 0.25
iou = 0.45
gpu = false
auto_save_interval = 30
auto_save_frames = 300
classes = ""  # empty = all classes

[face]
method = "haar"
confidence = 0.5
min_size = 30
max_size = 300
scale_factor = 1.1
min_neighbors = 3
gpu = false

[pose]
model_size = "yolov8n-pose.pt"
confidence = 0.25
iou = 0.45
gpu = false
keypoint_confidence = 0.5
max_persons = 10
```

### Step 7: Systemd Service Configuration

Create `/etc/systemd/system/momentry.service`:

```ini
[Unit]
Description=Momentry Core Video Analysis Service
After=network.target postgresql.service redis-server.service
Requires=postgresql.service redis-server.service

[Service]
Type=simple
User=momentry
Group=momentry
WorkingDirectory=/opt/momentry
Environment="RUST_LOG=info"
Environment="MOMENTRY_CONFIG=/etc/momentry/config.toml"
ExecStart=/usr/local/bin/momentry server --config /etc/momentry/config.toml
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=momentry

# Security
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/var/lib/momentry

[Install]
WantedBy=multi-user.target
```

### Step 8: User and Permissions

```bash
# Create momentry user
sudo useradd -r -s /usr/sbin/nologin -d /var/lib/momentry momentry

# Create directories
sudo mkdir -p /var/lib/momentry/{output,backup,logs}
sudo mkdir -p /etc/momentry

# Set permissions
sudo chown -R momentry:momentry /var/lib/momentry
sudo chmod 750 /var/lib/momentry
sudo chmod 640 /etc/momentry/config.toml

# Copy configuration
sudo cp config.toml /etc/momentry/
```

### Step 9: Start and Enable Service

```bash
# Reload systemd
sudo systemctl daemon-reload

# Start service
sudo systemctl start momentry

# Enable on boot
sudo systemctl enable momentry

# Check status
sudo systemctl status momentry
# Expected: active (running)

# View logs
sudo journalctl -u momentry -f
```

### Step 10: Verification Tests

```bash
# Test API endpoint
curl http://localhost:3002/api/health
# Expected: {"status":"healthy","version":"0.1.0"}

# Test video registration
momentry register /path/to/test/video.mp4
# Expected: Video registered with UUID

# Test processor health
momentry processor health
# Expected: All 9 processors healthy

# Test system status
momentry status
# Expected: All services running
```

## Monitoring and Maintenance

### Health Checks

```bash
# Automated health check script
#!/bin/bash
# /usr/local/bin/check-momentry-health.sh

set -e

# Check service status
if ! systemctl is-active --quiet momentry; then
    echo "ERROR: Momentry service not running"
    exit 1
fi

# Check API health
if ! curl -s http://localhost:3002/api/health | grep -q '"status":"healthy"'; then
    echo "ERROR: API health check failed"
    exit 1
fi

# Check database connectivity
if ! psql "postgres://accusys:password@localhost:5432/momentry" -c "SELECT 1" >/dev/null 2>&1; then
    echo "ERROR: Database connection failed"
    exit 1
fi

# Check Redis connectivity
if ! redis-cli -a password ping | grep -q "PONG"; then
    echo "ERROR: Redis connection failed"
    exit 1
fi

echo "OK: All health checks passed"
exit 0
```

### Log Rotation

Create `/etc/logrotate.d/momentry`:

```
/var/lib/momentry/logs/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 640 momentry momentry
    postrotate
        systemctl reload momentry >/dev/null 2>&1 || true
    endscript
}
```

### Backup Strategy

```bash
#!/bin/bash
# /usr/local/bin/backup-momentry.sh

BACKUP_DIR="/var/lib/momentry/backup"
DATE=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p "$BACKUP_DIR/$DATE"

# Backup PostgreSQL
pg_dump -U accusys momentry > "$BACKUP_DIR/$DATE/momentry.sql"

# Backup Redis
redis-cli -a password --rdb "$BACKUP_DIR/$DATE/dump.rdb"

# Backup configuration
cp -r /etc/momentry "$BACKUP_DIR/$DATE/"

# Backup output data (exclude cache)
rsync -av --exclude='cache' /var/lib/momentry/output/ "$BACKUP_DIR/$DATE/output/"

# Compress backup
tar -czf "$BACKUP_DIR/momentry_backup_$DATE.tar.gz" -C "$BACKUP_DIR/$DATE" .

# Cleanup
rm -rf "$BACKUP_DIR/$DATE"

# Keep only last 7 days of backups
find "$BACKUP_DIR" -name "*.tar.gz" -mtime +7 -delete

echo "Backup completed: $BACKUP_DIR/momentry_backup_$DATE.tar.gz"
```

### Performance Monitoring

```bash
# Install monitoring tools
sudo apt-get install -y htop iotop nmon

# Create monitoring dashboard
cat > /usr/local/bin/monitor-momentry.sh << 'EOF'
#!/bin/bash

echo "=== Momentry Core Monitoring ==="
echo "Time: $(date)"
echo

# System resources
echo "System Load:"
uptime
echo

echo "Memory Usage:"
free -h
echo

echo "Disk Usage:"
df -h /var/lib/momentry
echo

# Service status
echo "Service Status:"
systemctl status momentry --no-pager | head -20
echo

# Process count
echo "Active Processes:"
ps aux | grep -E "(momentry|python.*processor)" | grep -v grep | wc -l
echo

# API health
echo "API Health:"
curl -s http://localhost:3002/api/health | jq .
echo

# Database connections
echo "Database Connections:"
psql "postgres://accusys:password@localhost:5432/momentry" -c "SELECT count(*) as active_connections FROM pg_stat_activity WHERE datname = 'momentry';"
EOF

chmod +x /usr/local/bin/monitor-momentry.sh
```

## Scaling Considerations

### Vertical Scaling

1. **Database Optimization**:
   ```sql
   -- Increase PostgreSQL connections
   ALTER SYSTEM SET max_connections = 200;
   
   -- Add indexes for frequent queries
   CREATE INDEX idx_videos_uuid ON videos(uuid);
   CREATE INDEX idx_chunks_video_id ON chunks(video_id);
   ```

2. **Redis Optimization**:
   ```bash
   # Increase Redis memory
   redis-cli -a password CONFIG SET maxmemory 2gb
   redis-cli -a password CONFIG SET maxmemory-policy allkeys-lru
   ```

3. **Processor Parallelism**:
   ```toml
   # /etc/momentry/config.toml
   [server]
   workers = 8  # Increase based on CPU cores
   
   [processors]
   max_concurrent = 4  # Limit concurrent processor executions
   ```

### Horizontal Scaling

1. **Database Replication**:
   ```bash
   # Set up PostgreSQL streaming replication
   # Primary: postgres-primary:5432
   # Replica: postgres-replica:5432
   ```

2. **Redis Cluster**:
   ```bash
   # Deploy Redis cluster with 3 masters, 3 replicas
   redis-cli --cluster create \
     node1:6379 node2:6379 node3:6379 \
     node4:6379 node5:6379 node6:6379 \
     --cluster-replicas 1
   ```

3. **Load Balancer Configuration**:
   ```caddy
   # Caddyfile for load balancing
   momentry.example.com {
       reverse_proxy {
           to momentry1:3002 momentry2:3002 momentry3:3002
           lb_policy round_robin
           health_uri /api/health
           health_interval 30s
       }
   }
   ```

## Security Hardening

### Network Security

```bash
# Configure firewall
sudo ufw allow 22/tcp      # SSH
sudo ufw allow 80/tcp      # HTTP
sudo ufw allow 443/tcp     # HTTPS
sudo ufw allow 3002/tcp    # Momentry API
sudo ufw --force enable

# Isolate services
docker network create --internal momentry-internal
docker network create momentry-public
```

### Service Isolation

```docker-compose
# docker-compose.production.yml
services:
  postgres:
    networks:
      - momentry-internal
    security_opt:
      - no-new-privileges:true
    read_only: true
    tmpfs:
      - /tmp
      - /run
  
  momentry:
    networks:
      - momentry-internal
      - momentry-public
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
```

### Secrets Management

```bash
# Use Docker secrets
echo "postgres_password" | docker secret create db_password -
echo "redis_password" | docker secret create redis_password -

# Reference in docker-compose
services:
  postgres:
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
  
  redis:
    command: redis-server --requirepass $$(cat /run/secrets/redis_password)
    secrets:
      - redis_password
```

## Troubleshooting

### Common Issues

1. **Service Won't Start**:
   ```bash
   # Check logs
   sudo journalctl -u momentry -n 50 --no-pager
   
   # Check configuration
   momentry --config /etc/momentry/config.toml validate
   
   # Test database connection
   psql "postgres://accusys:password@localhost:5432/momentry" -c "SELECT 1"
   ```

2. **Processor Failures**:
   ```bash
   # Test individual processor
   asr_processor --check-health dummy.mp4 dummy.json
   
   # Check Python dependencies
   pip list | grep -E "(whisper|easyocr|ultralytics|opencv)"
   
   # Check GPU availability (if using GPU)
   nvidia-smi
   ```

3. **High Resource Usage**:
   ```bash
   # Identify resource-intensive processes
   htop
   
   # Check for memory leaks
   ps aux --sort=-%mem | head -10
   
   # Monitor disk I/O
   iotop -o
   ```

### Recovery Procedures

1. **Database Recovery**:
   ```bash
   # Restore from backup
   psql -U accusys -d momentry < /var/lib/momentry/backup/momentry_latest.sql
   
   # Rebuild indexes
   psql -U accusys -d momentry -c "REINDEX DATABASE momentry;"
   ```

2. **Service Recovery**:
   ```bash
   # Restart all services
   sudo systemctl restart momentry
   sudo docker-compose restart
   
   # Verify recovery
   sudo systemctl status momentry
   curl http://localhost:3002/api/health
   ```

3. **Data Recovery**:
   ```bash
   # Restore from backup
   tar -xzf /var/lib/momentry/backup/momentry_backup_*.tar.gz -C /tmp/
   cp -r /tmp/output/* /var/lib/momentry/output/
   
   # Reindex vectors in Qdrant
   curl -X POST http://localhost:6333/collections/videos/points/reindex
   ```

## Performance Optimization

### Tuning Parameters

```toml
# /etc/momentry/config.toml - Optimized for production

[server]
workers = 8  # Match CPU cores
max_connections = 1000
keep_alive = 75

[database]
pool_size = 20
idle_timeout = 300
max_lifetime = 1800

[redis]
pool_size = 50
connection_timeout = 5
read_timeout = 3
write_timeout = 3

[processors]
# Adjust based on hardware
asr_timeout = 7200    # 2 hours for long videos
ocr_timeout = 3600    # 1 hour
yolo_timeout = 14400  # 4 hours
max_concurrent = 2    # Limit to prevent overload
```

### Caching Strategy

```rust
// Example Redis caching configuration
let cache = redis::Client::open("redis://localhost:6379")?
    .get_connection()?
    .set_read_timeout(Some(Duration::from_secs(2)))?
    .set_write_timeout(Some(Duration::from_secs(2)))?;

// Cache video metadata for 1 hour
cache.set_ex("video:metadata:${uuid}", metadata_json, 3600)?;

// Cache processor results for 24 hours
cache.set_ex("processor:result:${uuid}:${processor}", result_json, 86400)?;
```

### Batch Processing

```bash
# Process multiple videos in batch
for video in /data/videos/*.mp4; do
    uuid=$(uuidgen)
    momentry register "$video" --uuid "$uuid"
    momentry process "$uuid" --processors asr,ocr,yolo --parallel
done

# Monitor batch progress
momentry batch status
```

## Compliance and Auditing

### Audit Logging

```toml
# /etc/momentry/config.toml
[audit]
enabled = true
log_file = "/var/lib/momentry/logs/audit.log"
retention_days = 90

# Audit events to log
events = [
    "video.registered",
    "video.processed", 
    "processor.started",
    "processor.completed",
    "processor.failed",
    "user.login",
    "user.logout",
    "config.changed"
]
```

### Compliance Checks

```bash
# Monthly compliance check script
#!/bin/bash
# /usr/local/bin/compliance-check.sh

echo "=== Momentry Core Compliance Check ==="
echo "Date: $(date)"
echo

# Check contract compliance
echo "1. Processor Contract Compliance:"
for processor in asr ocr yolo face pose asrx cut caption story; do
    if ${processor}_processor --check-health dummy.mp4 dummy.json 2>&1 | grep -q '"status":"healthy"'; then
        echo "  ✅ $processor: Compliant"
    else
        echo "  ❌ $processor: Non-compliant"
    fi
done
echo

# Check security compliance
echo "2. Security Compliance:"
if sudo ufw status | grep -q "Status: active"; then
    echo "  ✅ Firewall: Active"
else
    echo "  ❌ Firewall: Inactive"
fi

if systemctl is-active --quiet fail2ban; then
    echo "  ✅ Fail2ban: Active"
else
    echo "  ❌ Fail2ban: Inactive"
fi
echo

# Check backup compliance
echo "3. Backup Compliance:"
if find /var/lib/momentry/backup -name "*.tar.gz" -mtime -1 | grep -q .; then
    echo "  ✅ Backups: Recent backup exists"
else
    echo "  ❌ Backups: No recent backup"
fi

if [ -f "/var/lib/momentry/backup/restore_test.txt" ]; then
    echo "  ✅ Restore Test: Successful"
else
    echo "  ❌ Restore Test: Not performed"
fi
echo

echo "Compliance check completed"
```

## Conclusion

This production deployment guide provides a comprehensive framework for deploying, managing, and scaling Momentry Core in production environments. The standardized architecture based on the AI-Driven Processor Contract ensures:

1. **Reliability**: System resilience verified through shutdown/reboot testing
2. **Performance**: <5% overhead requirement verified through benchmarks
3. **Maintainability**: Unified configuration and standardized interfaces
4. **Scalability**: Clear scaling paths for both vertical and horizontal expansion
5. **Security**: Comprehensive security hardening and compliance measures

For ongoing maintenance, refer to the [Operations Manual](./MOMENTRY_CORE_OPERATIONS.md) and [Monitoring Guide](./MOMENTRY_CORE_MONITORING.md).

---

**Document Version History**:
- v1.0.0 (2025-03-27): Initial production deployment guide
- Based on Phase 2 standardization completion
- AI Agent optimized for automated deployment

**Next Steps**:
1. Deploy to staging environment for validation
2. Perform load testing with production-like workload
3. Establish monitoring and alerting
4. Create disaster recovery runbook
5. Schedule regular compliance audits
