# ANE Embedding 安裝指南（給 M4）

## 前置需求

```bash
# 1. 確認 Python venv 有 coremltools + transformers
pip install coremltools transformers torch numpy
```

## 轉換腳本

將以下內容存為 `convert_embed_to_coreml.py` 並執行：

```python
import os, numpy as np
from pathlib import Path
import torch
from transformers import AutoModel, AutoTokenizer
import coremltools as ct

MLPACKAGE_PATH = "/Users/accusys/momentry/mxbai-embed-large-v1.mlpackage"

print("Loading mxbai-embed-large-v1...")
model_name = "mixedbread-ai/mxbai-embed-large-v1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
print(f"Model: BERT {model.config.hidden_size}D, {model.config.num_hidden_layers} layers")
model.eval()

class EmbeddingWrapper(torch.nn.Module):
    def __init__(self, model):
        super().__init__()
        self.model = model
    def forward(self, input_ids, attention_mask):
        outputs = self.model(input_ids=input_ids, attention_mask=attention_mask)
        token_emb = outputs.last_hidden_state
        mask = attention_mask.unsqueeze(-1).expand(token_emb.size()).float()
        return torch.sum(token_emb * mask, 1) / torch.clamp(mask.sum(1), min=1e-9)

wrapper = EmbeddingWrapper(model).eval()
example = tokenizer("Hello world", return_tensors="pt", padding="max_length", truncation=True, max_length=512)

print("Converting to CoreML...")
traced = torch.jit.trace(wrapper, (example["input_ids"], example["attention_mask"]))

ct_model = ct.convert(
    traced,
    inputs=[
        ct.TensorType(name="input_ids", shape=(1, 512), dtype=np.int32),
        ct.TensorType(name="attention_mask", shape=(1, 512), dtype=np.int32),
    ],
    outputs=[
        ct.TensorType(name="embedding", dtype=np.float16),
    ],
    minimum_deployment_target=ct.target.macOS14,
    compute_precision=ct.precision.FLOAT16,
    compute_units=ct.ComputeUnit.ALL,
)

ct_model.save(MLPACKAGE_PATH)
size_mb = sum(f.stat().st_size for f in Path(MLPACKAGE_PATH).rglob('*') if f.is_file()) / 1e6
print(f"Saved: {MLPACKAGE_PATH} ({size_mb:.1f} MB)")
print("Done!")
```

## 測試腳本

存為 `test_coreml_embed.py`：

```python
import numpy as np, time
import coremltools as ct
from transformers import AutoTokenizer

MLPACKAGE_PATH = "/Users/accusys/momentry/mxbai-embed-large-v1.mlpackage"
mlmodel = ct.models.MLModel(MLPACKAGE_PATH, compute_units=ct.ComputeUnit.ALL)
tokenizer = AutoTokenizer.from_pretrained("mixedbread-ai/mxbai-embed-large-v1")

texts = {
    "English": "Hello and welcome to the old time movie show.",
    "Chinese": "歡迎收看老電影節目。",
}

for name, text in texts.items():
    tokens = tokenizer(text, return_tensors="np", padding="max_length", truncation=True, max_length=512)
    input_ids = tokens["input_ids"].astype(np.int32)
    attention_mask = tokens["attention_mask"].astype(np.int32)
    
    for _ in range(3):  # warmup
        mlmodel.predict({"input_ids": input_ids, "attention_mask": attention_mask})
    
    times = []
    for _ in range(10):
        t0 = time.perf_counter()
        result = mlmodel.predict({"input_ids": input_ids, "attention_mask": attention_mask})
        t1 = time.perf_counter()
        times.append((t1 - t0) * 1000)
    
    avg = sum(times) / len(times)
    emb = result["embedding"]
    print(f"{name:10s}: avg={avg:.1f}ms  dim={len(emb[0])}")

print("\nDone! ANE embedding is working.")
```

## 執行

```bash
cd /Users/accusys/momentry_core_0.1
python3 convert_embed_to_coreml.py   # 第一次轉換（~30秒）
python3 test_coreml_embed.py         # 測試 ANE 效能
```

## 預期結果（M4 Mac Mini）

| 項目 | 預期 |
|------|------|
| 轉換時間 | ~30秒 |
| 模型大小 | ~670MB |
| ANE 延遲 | ~10-15ms（M4 晶片 ANE 略慢於 M5 Max） |
| Ollama 延遲 | ~15-25ms |
| 中文支援 | ✅ |
