 MarkBase Admin
|
303fc748ac
|
partial fix: Metal kernel bits=8 support - Embedding OK but Router/Expert still NaN
CI / build-and-test (push) Has been cancelled
修复成果：
✅ 创建dequantize_row_8bit kernel
✅ Swift dequantizeRow函数检测bits并调用正确kernel
✅ loadExpertGroup groupSize计算修复
✅ Embedding测试：0 NaN/2816
待修复：
⚠️ Router/Expert forward pass仍有2 NaN
⚠️ Router matmul可能使用错误kernel
⚠️ Expert matmul可能使用错误kernel
测试对比：
修复前：Embedding 0 NaN, Forward 2 NaN
修复后：Embedding 0 NaN, Forward 2 NaN（未变）
结论：Embedding一直正常，NaN在Router/Expert/LM head
技术原理：
4-bit: packedIdx=g*(groupSize/8)+inG/8, shift=(inG%8)*4, &0xF
8-bit: packedIdx=g*(groupSize/4)+inG/4, shift=(inG%4)*8, &0xFF
下一步：
检查Router/Expert matmul是否使用8-bit kernels
或使用26B-Standard代替（完美0 NaN）
进度：60% ✅
推荐：使用26B-Standard ⭐⭐⭐⭐⭐
|
2026-06-24 02:31:48 +08:00 |
|