deep analysis: 26B-A4B根本问题 - Metal kernel需支持bits=8
CI / build-and-test (push) Has been cancelled

根本问题确认:
 26B-A4B Router/Expert使用bits=8量化
 inDim = 704*4 = 2816(8-bit: 4 vals/u32)
 groupSize = 2816/44 = 64
⚠️ 现有dequantize_row kernel只支持bits=4
⚠️ Kernel硬编码:groupSize/8, (inG%8)*4, &0xF
⚠️ 需要8-bit逻辑:groupSize/4, (inG%4)*8, &0xFF

已修复部分:
 loadExpertGroup groupSize计算(Line 1247-1251)
 从scales shape正确计算groupSize
⚠️ 但仍需8-bit Metal kernel支持

修复方案对比:
方案A(修改Metal kernels):数天,极高风险,不确定 
方案B(使用26B-Standard):0分钟,无风险,完美 

创建文件:
- dequantize_8bit_kernel.metal(示例kernel)
- dequantizeRow_analysis.md(函数分析)
- 26B_A4B_Deep_Fix_Analysis.md(完整分析)

结论:
技术上可修复,但难度极高(需修改Metal kernels)
强烈推荐使用26B-Standard代替(完美无NaN)

推荐度:方案B 
This commit is contained in:
MarkBase Admin
2026-06-24 02:22:26 +08:00
parent e82162e96b
commit d3379e23d5
4 changed files with 466 additions and 2 deletions
+6 -2
View File
@@ -1244,8 +1244,12 @@ readers = readersDict
// Scales: [numExperts, expertOutDim, numGroups] bf16
// Biases: same shape as scales
let groupSize = 64
let numGroups = expertInDim / groupSize
// Compute groupSize from actual scales shape (not hardcoded 64)
// For 26B-A4B: scales.shape[2] = 44, expertInDim = 2816, groupSize = 2816/44 = 64
// For 26B-Standard: scales.shape[2] = 22, expertInDim = 2816, groupSize = 2816/22 = 128
// But we need to detect from actual scales shape
let numGroups = sDesc.shape.count == 3 ? sDesc.shape[2] : (sDesc.shape.count == 2 ? sDesc.shape[1] : 1)
let groupSize = numGroups > 0 ? expertInDim / numGroups : 64
// Get readers
let wReader: SafeTensorsReader