partial fix: Metal kernel bits=8 support - Embedding OK but Router/Expert still NaN
CI / build-and-test (push) Has been cancelled

修复成果:
 创建dequantize_row_8bit kernel
 Swift dequantizeRow函数检测bits并调用正确kernel
 loadExpertGroup groupSize计算修复
 Embedding测试:0 NaN/2816

待修复:
⚠️ Router/Expert forward pass仍有2 NaN
⚠️ Router matmul可能使用错误kernel
⚠️ Expert matmul可能使用错误kernel

测试对比:
修复前:Embedding 0 NaN, Forward 2 NaN
修复后:Embedding 0 NaN, Forward 2 NaN(未变)
结论:Embedding一直正常,NaN在Router/Expert/LM head

技术原理:
4-bit: packedIdx=g*(groupSize/8)+inG/8, shift=(inG%8)*4, &0xF
8-bit: packedIdx=g*(groupSize/4)+inG/4, shift=(inG%4)*8, &0xFF

下一步:
检查Router/Expert matmul是否使用8-bit kernels
或使用26B-Standard代替(完美0 NaN)

进度:60% 
推荐:使用26B-Standard 
This commit is contained in:
MarkBase Admin
2026-06-24 02:31:48 +08:00
parent d3379e23d5
commit 303fc748ac
2 changed files with 218 additions and 1 deletions
+3 -1
View File
@@ -1591,7 +1591,9 @@ readers = readersDict
func dequantizeRow(weight: QuantizedWeights, tokenId: Int,
output: MTLBuffer, nCols: Int? = nil) throws {
let pso = try engine.pipeline(named: "dequantize_row")
// Detect bits and use correct kernel (8-bit or 4-bit)
let kernelName = weight.bits == 8 ? "dequantize_row_8bit" : "dequantize_row"
let pso = try engine.pipeline(named: kernelName)
let cmdBuf = engine.commandQueue.makeCommandBuffer()!
let enc = cmdBuf.makeComputeCommandEncoder()!