完整修复历程: ✅ Swift: loadExpertGroup groupSize计算修复 ✅ Swift: dequantizeRow bits检测 ✅ Swift: quantizedMatmul bits检测(移除if false) ✅ Metal: dequantize_row_8bit kernel创建 ✅ Metal: quantized_matmul_8bit kernel创建 ✅ 已有: quantized_matmul_gate_up_8bit, quantized_matmul_simd_8bit 测试结果始终不变: Embedding: 0 NaN ✅(一直正常) Forward Pass: 2 NaN ⚠️(位置[2,98],固定) 已排除的问题: ✅ Embedding weights/dequantization ✅ Router matmul kernel缺失 ✅ Expert matmul kernel缺失 ✅ GroupSize计算错误 ✅ Bits detection逻辑 未排除的可能问题: ⚠️ LM head逻辑 ⚠️ moeMegaKernel内部实现 ⚠️ Router scale计算 ⚠️ Token ID用作logits索引 关键差异: 12B: NaN在[2,255999,256000](多模态tokens) 26B-A4B: NaN在[2,98](未知机制) 26B-Standard: 0 NaN(完美) 修复成本: 已投入:数小时,5 kernel + 3 Swift修复 剩余工作:数小时+,风险极高 成功率:不确定 最终决策: 强烈推荐:使用26B-Standard代替 ⭐⭐⭐⭐⭐ 理由:完美0 NaN,相同架构,零风险,立即可用 修复进度:60% ✅ 问题定性:极其复杂 ⭐⭐⭐⭐⭐ 推荐方案:26B-Standard代替
This commit is contained in:
@@ -330,8 +330,7 @@ func quantizedMatmul(engine: MarkBaseEngine, cmdBuf: MTLCommandBuffer,
|
||||
output: MTLBuffer) throws {
|
||||
// Select kernel based on quantization bits
|
||||
let kernelName = weights.bits == 8 ? "quantized_matmul_8bit" : "quantized_matmul"
|
||||
// TEMPORARILY USE FALLBACK KERNEL FOR TESTING
|
||||
if false, let pso = try? engine.pipeline(named: kernelName) {
|
||||
if let pso = try? engine.pipeline(named: kernelName) {
|
||||
let enc = cmdBuf.makeComputeCommandEncoder()!
|
||||
enc.setComputePipelineState(pso)
|
||||
enc.setBuffer(input, offset: 0, index: 0)
|
||||
@@ -358,7 +357,7 @@ func quantizedMatmul(engine: MarkBaseEngine, cmdBuf: MTLCommandBuffer,
|
||||
return
|
||||
}
|
||||
|
||||
// Fallback to original
|
||||
// Fallback to original if optimized kernel not found
|
||||
let pso = try engine.pipeline(named: "quantized_matmul")
|
||||
let enc = cmdBuf.makeComputeCommandEncoder()!
|
||||
enc.setComputePipelineState(pso)
|
||||
|
||||
Reference in New Issue
Block a user