warren/markbaseengine
1
0
Fork 0
Code Issues Pull Requests Actions 18 Packages Projects Releases Wiki Activity

v2: Apply tokenizer UTF-8 fix + Engine writeFloats helper
CI / build (push) Waiting to run
Details
CI / unit-tests (push) Blocked by required conditions
Details
CI / lint (push) Blocked by required conditions
Details

Browse Source 
- Tokenizer fix: collect <0xXX> bytes and decode as UTF-8
  (fixes Chinese/non-ASCII character decoding)
- BPETokenizer + HuggingFaceTokenizer: both updated
- Engine.swift: added writeFloats() utility method
- FloatWeights struct added to Layer.swift (bf16 support)
- attnQBits/KBits/VBits/OBits detection added to Model.swift
- bf16 layer weight support from commit 48c0347 cherry-picked
This commit is contained in:
MarkBase Admin
2026-07-05 13:41:48 +08:00
parent 5a94501f95
commit 31427770b1
5 changed files with 40 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Sources/MarkBase/Model.swift
+5 -1
View File
@@ -716,9 +716,13 @@ readers = readersDict
print(" layer_scalar: NOT FOUND (using 1.0)")
}
// Detect quantization bits from weight shape (supports both uniform 4-bit and 8-bit MLP/router)
// Detect quantization bits from weight shape (supports both uniform 4-bit and 8-bit MLP/router)
let mlpGateBits = detectBits(for: "mlp.gate_proj", expectedInDim: hiddenSize, defaultBits: 4)
let mlpDownBits = detectBits(for: "mlp.down_proj", expectedInDim: intermediate, defaultBits: 4)
let attnQBits = detectBits(for: "self_attn.q_proj", expectedInDim: hiddenSize, defaultBits: 4)
let attnKBits = detectBits(for: "self_attn.k_proj", expectedInDim: hiddenSize, defaultBits: 4)
let attnVBits = detectBits(for: "self_attn.v_proj", expectedInDim: hiddenSize, defaultBits: 4)
let attnOBits = detectBits(for: "self_attn.o_proj", expectedInDim: hiddenSize, defaultBits: 4)
// Try bf16 weights first (for bf16 models)
let qpFloat = try fw("self_attn.q_proj")