SOURCE-LINKED INTELLIGENCE

Capability-Stratified Degradation in Ternary Language Models

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Extreme low-bit inference offers a route toward smaller models and constrained deployment. Ternary language models restrict weights to $\{-1,0,+1\}$, approaching the limit of $\log_2 3 \approx 1.585$ bits/weight. The practical question for a pretrained model is not simply whether weights can be quantised but which capabilities survive and whether it remains useful for adaptation. We explore this by converting Qwen3.5-0.8B (752M parameters) to ternary weights using 72.4M tokens of quantisation-aware training (QAT). The resulting model, Cloe, is evaluated across 29 benchmarks, representation dia

Read original source ↗ Open in workspace

recordType

paper

region

Global