SOURCE-LINKED INTELLIGENCE

dQwen3.5: Hybrid-Attention Diffusion Language Models

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

Adapting a pretrained autoregressive (AR) model is a cost-efficient route to a diffusion language model (DLM). While nearly all such adaptations start from a full-attention transformer, AR modeling has shifted toward hybrid architectures that interleave attention and RNN layers. This creates an obstacle for adaptation: unlike attention, RNNs are structurally causal and nontrivial to bidirectionalize. Despite this mismatch, we investigate whether such backbones can become effective DLMs by adapting Qwen3.5 at 0.8B, 2B, 4B, and 9B scales, yielding the dQwen3.5 family. We find that hybrid backbon

Read original source ↗ Open in workspace

recordType

paper

region

Global