SOURCE-LINKED INTELLIGENCE

ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual Regularization

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

Knowledge distillation effectively transfers reasoning capabilities from large language models to lightweight student models. To enable knowledge transfer across disparate model families, researchers increasingly explore cross-tokenizer distillation. However, cross-tokenizer distillation remains challenging due to vocabulary and sequence misalignment, while approximate vocabulary alignment can introduce additional noise into distillation. To address these challenges, we propose Anchor-Based Cross-Tokenizer Distillation with Residual Regularization (ACTD). ACTD bridges structural heterogeneity

Read original source ↗ Open in workspace

recordType

paper

region

Global