SOURCE-LINKED INTELLIGENCE

Component-Aware Differential Privacy for Federated Multilingual Speech-LLMs

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

Per-layer differential privacy (DP) clipping improves gradient fidelity in federated learning by allocating per-matrix clipping budgets proportional to parameter count. We show that this recipe breaks for speech large language models (speech-LLMs), when the acoustic encoder and the language decoder differ by an order of magnitude in update norm. Single-pool per-layer methods suffer \emph{cross-component budget collapse}, dragging word error rate (WER) far from flat global clipping or collapsing training entirely. When the norm imbalance is milder, adaptive single-pool methods partially recover

Read original source ↗ Open in workspace

recordType

paper

region

Global