SOURCE-LINKED INTELLIGENCE

Quantifying the Generation Modality Gap in Speech-Text Language Models

arXiv · AI, language, vision and robotics · article · Sep 13, 2026 · UTC

Pure speech language models often lag behind text and speech-text language models in generating coherent content, but this gap is difficult to quantify because speech and text systems are typically evaluated with different metrics and trained on different data. We study the speech-text modality gap in a family of spoken language models, based on flow matching for continuous acoustic feature generation. We construct a unified generation-based evaluation suite that compares speech-only, text-only, and speech-text language models trained on matched data distributions and evaluated in matched gene

Read original source ↗ Open in workspace

recordType

paper

region

Global