SOURCE-LINKED INTELLIGENCE

When Speech Meets Lips: Interpretable Audio-Visual Synchronization for L2 Pronunciation Assessment

arXiv · AI, language, vision and robotics · article · Sep 6, 2026 · UTC

Automatic Pronunciation Assessment (APA) systems have achieved strong performance with transformer-based models and self-supervised speech representations. However, most methods rely only on acoustic signals and overlook temporal synchronization between speech and articulatory movements, limiting diagnostic feedback on timing mismatches important for L2 pronunciation training. We propose an interpretable audio-visual synchronization framework that explicitly models speech-lip temporal alignment through feature encoding, cross-attention fusion, lag estimation, stability quantification, and visu

Read original source ↗ Open in workspace

recordType

paper

region

Global