SOURCE-LINKED INTELLIGENCE

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

An avatar that holds a conversation should decide what to say and to move while saying it, yet these abilities live in separate model families: spoken dialogue models produce speech without motion, and co-speech motion models produce motion only from audio handed to them. The standard remedy is a cascade that first generates the spoken response and then runs a motion model over the finished audio, which requires a second full inference pass and precludes any joint optimisation between the two. We present Motion-Omni, an end-to-end framework in which a spoken dialogue model natively outputs exp

Read original source ↗ Open in workspace

recordType

paper

region

Global