SOURCE-LINKED INTELLIGENCE

A frontend-backend architecture for tool calls in full-duplex speech models

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Full-duplex speech-to-speech (S2S) models provide natural, low-latency conversational interaction and would benefit from the ability to use external tools and complete voice-agent tasks. We propose a frontend-backend architecture where a duplex speech-to-text frontend learns to emit a delegation token and forwards streaming ASR transcripts to a text-based backend LLM for tool calls. Tool-call results from the backend are injected back into the frontend through a lightweight prefill-and-repeat mechanism and then synthesized using streaming TTS to the user. Our approach largely preserves regular

Read original source ↗ Open in workspace

recordType

paper

region

Global