Beyond Monologue: Interactive Talking-Listening Avatar Generation with Conversational Audio Context-Aware Kernels

In plain English

Real conversation is full-duplex, but most audio-driven avatar models are built for monologue and handle listening as an afterthought, bolting it onto frame-aligned talking-head machinery. The authors argue that talking and listening operate on different timescales and add a multi-head Gaussian kernel that builds that difference into the model, alongside a dual-stream agent that processes the speaker's and the listener's audio separately and a cleaned dataset, VoxHear, with decoupled speech and background tracks. For practitioners, this is the listening half of the duplex problem — the part that makes an avatar look present rather than frozen between turns. The abstract reports no benchmark numbers, so the claimed state of the art is unquantified at the abstract level.