preprint · arXiv · Sep 19, 2026
AVTR-1: Open Stack for Real-Time Interactive Avatars
Artem Kravtsov, Dmitrii Ziganshin, Vsevolod Poletaev, Gleb Balitskiy, Anastasia Tikhonova, Egor Burkov, Vadim Lebedev
In plain English
The authors argue that fast frame generation is no longer the bottleneck for interactive avatars — the hard part is synchronising the video with an external voice agent, scheduling playback and surviving interruptions. Their released stack pairs a compact 153M-parameter motion model with that plumbing, runs in real time on consumer as well as data-center GPUs, and ships with weights, renderer and serving backend. They also propose a metric (R-DGG) that tests whether a listening avatar's motion is genuinely driven by the other person's speech, and find that talking-head generators without paired audio fail it — a practical way to check whether an "attentive listener" avatar is actually listening.
Abstract
Talking-head and dyadic models now achieve real-time inference, yet fast motion generation alone does not produce an interactive conversation. A live system must synchronize the model's output with speech from an external voice agent, schedule video frames for playback, and handle interruptions. We introduce AVTR-1, an open stack for real-time interactive avatar conversations, built around a compact 153M-parameter autoregressive flow-matching motion generator conditioned on both participants' audio. We adapt its audio encoder for streaming through self-distillation. The stack turns the model's chunk-based generation into a continuous, synchronized audio-video stream driven by an external voice agent, and we analytically derive its contribution to the user-facing latencies and validate the resulting bounds with two commercial voice agents. Further experiments demonstrate that AVTR-1 leads the compared dyadic systems on all reported visual-quality metrics and most conventional listening-motion metrics while remaining competitive in lip synchronization. Its inference runtime operates in real time on data-center and consumer GPUs. However, conventional listening metrics do not establish whether the paired speaker's speech contributes to generated motion. We therefore introduce the Reference-Based Directed Granger Gain (R-DGG), which measures the additional predictive information carried by speaker speech after accounting for listener history and speaker motion. R-DGG finds statistically supported predictive dependence for recorded listeners and all evaluated dyadic systems, but not for talking-head generators without paired audio or mismatched speaker-listener pairs. We release the model weights, renderer, and serving backend under component-specific licenses.