preprint · arXiv
InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars
Quanyue Song, Yishan He, Yanfei Zhang, Shihao Cheng, Zhixiang He, Zhizhi Guo, Chi Zhang, Xuelong Li, Caigui Jiang
In plain English
A streaming avatar video generator built on a distilled diffusion transformer that keeps a two-tier visual memory — a short FIFO window of recent frames plus a fixed-size buffer of semantically distinct key frames — so appearance does not drift over a long session, and routes user input through an LLM that emits a spoken reply plus a separate action prompt. It reports 26.68 FPS against 21.94 for the next-fastest baseline and the best scores on its own object and temporal consistency metrics. Practitioners should note what is missing: no end-to-end latency figure, no lead on image-quality metrics such as FID or FVD, and an evaluation designed by the authors.
Related research
- Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
- Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
- EmbodiedHead: Real-Time Listening and Speaking Avatar for Conversational Agents