preprint · arXiv (2026) · May 28, 2026
VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents
Amrita Mazumdar, Seonwook Park, Rajarshi Roy, Nikhil Srihari, Shengze Wang, Yuhao Zhou, Julia Wang, Koki Nagano, Shalini De Mello
In plain English
The first benchmark for agents that must listen, speak and read nonverbal cues in video at the same time, built from 237 real video-call clips. Tested agents largely failed to use the visual stream during conversation, and speech-then-avatar pipelines could not produce simultaneous nonverbal responses.
benchmark
embodied conversational agents
evaluation
conversational ai