preprint · arXiv (2026) · Apr 6, 2026
Designing Digital Humans with Ambient Intelligence
Mengyu Chen, Pranav Deshpande, Runqing Yang, Elvir Azanli, Joseph Ligman, Shaohan Hu, Richard Chen
In plain English
Most deployed digital humans know only what is said to them in the conversation and nothing about the room, the device or the customer's history. This paper sets out a design framework for wiring sensors, IoT data and context models into a digital human so it can notice a situation and act before being asked, and it maps the trade-off between how much context the system holds and how much initiative it takes. Useful mainly as a structured vocabulary for scoping proactive avatar deployments and for arguing the privacy questions early; the retail-banking case study is a prototype, not an evaluated deployment.
Abstract
Digital humans are lifelike virtual agents capable of natural conversation and are increasingly deployed in domains like retail and finance. However, most current digital humans operate in isolation from their surroundings and lack contextual awareness beyond the dialogue itself. We address this limitation by integrating ambient intelligence (AmI) - i.e., environmental sensors, IoT data, and contextual modeling - with digital human systems. This integration enables situational awareness of the user's environment, anticipatory and proactive assistance, seamless cross-device interactions, and personalized long-term user support. We present a conceptual framework defining key roles that AmI can play in shaping digital human behavior, a design space highlighting dimensions such as proactivity levels and privacy strategies, and application-driven patterns with case studies in financial and retail services. We also discuss an architecture for ambient-enabled digital humans and provide guidelines for responsible design regarding privacy and data governance.