%0 Journal Article
%T Tele-Immersive Interaction with Intelligent Virtual Agents Based on Real-Time 3D Modeling
%A Shujun Zhang
%A Wan Ching Ho
%J Journal of Multimedia
%D 2012
%I Academy Publisher
%R 10.4304/jmm.7.1.57-65
%X To enable intelligent agents interacting smoothly with human users, researchers have been deploying novel interaction modalities (e.g. non-verbal cue, vision and touch) in addition to agents¡¯ conversational skills. Models of multi-modality interaction can enhance agents¡¯ real-time perception, cognition and reaction towards the user. In this paper we report a novel tele-immersive interaction system developed using real-time 3D modelling techniques. In such system user¡¯s full body is reconstructed using multi-view cameras and CUDA based visual hull reconstruction algorithm. User¡¯s mesh model is then loaded into a virtual environment for interacting with an autonomous agent. Technical details and initial results of the system are illustrated in this paper. Following that a novel interaction scenario is proposed which links the virtual agent with a remote physical robot who takes the role of mediating interactions between two geographically separated users. Finally we discuss in depth the implications of such human-agent interaction and possible future improvements and directions.
%K intelligent virtual agent
%K tele-immersion
%K interaction
%K real-time modeling
%U http://ojs.academypublisher.com/index.php/jmm/article/view/6229