Virtual Human

As part of the Institute for Creative Technologies and the School of Cinematic Arts at the University of Southern California, the Mixed Reality lab develops technologies and techniques for presenting realistic immersive training experiences. Such experiences typically place users within a complex ecology of social actors, physical objects, and collections of intents, motivations, relationships, and other psychological constructs. Currently, it remains infeasible to completely synthesize the interactivity and sensory signatures of such ecologies. For this reason, the lab advocates mixed reality methods for training and conducts experiments exploring such methods. Currently, the lab focuses on understanding and exploiting the elasticity of human perception with respect to representational differences between real and virtual environments. This paper presents an overview of three projects: techniques for redirected walking, displays for the representation of virtual humans, and audio processing to increase stress.

Material culture is the conventional name for the tangible yield of human conduct. We have things to study, and we must record them dutifully and examine them lovingly if the abstraction called culture is to be compassed."… "The design is a record of the process of design and … of the designer's mind and provides a resource for the historian and scholar created in time and shaped to cultural pattern." [1] Historic Costume Collections offer an opportunity for sociologists, anthropologists, historians and designers to study fashion design from concept to realization; the social milieu in which the garments were produced and worn; and the evolution of body type and posture. This paper discusses a project to enhance these investigations through advanced technology in virtual humans and a multicultural, multilingual development team.

Molecular medicine is undergoing a revolution, creating a data fog that may obscure understanding. The functioning human is analogous to a biological factory controlled by an incredibly complex Information and Communication (IC) network. It is proposed that 7 billion computational replicas be made of those 7 billion human IC networks to enable interrogation and manipulation, for understanding and personalized healthcare. This requires a revolutionary ICT that follows the organization of the biological information and communication flows, with implications for hardware, software and connectivity.

The audio of the spoken dialogue between a human and a virtual human (VH) is analyzed to explore the impact of H-VH interaction. The goal is to determine if conversing with a VH can elicit detectable and systematic vocal changes. To study this topic, we examined the H-VH scenario of pharmacy students speaking with immersive VHs playing the role of patients. The audio analysis focused on the students' reaction to scripted empathetic challenges designed to generate an unrehearsed affective response from the human. The responses were analyzed with software developed to analyze vocal patterns during H-VH conversations. The analysis discovered vocal changes that were consistent across participants groups and correlated with known H-H conversation patterns.

We describe the design, implementation and evaluation of a novel multi-agent scenario generation framework for interactive virtual reality simulations towards preventive medicine education. Our scenario generation framework is based on recordings of human movements from a distributed sensor networks deployed in a realworld physical setting. The components of our framework include the generation of unique virtual agent behaviors from the sensor data, and algorithms for the generation of low level or gross movement behaviors such as path determination, directional traffic flows, collision avoidance and overtaking. The framework also includes the generation of high level fine actions for multiagents such as techniques for interactive activities in pedagogical scenarios based on environment and temporal triggers. We applied our multi-agent scenario generation framework in an interactive simulation for hand hygiene education, and conduct an initial usability study to assess the educational benefits of the simulation to nursing students and evaluated the performance characteristics of our framework. Results of our quantitative and qualitative evaluations suggest that our framework was robust in creating engaging, compelling, and realistic interactive training scenarios with multiple virtual agents in simulated hospital situations.

Backchannel feedback is an important kind of nonverbal feedback within face-to-face interaction that signals a person's interest, attention and willingness to keep listening. Learning to predict when to give such feedback is one of the keys to creating natural and realistic virtual humans. Prediction models are traditionally learned from large corpora of annotated face-to-face interactions, but this approach has several limitations. Previously, we proposed a novel data collection method, Parasocial Consensus Sampling, which addresses these limitations. In this paper, we show that data collected in this manner can produce effective learned models. A subjective evaluation shows that the virtual human driven by the resulting probabilistic model significantly outperforms a previously published rule-based agent in terms of rapport, perceived accuracy and naturalness, and it is even better than the virtual human driven by real listeners' behavior in some cases.

Virtual humans are embodied software agents that should not only be realistic looking but also have natural and realistic behaviors. Traditional virtual human systems learn these interaction beha- viors by observing how individuals respond in face-to-face situa- tions (i.e., direct interaction). In contrast, this paper introduces a novel methodological approach called parasocial consensus sam- pling (PCS) which allows multiple individuals

This paper proposes a scripting approach to facial animations for embodied agents, in particular for X3D/VRML-based web agents. The use of a scripting technology to control facial expressions has the advantage that it results in highly flexible components; not only can scripts be re-used to define a wide range of facial animation scenarios, but they can also be applied with appropriate parameterization for different application contexts and different agents, despite the fact that agents may have different 3D facial geometrical data. The proposed approach has been implemented as an extention of STEP, a scripting language for embodied agents, by using the distributed logic programming language DLP. In the DLP language framework, a facial model as well as the animation engine are webbased and animation results can be viewed locally in a browser. In addition, facial animations in STEP can be generated on-the-fly, which is a requirement for web-based embodied agents that need to interact with users and other agents in real time.

STEP is a scripting language for embodied agents. STEP has been implemented in DLP, an object-oriented parallel logic programming language. This paper discusses how the scripting approach can be extended with facial expressions for embodied agents in STEP. The scripting language STEP can serve both as a high-level specification language for knowledge representation and reasoning, and as a lowerlevel programming language for animation and interaction of embodied agents. In addition, the STEP scripting framework provides a comprehensive set of sensors and effectors by means of which agents can perceive their world or can take appropriate actions, respectively.

The pace in computing, graphics and networking technologies together with the demand from real-life applications made it a requirement to develop more realistic virtual environments (VEs). Realism not only includes believable appearance and simulation of the virtual world, but also implies the natural representation of participants. This representation fulfills several functions: • The visual embodiment of the user, • The means of interaction with the world, • The means of feeling various attributes of the world using the senses.

Since a few years, the terms of virtual, synthetic or digital actors have been extensively used in order to describe very different kinds of entities. The term “actor” is well-known in the world of cinema and everybody knows it. A Hollywood-like actor is first a human personality. In the context of software, an actor as introduced by Hewitt [1] in 1971 is essentially an object that can send or receive messages. According to Greif and Hewitt [2], actors are defined by their behavior, whatever their shape: a cube that is able to jump by itself and avoid obstacles without intervention of the animator may be also defined as a virtual actor, because it is virtual and because it is able to behave itself.

We outline the main features of our synthetic virtual human sign language system, JASigning. We describe how we have extended its input notation, SiGML, to allow explicit control of performance time, and we describe our initial steps on the path to integrating virtual human sign language performance into annotation tools, where it may be compared with video depicting the corresponding real human performance.

Virtual human, with realistic behaviors and social skills, evoke in users a range of social behaviors normally only seen in human face-to-face interactions. One of the key challenges in creating such virtual humans is to give them human-like conversational skills, such as turn-taking skill. In this paper, we propose a multimodal end-of-turn prediction model. Instead of recording face-to-face conversation data, we collect the turn-taking data using Parasocial Consensus Sampling (PCS) framework. Then we analyze the relationship between verbal and nonverbal features and turn-taking behaviors based on the consensus data and show how these features influence the time people use to take turns. Finally, we present a probabilistic multimodal end-ofturn prediction model, which enables virtual humans to make real-time turn-taking predictions. The result shows that our model achieves a higher accuracy than previous methods did.

Abstract. In a computer supported distant collaboration, communication between users can be enhanced with a visual channel. Plain videos of individual users unfortunately fail to render their joint actions on the objects they share, which limits their mutual perception. Remote interaction can be enhanced by immersing user representations (avatars) and the shared objects in a networked 3D virtual environment, so user actions are rendered by avatar mimicry. Communication gesture (not actions) are captured by real-time computer ...

In a remote computer-mediated collaboration, co-workers have a limited perception of each other. Virtual humans can be used to increase interaction by immersion in 3D collaborative virtual environment. We discuss solutions to immerse shared 2D applications in a 3D world and to represent user actions on the application as avatar actions in the 3D virtual immersive inhabited world.

To my family for their love and support my daddy, my mommy, my brother, and my husband This doctoral thesis was realized during my employment in the Artificial Intelligence Group headed by Prof. Ipke Wachsmuth at the Faculty of Technology in Bielefeld University. The research done in this thesis was kindly supported by the Deutsche Forschungsgemeinschaft (DFG) in the Collaborative Research Center 673 (CRC 673), Alignment in Communication. First of all, I thank my professor Ipke Wachsmuth for giving me this opportunity and for believing in my qualification to succeed in this hard work. His wise advice and very helpful assistance were key contributors to the realization of the present thesis. I also thank Prof. Ana Maria Paiva and Prof. Dirk Reichardt for agreeing to review my thesis. I further thank Prof. Karl Grammer, Andrea Hofstätter, Prof. Pia Knoeferle, and Maria Nella Carminati for their collaboration and support. I thank the former and current members of the Artificial Intelligence Group for the very nice and supportive working atmosphere in our 'M4 Flur'. Especially, I thank Christian Becker-Asano, Stefan Kopp, and Thies Pfeiffer for the fruitful discussions during crucial moments of this thesis. Special thanks are directed to my friends and working colleagues, Nhung (my binom), Alexa, Maha, and Lars for their moral support and for keeping together. I also thank my Tunisian and German friends for allowing me to refrain from intense work and to enjoy other moments in life. Last but not least, I owe my dear family, Abderrahman, Dorra, Heithem, and Tarak, deepest thanks for everything, especially my daddy and mommy who were fully present despite the geographical distance. i Brooks claims that intelligent behavior emerges from an agent's interaction with its environment. Therefore, in order for an agent to exploit the actual situation and to interact with its environment, it needs a physical body composed of sensors and actuators that couple the agent with the environment, and thus influences the agent's internal processing. This is referred to as embodiment . Accordingly, an artificial agent's embodiment is crucial to perceiving and understanding an interaction partner's emotional states, and to expressing and communicating empathy through, for example, facial expressions and verbal utterances. Research on artificial agents exhibiting empathic behavior such as virtual humans and robots, provides valuable results for the integration of empathy in human-machine interaction, and substantiates the role of empathy in enhancing artificial agents' social behavior. Virtual humans are 3D animated characters with human like appearance. Prendinger & Ishizuka [95] found that a virtual human that provides empathic feedback through textual expressions can reduce the stress levels of candidates during job interview tasks. Brave et al. [19] found that in a game scenario of casino-style blackjack, an artificial agent that empathizes with the player's game situation is perceived as more likable, trustworthy, and caring. Leite et al. [69] found that a robot's empathic behavior in a chess game scenario is perceived by children as more engaging and helpful. Further, not only does an artificial agents' ability to empathize have a positive impact on humanmachine interaction, but also their ability to evoke empathy in humans. In this regard, Paiva et al. show that empathic virtual humans can evoke empathy in children and thus can teach them to deal with bullying situations. A virtual human's empathic behavior also contributes to its ability to build and sustain long-term socio-emotional relationships with human partners as demonstrated by Bickmore & Picard [9]. However, in the context of a competitive card game scenario, Becker et al. [7] found that a virtual human's positive empathic emotions are significantly arousing and stress inducing and thus inadequate. Therefore, in line with humans' ability to empathize with each other to different degrees, we believe that a modulation of a virtual human's empathic behavior through factors such as its mood, personality, and relationship to its interaction partner, will allow for a more adequate empathic behavior in the agent across different interaction scenarios. Although much effort has gone to providing artificial agents with features such as mood, personality and social relationships, little attention has been devoted to the role of such features in modulating their empathic behavior. Altogether, endowing artificial agents with appropriate empathic behavior

Allowing virtual humans to align to others' perceived emotions is believed to enhance their cooperative and communicative social skills. In our work, emotional alignment is realized by endowing a virtual human with the ability to empathize. Recent research shows that humans empathize with each other to different degrees depending on several factors including, among others, their mood, their personality, and their social relationships. Although providing virtual humans with features like affect, personality, and the ability to build social relationships, little attention has been devoted to the role of such features as factors modulating their empathic behavior. Supported by psychological models of empathy, we propose an approach to model empathy for the virtual human EMMA-an Empathic MultiModal Agent-consisting of three processing steps: First, the Empathy Mechanism by which an empathic emotion is produced. Second, the Empathy Modulation by which the empathic emotion is modulated. Third, the Expression of Empathy by which EMMA's multiple modalities are triggered through the modulated empathic emotion. The proposed model of empathy is illustrated in a conversational agent scenario involving the virtual humans MAX and EMMA.

Alignment can be defined as the ability to adjust to the behavior of others. This is relevant in research on virtual humans because it would allow them to behave more believably in human-computer interaction and would facilitate their communication with the human interactant. One aspect that can be considered important is emotional alignment, which can be understood as sharing the same feelings with the others. This relates to the definition of empathy as an affective reaction to the other's emotion . A fundamental mechanism of empathy is mimicry, the process of simulating another's facial expressions, voice and posture, that triggers an afferent feedback eliciting the same feelings in oneself as that of the others . Beside our virtual human Max [6] we are developing the virtual human Emma, whose face replicates 25 actions units -the core elements of the Facial Coding System [2]-enabling about 3000 facial nuances related to pleasurearousal-dominance values . Based on the study by Grammer et al. [3] and inspired by the concepts used to implement the imitation algorithms as in [1] and [5], we will describe how our virtual human Emma constitutes a testbed for realising emotional alignment through mimicry.

Allowing virtual humans to align to others' perceived emotions is believed to enhance their cooperative and communicative social skills. In our work, emotional alignment is realized by endowing a virtual human with the ability to empathize. Recent research shows that humans empathize with each other to different degrees depending on several factors including, among others, their mood, their personality, and their social relationships. Although providing virtual humans with features like affect, personality, and the ability to build social relationships, little attention has been devoted to the role of such features as factors modulating their empathic behavior. Supported by psychological models of empathy, we propose an approach to model empathy for the virtual human EMMA-an Empathic MultiModal Agent-consisting of three processing steps: First, the Empathy Mechanism by which an empathic emotion is produced. Second, the Empathy Modulation by which the empathic emotion is modulated. Third, the Expression of Empathy by which EMMA's multiple modalities are triggered through the modulated empathic emotion. The proposed model of empathy is illustrated in a conversational agent scenario involving the virtual humans MAX and EMMA.

Expressing and recognizing affective states with respect to facial expressions is an important aspect in perceiving virtual humans as more natural and believable. Based on the results of an empirical study a system for simulating emotional facial expressions for a virtual human has been evolved. This system consists of two parts: (1) a control architecture for simulating emotional facial expressions with respect to Pleasure, Arousal, and Dominance (PAD) values, (2) an expressive output component for animating the virtual human's facial muscle actions called Action Units (AUs), modeled following the Facial Action Coding System (FACS). A large face repertoire of about 6000 faces arranged in PAD-space with respect to two dominance values (dominant vs. submissive) is obtained as a result of the empirical study. Using the face repertoire an approach towards realizing facial mimicry for a virtual human based on backward mapping AUs displaying an emotional facial expression on PAD-values is outlined. A preliminary evaluation of this first approach is realized with AUs corresponding to the basic emotions Happy and Angry.

Addressing user's emotions in human-computer interaction significantly enhances the believability and lifelikeness of virtual humans. Emotion recognition and interpretation is realized in our approach by integrating empathy as a designated process within the agent's cognitive architecture. In this paper we describe this empathy process which comprises of two interconnected components: a belief-desire-intention (BDI) based cognitive component and an affective component based on the emotion simulation system of the virtual human Max. The application and a preliminary evaluation of this empathy system are reported on in the context of a 3D competitive card game scenario.

A computational agent model for monitoring and control of a virtual human agent's resources and exhaustion is presented. It models a physically grounded intelligent decision making process within the agent model for physical effort to be spent. Simulation results are discussed, and a formal analysis is presented on conditions under which the agent model functions properly, for example, such that it can be used to avoid running out of resources. The model is related to a model for monitoring or simulating a person's heart rate. Finally some validation experiments are briefly discussed.

This study assesses the integration of Mixed Reality (MR) technologies in medical first responder (MFR) training, focusing on identifying key factors influencing behavioral intention to use MR systems and practical implications for technology acceptance and enhanced realism through haptic feedback. Methods: Through a User-Centered Design approach, involving co-creation workshops, iterative development, and evaluations in pilot and field trials across six countries, this study evaluated technology acceptance, presence, user experience, and workload among MFRs. Both quantitative measures and qualitative 1 feedback were collected to analyze the determinants of technology acceptance and user engagement. Results: The MED1stMR Training System, developed as a result, demonstrates that performance expectancy, effort expectancy, and social presence are significant predictors of behavioral intention to use MR training systems among MFRs. High technology acceptance and positive user experience were reported, with specific emphasis on the educational value of haptic feedback in skill training. Trainer feedback highlighted the importance of real-time performance metrics and openness to AI-driven training assistance for enhancing training outcomes. Conclusion: The study underscores the critical role of realistic patient interaction and the importance of aligning training challenges with users' skills to create engaging MR training environments for MFRs. Identifying factors influencing behavioral intention offers valuable insights for the development of MR training systems, suggesting a focus on social presence and interactive capabilities to improve realism and educational value. The findings advocate for the integration of adaptive training features and further exploration of AI support in scenario optimization and performance enhancement.

Rendering and animating in real-time a multitude of articulated characters presents a real challenge and few hardware systems are up to the task. Up to now little research has been conducted to tackle the issue of real-time rendering of numerous virtual humans. However, due to the growing interest in collaborative virtual environments the demand for numerous realistic avatars is becoming stronger. This paper presents a hardware-independent technique that improves the display rate of animated characters by acting on the sole geometric and rendering information. We first review the acceleration techniques traditionally in use in computer graphics and highlight their suitability to articulated characters. Then we show how impostors can be used to render virtual humans. Finally we introduce concrete case studies that demonstrate the effectiveness of our approach.

Integrating virtual human's animation in networked virtual environments (NVEs) has the potential to create severe network load issues. In NVEs, users are represented by their virtual embodiments. The articulated structure of these embodiments introduces a new complexity in the communication, representation, and streaming of animations. This might create a significant overhead, especially as the number of participants in the simulation increases. In this paper we present problems and solutions to efficiently transmit virtual human data over the network.

Rendering and animating in real-time a multitude of articulated characters presents a real challenge and few hardware systems are up to the task. Up to now little research has been conducted to tackle the issue of real-time rendering of numerous virtual humans. This paper presents a hardware-independent technique that improves the display rate of animated characters by acting on the sole geometric and rendering information. We first review the acceleration techniques traditionally in use in computer graphics and highlight their suitability to articulated characters. Then we show how impostors can be used to render virtual humans. We introduce concrete case studies that demonstrate the effectiveness of our approach. Finally, we tackle the visibility issue.

At present, very few systems possess the multiple functions required to build real-time deformable humans who look believable and recognizable. In this paper, we describe our interactive system for building a virtual human, fitting texture to the body and head, and controlling skeleton motion. Emphasis is placed on those aspects of deformations that increase the realism of the humans appearance. Specific attention is also paid to the hand as it contains half of the body skeletal bones. The system includes facial motion control as well. We first detail the complete animation framework, integrating all the virtual human modules. We then present the first of our two case studies: CyberTennis, where two virtual humans play realtime tennis within our Virtual Life Network Environment system. One real player is in Geneva and her opponent is in Lausanne. An autonomous virtual judge referees the game. The second application combines high-tech and artistic choreography in a CyberDance performance in Geneva exhibition hall, as part of Computer Animation 97. In this performance, the movements of the choreographer are captured and are paralleled, in real time, by his virtual robot counterpart. The show also features virtual humans choreographed for an aerobic dance session. Additional presentations of this performance are now scheduled, including one at the prestigious Creativity Institute in Zermatt in January 1998.

We present in this paper an image-based rendering technique to represent virtual humans in real-time. Though we deal only with virtual humans, the concept can be applied to any articulated character. Our technique is termed animated impostors because it extends the notion of dynamic impostors to take into account changes in the character's appearance. We also show how impostors can be embedded into large-scale simulations/applications.

In this paper we present a new workflow allowing the creation of 3D characters in an automated way that does not require the expertise of an animator. This workflow is based of the acquisition of real human data captured by 3D body scanners, which is them processed to generate firstly animatable body meshes, secondly skinned body meshes and finally textured 3D garments.

This paper presents our progress in developing a Virtual Human capable of being an attentive speaker. Such a Virtual Human should be able to attend to its interaction partner while it is speaking-and modify its communicative behavior on-the-fly based on what it observes in the behavior of its partner. We report new developments concerning a number of aspects, such as scheduling and interrupting multimodal behavior, automatic classification of listener responses, generation of response eliciting behavior, and strategies for generating appropriate reactions to listener This paper is base upon a project report of the eNTERFACE'10 Summer Workshop on Multimodal Interfaces [42].

Virtual humans are computer‐generated characters designed to look and behave like real people. Studies have shown that virtual humans can mimic many of the social effects that one finds in human‐human interactions such as creating rapport, and people respond to virtual humans in ways that are similar to how they respond to real people. We believe that virtual humans represent a new metaphor for interacting with computers, one in which working with a computer becomes much like interacting with a person and this can bring social elements to the interaction that are not easily supported with conventional interfaces. We present two systems that embody these ideas. The first, the twins are virtual docents in the Museum of Science, Boston, designed to engage visitors and raise their awareness and knowledge of science. The second, Sim‐Coach, uses an empathetic virtual human to provide veterans and their families with information about PTSD and depression.

We present an online system that provides a complete web-based sandbox for creating, testing and publishing embodied conversational agents. The tool, called Roundtable, empowers many different types of authors and varying team sizes to create flexible interactions by automating many editing workflows while limiting complexity and hiding architectural concerns. Finished characters can be published directly to web servers, enabling highly interactive applications.

Gaze plays a large number of cognitive, communicative and affective roles in face-to-face human interaction. To build a believable virtual human, it is imperative to construct a gaze model that generates realistic gaze behaviors. However, it is not enough to merely imitate a person's eye movements. The gaze behaviors should reflect the internal states of the virtual human and users should be able to derive them by observing the behaviors. In this paper, we present a gaze model driven by the cognitive operations; the model processes the virtual human's reasoning, dialog management, and goals to generate behaviors that reflect the agent's inner thoughts. It has been implemented in our virtual human system and operates in real-time. The gaze model introduced in this paper was originally designed and developed by Jeff Rickel but has since been extended by the authors.

This paper describes the development of a multi-modal corpus based on multi-party multi-task driven common goal oriented spoken language interaction. The data consists of approximately 10 hours of audio human simulation radio data and nearly 5 hours of video and audio face-to-face sessions between human trainees and virtual agents.

The Twins corpus is a collection of utterances spoken in interactions with two virtual characters who serve as guides at the Museum of Science in Boston. The corpus contains about 200,000 spoken utterances from museum visitors (primarily children) as well as from trained handlers who work at the museum. In addition to speech recordings, the corpus contains the outputs of speech recognition performed at the time of utterance as well as the system interpretation of the utterances. Parts of the corpus have been manually transcribed and annotated for question interpretation. The corpus has been used for improving performance of the museum characters and for a variety of research projects, such as phonetic-based Natural Language Understanding, creation of conversational characters from text resources, dialogue policy learning, and research on patterns of user interaction. It has the potential to be used for research on children's speech and on language used when talking to a virtual human.

In this paper, we present current research developments in the Virtual Life of autonomous synthetic actors. After a brief description of the perception action principles with a few simple examples, we emphasize the concept of virtual sensors for virtual humans. In particular, we describe in details our experiences in implementing virtual vision, tactile and audition. We then describe perception-based locomotion, a multisensor based method of automatic grasping and vision-based ball games.

We describe the HUMANOID environment dedicated to human modeling and animation for general multimedia, VR, and CAD applications integrating virtual humans. We present the design of the system and the integration of the various features: generic modeling of a large class of entities with the BODY data structure, realistic skin deformation for body and hands, facial animation, collision detection, integrated motion control and parallelization of computation intensive tasks.

Virtual Life is a new area dedicated to the simulation of life in virtual worlds, human-virtual interaction and immersion inside virtual worlds. Virtual Life cannot exist without the growing development of Computer Animation techniques and corresponds to the most advanced concepts and techniques of it. In this chapter, we present current research developments in the Virtual Life of autonomous synthetic actors. After a brief description of the perception action principles with a few simple examples, we emphasize the concept of virtual sensors for virtual humans. In particular, we describe in details our experiences in implementing virtual vision, tactile and audition. We then describe perception-based locomotion, a multisensor based method of automatic grasping and vision-based ball games. We also discuss problems of integrating autonomous humans into virtual environments.

Evaluating Personality Trait Attribution Based on Gestures by Virtual Agents Kris Liu University of California, Santa Cruz, Santa Cruz, CA, United States Jackson Tolins University of California, Santa Cruz, Santa Cruz, CA, United States Jean Fox Tree University of California, Santa Cruz, Santa Cruz, CA, United States Marilyn Walker University of California, Santa Cruz, Santa Cruz, CA, United States Michael Neff University of California, Davis Abstract: Can human-normed personality scales alone give us a full picture of the attribution of personality to virtual agents? In humans, personality traits are often immediately ascribed: first impressions, influenced by posture and gesture, can be strong, lasting and accurate. Is there a similar immediate attribution of personality for virtual agents that can be similarly influenced by posture and gesture? Our study uses an open-ended question alongside a traditional Big Five personality inventory to probe the perceived personality of agents...

Complex, real-world environments may not be fully modeled for an agent, especially if the agent has never operated in the environment before. The agent’s ability to effectively plan and act in the environment is influenced by its knowledge of when it can perform specific actions and the effects of those actions. We describe progress on an explainable exploratory planning agent that is capable of learning action preconditions using an exploratory planning process. The agent’s architecture allows it to perform both exploratory actions as well as goal-directed actions, which opens up important considerations for how exploratory planning and goal planning should be controlled, as well as how the agent’s behavior should be explained to any teammates it may have. We describe initial approaches for both exploratory planning and action model learning, and evaluate them in the video game Dungeon Crawl Stone Soup.

We describe an initial prototype of a holodeck-like environment that we have created for the Mission Rehearsal Exercise Project. The goal of the project is to create an experience learning system where the participants are immersed in an environment where they can encounter the sights, sounds, and circumstances of realworld scenarios. Virtual humans act as characters and coaches in an interactive story with pedagogical goals.

To foster a better relationship with virtual agents, agents need to be emotionally intelligent. This study discusses papers which talks on the need for emotional intelligence (EI) in virtual agents and the effects of psychological effects on such virtual agents. We also discuss the designing strategy of EI agents. And then we discuss applications of such agents and the effects of it on building a social relationship with their users. For building agents that can create a relationship with their user it is imperative for agents to be emotionally intelligent. Such agents outperform user counterparts in overcoming phenomenon like morning morality. With models for emotional intelligent machines in place designing such an agent can be easily done with integrating models and technologies like Virtual Reality. Applications of emotionally intelligent agents are vast in the areas of interactive systems like games and in systems like crowd simulation. Although, creating a virtual agent with emotional intelligence is the road ahead it is still in a nascent state. A lot of work is required to create an emotionally intelligent agent which can be comparable to a human counterpart.

For a growing number of applications it is important that the perceptual behavior of a virtual human be at once realistic and believable. To be realistic, a model needs to be psychologically plausible, and to be believable the behavior must look right. To this end, I describe a model of attention that was implemented in a virtual helicopter pilot using the Soar cognitive architecture. It is based on psychological principles and where the pilot performs a number of realistic tasks in a real-time, synthetic environment. We are now extending the model to be more believable by incorporating gaze behaviors where the virtual human has a head and eyes. This paper describes the basic model of attention and the first steps toward achieving more believable gaze behaviors.

This paper presents an account of a Cyberdance performance involving both real and virtual dancers. Preparing for this performance required the integration of a number of virtual human modules into a single system framework capable of handling all aspects of the show. We describe the modules that deal with creation, modeling and texturing the virtual humans, as well as those that manage the animation of bodies within real-time applications, and control several bodies simultaneously, as choreographed. Real-time hand deformation and facial animation modules increase the realism of the virtual dancers. Two techniques are described for improving the quality of the rendering in the virtual component of the scenario. We describe the VR devices for merging the interactive dancers into the virtual environment. In conclusion, we report on experiences during the actual presentation of the show and suggest topics for future research into improvements of various parts of our system.

In this paper we present the Virtual Park (or VPARK) system. This includes a Networked Virtual Environment (NVE) System, called W-VLNET and an Attraction Building System, able to create and modify attractions used in the NVES. Both systems have been developed in the Windows NT environment. The paper outlines the techniques for communication, scene management, facial and body animation, and general user interaction modules. The use of VRML97 and MPEG-4 SHNC is overviewed for the purpose of outlining the compatibility of the system with other similar Virtual Reality systems. The software provides realistic virtual actors as well as sets of high-level actions applicable to them in real-time. Related issues on obtaining actor models and animating them in real-time are presented. The creation process of an attraction incorporates assembling animation units through a timeline. Using this software, the users are able to bring their own scenario-based applications into a shared virtual environment.

In order to enhance the sense of presence within networked Virtual Environments it is important to increase the quality of physical or social interaction of participants with each other and with the environment. We beleive that this increase of quality can be achieved by realistic modeling and animation of the human body representing the participant, together with the natural behaviors of the objects in the environment. The representation of self as a realistic-looking virtual human body with natural movement helps participants not only to perceive each other and thus feel together, but also to interact with the environment in a straightforward and natural fashion. This property is further enhanced if the objects in the environment behave in a natural, expected way. In the Virtual Life Network (VLNET) system we use realistic human body representation together with a set of motor functions giving behaviors to actors and objects to reach the mentioned goals.

In this paper, we present our work on a level of detail(LoD) technique for human-like face models in virtual environments. Conventional LoD techniques have been adapted to allow facial animation on simplified geometric models. This includes optimization on both geometry, and animation parameters. Simplified models are generated in a region-based manner in consideration with the mobility of each region. The animation process is decomposed into two sub-processes and each step is optimized. In the MPA(Minimum Perceptible Action) level optimization, a hierarchical structure was devised for the multi-level animation model. The deformation level is simplified by reducing the number of control points. During runtime, the level of animation is selected in combination with the viewpoint information the level of geometry.