User interaction in mixed reality interactive storytelling

Lecture Notes in Computer Science, 2003

Entertainment systems promise to be a significant application for Mixed Reality. Recently, a growing number of Mixed Reality applications have included interaction with synthetic characters and storytelling. However, AIbased Interactive Storytelling techniques have not yet been explored in the context of Mixed Reality. In this paper, we describe a first experiment in the adaptation of an Interactive Storytelling technique to a Mixed Reality system. After a description of the real time image processing techniques that support the creation of a hybrid environment, we introduce the storytelling technique and the specificities of user interaction in the Mixed Reality context. We illustrate these experiments by discussing examples obtained from the system.

2004

Abstract Entertainment systems promise to be a significant application for Mixed Reality. Recently, a growing number of Mixed Reality applications have included interaction with virtual actors and storytelling. However, AI-based Interactive Storytelling techniques have not yet been explored in the context of Mixed Reality. In this paper, we describe a fully-integrated first prototype based upon the adaptation of an Interactive Storytelling technique to a Mixed Reality system.

Multimedia, …, 2004

2015

DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User Agreement:

2003

User interaction with virtual agents generally takes place in virtual environments in which there is clear separation between the virtual actors and the user, due to the fact that in most cases, the user is in some way external to the virtual world. In Mixed-Reality Interactive Storytelling, the user's video image is captured in real time and inserted into a virtual world populated by autonomous synthetic actors with which the user interacts.

2005

Mixed Reality (MR) offers a unique challenge in integrating interacting agents, show-control devices, graphics and audio presentation, and human interaction into a single consolidated system. While each component may be addressed individually, combining their various functionalities via a dynamic script that delivers an interactive, non-linear story (scenario, world) requires a robust process and system. In this paper, we present a

Interactive storytelling in a mixed reality environment merges digital and physical information and features. It usually uses an augmentation of the real-world and physically-based interaction to create an immersive experience that corresponds to the dramatic storyline of the interactive narrative influenced by the actions of the user. Immersiveness is a crucial aspect of such an installation, and can be influenced by multiple factors such as video, sounds, interaction and, finally, the density of all combined stimuli. We used one of the stages from our interactive ALICE installation to investigate immersiveness and its contributing factors in a between-group design with a special focus on the effects of interactivity, and the feedback and feedforward stimuli of the environment on the users' experiences. The study was carried out with 41 participants and the results showed that immersiveness not necessarily depends on the modality of stimuli, but instead on their time-density.

Handbook of Research on Applied Data Science and Artificial Intelligence in Business and Industry, 2021

This chapter is focused on describing the history and the current relevance of user experience (UX) techniques that combine data science and AI in the research field of interactive and immersive storytelling, including virtual and augmented realities. It initially presents a brief history of interactive storytelling, video games, VR and AR, AI and data science, and the user experience (UX) techniques used in those areas. Later, the chapter describes the UX techniques in depth, using AI and data science that work best and are more useful for testing interactive media products, describing examples of its applications briefly. Finally, the chapter presents conclusions in relationship with utopias and dystopias regarding the future use of UX, AI, and data science in several areas such as edutainment, social media, media arts, and business, among others.

2005

The Institute for Creative Technologies was created at the University of Southern California with the goal of bringing together researchers in simulation technology to collaborate with people from the entertainment industry. The idea was that much more compelling simulations could be developed if researchers who understood state-of-the-art simulation technology worked together with writers and directors who knew how to create compelling stories and characters. This paper presents our first major effort to realize that vision, the Mission Rehearsal Exercise Project, which confronts a soldier trainee with the kinds of dilemmas he might reasonably encounter in a peacekeeping operation. The trainee is immersed in a synthetic world and interacts with virtual humans: artificially intelligent and graphically embodied conversational agents that understand and generate natural language, reason about world events and respond appropriately to the trainee's actions or commands. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have also joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. In this paper we describe the Mission Rehearsal Exercise system and the insights gained through this large-scale integration.

This chapter is focused on describing the history and the current relevance of user experience (UX) techniques that combine data science and AI in the research field of interactive and immersive storytelling, including virtual and augmented realities. It initially presents a brief history of interactive storytelling, video games, VR and AR, AI and data science, and the user experience (UX) techniques used in those areas. Later, the chapter describes the UX techniques in depth, using AI and data science that work best and are more useful for testing interactive media products, describing examples of its applications briefly. Finally, the chapter presents conclusions in relationship with utopias and dystopias regarding the future use of UX, AI, and data science in several areas such as edutainment, social media, media arts, and business, among others.