Authoring scenes for adaptive, interactive performances

Virtual Reality, 2003

CrossTalk is a self-explaining virtual character exhibition for public spaces. This paper presents the CrossTalk system, including its authoring tool Scene-Maker and the CarSales exhibit. CrossTalk extends the commonplace human-to-screen interaction to an interaction triangle. The user faces two separated screens inhabited with virtual characters and interacts through a frontal touch screen. One screen features the exhibition's hostess, an agent who explains exchangeable exhibits located in the opposing screen. The current exhibit is CarSales, a demonstration of automatically generated dialogue, performed by virtual actors. The physical presence of the characters is established through the separation of screens and intensified by inter-character conversations across screens, tying hostess and exhibit together. CrossTalk utilizes a combination of both automatically generated and pre-scripted scenes, and a context memory to adapt to the user and the environment. CrossTalk's authoring tool SceneMaker, in a strict separation of narrative structure and content, provides non-experts with a screenplay-like language to create installations for staging exhibitions.

The School of Creative Arts & Technologies at Ulster University (Magee) has brought together the subject of computing with creative technologies, cinematic arts (film), drama, dance, music and design in terms of research and education. We propose here the development of a flagship computer software platform, SceneMaker, acting as a digital laboratory workbench for integrating and experimenting with the computerprocessing of new theories and methods in these multidisciplinary fields. We discuss the architecture of SceneMaker and relevant technologies for processing within its component modules. SceneMaker will enable the automated production of multimodal animated scenes from film and drama scripts or screenplays. SceneMaker will highlight affective or emotional content in digital storytelling with particular focus on character body posture, facial expressions, speech, non-speech audio, scene composition, timing, lighting, music and cinematography. Applications of SceneMaker include automated simulation of productions.

Proc. of HCI-Italy, 2003

This paper presents a character-centered architecture for interactive presentation of information. The architecture is based on scripts that result from the off-line composition of pre-built multi-modal script units and are acted by animated characters in a reactive fashion. Scripts are stored in an a script library, that allows continuous switches over scripts in reaction to the user input. The system's interactional and dramatic goals are explicitly represented, and drive the script retrieval process.

Computers & Graphics, 2008

This paper presents a powerful animation engine for developing applications with embodied animated agents called Maxine. The engine, based on open source tools, allows management of scenes and virtual characters, and pays special attention to multimodal and emotional interaction with the user. Virtual actors are endowed with facial expressions, lip-synch, emotional voice, and they can vary their answers depending on their own emotional state and the relationship with the user during conversation. Maxine virtual agents have been used in several applications: a virtual presenter was employed in MaxinePPT, a specific application developed to allow non-programmers to create 3D presentations easily using classical PowerPoint presentations; a virtual character was also used as an interactive interface to communicate with and control a domotic environment; finally, an interactive pedagogical agent was used to simplify and improve the teaching and practice of Computer Graphics subjects.

2008

We present first results for the development of a general interface to believable agents. Central element of our approach is CharanisML, a character animation system meta language, which draws from a number of previous proposals for the description of interactive 3D content. To demonstrate the general character of this interface, we implemented two different clients in 2D and 3D that are able to interpret CharanisML. These clients may be adapted as animation engines for interactive digital storytelling engines like Scenejo. Using CharanisML it is possible for an author to control characters independently from both, storytelling engines and two or three-dimensional representations.

… Using Virtual Reality Technologies for Storytelling, 2007

The proposal of this research project is to develop a standard connection mechanism to make narrative environments and the external systems that control them interoperable. Thanks to a new communication interface called RCEI, different knowledge-based storytelling systems will be able to perform interactive drama using different narrative environments that were available. The controller has a reasonable level of granularity, being the only additional requirement the development of valid adapters for both extremes of the connection. Open source implementation of this protocol and language is provided in order to save extra parsing and serializing efforts and make software more interoperable.

Embodied …, 2000

Intelligent Virtual Agents, 2007

Interactive Storytelling, 2020

In this project, we introduce finite state machines as a way to simultaneously connect an audience's input with performer output during a live performance. Our approach is novel in that we can redefine the traditional notion of an author, by dividing and balancing the responsibilities for creating and developing emergent narrative between three elements: the audience, finite state machines, and the performers. This also allows audience members a large degree of freedom to input into the system, as we can consider audience inaction as a form of productive action. We have developed the Data Generation Engine (DGE), software that generates data to be used by performers for creating and developing narrative and to provide audiences with opportunities to manipulate the DGE's data generation and distribution mechanisms. We argue that using this approach in a live theatre context allows a consistent narrative to emerge while giving the audience the freedom to engage in the narrative without disrupting the performance.

Proceedings of the 2nd International Conference on INtelligent TEchnologies for interactive enterTAINment, 2008

Today there is a number of automatic systems for developing interactive digital storytelling applications. Each one uses its own architecture, data structure and user interface which make practically impossible to create a single universal quantitative metric to compare them. While these differences are intrinsic to the artistic nature of narrative applications, developers of underlying technology could be benefited from some "evaluation standards" for these systems' functionality, interoperability and performance. This paper describes a testbed environment that has been designed as an example scenario for testing how different interactive storytelling systems confront a set of "common challenges" of this kind of applications. In order to avoid additional programming efforts an adapter that allows the connection of this environment with other systems has been implemented and released as open source.