A Component-Oriented Approach for Mixed Reality Applications

This chapter describes the evolution of a software system specifically designed to support the creation and delivery of Mixed Reality (MR) experiences. We first describe some of the attributes required of such a system. We then present a series of MR experiences that we have developed over the last four years, with companion sections on lessons learned and lessons applied. We conclude with several sample scripts that one might write to create experiences within the current version of this system. The authors' goals are to show the readers the unique challenges in developing an MR system for multimodal, multi-sensory experiences and to demonstrate how developing MR applications informs the evolution of such a framework.

2001

There are many toolkits that aid the process of constructing virtual environments. Each of these development systems has a special set of features, they support a set of hardware and they give developers and end users a level of abstraction. This paper presents a new component based system for the development of VR based applications. With a pool of components everybody should be able to create a virtual environment.

2017

Virtual Reality (VR) is widely used in training simulators of dangerous or expensive vehicles such as aircraft or heavy mining machinery. The vehicles often have very complicated controls that users need to master before attempting to operate a real world version of the machine. VR allows users to safely train in a simulated environment without the risk of injury or damaging expensive equipment in the field. VR however visually cuts off the user from the real environment, which may obtain obstructions. Users are unable to safely move or gesture while wearing a VR headset. Additionally users are unable to use standard input devices such as mice and keyboards. By mixing in a live view of the the real world, the user can still see and interact with the physical environment. The contribution of this research is presenting ways of using Mixed Reality to enhance the user experience of traditional VR based simulators. Mixed Reality improves on traditional VR simulators by allowing the user...

Smart homes, smart cars, smart classrooms are now a reality as the world becomes increasingly interconnected by ubiquitous computing technology. The next step is to interconnect such environments; however there are a number of significant barriers to advancing research in this area, most notably the lack of available environments, standards and tools etc. A possible solution is the use of simulated spaces; nevertheless as realistic as strive to make them, they are, at best, only approximations to the real spaces, with important differences such as utilising idealised rather than noisy sensor data. In this respect, an improvement to simulation is emulation, which uses specially adapted physical components to imitate real systems and environments. In this paper we present our work-in-progress towards the creation of a development tool for intelligent environments based on the interconnection of simulated, emulated and real intelligent spaces using a distributed model of mixed reality. To do so, we propose the use of physical/virtual components (xReality objects) able to be combined through a 3D graphical user interface, sharing real-time information. We present three scenarios of interconnected real and emulated spaces, used for education, achieving integration between real and virtual worlds.

Springer eBooks, 2015

Mixed reality systems overlay real data with virtual information in order to assist users in their current task. They generally combine several hardware components operating at different time scales, and software that has to cope with these timing constraints. MIRELA, for MIxed REality LAnguage, is a framework aimed at modelling, analysing and implementing systems composed of sensors, processing units, shared memories and rendering loops, communicating in a well-defined manner and submitted to timing constraints. The framework is composed of (i) a language allowing a high level, and partially abstract, specification of a concurrent real-time system, (ii) the corresponding semantics, which defines the translation of the system to concrete networks of timed automata, (iii) a methodology for analysing various real-time properties, and (iv) an implementation strategy. We present here a summary of several of our papers about this framework, as well as some recent extensions concerning probability and non-deterministic choices.

IEEE Virtual Reality (VR), 2018

This poster formulates the concept of heterogeneous distributed mixed reality (HDMR) applications in order to state some interesting research questions in this domain. HDMR applications give synchronous access to shared virtual worlds, from diverse mixed reality (MR) hardware, and at similar levels of functionality. We show the relationship between HDMR and previous concepts, state challenges in their development, and illustrate this concept and its challenges with an example.

2003

This paper describes a software platform oriented to the augmented reality/mixed reality application developer, aiming at simplifying his/her programming tasks. This platform comprises a software development kit (SDK) for the Windows environment, consisting of a set of C++ classes packaged into modules. The platform is known as MX toolkit and utilises extensively the AR toolkit, for all matters regarding marker-based tracking, but is defined at a somewhat higher abstraction level than the AR toolkit software layer, by hiding from the programmer, low level implementation details and facilitating AR/MR object-oriented programming. The MX toolkit is presented by means of a comparison with the AR toolkit processing pipeline and API and by explaining its eight system modules. We expect to provide this package as an aid for AR toolkit-based applications development. The paper also describes a visual AR/MR authoring tool, the Mix It application, from the user interface point of view, as an illustration of the development support potential of the MX toolkit SDK.

The Engineering of Mixed …, pp. 57-78, 2010

The deployment of mixed reality environments for use by members of the public poses very different challenges to those faced during focused lab studies and in defined engineering settings. This chapter discusses and presents examples from a set of case studies that have implemented and evaluated fully functioning mixed reality environments in three different organisational settings. Based on these case studies, common themes that are critical in the engineering of publicly deployed mixed reality are drawn out. Specifically, it is argued how the creation of a mixed reality interaction space depends on the technology as well as the environment, how asymmetric access provided to different sets of participants can be desirable and how social interaction reflects the particulars of the embedded technology, the length of deployment and the existing social organisation.

Entertainment Computing, 2003

This paper describes a prototyping environment for rapid application development. We combine existing AR-technologies with a component-based 3D animation library and a scripting API. Through the development of an interface to a high-level 3D modelling system we are able to use this visual tool for modelling and basic animation features in MR design. This provides content experts with a powerful tool to quickly design and test mixed reality prototypes. We consider applications in the area of interactive mixed reality illustrations in the context of technical descriptions / user manuals and interactive exhibitions in museums.

Mixed Reality Architecture (MRA) dynamically links and overlays physical and virtual spaces. This paper investigates the topology of and the relationships between the components of MRA. As a phenomenon, MRA takes its place in a long history of technologies that have influenced conditions for social interaction as well as the environment we build around us. However, by providing a flexible spatial topology spanning physical and virtual environments it presents new opportunities for social interaction across electronic media. An experimental MRA is described that allowed us to study some of the emerging issues in this field. It provided material for the development of a framework describing virtual and physical spaces, the links between those and the types of mixed reality structure that we can envisage it being possible to design using these elements. We propose that by re-introducing a level of spatiality into communication across physical and virtual environments MRA will support everyday social interaction, and may convert digital communication media from being socially conservative to a more generative form familiar from physical space.