Spatial Augmented Reality

A GPU-based visualization approach is presented to show the discrepancies between the 3D model and the fabrication of that model. To show the differences, a 3D scanner is constructed to scan the fabrication for comparison with the 3D model. To compare the detailed differences, a high resolution camera with a projector is used. To demonstrate its application, a sculpting assistance system is implemented. The system can capture the three-dimensional model iteratively and provide information by rendering different colors on the surface to show the topological and geometric differences between the final target model and the current model. Then the user can see how to modify the current model to best approximate the final target model. The topological difference is obtained by rendering on the screen the 3D data from both the final target model and the current model. The user can manipulate and observe their differences. The local geometry is compared in the GPU and rendered on the real ...

In recent years, the academics have always been more and more interested into the study of the polychromies of ancient works and aware of their value. However, the audience who visits the museums it’s not aware of it, ignoring the fact that the white marble of the statues and buildings it’s not what we can call a stylistic choice but evidence of the flowing time. The collective imagination that has always considered White as the supreme colour of the ancient sculpture needs now to re-considerate its position facing with an interdisciplinary approach, that allows to revaluate and better appreciate the pieces of the ancient sculpture and its decorations, being aware of the importance that the colour had in that times. It is therefore important to use forms of communication that make possible to highlight the colour appearance of the original works in the eyes of the final users, pieces kept into our museums object of precise studies. It is important to pay attention to the new communication media that have the clear role of showing the final out- comes of the research to the public, a research carried out by more sophisticated means.
Spatial Augmented Reality (SAR) is one of the architectural technologies that lends itself best to such communication experimentations in archaeological and museum settings. The reasons of its fortune can be ascribe to its relative user-friendliness and the fact that can be used by a group, an audience. The archaeological and museum assets are well suited to the use of this technology because it allows to reconstruct missing fragments or original colours by overlapping the real and virtual model through the use of matching techniques.

Digital technology interventions become non-separable from contemporary urbanscape. This study discusses drone-based spatial video projection (D-SVP) as a digital alternative to revitalize under-valued public open spaces (UVPOSs). Spatial video projection is a projection-based augmented reality technology that enables the projection of visual content over 3-D objects of any scale. D-SVP is an integrated system suggested by this study where the drone will be supported by data collection and projection devices, which means that D-SVP works for both the data collection and showing at once.

This study suggests a digital model for revitalizing undervalued public open spaces (UVPOS) as one of the contemporary cities' phenomena. It discusses spatial video projection (SVP) as a digital do-it-yourself (DIY) paradigm for UVPOS revitalization. The application of tricycle SVP is suggested as a model for this paradigm. As a result, this research conceptualizes the SVP as a digital DIY paradigm for UVPOS revitalization and identifies the main components that must be achieved and integrated with the city's official planning and design systems.

In computer vision, many active illumination techniques employ Projector-Camera systems to extract useful information from the scenes. Known illumination patterns are projected onto the scene and their deformations in the captured images are then analyzed. We observe that the local frequencies in the captured pattern for the mirror-like surfaces is different from the projected pattern. This property allows us to design a custom Projector-Camera system to segment mirror-like surfaces by analyzing the local frequencies in the captured images. The system projects a sinusoidal pattern and capture the images from projector's point of view. We present segmentation results for the scenes including multiple reflections and inter-reflections from the mirror-like surfaces. The method can further be used in the separation of direct and global components for the mirror-like surfaces by illuminating the non-mirror-like objects separately. We show how our method is also useful for accurate estimation of shape of the non-mirror-like regions in the presence of mirror-like regions in a scene. CCS Concepts •Computing methodologies → Computational photography; Image processing;

We introduce ideas, proposed technologies, and initial results for an office of the future that is based on a unified application of computer vision and computer graphics in a system that combines and builds upon the notions of the CAVE™, tiled display systems, and image-based modeling. The basic idea is to use real-time computer vision techniques to dynamically extract per-pixel depth and reflectance information for the visible surfaces in the office including walls, furniture, objects, and people, and then to either project images on the surfaces, render images of the surfaces , or interpret changes in the surfaces. In the first case, one could designate everyday (potentially irregular) real surfaces in the office to be used as spatially immersive display surfaces, and then project high-resolution graphics and text onto those surfaces. In the second case, one could transmit the dynamic image-based models over a network for display at a remote site. Finally, one could interpret dynamic changes in the surfaces for the purposes of tracking, interaction, or augmented reality applications. To accomplish the simultaneous capture and display we envision an office of the future where the ceiling lights are replaced by computer controlled cameras and "smart" projectors that are used to capture dynamic image-based models with imperceptible structured light techniques, and to display high-resolution images on designated display surfaces. By doing both simultaneously on the designated display surfaces, one can dynamically adjust or autocalibrate for geometric, intensity, and resolution variations resulting from irregular or changing display surfaces, or overlapped projector images. Our current approach to dynamic image-based modeling is to use an optimized structured light scheme that can capture per-pixel depth and reflectance at interactive rates. Our system implementation is not yet imperceptible, but we can demonstrate the approach in the laboratory. Our approach to rendering on the designated (potentially irregular) display surfaces is to employ a two-pass projective texture scheme to generate images that when projected onto the surfaces appear correct to a moving headtracked observer. We present here an initial implementation of the overall vision, in an office-like setting, and preliminary demonstrations of our dynamic modeling and display techniques.

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Spatial augmented reality (SAR) makes possible the projection of virtual environments into the real world. In this demo, we propose to demonstrate our SAR tools dedicated to the creation of physical drawings. From the most simple tools: the projection on virtual guidelines enabling to trace lines and curves to more advanced techniques enabling stereoscopic drawing through the projection of a 3D scene. This demo presents how we can use computer graphics tools to ease the drawing, and how it will enable new kinds of physical drawings.

Cultural heritage artifacts often contain details that are difficult to distinguish due to aging effects such as erosion. We propose the revealing flashlight, a new interaction and visualization technique in spatial augmented reality that helps to reveal the detail of such artifacts. We locally and interactively augment a physical artifact by projecting an expressive 3D visualization that highlights its features, based on an analysis of its previously acquired geometry at multiple scales. Our novel interaction technique simulates and improves the behavior of a flashlight: according to 6-degree-of-freedom input, we adjust the numerous parameters involved in the expressive visualization-in addition to specifying the location to be augmented. This makes advanced 3D analysis accessible to the greater public with an everyday gesture, by naturally combining the inspection of the real object and the virtual object in a co-located interaction and visualization space. The revealing flashlight can be used by archeologists, for example, to help decipher inscriptions in eroded stones, or by museums to let visitors interactively discover the geometric details and meta-information of cultural artifacts. We confirm its effectiveness, ease-of-use and ease-of-learning in an initial preliminary user study and by the feedbacks of two public exhibitions.

In this paper we introduce a cooperative environment between the Interactive Digital TV (IDTV) and home networking with the aim of allowing the interaction between interactive TV applications and the controllers of the in-home appliances in a natural way. More specifically, our proposal consists of merging MHP (Multimedia Home Platform), one of the main standard frameworks for IDTV, with OSGi (Open Service Gateway Initiative), the most widely used open platform to set up Residential Gateways. To overcome the radically different nature of these specifications the function-oriented MHP middleware and the service-oriented OSGi framework , we define a new kind of application, coined as XbundLET. Although this software bridge is suitable to enable the interaction between MHP and OSGi applications in both directions, we concretely focus on exposing our implementation experience in only one direction: from MHP to the OSGi world.

Coupling is a key concept in the field of embodied interaction with digital products and systems, describing how digital phenomena relate to the physical world. In this paper, we present a Research through Design process in which the concept of coupling is explored and deepened. The use case that we employed to conduct our research is an industrial workplace proposed by Audi Brussels and Kuka. Our aim was to enrich this workplace with projection, or Spatial Augmented Reality, while focusing on operator interaction. We went through three successive design iterations, each of which resulted in a demonstrator. We present each of the three demonstrators, focusing on how they propelled our understanding of coupling. We establish a framework in which coupling between different events, be they physical or digital, emerges on four different aspects: time, location, direction, and expression. We bring the first three aspects together under one heading—coupling of meaning—and relate it to eas...

Coupling is a key concept in the field of embodied interaction with digital products and systems, describing how digital phenomena relate to the physical world. In this paper, we present a Research through Design process in which the concept of coupling is explored and deepened. The use case that we employed to conduct our research is an industrial workplace proposed by Audi Brussels and Kuka. Our aim was to enrich this workplace with projection, or Spatial Augmented Reality, while focusing on operator interaction. We went through three successive design iterations, each of which resulted in a demonstrator. We present each of the three demonstrators, focusing on how they propelled our understanding of coupling. We establish a framework in which coupling between different events, be they physical or digital, emerges on four different aspects: time, location, direction, and expression. We bring the first three aspects together under one heading—coupling of meaning—and relate it to eas...

Coupling is a key concept in the field of embodied interaction with digital products and systems, describing how digital phenomena relate to the physical world. In this paper, we present a Research through Design process in which the concept of coupling is explored and deepened. The use case that we employed to conduct our research is an industrial workplace proposed by Audi Brussels and Kuka. Our aim was to enrich this workplace with projection, or Spatial Augmented Reality, while focusing on operator interaction. We went through three successive design iterations, each of which resulted in a demonstrator. We present each of the three demonstrators, focusing on how they propelled our understanding of coupling. We establish a framework in which coupling between different events, be they physical or digital, emerges on four different aspects: time, location, direction, and expression. We bring the first three aspects together under one heading—coupling of meaning—and relate it to eas...

Coupling is a key concept in the field of embodied interaction with digital products and systems, describing how digital phenomena relate to the physical world. In this paper, we present a Research through Design process in which the concept of coupling is explored and deepened. The
use case that we employed to conduct our research is an industrial workplace proposed by Audi Brussels and Kuka. Our aim was to enrich this workplace with projection, or Spatial Augmented Reality, while focusing on operator interaction. We went through three successive design iterations, each of which resulted in a demonstrator. We present each of the three demonstrators, focusing on how they propelled our understanding of coupling. We establish a framework in which coupling
between different events, be they physical or digital, emerges on four different aspects: time, location, direction, and expression. We bring the first three aspects together under one heading—coupling of meaning—and relate it to ease of use and pragmatic usability. We uncover the characteristics of the fourth aspect—coupling of expression—and link it to the psychological wellbeing of the operator in the workplace. We conclude this paper by highlighting its contribution to the embodied interaction
research agenda.

We propose a toolkit for creating Tangible Out-of-Body Experiences: exposing the inner states of users using physiological signals such as heart rate or brain activity. Tobe can take the form of a tangible avatar displaying live physiological readings to reflect on ourselves and others. Such a toolkit could be used by researchers and designers to create a multitude of potential tangible applications, including (but not limited to) educational tools about Science Technologies Engineering and Mathematics (STEM) and cognitive science, medical applications or entertainment and social experiences with one or several users or Tobes involved. Through a co-design approach, we investigated how everyday people picture their physiology and we validated the acceptability of Tobe in a scientific museum. We also give a practical example where two users relax together, with insights on how Tobe helped them to synchronize their signals and share a moment.

This dissertation identifies the synergies between physical and virtual environments when designing for immersive experiences in interactive dance performances. The integration of virtual information in physical space is transforming our interactions and experiences with the world. By using the body and creative expression as the interface between real and virtual worlds, dance performance creates a privileged framework to research and design interactive mixed reality environments and immersive augmented architectures. The research is primarily situated in the fields of visual art and interaction design. It combines performance with transdisciplinary fields and intertwines practice with theory. The theoretical and conceptual implications involved in designing and experiencing immersive hybrid environments are analyzed using the reality-virtuality v continuum. These theories helped frame the ways augmented reality architectures are achieved through the integration of dance performance with digital software and reception displays. They also helped identify the main artistic affordances and restrictions in the design of augmented reality and augmented virtuality environments for live performance. These pervasive media architectures were materialized in three field experiments, the live dance performances. Each performance was created in three different stages of conception, design and production. The first stage was to "digitize" the performer's movement and brain activity to the virtual environment and our system. This was accomplished through the use of depth sensor cameras, 3D motion capture, and brain computer interfaces. The second stage was the creation of the computational architecture and software that aggregates the connections and mapping between the physical body and the spatial dynamics of the virtual environment. This process created real-time interactions between the performer's behavior and motion and the real-time generative computer 3D graphics. Finally, the third stage consisted of the output modality: 3D projector based augmentation techniques were adopted in order to overlay the virtual environment onto physical space. This thesis proposes and lays out theoretical, technical, and artistic frameworks between 3D digital environments and moving bodies in dance performance. By sensing the body and the brain with the 3D virtual environments, new layers of augmentation and interactions are established, and ultimately this generates mixed reality environments for embodied improvisational self-expression.

Modeling dynamic scenes is a challenging problem faced by applications such as digital content generation and motion analysis. Fast single-frame methods obtain sparse depth samples while multipleframe methods often reply on the rigidity of the object to correspond a small number of consecutive shots for decoding the pattern by feature tracking. We present a novel structured-light acquisition method which can obtain dense depth and color samples for moving and deformable surfaces undergoing repetitive motions. Our key observation is that for repetitive motion, different views of the same motion state under different structured-light patterns can be corresponded together by image matching. These images densely encode an effectively "static" scene with time-multiplexed patterns that we can use for reconstruction of the timevarying scene. At the same time, color samples are reconstructed by matching images illuminated using white light to those using structured-light patterns. We demonstrate our approach using several real-world scenes.

This paper discusses how Spatial Augmented Reality (SAR) can support design sessions in the fields of product, interface and packaging design. We analyse how the scope of a design session and the type of collaboration require different features of the SAR technology. We benchmark a SAR platform under development within the SPARK project (http://spark-project.net/) and state of the art solutions against the proposed classification framework to evaluate the current state of the platform, its limitations and to outline SAR technology requirements for future development possibilities.

Visual coherence between virtual and real objects is a major issue in creating convincing augmented reality (AR) applications. To achieve this seamless integration, actual light conditions must be determined in real time to ensure that virtual objects are correctly illuminated and cast consistent shadows. In this paper, we propose a novel method to estimate daylight illumination and use this information in outdoor AR applications to render virtual objects with coherent shadows. The illumination parameters are acquired in real time from context-aware live sensor data. The method works under unprepared natural conditions. We also present a novel and rapid implementation of a state-of-the-art skylight model, from which the illumination parameters are derived. The Sun's position is calculated based on the user location and time of day, with the relative rotational differences estimated from a gyroscope, compass and accelerometer. The results illustrated that our method can generate ...

This article explores issues associated with immersive storytelling in order to examine how the field of World Building can constitute a theoretical framework for practice in the context of VR-based and Full Dome artistic projects. With respect to immersion, the intent will be to interpret the concept of storytelling in relation with the recent formulation of the concept of extended reality (XR). The very concept of World Building is transauthor and transmedia by nature. The transauthor dimension of World Building resides in the idea of subcreation, i.e., designing environments and interaction rules that help create a storytelling basis for generating multiple stories. Once the universe has been conceived, stories written by different authors take shape through transmedia processes across multiple distribution media (film, video games, web, etc.). The question then arises: How can the World Building approach shape the construction of immersive experiences? The article sets out to an...

This article explores issues associated with immersive storytelling in order to examine how the field of World Building can constitute a theoretical framework for practice in the context of VR-based and Full Dome artistic projects. With respect to immersion, the intent will be to interpret the concept of storytelling in relation with the recent formulation of the concept of extended reality (XR). The very concept of World Building is transauthor and transmedia by nature. The transauthor dimension of World Building resides in the idea of subcreation, i.e., designing environments and interaction rules that help create a storytelling basis for generating multiple stories. Once the universe has been conceived, stories written by different authors take shape through transmedia processes across multiple distribution media (film, video games, web, etc.). The question then arises: How can the World Building approach shape the construction of immersive experiences? The article sets out to an...

We synthesize imagery using optical flow, color analysis, and heuristics to extrapolate beyond the edge of a television screen. Projectors provide the viewer's peripheral vision with low-detail dynamic patterns consistent with the input video, creating a sense of immersion and the impression that the scene wraps around the viewer.

Virtual studio technology plays an important role for modern television productions. Blue-screen matting is a common technique for integrating real actors or moderators into computer generated sceneries. Augmented reality offers the possibility to mix real and virtual in a more general context. This article proposes a new technological approach for combining real studio content with computer-generated information. Digital light projection allows a controlled spatial, temporal, chrominance and luminance modulation of ...

In this paper we present an innovative way to combine a laser (vector) projector with a video (raster) projector, using the former to enhance or augment the 3D images projected with the latter. As we will show, two main problems arise from this setting. One is to find a relation between the video projector space and the laser projector space, so that we can transform pixels into laser coordinates. The second problem is related to hiding virtually occluded laser segments to create a seamless fusion of the two projected parts. Also, we apply a calibration technique to compensate visual deformations caused by projector placement and/or non-planar screens.

Mixed reality is the combination of real and virtual scene content. Besides correct alignment of the two modalities and correct occlusion handling the core issues to be tackled are the degree of realism and the ease of use. For a convincing perception correct occlusion handling and shadowing is mandatory. We present a system for mixed reality applications which features enhanced realism through automatic shadow computation and supported placement functions for virtual content. The system relies on the incorporation of a ToF-camera device. This device delivers real-time depth maps of the environment and is used to build a static environment model. It also allows correct handling of mutual occlusion between real and virtual content, shadow computation and enhanced content planning. Other mixed reality systems require expensive studio setups with complex camera tracking installations. Often multi- camera approaches in combination with chroma-keying facilities are used. The presented sy...

Video display content can be extended to the walls of the living room around the TV using projection. The problem of providing appropriate projection content is hard for the computer and we solve this problem with deep neural network. We propose the peripheral vision system that provides the immersive visual experiences to the user by extending the video content using deep learning and projecting that content around the TV screen. The user may manually create the appropriate content for the existing TV screen, but it is too expensive to create it. The PCE (Pixel context encoder) network considers the center of the video frame as input and the outside area as output to extend the content using supervised learning. The proposed system is expected to pave a new road to the home appliance industry, transforming the living room into the new immersive experience platform.

This paper presents a case study of Half Real, a live action, interactive theater show employing spatial augmented reality. Half Real is based on a murder investigation, where the live audience votes on how the investigation proceeds. Half Real immerses live actors into a virtual world projected onto the set. Using spatial augmented reality technology in theater brings new possibilities that would not be possible with simple video projection. Half Real models the set as a 3D virtual environment. Actors are tracked as they move about on stage, with the projected content responding to their movements. A real time AR system allows content to be generated procedurally. Half Real makes use of this by projecting vote options and results directly into the virtual environment. This paper describes Half Real, with a focus on the technology used to make the production possible. We describe the benefits of the techniques used and the challenges faced during production.

A 3D imaging and mapping system that can handle both multiple-viewers and dynamic-objects is attractive for many applications. We propose a freeform structured light system that does not rigidly constrain camera(s) to the projector. By introducing an optimized phasecoded aperture in the projector, we transform the projector pattern to encode depth in its defocus robustly; this allows a camera to estimate depth, in projector coordinates, using local information. Additionally, we project a Kronecker-multiplexed pattern that provides global context to establish correspondence between camera and projector pixels. Together with aperture coding and projected pattern, the projector offers a unique 3D labeling for every location of the scene. The projected pattern can be observed in part or full by any camera, to reconstruct both the 3D map of the scene and the camera pose in the projector coordinates. This system is optimized using a fully differentiable rendering model and a CNN-based reconstruction. We build a prototype and demonstrate high-quality 3D reconstruction with an unconstrained camera, for both dynamic scenes and multi-camera systems.

Interactive technologies in museums enhance the visit experience by providing contextual information and fostering collaboration and participation. In this paper we revisit the design of the ubiquitous transparent exhibition case from a museum learning perspective. Transparent cases with interactive properties can complement other museum technologies and mitigate some of their shortcomings, such as the group isolation caused by audio guides and mobile devices. This paper focuses on the design of interactive cases and makes three contributions. First, based on field observations and interviews we present a list of requirements for interactive cases. Second, we propose a design space with dimensions grouped around the themes of hardware, interaction and information design. Our design space suggests interactive cases which present collocated information at increasing levels of detail, facilitate social interaction, and integrate with other technologies. Third, we demonstrate our design space through sample case designs and discuss the general technical challenges.

Effective designs rarely emerge from good structural design or aesthetics alone. It is more often the result of the end product's overall design integrity. Added to this, design is inherently an interdisciplinary collaborative activity. With this in mind, today's tools are not powerful enough to design complex physical environments, such as command control centers or hospital operating theaters. This paper presents the concept of employing projector-based augmented reality techniques to enhance interdisciplinary design processes.

In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for onset motion capture or real-time broadcasting of virtual sets.

The article investigates the 21st-century fashion through the theoretical lens of the mise en abyme, focusing on the mixed reality as an extended panorama between physical and digital. Four case studies were chosen within the Italian context, deemed as a place of experimentation and interaction between digital innovation and manufacturing tradition. Those cases have developed projects in mixed reality during the months of social distancing imposed by Covid-19, with significant consequences on creative processes, production, distribution, communication and consumption of fashion. In this scenario, the article questions the expansion of fashion experience, the ongoing transformation of the digital object, the conception and perception of time and the idea of intimacy at the time of Covid.

The article investigates the 21st-century fashion through the theoretical lens of the mise en abyme, focusing on the mixed reality as an extended panorama between physical and digital. Four case studies were chosen within the Italian context, deemed as a place of experimentation and interaction between digital innovation and manufacturing tradition. Those cases have developed projects in mixed reality during the months of social distancing imposed by Covid-19, with significant consequences on creative processes, production, distribution, communication and consumption of fashion. In this scenario, the article questions the expansion of fashion experience, the ongoing transformation of the digital object, the conception and perception of time and the idea of intimacy at the time of Covid.

Architecture practice is reshaped by new technological developments such as virtual reality, augmented reality, 3D printing, computational design, robotics, etc. Future architects are expected to deliver spatial solutions, to understand changing living conditions and to discover the different future by responsing technological improvements. In architecture schools, students should explore and investigate design practice with different production methods, systems and materials by using various technologies in collaborations with interdisciplinary partners and industry. The study aims to understand the STEAM concept which is considered as a promising education approach and formulated; science, technology, engineering, math and then adds an ‘A’ for arts which represents the creative thinking and design approach. The STEAM concept and its place in the architectural education are discussed. The discipline of architecture is an intersection of technology, art and science. The architecture...

We present a system that superimposes multiple projections onto an object of arbitrary shape and color to produce high-resolution appearance changes. Our system produces appearances at an improved resolution compared to prior works and can change appearances at near interactive rates. Three main components are central to our system. First, the problem of computing compensation images is formulated as a constrained optimization which yields high-resolution appearances. Second, decomposition of the target appearance into base and scale images enables fast swapping of appearances on the object by requiring the constrained optimization to be computed only once per object. Finally, to make high-quality appearance edits practical, an elliptical Gaussian is used to model projector pixels and their interaction between projectors. To the best of our knowledge, we build the first system that achieves high-resolution and high-quality appearance edits using multiple superimposed projectors on c...

Most Augmented Reality applications feature an information overlay over physical objects that consists of text, images, and animated 3-D models. We go a step further by using an AR-menu and linking multiple overlays using occlusion-based triggers that help navigate between the different overlays. This paper describes the development of an AR-based tutorial for a common real-world scenario. Using multiple overlays allows us to design a comprehensive AR tutorial by presenting more information on the same physical object in a hierarchical manner with a narrative structure. We describe the design features in an attempt to elucidate the issues related to the development of the AR tutorials.

This paper discusses how Spatial Augmented Reality (SAR) can support design sessions in the fields of product, interface and packaging design. We analyse how the scope of a design session and the type of collaboration require different features of the SAR technology. We benchmark a SAR platform under development within the SPARK project (http://spark-project.net/) and state of the art solutions against the proposed classification framework to evaluate the current state of the platform, its limitations and to outline SAR technology requirements for future development possibilities.

Projects in the area of architectural design and urban planning typically engage several architects as well as experts from other professions. While the design and review meetings thus often involve a large number of cooperating participants, the actual design is still done by the individuals in the time in between those meetings using desktop PCs and CAD applications. A real collaborative approach to architectural design and urban planning is often limited to early paper-based sketches. In order to overcome these limitations, we designed and realized the ARTHUR system, an Augmented Reality (AR) enhanced round table to support complex design and planning decisions for architects. While AR has been applied to this area earlier, our approach does not try to replace the use of CAD systems but rather integrates them seamlessly into the collaborative AR environment. The approach is enhanced by in

Television and movie images have been altered ever since it was technically possible. Nowadays embedding advertisements, or incorporating text and graphics in TV scenes, are common practice, but they can not be considered as integrated part of the scene. The introduction of new services for interactive augmented television is discussed in this paper. We analyse the main aspects related with the whole chain of augmented reality production. Interactivity is one of the most important added values of the digital television: This paper aims to break the model where all TV viewers receive the same final image. Thus, we introduce and discuss the new concept of interactive augmented television, i. e. real time composition of video and computer graphics-e.g. a real scene and freely selectable images or spatial rendered objects-edited and customized by the end user within the context of the user's set top box and TV Digital Peer Publishing Licence Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the current version of the Digital Peer Publishing Licence (DPPL). The text of the licence may be accessed and retrieved via Internet at http://www.dipp.nrw.de/. First presented at the 4th European Interactive TV Conference EuroITV 2006, extended and revised for JVRB receiver. We demonstrate a sample application introducing "Interactive Augmented Television" for sport broadcasts additionally with 3D virtual objects in order to enhance or alter the presentation of the match with a new interface. We also introduce a pure virtual world where the user can select the camera position.

A system of coordinated projectors and cameras enables the creation of projected light displays that are robust to environmental disturbances. This paper describes approaches for tackling both geometric and photometric aspects of the problem: (1) the projected image remains stable even when the system components (projector, camera or screen) are moved; (2) the display automatically removes shadows caused by users moving between a projector and the screen, while simultaneously suppressing projected light on the user. The former can be accomplished without knowing the positions of the system components. The latter can be achieved without direct observation of the occluder. We demonstrate that the system responds quickly to environmental disturbances and achieves low steady-state errors.

Two related problems of front projection displays which occur when users obscure a projector are: (i) undesirable shadows cast on the display by the users, and (ii) projected light falling on and distracting the users. This paper provides a computational framework for solving these two problems based on multiple overlapping projectors and cameras. The overlapping projectors are automatically aligned to display the same dekeystoned image. The system detects when and where shadows are cast by occluders and is able to determine the pixels which are occluded in different projectors. Through a feedback control loop, the contributions of unoccluded pixels from other projectors are boosted in the shadowed regions, thereby eliminating the shadows. In addition, pixels which are being occluded are blanked, thereby preventing the projected light from falling on a user when they occlude the display. This can be accomplished even when the occluders are not visible to the camera. The paper presents results from a number of experiments demonstrating that the system converges rapidly with low steady-state errors.

A system of coordinated projectors and cameras enables the creation of projected light displays that are robust to environmental disturbances. This paper describes approaches for tackling both geometric and photometric aspects of the problem: (1) the projected image remains stable even when the system components (projector, camera or screen) are moved; (2) the display automatically removes shadows caused by users moving between a projector and the screen, while simultaneously suppressing projected light on the user. The former can be accomplished without knowing the positions of the system components. The latter can be achieved without direct observation of the occluder. We demonstrate that the system responds quickly to environmental disturbances and achieves low steady-state errors.

We previously studied methods leveraging pixel-level visible light communication (PVLC) that embeds imperceptible information for human eyes in each pixel of an image. The PC computation load and amount of data transferred between the PC and projector in previous PVLC systems were excessive because the PC executed both the video and data encoding processes. As a result, it was impossible to achieve both high-dynamic-range images and dynamic updates of the images and data. In this paper, we propose a dynamic PVLC system that offers high video quality and interactively updates the PVLC information through hardware encoding processing. Our system can project a 24-bit gradation color PVLC video that contains 64-bit data at 120 fps by synchronously controlling the ON/OFF states of the DMD and LED light sources at the given performance limit of the projector.

We propose the system that provides an immersive experience by automatically extending the video content of the TV to the wall. Though it is possible to manually create the appropriate content that works on the existing TV screen, the cost is too expensive to create. The proposed deep neural network uses the center of the video as input and the outside area as output. The trained network automatically generates the extrapolated video according to the input video and a projector projects the resulting content on the living room wall around the TV thereby improving the immersive feeling. The proposed system is expected to pave a new road to the home appliance industry by transforming the living room into the new immersive experience platform.

Video display content can be extended to the walls of the living room around the TV using projection. The problem of providing appropriate projection content is hard for the computer and we solve this problem with deep neural network. We propose the peripheral vision system that provides the immersive visual experiences to the user by extending the video content using deep learning and projecting that content around the TV screen. The user may manually create the appropriate content for the existing TV screen, but it is too expensive to create it. The PCE (Pixel context encoder) network considers the center of the video frame as input and the outside area as output to extend the content using supervised learning. The proposed system is expected to pave a new road to the home appliance industry, transforming the living room into the new immersive experience platform.

In the last decade, the advances in mobile technologies and location-based applications reshaped our mutual relationship with the urban environment. These technologies, as both a mean and barrier to the engagement between humans and their environment, have transformed the urban experience in profound ways. Urban experience is a relatively new concept introduced with the rise of modern cities in the nineteenth century. Its loss due to the rapid urbanization has been a subject of debate since then. Among the discussions that take place in this debate, the figure of ‘flâneur’ plays an extensive role. The flâneur is a figure who wanders through and appropriates the metropolitan city in pursuit of urban experience and reaps aesthetic meaning from the spectacle of the teeming crowds. Flânerie, or the act of wandering, and its implications for our understanding of urban life have been profound. Today, mobile technologies create a new kind of urban wanderer which is described as “post-flâne...