Spatial Augmented Reality

Video display content can be extended to the walls of the living room around the TV using projection. The problem of providing appropriate projection content is hard for the computer and we solve this problem with deep neural network. We propose the peripheral vision system that provides the immersive visual experiences to the user by extending the video content using deep learning and projecting that content around the TV screen. The user may manually create the appropriate content for the existing TV screen, but it is too expensive to create it. The PCE (Pixel context encoder) network considers the center of the video frame as input and the outside area as output to extend the content using supervised learning. The proposed system is expected to pave a new road to the home appliance industry, transforming the living room into the new immersive experience platform.

Contemporary Art is not always well acknowledged even when work of successful talents. Forgotten, if not abandoned by its own community, it is often not recognised as Art by the potential users, as the Italian educational system does not always provide the right tools for its understanding. This paper focuses on two case studies which final aim is to increase the value of Contemporary Art and its educational foundations using the Spatial Augmented Reality (SAR), well known as video mapping, a performative action that allows to "augmenting" the existing contents. Today this artistic form is undergoing a split: on the one hand a growing graphic data processing are turning into independent performance whose immersiveness recalls that of gaming, on the other its use as a means of enhancement and education for Art, including the contemporary one, without taking anything away from the artistic performance. Here we want to show two preparatory performances made recently by the Studio Glowarp that follows the second mentioned front.

In questo volume vengono esposte in maniera organica e quanto più completa possibile le tecniche che, anche se non nate propriamente per la SAR, sono state utilizzate nel settore con ottimi risultati. I temi trattati hanno come filo conduttore il loro utilizzo per progettare l'edutainment, consentendo così di avere un potente strumento di comunicazione plasmabile su ogni tipo di superficie. Ciò implica una grande responsabilità in termini di messaggio, qualità e scientificità finali, soprattutto quando ad essere cinvolto è il bene culturale. Video proiettare significa comunicare ma è ovvio che per farlo correttamente è necessario conoscere gli strumenti che abbiamo a disposizione così da averne chiari i relativi limiti e potenzialità. A tal proposito il libro vuole essere una sorta di laboratorio dal quale il lettore possa far proprie le esperienze dell'autore, attingere nozioni e approfondire argomenti. Un laboratorio che ponga delle solide basi per poi progettare il proprio intervento. Effetti video, stereoscopia, HP Reveal, Resolume Arena 6, MadMapper, controller esterni, la proposizione di un visore pseudo-olografico ed infine la stampa 3D sono alcuni dei temi trattati. Infine, visto che il successo di un'installazione multimediale non può prescindere dalla progettazione del suono, uno specifico capitolo su questo tema è stato curato dal sound designer Luca Canciello che introdurrà il lettore nell’affascinante mondo del Sound Design.

Il presente volume intende essere una guida il più esaustiva possibile per tutti coloro che, già padroni delle tecniche base di video mapping, intendo- no approfondire l’argomento ed in particolare vogliono focalizzare l’attenzione sulla Spatial Augmented Reality (SAR) applicata ai beni culturali, siano essi artistici, archeologici o museali. La SAR offre in questo settore grandi potenzialità ancora non del tutto esplorate, soprattutto se si pensa ad una sua ibridazione con i sistemi museografici tradizionali o con i sistemi più avveniristici di mixed reality. Spesso infatti si configura come l'azione migliore da intraprendere per comunicare il passato in quanto, rendendo libera la fruizione da qualsiasi strumentazione, ne favorisce la funzione sociale di gruppo. Il testo, partendo da un rapido excursus sulle tecnologie attuali che stanno ponendo le basi per quelle del fututo, tratta le tecniche avanzate di acquisizione delle superfici per poi passare al tema della modellazione 3D e a MadMapper, attualmente l'unico software con specifiche funzioni per il video mapping come ad esempio la funzione per la scansione a luce strutturata. L'ultimo capitolo è infine dedicato al tema dell'Augmented Heritage e focalizza l'attenzione sull'utilizzo delle procedure multimediali volte alla comunicazione dei beni culturali attraverso un percorso ragionato che miri all’applicazione di tale tecnologia secondo un approccio critico e mediato tra analogico e digitale.

This article explores issues associated with immersive storytelling in order to examine how the field of World Building can constitute a theoretical framework for practice in the context of VR-based and Full Dome artistic projects. With respect to immersion, the intent will be to interpret the concept of storytelling in relation with the recent formulation of the concept of extended reality (XR). The very concept of World Building is transauthor and transmedia by nature. The transauthor dimension of World Building resides in the idea of subcreation, i.e., designing environments and interaction rules that help create a storytelling basis for generating multiple stories. Once the universe has been conceived, stories written by different authors take shape through transmedia processes across multiple distribution media (film, video games, web, etc.). The question then arises: How can the World Building approach shape the construction of immersive experiences? The article sets out to an...

This article explores issues associated with immersive storytelling in order to examine how the field of World Building can constitute a theoretical framework for practice in the context of VR-based and Full Dome artistic projects. With respect to immersion, the intent will be to interpret the concept of storytelling in relation with the recent formulation of the concept of extended reality (XR). The very concept of World Building is transauthor and transmedia by nature. The transauthor dimension of World Building resides in the idea of subcreation, i.e., designing environments and interaction rules that help create a storytelling basis for generating multiple stories. Once the universe has been conceived, stories written by different authors take shape through transmedia processes across multiple distribution media (film, video games, web, etc.). The question then arises: How can the World Building approach shape the construction of immersive experiences? The article sets out to answer this question, and in doing so, to contribute to the research on environmental storytelling.

This article explores issues associated with immersive storytelling in order to examine how the field of World Building can constitute a theoretical framework for practice in the context of VR-based and Full Dome artistic projects. With respect to immersion, the intent will be to interpret the concept of storytelling in relation with the recent formulation of the concept of extended reality (XR). The very concept of World Building is transauthor and transmedia by nature. The transauthor dimension of World Building resides in the idea of subcreation, i.e., designing environments and interaction rules that help create a storytelling basis for generating multiple stories. Once the universe has been conceived, stories written by different authors take shape through transmedia processes across multiple distribution media (film, video games, web, etc.). The question then arises: How can the World Building approach shape the construction of immersive experiences? The article sets out to an...

Thanks to the use of digital media, which is increasingly present in contemporary museography, the concept of anastylosis, and consequently its use, can be reviewed in the light of the results achieved to date in narrating heritage. The aim is to make the understanding of evidence in the field of archaeology and art history, which is often fragmented, more accessible.The anastylosis of such evidence can be enhanced both through digital reconstructions and the possibility of integrating them through a video projection, that makes use of SAR (Spatial Augmented Reality), directly onto the artefact, allowing appreciable and interesting levels of understanding of it to be reached. The transformation of spaces into knowledge-sensitive environments allows a perfect and natural hybridisation of analogue and digital technologies. The versatility of this technique-albeit with some limitations-allows correction and addition of all information that the advance of the studies and researches can bring over time, with relative operational simplicity. The restored and rebuilt portions, in fact immaterial and in compliance with the scientific transparency of the data, will be based on interdisciplinary studies and research in order to not alter the meaning of the artefact (considering the ease with which this information can be manipulated), thus giving a degree of accuracy that guarantees scientificity. In particular, this contribution will focus on the use of SAR in two specific case studies, in order to highlight the limits and potential of this hybrid technology; this discussion will take place in light of the ever current issues on virtual archaeology expressed both in the Charter of London and in the Principles of Seville. Digital anastylosis is a further tool of knowledge, which can thus favour the process of appropriation related to cultural heritage without this compromising conservation needs.

Most Augmented Reality applications feature an information overlay over physical objects that consists of text, images, and animated 3-D models. We go a step further by using an AR-menu and linking multiple overlays using occlusion-based triggers that help navigate between the different overlays. This paper describes the development of an AR-based tutorial for a common real-world scenario. Using multiple overlays allows us to design a comprehensive AR tutorial by presenting more information on the same physical object in a hierarchical manner with a narrative structure. We describe the design features in an attempt to elucidate the issues related to the development of the AR tutorials.

Studio Glowarp gained experiences into the application of the Spatial Augmented Reality (SAR) that allowed creating a cultural exhibition called GLOWFestival (www.glowarp.com/glowfestival.html) whose aim was the revaluation of the cultural heritage set in Ostuni, using digital technologies. The enhancement, intended not as a substitute of the historic and cultural value, provides an alternate use through a multimedia installation. The light and its linked technologies promote the culture sustainability infused in an interested audience. Thought as a four-year festival come to an end in 2016, it evoked an event totally dedicated to light as artistic medium. The web had a fundamental role as mean of sharing and diffusion that helped to start important inter-disciplinary collaboration in the ICT area. The share of the event via social network had made possible the involvement of artists from all over the world who merged into an environment open to sharing digital skills. The decisions made brought different age groups closer to the expressive potential that this medium offers as a show through the form of the artistic edutainment. The participation of artists from all over the world was free of charge. This paper wants to show the results, the findings and potentialities of the GLOWFestival.

In this presentation I will present the research developed lately in ISTAR-IUL regarding the use of digital technologies, namely Augmented Reality (AR), Virtual Reality (VR) and Spatial Augmented Reality (SAR) to explore architecture contents, namely the ones focusing cultural tourism, through several perspectives.
The goal of this research is to combine knowledge from several disciplines, including History of Architecture and the City, Architecture in the broadest sense, Information and Communication Technologies (with an emphasis on Computer Graphics) and Tourism studies. These fields have been integrated in the development of a common base of information on the architecture of cities. In general, the main goals of our research are: i) to create an information system that allows access to digital information useful for the understanding of the built and urban areas; ii) to increase the knowledge about the built environment through the use of more informational media; iii) to collaborate with IT researchers in the development of real VR and AR implementations for the city; iv) to enhance collaboration between education architecture and multimedia students through the use and development of new technologies and processes.
In the book Theorizing Digital Cultural Heritage, Fiona Cameron and Sarah Kenderdine (2007), present a compilation of papers that discuss the mutual transformation that emerging digital technologies and heritage organizations cause on each others due the current trend of combining both. In this line of thinking the research we have been doing contributes to the discussion on the combination between the digital technologies and the elements and places they refer to. Both AR and VR tools have been developed by our team to enhance a better understanding of architecture, namely the cultural heritage, to students and the broad public. All these technologies have been presented in exhibitions on cultural heritage such as the “Escola de Chicago: arranha-céus digitais” exhibition in 2014 at ISCTE-IUL where digital technologies allowed visitors to observe the story of such period in an innovative way. In this exhibition several digital tools developed were presented. Semi-immersive Virtual Reality enabled participants to explore the downtown
city of Chicago by navigating in a 3D model (Coroado, Pedro, D’Alpuim, Eloy, & Dias, 2015). Individual buildings representative of the period, as the Old Colony, were presented with an AR app (ARch) that enables
users to navigate into these buildings using several functionalities as hide/show parts of the building (Mendonça, 2014). Spatial Augmented Reality was used to augment physical models at the scale 1:10 that
were positioned in the exhibition as a complement to the ones digitally presented by AR (Velhinho, 2014).
The mobility and ease of use enabled by the augmented reality apps lead us to continuing using this technology for creating solutions of visualization of cultural information targeting tourists. In this scope we
developed an app to be used for the augmented visualization of paper maps (ARch4maps) (Gaspar et al., 2016). In this case the Lisbon map of the Prémios Valmor was used and the awarded buildings were mapped
by AR in the physical map. SeeARch is a recent AR app developed to be use outside while e.g. a tourist is walking through the city and aims to acquire information on buildings and places. All the developed apps were tested (satisfaction and usability tests) and they use of digital visualization tools proved to be of outmost interest for linking city knowledge and visitors and therefore heritage and cultural tourism.

This article wants to present the Spatial Augmented Reality technique (hereafter S.A.R.), better known as video mapping, used by the Studio Glowarp on scale models of archeological finds and cultural heritage. The use of S.A.R. technique offers new communication ways for archeology and museum fields. It helps to show fragile or destroyed goods, sometimes far away from the exhibition set, for the listed reasons considered as fragmented and hard to be reached by a bigger audience, become interesting case studies of digital anastylosis. The aim of this paper is to show from a different point of view the S. A.R. method that now can be considered not only as an " artistic " technique but also as a technique serving the cultural heritage, and to recommend it as one of the possible means of the future restoration and of the augmented heritage field.

The paper presents the results of an experiment which aims on the one hand to the elaboration of a "missing" cartography able to illustrate with suitable graphic codes the main evolutionary phases of the urban fabric of the city of Ascoli Piceno taken from archival sources and historical maps and on the other hand the experimentation aims to verify the functionality and the possibilities that the use of Spatial Augmented Reality projection-based offers in the field of communication of cartographic heritage.

У цьому дослідженні для уникнення проблем з перекладом на різні мови світу тотожних по суті за значеннєвим наповненням термінів «просторова додана реальність», «відеомеппінг» та «проекційний меппінг», які вживаються у науковому обігу для позначення однієї і тієї ж технології, автором запропоновано використовувати у міжнародному науковому дискурсі уніфікований термін «AR-проекції». Класифіковано основні різновиди AR-проекцій, які використовуються комерційними брендами з метою промоції, а також встановлено їх специфічні особливості.
Ключові слова: додана (доповнена) реальність (AR), змішана реальність (MR), AR-проекції, просторова додана реальність, відеомеппінг, проекційний меппінг, 2D, 3D.

Yu. P. Shchehelska
Two- and Three-dimensional Projections of Augmented Reality in Promotional Communications

Abstract. In this study, with the view to avoiding problems with translating into different languages of the essentially identical in meaning terms «spatial augmented reality», «video mapping» and «projection mapping», which are used in scientific circulation all over the world to denote one and the same technology, the author proposed to use in international scientific discourse the unified term «AR projections». The article also classifies the main varieties of AR projections, which are used by commercial brands for the purpose of promotion, and defines their specific features.
Keywords: augmented reality (AR), mixed reality (MR), AR-projections, spatial augmented reality, video mapping, projection mapping, 2D, 3D.

Pseudo-valorizzare i centri storici con effetti globali stellati Articolo del 7 Aprile 2019, scritto da Donato Maniello "La Notte stellata di Van Gogh illumina Trani con il videomapping". Questo il titolo di un articoletto apparso su un noto quotidiano on line che ad un'attenta lettura è contraddetto da quanto scritto successivamente dallo stesso autore. Nelle righe successive egli scrive più correttamente di "ispirazione" all'opera di Van Gogh e non più di video mapping ma di semplici proiezioni che ricordano quelle a tema natalizio. Le stesse, per intenderci, che uniformano i nostri bellissimi centri storici con effetti a "pan di stelle" che li rendono perfettamente uguali gli uni agli altri. Cambia stagione, cambiano gli effetti -che cercano di mimare un'opera d'arte con risultati sgraziati che nulla hanno a che vedere con l'originale -ma il risultato è il medesimo. Un luna park mediatico il cui unico "successo" è a tutto vantaggio del selfie tra turisti fotografi e curiosi che

Virtual studio technology plays an important role for modern television productions. Blue-screen matting is a common technique for integrating real actors or moderators into computer generated sceneries. Augmented reality offers the possibility to mix real and virtual in a more general context. This article proposes a new technological approach for combining real studio content with computer-generated information. Digital light projection allows a controlled spatial, temporal, chrominance and luminance modulation of ...

This paper analyzes what technologies, devices and equipment are available today that can contribute to the creation of an interactive environment based on Spatial Augmented Reality (SAR). The objective is to define what should be considered to build a room-sized workspace equipped with SAR tools, aimed to facilitate both artists and designers to develop their creations.
This workspace will be designed as a "white box" cabin in which the user will be placed, The user will be able to interact with images projected on the walls and primary volumes by using gestures and their own body movements. Interaction with an interface based only on projected images can be carried out in an intuitive and gestural way, removing the technological constraints that appear when working simultaneously with different devices and interfaces. To be immersed in a physical space where the environment itself acts as a working tool would change the concept of a workplace by eliminating everything that could cause problems for the creative task.
Once built, this prototype would be used to raise a methodology for analyzing the influence of SAR in the process of creation of indoor spaces, both at creative and emotional level in designers and artists, as well as the influence in the design itself obtained.

Virtual studio technology plays an important role for modern television productions. Blue-screen matting is a common technique for integrating real actors or moderators into computer generated sceneries. Augmented reality offers the possibility to mix real and virtual in a more general context. This article proposes a new technological approach for combining real studio content with computer-generated information. Digital light projection allows a controlled spatial, temporal, chrominance and luminance modulation of illumination opening new possibilities for TV studios.

This paper analyzes what technologies, devices and equipment are available today that can contribute to the creation of an interactive environment based on Spatial Augmented Reality (SAR). The objective is to define what should be considered to build a room-sized workspace equipped with SAR tools, aimed to facilitate both artists and designers to develop their creations. This workspace will be designed as a "white box" cabin in which the user will be placed, The user will be able to interact with images projected on the walls and primary volumes by using gestures and their own body movements. Interaction with an interface based only on projected images can be carried out in an intuitive and gestural way, removing the technological constraints that appear when working simultaneously with different devices and interfaces. To be immersed in a physical space where the environment itself acts as a working tool would change the concept of a workplace by eliminating everything th...

We present a novel multi-step technique for imperceptible geometry and radiometry calibration of projectorcamera systems. Our approach can be used to display geometry and color corrected images on non-optimized surfaces at interactive rates while simultaneously performing a series of invisible structured light projections during runtime. It supports disjoint projector-camera configurations, fast and progressive improvements, as well as real-time correction rates of arbitrary graphical content. The calibration is automatically triggered when misregistrations between camera, projector and surface are detected.

In this presentation I will present the research developed lately in ISTAR-IUL regarding the use of digital technologies, namely Augmented Reality (AR), Virtual Reality (VR) and Spatial Augmented Reality (SAR) to explore architecture contents, namely the ones focusing cultural tourism, through several perspectives. The goal of this research is to combine knowledge from several disciplines, including History of Architecture and the City, Architecture in the broadest sense, Information and Communication Technologies (with an emphasis on Computer Graphics) and Tourism studies. These fields have been integrated in the development of a common base of information on the architecture of cities. In general, the main goals of our research are: i) to create an information system that allows access to digital information useful for the understanding of the built and urban areas; ii) to increase the knowledge about the built environment through the use of more informational media; iii) to collaborate with IT researchers in the development of real VR and AR implementations for the city; iv) to enhance collaboration between education architecture and multimedia students through the use and development of new technologies and processes. In the book Theorizing Digital Cultural Heritage, Fiona Cameron and Sarah Kenderdine (2007), present a compilation of papers that discuss the mutual transformation that emerging digital technologies and heritage organizations cause on each others due the current trend of combining both. In this line of thinking the research we have been doing contributes to the discussion on the combination between the digital technologies and the elements and places they refer to. Both AR and VR tools have been developed by our team to enhance a better understanding of architecture, namely the cultural heritage, to students and the broad public. All these technologies have been presented in exhibitions on cultural heritage such as the “Escola de Chicago: arranha-céus digitais” exhibition in 2014 at ISCTE-IUL where digital technologies allowed visitors to observe the story of such period in an innovative way. In this exhibition several digital tools developed were presented. Semi-immersive Virtual Reality enabled participants to explore the downtown city of Chicago by navigating in a 3D model (Coroado, Pedro, D’Alpuim, Eloy, & Dias, 2015). Individual buildings representative of the period, as the Old Colony, were presented with an AR app (ARch) that enables users to navigate into these buildings using several functionalities as hide/show parts of the building (Mendonça, 2014). Spatial Augmented Reality was used to augment physical models at the scale 1:10 that were positioned in the exhibition as a complement to the ones digitally presented by AR (Velhinho, 2014). The mobility and ease of use enabled by the augmented reality apps lead us to continuing using this technology for creating solutions of visualization of cultural information targeting tourists. In this scope we developed an app to be used for the augmented visualization of paper maps (ARch4maps) (Gaspar et al., 2016). In this case the Lisbon map of the Prémios Valmor was used and the awarded buildings were mapped by AR in the physical map. SeeARch is a recent AR app developed to be use outside while e.g. a tourist is walking through the city and aims to acquire information on buildings and places. All the developed apps were tested (satisfaction and usability tests) and they use of digital visualization tools proved to be of outmost interest for linking city knowledge and visitors and therefore heritage and cultural tourism.

ABSTRACT This paper analyzes what technologies, devices and equipment are available today that can contribute to the creation of an interactive environment based on Spatial Augmented Reality (SAR). The objective is to define what should be considered to build a room-sized workspace equipped with SAR tools, aimed to facilitate both artists and designers to develop their creations. This workspace will be designed as a "white box" cabin in which the user will be placed, The user will be able to interact with images projected on the walls and primary volumes by using gestures and their own body movements. Interaction with an interface based only on projected images can be carried out in an intuitive and gestural way, removing the technological constraints that appear when working simultaneously with different devices and interfaces. To be immersed in a physical space where the environment itself acts as a working tool would change the concept of a workplace by eliminating everything that could cause problems for the creative task. Once built, this prototype would be used to raise a methodology for analyzing the influence of SAR in the process of creation of indoor spaces, both at creative and emotional level in designers and artists, as well as the influence in the design itself obtained.

ABSTRACT This paper analyzes what technologies, devices and equipment are available today that can contribute to the creation of an interactive environment based on Spatial Augmented Reality (SAR). The objective is to define what should be considered to build a room-sized workspace equipped with SAR tools, aimed to facilitate both artists and designers to develop their creations. This workspace will be designed as a "white box" cabin in which the user will be placed, The user will be able to interact with images projected on the walls and primary volumes by using gestures and their own body movements. Interaction with an interface based only on projected images can be carried out in an intuitive and gestural way, removing the technological constraints that appear when working simultaneously with different devices and interfaces. To be immersed in a physical space where the environment itself acts as a working tool would change the concept of a workplace by eliminating everything that could cause problems for the creative task. Once built, this prototype would be used to raise a methodology for analyzing the influence of SAR in the process of creation of indoor spaces, both at creative and emotional level in designers and artists, as well as the influence in the design itself obtained.

This article aims to enrich the conversation about the colonization of students' horizontal discourse by the vertical discourse in school, according the Bernstein's distinction, researching the creative work of students in the context of a project, as a significant factor to this colonization. LiteraMovie Cartoon story© is a hybrid creation combining literature, movie and visualization based on cartoons. The reformation of the classic texts in a post-typography educational environment should be considered as a recreation of the text, based on new, as well as on old literacies. So, during two successive projects in the last two years, students of different grades were led to edit classic texts, creating a multilingual and multimedia e-edition with a cross-cultural perception. Finally, it is proposed that this creativity in producing e-learning objects could be the meeting point of formal and non-formal education, of the claim for knowledge acquired after a planned educational procedure and for learning inaction .

In this paper we propose a feature-based alignment method for visualizing geographical content on a movable paper map using non-calibrated projector-camera pair. In order to project correctly aligned geographical content on a movable paper map, we detect two kinds of features: printed features and projected features on a paper map. First, we print landmark points as map features on the paper map for registration purpose. We then project landmark points onto the paper map. We then detect these printed and projected features simultaneously. The transformation from the projector coordinate system to the image coordinate system is estimated using the feature correspondences. This transformation is used to warp the geographical content before the corrected projection. In succeeding frames, the features are tracked in order to update the transformation. We evaluated our method against the motion of the paper map, the camera and the projector with average projection error of 9.5 pixels.