Papers by Maurício Sousa
Interactions, Oct 25, 2017
Three-dimensional virtual environments are present in many different applications, being used eve... more Three-dimensional virtual environments are present in many different applications, being used even in small handheld devices. To navigate in these environments using such devices, most of current solutions rely on multi-touch interactions. However, previous works have already stated that multi-touch gestures on smartphones are not always feasible. In this work we present ThumbCam, a novel single-touch technique for camera manipulation on 3D virtual environments. With our solution, the user is able to move and look around and circle points of interest, while interacting using only his thumb. We compare ThumbCam with other state-of-the-art techniques, showing that it can offer more operations with a single touch. A qualitative user evaluation revealed that users found our solution appealing.
Head-Mounted Displays (HMDs) and similar 3D visualization devices are becoming ubiquitous. Going ... more Head-Mounted Displays (HMDs) and similar 3D visualization devices are becoming ubiquitous. Going a step forward, HMD seethrough systems bring virtual objects to real world settings, allowing augmented reality to be used in complex engineering scenarios. Of these, optical and video see-through systems differ on how the real world is captured by the device. To provide a seamless integration of real and virtual imagery, the absolute depth and size of both virtual and real objects should match appropriately. However, these technologies are still in their early stages, each featuring different strengths and weaknesses which affect the user experience. In this work we compare optical to video see-through systems, focusing on depth perception via exocentric and egocentric methods. Our study pairs Meta Glasses, an off-the-shelf optical see-through, to a modified Oculus Rift setup with attached video-cameras, for video see-through. Results show that, with the current hardware available, the video see-through configuration provides better overall results.These experiments and our results can help interaction designers for both virtual and augmented reality conditions.
When engaged in communication, people often rely on pointing gestures to refer to out-of-reach co... more When engaged in communication, people often rely on pointing gestures to refer to out-of-reach content. However, observers frequently misinterpret the target of a pointing gesture. Previous research suggests that to perform a pointing gesture, people place the index finger on or close to a line connecting the eye to the referent, while observers interpret pointing gestures by extrapolating the referent using a vector defined by the arm and index finger. In this paper we present Warping Deixis, a novel approach to improving the perception of pointing gestures and facilitate communication in collaborative Extended Reality environments. By warping the virtual representation of the pointing individual, we are able to match the pointing expression to the observer's perception. We evaluated our approach in a co-located side by side virtual reality scenario. Results suggest that our approach is effective in improving the interpretation of pointing gestures in shared virtual environments.
arXiv (Cornell University), Nov 29, 2019
New technologies allow ordinary people to access Virtual Reality at affordable prices in their ho... more New technologies allow ordinary people to access Virtual Reality at affordable prices in their homes. One of the most important tasks when interacting with immersive Virtual Reality is to navigate the virtual environments (VEs). Arguably, the best methods to accomplish this use direct control interfaces. Among those, natural walking (NW) makes for an enjoyable user experience. However, common techniques to support direct control interfaces in VEs feature constraints that make it difficult to use those methods in cramped home environments. Indeed, NW requires unobstructed and open space, to allow users to roam around without fear of stumbling on obstacles while immersed in a virtual world. To approach this problem, we propose a new virtual locomotion technique, which we call Combined Walking in Place (CWIP). CWIP allows people to take advantage of the available physical space and empowers them to use NW to navigate in the virtual world. For longer distances, we adopt Walking in Place (WIP) to enable them to move in the virtual world beyond the confines of a cramped real room. However, roaming in an immersive alternate reality, while moving in the confines of a cluttered environment can lead people to stumble and fall. To approach these problems we developed a technique called Augmented Virtual Reality (AVR), to inform users about real world hazards, such as chairs, drawers, walls via proxies and signs placed in the virtual world. We propose thus Combined Walking in Place in Augmented Virtual Reality (CWIP-AVR) as a way to safely explore VR in the cramped confines of your own home. To our knowledge, this is the first approach to combined different locomotion modalities in a safe manner. We assessed its effectiveness in a user study with 20 participants to validate their ability to navigate a virtual world while walking in a confined and cluttered real space. Our results show that CWIP-AVR allows people to navigate VR safely, while switching between locomotion modes flexibly and maintaining a good degree of immersion.
Modern volumetric projection-based telepresence approaches are capable of providing realistic ful... more Modern volumetric projection-based telepresence approaches are capable of providing realistic full-size virtual representations of remote people. Interacting with full-size people may not be desirable due to the spatial constraints of the physical environment, application context, or display technology. However, the miniaturization of remote people is known to create an eye gaze matching problem. Eye-contact is essential to communication as it allows for people to use natural nonverbal cues and improves the sense of “being there”. In this paper we discuss the design space for interacting with volumetric representations of people and present an approach for dynamically manipulating scale, orientation and the position of holograms which guarantees eye-contact. We created a working augmented reality-based prototype and validated it with 14 participants.
Proceedings of the 33rd Spring Conference on Computer Graphics
In Virtual Reality (VR), the action of selecting virtual objects outside arms-reach still poses s... more In Virtual Reality (VR), the action of selecting virtual objects outside arms-reach still poses significant challenges. In this work, after classifying, with a new taxonomy, and analyzing existing solutions, we propose a novel technique to perform out-of-reach selections in VR. It uses natural pointing gestures, a modifiable cone as selection volume, and an iterative progressive refinement strategy. This can be considered a VR implementation of a discrete zoom approach, although we modify users' position instead of the field-of-view. When the cone intersects several objects, users can either activate the refinement process, or trigger a multiple object selection. We compared our technique against two techniques from literature. Our results show that, although not being the fastest, it is a versatile approach due to the lack of errors and uniform completion times.
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
Laparoscopic surgery has the advantage of avoiding large open incisions and thereby decreasing bl... more Laparoscopic surgery has the advantage of avoiding large open incisions and thereby decreasing blood loss, pain, and discomfort to patients. However, on the other side, it is hampered by restricted workspace, ambiguous communication, and surgeon fatigue caused by non-ergonomic head positioning. We aimed to identify critical problems and suggest design requirements and solutions. We used user and task analysis methods to learn about practices performed in an operating room by observing surgeons in their working environment to understand how they performed tasks and achieved their intended goals. Drawing on observations and analysis from recorded laparoscopic surgeries, we have identified several constraints and design requirements to propose potential solutions to address the issues. Surgeons operate in a dimly lit environment, surrounded by monitors, and communicate through verbal commands and pointing gestures. Therefore, performing user and task analysis allowed us to understand the existing problems in laparoscopy better while identifying several communication constraints and design requirements, which a solution has to follow to address those problems. Our contributions include identifying design requirements for laparoscopy surgery through a user and task analysis. These requirements propose design solutions towards improved surgeons' comfort and make the surgical procedure less laborious.
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, 2019
Common natural walking techniques for navigating in virtual environments feature constraints that... more Common natural walking techniques for navigating in virtual environments feature constraints that make it difficult to use those methods in cramped home environments. Indeed, natural walking requires unobstructed and open space, to allow users to roam around without fear of stumbling on obstacles while immersed in a virtual world. In this work, we propose a new virtual locomotion technique, CWIP-AVR, that allows people to take advantage of the available physical space and empowers them to use natural walking to navigate in the virtual world. To inform users about real world hazards our approach uses augmented virtual reality visual indicators. A user evaluation suggests that CWIP-AVR allows people to navigate safely, while switching between locomotion modes flexibly and maintaining a adequate degree of immersion. CCS CONCEPTS • Human-centered computing → Computer supported cooperative work; Mixed / augmented reality.
Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, 2019
Face-to-face telepresence promotes the sense of "being there" and can improve collaboration by al... more Face-to-face telepresence promotes the sense of "being there" and can improve collaboration by allowing immediate understanding of remote people's nonverbal cues. Several approaches successfully explored interactions with 2D content using a see-through whiteboard metaphor. However, with 3D content, there is a decrease in awareness due to ambiguities originated by participants' opposing points-of-view. In this paper, we investigate how people and content should be presented for discussing 3D renderings within face-to-face collaborative sessions. To this end, we performed a user evaluation to compare four different conditions, in which we varied reflections of both workspace and remote people representation. Results suggest potentially more benefits to remote collaboration from workspace consistency rather than people's representation fidelity. We contribute a novel design space, the Negative Space, for remote face-to-face collaboration focusing on 3D content. CCS CONCEPTS • Human-centered computing → Computer supported cooperative work; Mixed / augmented reality.
2021 IEEE Virtual Reality and 3D User Interfaces (VR), 2021
Head-Mounted Virtual reality (VR) systems provide full-immersive experiences to users and complet... more Head-Mounted Virtual reality (VR) systems provide full-immersive experiences to users and completely isolate them from the outside world, placing them in unsafe situations. Existing research proposed different alert-based solutions to address this. Our work builds on these studies on notification systems for VR environments from a different perspective. We focus on: (i) exploring alert systems to notify VR users about non-immersed bystanders' in socially related, non-critical interaction contexts; (ii) understanding how best to provide awareness of non-immersed bystanders while maintaining presence and immersion within the Virtual Environment(VE). To this end, we developed single and combined alert cues-leveraging proxemics, perception channels, and push/pull approaches and evaluated those via two user studies. Our findings indicate a strong preference towards maintaining immersion and combining audio and visual cues, push and pull notification techniques that evolve dynamically based on proximity.
Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, 2017
Virtual Reality environments are able to o er natural interaction metaphors. However, it is di cu... more Virtual Reality environments are able to o er natural interaction metaphors. However, it is di cult to accurately place virtual objects in the desired position and orientation using gestures in mid-air. Previous research concluded that the separation of degrees-of-freedom (DOF) can lead to be er results, but these bene ts come with an increase in time when performing complex tasks, due to the additional number of transformations required. In this work, we assess whether custom transformation axes can be used to achieve the accuracy of DOF separation without sacri cing completion time. For this, we developed a new manipulation technique, MAiOR, which o ers translation and rotation separation, supporting both 3-DOF and 1-DOF manipulations, using personalized axes for the la er. Additionally, it also has direct 6-DOF manipulation for coarse transformations, and scaled object translation for increased placement. We compared MAiOR against an exclusively 6-DOF approach and a widget-based approach with explicit DOF separation. Results show that, contrary to previous research suggestions, single DOF manipulations are not appealing to users. Instead, users favored 3-DOF manipulations above all, while keeping translation and rotation independent. CCS CONCEPTS •Human-centered computing →Interaction techniques;
Journal of Biomedical Informatics, 2020
One of the most promising applications of Optical See-Through Augmented Reality is minimally lapa... more One of the most promising applications of Optical See-Through Augmented Reality is minimally laparoscopic surgery, which currently suffers from problems such as surgeon discomfort and fatigue caused by looking at a display positioned outside the surgeon's visual field, made worse by the length of the procedure. This fatigue is especially felt on the surgeon's neck, as it is strained from adopting unnatural postures in order to visualise the laparoscopic video feed. Throughout this paper, we will present work in Augmented Reality, as well as developments in surgery and Augmented Reality applied to both surgery in general and laparoscopy in particular to address these issues. We applied user and task analysis methods to learn about practices performed in the operating room by observing surgeons in their working environment in order to understand, in detail, how they performed their tasks and achieved their intended goals. Drawing on observations and analysis of video recordings of laparoscopic surgeries, we identified relevant constraints and design requirements. Besides proposals to approach the ergonomic issues, we present a design and implementation of a multimodal interface to enhance the laparoscopic procedure. Our method makes it more comfortable for surgeons by allowing them to keep the laparoscopic video in their viewing area regardless of neck posture. Also, our interface makes it possible to access patient imaging data without interrupting the operation. It also makes it possible to communicate with team members through a pointing reticle. We evaluated how surgeons perceived the implemented prototype, in terms of usefulness and usability, via a think-aloud protocol to conduct qualitative evaluation sessions which we describe in detail in this paper. In addition to checking the advantages of the prototype as compared to traditional laparoscopic settings, we also conducted a System Usability Scale questionnaire for measuring its usability, and a NASA Task Load Index questionnaire to rate perceived workload and to assess the prototype effectiveness. Our results show that surgeons consider that our prototype can improve surgeon-to-surgeon communication using head pose as a means of pointing. Also, surgeons believe that our approach can afford a more comfortable posture throughout the surgery and enhance hand-eye coordination, as physicians no longer need to twist their necks to look at screens placed outside the field of operation.
Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, 2017
Figure 1. Creepy Tracker is an open-source toolkit that provides spatial information about people... more Figure 1. Creepy Tracker is an open-source toolkit that provides spatial information about people and interactive surfaces. To do this, it resorts to multiple depth sensing cameras (A, B). It helps the design of systems that handle, for instance, (C) interactive tabletops, (D) vertical surfaces, (E) floor projections and even capture avatars for (F) telepresence or (G) virtual reality.
Computers & Graphics, 2019
3D reconstruction from anatomical slices allows anatomists to create three dimensional depictions... more 3D reconstruction from anatomical slices allows anatomists to create three dimensional depictions of real structures by tracing organs from sequences of cryosections. However, conventional user interfaces rely on single-user experiences and mouse-based input to create content for education or training purposes. In this work, we present Anatomy Studio, a collaborative Mixed Reality tool for virtual dissection that combines tablets with styli and see-through head-mounted displays to assist anatomists by easing manual tracing and exploring cryosection images. We contribute novel interaction techniques intended to promote spatial understanding and expedite manual segmentation. By using mid-air interactions and interactive surfaces, anatomists can easily access any cryosection and edit contours, while following other user's contributions. A user study including experienced anatomists and medical professionals, conducted in real working sessions, demonstrates that Anatomy Studio is appropriate and useful for 3D reconstruction. Results indicate that Anatomy Studio encourages closely-coupled collaborations and group discussion, to achieve deeper insights.
IEEE Transactions on Visualization and Computer Graphics, 2019
Locomotion in virtual environments is currently a difficult and unnatural task to perform. Normal... more Locomotion in virtual environments is currently a difficult and unnatural task to perform. Normally, researchers tend to devise ground-or floor-based metaphors to constrain the degrees of freedom (DoFs) during motion. These restrictions enable interactions that accurately emulate the human gait to provide high interaction fidelity. However, flying allows users to reach specific locations in a virtual scene more expeditiously. Our experience suggests that even though flying is not innate to humans, high-interaction-fidelity techniques may also improve the flying experience since flying requires simultaneously controlling additional DoFs. We present the Magic Carpet, an approach to flying that combines a floor proxy with a full-body representation to avoid balance and cybersickness issues. This design space enables DoF separation by treating direction indication and speed control as two separate phases of travel, thereby enabling techniques with higher interaction fidelity. To validate our design space, we conducted two complementary studies, one for each of the travel phases. In this paper, we present the results of both studies and report the best techniques for use within the Magic Carpet design space. To this end, we use both objective and subjective measures to evaluate the efficiency, embodiment effect, and side effects, such as physical fatigue and cybersickness, of the tested techniques in our design space. Our results show that the proposed approach enables high-interaction-fidelity techniques while improving the user experience.
Computers & Graphics, 2017
In interactive systems, the ability to select virtual objects is essential. In immersive virtual ... more In interactive systems, the ability to select virtual objects is essential. In immersive virtual environments, object selection is usually done at arm's length in mid-air by directly intersecting the desired object with the user's hand. However, selecting objects outside user's arm-reach still poses significant challenges, which direct approaches fail to address. Techniques proposed to overcome such limitations often follow an arm-extension metaphor or favor selection volumes combined with ray-casting. Nonetheless, while these approaches work for room sized environments, they hardly scale up to larger scenarios with many objects. In this paper, we introduce a new taxonomy to classify existing selection techniques. In its wake, we propose PRECIOUS, a novel mid-air technique for selecting out-of-reach objects, featuring iterative refinement in Virtual Reality, an hitherto untried approach in this context. While comparable techniques have been developed for non-stereo and non-immersive environments, these are not suitable to Immersive Virtual Reality. Our technique is the first to employ an iterative progressive refinement in such settings. It uses cone-casting to select multiple objects and moves the user closer to them in each refinement step, to allow accurate selection of the desired target. A user evaluation showed that PRECIOUS compares favorably against state-of-the-art approaches. Indeed, our results indicate that PRECIOUS is a versatile approach to out-of-reach target acquisition, combining accurate selection with consistent task completion times across different scenarios.
2017 IEEE Symposium on 3D User Interfaces (3DUI), 2017
Virtual Reality (VR) is again in the spotlight. However, interactions and modeling operations are... more Virtual Reality (VR) is again in the spotlight. However, interactions and modeling operations are still major hurdles to its complete success. To make VR Interaction viable, many have proposed mid-air techniques because of their naturalness and resemblance to physical world operations. Still, natural mid-air metaphors for Constructive Solid Geometry (CSG) are still elusive. This is unfortunate, because CSG is a powerful enabler for more complex modeling tasks, allowing to create complex objects from simple ones via Boolean operations. Moreover, Head-Mounted Displays occlude the real self, and make it difficult for users to be aware of their relationship to the virtual environment. In this paper we propose two new techniques to achieve Boolean operations between two objects in VR. One is based on direct-manipulation via gestures while the other uses menus. We conducted a preliminary evaluation of these techniques. Due to tracking limitations, results allowed no significant conclusions to be drawn. To account for self-representation, we compared full-body avatar against an iconic cursor depiction of users' hands. In this matter, the simplified hands-only representation improved efficiency in CSG modelling tasks.
Uploads
Papers by Maurício Sousa
radiodiagnostics by considerably diminishing or canceling out the effects of unsuitable ambient conditions. Our approach combines immersive head-mounted displays with interactive surfaces to
support professional radiologists in analyzing medical images and formulating diagnostics. We evaluated our prototype with two senior medical doctors and four seasoned radiology fellows.
Results indicate that our approach constitutes a viable, flexible, portable and cost-efficient option to traditional radiology reading rooms.