Papers by Roberto F U S C O Pugliese
IEEE MultiMedia, 2015
ABSTRACT In this paper we examine the influence of auditory augmentation on the act of jumping on... more ABSTRACT In this paper we examine the influence of auditory augmentation on the act of jumping on an elastic trampoline. To do so we have developed a system that interactively augments the sound of a trampoline when a user is jumping on it. The sound design is inspired by iconic jumping sounds from games and the synthesis engine allows for parametric control of the sound features. The sensing technology is based on a combination of motion tracking with depth camera and audio-based contact sensing between the feet and the trampoline. The system lets the user interactively control the sound by jumping. We conducted a study to evaluate the effect of manipulating the auditory feedback during the jumping exercises. Results show that our interactive sonification positively affects the user experience during the exercise and stimulates changes in the user's behavior toward increased performance. Our system and study provide evidence that interactive sonification can act as a motivational tool in training and add an extra fun-factor to body-controlled games.
Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '14, 2015
ABSTRACT This paper presents ATSI, a system for sonic augmentation of physical objects with spati... more ABSTRACT This paper presents ATSI, a system for sonic augmentation of physical objects with spatialized sounds and their control by gestural interaction. The implementation combines tracking of the users’ hands and head with commodity hardware, and binaural spatialized sound rendering over headphones. The user may attach sounds to common objects and the system maintains the correct spatial auditory perspective inside the augmented scene during the actions of attaching sounds, pick- ing and moving the sounding objects, or exploring the scene. Attention has been given to the binaural rendering of distance cues to support the above actions with perceptual realism in small scale environments and for many objects. Through a user study assessing the localization accuracy that can be achieved with the system, we show that sound rendering per- formance looks appropriate for applications such as auditory displays for context and object specific information, sonic de- sign for architectural planning and interior design, and music applications.
To investigate how auditory augmentation influences user jumping activity on an elastic trampolin... more To investigate how auditory augmentation influences user jumping activity on an elastic trampoline, the authors developed a system to interactively augment the trampoline's sound. The system's sound design is inspired by iconic jumping sounds from games, with sensing technology based on a combination of motion tracking with depth camera and audio-based contact sensing between the feet and trampoline. Results show that such interactive sonification stimulates changes in the user's behavior toward increased performance during the exercise; it also positively affects the user experience.
In this paper we describe a motion-controlled game based on a paradigm of a player enacting the c... more In this paper we describe a motion-controlled game based on a paradigm of a player enacting the character, rather than a character mimicking the player’s action. Our hypothesis is that a controlling scheme based on the adaptation of a player to the way the avatar is able to perform actions, can result in a stronger presence and psychological bond to the character. The approach is based on previous studies showing that features, attitudes and behaviors of the digital representation of players in a virtual reality setting, alter the players self-perception in the virtual environment (Proteus effect). The interaction mechanism is inspired by enactive approach to cognition and embodied action. In a mini-game we explore effects of controlling a weak character on self-presence and identification with the avatar. We show that increasing degrees of effort in the controlled bodies resulted in different impressions of the physical state of the character. Additionally, we provide our interpretation of relation between game experience and the kinetic parameters and adaptation indicators extracted from the motion of the player and avatar. Finally, we address scenarios where this enaction-based approach to motion controlled avatar can find application.
This paper presents ATSI, a system for sonic augmentation of physical objects with spatialized so... more This paper presents ATSI, a system for sonic augmentation of physical objects with spatialized sounds and their control by gestural interaction. The implementation combines tracking of the users' hands and head with commodity hardware, and binaural spatialized sound rendering over headphones. The user may attach sounds to common objects and the system maintains the correct spatial auditory perspective inside the augmented scene during the actions of attaching sounds, picking and moving the sounding objects, or exploring the scene. Attention has been given to the binaural rendering of distance cues to support the above actions with perceptual realism in small scale environments and for many objects. Through a user study assessing the localization accuracy that can be achieved with the system, we show that sound rendering performance looks appropriate for applications such as auditory displays for context and object specific information, sonic design for architectural planning and interior design, and music applications.
eecs.umich.edu
In this paper strategies for augmenting the social dimension of collaborative music making, in pa... more In this paper strategies for augmenting the social dimension of collaborative music making, in particular in the form of bodily and situated interaction are presented. Mobile instruments are extended by means of relational descriptors democratically controlled by the group and mapped to sound parameters. A qualitative evaluation approach is described and a user test with participants playing in groups of three conducted. The results of the analysis show corecategories such as familiarity with instrument and situation, shift of focus in activity , family of interactions and different categories of the experience emerging from the interviews. Our evaluation shows the suitability of our approach but also the need for iterating on our design on the basis of the perspectives brought forth by the users. This latter observation confirms the importance of conducting a thorough interview session followed by data analysis on the line of grounded theory.
In this paper an integrated system for the creation of a combined audio and tactile display is de... more In this paper an integrated system for the creation of a combined audio and tactile display is described. In order to get an illusion of physically being among virtual sounding objects, we used vibration motors attached to a belt to give tactile stimulus, and sensed the user’s position and orientation with a 3D tracker. Collisions with free-to-move virtual objects is provided through semi-realistic vibration on the correct collision point with respect to the position and orientation of the user. The tactile vibration is encoded on 8 vibrotactile motors using a calculation of the gains on the motors similar to a panning law and improved to convey the perceptual illusion of proximity of the object and collision with it. We combine the tactile stimulus with a spatialization system augmented with distance cues. As a case scenario, simpleLife, an immersive audio-tactile installation for one participant, inspired by the concept of performance ecosystems and ecological approaches to musical interaction is shown.
This paper describes a system to interactively sonify the foot-floor contacts resulting from jump... more This paper describes a system to interactively sonify the foot-floor contacts resulting from jumping on an elastic trampoline. The sonification was achieved by means of a synthesis engine based on physical models reproducing the sounds of jumping on several surface materials. The engine was controlled in real-time by processing the signal captured by a contact microphone which was attached to the membrane of the trampoline in order to detect each jump. A user study was conducted to evaluate the quality of the interactive sonification. Results proved the success of the proposed algorithms and their control. In addition, results provided indications that the proposed auditory feedback can modulate the perception of the foot-haptic sensations of the surface utilized when jumping. The system can find application in augmented reality contexts for sport and entertainment, and is suitable for studies on multi-sensory perception involving the auditory and the foothaptic modalities.
We explore motion capture as a means for generating expressive bodily interaction between humans ... more We explore motion capture as a means for generating expressive bodily interaction between humans and virtual characters. Recorded interactions between humans are used as examples from which rules are formed that control reactions of a virtual character to human actions. The author of the rules selects segments considered important and features that best describe the desired interaction.
We propose a novel methodology for authoring interactive behaviors of virtual characters. Our app... more We propose a novel methodology for authoring interactive behaviors of virtual characters. Our approach is based on enaction, which means a continuous two-directional loop of bodily interaction. We have implemented the case of two characters, one human and one virtual, who are separated by a glass wall and can interact only through bodily motions. Animations for the virtual character are based on captured motion segments and descriptors for the style of motions that are automatically calculated from the motion data.
We introduce Reality-based User Interface System (RUIS), a platform that even novice programmers ... more We introduce Reality-based User Interface System (RUIS), a platform that even novice programmers can easily use for prototyping novel user interfaces. We implemented a puzzle application with a 3D user interface employing modified Nintendo Wii Remotes as 6-DOF controllers to showcase some of the features that are possible with our platform: two-handed gestural interaction, natural interaction metaphors, and haptic feedback.
-Elliptically-shaped bow-tie antenna is the generator and the receiver circuit (VSWR should be op... more -Elliptically-shaped bow-tie antenna is the generator and the receiver circuit (VSWR should be optimized to satisfy demands of future U-impulse radio less than 2 within the whole operational hand). Finally. system After numerid optimization the antenna exbibits the antenna might he mounted on a dielectric substrate, acceptable performance in terms of the radiated waveform and to the generator, receiver, of the which will Serve also as a protective mechanical shield, optimized antenna has been verified experimentally.
In our previous publications we have introduced a method for applying a combination of resistive ... more In our previous publications we have introduced a method for applying a combination of resistive (R) and capacitive (C) loading on a circular-end bow-tie antenna to considerably improve the antenna bandwidth [1]–[4]. The application of the RC loading allows the antenna to transmit ultra-wideband pulses with relatively high radiation efficiency and very small late-time ringing, suitable for impulse ground penetrating radars for detection of small shallowly-buried objects.
ABSTRACT This paper proposes the concept of enactive system as an alternative to the standard hum... more ABSTRACT This paper proposes the concept of enactive system as an alternative to the standard human-computer system and elaborates the idea of content mediation as enactive media. The authors' system consists of the two elements coupled in a holistic manner by means of bodily and spatial involvement, or enactment. Such a system is recursive by nature, involving the impact of the technology on the human agent as well as the effect of the human experience on the technology.
[2] Ultrawideband (UWB) communication is a promising technology that is expected to solve the pro... more [2] Ultrawideband (UWB) communication is a promising technology that is expected to solve the problem of the shortage of the available frequency bands in wireless, short-distance networks. It is characterized by a potentially high capacity of the channel, low spectral density of the transmitted power (and thus low energy consumption), and high immunity against electromagnetic (EM) interference and fading robustness [Roy et al., 2004].
ABSTRACT In this paper strategies for augmenting the social dimension of collaborative music maki... more ABSTRACT In this paper strategies for augmenting the social dimension of collaborative music making, in particular in the form of bodily and situated interaction are presented. Mobile instruments are extended by means of relational descriptors democratically controlled by the group and mapped to sound parameters. A qualitative evaluation approach is described and a user test with participants playing in groups of three conducted.
Uploads
Papers by Roberto F U S C O Pugliese