ABSTRACT More and more recent studies of emotion estimation have taken the approach of ambient se... more ABSTRACT More and more recent studies of emotion estimation have taken the approach of ambient sensing, where users are monitored without constraining their bodies and without being aware of the sensing system. Few of them, however, focus on audience emotions. We attempted to examine audience emotions, which are difficult to estimate when using only individual data, and which come to be estimated more easily when using ambient sensing of synchrony among multiple users. We developed an ambient sensing chair, which employs wireless sensors to determine the center of force of the seated user, and designed an experiment for an audience watching a soccer game on TV or a technical presentation in order to examine the relationship between audience body sway and emotional states in a natural situation. We found that synchrony could be well correlated with audience excitement and boredom, something not observable in the individual data. These results indicate the possibility of development of service marketing systems that evaluate TV content, and feedback systems that give notice of the emotional state of an audience to a presenter.
Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, 2013
ABSTRACT The authors aim to develop a community detection system without constraining users. This... more ABSTRACT The authors aim to develop a community detection system without constraining users. This paper introduces a method to detect communities in which users are engaging in a conversation with each other, by an ambient sensing of body sways and their synchrony between users. We apply an improved ambient sensing chair, which had been developed in our previous study. The system employs wireless force sensors to measure the center of pressure. By using measured speaker/listener information and synchrony between any users' body sways, it is confirmed to detect the community information in experiment.
The 1st IEEE Global Conference on Consumer Electronics 2012, 2012
ABSTRACT The authors aimed to develop a practical system to recognize group emotion without const... more ABSTRACT The authors aimed to develop a practical system to recognize group emotion without constraining users. This paper introduces a method to recognize the emotion of an audience attending a presentation by the ambient sensing of body sways or synchrony of multiple users. We developed ambient sensing chair, which employs wireless sensors to determine the center of force of seated users and a methodology for recognizing individual and group-acting-out behaviors. We designed an experiment to examine the relationship between audience body sways and emotions in a natural situation. Subjects watched five technical presentations, each of which consisted of 20 slides that switched automatically every 20 [s] and responded to questionnaires about their subjective impressions of each slide in each presentation. As a result of comparing subjects' acting-out behaviors and their synchrony to subjective impressions, synchrony could be well correlated to boredom. Our system was able to identify boredom at a recognition rate of more than 80%.
ABSTRACT This paper presents a shadow removal technique for a multiple overlapping projection sys... more ABSTRACT This paper presents a shadow removal technique for a multiple overlapping projection system. In particular, this paper deals with situations where cameras cannot be placed between the occluder and projection surface. We apply a synthetic aperture capturing technique to estimate the appearance of the projection surface, and a visual hull reconstruction technique to measure the shape of the occluder. Once the shape is acquired, shadow regions on the surface can be estimated. The proposed shadow removal technique allows users to balance between the following two criteria: the likelihood of new shadow emergence and the spatial resolution of the projected results. Through a real projection experiment, we evaluate the proposed shadow removal technique
Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '11, 2011
We propose the concept and implementation of a graphical information hiding technique for interac... more We propose the concept and implementation of a graphical information hiding technique for interactive tabletops where users can view the information by simply casting real shadows. We placed three projectors (one in the rear and two in the front) in such a way that the rear one projects graphical information onto a tabletop surface, and the front ones project a complementary image, so that the combined image displayed on the surface becomes uniformly gray, thus hiding the information from the viewer. Users can view the hidden information by blocking the light from the front projector, revealing the complementary image that is being projected onto the occluder. We use the other front projector and polarization filters to make the complementary image projected onto the occluder also uniformly gray. Because the technique completely relies on optical phenomena, users can interact with the system without suffering from any false recognitions or delays.
Proceedings of the ACM symposium on Virtual reality software and technology - VRST '06, 2006
We propose Limpid Desk which supports document search on a real desktop with virtual transparenti... more We propose Limpid Desk which supports document search on a real desktop with virtual transparentizing of the upper layer of a document stack in projection-based mixed reality (MR) environments. In the system, users can visually access a lower layer document without physically removing the upper documents. This is accomplished by projecting a special pattern of light that is calculated to compensate the appearances of the upper layer documents as if they are transparent. In addition, we propose two types of visual effects for users to cognize the progress of the transparentizing of real documents and to recognize the layer number of the virtually exposing document, and excecute psychological tests to confirm the intuitiveness of these effects. This paper also presents an intuitive document search interaction in the proposed system.
In this paper, we present an image-based 3D posi-tioning method by using produced motion blur for... more In this paper, we present an image-based 3D posi-tioning method by using produced motion blur for a novel camera-based indoor navigation system. This sys-tem, avoiding adopting any expensive equipment, could provide exact 3D location and orientation for mobile users. High-intensity LED tube, which is expected to be the next generation of illumination source, is used as optical marker in this system. LED tubes are modulated to transmit ID information and the message is received by handy terminal which is equipped with one normal camera. The exposure time of a common camera is usually longer than flicker period of LED lamps, and this makes difficulties for ID recognition. We study a method to solve this problem. If camera is shaken by user's hand while the period of exposure, on captured image a streaked pattern would be produced and that could be used for retrieving ID number. Result of experiment shows this method is feasible.
We propose "Limpid Desk" in which users can visually access to the lower layer of real ... more We propose "Limpid Desk" in which users can visually access to the lower layer of real documents multiply stacked on a real desktop using projection based mixed reality. Special pattern light which is photometrically compensated as if the upper documents would be translucent is projected to the stacked documents which users touch. We also propose a touch sensor, as an interface of the proposed system, witch can recognize the users' touch regions on the real objects, without any tools witch users ware and hold, using thermal (far infrared) image. Like PC desktop graphics, in the proposed system, users can realize the lower layer of documents without physically moving the upper documents on the real desktop. In the proposed system, since the stack space, the manipulating space, and the display space are completely unified, users can directly access any documents without PC interface. We claim that this kind of consistency of the manipulation is the key for realizing the ...
Searching of spread documents on a real desk is an arduous task. The chaos on the desk makes user... more Searching of spread documents on a real desk is an arduous task. The chaos on the desk makes users searching of the desired documents difficult. In this paper, we propose Limpid Desk which supports a document search on a desk by transparentizing the upper layer of a document stack in projection-based mixed reality environments. In the system, the special pattern light which is calculated to compensate the appearances of the upper layer documents as if they are transparent is projected to the stack, and as a result, users can visually access to the lower layer document. We propose a touch sensing method using a thermal image for the input interface of the system. The method realizes that users touch areas on real documents can be detected with no worn or hold devices. Hence, users can intuitively select the stack, which they would like to transparentize, by their simple touch gestures. We present three intuitive interaction techniques which allow users to figure out what the lower ...
IEEE Transactions on Circuits and Systems for Video Technology
This paper presents a novel technique to reduce the motion blur artifacts of foveal projection in... more This paper presents a novel technique to reduce the motion blur artifacts of foveal projection in a dynamic focus-plus-context (DF+C) display. The DF+C display is generally configured with multiple projectors and provides a nonuniform spatial resolution that consists of high-resolution (hi-res) regions (foveal projection) and low-resolution regions (peripheral projection). A serious problem of the DF+C display is motion blur, which inevitably occurs when a foveal projection is moved by a pan-tilt mirror or gantry. We propose a solution that reduces the motion blur artifacts, and evaluate how this solution improves the image quality using both qualitative and quantitative experiments. Our proposed method defines an error function to assess the displayed image quality as the difference between an original hi-res image and the displayed image by considering the nonuniform spatial property of human visual acuity. Then, it decides the set of positions and moving techniques of foveal proj...
IEEE Transactions on Visualization and Computer Graphics
A simple and cost-efficient method for extending a projector's depth-of-field (DOF) is propos... more A simple and cost-efficient method for extending a projector's depth-of-field (DOF) is proposed. By leveraging liquid lens technology, we can periodically modulate the focal length of a projector at a frequency that is higher than the critical flicker fusion (CFF) frequency. Fast periodic focal length modulation results in forward and backward sweeping of focusing distance. Fast focal sweep projection makes the point spread function (PSF) of each projected pixel integrated over a sweep period (IPSF; integrated PSF) nearly invariant to the distance from the projector to the projection surface as long as it is positioned within sweep range. This modulation is not perceivable by human observers. Once we compensate projection images for the IPSF, the projected results can be focused at any point within the range. Consequently, the proposed method requires only a single offline PSF measurement; thus, it is an open-loop process. We have proved the approximate invariance of the project...
Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology - ACE '05, 2005
We introduce how to involve the heat sensation in image creation by using thermal vision. We deve... more We introduce how to involve the heat sensation in image creation by using thermal vision. We develop "ThermoTablet" which can detect touch regions of physical input objects on a sensing surface using changes of a temperature distribution of the surface when those objects are hotter or colder than the surface. Image creation applications of painting, image modifications are implemented on it. In the painting application, users can directly use physical paintbrushes and airbrushes with hot water in spite of paint, and even use their own fingers, hands, and breaths directly. "ThermoInk" transforms the temperature information into a variation of painting. Image modification application aims an intuitive user interface for color saturation modification, hue modification, shape modification and CAD model deformation which are strongly related to heat sensations.
Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology - ACE '10, 2010
We propose a novel video projection system, the Snail Light Projector, where light emitted from a... more We propose a novel video projection system, the Snail Light Projector, where light emitted from a video projector travels at virtual hyper-slow speed. We define hyper-slow light as the light that creates a number of spatio-temporal video containing images with emitting time that differ between the video projector lens and the projection screen. As a result, the user can browse the images by moving a projection screen, and control the emitted light by moving a handheld projector. We believe the Snail Light Projector can be an innovative entertainment system, achieving interaction by using hyper-slow speed light, which enables the user to experience virtual physical phenomena directly.
ABSTRACT More and more recent studies of emotion estimation have taken the approach of ambient se... more ABSTRACT More and more recent studies of emotion estimation have taken the approach of ambient sensing, where users are monitored without constraining their bodies and without being aware of the sensing system. Few of them, however, focus on audience emotions. We attempted to examine audience emotions, which are difficult to estimate when using only individual data, and which come to be estimated more easily when using ambient sensing of synchrony among multiple users. We developed an ambient sensing chair, which employs wireless sensors to determine the center of force of the seated user, and designed an experiment for an audience watching a soccer game on TV or a technical presentation in order to examine the relationship between audience body sway and emotional states in a natural situation. We found that synchrony could be well correlated with audience excitement and boredom, something not observable in the individual data. These results indicate the possibility of development of service marketing systems that evaluate TV content, and feedback systems that give notice of the emotional state of an audience to a presenter.
Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, 2013
ABSTRACT The authors aim to develop a community detection system without constraining users. This... more ABSTRACT The authors aim to develop a community detection system without constraining users. This paper introduces a method to detect communities in which users are engaging in a conversation with each other, by an ambient sensing of body sways and their synchrony between users. We apply an improved ambient sensing chair, which had been developed in our previous study. The system employs wireless force sensors to measure the center of pressure. By using measured speaker/listener information and synchrony between any users' body sways, it is confirmed to detect the community information in experiment.
The 1st IEEE Global Conference on Consumer Electronics 2012, 2012
ABSTRACT The authors aimed to develop a practical system to recognize group emotion without const... more ABSTRACT The authors aimed to develop a practical system to recognize group emotion without constraining users. This paper introduces a method to recognize the emotion of an audience attending a presentation by the ambient sensing of body sways or synchrony of multiple users. We developed ambient sensing chair, which employs wireless sensors to determine the center of force of seated users and a methodology for recognizing individual and group-acting-out behaviors. We designed an experiment to examine the relationship between audience body sways and emotions in a natural situation. Subjects watched five technical presentations, each of which consisted of 20 slides that switched automatically every 20 [s] and responded to questionnaires about their subjective impressions of each slide in each presentation. As a result of comparing subjects' acting-out behaviors and their synchrony to subjective impressions, synchrony could be well correlated to boredom. Our system was able to identify boredom at a recognition rate of more than 80%.
ABSTRACT This paper presents a shadow removal technique for a multiple overlapping projection sys... more ABSTRACT This paper presents a shadow removal technique for a multiple overlapping projection system. In particular, this paper deals with situations where cameras cannot be placed between the occluder and projection surface. We apply a synthetic aperture capturing technique to estimate the appearance of the projection surface, and a visual hull reconstruction technique to measure the shape of the occluder. Once the shape is acquired, shadow regions on the surface can be estimated. The proposed shadow removal technique allows users to balance between the following two criteria: the likelihood of new shadow emergence and the spatial resolution of the projected results. Through a real projection experiment, we evaluate the proposed shadow removal technique
Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '11, 2011
We propose the concept and implementation of a graphical information hiding technique for interac... more We propose the concept and implementation of a graphical information hiding technique for interactive tabletops where users can view the information by simply casting real shadows. We placed three projectors (one in the rear and two in the front) in such a way that the rear one projects graphical information onto a tabletop surface, and the front ones project a complementary image, so that the combined image displayed on the surface becomes uniformly gray, thus hiding the information from the viewer. Users can view the hidden information by blocking the light from the front projector, revealing the complementary image that is being projected onto the occluder. We use the other front projector and polarization filters to make the complementary image projected onto the occluder also uniformly gray. Because the technique completely relies on optical phenomena, users can interact with the system without suffering from any false recognitions or delays.
Proceedings of the ACM symposium on Virtual reality software and technology - VRST '06, 2006
We propose Limpid Desk which supports document search on a real desktop with virtual transparenti... more We propose Limpid Desk which supports document search on a real desktop with virtual transparentizing of the upper layer of a document stack in projection-based mixed reality (MR) environments. In the system, users can visually access a lower layer document without physically removing the upper documents. This is accomplished by projecting a special pattern of light that is calculated to compensate the appearances of the upper layer documents as if they are transparent. In addition, we propose two types of visual effects for users to cognize the progress of the transparentizing of real documents and to recognize the layer number of the virtually exposing document, and excecute psychological tests to confirm the intuitiveness of these effects. This paper also presents an intuitive document search interaction in the proposed system.
In this paper, we present an image-based 3D posi-tioning method by using produced motion blur for... more In this paper, we present an image-based 3D posi-tioning method by using produced motion blur for a novel camera-based indoor navigation system. This sys-tem, avoiding adopting any expensive equipment, could provide exact 3D location and orientation for mobile users. High-intensity LED tube, which is expected to be the next generation of illumination source, is used as optical marker in this system. LED tubes are modulated to transmit ID information and the message is received by handy terminal which is equipped with one normal camera. The exposure time of a common camera is usually longer than flicker period of LED lamps, and this makes difficulties for ID recognition. We study a method to solve this problem. If camera is shaken by user's hand while the period of exposure, on captured image a streaked pattern would be produced and that could be used for retrieving ID number. Result of experiment shows this method is feasible.
We propose "Limpid Desk" in which users can visually access to the lower layer of real ... more We propose "Limpid Desk" in which users can visually access to the lower layer of real documents multiply stacked on a real desktop using projection based mixed reality. Special pattern light which is photometrically compensated as if the upper documents would be translucent is projected to the stacked documents which users touch. We also propose a touch sensor, as an interface of the proposed system, witch can recognize the users' touch regions on the real objects, without any tools witch users ware and hold, using thermal (far infrared) image. Like PC desktop graphics, in the proposed system, users can realize the lower layer of documents without physically moving the upper documents on the real desktop. In the proposed system, since the stack space, the manipulating space, and the display space are completely unified, users can directly access any documents without PC interface. We claim that this kind of consistency of the manipulation is the key for realizing the ...
Searching of spread documents on a real desk is an arduous task. The chaos on the desk makes user... more Searching of spread documents on a real desk is an arduous task. The chaos on the desk makes users searching of the desired documents difficult. In this paper, we propose Limpid Desk which supports a document search on a desk by transparentizing the upper layer of a document stack in projection-based mixed reality environments. In the system, the special pattern light which is calculated to compensate the appearances of the upper layer documents as if they are transparent is projected to the stack, and as a result, users can visually access to the lower layer document. We propose a touch sensing method using a thermal image for the input interface of the system. The method realizes that users touch areas on real documents can be detected with no worn or hold devices. Hence, users can intuitively select the stack, which they would like to transparentize, by their simple touch gestures. We present three intuitive interaction techniques which allow users to figure out what the lower ...
IEEE Transactions on Circuits and Systems for Video Technology
This paper presents a novel technique to reduce the motion blur artifacts of foveal projection in... more This paper presents a novel technique to reduce the motion blur artifacts of foveal projection in a dynamic focus-plus-context (DF+C) display. The DF+C display is generally configured with multiple projectors and provides a nonuniform spatial resolution that consists of high-resolution (hi-res) regions (foveal projection) and low-resolution regions (peripheral projection). A serious problem of the DF+C display is motion blur, which inevitably occurs when a foveal projection is moved by a pan-tilt mirror or gantry. We propose a solution that reduces the motion blur artifacts, and evaluate how this solution improves the image quality using both qualitative and quantitative experiments. Our proposed method defines an error function to assess the displayed image quality as the difference between an original hi-res image and the displayed image by considering the nonuniform spatial property of human visual acuity. Then, it decides the set of positions and moving techniques of foveal proj...
IEEE Transactions on Visualization and Computer Graphics
A simple and cost-efficient method for extending a projector's depth-of-field (DOF) is propos... more A simple and cost-efficient method for extending a projector's depth-of-field (DOF) is proposed. By leveraging liquid lens technology, we can periodically modulate the focal length of a projector at a frequency that is higher than the critical flicker fusion (CFF) frequency. Fast periodic focal length modulation results in forward and backward sweeping of focusing distance. Fast focal sweep projection makes the point spread function (PSF) of each projected pixel integrated over a sweep period (IPSF; integrated PSF) nearly invariant to the distance from the projector to the projection surface as long as it is positioned within sweep range. This modulation is not perceivable by human observers. Once we compensate projection images for the IPSF, the projected results can be focused at any point within the range. Consequently, the proposed method requires only a single offline PSF measurement; thus, it is an open-loop process. We have proved the approximate invariance of the project...
Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology - ACE '05, 2005
We introduce how to involve the heat sensation in image creation by using thermal vision. We deve... more We introduce how to involve the heat sensation in image creation by using thermal vision. We develop "ThermoTablet" which can detect touch regions of physical input objects on a sensing surface using changes of a temperature distribution of the surface when those objects are hotter or colder than the surface. Image creation applications of painting, image modifications are implemented on it. In the painting application, users can directly use physical paintbrushes and airbrushes with hot water in spite of paint, and even use their own fingers, hands, and breaths directly. "ThermoInk" transforms the temperature information into a variation of painting. Image modification application aims an intuitive user interface for color saturation modification, hue modification, shape modification and CAD model deformation which are strongly related to heat sensations.
Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology - ACE '10, 2010
We propose a novel video projection system, the Snail Light Projector, where light emitted from a... more We propose a novel video projection system, the Snail Light Projector, where light emitted from a video projector travels at virtual hyper-slow speed. We define hyper-slow light as the light that creates a number of spatio-temporal video containing images with emitting time that differ between the video projector lens and the projection screen. As a result, the user can browse the images by moving a projection screen, and control the emitted light by moving a handheld projector. We believe the Snail Light Projector can be an innovative entertainment system, achieving interaction by using hyper-slow speed light, which enables the user to experience virtual physical phenomena directly.
Uploads
Papers by Daisuke Iwai