Dynamic shadow removal from front projection displays

IEEE Transactions on Visualization and Computer Graphics, 2004

Front-projection displays are a cost-effective and increasingly popular method for large format visualization and immersive rendering of virtual models. New approaches to projector tiling, automatic calibration, and color balancing have made multiprojector display systems feasible without undue infrastructure changes and maintenance. As a result, front-projection displays are being used to generate seamless, visually immersive worlds for virtual reality and visualization applications with reasonable cost and maintenance overhead. However, these systems suffer from a fundamental problem: Users and other objects in the environment can easily and inadvertently block projectors, creating shadows on the displayed image. Shadows occlude potentially important information and detract from the sense of presence an immersive display may have conveyed. We introduce a technique that detects and corrects shadows in a multiprojector display while it is in use. Cameras observe the display and compare observations with an expected image to detect shadowed regions. These regions are transformed to the appropriate projector frames, where corresponding pixel values are increased and/or attenuated. In display regions where more than one projector contributes to the image, shadow regions are eliminated.

Front-projectiondisplay systems suffer from a fundamental problem: the user can easily and inadvertently block the projector, creating shadows. The framework we present in this paper enables a front-projected display system to solve this problem,and remove,display surface shadows,in real-time. Shadows,are an example of radiometric inconsistencies,which cause a number of challenges in building large display systems. Other radiometric anomalies arise from reflectance variations across the display surface, inconsistencies between projectors, and inter-reflections from the display surfaces themselves. We present a new automated,technique for dynamically,creating a radiometricall y constant image on the display surface. Our technique detects and corrects for radiometric anomalies as transient as shadows, accommodates variations in the reflectance characteristics of the display surface, and can address the display-surface inter-reflection problem. Using an automatically-derived relative p...

Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001

A major problem with interactive displays based on frontprojection is that users cast undesirable shadows on the display surface. This situation is only partially-addressed by mounting a single projector at an extreme angle and prewarping the projected image to undo keystoning distortions. This paper demonstrates that shadows can be muted by redundantly-illuminating the display surface using multiple projectors, all mounted at different locations. However, this technique alone does not eliminate shadows: multiple projectors create multiple dark regions on the surface (penumbral occlusions). We solve the problem by using cameras to automatically identify occlusions as they occur and dynamically adjust each projector's output so that additional light is projected onto each partially-occluded patch. The system is self-calibrating: relevant homographies relating projectors, cameras and the display surface are recovered by observing the distortions induced in projected calibration patterns. The resulting redundantly-projected display retains the high image quality of a single-projector system while dynamically correcting for all penumbral occlusions. Our initial two-projector implementation operates at 3 Hz.

2001

A major problem with interactive displays based on frontprojection is that users cast undesirable shadows on the display surface. This situation is only partially-addressed by mounting a single projector at an extreme angle and prewarping the projected image to undo keystoning distortions. This paper demonstrates that shadows can be muted by redundantly-illuminating the display surface using multiple projectors, all mounted at different locations. However, this technique alone does not eliminate shadows: multiple projectors create multiple dark regions on the surface (penumbral occlusions). We solve the problem by using cameras to automatically identify occlusions as they occur and dynamically adjust each projector's output so that additional light is projected onto each partially-occluded patch. The system is self-calibrating: relevant homographies relating projectors, cameras and the display surface are recovered by observing the distortions induced in projected calibration patterns. The resulting redundantly-projected display retains the high image quality of a single-projector system while dynamically correcting for all penumbral occlusions. Our initial two-projector implementation operates at 3 Hz.

2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., 2003

Two related problems of front projection displays which occur when users obscure a projector are: (i) undesirable shadows cast on the display by the users, and (ii) projected light falling on and distracting the users. This paper provides a computational framework for solving these two problems based on multiple overlapping projectors and cameras. The overlapping projectors are automatically aligned to display the same dekeystoned image. The system detects when and where shadows are cast by occluders and is able to determine the pixels which are occluded in different projectors. Through a feedback control loop, the contributions of unoccluded pixels from other projectors are boosted in the shadowed regions, thereby eliminating the shadows. In addition, pixels which are being occluded are blanked, thereby preventing the projected light from falling on a user when they occlude the display. This can be accomplished even when the occluders are not visible to the camera. The paper presents results from a number of experiments demonstrating that the system converges rapidly with low steady-state errors.

Computer Graphics Forum, 2009

Display devices, more than ever, are finding their ways into electronic consumer goods as a result of recent trends in providing more functionality and user interaction. Combined with the new developments in display technology towards higher reproducible luminance range, the mobility and variation in capability of display devices are constantly increasing. Consequently, in real life usage it is now very likely that the display emission to be distorted by spatially and temporally varying reflections, and the observer's visual system to be not adapted to the particular display that she is viewing at that moment. The actual perception of the display content cannot be fully understood by only considering steady-state illumination and adaptation conditions. We propose an objective method for display visibility analysis formulating the problem as a full-reference image quality assessment problem, where the display emission under "ideal" conditions is used as the reference for real-life conditions. Our work includes a human visual system model that accounts for maladaptation and temporal recovery of sensitivity. As an example application we integrate our method to a global illumination simulator and analyze the visibility of a car interior display under realistic lighting conditions.

2003

Photometric variation in multi-projector displays is ar- guably the most vexing problem that needs to be addressed to achieve seamless tiled multi-projector displays. In this paper, we present a scalable real-time solution to correct the spatial photometric variation in multi-projector displays. A digital camera is used to capture the intensity variation across the display in a scalable fashion. This information

Rear projection of large-scale upright displays is often preferred over front projection because of the elimination of shadows that occlude the projected image. However, rear projection is not always a feasible option for space and cost reasons. Recent research suggests that many of the desirable features of rear projection, in particular shadow elimination, can be reproduced using new front projection techniques. We report on an empirical study to determine how two of these new projection techniques compare with traditional rear projection and front projection, with the hope of motivating the continued advance of improved virtual rear projection techniques.

2003

A system of coordinated projectors and cameras enables the creation of projected light displays that are robust to environmental disturbances. This paper describes approaches for tackling both geometric and photometric aspects of the problem: (1) the projected image remains stable even when the system components (projector, camera or screen) are moved; (2) the display automatically removes shadows caused by users moving between a projector and the screen, while simultaneously suppressing projected light on the user. The former can be accomplished without knowing the positions of the system components. The latter can be achieved without direct observation of the occluder. We demonstrate that the system responds quickly to environmental disturbances and achieves low steady-state errors.

2003

A major limitation of existing projection display systems is that they rely on a high quality screen for projecting images. We believe that relaxing this restriction will make projectors more useful and widely applicable. The fundamental problem with using an arbitrary surface for a screen is that the surface is bound to have its own colors and textures (bricks of a wall, painting on a wall, tiles of a ceiling, grain of a wooden door, etc.) or surface markings (paint imperfections, scratches, nails, etc.). As a result, when an image is projected onto the surface, the appearance of the image is modulated by the spatially varying reflectance properties of the surface. Humans are very sensitive to such modulations.