Orientation-ocularity maps: A technique for computer vision

Journal of Neuroscience, 2006

The left and right eyes receive subtly different images from a visual scene. Binocular disparities of retinal image locations are correlated with variation in the depth of objects in the scene and make stereoscopic depth perception possible. Disparity stereoscopically specifies a stimulus; changing the stimulus in a way that conserves its disparity leaves the stimulus stereoscopically unchanged. Therefore, a person's ability to use stereo to see the depth separating any two objects should depend only on the disparities of the objects, which in turn depend on where the objects are, not what they are. However, I find that the disparity difference between two stimuli by itself predicts neither stereoacuity nor perceived depth. Human stereo vision is shown here to be most sensitive at detecting the relative depth of two gratings when they are parallel. Rotating one grating by as little as 10°lowers sensitivity. The rotation can make a perceptible depth separation invisible, although it changes neither the relative nor absolute disparities of the gratings, only their relative orientations. The effect of relative orientation is not confined to stimuli that, like gratings, vary along one dimension or to stimuli perceived to have a dominant orientation. Rather, it is the relative orientation of the one-dimensional components of stimuli, even broadband stimuli, that matters. This limit on stereoscopic depth perception appears to be intrinsic to the visual system's computation of disparity; by taking place within orientation bands, the computation renders the coding of disparity inseparable from the coding of orientation.

Neuroscience journal, 2016

Stereopsis or depth perception is a critical aspect of information processing in the brain and is computed from the positional shift or disparity between the images seen by the two eyes. Various algorithms and their hardware implementation that compute disparity in real time have been proposed; however, most of them compute disparity through complex mathematical calculations that are difficult to realize in hardware and are biologically unrealistic. The brain presumably uses simpler methods to extract depth information from the environment and hence newer methodologies that could perform stereopsis with brain like elegance need to be explored. This paper proposes an innovative aVLSI design that leverages the columnar organization of ocular dominance in the brain and uses time-staggered Winner Take All (ts-WTA) to adaptively create disparity tuned cells. Physiological findings support the presence of disparity cells in the visual cortex and show that these cells surface as a result o...

Proceedings of the Royal Society of London. Series B. Biological Sciences, 1979

An algorithm is proposed for solving the stereoscopic matching problem. The algorithm consists of five steps: (1) Each image is filtered at different orientations with bar masks of four sizes that increase with eccentricity; the equivalent filters are one or two octaves wide. (2) Zero-crossings in the filtered images, which roughly correspond to edges, are localized. Positions of the ends of lines and edges are also found. (3) For each mask orientation and size, matching takes place between pairs of zero-crossings or terminations of the same sign in the two images, for a range of dis­parities up to about the width of the mask’s central region. (4) Wide masks can control vergence movements, thus causing small masks to come into correspondence. (5) When a correspondence is achieved, it is stored in a dynamic buffer, called the 2½-D sketch. It is shown that this proposal provides a theoretical framework for most existing psychophysical and neurophysiological data about stereopsis. Seve...

Electronic Imaging HVEI conf, 2019

As a biologically inspired guess, we consider two stereo information channels. One is the traditional channel that works on the basis of the horizontal disparity between the left and right projections of single points in the 3D scene; this channel carries information regarding the absolute depth of the point. The second channel works on the basis of the projections of pairs of points in the 3D scene and carries information regarding the relative depth of the points; equivalently, for a given azimuth disparity of the points, the channel carries information of the ratio of the orienta-tions of the left and right projections of the line segment between the pair of points.

Progress in Biophysics and Molecular Biology, 2005

Stereoscopic depth perception is a fascinating ability in its own right and also a useful model of perception. In recent years, considerable progress has been made in understanding the early cortical circuitry underlying this ability. Inputs from left and right eyes are first combined in primary visual cortex (V1), where many cells are tuned for binocular disparity. Although the observation of disparity tuning in V1, combined with psychophysical evidence that stereopsis must occur early in visual processing, led to initial suggestions that V1 was the neural correlate of stereoscopic depth perception, more recent work indicates that this must occur in higher visual areas. The firing of cells in V1 appears to depend relatively simply on the visual stimuli within local receptive fields in each retina, whereas the perception of depth reflects global properties of the stimulus. However, V1 neurons appear to be specialized in a number of respects to encode ecologically relevant binocular disparities. This suggests that they carry out essential preprocessing underlying stereoscopic depth perception in higher areas. This article reviews recent progress in developing accurate models of the computations carried out by these neurons. We seem close to achieving a mathematical description of the initial stages of the brain's stereo algorithm. This is important in itself--for instance, it may enable improved stereopsis in computer vision--and paves the way for a full understanding of how depth perception arises.

PLoS ONE, 2013

This work describes an approach inspired by the primary visual cortex using the stimulus response of the receptive field profiles of binocular cells for disparity computation. Using the energy model based on the mechanism of log-Gabor filters for disparity encodings, we propose a suitable model to consistently represent the complex cells by computing the wide bandwidths of the cortical cells. This way, the model ensures the general neurophysiological findings in the visual cortex (V1), emphasizing the physical disparities and providing a simple selection method for the complex cell response. The results suggest that our proposed approach can achieve better results than a hybrid model with phase-shift and positionshift using position disparity alone.

Lecture Notes in Computer Science, 1992

Visual information is represented in the primate visual cortex (area 17, layer 4B) in a peculiar structure of alternating bands of left and right eye dominance. Recently, a number of computational algorithms based on this ocular stripe map architecture have been proposed, from which we selected the cepstral filtering method of Y. Yeshurun & E.L. Schwartz for fast disparity computation due to its simplicity and robustness. The algorithm has been implemented and analyzed. Some special deficiencies have been identified. The robustness against noise and image degradations such as rotation and scaling has been evaluated. We made several improvements to the algorithm. For real image data the cepstral filter behaves like a square autocorrelation of a bandpass filtered version of the original image. The discussed framework is now a reliable single-step method for local depth estimation.

F1000 - Post-publication peer review of the biomedical literature, 2009

In invertebrate predators like the praying mantis and vertebrate predators such as wild cats, the ability to detect small differences in inter-ocular retinal disparities is a critical means for accurately determining the depth of moving objects such as prey1. In mammals, the first neurons along the visual pathway that encode binocular disparities are found in the visual cortex. However, a precise functional architecture for binocular disparity has never been demonstrated in any species, and coarse maps for disparity have been found in only one primate species2,3. Moreover, the dominant approach for assaying the developmental plasticity of binocular cortical neurons employed monocular tests of ocular dominance to infer binocular function4. The few studies that examined the relationship between ocular dominance and binocular disparity of individual cells used single-unit recordings and have provided conflicting results as to whether ocular dominance can predict the selectivity or sensitivity to binocular disparity5-9. Here we use two-photon calcium imaging to sample the response to monocular and binocular visual stimuli from nearly every adjacent neuron in a small region of the cat visual cortex, area 18. We show that local circuits for ocular dominance always have smooth and graded transitions from one apparently monocular functional domain to an adjacent binocular region. Most unexpectedly, we discovered a new map in the cat visual cortex that had a precise functional micro-architecture for binocular disparity selectivity. At the level of single cells, ocular dominance was unrelated to binocular disparity selectivity or sensitivity. When the local maps for ocular dominance and binocular disparity both had measurable gradients at a given cortical site, the two gradient directions were orthogonal to each other. Together, these results suggest that from the perspective of the spiking activity of individual neurons, ocular dominance cannot predict binocular disparity tuning. However, the precise local arrangement of ocular dominance and binocular disparity maps provide new clues on how monocular and binocular depth cues may be combined and decoded. Binocular vision and depth discrimination evolved more than 100 million years ago10. In mammals, the first single-cell description of a binocular disparity detector in the brain was made in the cerebral cortex of the cat approximately 40 years ago11. Numerous single-unit Users may view, print, copy, and download text and data-mine the content in such documents, for the purposes of academic research, subject always to the full Conditions of use:

Frontiers in Computational Neuroscience, 2012

Perception of stereoscopic depth requires that visual systems solve a correspondence problem: find parts of the left-eye view of the visual scene that correspond to parts of the right-eye view. The standard model of binocular matching implies that similarity of left and right images is computed by inter-ocular correlation. But the left and right images of the same object are normally distorted relative to one another by the binocular projection, in particular when slanted surfaces are viewed from close distance. Correlation often fails to detect correct correspondences between such image parts. We investigate a measure of inter-ocular similarity that takes advantage of spatially invariant computations similar to the computations performed by complex cells in biological visual systems. This measure tolerates distortions of corresponding image parts and yields excellent performance over a much larger range of surface slants than the standard model. The results suggest that, rather than serving as disparity detectors, multiple binocular complex cells take part in the computation of inter-ocular similarity, and that visual systems are likely to postpone commitment to particular binocular disparities until later stages in the visual process.

Perception, 1985

Burt and Julesz experimentally demonstrated that, in addition to Panum's fusional area, a quantity defined by them and named disparity gradient also plays a crucial part in deciding whether the human visual system would be able to fuse the images seen by the left and right eyes. The physical meaning of this quantity remains obscure despite attempts to interpret it in terms of depth gradient. Nevertheless, it has been found to be an effective selector of matches in stereo correspondence algorithms. A proof is provided that a disparity gradient limit of less than 2 implies that the matches between the two images preserve the topology of the images. The result, which is invariant under rotations and under relative as well as overall magnifications, holds for pairs of points separated in any direction, not just along epipolar lines. This in turn can be shown to prevent correspondences being established between points which would have to be located in three dimensions on a surface in...