{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T17:25:51Z","timestamp":1787851551227,"version":"build-2784847793"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T00:00:00Z","timestamp":1687305600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Image understanding is often conceived as a hierarchical process with many levels, where complexity and invariance of object representation gradually increase with level in the hierarchy. In contrast, neurophysiological studies have shown that figure-ground organization and border ownership coding, which imply understanding of the object structure of an image, occur at levels as low as V1 and V2 of the visual cortex. This cannot be the result of back-projections from object recognition centers because border-ownership signals appear well-before shape selective responses emerge in inferotemporal cortex. Ultra-fast border-ownership signals have been found not only for simple figure displays, but also for complex natural scenes. In this paper I review neurophysiological evidence for the hypothesis that the brain uses dedicated grouping mechanisms early on to link elementary features to larger entities we might call \u201cproto-objects\u201d, a process that is pre-attentive and does not rely on object recognition. The proto-object structures enable the system to individuate objects and provide permanence, to track moving objects and cope with the displacements caused by eye movements, and to select one object out of many and scrutinize the selected object. I sketch a novel experimental paradigm for identifying grouping circuits, describe a first application targeting area V4, which yielded negative results, and suggest targets for future applications of this paradigm.<\/jats:p>","DOI":"10.3389\/fcomp.2023.1136987","type":"journal-article","created":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T08:47:23Z","timestamp":1687337243000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Visual cortical processing\u2014From image to object representation"],"prefix":"10.3389","volume":"5","author":[{"given":"R\u00fcdiger","family":"von der Heydt","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1038\/nn0901-937","article-title":"Shape-coding in IT cells generalizes over contrast and mirror reversal, but not figure-ground reversal","volume":"4","author":"Baylis","year":"2001","journal-title":"Nat. Neurosci."},{"key":"B2","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.neuron.2005.11.026","article-title":"Dynamic shape synthesis in posterior inferotemporal cortex","volume":"49","author":"Brincat","year":"2006","journal-title":"Neuron"},{"key":"B3","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1016\/S0079-6123(01)34014-1","article-title":"the role of feedback connections in shaping the responses of visual cortical neurons","volume":"134","author":"Bullier","year":"2001","journal-title":"Prog. Brain Res."},{"key":"B4","doi-asserted-by":"publisher","first-page":"4012","DOI":"10.1523\/JNEUROSCI.4766-10.2011","article-title":"Partial occlusion modulates contour-based shape encoding in primate area V4","volume":"31","author":"Bushnell","year":"2011","journal-title":"J. Neurosci."},{"key":"B5","doi-asserted-by":"publisher","first-page":"4310","DOI":"10.1152\/jn.00203.2007","article-title":"A neural model of figure-ground organization","volume":"97","author":"Craft","year":"2007","journal-title":"J. Neurophysiol."},{"key":"B6","doi-asserted-by":"publisher","first-page":"e72573","DOI":"10.7554\/eLife.72573.sa2","article-title":"Columnar processing of border ownership in primate visual cortex","volume":"10","author":"Franken","year":"2021","journal-title":"Elife"},{"key":"B7","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1113\/jphysiol.2002.033555","article-title":"The coding of uniform color figures in monkey visual cortex","volume":"548","author":"Friedman","year":"2003","journal-title":"J. Physiol."},{"key":"B8","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/B978-0-12-415804-7.00019-8","article-title":"\u201cChapter 19 - visual network,\u201d","volume-title":"Neuronal Networks in Brain Function, CNS Disorders, and Therapeutics","author":"Furman","year":"2014"},{"key":"B9","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/s10827-017-0662-8","article-title":"Short-term depression and transient memory in sensory cortex","volume":"43","author":"Gillary","year":"2017","journal-title":"J. Comput. Neurosci."},{"key":"B10","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1037\/0033-295X.92.2.173","article-title":"Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading","volume":"92","author":"Grossberg","year":"1985","journal-title":"Psychol. Rev."},{"key":"B11","doi-asserted-by":"publisher","first-page":"963","DOI":"10.1016\/0042-6989(92)90039-L","article-title":"Simulation of neural contour mechanisms: from simple to end-stopped cells","volume":"32","author":"Heitger","year":"1992","journal-title":"Vis. Res."},{"key":"B12","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1016\/S0262-8856(97)00083-8","article-title":"Simulation of neural contour mechanisms: representing anomalous contours","volume":"16","author":"Heitger","year":"1998","journal-title":"Image Vis. Comp. Comput. Psychophys. Stud. Early Vis."},{"key":"B13","first-page":"1","article-title":"\u201cFigure-ground representation in deep neural networks,\u201d","volume-title":"2019 53rd Annual Conference on Information Sciences and Systems (CISS)","author":"Hu","year":""},{"key":"B14","doi-asserted-by":"publisher","first-page":"ENEURO.0479-18","DOI":"10.1523\/ENEURO.0479-18.2019","article-title":"Figure-ground organization in natural scenes: performance of a recurrent neural model compared with neurons of area V2","volume":"6","author":"Hu","year":"","journal-title":"ENeuro"},{"key":"B15","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1006\/cogp.2001.0755","article-title":"The spatial resolution of visual attention","volume":"43","author":"Intriligator","year":"2001","journal-title":"Cognit. Psychol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1016\/j.visres.2006.12.018","article-title":"Boundary assignment in a recurrent network architecture","volume":"47","author":"Jehee","year":"2007","journal-title":"Vis. Res."},{"key":"B17","doi-asserted-by":"publisher","first-page":"e14320","DOI":"10.7554\/eLife.14320","article-title":"Serial grouping of 2D-image regions with object-based attention in humans","volume":"5","author":"Jeurissen","year":"2016","journal-title":"Elife"},{"key":"B18","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1037\/a0019076","article-title":"Surface construction by a 2-D differentiation-integration process: a neurocomputational model for perceived border ownership, depth, and lightness in Kanizsa Figures","volume":"117","author":"Kogo","year":"2010","journal-title":"Psychol. Rev."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1605","DOI":"10.1523\/JNEUROSCI.15-02-01605.1995","article-title":"The neurophysiology of figure-ground segregation in primary visual cortex","volume":"15","author":"Lamme","year":"1995","journal-title":"J. Neurosci."},{"key":"B20","doi-asserted-by":"publisher","first-page":"6860","DOI":"10.1523\/JNEUROSCI.3590-14.2015","article-title":"Spike synchrony reveals emergence of proto-objects in visual cortex","volume":"35","author":"Martin","year":"2015","journal-title":"J. Neurosci."},{"key":"B21","first-page":"416","article-title":"\u201cA database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,\u201d","volume-title":"Proceedings of Eighth IEEE International Conference on Computer Vision","author":"Martin","year":"2001"},{"key":"B22","doi-asserted-by":"publisher","first-page":"7583","DOI":"10.1073\/pnas.1014655108","article-title":"Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects","volume":"108","author":"Mihalas","year":"2011","journal-title":"Proc. Nat. Acad. Sci. U. S. A."},{"key":"B23","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1068\/p180055","article-title":"Stereoscopic depth: its relation to image segmentation, grouping, and the recognition of occluded objects","volume":"18","author":"Nakayama","year":"1989","journal-title":"Perception"},{"key":"B24","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1016\/j.neuron.2009.01.014","article-title":"Short-term memory for figure-ground organization in the visual cortex","volume":"61","author":"O'Herron","year":"2009","journal-title":"Neuron"},{"key":"B25","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1167\/11.2.12","article-title":"Representation of object continuity in the visual cortex","volume":"11","author":"O'Herron","year":"2011","journal-title":"J. Vis."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.1523\/JNEUROSCI.2797-12.2013","article-title":"Remapping of border ownership in the visual cortex","volume":"33","author":"O'Herron","year":"2013","journal-title":"J. Neurosci."},{"key":"B27","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1016\/B0-12-227210-2\/00346-0","article-title":"\u201cThalamus and thalamic damage,\u201d","volume-title":"Encyclopedia of the Human Brain","author":"Ohye","year":"2002"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1492","DOI":"10.1038\/nn1989","article-title":"Figure-ground mechanisms provide structure for selective attention","volume":"10","author":"Qiu","year":"2007","journal-title":"Nat. Neurosci."},{"key":"B29","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.neuron.2005.05.028","article-title":"Figure and ground in the visual cortex: v2 combines stereoscopic cues with gestalt rules","volume":"47","author":"Qiu","year":"2005","journal-title":"Neuron"},{"key":"B30","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1038\/nn1853","article-title":"Neural representation of transparent overlay","volume":"10","author":"Qiu","year":"2007","journal-title":"Nat. Neurosci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1038\/26475","article-title":"Object-based attention in the primary visual cortex of the macaque monkey","volume":"395","author":"Roelfsema","year":"1998","journal-title":"Nature."},{"key":"B32","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1126\/science.1223082","article-title":"The pulvinar regulates information transmission between cortical areas based on attention demands","volume":"337","author":"Saalmann","year":"2012","journal-title":"Science"},{"key":"B33","doi-asserted-by":"publisher","first-page":"12591","DOI":"10.1523\/JNEUROSCI.2929-08.2008","article-title":"Spatial and temporal scales of neuronal correlation in primary visual cortex","volume":"28","author":"Smith","year":"2008","journal-title":"J. Neurosci."},{"key":"B34","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1152\/jn.00928.2010","article-title":"The speed of context integration in the visual cortex","volume":"106","author":"Sugihara","year":"2011","journal-title":"J. Neurophysiol."},{"key":"B35","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1093\/cercor\/bhm061","article-title":"Cortical connections of area V4 in the Macaque","volume":"18","author":"Ungerleider","year":"2008","journal-title":"Cereb. Cortex"},{"key":"B36","first-page":"549","article-title":"\u201cTwo cortical visual systems,\u201d","volume-title":"Analysis of Visual Behavior","author":"Ungerleider","year":"1982"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1695","DOI":"10.3389\/fpsyg.2015.01695","article-title":"Figure-ground organization and the emergence of proto-objects in the visual cortex","volume":"6","author":"von der Heydt","year":"2015","journal-title":"Front. Psychol."},{"key":"B38","doi-asserted-by":"publisher","first-page":"2222","DOI":"10.1364\/JOSAA.22.002222","article-title":"Border-ownership dependent tilt aftereffect","volume":"22","author":"von der Heydt","year":"2005","journal-title":"J. Opt. Soc. Am. Opt. A."},{"key":"B39","doi-asserted-by":"publisher","first-page":"1731","DOI":"10.1523\/JNEUROSCI.09-05-01731.1989","article-title":"Mechanisms of contour perception in monkey visual cortex. I. Lines of pattern discontinuity","volume":"9","author":"von der Heydt","year":"1989","journal-title":"J. Neurosci."},{"key":"B40","doi-asserted-by":"publisher","first-page":"1260","DOI":"10.1126\/science.6539501","article-title":"Illusory contours and cortical neuron responses","volume":"224","author":"von der Heydt","year":"1984","journal-title":"Science"},{"key":"B41","doi-asserted-by":"publisher","first-page":"1416","DOI":"10.1523\/JNEUROSCI.12-04-01416.1992","article-title":"Periodic-pattern-selective cells in monkey visual cortex","volume":"12","author":"von der Heydt","year":"1992","journal-title":"J. Neurosci."},{"key":"B42","doi-asserted-by":"publisher","first-page":"666","DOI":"10.1167\/3.9.666","article-title":"Neural mechanisms in border ownership assignment: motion parallax and gestalt cues","author":"von der Heydt","year":"2003","journal-title":"J. Vis."},{"key":"B43","doi-asserted-by":"publisher","first-page":"3085","DOI":"10.1152\/jn.00125.2018","article-title":"Figure and ground: how the visual cortex integrates local cues for global organization","volume":"120","author":"von der Heydt","year":"2018","journal-title":"J. Neurophysiol."},{"key":"B44","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1007\/BF00622503","article-title":"Das Reafferenzprinzip. Wechselwirkungen Zwischen Zentralnervensystem Und Peripherie","volume":"37","author":"von Holst","year":"1950","journal-title":"Naturwissenschaften"},{"key":"B45","doi-asserted-by":"publisher","first-page":"1418","DOI":"10.1152\/jn.01142.2015","article-title":"Spike synchrony generated by modulatory common input through NMDA-type synapses","volume":"116","author":"Wagatsuma","year":"2016","journal-title":"J. Neurophysiol."},{"key":"B46","doi-asserted-by":"publisher","first-page":"ENEURO.0127","DOI":"10.1523\/ENEURO.0127-16.2016","article-title":"Figure-ground organization in visual cortex for natural scenes","volume":"3","author":"Williford","year":"","journal-title":"ENeuro"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.7281\/T1C8276W","author":"Williford","year":"","journal-title":"Data Associated with Publication \u2018Figure-Ground Organization in Visual Cortex for Natural Scenes,' Version 1"},{"key":"B48","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1038\/nrn1411","article-title":"What attributes guide the deployment of visual attention and how do they do it?","volume":"5","author":"Wolfe","year":"2004","journal-title":"Nat. Rev. Neurosci."},{"key":"B49","doi-asserted-by":"publisher","first-page":"6482","DOI":"10.1523\/JNEUROSCI.5168-09.2010","article-title":"Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex","volume":"30","author":"Zhang","year":"2010","journal-title":"J. Neurosci."},{"key":"B50","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.neuron.2005.04.005","article-title":"Border ownership from intracortical interactions in visual area V2","volume":"47","author":"Zhaoping","year":"2005","journal-title":"Neuron"},{"key":"B51","doi-asserted-by":"publisher","first-page":"6594","DOI":"10.1523\/JNEUROSCI.20-17-06594.2000","article-title":"Coding of border ownership in monkey visual cortex","volume":"20","author":"Zhou","year":"2000","journal-title":"J. Neurosci."},{"key":"B52","doi-asserted-by":"publisher","first-page":"1979","DOI":"10.1152\/jn.00112.2020","article-title":"Searching for object pointers in the visual cortex","volume":"123","author":"Zhu","year":"2020","journal-title":"J. Neurophysiol."}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1136987\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T08:47:31Z","timestamp":1687337251000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1136987\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,21]]},"references-count":52,"alternative-id":["10.3389\/fcomp.2023.1136987"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2023.1136987","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,21]]},"article-number":"1136987"}}