Our perceptual systems are products of evolution and have been shaped, in part, by natural select... more Our perceptual systems are products of evolution and have been shaped, in part, by natural selection. It is widely assumed that natural selection favors veridical perceptions—that is, perceptions that accurately describe aspects of the objective world relevant to fitness. This assumption has been tested using the mathematics of evolutionary game theory. It is false. Monte Carlo simulations reveal that veridical perceptions are never more fit, and generically are less fit, than nonveridical perceptions of equal complexity that are tuned to fitness. Veridical perceptions go extinct, and their extinction rate increases as complexity increases. These results motivate a new theory of perceptual systems—as species-specific interfaces shaped by natural selection to hide objective reality and guide adaptive behavior. For Homo sapiens, space-time is the desktop of the interface and physical objects are icons on the desktop. The shapes and colors of physical objects no more resemble objective...
What are conscious experiences? Can they combine to form new experiences? What are conscious subj... more What are conscious experiences? Can they combine to form new experiences? What are conscious subjects? Can they combine to form new subjects? Most attempts to answer these questions assume that spacetime, and some of its particles, are fundamental. However, physicists tell us that spacetime cannot be fundamental. Spacetime, they say, is doomed. We heed the physicists, and drop the assumption that spacetime is fundamental. We assume instead that subjects and experiences are entities beyond spacetime, not within spacetime. We make this precise in a mathematical theory of conscious agents, whose dynamics are described by Markov chains. We show how (1) agents combine into more complex agents, (2) agents fuse into simpler agents, and (3) qualia fuse to create new qualia. The possible dynamics of n agents form an n(n -1)-dimensional polytope with n n vertices-the Markov polytope M n . The total fusions of n agents and qualia form an (n -1)dimensional simplex-the fusion simplex F n . To project the Markovian dynamics of conscious agents onto scattering processes in spacetime, we define a new map from Markov chains to decorated permutations. Such permutations-along with helicities, or masses and spins-invariantly encode all physical information used to compute scattering amplitudes. We propose that spacetime and scattering processes are a data structure that codes for interactions of conscious agents: a particle in spacetime is a projection of the Markovian dynamics of a communicating class of conscious agents.
Does natural selection favor veridical perceptions, those that more accurately depict the objecti... more Does natural selection favor veridical perceptions, those that more accurately depict the objective environment? Vision researchers often claim that it does. But this claim, though influential, has not been adequately tested. In this paper we formalize the claim and a few alternatives. We then discuss the results of evolutionary games and genetic algorithms that indicate that veridical perceptions can be driven to extinction by non-veridical strategies that are tuned to fitness rather than to objective reality. This suggests that natural selection need not favor veridical perceptions, and that the effects of natural selection on perception deserve further study. We then address the question: How can an organism act effectively in an environment that it does not perceive veridically? (Acknowledgement: Brian Marion and Kyle Stevens are collaborators in this research.
Marr proposed that human vision constructs “a true description of what is there”. He argued that ... more Marr proposed that human vision constructs “a true description of what is there”. He argued that to understand human vision one must discover the features of the world it recovers and the constraints it uses in the process. Bayesian decision theory (BDT) is used in modern vision research as a probabilistic framework for understanding human vision along the lines laid out by Marr. Marr's contribution to vision research is substantial and justly influential. We propose, however, that evolution by natural selection does not, in general, favor perceptions that are true descriptions of the objective world. Instead, research with evolutionary games shows that perceptual systems tuned solely to fitness routinely outcompete those tuned to truth. Fitness functions depend not just on the true state of the world, but also on the organism, its state, and the type of action. Thus, fitness and truth are distinct. Natural selection depends only on expected fitness. It shapes perceptual systems...
Pylyshyn's target article argues that perception is not inferential, but this is true only un... more Pylyshyn's target article argues that perception is not inferential, but this is true only under a narrow construal of inference. A more general construal is possible, and has been used to provide formal theories of many visual capacities. This approach also makes clear that the evolution of natural constraints need not converge to the “veridical” state of the world.
Visual images are ambiguous. Any image, or collection of images, is consistent with an infinite n... more Visual images are ambiguous. Any image, or collection of images, is consistent with an infinite number of possible scenes in the world. Yet we are generally unaware of this ambiguity. During ordinary perception we are generally aware of only one, or perhaps a few of these possibilities. Human vision evidently exploits certain constraints—assumptions about the world and images formed of it—in order to generate its perceptions. One constraint that has been widely studied by researchers in human and machine vision is the generic-viewpoint assumption. We show that this assumption can help to explain the widely discussed fact that outlines of blobs are ineffective inducers of illusory contours. We also present a number of novel effects and report an experiment suggesting that the generic-viewpoint assumption strongly influences illusory-contour perception.
Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have bee... more Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natu-ral selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and al-low them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that verid-ical perceptions—strategies tuned to the true structure of the world—are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a win-dow on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our percep-tions of space-time and objects do not entail (by the Invention of Space-Time ...
Networks of ''conscious agents" (CAs) as defined by Hoffman and Prakash (2014) are shown to provi... more Networks of ''conscious agents" (CAs) as defined by Hoffman and Prakash (2014) are shown to provide a robust and intuitive representation of perceptual and cognitive processes in the context of the Interface Theory of Perception (Hoffman, Singh and Prakash, 2015). The behavior of the simplest CA networks is analyzed exhaustively. The construction of short-and long-term memories and the implementation of attention, categorization and case-based planning are demonstrated. These results show that robust perception and cognition can be modelled independently of any ontological assumptions about the world in which an agent is embedded. Any agent-world interaction can, in particular, also be represented as an agent-agent interaction.
Inverting a face impairs perception of its features and recognition of its identity. Whether face... more Inverting a face impairs perception of its features and recognition of its identity. Whether faces are special in this regard is a current topic of research and debate. Kanizsa studied the role of facial features and environmental context in perceiving the emotion and identity of upright and inverted faces. He found that observers are biased to interpret faces in a retinal coordinate frame, and that this bias is readily overruled by increased realism of facial features, but not easily overruled by environmental context. An additional factor contributing to a retinal coordinate-frame interpretation may be the ambiguous nature of the face stimuli. Since his facial expressions are interpretable both upright and inverted, they may in both orientations activate an endogenous attentional process for faces. We present visual search and change-blindness experiments that explore how inversion, negation, and facial emotion affect visual attention to static faces. We find that attention to faces is impaired by inversion and negation. We also find that the parts of the face that receive greater attention can be influenced by the emotional expression of the face. We propose to extend these experiments to dynamic faces. To this end, we develop a theory of the visual representation of dynamic faces, in which faces are represented by classes of 'spacetime fragments'-moving regions of the face with high informational content. We then present ideas for future experiments which are motivated by the spacetime fragment theory, and which should serve to constrain its further development.
We propose that selection favors nonveridical perceptions that are tuned to fitness. Current text... more We propose that selection favors nonveridical perceptions that are tuned to fitness. Current textbooks assert, to the contrary, that perception is useful because, in the normal case, it is veridical. Intuition, both lay and expert, clearly sides with the textbooks. We thus expected that some commentators would reject our proposal and provide counterarguments that could stimulate a productive debate. We are pleased that several commentators did indeed rise to the occasion and have argued against our proposal. We are also pleased that several others found our proposal worth exploring and have offered ways to test it, develop it, and link it more deeply to the history of ideas in the science and philosophy of perception. To both groups of commentators: thank you. Point and counterpoint, backed by data and theory, is the essence of science. We hope that the exchange recorded here will advance the scientific understanding of perception and its evolution. In what follows, we respond to th...
Visual forms come in countless varieties, from the simplicity of a sphere, to the geometric compl... more Visual forms come in countless varieties, from the simplicity of a sphere, to the geometric complexity of a face, to the fractal complexity of a rugged coast. These varieties have been studied with mathematical tools such as topology, differential geometry and fractal geometry. They have also been examined, largely in the last three decades, in terms of mereology, the study of part-whole relationships. The result is a fascinating body of theoretical and empirical results. In this paper I review these results, and describe a new development that applies them to the problem of learning names for visual forms and their parts.
Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have bee... more Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natural selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that veridical perceptions-strategies tuned to the true structure of the world-are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the Invention of Space-Time Theorem) that objective reality has the structure of space-time and objects. An interface serves to guide useful actions, not to resemble truth. Indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. For the perceptions of H. sapiens, space-time is the desktop and physical objects are the icons. Our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. Perception is an adaptive interface.
Human vision readily constructs subjective contours from displays of kinetic occlusion and color ... more Human vision readily constructs subjective contours from displays of kinetic occlusion and color from motion. To construct these contours from kinetic displays it is argued that human vision must solve the point-aperture problem, a problem more general and more difficult than the well-known aperture problem. In the aperture problem one is given a contour and its orthogonal velocity field, and must compute the full velocity field; in the point-aperture problem one is given neither the curve nor any components of its velocity field, and must constn,ict both the curve and its full velocity field. We formalize the point-aperture problem and present, in special cases, two simple algorithms for its solution.
Do images provide a foundation for interstellar communication that is relatively unam- biguous? I... more Do images provide a foundation for interstellar communication that is relatively unam- biguous? Is there something special about images that makes them a universal medium of communication, likely to be understood with little difficulty by extraterrestrial intelli- gences sufficiently advanced that they attempt to contact us via, say, radio or laser? This is the question we address in this chapter. It is natural to suppose pretheoretically that images are a particularly easy medium of communication, one that requires little if any interpretation. Our own subjective experience when we view images is that the process of seeing is usually effortless and almost instantaneous. Moreover, if we wish to create road signs that can be understood by drivers of different native languages, we use pic- tures rather than words, and the pictures are so effective that we trust them to guide the actions of drivers even in cases where a misinterpretation of a sign could have serious consequences. Since...
We analyze the properties of a dynamic color-spreading display created by adding narrow colored f... more We analyze the properties of a dynamic color-spreading display created by adding narrow colored flanks to rigidly moving black lines where these lines fall in the interior of a stationary virtual disk. This recently introduced display (Wollschläger et al, 2001 Perception30 1423–1426) induces the perception of a colored transparent disk bounded by strong illusory contours. It provides a link between the classical neon-color-spreading effect and edge-induced color spreading as discussed by Pinna et al (2001 Vision Research41 2669–2676). We performed three experiments to quantitatively study (i) the enhancing influence of apparent motion; (ii) the degrading effect of small spatial discontinuities (gaps) between lines and flanks; and (iii) the spatial extent of the color spreading. We interpret the results as due to varying degrees of objecthood of the dynamically specified disk: increased objecthood leads to increased surface visibility in both contour and color.
Our perceptual systems are products of evolution and have been shaped, in part, by natural select... more Our perceptual systems are products of evolution and have been shaped, in part, by natural selection. It is widely assumed that natural selection favors veridical perceptions—that is, perceptions that accurately describe aspects of the objective world relevant to fitness. This assumption has been tested using the mathematics of evolutionary game theory. It is false. Monte Carlo simulations reveal that veridical perceptions are never more fit, and generically are less fit, than nonveridical perceptions of equal complexity that are tuned to fitness. Veridical perceptions go extinct, and their extinction rate increases as complexity increases. These results motivate a new theory of perceptual systems—as species-specific interfaces shaped by natural selection to hide objective reality and guide adaptive behavior. For Homo sapiens, space-time is the desktop of the interface and physical objects are icons on the desktop. The shapes and colors of physical objects no more resemble objective...
What are conscious experiences? Can they combine to form new experiences? What are conscious subj... more What are conscious experiences? Can they combine to form new experiences? What are conscious subjects? Can they combine to form new subjects? Most attempts to answer these questions assume that spacetime, and some of its particles, are fundamental. However, physicists tell us that spacetime cannot be fundamental. Spacetime, they say, is doomed. We heed the physicists, and drop the assumption that spacetime is fundamental. We assume instead that subjects and experiences are entities beyond spacetime, not within spacetime. We make this precise in a mathematical theory of conscious agents, whose dynamics are described by Markov chains. We show how (1) agents combine into more complex agents, (2) agents fuse into simpler agents, and (3) qualia fuse to create new qualia. The possible dynamics of n agents form an n(n -1)-dimensional polytope with n n vertices-the Markov polytope M n . The total fusions of n agents and qualia form an (n -1)dimensional simplex-the fusion simplex F n . To project the Markovian dynamics of conscious agents onto scattering processes in spacetime, we define a new map from Markov chains to decorated permutations. Such permutations-along with helicities, or masses and spins-invariantly encode all physical information used to compute scattering amplitudes. We propose that spacetime and scattering processes are a data structure that codes for interactions of conscious agents: a particle in spacetime is a projection of the Markovian dynamics of a communicating class of conscious agents.
Does natural selection favor veridical perceptions, those that more accurately depict the objecti... more Does natural selection favor veridical perceptions, those that more accurately depict the objective environment? Vision researchers often claim that it does. But this claim, though influential, has not been adequately tested. In this paper we formalize the claim and a few alternatives. We then discuss the results of evolutionary games and genetic algorithms that indicate that veridical perceptions can be driven to extinction by non-veridical strategies that are tuned to fitness rather than to objective reality. This suggests that natural selection need not favor veridical perceptions, and that the effects of natural selection on perception deserve further study. We then address the question: How can an organism act effectively in an environment that it does not perceive veridically? (Acknowledgement: Brian Marion and Kyle Stevens are collaborators in this research.
Marr proposed that human vision constructs “a true description of what is there”. He argued that ... more Marr proposed that human vision constructs “a true description of what is there”. He argued that to understand human vision one must discover the features of the world it recovers and the constraints it uses in the process. Bayesian decision theory (BDT) is used in modern vision research as a probabilistic framework for understanding human vision along the lines laid out by Marr. Marr's contribution to vision research is substantial and justly influential. We propose, however, that evolution by natural selection does not, in general, favor perceptions that are true descriptions of the objective world. Instead, research with evolutionary games shows that perceptual systems tuned solely to fitness routinely outcompete those tuned to truth. Fitness functions depend not just on the true state of the world, but also on the organism, its state, and the type of action. Thus, fitness and truth are distinct. Natural selection depends only on expected fitness. It shapes perceptual systems...
Pylyshyn's target article argues that perception is not inferential, but this is true only un... more Pylyshyn's target article argues that perception is not inferential, but this is true only under a narrow construal of inference. A more general construal is possible, and has been used to provide formal theories of many visual capacities. This approach also makes clear that the evolution of natural constraints need not converge to the “veridical” state of the world.
Visual images are ambiguous. Any image, or collection of images, is consistent with an infinite n... more Visual images are ambiguous. Any image, or collection of images, is consistent with an infinite number of possible scenes in the world. Yet we are generally unaware of this ambiguity. During ordinary perception we are generally aware of only one, or perhaps a few of these possibilities. Human vision evidently exploits certain constraints—assumptions about the world and images formed of it—in order to generate its perceptions. One constraint that has been widely studied by researchers in human and machine vision is the generic-viewpoint assumption. We show that this assumption can help to explain the widely discussed fact that outlines of blobs are ineffective inducers of illusory contours. We also present a number of novel effects and report an experiment suggesting that the generic-viewpoint assumption strongly influences illusory-contour perception.
Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have bee... more Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natu-ral selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and al-low them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that verid-ical perceptions—strategies tuned to the true structure of the world—are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a win-dow on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our percep-tions of space-time and objects do not entail (by the Invention of Space-Time ...
Networks of ''conscious agents" (CAs) as defined by Hoffman and Prakash (2014) are shown to provi... more Networks of ''conscious agents" (CAs) as defined by Hoffman and Prakash (2014) are shown to provide a robust and intuitive representation of perceptual and cognitive processes in the context of the Interface Theory of Perception (Hoffman, Singh and Prakash, 2015). The behavior of the simplest CA networks is analyzed exhaustively. The construction of short-and long-term memories and the implementation of attention, categorization and case-based planning are demonstrated. These results show that robust perception and cognition can be modelled independently of any ontological assumptions about the world in which an agent is embedded. Any agent-world interaction can, in particular, also be represented as an agent-agent interaction.
Inverting a face impairs perception of its features and recognition of its identity. Whether face... more Inverting a face impairs perception of its features and recognition of its identity. Whether faces are special in this regard is a current topic of research and debate. Kanizsa studied the role of facial features and environmental context in perceiving the emotion and identity of upright and inverted faces. He found that observers are biased to interpret faces in a retinal coordinate frame, and that this bias is readily overruled by increased realism of facial features, but not easily overruled by environmental context. An additional factor contributing to a retinal coordinate-frame interpretation may be the ambiguous nature of the face stimuli. Since his facial expressions are interpretable both upright and inverted, they may in both orientations activate an endogenous attentional process for faces. We present visual search and change-blindness experiments that explore how inversion, negation, and facial emotion affect visual attention to static faces. We find that attention to faces is impaired by inversion and negation. We also find that the parts of the face that receive greater attention can be influenced by the emotional expression of the face. We propose to extend these experiments to dynamic faces. To this end, we develop a theory of the visual representation of dynamic faces, in which faces are represented by classes of 'spacetime fragments'-moving regions of the face with high informational content. We then present ideas for future experiments which are motivated by the spacetime fragment theory, and which should serve to constrain its further development.
We propose that selection favors nonveridical perceptions that are tuned to fitness. Current text... more We propose that selection favors nonveridical perceptions that are tuned to fitness. Current textbooks assert, to the contrary, that perception is useful because, in the normal case, it is veridical. Intuition, both lay and expert, clearly sides with the textbooks. We thus expected that some commentators would reject our proposal and provide counterarguments that could stimulate a productive debate. We are pleased that several commentators did indeed rise to the occasion and have argued against our proposal. We are also pleased that several others found our proposal worth exploring and have offered ways to test it, develop it, and link it more deeply to the history of ideas in the science and philosophy of perception. To both groups of commentators: thank you. Point and counterpoint, backed by data and theory, is the essence of science. We hope that the exchange recorded here will advance the scientific understanding of perception and its evolution. In what follows, we respond to th...
Visual forms come in countless varieties, from the simplicity of a sphere, to the geometric compl... more Visual forms come in countless varieties, from the simplicity of a sphere, to the geometric complexity of a face, to the fractal complexity of a rugged coast. These varieties have been studied with mathematical tools such as topology, differential geometry and fractal geometry. They have also been examined, largely in the last three decades, in terms of mereology, the study of part-whole relationships. The result is a fascinating body of theoretical and empirical results. In this paper I review these results, and describe a new development that applies them to the problem of learning names for visual forms and their parts.
Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have bee... more Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natural selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that veridical perceptions-strategies tuned to the true structure of the world-are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the Invention of Space-Time Theorem) that objective reality has the structure of space-time and objects. An interface serves to guide useful actions, not to resemble truth. Indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. For the perceptions of H. sapiens, space-time is the desktop and physical objects are the icons. Our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. Perception is an adaptive interface.
Human vision readily constructs subjective contours from displays of kinetic occlusion and color ... more Human vision readily constructs subjective contours from displays of kinetic occlusion and color from motion. To construct these contours from kinetic displays it is argued that human vision must solve the point-aperture problem, a problem more general and more difficult than the well-known aperture problem. In the aperture problem one is given a contour and its orthogonal velocity field, and must compute the full velocity field; in the point-aperture problem one is given neither the curve nor any components of its velocity field, and must constn,ict both the curve and its full velocity field. We formalize the point-aperture problem and present, in special cases, two simple algorithms for its solution.
Do images provide a foundation for interstellar communication that is relatively unam- biguous? I... more Do images provide a foundation for interstellar communication that is relatively unam- biguous? Is there something special about images that makes them a universal medium of communication, likely to be understood with little difficulty by extraterrestrial intelli- gences sufficiently advanced that they attempt to contact us via, say, radio or laser? This is the question we address in this chapter. It is natural to suppose pretheoretically that images are a particularly easy medium of communication, one that requires little if any interpretation. Our own subjective experience when we view images is that the process of seeing is usually effortless and almost instantaneous. Moreover, if we wish to create road signs that can be understood by drivers of different native languages, we use pic- tures rather than words, and the pictures are so effective that we trust them to guide the actions of drivers even in cases where a misinterpretation of a sign could have serious consequences. Since...
We analyze the properties of a dynamic color-spreading display created by adding narrow colored f... more We analyze the properties of a dynamic color-spreading display created by adding narrow colored flanks to rigidly moving black lines where these lines fall in the interior of a stationary virtual disk. This recently introduced display (Wollschläger et al, 2001 Perception30 1423–1426) induces the perception of a colored transparent disk bounded by strong illusory contours. It provides a link between the classical neon-color-spreading effect and edge-induced color spreading as discussed by Pinna et al (2001 Vision Research41 2669–2676). We performed three experiments to quantitatively study (i) the enhancing influence of apparent motion; (ii) the degrading effect of small spatial discontinuities (gaps) between lines and flanks; and (iii) the spatial extent of the color spreading. We interpret the results as due to varying degrees of objecthood of the dynamically specified disk: increased objecthood leads to increased surface visibility in both contour and color.
Uploads
Papers by Donald Hoffman