Social interaction involves the active visual perception of facial expressions and communicative ... more Social interaction involves the active visual perception of facial expressions and communicative gestures. This study examines the distribution of gaze fixations while watching videos of expressive talking faces. The knowledge-driven factors that influence the selective visual processing of facial information were examined by using the same set of stimuli, and assigning subjects to either a speech recognition task or an emotion judgment task. For half of the subjects assigned to each of the tasks, the intelligibility of the speech was manipulated by the addition of moderate masking noise. Both tasks and the intelligibility of the speech signal influenced the spatial distribution of gaze. Gaze was concentrated more on the eyes when emotion was being judged as compared to when words were being identified. When noise was added to the acoustic signal, gaze in both tasks was more centralized on the face. This shows that subject's gaze is sensitive to the distribution of information on the face, but can also be influenced by strategies aimed at maximizing the amount of visual information processed.
During face-to-face conversation the face provides auditory and visual linguistic information, an... more During face-to-face conversation the face provides auditory and visual linguistic information, and also conveys information about the identity of the speaker. This study investigated behavioral strategies involved in gathering visual information while watching talking faces. The effects of varying talker identity and varying the intelligibility of speech (by adding acoustic noise) on gaze behavior were measured with an eyetracker. Varying the intelligibility of the speech by adding noise had a noticeable effect on the location and duration of fixations. When noise was present subjects adopted a vantage point that was more centralized on the face by reducing the frequency of the fixations on the eyes and mouth and lengthening the duration of their gaze fixations on the nose and mouth. Varying talker identity resulted in a more modest change in gaze behavior that was modulated by the intelligibility of the speech. Although subjects generally used similar strategies to extract visual information in both talker variability conditions, when noise was absent there were more fixations on the mouth when viewing a different talker every trial as opposed to the same talker every trial. These findings provide a useful baseline for studies examining gaze behavior during audiovisual speech perception and perception of dynamic faces.
Communication impairments pose a major threat to an individual's quality of life. However, the im... more Communication impairments pose a major threat to an individual's quality of life. However, the impact of visual impairments on communication is not well understood, despite the important role that vision plays in the perception of speech. Here we present 2 experiments examining the impact of discrete central scotomas on speech perception. In the first experiment, 4 patients with central vision loss due to unilateral macular holes identified utterances with conflicting auditory-visual information, while simultaneously having their eye movements recorded. Each eye was tested individually. Three participants showed similar speech perception with both the impaired eye and the unaffected eye. For 1 participant, speech perception was disrupted by the scotoma because the participant did not shift gaze to avoid obscuring the talker's mouth with the scotoma. In the second experiment, 12 undergraduate students with gaze-contingent artificial scotomas (10 visual degrees in diameter) identified sentences in background noise. These larger scotomas disrupted speech perception, but some participants overcame this by adopting a gaze strategy whereby they shifted gaze to prevent obscuring important regions of the face such as the mouth. Participants who did not spontaneously adopt an adaptive gaze strategy did not learn to do so over the course of 5 days; however, participants who began with adaptive gaze strategies became more consistent in their gaze location. These findings confirm that peripheral vision is sufficient for perception of most visual information in speech, and suggest that training in gaze strategy may be worthwhile for individuals with communication deficits due to visual impairments.
Journal of The Acoustical Society of America, 2001
In natural conversation, visual and auditory information about speech not only provide linguistic... more In natural conversation, visual and auditory information about speech not only provide linguistic information but also provide information about the identity and the emotional state of the speaker. Thus, listeners must process a wide range of information in parallel to understand the full meaning in a message. In this series of studies, we examined how different types of visual information conveyed by a speaker's face are processed by measuring the gaze patterns exhibited by subjects watching audiovisual recordings of spoken sentences. In three experiments, subjects were asked to judge the emotion and the identity of the speaker, and to report the words that they heard under different auditory conditions. As in previous studies, eye and mouth regions dominated the distribution of the gaze fixations. It was hypothesized that the eyes would attract more fixations for more social judgment tasks, rather than tasks which rely more on verbal comprehension. Our results support this hypothesis. In addition, the location of gaze on the face did not influence the accuracy of the perception of speech in noise.
We conducted three experiments in order to examine the influence of gaze behavior and fixation on... more We conducted three experiments in order to examine the influence of gaze behavior and fixation on audiovisual speech perception in a task that required subjects to report the speech sound they perceived during the presentation of congruent and incongruent (McGurk) audiovisual stimuli. Experiment 1 showed that the subjects’ natural gaze behavior rarely involved gaze fixations beyond the oral and ocular regions of the talker’s face and that these gaze fixations did not predict the likelihood of perceiving the McGurk effect. Experiments 2 and 3 showed that manipulation of the subjects’ gaze fixations within the talker’s face did not influence audiovisual speech perception substantially and that it was not until the gaze was displaced beyond 10°–20° from the talker’s mouth that the McGurk effect was significantly lessened. Nevertheless, the effect persisted under such eccentric viewing conditions and became negligible only when the subject’s gaze was directed 60° eccentrically. These findings demonstrate that the analysis of high spatial frequency information afforded by direct oral foveation isnot necessary for the successful processing of visual speech information.
Two long-standing questions in neuroscience concern the mechanisms underlying our abilities to ma... more Two long-standing questions in neuroscience concern the mechanisms underlying our abilities to make decisions and to store goal-relevant information in memory for seconds at a time. Recent experimental and theoretical advances suggest that NMDA receptors at intrinsic cortical synapses play an important role in both these functions. The long NMDA time constant is suggested to support persistent mnemonic activity by maintaining excitatory drive after the removal of a stimulus and to enable the slow integration of afferent information in the service of decisions. These findings have led to the hypothesis that the local circuit mechanisms underlying decisions must also furnish persistent storage of information. We use a local circuit cortical model of spiking neurons to test this hypothesis, controlling intrinsic drive by scaling NMDA conductance strength. Our simulations provide further evidence that persistent storage and decision making are supported by common mechanisms, but under biophysically realistic parameters, our model demonstrates that the processing requirements of persistent storage and decision making may be incompatible at the local circuit level. Parameters supporting persistent storage lead to strong dynamics that are at odds with slow integration, whereas weaker dynamics furnish the speed–accuracy trade-off common to psychometric data and decision theory.► We model a local cortical circuit participating in a perceptual decision. ► Network dynamics are controlled by NMDA receptor conductance at intrinsic synapses. ► Dynamics that support persistent mnemonic activity lead to poor decision making. ► Weaker dynamics support decision making that is consistent with experimental data.
The superior colliculus has long been recognized as an important structure in the generation of s... more The superior colliculus has long been recognized as an important structure in the generation of saccadic displacements of the visual axis. Neurons with presaccadic activity encoding saccade vectors are topographically organized and form a “motor map.” Recently, neurons with fixation-related activity have been recorded at the collicular rostral pole, at the area centralis representation or fixation area. Another collicular function which deals with the maintenance of fixation behavior by means of active inhibition of orientation commands was then suggested. We tested that hypothesis as it relates to the suppression of gaze saccades (gaze = eye in space = eye in head + head in space) in the head-free cat by increasing the activity of the fixation cells at the rostral pole with electrical microstimulation. Long stimulation trains applied before gaze saccades delayed their initiation. Short stimuli, delivered during the gaze saccades, transiently interrupted both eye and head components. These results provide further support for a role in fixation behavior for collicular fixation neurons. Brainstem omnipause neurons also exhibit fixation-related activity and have been shown to receive a direct excitatory input from the superior colliculus. To determine whether the collicular projection to omnipause neurons arises from the fixation area, the deep layers of the superior colliculus were electrically stimulated either at the rostral pole including the fixation area or in more caudal regions where stimulation evokes orienting responses. Forty-nine neurons were examined in three cats. 61% of the neurons were found to be orthodromically excited by single-pulse stimulation of the rostral pole, whereas only 29% responded to caudal stimulation. In addition, stimuli delivered to the rostral pole activated, on average, omnipause neurons at shorter latencies and with lower currents than those applied in caudal regions. These results suggest that excitatory inputs to omnipause neurons from the superior colliculus are principally provided by the fixation area, via which the superior colliculus could play a role in suppression of gaze shifts.
In our previous paper we demonstrated that electrical microstimulation of the fixation area at th... more In our previous paper we demonstrated that electrical microstimulation of the fixation area at the rostral pole of the cat superior colliculus (SC) elicits no gaze movement but, rather, transiently suppresses eye-head gaze saccades. In this paper, we investigated the more caudal region of the SC and its interaction with the fixation area. In the alert head-free cat, supra-threshold stimulation in the anterior portion of the SC but outside the fixation area evoked small saccadic shifts of gaze consisting mainly of an eye movement, the head's contribution being small. Stimulating more posteriorly elicited large gaze saccades consisting of an ocular saccade combined with a rapid head movement. At these latter stimulation sites, craniocentric (goal-directed) eye movements were evoked when the cat's head was restrained. The amplitude of eye-head gaze saccades elicited at a particular stimulation site increased with stimulus duration, current strength, and pulse rate, until a constant or “unit” value was reached. The peak velocity of gaze shifts depended on both pulse rate and current strength. The movement direction was not affected by stimulus parameters. The unit gaze vector evoked, in the head-free condition, by stimulating one collicular site was similar to that coded by efferent neurons recorded at that site, thereby indicating a retinotopically coded gaze error representation on the collicular motor map which is not revealed by stimulating the head-fixed animal. Evoked gaze saccades were found to be influenced by fixation behavior. The amplitude of evoked gaze shifts was reduced if stimulation occurred when the hungry animal fixated a food target. Electrical activation of the collicular fixation area was found to mimic well the effects of natural fixation on evoked gaze shifts. Taken together, our results support the view that the overall distribution and level of collicular activity contributes to the encoding of the metrics of gaze saccades. We suggest that the combined levels of activity at the site being stimulated and at the fixation area influence the amplitude of evoked gaze saccades through competition. When stimulation is at low intensities, fixation-related activity reduces the amplitude of evoked gaze saccades. At high activation levels, the site being stimulated dominates and the gaze vector is specified only by that site's collicular output neurons, from which arises the close correspondence between the unit-evoked gaze saccades and the neurally coded gaze vector at that site.
The ability of primates to make rapid and accurate saccadic eye movements for exploring the natur... more The ability of primates to make rapid and accurate saccadic eye movements for exploring the natural world is based on a neuronal system in the brain that has been studied extensively and is known to include multiple brain regions extending throughout the neuraxis. We examined the characteristics of signal flow in this system by recording from identified output neurons of two cortical regions, the lateral intraparietal area (LIP) and the frontal eye field (FEF), and from neurons in a brainstem structure targeted by these output neurons, the superior colliculus (SC). We compared the activity of neurons in these three populations while monkeys performed a delayed saccade task that allowed us to quantify visual responses, motor activity, and intervening delay activity. We examined whether delay activity was related to visual stimulation by comparing the activity during interleaved trials when a target was either present or absent during the delay period. We examined whether delay activity was related to movement by using a Go/Nogo task and comparing the activity during interleaved trials in which a saccade was either made (Go) or not (Nogo). We found that LIP output neurons, FEF output neurons, and SC neurons can all have visual responses, delay activity, and presaccadic bursts; hence in this way they are all quite similar. However, the delay activity tended to be more related to visual stimulation in the cortical output neurons than in the SC neurons. Complementing this, the delay activity tended to be more related to movement in the SC neurons than in the cortical output neurons. We conclude, first, that the signal flow leaving the cortex represents activity at nearly every stage of visuomotor transformation, and second, that there is a gradual evolution of signal processing as one proceeds from cortex to colliculus. Published by Elsevier Science Ltd.
Buschman and Miller (Reports, 30 March 2007, p. 1860) described the activity of ensembles of neur... more Buschman and Miller (Reports, 30 March 2007, p. 1860) described the activity of ensembles of neurons in parietal and frontal cortex of monkeys performing visual search for targets that were easy or hard to distinguish from distractors. However, their conclusions are called into question by discrepancies between their results and publications from other laboratories measuring the same neural process.
We investigated and quantified the ability of the primate saccadic system to generate accurate ey... more We investigated and quantified the ability of the primate saccadic system to generate accurate eye movements in spite of naturally occurring variations in saccadic speed and trajectory. We show that the amplitude of a series of saccades directed to the same target is positively correlated to their peak speed, i.e., the faster the saccade, the bigger its amplitude. We demonstrate that this result cannot be simply accounted for by the main sequence, and that on average the saccadic system is able to compensate for only 61% of the variability in speed. Deviations from the average trajectory are also only partially compensated: the underlying mechanism, which tends to bring the eyes back toward the desired trajectory, underperforms for small movements and overperforms for large movements. We also demonstrate that the performance of this compensatory mechanism, and the metrics of saccades in general, do not depend on the presence of visual information during the movement. By showing that deviations from the desired behavior are corrected during the saccade, our results further support the hypothesis that the innervation signal that generates saccadic eye movements is not pre-programmed but rather is dynamically adjusted during the movement. However, the compensation for deviations from the desired behavior is only partial, and the underlying mechanisms have yet to be completely understood. Although none of the current models of the saccadic system can account for our results, some of them, if appropriately modified, probably could.
Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye ... more Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845-858, 2002; 10.1152/ jn.00317.2001. Many neurons in the frontal eye field (FEF) and lateral intraparietal (LIP) areas of cerebral cortex are active during the visualmotor events preceding the initiation of saccadic eye movements: they respond to visual targets, increase their activity before saccades, and maintain their activity during intervening delay periods. Previous experiments have shown that the output neurons from both LIP and FEF convey the full range of these activities to the superior colliculus (SC) in the brain stem. These areas of cerebral cortex also have strong interconnections, but what signals they convey remains unknown. To determine what these cortico-cortical signals are, we identified the LIP neurons that project to FEF by antidromic activation, and we studied their activity during a delayed-saccade task. We then compared these cortico-cortical signals to those sent subcortically by also identifying the LIP neurons that project to the intermediate layers of the SC. Of 329 FEF projection neurons and 120 SC projection neurons, none were co-activated by both FEF and SC stimulation. FEF projection neurons were encountered more superficially in LIP than SC projection neurons, which is consistent with the anatomical projection of many cortical layer III neurons to other cortical areas and of layer V neurons to subcortical structures. The estimated conduction velocities of FEF projection neurons (16.7 m/s) were significantly slower that those of SC projection neurons (21.7 m/s), indicating that FEF projection neurons have smaller axons. We identified three main differences in the discharge properties of FEF and SC projection neurons: only 44% of the FEF projection neurons changed their activity during the delayed-saccade task compared with 69% of the SC projection neurons; only 17% of the task-related FEF projection neurons showed saccadic activity, whereas 42% of the SC projection neurons showed such increases; 78% of the FEF projection neurons had a visual response but no saccadic activity, whereas only 55% of the SC projection neurons had similar activity. The FEF and SC projection neurons had three similarities: both had visual, delay, and saccadic activity, both had stronger delay and saccadic activity with visually guided than with memory-guided saccades, and both had broadly tuned responses for disparity stimuli, suggesting that their visual receptive fields have a three-dimensional configuration. These observations indicate that the activity carried between parietal and frontal cortical areas conveys a spectrum of signals but that the preponderance of activity conveyed might be more closely related to earlier visual processing than to the later saccadic stages that are directed to the SC.
We investigated and quantified the ability of the primate saccadic system to generate accurate ey... more We investigated and quantified the ability of the primate saccadic system to generate accurate eye movements in spite of naturally occurring variations in saccadic speed and trajectory. We show that the amplitude of a series of saccades directed to the same target is positively correlated to their peak speed, i.e., the faster the saccade, the bigger its amplitude. We demonstrate that this result cannot be simply accounted for by the main sequence, and that on average the saccadic system is able to compensate for only 61% of the variability in speed. Deviations from the average trajectory are also only partially compensated: the underlying mechanism, which tends to bring the eyes back toward the desired trajectory, underperforms for small movements and overperforms for large movements. We also demonstrate that the performance of this compensatory mechanism, and the metrics of saccades in general, do not depend on the presence of visual information during the movement. By showing that deviations from the desired behavior are corrected during the saccade, our results further support the hypothesis that the innervation signal that generates saccadic eye movements is not pre-programmed but rather is dynamically adjusted during the movement. However, the compensation for deviations from the desired behavior is only partial, and the underlying mechanisms have yet to be completely understood. Although none of the current models of the saccadic system can account for our results, some of them, if appropriately modified, probably could.
You might find this additional info useful... 70 articles, 37 of which you can access for free at... more You might find this additional info useful... 70 articles, 37 of which you can access for free at: This article cites http://jn.physiology.org/content/87/2/845.full#ref-list-1 30 other HighWire-hosted articles: This article has been cited by http://jn.physiology.org/content/87/2/845#cited-by including high resolution figures, can be found at: Updated information and services http://jn.physiology.org/content/87/2/845.full can be found at: Journal of Neurophysiology about Additional material and information http://www.the-aps.org/publications/jn This information is current as of August 22, 2013. Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845-858, 2002; 10.1152/ jn.00317.2001. Many neurons in the frontal eye field (FEF) and lateral intraparietal (LIP) areas of cerebral cortex are active during the visualmotor events preceding the initiation of saccadic eye movements: they respond to visual targets, increase their activity before saccades, and maintain their activity during intervening delay periods. Previous experiments have shown that the output neurons from both LIP and FEF convey the full range of these activities to the superior colliculus (SC) in the brain stem. These areas of cerebral cortex also have strong interconnections, but what signals they convey remains unknown. To determine what these cortico-cortical signals are, we identified the LIP neurons that project to FEF by antidromic activation, and we studied their activity during a delayed-saccade task. We then compared these cortico-cortical signals to those sent subcortically by also identifying the LIP neurons that project to the intermediate layers of the SC. Of 329 FEF projection neurons and 120 SC projection neurons, none were co-activated by both FEF and SC stimulation. FEF projection neurons were encountered more superficially in LIP than SC projection neurons, which is consistent with the anatomical projection of many cortical layer III neurons to other cortical areas and of layer V neurons to subcortical structures. The estimated conduction velocities of FEF projection neurons (16.7 m/s) were significantly slower that those of SC projection neurons (21.7 m/s), indicating that FEF projection neurons have smaller axons. We identified three main differences in the discharge properties of FEF and SC projection neurons: only 44% of the FEF projection neurons changed their activity during the delayed-saccade task compared with 69% of the SC projection neurons; only 17% of the task-related FEF projection neurons showed saccadic activity, whereas 42% of the SC projection neurons showed such increases; 78% of the FEF projection neurons had a visual response but no saccadic activity, whereas only 55% of the SC projection neurons had similar activity. The FEF and SC projection neurons had three similarities: both had visual, delay, and saccadic activity, both had stronger delay and saccadic activity with visually guided than with memory-guided saccades, and both had broadly tuned responses for disparity stimuli, suggesting that their visual receptive fields have a three-dimensional configuration. These observations indicate that the activity carried between parietal and frontal cortical areas conveys a spectrum of signals but that the preponderance of activity conveyed might be more closely related to earlier visual processing than to the later saccadic stages that are directed to the SC.
The ability of primates to make rapid and accurate saccadic eye movements for exploring the natur... more The ability of primates to make rapid and accurate saccadic eye movements for exploring the natural world is based on a neuronal system in the brain that has been studied extensively and is known to include multiple brain regions extending throughout the neuraxis. We examined the characteristics of signal flow in this system by recording from identified output neurons of two cortical regions, the lateral intraparietal area (LIP) and the frontal eye field (FEF), and from neurons in a brainstem structure targeted by these output neurons, the superior colliculus (SC). We compared the activity of neurons in these three populations while monkeys performed a delayed saccade task that allowed us to quantify visual responses, motor activity, and intervening delay activity. We examined whether delay activity was related to visual stimulation by comparing the activity during interleaved trials when a target was either present or absent during the delay period. We examined whether delay activity was related to movement by using a Go/Nogo task and comparing the activity during interleaved trials in which a saccade was either made (Go) or not (Nogo). We found that LIP output neurons, FEF output neurons, and SC neurons can all have visual responses, delay activity, and presaccadic bursts; hence in this way they are all quite similar. However, the delay activity tended to be more related to visual stimulation in the cortical output neurons than in the SC neurons. Complementing this, the delay activity tended to be more related to movement in the SC neurons than in the cortical output neurons. We conclude, first, that the signal flow leaving the cortex represents activity at nearly every stage of visuomotor transformation, and second, that there is a gradual evolution of signal processing as one proceeds from cortex to colliculus. Published by Elsevier Science Ltd.
The ability of sensory-motor circuits to integrate sensory evidence over time is thought to under... more The ability of sensory-motor circuits to integrate sensory evidence over time is thought to underlie the process of decision-making in perceptual discrimination. Recent work has suggested that the NMDA receptor contributes to mediating neural activity integration. To test this hypothesis, we trained three female rhesus monkeys (Macaca mulatta) to perform a visual search task, in which they had to make a saccadic eye movement to the location of a target stimulus presented among distracter stimuli of lower luminance. We manipulated NMDA-receptor function by administering an intramuscular injection of the noncompetitive NMDA antagonist ketamine and assessed visual search performance before and after manipulation. Ketamine was found to lengthen response latency in a dose-dependent fashion. Surprisingly, it was also observed that response accuracy was significantly improved when lower doses were administered. These findings suggest that NMDA receptors play a crucial role in the process of decision-making in perceptual discrimination. They also further support the idea that multiple neural representations compete with one another through mutual inhibition, which may explain the speedaccuracy trade-off rule that shapes discrimination behavior: lengthening integration time helps resolve small differences between choice alternatives, thereby improving accuracy.
Social interaction involves the active visual perception of facial expressions and communicative ... more Social interaction involves the active visual perception of facial expressions and communicative gestures. This study examines the distribution of gaze fixations while watching videos of expressive talking faces. The knowledge-driven factors that influence the selective visual processing of facial information were examined by using the same set of stimuli, and assigning subjects to either a speech recognition task or an emotion judgment task. For half of the subjects assigned to each of the tasks, the intelligibility of the speech was manipulated by the addition of moderate masking noise. Both tasks and the intelligibility of the speech signal influenced the spatial distribution of gaze. Gaze was concentrated more on the eyes when emotion was being judged as compared to when words were being identified. When noise was added to the acoustic signal, gaze in both tasks was more centralized on the face. This shows that subject's gaze is sensitive to the distribution of information on the face, but can also be influenced by strategies aimed at maximizing the amount of visual information processed.
During face-to-face conversation the face provides auditory and visual linguistic information, an... more During face-to-face conversation the face provides auditory and visual linguistic information, and also conveys information about the identity of the speaker. This study investigated behavioral strategies involved in gathering visual information while watching talking faces. The effects of varying talker identity and varying the intelligibility of speech (by adding acoustic noise) on gaze behavior were measured with an eyetracker. Varying the intelligibility of the speech by adding noise had a noticeable effect on the location and duration of fixations. When noise was present subjects adopted a vantage point that was more centralized on the face by reducing the frequency of the fixations on the eyes and mouth and lengthening the duration of their gaze fixations on the nose and mouth. Varying talker identity resulted in a more modest change in gaze behavior that was modulated by the intelligibility of the speech. Although subjects generally used similar strategies to extract visual information in both talker variability conditions, when noise was absent there were more fixations on the mouth when viewing a different talker every trial as opposed to the same talker every trial. These findings provide a useful baseline for studies examining gaze behavior during audiovisual speech perception and perception of dynamic faces.
Communication impairments pose a major threat to an individual's quality of life. However, the im... more Communication impairments pose a major threat to an individual's quality of life. However, the impact of visual impairments on communication is not well understood, despite the important role that vision plays in the perception of speech. Here we present 2 experiments examining the impact of discrete central scotomas on speech perception. In the first experiment, 4 patients with central vision loss due to unilateral macular holes identified utterances with conflicting auditory-visual information, while simultaneously having their eye movements recorded. Each eye was tested individually. Three participants showed similar speech perception with both the impaired eye and the unaffected eye. For 1 participant, speech perception was disrupted by the scotoma because the participant did not shift gaze to avoid obscuring the talker's mouth with the scotoma. In the second experiment, 12 undergraduate students with gaze-contingent artificial scotomas (10 visual degrees in diameter) identified sentences in background noise. These larger scotomas disrupted speech perception, but some participants overcame this by adopting a gaze strategy whereby they shifted gaze to prevent obscuring important regions of the face such as the mouth. Participants who did not spontaneously adopt an adaptive gaze strategy did not learn to do so over the course of 5 days; however, participants who began with adaptive gaze strategies became more consistent in their gaze location. These findings confirm that peripheral vision is sufficient for perception of most visual information in speech, and suggest that training in gaze strategy may be worthwhile for individuals with communication deficits due to visual impairments.
Journal of The Acoustical Society of America, 2001
In natural conversation, visual and auditory information about speech not only provide linguistic... more In natural conversation, visual and auditory information about speech not only provide linguistic information but also provide information about the identity and the emotional state of the speaker. Thus, listeners must process a wide range of information in parallel to understand the full meaning in a message. In this series of studies, we examined how different types of visual information conveyed by a speaker's face are processed by measuring the gaze patterns exhibited by subjects watching audiovisual recordings of spoken sentences. In three experiments, subjects were asked to judge the emotion and the identity of the speaker, and to report the words that they heard under different auditory conditions. As in previous studies, eye and mouth regions dominated the distribution of the gaze fixations. It was hypothesized that the eyes would attract more fixations for more social judgment tasks, rather than tasks which rely more on verbal comprehension. Our results support this hypothesis. In addition, the location of gaze on the face did not influence the accuracy of the perception of speech in noise.
We conducted three experiments in order to examine the influence of gaze behavior and fixation on... more We conducted three experiments in order to examine the influence of gaze behavior and fixation on audiovisual speech perception in a task that required subjects to report the speech sound they perceived during the presentation of congruent and incongruent (McGurk) audiovisual stimuli. Experiment 1 showed that the subjects’ natural gaze behavior rarely involved gaze fixations beyond the oral and ocular regions of the talker’s face and that these gaze fixations did not predict the likelihood of perceiving the McGurk effect. Experiments 2 and 3 showed that manipulation of the subjects’ gaze fixations within the talker’s face did not influence audiovisual speech perception substantially and that it was not until the gaze was displaced beyond 10°–20° from the talker’s mouth that the McGurk effect was significantly lessened. Nevertheless, the effect persisted under such eccentric viewing conditions and became negligible only when the subject’s gaze was directed 60° eccentrically. These findings demonstrate that the analysis of high spatial frequency information afforded by direct oral foveation isnot necessary for the successful processing of visual speech information.
Two long-standing questions in neuroscience concern the mechanisms underlying our abilities to ma... more Two long-standing questions in neuroscience concern the mechanisms underlying our abilities to make decisions and to store goal-relevant information in memory for seconds at a time. Recent experimental and theoretical advances suggest that NMDA receptors at intrinsic cortical synapses play an important role in both these functions. The long NMDA time constant is suggested to support persistent mnemonic activity by maintaining excitatory drive after the removal of a stimulus and to enable the slow integration of afferent information in the service of decisions. These findings have led to the hypothesis that the local circuit mechanisms underlying decisions must also furnish persistent storage of information. We use a local circuit cortical model of spiking neurons to test this hypothesis, controlling intrinsic drive by scaling NMDA conductance strength. Our simulations provide further evidence that persistent storage and decision making are supported by common mechanisms, but under biophysically realistic parameters, our model demonstrates that the processing requirements of persistent storage and decision making may be incompatible at the local circuit level. Parameters supporting persistent storage lead to strong dynamics that are at odds with slow integration, whereas weaker dynamics furnish the speed–accuracy trade-off common to psychometric data and decision theory.► We model a local cortical circuit participating in a perceptual decision. ► Network dynamics are controlled by NMDA receptor conductance at intrinsic synapses. ► Dynamics that support persistent mnemonic activity lead to poor decision making. ► Weaker dynamics support decision making that is consistent with experimental data.
The superior colliculus has long been recognized as an important structure in the generation of s... more The superior colliculus has long been recognized as an important structure in the generation of saccadic displacements of the visual axis. Neurons with presaccadic activity encoding saccade vectors are topographically organized and form a “motor map.” Recently, neurons with fixation-related activity have been recorded at the collicular rostral pole, at the area centralis representation or fixation area. Another collicular function which deals with the maintenance of fixation behavior by means of active inhibition of orientation commands was then suggested. We tested that hypothesis as it relates to the suppression of gaze saccades (gaze = eye in space = eye in head + head in space) in the head-free cat by increasing the activity of the fixation cells at the rostral pole with electrical microstimulation. Long stimulation trains applied before gaze saccades delayed their initiation. Short stimuli, delivered during the gaze saccades, transiently interrupted both eye and head components. These results provide further support for a role in fixation behavior for collicular fixation neurons. Brainstem omnipause neurons also exhibit fixation-related activity and have been shown to receive a direct excitatory input from the superior colliculus. To determine whether the collicular projection to omnipause neurons arises from the fixation area, the deep layers of the superior colliculus were electrically stimulated either at the rostral pole including the fixation area or in more caudal regions where stimulation evokes orienting responses. Forty-nine neurons were examined in three cats. 61% of the neurons were found to be orthodromically excited by single-pulse stimulation of the rostral pole, whereas only 29% responded to caudal stimulation. In addition, stimuli delivered to the rostral pole activated, on average, omnipause neurons at shorter latencies and with lower currents than those applied in caudal regions. These results suggest that excitatory inputs to omnipause neurons from the superior colliculus are principally provided by the fixation area, via which the superior colliculus could play a role in suppression of gaze shifts.
In our previous paper we demonstrated that electrical microstimulation of the fixation area at th... more In our previous paper we demonstrated that electrical microstimulation of the fixation area at the rostral pole of the cat superior colliculus (SC) elicits no gaze movement but, rather, transiently suppresses eye-head gaze saccades. In this paper, we investigated the more caudal region of the SC and its interaction with the fixation area. In the alert head-free cat, supra-threshold stimulation in the anterior portion of the SC but outside the fixation area evoked small saccadic shifts of gaze consisting mainly of an eye movement, the head's contribution being small. Stimulating more posteriorly elicited large gaze saccades consisting of an ocular saccade combined with a rapid head movement. At these latter stimulation sites, craniocentric (goal-directed) eye movements were evoked when the cat's head was restrained. The amplitude of eye-head gaze saccades elicited at a particular stimulation site increased with stimulus duration, current strength, and pulse rate, until a constant or “unit” value was reached. The peak velocity of gaze shifts depended on both pulse rate and current strength. The movement direction was not affected by stimulus parameters. The unit gaze vector evoked, in the head-free condition, by stimulating one collicular site was similar to that coded by efferent neurons recorded at that site, thereby indicating a retinotopically coded gaze error representation on the collicular motor map which is not revealed by stimulating the head-fixed animal. Evoked gaze saccades were found to be influenced by fixation behavior. The amplitude of evoked gaze shifts was reduced if stimulation occurred when the hungry animal fixated a food target. Electrical activation of the collicular fixation area was found to mimic well the effects of natural fixation on evoked gaze shifts. Taken together, our results support the view that the overall distribution and level of collicular activity contributes to the encoding of the metrics of gaze saccades. We suggest that the combined levels of activity at the site being stimulated and at the fixation area influence the amplitude of evoked gaze saccades through competition. When stimulation is at low intensities, fixation-related activity reduces the amplitude of evoked gaze saccades. At high activation levels, the site being stimulated dominates and the gaze vector is specified only by that site's collicular output neurons, from which arises the close correspondence between the unit-evoked gaze saccades and the neurally coded gaze vector at that site.
The ability of primates to make rapid and accurate saccadic eye movements for exploring the natur... more The ability of primates to make rapid and accurate saccadic eye movements for exploring the natural world is based on a neuronal system in the brain that has been studied extensively and is known to include multiple brain regions extending throughout the neuraxis. We examined the characteristics of signal flow in this system by recording from identified output neurons of two cortical regions, the lateral intraparietal area (LIP) and the frontal eye field (FEF), and from neurons in a brainstem structure targeted by these output neurons, the superior colliculus (SC). We compared the activity of neurons in these three populations while monkeys performed a delayed saccade task that allowed us to quantify visual responses, motor activity, and intervening delay activity. We examined whether delay activity was related to visual stimulation by comparing the activity during interleaved trials when a target was either present or absent during the delay period. We examined whether delay activity was related to movement by using a Go/Nogo task and comparing the activity during interleaved trials in which a saccade was either made (Go) or not (Nogo). We found that LIP output neurons, FEF output neurons, and SC neurons can all have visual responses, delay activity, and presaccadic bursts; hence in this way they are all quite similar. However, the delay activity tended to be more related to visual stimulation in the cortical output neurons than in the SC neurons. Complementing this, the delay activity tended to be more related to movement in the SC neurons than in the cortical output neurons. We conclude, first, that the signal flow leaving the cortex represents activity at nearly every stage of visuomotor transformation, and second, that there is a gradual evolution of signal processing as one proceeds from cortex to colliculus. Published by Elsevier Science Ltd.
Buschman and Miller (Reports, 30 March 2007, p. 1860) described the activity of ensembles of neur... more Buschman and Miller (Reports, 30 March 2007, p. 1860) described the activity of ensembles of neurons in parietal and frontal cortex of monkeys performing visual search for targets that were easy or hard to distinguish from distractors. However, their conclusions are called into question by discrepancies between their results and publications from other laboratories measuring the same neural process.
We investigated and quantified the ability of the primate saccadic system to generate accurate ey... more We investigated and quantified the ability of the primate saccadic system to generate accurate eye movements in spite of naturally occurring variations in saccadic speed and trajectory. We show that the amplitude of a series of saccades directed to the same target is positively correlated to their peak speed, i.e., the faster the saccade, the bigger its amplitude. We demonstrate that this result cannot be simply accounted for by the main sequence, and that on average the saccadic system is able to compensate for only 61% of the variability in speed. Deviations from the average trajectory are also only partially compensated: the underlying mechanism, which tends to bring the eyes back toward the desired trajectory, underperforms for small movements and overperforms for large movements. We also demonstrate that the performance of this compensatory mechanism, and the metrics of saccades in general, do not depend on the presence of visual information during the movement. By showing that deviations from the desired behavior are corrected during the saccade, our results further support the hypothesis that the innervation signal that generates saccadic eye movements is not pre-programmed but rather is dynamically adjusted during the movement. However, the compensation for deviations from the desired behavior is only partial, and the underlying mechanisms have yet to be completely understood. Although none of the current models of the saccadic system can account for our results, some of them, if appropriately modified, probably could.
Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye ... more Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845-858, 2002; 10.1152/ jn.00317.2001. Many neurons in the frontal eye field (FEF) and lateral intraparietal (LIP) areas of cerebral cortex are active during the visualmotor events preceding the initiation of saccadic eye movements: they respond to visual targets, increase their activity before saccades, and maintain their activity during intervening delay periods. Previous experiments have shown that the output neurons from both LIP and FEF convey the full range of these activities to the superior colliculus (SC) in the brain stem. These areas of cerebral cortex also have strong interconnections, but what signals they convey remains unknown. To determine what these cortico-cortical signals are, we identified the LIP neurons that project to FEF by antidromic activation, and we studied their activity during a delayed-saccade task. We then compared these cortico-cortical signals to those sent subcortically by also identifying the LIP neurons that project to the intermediate layers of the SC. Of 329 FEF projection neurons and 120 SC projection neurons, none were co-activated by both FEF and SC stimulation. FEF projection neurons were encountered more superficially in LIP than SC projection neurons, which is consistent with the anatomical projection of many cortical layer III neurons to other cortical areas and of layer V neurons to subcortical structures. The estimated conduction velocities of FEF projection neurons (16.7 m/s) were significantly slower that those of SC projection neurons (21.7 m/s), indicating that FEF projection neurons have smaller axons. We identified three main differences in the discharge properties of FEF and SC projection neurons: only 44% of the FEF projection neurons changed their activity during the delayed-saccade task compared with 69% of the SC projection neurons; only 17% of the task-related FEF projection neurons showed saccadic activity, whereas 42% of the SC projection neurons showed such increases; 78% of the FEF projection neurons had a visual response but no saccadic activity, whereas only 55% of the SC projection neurons had similar activity. The FEF and SC projection neurons had three similarities: both had visual, delay, and saccadic activity, both had stronger delay and saccadic activity with visually guided than with memory-guided saccades, and both had broadly tuned responses for disparity stimuli, suggesting that their visual receptive fields have a three-dimensional configuration. These observations indicate that the activity carried between parietal and frontal cortical areas conveys a spectrum of signals but that the preponderance of activity conveyed might be more closely related to earlier visual processing than to the later saccadic stages that are directed to the SC.
We investigated and quantified the ability of the primate saccadic system to generate accurate ey... more We investigated and quantified the ability of the primate saccadic system to generate accurate eye movements in spite of naturally occurring variations in saccadic speed and trajectory. We show that the amplitude of a series of saccades directed to the same target is positively correlated to their peak speed, i.e., the faster the saccade, the bigger its amplitude. We demonstrate that this result cannot be simply accounted for by the main sequence, and that on average the saccadic system is able to compensate for only 61% of the variability in speed. Deviations from the average trajectory are also only partially compensated: the underlying mechanism, which tends to bring the eyes back toward the desired trajectory, underperforms for small movements and overperforms for large movements. We also demonstrate that the performance of this compensatory mechanism, and the metrics of saccades in general, do not depend on the presence of visual information during the movement. By showing that deviations from the desired behavior are corrected during the saccade, our results further support the hypothesis that the innervation signal that generates saccadic eye movements is not pre-programmed but rather is dynamically adjusted during the movement. However, the compensation for deviations from the desired behavior is only partial, and the underlying mechanisms have yet to be completely understood. Although none of the current models of the saccadic system can account for our results, some of them, if appropriately modified, probably could.
You might find this additional info useful... 70 articles, 37 of which you can access for free at... more You might find this additional info useful... 70 articles, 37 of which you can access for free at: This article cites http://jn.physiology.org/content/87/2/845.full#ref-list-1 30 other HighWire-hosted articles: This article has been cited by http://jn.physiology.org/content/87/2/845#cited-by including high resolution figures, can be found at: Updated information and services http://jn.physiology.org/content/87/2/845.full can be found at: Journal of Neurophysiology about Additional material and information http://www.the-aps.org/publications/jn This information is current as of August 22, 2013. Comparison of cortico-cortical and cortico-collicular signals for the generation of saccadic eye movements. J Neurophysiol 87: 845-858, 2002; 10.1152/ jn.00317.2001. Many neurons in the frontal eye field (FEF) and lateral intraparietal (LIP) areas of cerebral cortex are active during the visualmotor events preceding the initiation of saccadic eye movements: they respond to visual targets, increase their activity before saccades, and maintain their activity during intervening delay periods. Previous experiments have shown that the output neurons from both LIP and FEF convey the full range of these activities to the superior colliculus (SC) in the brain stem. These areas of cerebral cortex also have strong interconnections, but what signals they convey remains unknown. To determine what these cortico-cortical signals are, we identified the LIP neurons that project to FEF by antidromic activation, and we studied their activity during a delayed-saccade task. We then compared these cortico-cortical signals to those sent subcortically by also identifying the LIP neurons that project to the intermediate layers of the SC. Of 329 FEF projection neurons and 120 SC projection neurons, none were co-activated by both FEF and SC stimulation. FEF projection neurons were encountered more superficially in LIP than SC projection neurons, which is consistent with the anatomical projection of many cortical layer III neurons to other cortical areas and of layer V neurons to subcortical structures. The estimated conduction velocities of FEF projection neurons (16.7 m/s) were significantly slower that those of SC projection neurons (21.7 m/s), indicating that FEF projection neurons have smaller axons. We identified three main differences in the discharge properties of FEF and SC projection neurons: only 44% of the FEF projection neurons changed their activity during the delayed-saccade task compared with 69% of the SC projection neurons; only 17% of the task-related FEF projection neurons showed saccadic activity, whereas 42% of the SC projection neurons showed such increases; 78% of the FEF projection neurons had a visual response but no saccadic activity, whereas only 55% of the SC projection neurons had similar activity. The FEF and SC projection neurons had three similarities: both had visual, delay, and saccadic activity, both had stronger delay and saccadic activity with visually guided than with memory-guided saccades, and both had broadly tuned responses for disparity stimuli, suggesting that their visual receptive fields have a three-dimensional configuration. These observations indicate that the activity carried between parietal and frontal cortical areas conveys a spectrum of signals but that the preponderance of activity conveyed might be more closely related to earlier visual processing than to the later saccadic stages that are directed to the SC.
The ability of primates to make rapid and accurate saccadic eye movements for exploring the natur... more The ability of primates to make rapid and accurate saccadic eye movements for exploring the natural world is based on a neuronal system in the brain that has been studied extensively and is known to include multiple brain regions extending throughout the neuraxis. We examined the characteristics of signal flow in this system by recording from identified output neurons of two cortical regions, the lateral intraparietal area (LIP) and the frontal eye field (FEF), and from neurons in a brainstem structure targeted by these output neurons, the superior colliculus (SC). We compared the activity of neurons in these three populations while monkeys performed a delayed saccade task that allowed us to quantify visual responses, motor activity, and intervening delay activity. We examined whether delay activity was related to visual stimulation by comparing the activity during interleaved trials when a target was either present or absent during the delay period. We examined whether delay activity was related to movement by using a Go/Nogo task and comparing the activity during interleaved trials in which a saccade was either made (Go) or not (Nogo). We found that LIP output neurons, FEF output neurons, and SC neurons can all have visual responses, delay activity, and presaccadic bursts; hence in this way they are all quite similar. However, the delay activity tended to be more related to visual stimulation in the cortical output neurons than in the SC neurons. Complementing this, the delay activity tended to be more related to movement in the SC neurons than in the cortical output neurons. We conclude, first, that the signal flow leaving the cortex represents activity at nearly every stage of visuomotor transformation, and second, that there is a gradual evolution of signal processing as one proceeds from cortex to colliculus. Published by Elsevier Science Ltd.
The ability of sensory-motor circuits to integrate sensory evidence over time is thought to under... more The ability of sensory-motor circuits to integrate sensory evidence over time is thought to underlie the process of decision-making in perceptual discrimination. Recent work has suggested that the NMDA receptor contributes to mediating neural activity integration. To test this hypothesis, we trained three female rhesus monkeys (Macaca mulatta) to perform a visual search task, in which they had to make a saccadic eye movement to the location of a target stimulus presented among distracter stimuli of lower luminance. We manipulated NMDA-receptor function by administering an intramuscular injection of the noncompetitive NMDA antagonist ketamine and assessed visual search performance before and after manipulation. Ketamine was found to lengthen response latency in a dose-dependent fashion. Surprisingly, it was also observed that response accuracy was significantly improved when lower doses were administered. These findings suggest that NMDA receptors play a crucial role in the process of decision-making in perceptual discrimination. They also further support the idea that multiple neural representations compete with one another through mutual inhibition, which may explain the speedaccuracy trade-off rule that shapes discrimination behavior: lengthening integration time helps resolve small differences between choice alternatives, thereby improving accuracy.
Uploads
Papers by Martin Pare