IJCNN-91-Seattle International Joint Conference on Neural Networks
Working memory neural networks are characterized which encode the invariant temporal order of seq... more Working memory neural networks are characterized which encode the invariant temporal order of sequential events that may be presented at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such a competence is needed in neural architectures which self-organize learned codes for variable-rate speech perception, sensory-motor planning, or 3-D visual object recognition. Using such a working memory, a self-organizing architecture for invariant 3-D visual object recognition is described that is based on the model of Seibert and Waxman [1].
[Proceedings 1992] IJCNN International Joint Conference on Neural Networks
A working memory model is describ<~d that is capable of storing and recalling arbitrary temporal ... more A working memory model is describ<~d that is capable of storing and recalling arbitrary temporal sequences of events, including repeated items. These memories encode the invariant temporal order of sequential events that may be present<~d at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and re1nembered in real ti1ne, even as new events perturb the syste1n.
A theory is developed of how recognition categories can be learned in response to a temporal stre... more A theory is developed of how recognition categories can be learned in response to a temporal stream of input patterns. Interactions between an attentional subsystem and an orienting subsystem enable the network to self-stabilize its learning, without an external teacher, as the code becomes globally self-consistent. Category learning is thus determined by global contextual information in this system. The attentional subsystem learns bottomup codes and top-down templates, or expectancies. The internal representations formed in this way stabilize themselves against recoding by matching the learned top-down templates against input patterns. This matching process detects structural pattern properties in addition to local feature matches. The top-down templates can also suppress noise in the input patterns, and can subliminally prime the network to anticipate a set of input patterns. Mismatches activate an orienting subsystem, which resets incorrect codes and drives a rapid search for new or more appropriate codes. As the learned code becomes globally self-consistent, the orienting subsystem is automatically disengaged and the memory consolidates. After the recognition categories for a set of input patterns self-stabilize, those patterns directly access their categories without any search or recoding on future recognition trials. A novel pattern exemplar can directly access an established category if it shares invariant properties with the set of familiar exemplars of that category. Several attentional and nonspecific arousal mechanisms modulate the course of search and learning. Three types of attentional mechanism-priming, gain control, and vigilance-are distinguished. Three types of nonspecific arousal are also mechanistically characterized. The nonspecific vigilance process determines how fine the learned categories will be. If vigilance increases due, for example, to a negative reinforcement, then the system automatically searches for and learns finer recognition categories. The learned top-down expectancies become more abstract as the recognition categories become broader. The learned code is a property of network interactions and the entire history of input pattern presentations. The interactions generate emergent rules such as a Weber Law Rule, a 2/3 Rule, and an Associative Decay Rule. No serial programs or algorithmic rule structures are used. The interactions explain and predict properties of evoked potentials (processing negativity, mismatch negativity,-1
A neural network based cognitive architecture, known as Fusion Architecture for Learning and Cogn... more A neural network based cognitive architecture, known as Fusion Architecture for Learning and Cognition-eXtension (FALCON-X), is proposed herein, based on an integration of the Adaptive Control of Thought (ACT-R) cognitive architecture [1] and the fusion Adaptive Resonance Theory (fusion ART) neural model [2]. Fusion ART is a generalization of self-organizing neural models known as Adaptive Resonance Theory (ART). By extending the original ART model consisting of a single pattern field into a multi-channel architecture, fusion ART unifies a number of network designs as well as a myriad of learning paradigms, including unsupervised learning, supervised learning and reinforcement learning. While retaining the structure of the perceptual (visual), motor (manual), intentional and declarative memory modules, the proposed architecture replaces the central production system of ACT-R with a fusion ART neural network serving as the core inference area for fusing and updating the pattern activities in the four memory buffers. In addition, a critic module is incorporated that provides reinforcement signals to regulate the attentional and learning processes of the core inference area.
Adaptive Hesonance Theory (AR:I') is a neural theory of human and primate information processing ... more Adaptive Hesonance Theory (AR:I') is a neural theory of human and primate information processing and of adaptive pattern recognition and prediction for technology. Biological applications to attentive learning of visual recognition categories by inferotcrnporal cortex and hippocampal system, medial temporal amnesia., corticogeniculate synchronization, auditory streaming, speech recognition, and eye rnovement control arc noted. ART'MAP systems for technology integrate neural networks, fuzzy logic, and expert production systems to carry out both unsupervised and supervised learning. Fast and slow learning arc both stable response to large non stationary databases. Match tracking search conjointly maximizes learned compression while minimizing predictive error. Spatial and temporal evidence accumulation irnprove a.ccmacy in :J-D object recognition. Other applications are noted.
... without forgetting familiar ones? Gail A. Carpenter and Stephen Grossberg GAILCARPENTER o pro... more ... without forgetting familiar ones? Gail A. Carpenter and Stephen Grossberg GAILCARPENTER o professor of Cognitive and Neural Systems ICNS) and Mathematics at Boston University. STEPHEN GROSSBERG is Wang Professor ...
STABLE SELF-ORGANIZATION OF NEURAL RECOGNITION CODES FOR COMPLEX ENVIRONMENTS Many recent contrib... more STABLE SELF-ORGANIZATION OF NEURAL RECOGNITION CODES FOR COMPLEX ENVIRONMENTS Many recent contributions to dynamical system theory have analyzed how simple systems can generate surprisingly complex and unpredictable dynamics. Here we describe a class of dynamical systems that discovers order in a complex environment, and in so doing becomes more complex and predictive through time. The design of these dynamical systems is based upon an analysis of how humans discover, learn, and recognize invariant properties of the complex, fluctuating environments to which they are exposed. When such recognition codes spontaneously emerge through an individual's interaction with an environment, the processes are said to undergo selforganization.' This article reviews a theory of how recognition codes are self-organized by a class of neural networks whose qualitative features have been used to analyze data about speech perception, word recognition and recall, visual perception, olfactory coding, evoked potentials, thalamocortical interactions, attentional modulation of critical period termination, and amnesias.*-l' These networks are described by the adaptive resonance theory (ART), which was introduced by Grossberg.* This article presents a system of differential equations that completely characterizes one class of ART networks. The network is capable of self-organizing, selfstabilizing, and self-scaling its recognition codes in response to arbitrary temporal sequences of arbitrarily many input patterns of variable complexity. The model hereby illustrates how humans can spontaneously organize into coherent contexts the chaotic sensory inputs to which they are continuously exposed. Before proceeding to a description of this class of ART systems, we summarize some of their major properties and some scientific problems for which they provide a solution.
[Proceedings 1992] IJCNN International Joint Conference on Neural Networks
Page 1. FUZZY ARTMAP: AN ADAPTIVE RESONANCE ARCHITECTURE FOR INCREMENTAL LEARNING OF ANALOG MAPS ... more Page 1. FUZZY ARTMAP: AN ADAPTIVE RESONANCE ARCHITECTURE FOR INCREMENTAL LEARNING OF ANALOG MAPS Gail A. Carpentert, Stephen Grossberg$, Natalya Markuzons, John H. Reynoldsl, and David B. Rosena ...
Abstract : The Symposium on Neural Models of Sensory-Motor Control was held on August 19 and 20, ... more Abstract : The Symposium on Neural Models of Sensory-Motor Control was held on August 19 and 20, 1986, at Harvard University as part of the annual meeting of the Society for Mathematical Psychology. The Symposium was divided into two sessions, each with four 50-minute presentations. The first session focused on neural models of the human oculo-motor system. The second session began with two papers on arm movement planning. Speakers were not asked to prepare papers for a formal publication. Papers supplied by Bullock and Grossberg, by Grossbers, by Hollerback and Atkeson, by Kelso, and Scholz, and by Optician and Miles are enclosed.
IJCNN-91-Seattle International Joint Conference on Neural Networks
Working memory neural networks are characterized which encode the invariant temporal order of seq... more Working memory neural networks are characterized which encode the invariant temporal order of sequential events that may be presented at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such a competence is needed in neural architectures which self-organize learned codes for variable-rate speech perception, sensory-motor planning, or 3-D visual object recognition. Using such a working memory, a self-organizing architecture for invariant 3-D visual object recognition is described that is based on the model of Seibert and Waxman [1].
[Proceedings 1992] IJCNN International Joint Conference on Neural Networks
A working memory model is describ<~d that is capable of storing and recalling arbitrary temporal ... more A working memory model is describ<~d that is capable of storing and recalling arbitrary temporal sequences of events, including repeated items. These memories encode the invariant temporal order of sequential events that may be present<~d at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and re1nembered in real ti1ne, even as new events perturb the syste1n.
A theory is developed of how recognition categories can be learned in response to a temporal stre... more A theory is developed of how recognition categories can be learned in response to a temporal stream of input patterns. Interactions between an attentional subsystem and an orienting subsystem enable the network to self-stabilize its learning, without an external teacher, as the code becomes globally self-consistent. Category learning is thus determined by global contextual information in this system. The attentional subsystem learns bottomup codes and top-down templates, or expectancies. The internal representations formed in this way stabilize themselves against recoding by matching the learned top-down templates against input patterns. This matching process detects structural pattern properties in addition to local feature matches. The top-down templates can also suppress noise in the input patterns, and can subliminally prime the network to anticipate a set of input patterns. Mismatches activate an orienting subsystem, which resets incorrect codes and drives a rapid search for new or more appropriate codes. As the learned code becomes globally self-consistent, the orienting subsystem is automatically disengaged and the memory consolidates. After the recognition categories for a set of input patterns self-stabilize, those patterns directly access their categories without any search or recoding on future recognition trials. A novel pattern exemplar can directly access an established category if it shares invariant properties with the set of familiar exemplars of that category. Several attentional and nonspecific arousal mechanisms modulate the course of search and learning. Three types of attentional mechanism-priming, gain control, and vigilance-are distinguished. Three types of nonspecific arousal are also mechanistically characterized. The nonspecific vigilance process determines how fine the learned categories will be. If vigilance increases due, for example, to a negative reinforcement, then the system automatically searches for and learns finer recognition categories. The learned top-down expectancies become more abstract as the recognition categories become broader. The learned code is a property of network interactions and the entire history of input pattern presentations. The interactions generate emergent rules such as a Weber Law Rule, a 2/3 Rule, and an Associative Decay Rule. No serial programs or algorithmic rule structures are used. The interactions explain and predict properties of evoked potentials (processing negativity, mismatch negativity,-1
A neural network based cognitive architecture, known as Fusion Architecture for Learning and Cogn... more A neural network based cognitive architecture, known as Fusion Architecture for Learning and Cognition-eXtension (FALCON-X), is proposed herein, based on an integration of the Adaptive Control of Thought (ACT-R) cognitive architecture [1] and the fusion Adaptive Resonance Theory (fusion ART) neural model [2]. Fusion ART is a generalization of self-organizing neural models known as Adaptive Resonance Theory (ART). By extending the original ART model consisting of a single pattern field into a multi-channel architecture, fusion ART unifies a number of network designs as well as a myriad of learning paradigms, including unsupervised learning, supervised learning and reinforcement learning. While retaining the structure of the perceptual (visual), motor (manual), intentional and declarative memory modules, the proposed architecture replaces the central production system of ACT-R with a fusion ART neural network serving as the core inference area for fusing and updating the pattern activities in the four memory buffers. In addition, a critic module is incorporated that provides reinforcement signals to regulate the attentional and learning processes of the core inference area.
Adaptive Hesonance Theory (AR:I') is a neural theory of human and primate information processing ... more Adaptive Hesonance Theory (AR:I') is a neural theory of human and primate information processing and of adaptive pattern recognition and prediction for technology. Biological applications to attentive learning of visual recognition categories by inferotcrnporal cortex and hippocampal system, medial temporal amnesia., corticogeniculate synchronization, auditory streaming, speech recognition, and eye rnovement control arc noted. ART'MAP systems for technology integrate neural networks, fuzzy logic, and expert production systems to carry out both unsupervised and supervised learning. Fast and slow learning arc both stable response to large non stationary databases. Match tracking search conjointly maximizes learned compression while minimizing predictive error. Spatial and temporal evidence accumulation irnprove a.ccmacy in :J-D object recognition. Other applications are noted.
... without forgetting familiar ones? Gail A. Carpenter and Stephen Grossberg GAILCARPENTER o pro... more ... without forgetting familiar ones? Gail A. Carpenter and Stephen Grossberg GAILCARPENTER o professor of Cognitive and Neural Systems ICNS) and Mathematics at Boston University. STEPHEN GROSSBERG is Wang Professor ...
STABLE SELF-ORGANIZATION OF NEURAL RECOGNITION CODES FOR COMPLEX ENVIRONMENTS Many recent contrib... more STABLE SELF-ORGANIZATION OF NEURAL RECOGNITION CODES FOR COMPLEX ENVIRONMENTS Many recent contributions to dynamical system theory have analyzed how simple systems can generate surprisingly complex and unpredictable dynamics. Here we describe a class of dynamical systems that discovers order in a complex environment, and in so doing becomes more complex and predictive through time. The design of these dynamical systems is based upon an analysis of how humans discover, learn, and recognize invariant properties of the complex, fluctuating environments to which they are exposed. When such recognition codes spontaneously emerge through an individual's interaction with an environment, the processes are said to undergo selforganization.' This article reviews a theory of how recognition codes are self-organized by a class of neural networks whose qualitative features have been used to analyze data about speech perception, word recognition and recall, visual perception, olfactory coding, evoked potentials, thalamocortical interactions, attentional modulation of critical period termination, and amnesias.*-l' These networks are described by the adaptive resonance theory (ART), which was introduced by Grossberg.* This article presents a system of differential equations that completely characterizes one class of ART networks. The network is capable of self-organizing, selfstabilizing, and self-scaling its recognition codes in response to arbitrary temporal sequences of arbitrarily many input patterns of variable complexity. The model hereby illustrates how humans can spontaneously organize into coherent contexts the chaotic sensory inputs to which they are continuously exposed. Before proceeding to a description of this class of ART systems, we summarize some of their major properties and some scientific problems for which they provide a solution.
[Proceedings 1992] IJCNN International Joint Conference on Neural Networks
Page 1. FUZZY ARTMAP: AN ADAPTIVE RESONANCE ARCHITECTURE FOR INCREMENTAL LEARNING OF ANALOG MAPS ... more Page 1. FUZZY ARTMAP: AN ADAPTIVE RESONANCE ARCHITECTURE FOR INCREMENTAL LEARNING OF ANALOG MAPS Gail A. Carpentert, Stephen Grossberg$, Natalya Markuzons, John H. Reynoldsl, and David B. Rosena ...
Abstract : The Symposium on Neural Models of Sensory-Motor Control was held on August 19 and 20, ... more Abstract : The Symposium on Neural Models of Sensory-Motor Control was held on August 19 and 20, 1986, at Harvard University as part of the annual meeting of the Society for Mathematical Psychology. The Symposium was divided into two sessions, each with four 50-minute presentations. The first session focused on neural models of the human oculo-motor system. The second session began with two papers on arm movement planning. Speakers were not asked to prepare papers for a formal publication. Papers supplied by Bullock and Grossberg, by Grossbers, by Hollerback and Atkeson, by Kelso, and Scholz, and by Optician and Miles are enclosed.
Uploads
Papers by S. Grossberg