Tutorial on
Neural Systems Modeling
Thomas J. Anastasio
Sinauer Associates Inc. Publishers Sunderland, Massachusetts U.S.A.
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
Brief Contents
CHapter 1
Vectors, Matrices, and Basic Neural Computations 1
CHapter 2
Recurrent Connections and Simple Neural Circuits 27 Forward and Recurrent Lateral Inhibition 65 Covariation Learning and Auto-Associative Memory 97 Unsupervised Learning and Distributed Representations 135 Supervised Learning and Non-Uniform Representations 171 Reinforcement Learning and Associative Conditioning 213 Information Transmission and Unsupervised Learning 251 Probability Estimation and Supervised Learning 299
CHapter 3 CHapter 4 CHapter 5 CHapter 6 CHapter 7 CHapter 8 CHapter 9
CHapter 10 CHapter 11 CHapter 12 CHapter 13 CHapter 14
Time Series Learning and Nonlinear Signal Processing 341
Temporal-Difference Learning and Reward Prediction 387 PredictorCorrector Models and Probabilistic Inference 423 Simulated Evolution and the Genetic Algorithm 481
Future Directions in Neural Systems Modeling 515
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
Contents
CHapter 1 Vectors, Matrices, and Basic Neural Computations 1
1.1 Neural Systems, Neural Networks, and Brain Function 2 1.2 Using MATLAB: The Matrix Laboratory Programming Environment 3 1.3 Imitating the Solution by Gauss to the Busywork Problem 4 1.4 Operation and Habituation of the Gill Withdrawal Reflex of Aplysia 7
MAth Box 1.1 Geometric decay of the discrete
pulse response 16
1.6 Neural Networks: Neural Systems with Multiple Interconnected Units 18
Math Box 1.2 Vector and matrix
multiplication: Across the row and down the column 19
Matlab Box 1.3 This script simulates the pulse or
Matlab Box 1.1 This script implements a simple
simulation of habituation of the Aplysia gill-withdrawal reflex 12
step response of a neural network with ten input and ten output units that are randomly connected 23
Exercises 24 References 25
1.5The Dynamics of a Single Neural Unit with Positive Feedback 13
Matlab Box 1.2 This script simulates the pulse
or step response of a single neuron with positive feedback 15
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
viii C o n t e n t s
CHapter 2 Recurrent Connections and Simple Neural Circuits 27
Math Box 2.1 the geometric decay is the discretetime analog of the exponential decay 29
Matlab Box 2.2 The script implements the parallel-
pathway and positive-feedback models of velocity storage, and the negative-feedback model of velocity leakage 42
Math Box 2.2 Discrete approximation of a
continuous differential equation model of a single neuron 30
2.4 The Positive-Feedback Model of Velocity Storage in the Primate VOR 44 2.5 The Negative-Feedback Model of Velocity Leakage in the Pigeon VOR 46 2.6 Oculomotor Neural Integration via Reciprocal Inhibition 48
2.1 The Dynamics of Two Neural Units with Feedback in Series 30
Matlab Box 2.1 This script implements a model having
two units in series, each with recurrent, excitatory selfconnections allowing the units to exert positive feedback on themselves (the two units are leaky integrators) 32
Matlab Box 2.3 The script implements the two-unit model of the integrator of the oculomotor system
2.7 Simulating the Insect-Flight Central Pattern Generator 53
Math Box 2.3 Discrete approximation of a system
of two coupled, continuous differential equations that model two interconnected neurons 34
Math Box 2.4 Eigenmode analysis of discrete,
linear dynamic systems 35
Matlab Box 2.4 The script implements a linear version
of Wilsons model of the locust-flight central pattern generator 56
2.2 Signal Processing in the Vestibulo-Ocular Reflex (VOR) 38 2.3 The Parallel-Pathway Model of Velocity Storage in the Primate VOR 40
Exercises 60 References 61
CHapter 3 Forward and Recurrent Lateral Inhibition 65
3.1 Simulating Edge Detection in the Early Visual System of Limulus 69
Matlab Box 3.2 This function will make a Gaussian
connectivity profile 81
Matlab Box 3.1 This function will make a connectivity
matrix by laminating shifted versions of a connectivity profile 72
3.3 Simulating Activity Bubbles and Stable Pattern Formation 85
Math Box 3.4 summation notation 86
take-all network that has recurrent, lateral inhibitory and central excitatory connections and nonlinear units 87
Math Box 3.1 Finite (discrete) difference Math Box 3.2 The Marr filter 79
approximations t0 continuous derivatives 74
Matlab Box 3.3 This script implements the winners-
Math Box 3.3 One- and two-dimensional Gabor
functions 80
3.4 Separating Signals from Noise and Modeling Target Selection in the Superior Colliculus 89
Exercises 94 References 95
3.2 Simulating Center/Surround Receptive Fields Using the Difference of Gaussians 81
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
c o n t e n t s ix
CHapter 4 Covariation Learning and Auto-Associative Memory 97
Math Box 4.1 Covariation and its relationship
with covariance 100
4.3 Recalling Distinct Memories Using Negative Connections in Auto-Associators 116 4.4 Synchronous versus Asynchronous Updating in Recurrent Auto-Associators 118
4.1 The Four Hebbian Learning Rules for Neural Networks 101
Matlab Box 4.1 This script implements three different
methods to make connection weight matrices for recurrent auto-associative networks using the Hebb, post-synaptic, pre-synaptic, and Hopfield (covariation) rules 104
Matlab Box 4.3 This function computes asynchronous
updates of recurrent auto-associative networks. The state of only one unit, chosen at random, is updated on each time step 120
Math Box 4.2 The outer product of two
vectors 106
4.5 Graceful Degradation and Simulated Forgetting 121 4.6 Simulating Storage and Recall of a Sequence of Patterns 124
Math Box 4.3 Matrix multiplication: across the row and down the column 108
4.2 Simulating Memory Recall Using Recurrent Auto-Associator Networks 109
Math Box 4.4 Liapunov functions 112
Matlab Box 4.4 This script makes asymmetric Hopfield
connection weight matrices for sequence recall in recurrent auto-associative neural networks 124
Matlab
Box 4.2 This function computes synchronous updates of recurrent auto-associative networks. The entire state vector is updated on each time step 113
4.7 Hebbian Learning, Recurrent Auto-Association, and Models of Hippocampus 126
Exercises 131 References 133
CHapter 5 Unsupervised Learning and Distributed Representations 135
Math Box 5.1 Vector norms, normalization, and
inner products revisited 141
5.4 Modeling the Formation of Tonotopic Maps in the Auditory System 153
5.1 Learning through Competition to Specialize for Specific Inputs 142
Matlab Box 5.1 This script implements the Kohonen self-organizing map (SOM) algorithm 143
5.2 Training Few Output Neurons to Represent Many Input Patterns 145
output responses to sets of input patterns as black-andwhite images 146
Matlab Box 5.3 This script uses the Kohonen SOM algorithm to simulate the formation of a tonotopic map 156
5.5 Simulating the Development of Orientation Selectivity in Visual Cortex 158 5.6 Modeling a Possible Multisensory Map in the Superior Colliculus 164
Exercises 167 References 168
Matlab Box 5.2 This script can be used to visualize SOM
5.3 Simulating the Formation of Brain Maps using Cooperative Mechanisms 148
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
x C o n t e n t s
CHapter 6 Supervised Learning and Non-Uniform Representations 171
6.1 Using the Classic Hebb Rule to Learn a Simple Labeled Line Response 173 6.4 Learning Interneuronal Representations using Back-Propagation 189
Math Box 6.4 Multilayered networks of linear
processing units 190
Matlab Box 6.1 This script trains two-layered networks of binary units to associate patterns using the Hebb rule 175
6.2 Learning a Simple Contingency Using the Covariation Rule 178 6.3 Using the Delta Rule to Learn a Complex Contingency 180
Math Box 6.1 The gradient of vector
calculus 181
Math Box 6.5 Derivation of the back-propagation
learning algorithm 191
Matlab Box 6.3 This script trains three-layered networks of sigmoidal units to associate patterns using backpropagation 194
6.5 Simulating Catastrophic Retroactive Interference in Learning 196 6.6 Simulating the Development of Non-Uniform Distributed Representations 198 6.7 Modeling Non-Uniform Distributed Representa tions in the Vestibular Nuclei 201
Exercises 209 References 210
Math Box 6.2 Derivation of the delta rule learning algorithm 183 Math Box 6.3 Differentiating the squashing
function 184
Matlab Box 6.2 This script trains two-layered networks of sigmoidal units to associate patterns using the delta rule 186
CHapter 7 Reinforcement Learning and Associative Conditioning 213
7.1 Learning the Labeled-Line Task via Perturbation of One Weight at a Time 216 7.4 Reinforcement Learning and Non-Uniform Distributed Representations 230
Matlab Box 7.1 This script trains two-layered networks of sigmoidal units (with only one output unit) to associate patterns using perturbation to estimate the error gradient at one weight at a time 220
7.2 Perturbing All Weights Simultaneously and the Importance of Structure 221
Math Box 7.1 the Brownian motion
algorithm 222
Matlab Box 7.4 This script trains three-layered networks of sigmoidal units to develop a distributed representation using parallel, perturbative reinforcement learning 232
7.5 Reinforcement in a Schema Model of Avoidance Conditioning 234
Matlab Box 7.5 This script simulates avoidance conditioning as reinforcement learning 239
7.6 Exploration and Exploitation in a Model of Avoidance Conditioning 242
networks of sigmoidal units to associate patterns using perturbation to estimate the error gradient at all weights simultaneously 223
Matlab Box 7.2 This script trains two-layered
7.3 Plausible Weight Modification using Perturbative Reinforcement Learning 227
of sigmoidal units to associate patterns by perturbing all weights simultaneously 228
Matlab Box 7.6 This script simulates avoidance conditioning as reinforcement learning with automatic adjustment of exploration 244
Exercises 247 References 248
Matlab Box 7.3 This script trains two-layered networks
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
c o n t e n t s xi
CHapter 8 Information Transmission and Unsupervised Learning 251
8.1 Some Basic Concepts in Information Theory 253 8.2 Measuring Information Transmission through a Neural Network 257 8.4 Information Transmission and Competitive Learning in Neural Networks 276
Matlab Box 8.1 This script computes inputoutput mutual information in a feedforward network with two stochastic, binary input units and two deterministic, binary output units 258 Matlab Box 8.2 This function computes the input and output entropy and the inputoutput mutual information for a neural network 261
8.3 Maximizing Information Transmission in a Neural Network 265
Math Box 8.1 Derivation of the BellSejnowski
infomax algorithm for A network with one input unit and one output unit 266
Matlab Box 8.4 This script trains a 2-by-2, feedforward network using competitive, unsupervised learning, and finds the inputoutput mutual information 277
8.5 Information Transmission in Self-Organized Map Networks 279
Matlab Box 8.5 This script trains a self-organizing map network and finds informational measures, including estimated distortion, following training 281
Math Box 8.2 Rate-distortion theory 284
8.6 Information Transmission in Stochastic Neural Networks 290
network using the BellSejnowski infomax algorithm, and finds the inputoutput mutual information 268
Matlab Box 8.3 This script trains a 2-by-2, feedforward
Matlab Box 8.6 This script computes informational measures for a stochastic network of binary units with one input unit and arbitrarily many output units 293
Exercises 296 References 298
CHapter 9 Probability Estimation and Supervised Learning 299
9.1 Implementing a Simple Classifier as a Three Layered Neural Network 302
Matlab Box 9.1 This script trains a three-layered network of sigmoidal units using back-propagation to classify fish according to their lengths 303
Math Box 9.1 Univariate, multivariate, and bivariate Gaussian distributions 305
probability of a target given sensory input of one modality (i.e., visual) 317
Matlab Box 9.3 This script computes the posterior
Matlab Box 9.4 This script trains a single sigmoidal unit using the delta rule to estimate posterior target probability given sensory input of one modality (i.e., visual) 319
Math Box 9.3 Solving for the input and bias
weights of a sigmoidal unit that computes a posterior probability 321
9.2 Predicting Rain as an Everyday Example of Probabilistic Inference 308
Math Box 9.2 The derivation of Bayes rule 309
9.5 Modeling Multisensory Collicular Neurons as Probability Estimators 323
9.3 Implementing a Simple Classifier Using Bayes Rule 311
Matlab Box 9.5 This script computes the posterior
probability of a target given sensory input of two modalities (i.e., visual and auditory) 328
Matlab Box 9.2 This script computes the posterior probabilities of each of three hypothetical fish classes using Bayes rule 313
9.4 Modeling Neural Responses to Sensory Input as Probabilistic Inference 315
Matlab Box 9.6 This script trains a single sigmoidal unit using the delta rule to estimate posterior target probability given sensory input of two modalities (i.e., visual and auditory) 330
Exercises 338 References 339
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
xii C o n t e n t s
CHapter 10 Time Series Learning and Nonlinear Signal Processing 341
10.1 Training Connection Weights in Recurrent Neural Networks 344
Math Box 10.1 Derivation of real-time recurrent back-propagation 345
10.5 Training Networks of Nonlinear Units to Produce Velocity Storage 363
10.2 Training a Two-Unit Network to Simulate the Oculomotor Neural Integrator 348
Matlab Box 10.3 This script uses recurrent backpropagation to train a nonlinear recurrent network to perform velocity storage (Training requires several minutes) 364
10.6 Training a Recurrent Neural Network to Simulate Short-Term Memory 374
Matlab Box 10.1 This script uses recurrent backpropagation to train a recurrent network with two linear output units to act as a leaky integrator 351
10.3 Velocity Storage in the Vestibulo-Ocular Reflex 354 10.4 Training a Network of Linear Units to Produce Velocity Storage 357
propagation to train a linear recurrent network to perform velocity storage 360
Matlab Box 10.4 This script uses recurrent backpropagation to train a nonlinear recurrent network to simulate short-term memory (Training requires several minutes) 377 Matlab Box 10.5 This script tests the ability of recurrent neural networks to simulate short-term memory 378
Exercises 383 References 384
Matlab Box 10.2 This script uses recurrent back-
CHapter 11 Temporal-Difference Learning and Reward Prediction 387
gridworld 391
Matlab Box 11.1 This script sets up a stochastic
11.3 Learning State Values Using the Method of Temporal Differences 403
Math Box 11.1 Finding state values by solving a set of simultaneous linear equations 394
Matlab Box 11.4 This script updates state value estimates for the stochastic gridworld using temporaldifference learning 405
11.4 Simulating Dopamine Neuron Responses Using Temporal-Difference Learning 408
11.1 Learning State Values Using Iterative Dynamic Programming 395
estimates for the stochastic gridworld using iterative dynamic programming 396
Matlab Box 11.2 This script updates state value
Matlab Box 11.5 This script simulates the responses of midbrain dopamine neurons using temporal-difference learning 414
11.5 Temporal-Difference Learning as a Form of Supervised Learning 416
Exercises 419 References 420
11.2 Learning State Values Using Least Mean Squares 399
Matlab Box 11.3 This script updates state value estimates for the stochastic gridworld using least-meansquares learning 401
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
c o n t e n t s xiii
CHapter 12 PredictorCorrector Models and Probabilistic
Inference 423
12.3 A PredictorCorrector Model of Predictive Tracking by Midbrain Neurons 450
Math Box 12.3 decision-theoretic agent
design 454
Math Box 12.1 the Kalman filter 425
average 426
Matlab Box 12.1 This script implements a running
12.1 Modeling Visual System Direction Selectivity Using Asymmetric Inhibition 428
selectivity in the visual system 431
Matlab Box 12.2 This script simulates direction
predictorcorrector model 462
Matlab Box 12.5 This script sets up the target-tracking Matlab Box 12.6 This script implements a predictor
12.2 Modeling Visual Processing as Bottom-Up/ Top-Down Probabilistic Inference 434
top-down processing in the visual system using the joint distribution 442
corrector simulation of the responses of neurons in the parabigeminal nucleus 463
Matlab Box 12.3 This script simulates bottom-up/
12.4 Training a Sigmoidal Unit to Simulate Trajectory Prediction by Neurons 469
a single sigmoidal unit with feedback to simulate the responses of neurons in the parabigeminal nucleus 472
Matlab Box 12.7 This script uses the delta rule to train
Math Box 12.2 belief propagation 446
down processing in the visual system using probabilistic inference 447
Matlab Box 12.4 This script simulates bottom-up/top-
Exercises 475 References 477
CHapter 13 Simulated Evolution and the Genetic Algorithm 481
13.1 Simulating Genes and Genetic Operators 487 13.2 Exploring a Simple Example of Simulated Genetic Evolution 488
Math Box 13.1 The schema theorem for the
genetic algorithm 489
Matlab Box 13.3 This script trains a three-layered network of sigmoidal units to associate patterns using back-propagation 500
13.4 Evolving Optimal Learning Rules for Auto Associative Memories 502
Math Box 13.2 Finding the minima (maxima) of
functions 490
Matlab Box 13.4 This script uses the genetic algorithm to optimize the Hopfield rule 504
13.5 Evolving Connectivity Profiles for Activity-Bubble Neural Networks 506
with binary genes to find the minimum of a function 491
Matlab Box 13.1 The script uses the genetic algorithm
13.3 Evolving the Sizes of Neural Networks to Improve Learning 495
to optimize the speed of learning by evolving the number of hidden units in three-layered, feedforward networks trained using back-propagation 499
Matlab Box 13.5 This script uses the genetic algorithm to optimize the connectivity profile of the activity-bubble network 506
Exercises 511 References 513
Matlab Box 13.2 This script uses the genetic algorithm
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
xiv C o n t e n t s
CHapter 14 Future Directions in Neural Systems Modeling 515
14.1 Neuroinformatics and Molecular Networks 516 14.2 Enhanced Learning in Neural Networks with Smart Synapses 522 14.3 Combining Complementary Network Paradigms for Memory Formation 526 14.4 Smart Synapses and Complementary Rules in Cerebellar Learning 531 14.5 A Final Word 539
References 541
Sinauer Associates, Inc. This material cannot be copied, reproduced, manufactured or disseminated in any form without express written permission from the publisher.
