Sensorimotor processing
in health and disease
Vision
11.12.23
NEUROBIOLOGY
Spike trains
• Information about environment or own body is transmitted to the brain
in the form of action potentials (AP)
• If the very short duration of AP is ignored they can be treated as all-or-
none events in time à spike train
11.12.23 Fig 1 from Il Memming Park et al. (2013)
NEUROBIOLOGY
Hypothesized coding schemes
How is information encoded?
• There is an ongoing debate about which coding schemes are used by
neurons
1. rate coding: frequency of action potentials
2. temporal coding: precise timing of spike carries information
3. population coding: the combination of responses from many neurons
is necessary
4. …
11.12.23
11.12.23
VISION
11.12.23
Eagleman, 2001. Nature Perspectives.
11.12.23
VISION
Visible light
• Light is electromagnetic radiation and we can see a small spectrum of
it.
• Visible light: 400-700nm
Bear et al. (2018)
11.12.23
VISION
Visible light
• We can see objects because they reflect light.
SxSchroeder, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via
Wikimedia Commons
11.12.23
VISION
The eye
Retina
light
Optic nerve
11.12.23
VISION
The retina
Ganglion
cells
Bipolar
cells
Photorecep
tors
11.12.23
VISION
Organization of the retina
'Light in Retina’ by Casey Henley (CC BY-NC-SA) 4.0 Licence. https://openbooks.lib.msu.edu/neuroscience/chapter/vision-the-retina/
11.12.23
VISION
Photoreceptors
• are the only light sensitive cells in the
retina
• Transform light energy into neural activity
• Neurotransmitter: Glutamate
• Photoreceptors are special in that they
hyperpolarize in response to light.
• Rods: low light vision
• Cones: color vision 'Rod and Cone’ by Casey Henley (CC BY-NC-SA) 4.0 Licence.
https://openbooks.lib.msu.edu/neuroscience/chapter/vision-the-
retina/
11.12.23
VISION
Photoreceptors
• Photoreceptors are special in that they
• A) hyperpolarize in response to light
11.12.23
VISION
Photoreceptors
Photoreceptors are special in that they
A) hyperpolarize in response to light
B) do not generate action potentials, but only graded potentials (change
of membrane potential)
11.12.23
VISION
Cones
• There are three types of cones
• Based on the information coming
from the cones, the brain constructs
a colorful world
• Cones have different sensitivities
with maxima either in blue, green or
red light
11.12.23
VISION
Rods
• Rods are a lot more sensitive to photons than cones (about 100 times)
• Visual information at night
11.12.23
INFORMATION PROCESSING
IN THE RETINA
11.12.23
VISION
The receptive field
The area on, e.g., retina/skin/… that is innervated by an
(afferent) neuron/population of neurons and that when
stimulated leads to a change in response of that specific
(afferent) neuron/ neuron population
11.12.23
VISION
The receptive field
11.12.23
VISION
Side note: receptive field in deep neural networks
• “it is the region in the input space that a particular CNN’s feature is
affected by. More informally, it is the part of a tensor that after
convolution results in a feature. So basically, it gives us an idea of
where we’re getting our results from as data flows through the
layers of the network.” - https://www.baeldung.com/cs/cnn-receptive-field-size
https://www.baeldung.com/cs/cnn-receptive-field-size
Also see: https://theaisummer.com/receptive-field/
11.12.23
VISION
Edge detection – The Mach bands
11.12.23
VISION
Organization of the retina
Transmission to brain
Ganglion cells
Amacrine cell
Bipolar cells
Horizontal cell
Photoreceptors
photoreceptors
11.12.23
VISION
Organization of the retina
Transmission to brain
Ganglion cells
Amacrine cell
Bipolar cells
Horizontal cell
Photoreceptors
photoreceptors
11.12.23
VISION
OFF and ON bipolar cells
• OFF bipolar cells: hyperpolarize in response to light
• ON bipolar cells: depolarize in response to light
11.12.23
VISION
OFF bipolar cells
Ligand gated
ion channel
11.12.23
VISION
ON bipolar cells
11.12.23
OFF and ON bipolar
• Different membrane receptors lead to different reactions based on
photoreceptor input
11.12.23
VISION
Ganglion Cells
• OFF and ON bipolar cells synapse on OFF-center and ON-center
ganglion cells
• Ganglion cells are the only cell type in retina that fire action potentials
• They send information out of the retina and to the brain
11.12.23
VISION
Ganglion Cells
11.12.23
VISION
Receptive field ganglion cell
11.12.23
VISION
Center-Surround Receptive fields
The receptive field of a bipolar cell: the center
11.12.23
VISION
The receptive field of a bipolar cell: the surround
11.12.23
VISION
Center-Surround Receptive field & lateral inhibition
http://www.yorku.ca/eye/recepfld.htm
11.12.23
VISION
Example of an off-center surround ganglion cell
• An off-center ganglion
cell fires most action
potentials when there
is no light (off-center) in
the center and light in
the surround
• The response to the
surround is always
opposite to the one in
the center.
11.12.23
Center-surround receptive fields
https://www.youtube.com/watch?v=9ptnmfpDThk
11.12.23
Eagleman, 2001. Nature Perspectives.
11.12.23
VISION
http://www.yorku.ca/eye/hermann1.htm
11.12.23
VISION
Explanation for Mach bands
http://www.yorku.ca/eye/machban1.htm
11.12.23
VISION
Visual Illusion: Mach band
11.12.23
VISION
Color-opponent ganglion cells
11.12.23
VISION
‘There is a sense in which all of vision is an illusion.
[…] We depend on […] illusions as a normal aspect
of our lives.’
Eagleman, 2001. Nature Perspectives
11.12.23
VISION
https://de.wikipedia.org/wiki/Optische_T%C3%A4uschung/media
/Datei:Opt_taeuschung_groesse.jpg
11.12.23
https://de.wikipedia.org/wiki/Optische_T%C3%A4uschung/media
/Datei:Opt_taeuschung_groesse.jpg
11.12.23
https://metanamorph.com
11.12.23
VISION
Exercise: Read through the text and perform the activities in
the section ‘Some Things To Try’
https://elvers.us/perception/rgc/
11.12.23
VISION
Resources
• https://elvers.us/perception/
• https://openbooks.lib.msu.edu/neuroscience/chapter/vision-the-retina/
• Eagleman, D. M. (2001). Visual illusions and neurobiology. Nature
Reviews Neuroscience, 2(12), 920–926.
https://doi.org/10.1038/35104092
11.12.23
VISION II
The central visual system
11.12.23 48
RECAP
11.12.23
VISION
ON bipolar cells
11.12.23
VISION
Center-Surround Receptive fields
The receptive field of a bipolar cell: the center
11.12.23
VISION
The receptive field of a bipolar cell: the surround
11.12.23
VISION
Exercise: Read through the text and perform the activities in the section ‘Some
Things To Try’
https://elvers.us/perception/rgc/
11.12.23
VISION II
The central visual system
11.12.23 54
VISION II
The central visual system
Lateral Geniculate
Nucleus
Primary visual
Retina
cortex (V1)
Retina à LGN à primary
visual cortex
11.12.23 55
VISION II
Visual field processing
• https://www.youtube.com/watch?v=aobWzlXIooQ
11.12.23 56
VISION II
Visual field processing
Binocular visual field
n
Fixatio
poin t
Right visual
Left visual field field
C D
B
A Right eye
Right visual nerve
Left eye
Right optical
Left visual tract
nerve
Left optical
chiasm
tract
11.12.23 57
VISION II: SHORT EXERCISE
The patient is not able to see
1. A
2. B
3. C
4. D
5. A and B
6. C and D
11.12.23 59
The patient is not able to see
1. A and C
2. A and B C D
3. C and D B
4. A A
5. C
6. B
11.12.23 60
The patient is not able to see
1. B and C
2. A
3. B
4. C C D
5. D B
6. B and C A
11.12.23 61
VISION II
Visual field processing
11.12.23 62
VISION
Inputs to the LGN
Lateral
Geniculate
• Retina is not main input to LGN Nucleus
• 80% excitatory input from V1 à ‘top
down’ control, e.g., for attention
Retina Primary visual
• Also input from brain stem à activity is
related to alertness / attentiveness cortex (V1)
11.12.23 63
VISION
https://www.youtube.com/watch?v=vJG698U2Mvo
11.12.23 64
VISION II
Primary Visual Cortex (V1)
Lateral Geniculate
Nucleus
Primary visual
Retina
cortex (V1)
11.12.23 65
VISION II
V1: Retinotopy
Retinotopy: Neighbouring areas in the
retina transmit information to
V1
neighbouring areas in the target
LGN
structure
Retina
Primary visual
cortex (V1)
11.12.23 66
VISION II
Receptive fields in V1
• Orientation selectivity
• Direction selectivity
11.12.23 67
VISION II
Wiesel and Hubel: receptive fields
• https://www.youtube.com/watch?v=8VdFf3egwfg
11.12.23 68
VISION II
V1: Orientation selectivity
11.12.23 69
VISION II
V1: Direction selectivity
Visual Receptive field Receptive field
stimulus
Movement Movement
direction direction
11.12.23 70
VISION II
After V1: Dorsal and ventral stream
• ganglion cells à LGN à V1
• After V1 there are many different
areas that are involved in visual
processing
• In general two different pathways:
• Dorsal stream
• ventral stream
11.12.23 71
VISION II
Dorsal (where) stream Ventral (what) stream
• ‘Vision for action’ • ‘Vision for perception’
• Analysis of object movement • Conscious perception of visual world and
recognition and memory of objects
• E.g. visually guided reaching
11.12.23 72
VISION II
Hierarchy of receptive fields
• Photoreceptors: small area on the retina
• Ganglion cells: center surround organisation à sensitive for contrast and wavelength (color)
• V1: e.g. orientation/direction selective
• Beyond V1: selective for more complex forms/object movement, faces
à hierarchy of areas with increasingly more complex receptive fields
à Parallel processing
11.12.23 73
Questions?
11.12.23 74
Vision prostheses
11.12.23
VISION EXERCISE
Retinal Implants
• Replace funciton of damaged or missing photoreceptors
• Two ways:
• Electrodes: implanted in or near retina
• Photodiodes: utilise incident light to trigger electrical stimulation
• Direct electrical stimulation via electrodes:
• External system: cambera-based system that captures image of
outside world
• Processing unit: image processing algortihms
• Internal system: electrodes implanted in eye
• Disadvantage: mismatch between direction of eye movement and
head-fixed camera
11.12.23 76
VISION EXERCISE
Retinal implants
• https://www.youtube.com/watch?v=x32tiwfhy0U&list=PLgMhAWVSMugcDQhgTa4lFh8-LKj-
eQ3JF&index=10
11.12.23 77
Retinal Implants
11.12.23
Retinal implants: Photodiode
11.12.23
REFERENCE RETINAL IMPLANT
• Ayton et al. (2020). An update on retinal prostheses. Clin Neurophysiol.
doi:10.1016/j.clinph.2019.11.029.
11.12.23
Exercise: Retinal Implant
• https://pulse2percept.readthedocs.io/en/stable/examples/index.htm
l
• Go to: Example Gallery / Stimuli
• Download ‘Generate a stimulus from an image’
• You can use Google Colab (or any other program that is working for
you)
• You might have to add !pip install pulse2percept to the
first cell
• Go through the script and read the explanations
Exercise: Retinal Implant
part II
• Now instead of the logo, use either the Dog.png or Salamander.png
• Insert these two cells to be able to use one of these pictures as
stimulus
Exercise: Retinal Implant
part II
• Note for Mac users: The upload function might not work in Safari, try
Google Chrome instead (or another browser)
• Play around with different pre-processing steps: How do they change
the predicted percept?
• Why can image pre-processing enhance the final percept?
• Also try out different models and implants. How do results change?
• You can find info about the models and implants on the pulse2percept
website.
Task
• Take a screenshot of your best predicted percept and upload it in
Moodle in the Task ‘Retinal Implants’.
