Cognitive Aspects
Objectives
• Explain what cognition is and why it is important for interaction
design.
• Discuss what attention is and its effects on our ability to multitask.
• Describe how memory can be enhanced through technology aid
• Show the difference between classic internal and external cognitive
frameworks that have been applied to HCI.
What is Cognition?
◇ the mental action or process of
acquiring knowledge and
understanding through thought,
experience, and the senses
What is Cognition?
“distinguishes between two general modes:
experiential and reflective cognition.”
Norman (1993)
“describes them in terms of fast and slow thinking.
A state of mind in which we perceive, act, and react to events
around us intuitively and effortlessly.”
Kahneman (2011)
Why do we need to understand
users?
◇ Interacting with technology is cognitive
◇ Need to take into account cognitive processes involved
and cognitive limitations of users
◇ Provides knowledge about what users can and cannot
be expected to do
www.id-book.com 5
Why do we need to understand
users?
◇ Identifies and explains the nature and causes of
problems users encounter
◇ Supply theories, modelling tools, guidance and
methods that can lead to the design of better
interactive products
www.id-book.com 6
Process of Cognition
◇ attention
◇ perception
◇ memory
◇ learning
◇ reading, speaking, and listening
◇ problem solving, planning, reasoning,
and decision making.
ATTE N T I O N
◇ the process of selecting things to
concentrate on
◇ involves auditory and/or visual senses
◇ allows us to focus on information that is relevant
to what we are doing
◇ easy or difficult depends on whether
we have clear goals and whether the
information we need is salient in the
environment.
Multitasking vs Attention
◇ it depends on the nature of the tasks and how
much attention each demands
Design implications for attention
◇ Make information salient when it needs attending to
◇ Use techniques that make things stand out like color, ordering,
spacing, underlining, sequencing and animation
◇ Avoid cluttering the interface with too much information
◇ Search engines and form fill-ins that have simple and clean
interfaces are easier to use
www.id-book.com 11
PERCEPTION
◇ refers to how information is acquired from the environment via
the different sense organs
– eyes, ears, fingers – and transformed into experiences of
objects, events, sounds, and tastes
( Roth, 1986 )
◇ it involves memory, attention, and language
◇ vision hearing touch
◇ whitespace
The findings suggest that using contrasting color is not It was found that people took less time to locate items
a good way to group information on a screen and that from information that was grouped using a border
using borders is more than when using color contrast
Effective (Galitz, 1997) (Weller, 2004)
Activity
◇ Weller (2004) found people took less time to
locate items for information that was grouped
◇ Some argue that too much white space on web
pages is detrimental to search
www.id-book.com 14
Which is easiest to read and why?
What is the time? What is the time?
What is the time? What is the time?
What is the time?
www.id-book.com 15
Design Implications
￭ Icons should enable users to readily distinguish their
meaning
￭ Bordering and spacing are effective visual ways of
grouping information
￭ Sounds should be audible and distinguishable
www.id-book.com 16
Design Implications
￭ Speech output should enable users to distinguish between
the set of spoken words
￭ Text should be legible and distinguishable from the
background
￭ Tactile feedback should allow users to recognize and
distinguish different meanings
www.id-book.com 17
MEMORY
◇ involves recalling various kinds of knowledge that
allow us to act appropriately
◇ The more attention that is paid to something and the
more it is processed in terms of thinking about it and
comparing it with other
knowledge, the more likely it is to be remembered.
MEMORY
◇ 7+2 Miller’s Theory (1956)
- chunks of information can be held in short-term
memory at any one time.
MEMORY TEST
3, 12, 6, 20, 9, 4, 0, 1, 19, 8, 97, 13, 84
cat, house, paper, laugh, people, red, yes,
number, shadow, broom, rain,
plant, lamp, chocolate, radio, one,
coin, jet
t, k, s, y, r, q, x, p, a, z, l, b, m, e
MEMORY
Guidelines in Creating Interface (Bailey 2000)
◇ Have only seven options on a menu.
◇ Display only seven icons on a menu bar.
◇ Never have more than seven bullets in a list.
◇ Place only seven tabs at the top of a website page.
◇ Place only seven items on a pull-down menu.
Processing in memory
◇ Encoding is first stage of memory
◇ The more attention paid to something…
◇ The more it is processed in terms of thinking about it and
comparing it with other knowledge…
◇ The more likely it is to be remembered
www.id-book.com 22
Context is important
◇ Context affects the extent to which information can be
subsequently retrieved
◇ Sometimes it can be difficult for people to recall
information that was encoded in a different context:
www.id-book.com 23
Memory aids
◇ SenseCam developed by Microsoft Research Labs (now
Autographer)
◇ a wearable device that intermittently takes photos without
any user intervention while worn
◇ digital images taken are stored and revisited using special
software
◇ Has been found to improve people’s memory, suffering
from Alzheimers
www.id-book.com 24
Activity
◇ Try to remember the dates of your grandparents’ birthday
◇ Try to remember the cover of the last two DVDs you bought
or rented
◇ Which was easiest? Why?
www.id-book.com 27
Activity
◇ People are very good at remembering visual cues about things
￭ e.g. the color of items, the location of objects and marks on an object
◇ They find it more difficult to learn and remember arbitrary
material
￭ e.g. birthdays and phone numbers
www.id-book.com 28
MEMORY
Design Implications
◇ Do not overload users’ memories with complicated procedures
for carrying out tasks.
◇ Design interfaces that promote recognition rather than recall by
using menus, icons, and consistently placed objects.
◇ Provide users with a variety of ways of encoding digital
information (e.g. files, emails, images) to help them access them
again easily, through the use of categories, color, tagging, time
stamping, icons, etc.
Recognition versus recall
◇ Command-based interfaces require users to recall from
memory a name from a possible set of 100s
◇ GUIs provide MP3 players visually-based options that
users need only browse through until they recognize one
◇ Web browsers, etc., provide lists of visited URLs, song
titles etc., that support recognition memory
www.id-book.com 30
The problem with the classic ‘72’
◇ George Miller’s (1956) theory of how much information
people can remember
◇ People’s immediate memory capacity is very limited
◇ Many designers think this is useful finding for interaction
design
www.id-book.com 31
What some designers get up to…
◇ Present only 7 options on a menu
◇ Display only 7 icons on a tool bar
◇ Have no more than 7 bullets in a list
◇ Place only 7 items on a pull down menu
◇ Place only 7 tabs on the top of a website page
But this is wrong? Why?
www.id-book.com 32
Why?
◇ Inappropriate application of the theory
◇ People can scan lists of bullets, tabs, menu items for the one they
want
◇ They don’t have to recall them from memory having only briefly
heard or seen them
◇ Sometimes a small number of items is good
◇ But depends on task and available screen estate
www.id-book.com 33
Digital content management
◇ Is a growing problem for many users
◇ vast numbers of documents, images, music files, video
clips, emails, attachments, bookmarks, etc.,
◇ where and how to save them all, then remembering what
they were called and where to find them again
www.id-book.com 34
LEARNING
◇ people find it hard to learn by following a set
of instructions in a manual
DESIGN IMPLICATIONS:
◇ Design interfaces that encourage exploration.
◇ Design interfaces that constrain and guide users to
select appropriate actions when initially learning.
◇ Dynamically link concrete representations and
abstract concepts to facilitate the learning of
complex material.
Cognitive prosthetic devices
◇ We rely more and more on the internet and smartphones
to look things up
◇ Cognitive resource cf. extended mind
◇ Expecting to have internet access reduces the need and
extent to which we remember
www.id-book.com 36
Cognitive prosthetic devices
◇ Also enhances our memory for knowing where to find it
online (Sparrow et al,2011)
◇ What are implications for designing technologies to
support how people will learn, and what they learn?
www.id-book.com 37
Design implications
◇ Design interfaces that encourage exploration
◇ Design interfaces that constrain and guide learners
◇ Dynamically linking concepts and representations can
facilitate the learning of complex material
www.id-book.com 38
READING, SPEAKING &
LISTENING
◇ three forms of language processing that have
similar and different properties.
READING, SPEAKING &
LISTENING
◇ Written language is permanent while listening is transient
◇ Reading can be quicker than speaking or listening,
◇ Listening requires less cognitive effort than reading or speaking.
◇ Written language tends to be grammatical while spoken
language is often ungrammatical.
◇ Dyslexics have difficulties understanding and recognizing
written words.
READING, SPEAKING &
LISTENING
Applications to support or replace the lack & difficulty:
◇ Interactive books and e-Learning
◇ Speech-recognition systems
◇ Speech-output Systems
◇ Natural Language
READING, SPEAKING &
LISTENING
DESIGN IMPLICATIONS:
◇ Keep the length of speech-based menus & instructions to
a minimum
◇ Accentuate the intonation of artificially generated speech
voices
◇ Provide opportunities for making text large on a screen,
without affecting the formatting
Problem Solving, Planning, Reasoning,
and Decision Making
◇ processes involving reflective cognition
◇ thinking about what to do, what the options are,
and what the consequences might be of carrying
out a given action.
COGNITIVE
FRAMEWORK
MENTAL MODELS
◇ used by people to reason about a system
◇ is an explanation of someone's thought process about
how something works in the real world.
Gulfs of Execution and
Evaluation
◇ describe the gaps that exist between the user and the
interface (Norman, 1986; Hutchins et al, 1986)
GULF OF EXECUTION - describes the distance from the
user to the physical system
GULF OF EVALUATION – distance from the physical
system to the user
INFORMATION PROCESSING
◇ used by people to reason about a system
COGNITIVE FRAMEWORK
Internal
1. Mental models
2. Gulfs of execution and evaluation
3. Information processing.
External
1. Distributed cognition
2. External cognition
3. Embodied interaction.
DILEMMA
The app mentality developing in the psyche of the
younger generation is making it worse for them to
make their own decisions because they are
becoming risk averse
(Gardner and Davis, 2013)
www.id-book.com 51
DILEMMA
“ Relying on a multitude of apps means that
they are becoming increasingly more anxious
about making decisions by themselves “
www.id-book.com 52
Mental models
◇ Users develop an understanding of a system through learning about and
using it
◇ Knowledge is sometimes described as a mental model:
How to use the system (what to do next)
What to do with unfamiliar systems or unexpected situations (how the system
works)
◇ People make inferences using mental models of how to carry out tasks
www.id-book.com 53
Mental models
◇ Craik (1943) described mental models as:
￭ internal constructions of some aspect of the external world enabling
predictions to be made
◇ Involves unconscious and conscious processes
￭ images and analogies are activated
◇ Deep versus shallow models
￭ e.g. how to drive a car and how it works
www.id-book.com 54
Gulfs of execution and evaluation
The ‘gulfs’ explicate the gaps that exist
between the user and the interface
Norman, 1986; Hutchins et al, 1986
www.id-book.com 55
Gulfs of execution and evaluation
◇ The gulf of execution
￭ the distance from the user to the physical system
◇ The gulf of evaluation
￭ the distance from the physical system to the user
◇ Bridging the gulfs can reduce cognitive effort
required to perform tasks
Norman, 1986; Hutchins et al, 1986
www.id-book.com 56
Information processing
Conceptualizes human performance in
metaphorical terms of information processing
stages
www.id-book.com 57
Bridging the gulfs
www.id-book.com 58
Cognitive modeling is an area of computer
science that deals with simulating human
problem solving and mental task processes in a
computerized model. Such a model can be used
to simulate or predict human behavior or
performance on tasks similar to the ones
modeled.
One of the assumptions underlying the programmable user model
approach is that it is possible to provide an algorithm to describe
the user's behavior in interacting with a system. Taking this
position to the extreme, choose some common task with a
familiar interactive system (for example, creating a column of
numbers in a spreadsheet and calculating their sum, or any other
task you can think of) and describe the algorithm needed by the
user to accomplish this task. Write the description in pseudo
code. Does this exercise suggest any improvements in the
system?
Externalizing to reduce
memory load
◇ Diaries, reminders, calendars, notes, shopping lists,
to-do lists
written to remind us of what to do
◇ Post-its, piles, marked emails
where placed indicates priority of what to do
www.id-book.com 61
Externalizing to reduce
memory load
◇ Diaries, reminders, calendars, notes, shopping lists, to-do lists
written to remind us of what to do
◇ Post-its, piles, marked emails
where placed indicates priority of what to do
◇ External representations:
Remind us that we need to do something (e.g. to buy something for mother’s day)
Remind us of what to do (e.g. buy a card)
Remind us when to do something (e.g. send a card by a certain date)
www.id-book.com 62
Computational offloading
◇ When a tool is used in conjunction with an external representation to
carry out a computation (e.g. pen and paper)
◇ Try doing the two sums below (a) in your head, (b) on a piece of
paper and c) with a calculator.
￭ 234 x 456 =??
￭ CCXXXIIII x CCCCXXXXXVI = ???
◇ Which is easiest and why? Both are identical sums
www.id-book.com 63
Annotation and cognitive tracing
◇ Annotation involves modifying existing
representations through making marks
￭ e.g. crossing off, ticking, underlining
www.id-book.com 64
Annotation and cognitive tracing
◇ Cognitive tracing involves externally
manipulating items into different orders or
structures
￭ e.g. playing Scrabble, playing cards
www.id-book.com 65
Summary
◇ Cognition involves several processes including attention, memory,
perception and learning
◇ The way an interface is designed can greatly affect how well users
can perceive, attend, learn and remember how to do their tasks
◇ Theoretical frameworks, such as mental models and external
cognition, provide ways of understanding how and why people
interact with products
◇ This can lead to thinking about how to design better products
www.id-book.com 67
Group Activity
