HUMAN-COMPUTER
INTERACTION
C h r i s t o p h e r Ta n
PSY 340 c h r i s t o p h e r. t a n @ h e l p . e d u . m y
OVERVIEW
• Human-computer interaction
o HCI principles in UI design
• Website & application design
• HCI in other contexts
o Wearable technology
o Driving
HUMAN-COMPUTER INTERACTION
• The study of how people use complex technological artifacts (May, 2001)
o And how these artifacts can be designed to facilitate this use
• HCI activity → Computer mediating between user and task (Card, 2014)
o Task can be done without computer but with other tools
o Similar in principle, but not HCI
HUMAN COMPUTER INTERACTION
• Goals • Software interface (i.e., info • Tasks performed with use of
• Cognitive & perceptual abilities provided, controls, input computer technology
• Mental states (i.e. fatigue, mechanisms, layout)
workload, stress, anxiety) • Hardware (i.e., physical design,
• Behavioural tendencies computer performance, speed,
responsiveness, capacities)
HUMAN-COMPUTER INTERACTION
• HCI goes beyond specifying displays & controls
• Asks the question: How can technology help people achieve their goals?
o How does interface design support goal-directed behaviour?
• Examples:
o Personal change – health & fitness, time management, financial monitoring
o Consumers – shopping, banking, gaming
o Work – productivity, collaboration
o Social relationships – social media, communication
HUMAN-COMPUTER INTERACTION
User experience & interface
• User experience (UX) → Overall experience when using a product/system
o How do users perceive and feel about the interface/technology?
• Usability
o Learnability; ease of understanding system
o Ease and efficiency of achieving goals
• User interface (UI) design
o How is information organized?
o Interface layout
HUMAN-COMPUTER INTERACTION
Principles & guidelines
• Interface & HCI design involves displays & controls
o Hence, HCI design requires consideration of display & control design
• Similar to/overlap with principles & guidelines covered in previous chapters
o Basic design principles (Norman)
o Display design principles (Wickens)
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Anticipate user needs
• Design is proactive by nature; user needs, wishes, expectations, etc.
• Avoid requiring users to search for info/remember from one page to next
• Provide necessary info and tools for task and context
o E.g. Price of shopping cart; “?” buttons beside data fields
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Limit interruptions & distractions
• Attention = finite cognitive resource
• Attention-switching diminishes task performance
o E.g., Alerts, notifications – some less important than others; should not interrupt primary task
• Give people control over when to respond to interruption
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Minimize info access cost
• Recall: Display design
• Physical & cognitive effort to retrieve info – sometimes excessive
o E.g. No. of clicks/steps to reach desired page
• Interfere with concurrent tasks – e.g. retaining info in WM
Example: Online banking fund transfer (steps from homepage)
Maybank RHB
1. Click “pay & transfer” 1. Hover over pull-out menu
2. Click “transfer” 2. Click “fund transfer”
3. Click “other bank normal transfer IBG”
4. Click “open transfer”
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Make system structure & affordances visible
• Recall: Affordances & signifiers (Norman)
• Affordances in real world are present everywhere → makes things intuitive
• BUT not a natural feature in computers
o Requires intentional design to guide users
o What actions are possible within the system?
• Graphical user interfaces signify possible actions
o Using icons, buttons, colour, design
o E.g. 3D-shaped buttons, greyed out links, etc.
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Be consistent
• Internal consistency (within system)
o Design elements repeated in consistent manner
• Similar info should be located in same locations
o Or info that serves similar functions → grouped together
• Same actions should accomplish same tasks
• Also: External consistency
o Similar to mental models
o Consistency with other systems used in same context
o E.g. Different apps have similar icons, groupings, colours, etc. within Windows platform – mental
model of how Windows functions
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Be credible & trustable
• People trust and rely on technology/interfaces when it is credible
o E.g., More likely to share credit card details to purchase from a reputable online store
• Features that degrade trust & credibility:
o Typos
o Broken links
o Poor design
o Dodgy aesthetics
• Features that enhance trust:
o Connection with real people
o Ability to contact assistance
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Consider aesthetics/simplicity
• Eliminate irrelevant elements that compete with the relevant
• Limit amount of typefaces/fonts, colours, etc. used
o Colour should be used as “redundant coding” – i.e., can be understood without it
• Also: Overall look & feel
o UX is about satisfaction as well
HUMAN-COMPUTER INTERACTION
Principles & guidelines
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Support flexibility, efficiency, & personalization
• Software flexibility to match needs of user
o Shorten tasks that users perform frequently/routinely
• Users often benefit from interfaces tailored to their preferences
o Features such as background colour, theme, wallpaper, font, etc.
o Customized shortcuts & commands – e.g., swipe direction to scroll
• E.g., Workers performing same task may configure their computers differently
HUMAN-COMPUTER INTERACTION
Principles & guidelines
Make system robust to errors
• Make commission of severe errors difficult – i.e., avoid accidental activation
o Confirmation buttons, warnings, multiple steps
• Make warnings clear; not vague
o E.g., Lengthy warnings, error codes → takes effort to interpret; unsure
• Errors are sometimes unavoidable, so minimize negative consequences
o Or allow for easy recovery from errors
o E.g., Ctrl + Z, undo action, unsend email
• User must know that error occurred, what the error is, and how to undo it
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• HCI in typical computers focused on supporting info search & retrieval
o How easily & efficiently can the user navigate the system?
• Legibility
o Font choice – Arial, Helvetica, Verdana, etc.
o Size
• Navigation
o Good UI design guides users through task; makes navigation easy
o Simplify steps needed to achieve goals
o “3 click rule” → find important info within 3 clicks
• Always doable?
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• Readability – The “F-Shaped Visual Scan”
o Users scan computer screens in an F-shaped
pattern (Nielson, 2006)
• Content area, not page itself
o 2 horizontal lines followed by 1 vertical line
o Not a perfect F-shape; gaze time varies from
left to right
o First lines of text & first few words on each
line receive more gaze-time
o Consider placement of important info
• Subheadings, bullet points
WEBSITE & APPLICATION DESIGN
Guidelines & practices
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• Vertical attention – “The Fold”
o Above the fold
• What you see without scrolling
• Info immediately present on screen without further action
o Below the fold
• Info visible only after scrolling downwards
o Users will scroll for info
• BUT will pay less attention as they scroll further down
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• Vertical attention – “The Fold”
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• Vertical attention – “The Fold” (NN Group, 2018)
o Reserve top of pages for high-priority content → key user goals?
• E.g., What do users use online banking websites for?
o Beware of “false floors” – illusion of completeness
• Illusion of completeness can interfere/prevent scrolling behaviour
• Include signifiers to inform user that content is present below the fold
• E.g., Arrows, blurred out texts, “read more” buttons
o Attract attention to important content
• Users’ scanning behaviour is influenced by their search for important info
• Visually distinct and consistently styled
• E.g., Use headers, bolded & italicized text, etc. – help identify when info is important
WEBSITE & APPLICATION DESIGN
Guidelines & practices
• Banner blindness (NN Group, 2007)
o Desensitization to banner ad-like pointlessness
o Users almost never look at anything that resemble ads
• Heatmaps show close to zero fixations on ads
• Whether or not it is actually an ad
o The more an ad looks like a native site component, the more fixation time it gets
• For web designers → ensure content on page does not resemble banner ads; make all
components in line with other design elements
• For advertisers → masquerade ads to look a part of the webpage it is placed on
o People have learned to ignore the right rail of webpages
• Putting content in right-hand corners may be a bad idea
WEBSITE & APPLICATION DESIGN
Guidelines & practices
HCI IN OTHER CONTEXTS
• HCI design must consider context of use
o What are the tasks/goals of user?
o What will the user be doing while achieving those goals?
Wearable technology
• HCI integrated into clothing/accessories → practical functions/features
o E.g., Fitbit, Apple Watch, Android Wear, Google Glass
• HCI in wearable tech should consider:
o Comfort – extended period of use
o Prioritization of info – constrained display spaces; filter out critical info
o Accessible – simplify info to ease access (i.e., single-glance accessibility)
o Non-visual cues – does not limit user to visual scanning; ensures attentional capture
o Simplify tasks – collapse lengthy processes into simple commands/gestures
HCI IN OTHER CONTEXTS
Computers in cars
• Cars now provide drivers with a lot of info and added functionality/features
o E.g., entertainment, vehicle control, connectivity, automated activities
• HCI in cars → concerned with integrating added functionality without compromising
drivers’ performance
HCI IN OTHER CONTEXTS
Computers in cars
HCI design considerations
• Limit distractions & amount of visual info on
displays
o Distracts drivers from primary task of driving
o E.g., Lengthy written texts, info clutter
• Simplify interactions
o Reduce no. of options, steps, & screens
o Systems demanding glances > 2s → risk
A U TO M AT I O N
C h r i s t o p h e r Ta n
PSY 340 c h r i s t o p h e r. t a n @ h e l p . e d u . m y
OVERVIEW
• Benefits of automation
• Problems with automation
AUTOMATION
• A machine that performs task otherwise performed by humans
o Also: Tasks that humans are incapable of performing – i.e., beyond perceptual & physical
capabilities
• Contrast with HCI
o HCI → human in control; role of computer is relatively small
o Automation → shifts human from direct control to supervisory control
• Supervisory control – human’s role is to manage automation
AUTOMATION
Examples:
• Manufacturing, lifting
• Kneading bread
• Floor sweeping
• Heating/cooling systems
• Driving → automated parking, adaptive cruise control
• Autopilot function
• Hazard detectors
• Predictive displays
AUTOMATION
Why automate?
• When tasks are impossible/hazardous
o Robotic handling of hazardous materials (or in hazardous environments)
o Heavy-lifting beyond human capacities
o Complex mathematical processes (statistical analysis)
o Automatic readers for visually impaired
AUTOMATION
Why automate?
• When tasks are difficult
o Operators can carry out, but effortful & error-prone
• E.g., ‘Simple’ calculations, assembly, autopilot systems, medical diagnosis & decision-making
o Routine tasks – automate repetitive & fatiguing human operations
• E.g., Assembly line work, long-haul flights/drives
o Vigilant monitoring (low signal base rate)
• E.g., Warning & alert systems
AUTOMATION
Why automate?
• Extend human capability
o Human WM vulnerable to many factors
o Aid humans in doing things in difficult circumstances
o Automated aids → supplements WM; relieves operator of cognitive load
• E.g., Predictive displays, decision aids
o Extends multitasking capabilities
• E.g., Autopilot function relieves from aircraft control when other task demands temporarily spike
AUTOMATION
Why automate?
• Automation is technically possible & inexpensive
o Economics; productivity
o Inexpensive; reduce labour costs (although not necessarily the best use of automation)
• E.g., Automated phone menus, restaurant kiosks
o Simply because technology is available
PROBLEMS WITH AUTOMATION
• Automation reliability
• Trust: calibration & mistrust
• Overtrust, complacency, & out-of-the-loop behaviour
• Workload & situation awareness
PROBLEMS WITH AUTOMATION
Automation reliability
• Automation (computers in general) does what operators expect it to do
• In human-automation interaction, issue is not reliability
o Perceived reliability
• Not “unreliable”, but “imperfect”
o Often asked to do tasks that are impossible to do perfectly (dynamic environment)
o Imperfect automation still provides value
PROBLEMS WITH AUTOMATION
Automation reliability
Possible reasons for perceived unreliability:
• Actual unreliability
o Automated systems → more complex; more components than manual
o More can go wrong
• Automation does not operate well in certain situations
o All automation → limited operating range (designers’ assumption of its intended use)
o E.g., cruise control on downward slope
o Appears to be erroneous; in reality, lack understanding of how system was designed
• Incorrect “set up” of automation
o Keypress errors, slips, etc. → configuration errors not uncommon
o E.g., periodic painkiller administration
o Automation appears “dumb & dutiful”; blind obedience to operator
PROBLEMS WITH AUTOMATION
Trust calibration & distrust
• Trust in automation
o Degree to which human believes automation will operate as intended
• Trust is linked to perceived automation reliability
o Trust (and automation dependence) increases with perceived reliability
• Trust should be calibrated – in proportion to its reliability
o I.e., Trust should be high when automation is reliable
PROBLEMS WITH AUTOMATION
Trust calibration & distrust
Over-trust
Subjective Trust
Under-trust
Automation Reliability
Wickens et al. (2013)
PROBLEMS WITH AUTOMATION
Trust calibration & distrust
• Poor calibration of trust
o Distrust – fail to trust automation as much as is appropriate; leads to disuse
• E.g., Preference for manual control; alarms; Excel formulas; perception-enhancing automation
• Common causes of distrust:
o “Cry wolf effect” – i.e., under trust; systems with high FA rates
o Failure to understand how automated algorithms produce outputs
• E.g., Perceive FAs as failures; actually low response criterion to ensure safety
• Consequences of distrust → not necessarily severe (except with alarms)
o Leads people to reject good assistance – inefficiency (e.g., Excel)
PROBLEMS WITH AUTOMATION
Trust calibration & distrust
• Rate of automation distrust is alarming – when valid warnings are not heeded
• Recall: Issues with high FA systems → ignore when actual signal is detected
Examples:
Sorkin (1989)
• Train engineers taping over alert speakers due to typically false alarms
Seagull & Sanderson (2001)
• 42% of alarms heard by anaesthesiology nurses ignored (no action taken)
Wickens et al. (2009)
• 45% of conflict alerts received by air traffic controllers required no action (and no action taken)
PROBLEMS WITH AUTOMATION
Overtrust, complacency, OOTLUF
• Overtrust in automation
o Trust in automation more than is warranted
o A.k.a. complacency/automation bias
o Operator expecting automation to be functioning well → less likely to monitor its job; trust that
machine has done the necessary analysis/calculations
• Causes of complacency:
o Top-down processing – Human tendency to let experience guide our expectations; TDP > BUP
o Path of least cognitive effort
o Perceived authority or reliability
• E.g., Pilot following advice of flight planning automated system although wrong
• E.g., Flight simulation experiment (Mosier et al., 1992):
o 75% of pilots wrongly shut down engine due to wrong diagnosis & recommendation of automation
o Only 25% of pilots committed same error when using traditional checklist (i.e., checking raw data)
PROBLEMS WITH AUTOMATION
Overtrust, complacency, OOTLUF
• Automation overtrust & overdependence → lead to deskilling
o Ability to manually perform automated task declines over time
o E.g., Skill loss among pilots of highly automated aircrafts; mitigated by occasionally hand flying
(Wiener, 1988)
o E.g., Calculators
• OOTLUF – Out-of-the-loop unfamiliarity
o Degraded detection through complacency, awareness/diagnosis, and manual skill loss
o Overall system unfamiliarity; become unaware of system states (i.e., out-of-the-loop)
• Dangerous when automation fails and operator has to retake control
• Automation vs. OOTLUF → opposing concerns
o Too much workload vs. too little workload
PROBLEMS WITH AUTOMATION
Workload & situation awareness
• Automation intended to reduce operator workload
o Free mental resources to focus on primary task
o E.g., Automated lane-keeping, blind spot monitors, aircraft alert automation
• In reality, reduces arousal & situation awareness
o Increased automation correlated with decreased SA & workload
PROBLEMS WITH AUTOMATION
Workload & situation awareness
• Automation can undermine situation awareness
o Undermines SA → operator not actively involved in choosing & executing actions
• Similar to “Generation Effect”
• In practice, sometimes causes “clumsy automation”
o Recall: Work overload & underload (i.e., loss of arousal)
o Reduce workload in low-workload periods; increase workload in high-workload periods
• I.e., makes easy tasks easier and hard tasks harder
o Automation failure often occurs in complex situations/problems
• Magnifies the problem; operator has to “re-enter the loop” in more challenging situations
RESOURCES
• Designing for People: An Introduction to Human Factors Engineering (Lee, Wickens, Liu,
& Boyle, 2017)
• Engineering Psychology and Human Performance (4th Ed.) (Wickens et al., 2013)
