AI Brainstorming Kit

A design resource to explore what to build with AI

aidesignkit.github.io
AI innovation is failing

Despite the hype, AI innovation is failing

because teams select the wrong things to build.

This brainstorming kit blends user-centered

and tech-centered innovation to simultaneously

consider both AI capabilities and user needs.

AI Brainstorming Kit v1.0

Broadly exploring
AI’s design space

Effective ideation can reduce the risk of

developing unwanted AI products and services.

This kit includes an overview of AI capabilities

and 40 AI product examples to inspire ideation.

AI Brainstorming Kit v1.0

How to use this?

Review AI capabilities and a subset of examples

at the beginning of an ideation session.

You can also use this resource for educational

purposes within your organization or classroom.

AI Brainstorming Kit v1.0

Learn more

This resource was created by researchers at

Carnegie Mellon University’s HCI Institute.

Visit aidesignkit.github.io to learn more

about this project.

AI Brainstorming Kit v1.0

AI Capabilities
Detect
(Monitor, Sense, Notice, Classify, Discriminate)

Notice if a kind of a thing is in a data set Detect face in image (camera)

or if it shows up in a sensor stream.

Detect human voice in audio (smart speaker)

Detect people, objects and vehicles in sensor stream

(autonomous parking)

AI Brainstorming Kit v1.0

Identify
(Recognize, Discern, Find, Classify, Perceive)

Notice if a specific item or class of items Identify if Jane’s face (security)

shows up in a set of like items.

Identify if message is spam (email)

Identify the type of cancer (medical imaging)

AI Brainstorming Kit v1.0

Estimate
(Rate, Grade, Measure, Assess)

Infer a value (e.g., position, size, duration, risk) to Estimate driving time (navigation planner)

make an inference about what’s happening now.

Estimate chances this is spam (email)

Estimate direction sound came from (smart speaker)

AI Brainstorming Kit v1.0

Forecast
(Predict, Guess, Speculate)

Infer a value that will be true or some attribute Forecast best time to buy stock (financial planner)

of a future situation that may or may not happen

(e.g., prices, sales, weather). Forecast tomorrow’s weather (weather app)

Forecast max price for my house (real estate app)

AI Brainstorming Kit v1.0

Compare
(Rank, Order, Find Best, Find Cheapest, Recommend)

Compare a collection of like items based on Compare items by likelihood of purchase (online store)

a metric. Allows services to select, rank, or

curate a collection of things. Compare posts by likely engagement (social media)

Compare movies by likelihood of watching (media)

AI Brainstorming Kit v1.0

Discover
(Extract, Notice, Organize, Cluster, Group, Connect, Reveal)

Analyze a dataset and notice a pattern that Discover unusual bank transactions (fraud detection)

allows clustering of similar things or identification

of outlying entities. Discover person’s routine (energy optimization)

Discover how people use this website (web usage mining)

AI Brainstorming Kit v1.0

Generate
(Make, Compose, Construct, Create, Author)

Generate something new (message, image, sound) Generate conversational responses (chat agent)

based on knowledge of similar things.

Generate detail in image (photo retouching)

Generate synthetic medical records (medical data)

AI Brainstorming Kit v1.0

Act
(Do, Execute, Play, Go, Learn, Operate)

Execute a strategy to achieve a specific goal and Act: Park the car (autonomous parking)

continue to update the strategy based on advance

towards the goal. Act: Play poker (gambling agent)

Act: Fly drone to location (drone pilot)

AI Brainstorming Kit v1.0

Capability + Synonyms
Definition
Examples

Estimate
Infer a value (e.g., position, size, duration, risk) to Estimate driving time (navigation maps)

Rate, Grade, Measure, Assess

make an inference about what’s happening now.
Estimate chances this is spam (email)

Estimate direction sound came from (smart speaker)

Forecast
Infer a value that will be true or some attribute of Forecast best time to buy stock (financial planner)

Predict, Guess, Speculate

a future situation that may or may not happen.
Forecast tomorrow's weather (weather app)

Forecast max price for my house (real estate app)

Compare
Compare a collection of like items based on a
Compare items by likelihood of purchase (online store)

Rank, Order, Find Best, Recommend

metric to select, rank or curate a collection of things.
Compare posts by likely engagement (social media)

Compare movies by likelihood of watching (media)

Detect
Notice if a kind of a thing is in a data set or if it Detect human voice in audio (smart speaker)

Sense, Notice, Classify, Discriminate

shows up in a sensor stream.
Detect face in image (camera)

Detect step in motion sensor stream (smartwatch)

Identify
Notice if a specific item or class of items shows Identify if message is spam (email)

Recognize, Discern, Find, Classify

up in a set of like items.

Identify if Jane’s face (security)

Identify the type of cancer (medical imaging)

Discover
Notice a pattern that allows clustering of similar Discover how people use this site (usage mining)

Notice, Organize, Cluster, Reveal

things or identification of outlying entities.
Discover unusual bank transactions (fraud detection)

Discover relationships between drugs and diseases

Generate
Generate something new (message, image, sound) Generate chat response (chat agent)

Make, Compose, Construct, Author

based on knowledge of similar things.
Generate detail in image (photo retouching)

Generate synthetic medical records (medical data)

Act
Execute a strategy to achieve a specific goal and Act: Park the car (autonomous parking)

Do, Execute, Play, Go, Learn, Operate

continue to update the strategy based on advance Act: Play poker (gambling agent)

towards the goal.

Act: Fly drone to location (drone pilot)

AI Brainstorming Kit v1.0

AI Examples
40 AI examples Aerial Wildlife Monitoring

Automated Essay Scoring

Meeting Summarization

Mobile App Face Filter

from 14 domains Autonomous Parking

Navigation Route Planner

AR Item Viewer
Personalized Advertisements

Biometric Security
Personalized Lesson Plans

Child Welfare Risk Assessment

Predictive Maintenance

Crop Monitoring
Resume Screening

Deepfakes
Review Analytics

Defect Detection
Robotic Invoice Processing

Drug Discovery
Robotic Pick and Place

Fraudulent Transaction Detection

Smart Pricing

Game Player
Smart Speaker Question Answering

Home Energy Optimization

Smartwatch Workout Detection

HR Chatbot
Spam Filter

Image Style Transfer

Stock Trading Recommendations

Infectious Disease Forecasting

Synthetic Health Data Generation

Lane Departure Prediction

Text Generation

Language Translation
Weather Prediction

Media Feed
Web Usage Analytics

Medical Imaging Analysis

Workforce Scheduling

AI Brainstorming Kit v1.0

AI examples
contain unique
capabilities

AI Brainstorming Kit v1.0

AI capabilities
derived across
examples

AI Brainstorming Kit v1.0

Anatomy of an Domain

AI example
Name

Description

Capabilities

AI Brainstorming Kit v1.0

Selecting examples
for brainstorming

Curate 10-15 examples that demonstrate

a broad range of capabilities.

AI Brainstorming Kit v1.0

Aerial Wildlife
Science

Monitoring
Allows tracking animal species for environmental Detect animal in image
research and wildlife preservation. Identify animals in image
Estimate number of animals
Discover animal movement patterns in image
Discover animal habitats in image

AI Brainstorming Kit v1.0

 Marketing & Sales

AR Item Viewer

Allows users to visualize online products in Detect room and room objects in depth map

their physical space using their mobile devices,

Identify room objects in depth map
such as previewing a couch in a living room.
Detect virtual-physical collision in AR

Estimate object size and location in room

Estimate size of room

Generate room with virtual and physical objects

AI Brainstorming Kit v1.0

Automated
Education

Essay Scoring
Assigns grades to essays by identifying Identify style in text
text attributes such as style, coherence, Identify organization in text
and organization.
Identify coherence in text
Estimate grade of essay

AI Brainstorming Kit v1.0

Autonomous
Transportation

Parking
Enables vehicles to autonomously move Identify driver’s intent to park in vehicle telemetry
from a traffic lane into a parking spot. Detect objects in sensor stream
Identify objects in sensor stream
Detect parking space in image
Estimate size of parking space
Generate motion path to parking space
Act motion path to park by minimum moves
AI Brainstorming Kit v1.0
 Risk Mitigation & Security

Biometric
Security
Ensures secure access through the verification Detect face and fingerprint in image
of biological signals from a user. Identify face and fingerprint in image
Detect face in depth map
Identify face in depth map
Detect voice in audio
Identify voice in audio

AI Brainstorming Kit v1.0

 Governance & Policy

Child Welfare
Risk Assessment
Assesses the likelihood of current or future Forecast maltreatment risk for child
maltreatment to a child or adolescent to help Forecast likelihood of repeated maltreatment
welfare workers assess families.
Discover relationships between child maltreatment
and locations

AI Brainstorming Kit v1.0

 Manufacturing & Agriculture

Crop Monitoring

Monitors and tracks crop health Detect crop stress in image

using aerial images.

Identify crop stress type in image

Estimate growth of crop

Forecast yield of crops

Forecast yield impact of resource plans

AI Brainstorming Kit v1.0

 Leisure, Content & Media

Deepfakes Detect face in reference image

Detect eye, mouth, and face landmarks in reference image

Detect human body in reference image

Identify body pose in reference image

Allows producers to generate synthetic content Detect face in target image
that replaces a person in the content with Identify face in target image
someone else’s likeness.
Estimate similarity of images

Generate image of target person

Identify voice in target audio

Estimate similarity of audio

Generate voice of target person

AI Brainstorming Kit v1.0

 Manufacturing & Agriculture

Defect Detection
Used in manufacturing to continuously monitor Detect product defect in image
manufactured goods for quality control and Identify defect type in image
assurance.
Identify defect cause in image
Estimate defect likelihood in product
Discover product defect in image

AI Brainstorming Kit v1.0

 Healthcare

Drug Discovery
Discover relationships between drugs & treatment outcomes
Generate protein structure of drug
Learns the relationships between current drugs’ Generate protein interaction of drug
protein structures and their effect on people, Generate physio-chemical reaction of drug
diseases, and medical conditions.
Generate bioactivity of drug
Estimate toxicity of drug
Estimate promise of drug
Discover new uses of drug in drug-treatment relationships

AI Brainstorming Kit v1.0

 Risk Mitigation & Security

Fraudulent
Transaction Detection

Monitors for suspicious transactions in services Identify fraud in transaction

like banking and healthcare.

Estimate fraud likelihood of transaction

Discover fraud in transactions

AI Brainstorming Kit v1.0

 Leisure, Content & Media

Game Player

Enables an AI agent to play against an opponent Compare game moves by game impact

in games such as chess or poker.

Generate game strategy for game

Act game moves to win by minimum moves

AI Brainstorming Kit v1.0

 Energy & Infrastructure

Home Energy
Optimization
Detect human presence in sensor stream

Optimizes energy use at home based on users’ Estimate preferred temperature of user

schedule and temperature preferences.

Discover person’s routine in temperatures

Discover group routines in temperatures

Identify person’s routine in temperatures

Forecast peak usage of energy

Generate temperature plan for user

AI Brainstorming Kit v1.0

HR Chatbot
Human Resources & Management

Assists employees in the workplace by answering Identify subject in user query

frequent questions about their work, such as Generate response to user query
requesting PTO balance.
Compare responses to query fit

AI Brainstorming Kit v1.0

 Leisure, Content & Media

Image Style
Transfer
Allows users to apply a visual style of an image to Identify content in target image

another image, such as recreating a photo in a

Identify style in reference image
famous painter’s style.
Estimate content similarity of images

Estimate style similarity of images

Generate stylized version of target image

AI Brainstorming Kit v1.0

Infectious Disease
Governance & Policy

Forecasting
Predicts characteristics of epidemics or pandemics Identify words in user query
to inform preparation and mitigation efforts. Estimate symptom-relevance of user query
Discover relationship between queries and infections
Estimate number of infected people
Forecast rate of infection
Forecast peak of infection

AI Brainstorming Kit v1.0

 Transportation

Lane Departure
Prediction

Alerts the driver of a vehicle when they may be Detect lane in image

unintentionally leaving their lane.

Estimate lane position of vehicle

Estimate lane departure likelihood of vehicle

Identify driver’s intent to depart in vehicle telemetry

Detect objects in sensor stream

Estimate collision likelihood of lane departure

AI Brainstorming Kit v1.0

 Office Productivity & Business Workflow

Language
Translation

Generates machine translations for text. Detect text in image

Identify language in text

Identify word translation in text

Identify phrase translation in text

Compare phrases by partial sentence fit

Generate translation of sentence

AI Brainstorming Kit v1.0

 Leisure, Content & Media

Media Feed Identify person’s name in text

Identify content in image

Generate description of image content

Identify company, organization, or product in text

Curates a list of media elements users can Identify place-of-interest in text

select or that might autoplay.

Identify sentiment in text

Identify bullying in text

Identify inappropriate content in audio, image, text

Identify copyrighted content in audio, image, text

Estimate user engagement of media post

Compare media posts by engagement

AI Brainstorming Kit v1.0

 Healthcare

Medical Imaging
Analysis
Looks for cancer in medical images to assist Detect medical anomaly in image
clinicians in making decisions. Identify anomaly as tumor in image

Identify malignant tumor in image

Estimate size of tumor

Identify tumor type in image

Discover medical anomaly in image

AI Brainstorming Kit v1.0

 Office Productivity & Business Workflow

Meeting
Summarization
Automatically generates a meeting transcript, Detect voice in audio
and a meeting summary based on the transcript. Identify words in audio
Identify phrase in text
Identify sentence in text
Compare words by partial sentence fit
Generate summary of transcript

AI Brainstorming Kit v1.0

 Leisure, Content & Media

Mobile App
Face Filter
Detect background in image

Allows users to create virtually augmented Detect eye, mouth, hair, face, and face landmarks in image

visual content using their mobile devices.

Detect hand in image

Detect human body in image

Identify body pose in image

Identify hand gesture in image

Identify face gesture and expression in image

Generate virtual effects on user face and body

AI Brainstorming Kit v1.0

 Transportation

Navigation
Route Planner
Generates users an optimal route to a destination Estimate street and direction of vehicle

based on the provided criteria and constraints,

Forecast traffic impact of route
such as weather, traffic, and road closures.
Estimate travel time of route

Compare routes by driver preferences

AI Brainstorming Kit v1.0

 Marketing & Sales

Personalized
Advertisements
Analyzes user behavior data to determine the best Discover user similarities from user behavior

match between an ad and a set of users to allow

Discover ad similarities from user behavior
services reach their audiences more effectively.
Compare ads by will-user-click

Compare users to ad fit

AI Brainstorming Kit v1.0

 Education

Personalized
Lesson Plans
Allows students to follow a personalized learning Estimate skill level of student
plan, particularly in massive open online courses. Estimate skill acquisition of student
Compare math problems by skill acquisition
Compare unknown skills by learning impact
Discover student stereotypes from student behavior
Generate learning plan for student

AI Brainstorming Kit v1.0

Predictive
Energy & Infrastructure

Maintenance
Builds a model that describes the conditions Detect machine sound in audio
under which a system breaks down. Identify machine breakdown in audio
Estimate breakdown likelihood of machine
Forecast breakdown point of machine
Estimate breakdown duration of machine
Estimate breakdown cost of machine
Forecast repair impact of machine
AI Brainstorming Kit v1.0
Resume
Human Resources & Management

Screening
Assists hiring managers of companies by Identify skills in text
prescreening candidate resumes. Identify competence in text
Identify specialization in text
Compare resumes by job fit

AI Brainstorming Kit v1.0

 Hospitality

Review
Analytics
Analyzes customer reviews for services to Identify subject in text
provide insights on customer behavior and Identify sentiment in text
service breakdowns.
Identify user intent in text
Identify context in text
Discover topics in documents

AI Brainstorming Kit v1.0

Robotic Invoice
Finance

Processing
Digitizes and validates data from purchase orders Identify document type in image
to speed up invoice processing. Detect document structure in image
Identify document structure in image
Identify paired content in image
Identify handwritten words in image

AI Brainstorming Kit v1.0

 Manufacturing & Agriculture

Robotic Pick
and Place

Speeds up the process of picking up items and Detect objects in image

placing them in other locations, typically in industrial

Identify object in image
contexts such as manufacturing or recycling.
Estimate location and orientation of object

Generate motion and grasping path to object

Act motion and grasping path to pick by minimum moves

AI Brainstorming Kit v1.0

Smart Pricing
Hospitality

Predicts an optimal price for a rental property Forecast demand for listing
based on changing demand to assist property Forecast maximum price for listing
owners in setting prices.

AI Brainstorming Kit v1.0

Smart Speaker
Leisure, Content & Media

Question Answering
Assists users by responding to their Detect voice in audio
verbal questions. Identify voice in audio
Identify words in audio
Identify user query in text
Identify subject in text
Compare responses to query fit
Generate human speech from response
AI Brainstorming Kit v1.0
 Healthcare

Smartwatch
Workout Detection

Automatically detects and records the workout Detect motion in sensor stream

activities users perform, such as walking, running,

Identify workout in sensor stream
and cycling.
Detect step in sensor stream

Detect hard fall in sensor stream

Estimate energy expenditure of user

AI Brainstorming Kit v1.0

 Office Productivity & Business Workflow

Spam Filter
Filters spam emails by identifying words associated Estimate spam likelihood of email
with spam emails, as well as other characteristics Identify spam in email
such as the sender.
Identify spam words in text

AI Brainstorming Kit v1.0

 Finance

Stock Trading
Recommendations
Provides insights on profitable stocks to help stock Discover relationships between news and stock prices
market traders, investors, and fund managers make Forecast peak price of stock
trading decisions.
Forecast price of stocks

AI Brainstorming Kit v1.0

 Healthcare

Synthetic Health
Data Generation
Examines a corpus of data to generate new data Generate new patient data from patient data

that is different, but highly similar to the source data.

Generate missing elements of patient data

Generate missing view for medical image

Generate detail for occulded area of medical image

Generate high-res detail for low-res medical image

Generate new medical images from medical image

AI Brainstorming Kit v1.0

 Office Productivity & Business Workflow

Text Generation

Generates guesses for the word or sentence Compare phrases by partial sentence fit

users are typing as part of searching, messaging,

Generate ending of sentence
and word processing services.
Generate next word of sentence

AI Brainstorming Kit v1.0

 Science

Weather Prediction

Predicts weather conditions for a given location and Estimate weather condition for location

time based on current conditions and historic data.

Estimate intensity of weather condition

Forecast temperature for location

AI Brainstorming Kit v1.0

 Marketing & Sales

Web Usage
Analytics
Provides an analysis of web usage data to Compare products by will-user-click
assess and improve websites. Compare users to product fit
Discover navigation patterns from user behavior
Discover user interests from user behavior
Discover user similarities from user behavior
Identify content in web page

AI Brainstorming Kit v1.0

Workforce
Human Resources & Management

Scheduling
Generates a schedule for employees by assigning Forecast demand for staffing
them to shifts or to customers, typically in contexts Estimate priority of cases
such as hospitality, maintenance and healthcare.
Generate schedule for employees

AI Brainstorming Kit v1.0

Ideation
Review a subset of AI examples with your team to
seed ideation. Then run a targeted ideation session,
probing domain experts if a capability might be useful.

AI Brainstorming Kit v1.0

 Idea Idea

Idea Idea

Start with individual brainstorming.

Then ideate with teammates.

AI Brainstorming Kit v1.0

Concept Name

One line description here. Target user need

Who is the user? What is the need you are addressing?

AI Capability

What is the system reasoning about?

Data

What information does the system need to know?

Action

What action does the system take with the AI output?

Template for detailing ideas

into concepts.

AI Brainstorming Kit v1.0

Prioritization &
Assessment
Assess and prioritize ideas using the matrices in
the following slides. Aim for a broad coverage of
the design space.

AI Brainstorming Kit v1.0

Impact-Effort
Matrix High Impact

Low Impact
Map your ideas based on their potential
value for customers and services, and Low Effort High Effort
the effort required to build it.

AI Brainstorming Kit v1.0

Impact-Effort
Matrix High Impact

Low Impact
Look for high impact, low effort ideas.
Low Effort High Effort

AI Brainstorming Kit v1.0

Task Expertise-AI
Performance Matrix High Expertise

Low Expertise
Map your ideas based on how well

the AI system needs to perform, and

Moderate Excellent
the level of expertise required for the task. Performance Performance

AI Brainstorming Kit v1.0

Task Expertise-AI
Performance Matrix High Expertise

Low Expertise
Look for use cases where moderate
AI performance can create value. Moderate Excellent
Performance Performance

AI Brainstorming Kit v1.0

What’s Next

Sketch out solutions to think further think

about feasibility, desirability, viability, and risks.

AI Brainstorming Kit v1.0

Drop us a line at [email protected] – we would love
to hear your feedback.

aidesignkit.github.io