Framing and refining a data science question
Stating and
Refining the
question
Six types of question
• Assume that you are collecting a set of
data from a group of individuals: their
dietary patterns, sex, infections, age, etc.
• What is the proportion of males in the
dataset?
• What proportion consume
vegetables/fruits?
• How many hours do females in the
population sleep?
• Hypothesis-forming questions
• Suppose you have a feeling that
dietary patterns influence certain
infections.
• Probing the relationship between
dietary patterns and a set of
infections
• An inferential data analysis quantifies whether an observed pattern will likely hold
beyond the data set in hand.
• This is the most common statistical analysis in the formal scientific literature.
• An example is a study of whether air pollution correlates with life expectancy at the
state level in the United States
• Hypothesis testing questions
• For example, if your EDA points to a lower infection rate for people with a ketogenic
diet, you test it in a representative population of the US.
• You will be able to infer what is true, on average, for the adult population in the US
from the analysis you perform on the representative sample.
• Prediction of Cancer probability
from features present on CT
scans
• What sort of people are more
likely to consume a diet rich in
fruits and vegetables
Correlation does not imply causation
• A causal data analysis seeks to find out what
happens to one variable on average if you
change the other
• Identifies both the magnitude and direction of
relationships between variables.
• For example, decades of data show a clear
causal relationship between smoking and
cancer
• Tells whether eating a ketogenic diet reduces
the probability of infections
• Randomized controlled trial or a series of
analyses where you control certain variables
(regression, stratification)
Randomised Controlled Trial (RCT)
Prospective cohort study
Prospective cohort study
Case-control studies
• However, case-control studies establish only
correlation and not causation.
• Be aware of spurious correlations!
• Mechanistic questions ask how a certain relationship occurs.
• This is the very heart of every real scientific inquiry and requires lots
of attention.
• A mechanistic question should always be the final stage of a scientific
inquiry; if a mechanistic question is not preceded by the
aforementioned stages, the mechanistic question may likely lack of
foundation to be asked.
• For example, how does consuming a ketogenic diet lead to a
reduction in infection?
Data analysis
flow chart
Does cell phone use cause brain tumours?
The Swedish Case-Control Study
Prospective cohort study
The UK Cohort Study
The Danish Cohort Study
The Interphone
Study
A final confirmation study
• A study in the US compared
observed rates of glioma to
projected rates (1997-2008)
• It found that brain tumor
rates have been pretty much
unchanged since mobile
phones arrived.
• If the Swedish team is right
about the size of the
cellphone effect, tumor rates
would be 40 percent higher
than they are
1990 2010
Characteristics of a good research question
• Interest to the audience
• Previously unanswered
• Plausibility
• Answerability or Feasibility
• Specificity
Interest to the audience
• Academic – Collaborators, scientific
community
• Corporate – boss, governing board, investors
• Is there a market for new asthma drug –
pharmaceutical company
• Does the sale of pasta sauce is high when
placed next to the pasta – grocery stores
Previously unanswered
• Explosion of data and the internet – possible
that a solution exists
• Proper literature search, networking
Plausibility
• The question should stem from a plausible
framework
• Placement of pasta and pasta sauce – plausible
• Convince consumers that when buying one of
these items, they also really need to buy the
others
• Placement of pasta sauce and hair shampoo –
not plausible
Answerability or Feasibility
• It should be feasible to answer the question
with existing resources, tools, and techniques
• What are the characteristics of human brain
cells in Alzheimer’s disease? – need for
biopsies
• Will the remedies to climate change be worse
than the disease? Will it drive more people into
poverty with higher costs?
• Why do different people like different foods?
Not just the candy you grew up with;
everything.
• Why hasn't rock climbing been universally
accepted as the best physical activity?
Specificity
• Questions should be focussed
• What affects people’s mental health vs How
does social media affect people’s self-esteem?
• Is air travel deleterious to human health vs. Are
frequent fliers at risk of certain types of
cancers?
• Is eating a healthier diet better for you? vs.
Does eating at least five servings of fresh fruits
lead to few respiratory infections (ex., colds)?
What sorts of questions don’t lead to
interpretable answers?
• Use of appropriate data
• Investigate whether taking
vitamin D supplements is
associated with fewer headaches
• You plan on answering that
question by using the number of
times a person took a pain reliever
– incorrect
• Investigate the relationship house
sizes and well being – you can’t
substitute house size by household
income
Confounding bias
Confounding bias example
Confounding bias example
Selection bias
Selection bias
Selection bias
Selection bias example
Fish during pregnancy vs Autism
Recall bias
Recall bias
Recall bias
[Link]
