Questionnaire Design

Prof. Caroline Saunders

Dr. Peter Tait
Meike Guenther
John Saunders

Content based on William et al. (2008) Business Research

Methods, 8th Ed. South Western, Cengage Learning, USA.
Key Elements of Research
• Problem Statement
• Objective
• Hypothesis/ Question
• Methods Surveying

• Analysis
• Results
• Discussion/ Conclusion/ Implications
Setting the scene
• Survey methodology is the study of sampling of individuals from a
population with the goal of making statistical inference about that
population. A survey that measures an entire target population is
called a Census.
• A Survey can refer to many different types or techniques of
observation, but in context of survey sampling it most often
involves a questionnaire.
• Surveys are conducted in numerous research fields including:
marketing, psychology, political economy, sociology, economics and
more.
• Surveys elicit information such as: preferences and attitudes (e.g.
type of car), behaviour (e.g. how often you eat fruit), or factual
information (e.g. income).
• Survey sampling is the process of selecting a sample of elements from
a target population.
• The representativeness of the population is paramount in gaining
useful results.
• Non-response bias – if those selected in the sample do not respond -
observed value deviates from the population parameter due to
differences between respondents and non-respondents.
• Coverage bias – sampling frame does not include all population
elements – observed value deviates from the population parameter
due to differences between covered and non-covered elements.
• Survey samples can be broadly divided into two types: Probability and
non-probability samples. Probability samples require each element of
a population to have a positive probability of being selected into the
sample. Unresolved debate on importance of this issue.
• Choice of Survey mode is determined by numerous factors
including: research objectives, budget, survey type, target
population etc.
• Telephone
• Mail
• Online
• Face-to-face
• Intercept
• Mixed mode
• All modes can be affected by Survey Mode Effects, for example
Social desirability bias is likely to be higher for telephone and face-
to-face interviews and lower for mail and online modes.
• Whether this will be a problem needs to be assessed within the
context of the survey content and research objectives.
Basic considerations
• Questionnaire Design is a crucial stage in the
survey research process.
• A survey is only as good as the questions ask.
Garbage in – garbage out!
• Creating a survey seems easy but is usually a
result of a long process.
• Questions must meet the basic criteria of
relevance and accuracy.
Decisions in Questionnaire Design
1. What should be asked?
2. How would questions be phrased?
3. In what sequence should the questions be arranged?
4. What survey layout will best serve the research objectives?
5. How should the survey be pretested? Does the survey need
to be revised?
Survey components

• Information Sheet : research purpose, researcher affiliation,

confidentiality, anonymity, inducements.
• Framing questions: set context, put respondents in the
relevant frame of mind.
• Specific questions: the crux of the survey.
• Demographics.
• “Thank you for your participation”
 What should be asked?
Survey Relevancy
• All information collected should address a research question
helping the decision maker in solving the current problem.
Survey Accuracy
• Increasing the reliability and validity of respondent information
requires that:
-Surveys should use simple, understandable, unbiased,
unambiguous and non-irritating words.
-Survey design should facilitate, recall and motivate respondents
to cooperate.
-Proper question wording and sequencing to avoid confusion and
biased answers.
Wording Questions: Open-ended
Open-ended response questions: pose some problem and
ask respondents to answer in their own words.
Advantages:
• Are most beneficial in exploratory research, especially when the range of
responses is not know.
• May reveal unanticipated reactions towards the product.
• Are good first questions because they allow respondents to warm up.
Disadvantages:
• High cost of administering open-ended questions.
• The possibility that interviewer bias will influence the response.
• Bias introduced by articulate individuals longer response.
Example: open – ended
• What things do you like most about your
study?
• What comes to your mind when you look at
this advertisement?
 Wording Questions: Fixed-alternative
Closed – ended questions are questions in which
respondents are given specific, limited –alternative
responses and asked to choose the one closest to their
standpoint.
Advantages:
• Require less interviewer skills.
• Take less time to answer.
• Are easier for respondents to answer.
• Provides comparability of answers.
Disadvantages:
• Lack of range in the response alternatives.
• Tendency of respondents to choose convenient alternative.
Types of fixed-alternative questions
• Simple-dichotomy questions: Choice of one of two
alternatives (e.g. yes/no).
• Determinant-choice questions: Choice of one from among
multiple alternative (e.g. A,B or C).
• Frequency-determination question: Asks for an answer about
general frequency of occurrence (e.g. always, often,
occasionally, or never).
• Attitude rating question: Required to measure an attitude or
perception of a person, most common is Likert scale.
• Checklist-question: Allows respondent to provide multiple
answers to a single questions by checking off items.
Example: Frequency-determination

How often do eat fresh fruit?

e> Da ily

t) 2-3 tim es a w ee k

t) Wee kly

t) Fo rtn ightly

t) Month ly

t) Le ss than once a month

»
Example: Attitude rating scale/ Likert
scale

How important are the following aspects when making a decision on what products to buy at the supermarket?
Somew hat Neither important Not at all
Very important important nor un important important Don1 know

Quality ~ ~ ~ ~ ~
Impact on the environment eJ eJ IE> eJ rE)

Brand ~ IE> e> e> It')

Price e'l eJ e'l eJ IE>

»
Question Sequencing
• Order bias: Bias caused by the influence of earlier questions
in a questionnaire or by an answer’s position in a set of
answers.
• Funnel technique: Asking general questions before specific
questions in order to obtain unbiased responses.
• Filter questions: Questions that screens out respondents who
are not qualified to answer a second question.
• Pivot questions: A filter question used to determine which
versions of a second question will be asked.
Guidelines for constructing questions
• Avoid complexity: the simpler, the better.
• Avoid leading and loaded questions.
• Avoid ambiguity: be as specific as possible.
• Avoid double-negative items.
• Avoid making assumptions.
• Avoid burdensome questions that may tax the respondents
memory.
• Make certain questions generate variance.
 Your task:
Survey development & administration
• Get together into four groups.
• Choose one of the four survey topics.
• Develop a survey with not more than 10 questions (incl.
framing questions and demographics).
• Administer survey (i.e. get a minimum of five completed
surveys).
• Analyse responses and present averages (descriptives)
tomorrow (5 min presentation & 5 min discussion).
Topic 1: Non-renewable resource
extraction
• Problem statement: Environmental damage,
harms the 100% pure image vs. $ & Jobs
• Objective: Assess the role of these types of
industries in the future of New Zealand?
Topic 2: Tobacco Policy
• Problem statement: Costs are greater than
taxes, Smoking causes public harm.
• Objective: Which consumer policies will help
to reduce quantity smoked?
Topic 3: Canterbury Earthquakes
• Problem statement: Heritage buildings are
costly to restore and money could be spent on
other priorities.
• Objective: Which types of buildings should be
chosen for rebuild and which types should be
chosen for demolition?
Topic 4: Farm Animal Welfare
• Problem statement: Industrial food production
can have consequences for animal welfare.
• Objective: Assess consumers level of support
for different levels of animal welfare.