Design and

Developntent Research
This page intentionally left blank
 Design and
Developtnent Research
Methods, Strategies, and Issues

Rita C. Richey
Wayne State University

James D. Klein
Arizona State University
Director of Editorial: Lane Akers
Editorial Assistant: Anthony Messina
Cover Design: Kathryn Houghtaling-Lacey
Full-Service Compositor: MidAtlantic Books and Journals, Inc.

This bwk was typeset in 10.5/12 pt Goudy Old Style, Italic, Bold, and Bold Italic.
Headings were typeset in Nueva Bold Extended and Nueva Roman Bold.

Copyright © 2007 by Lawrence Erlbaum Associates, Inc.

All rights reserved. No part of this book may be reproduced in

any form, by photostat, microform, retrieval system, or any
other means, without prior written permission of the publisher.

First published by
Lawrence Erlbaum Associates, Inc., Publishers
10 Industrial Avenue
Mahwah, New Jersey 07430
www.erlbaum.com

Reprinted 2009 by Routledge

Routledge
Routledge Taylor & Francis Group
Taylor & Francis Group 2 Park Square
270 Madison Avenue Milton Park. Abingdon
New York. NY 10016 Oxon OX14 4RN

Transferred to Digital Printing 2010

CIP information for this volume can be obtained by contacting the Library
of Congress.

ISBN 978-0-8058-5731-3-0-8058-5731-1 (case)

ISBN 978-0-8058-5732-0-O-8058-5732-X (paper)
ISBN 978-1-4106-1692-0-\-4106-1692-4 (ebook)

Publisher's Note
The publisher has gone to great lengths to ensure the quality of this reprint
but points out that some imperfections in the original may be apparent.
 rorCharlie

My partner in the best parts of life

and
Allison and Matt
Michelle and Andy

May their partnerships be as special as ours

For ]ayne, Mary, and Leslie

Who make me realize the important things in life

This page intentionally left blank
 Contents

List of Tables xiii

List of Figures xiv
Preface xv
Acknowledgments xix

1 An Overview of Design and Development Research 1

The Need for Design and Development Research 2

Design and Development as a Science • The Design and
Development Knowledge Base and Its Foundations • Design and
DevelolJment Research and the Advancement of the Field

The Scope of Design and Development Research 7

Research on Product and Tool Design and Development • Research on
Design and Development Models • The Outcomes of Product and
Tool Research and Model Research • The Unique Role of Design
and Development Research

Looking Ahead 14

2 Identifying Design and Development Research Problems 15

Guidelines for Selecting a Research Problem 15

vii
vIii DESIGN AND DEVELOPMENT RESEARCII

Sources of Design and Development Research Problems 16

Problems from the Workplace • Problems Related to
Emerging Technology • Problems Related to Design
and Development Theory

Using the Literature to Identify and Refine a Research Problem 25

The Role of Literature in Problem Identification • Sources for
a Literature Review in Design and Development

Focusing the Design and Development Research Problem 29

Transforming Research Problems into Research Questions •
Determining the Parameters of the Study

Looking Ahead 32

A Checklist for Identifying and Focusing the Problem

of a Design and Development Study 33

3 Design and Development Research Methodology 35

The Nature of Research Design 36

Establishing Validity • Facilitating Notions of Causality •
Facilitating Generalization and Interpretation • Anticipating and
Avoiding Problems • Components of a Research Design

The Characteristics of Design and Development

Research Methodology 39
Commonly Used Design and Development Research Methods
and Strategies • Mixed and Multiple Methods Research

Matching Research Methods and Questions 42

Classifying Design and Development Research Goals •
Matching Research Purposes and Methods

Looking Ahead 45

A Checklist for Matching Design and Development Research

Questions and Methods 46
Contents ix

4 Product and Tool Research: Methods and Strategies 47

Strategies of Product Development Research 48

A Representative Mixed Met/wds Case Study • A Representative
Multiple Qualitative Methods Study

Strategies of Program Development Research

A Representative Program Evaluation Study

Strategies for Research on Design and Development Phases 52

A Representative Mixed Methods Study of Formative Evaluation •
A Representative Multiple Quantitative Methods Study
of Integrated Evaluation

Strategies for Research on Tool Development and Use 54

A Representative Tool Development Case Study • A Representative
Tool Use Study

A Summary of Product and Tool Research Designs 57

Unique Product and Tool Research Design Concerns 61

Researcher-Participant Dual Roles • Research in a Natural
Work Environment

Looking Ahead 63

A Checklist for Addressing Common Concerns

of Product and Tool Research Design 64

5 Model Research: Methods and Strategies 65

Strategies of Model Development Research 66

A Representative Multiple Qualitative Methods Study •
A Representative Mixed Methods Study

Strategies of Model Validation Research 67

A Representative Expert Review Study • A Representative Usability
Documentation Study • A Representative Investigation of Component
Variables • A Representative Field-Evaluation Study •
A ReJ)resentative Controlled Testing Study
x DESIGN AND DEVELOPMENT RESEARCH

Strategies of Model Use Research n

A Representative Expertise Study Using Think-Aloud Strategies •
A Representative Designer Decision-Making Study Using In-Depth
Interviews • A Representative Qualitative Study of Conditions
Impacting Model Use

A Summary of Model Research Designs 75

Unique Model Research Design Concerns 78

Working with Recall Data • Research in Multiple Natural Work
Environments • Distinguishing Participant Characteristics

Looking Ahead 81

A Checklist for Addressing Common Concerns

of Model Research Design 82

6 Selecting Participants and Settings 83

Selecting the Setting of the Study 84

Types of Settings and Their Characteristics • Matching Settings
to Research Questions • Practical Issues in Setting Selection

Selecting the Participants of the Study 87

Types of Participants • Sampling Participants

Participant and Setting Solutions from the Design

and Development Literature 90
Participants and Settings in Product and Tool Research •
Participants and Settings in Model Research

Ethical Consideration for the Protection of Participants 94

Getting Approval to Conduct Research • Obtaining Informed
Consent • Avoiding Coercion • Maintaining Confidentiality
and Ensuring Anonymity

Looking Ahead 96

A Checklist for Selecting Participants and Settings

in a Design and Development Study 97
C(lntent~ xi

7 Collecting Design and Development Research Data 99

Critical Design and Development Research Data 100

Profile Data • Context Data • In- Progress Project Data •
Try-Out Data

Data Collection Instruments 106

Work Logs • Surveys and Questionnaires • Interview
Protocols • Observation Guides

Technology-Based Data Collection Strategies 117

Web-Based Data Collection • Software-Based Data
Collection • Laboratory-Based Data Collection

Data Collection Issues 123

Ensuring Data Integrity • Establishing Appropriate Data Sets

Looking Ahead 125

A Checklist for Collecting Data in a Design

and Development Study 126

8 Interpreting Design and Development Findings 127

The Contributions of Design and Development Research 127

Expanding the Design and Development Knowledge Base •
Creating the Foundations for Design and Development Theory •
Creating the Foundations for Future Research

Interpreting Product and Tool Research Findings 132

Lessons Learned about Product and Tool Design and Development
Processes • Lessons Learned about Product and Tool Use

Interpreting Model Research Findings 135

Understanding Model Development Findings • Understanding Model
Validation Findings • Understanding Model Use Findings

Interpretation Issues 139

Generalizations and Project-Specific Data • The Impact
of Organizational Concerns
xii DESIGN AND DEVELOPMENT RESEARCH

Looking Ahead 142

A Checklist for Interpreting Findings of Design

and Development Research 143

9 The Status and Future of Design and Development Research 145

The Expansion of Design and Development Research 146

Alternative Approaches to Design and Development Research •
New Opportunities for Design and Development Research

Conditions that Facilitate Design and Development Research 151

Dissatisfaction with Existing Research Orientations • Knowledge of the
Innovation • Availability of Resources

Conclusions 153

Glossary of Terms 155

References 161
Author Index 173
Subject Index 177
 Tables

1-1 Representative Clusters of Design and Development Research

1-2 The Varying Outcomes of Design and Development Research
2-1 Journals Relating to Design and Development Research
3-1 Common Methods Employed in Design and Development Research
3-2 Representative Matches between Purposes, Methods and Types of
Design and Development Research
4-1 Representative Research Design Techniques Used in Product and
Tool Research
5-1 Representative Research Design Techniques Used in Model Research
6-1 Settings of Design and Development Applications
6-2 Setting Elements that Impact Design and Development Research
6-3 Common Participants in Design and Development Studies
7-1 Critical Profile Data in Design and Development Research
7-2 Critical Context Data in Design and Development Research
7-3 Critical In-Progress Data in Design and Development Research
7-4 Critical Try-Out Data in Design and Development Research
7-5 Design and Development Data Accessible via Technology
7-6 Sample Hardware and Software Specifications for a Basic Design
and Development Research Laboratory
8-1 Product and Tool Design and Development Process Conclusions
and Supporting Data
8-2 Product and Tool Use Conclusions and Supporting Data
8-3 Model Development Conclusions and Supporting Data
8-4 Model Validation Conclusions and Supporting Data
8-5 Model Use Conclusions and Supporting Data

xiii
 Figures

1-1 The IOD Knowledge Base

1-2 The Research and Theory Foundations of the Design and
Development Knowledge Base
2-1 Narrowing the Research Topic
7-1 An Overview of Data Collection Tools Used in McKenney (2002)
7-2 Basic Designer/Developer Work Log for In-Progress Projects
7-3 Task-Oriented DesignerIDeveloper Work Log for Past Projects
7-4 Sample Designer Characteristic Survey
7-5 Sample Product Evaluation Survey
7-6 Sample Designer Interview Questions
7-7 A Critical Incident Interview Protocol With Probes
7-8 Sample Classroom Observation Checklist from Ottevanger (2000)
7-9 Sample Open-Ended Observation Instrument
8-1 Expanding the Designer and Design Processes Knowledge Base
Through Design and Development Research Findings

xiv
 Preface

This book is intended for scholars who are interested in planning and con-
ducting design and development research. It is written for experienced re-
searchers, as well as those who are preparing to become researchers. We made
two main assumptions about our audience while writing this book. First, you
should be familiar with concepts and principles related to research design and
methods. The book is intended to supplement a standard research methods
text, not replace it. Second, you should have knowledge related to processes
and models of design and development.

THE EVOLUTION OF DESIGN

AND DEVELOPMENT RESEARCH

Many examples of design and development research can be found in the liter-
ature spanning up to four decades. In recent years, however, there has been
renewed interest in this research orientation and the benefits it offers the field.
Furthermore, notions of design and development research continue to evolve.
We are defining design and development research here as

the systematic study of design, development and evaluation processes with the
aim of establishing an empirical basis for the creation of instructional and non-
instructional products and tools and new or enhanced models that govern their
development.

Our current thinking about this type of research has changed in two ways.
Readers familiar with our earlier work will note the use of the term "design and
development research" in this book. Previously, we used the term "develop-
mental research" to describe this approach (Richey, Klein, & Nelson, 2004;
Richey & Klein, 2005; Richey & Nelson, 1996; Seels & Richey, 1994). Dis-

xv
xvi DESIGN AND DEVELOPMENT RESEARCII

cuss ions with colleagues and students suggest, however, that the word "devel-
opmental" creates confusion since it refers to many other areas of study, such
as human development, international development, organizational develop-
ment, and staff development.
Furthermore, over the years, the term "development" has been ambiguous
to many in the field and has generated considerable discussion regarding its
meaning. This debate has focused typically upon the distinctions between in-
structional design and instructional development. In defining the domains of
the field, Seels and Richey (1994) viewed design as the planning phase in
which specifications are constructed, and development as the production
phase in which the design specifications are actualized. Others broadly define
each of the terms so that they have similar meanings. For example, Briggs
(1977) defined instructional design as "the entire process of analysis of learning
needs and goals and the development of a delivery system to meet the needs;
includes development of instructional materials and activities; and tryout and
revision of all instruction and learner assessment activities." In this interpre-
tation, design is the more generic term, encompassing both planning and pro-
duction. In contrast, Smaldino, Russell, Heinich, and Molenda (2005) define
instructional development as "the process of analyzing needs, determining what
content must be mastered, establishing educational goals, designing materials
to reach the objectives, and trying out and revising the program in terms of
learner achievement" (p. 386). To many in the field, this is a definition of the
instructional systems design (ISD) process.
We purposely use the term design and development throughout this book
because together they have a broad meaning especially in the research context.
The focus of a design and development study can be on front-end analysis,
planning, production, and/or evaluation. This approach can also center on
the design and development of products and tools or on the development,
validation and use of design and development models. In essence, this is the
study of design and development processes as opposed to performing them.
Another change in our thinking relates to the field's traditional emphasis on
instructional interventions. While many of the examples of design and devel-
opment research do focus on instructional products, tools and models, our def-
inition includes the study of both instructional and non-instructional inter-
ventions. This is in keeping with expanded definitions of the field that
encompass notions of performance improvement and non-instructional inter-
ventions (Reiser, 2002).

OVERVIEW OF THE BOOK

Throughout this book, we have included an explanation of the methods and

strategies appropriate for design and development research. We also discuss
Pre/uC'e xvii

issues and problems that design and development researchers often confront
when planning and conducting a study. Procedural explanations are supported
by a wide variety of examples of research studies from the literature and by
samples of actual research tools. We provide checklists to assist you in planning
and conducting each phase of a design and development study. An extensive
reference list and a glossary of terms are also included. Below we describe the
nine chapters that make up the main body of the book.
Chapter 1, ''An Overview of Design and Development Research," examines
why this type of research is important to advance the knowledge base of the
field and offers a rationale for further studies of this type. The scope of design
and development research is examined, and its two categories-product and
tool studies and model studies-are defined.
Chapter 2, "Identifying Design and Development Research Problems," pro-
vides practical guidelines for how to select and evaluate a research topic and
problem to study. Sources of design and development research problems are
discussed, including the workplace, emerging technology, and theory. Several
sources of design and development literature are also provided. The chapter
closes with an explanation of how to transform a research problem into a ques-
tion and establish the parameters of a design and development study.
Chapter 3, "Design and Development Research Methodology," explores the
nature of research design and describes some general concerns that should be
addressed in any research project. Concerns such as validity, generalizability,
and causality are discussed. The methods and strategies commonly employed
in design and development research studies are described. Suggestions for how
to match research questions to methods are also provided.
Chapter 4, "Product and Tool Research: Methods and Strategies," exam-
ines the research design strategies used in representative studies of this type.
Several in-depth examples are provided to illustrate how to plan research stud-
ies related to product and program development, the various phases of design
and development, and the development and use of tools. The chapter ends with
a discussion of two issues that should be addressed when conducting product
and tool research: (a) avoiding bias when researchers assume participant roles
and (b) recognizing the influence of work environment characteristics.
Chapter 5, "Model Research: Methods and Strategies," focuses on the de-
sign of studies of the development, validation, and use of design and develop-
ment models. Numerous detailed examples are used to show how to plan and
conduct this type of design and development research. Issues such as working
with recall data, conducting research in multiple natural work settings, and
distinguishing participant characteristics are discussed.
Chapter 6, "Selecting Participants and Settings," explores issues related to
who will participate in the design and development research project and where
the study will be conducted. Several examples of participants and settings
that have been examined in actual design and development studies, and brief
xviii DESIGN AND DEVELOPMENT RESEARCH

analyses of techniques that have been employed are provided. Ethical consid-
erations for the protection of participants such as obtaining informed con-
sent, avoiding coercion, and maintaining confidentiality and anonymity are
also discussed.
Chapter 7, "Collecting Data in Design and Development Research," ex-
amines the critical types of information that are often gathered in a design
and development study. These include profile data, context data, in-progress
project data, and try-out data. Commonly used data collection tools such as
work logs, surveys, questionnaires, interview protocols, and observation guides
are described, as is the use of technology for data collection. Sample instru-
ments are provided. In addition, iss~es related to ensuring data integrity are
discussed.
Chapter 8, "Interpreting Design and Development Findings," examines how
results from these studies can be used to expand the knowledge base and cre-
ate a foundation for theory and future research. The major types of conclusions
that can be made by design and development researchers, as well as the key
types of data that are used to support these conclusions, are discussed.
Chapter 9, "The Status and Future of Design and Development Research,"
explores the manner in which design and development research is expanding,
as well as the conditions that facilitate such growth. This chapter includes a
description of design-based research and formative research, two alternative
approaches that can be used to address problems similar to those dealt with
by design and development researchers. Finally, we speculate on the future
design and development research pertaining to technology, work place issues
and theory.
This is a book of methods, strategies and issues related to design and devel-
opment research. We have tried to provide concrete direction for those em-
barking on such projects, as well as stimulate future thinking about this type of
endeavor. Hopefully, this book will increase your understanding of design and
development research, and motivate you to tackle such a project yoursel£

Rita C. Richey James D. Klein

Detroit, Michigan Tempe, Arizona
 Acknowledgntents

We want to recognize the many people who have supported us in this project
and in the advancement of design and development research. Bob Reiser has
been a great proponent of this research for many years. His public advocacy
and private encouragement have meant a great deal to both of us. He has car-
ried on the support Barb Seels and Wait Dick gave to this topic in the early
years of our work. Monica Tracey became an adherent of design and develop-
ment research first as a doctoral student, and now as a faculty member. She
regularly spreads the word and has served us especially in terms of her feedback
and reviews of this manuscript. Likewise, Kathryn Ley has enthusiastically pro-
moted this research through conference presentations and repeated calls of
"So when's the book going to be done?" Finally, Susan McKenney has been a
great help to the preparation of this book. She has shared much of her own
work with us and provided insights into her experiences with design and de-
velopment research.
Some of the best assessments of this book throughout its development have
come from the students at Wayne State University and Arizona State Univer-
sity. They have read chapters (often not by choice), critiqued them, and told
us what helps and what doesn't, always with great clarity.
There has also been a level of support from the Erlbaum group that we
greatly appreciate. Both Lori Kelly and Lane Akers have been behind this pro-
ject from the beginning. They have secured wonderful reviewers who provided
direction for us throughout the writing process, and Lane has been particularly
helpful as we reach the end of the journey.
Finally, we want to acknowledge Bill Winn's contributions to the research in
this field. We hope that this book will help advance the high standards that
he set for us.

xix
This page intentionally left blank
 1
An Overview
of Design and
Developntent Research

This book is about how to plan and conduct design and development research.
These methods are also known as developmental or development research
(Richey, Klein, & Nelson, 2004; Seels & Richey, 1994; van den Akker, 1999).
Design and development research seeks to create knowledge grounded in data
systematically derived from practice. We define this type of research as:

the systematic study of design, development and evaluation processes with the
aim of establishing an empirical basis for the creation of instructional and non-
instructional products and tools and new or enhanced models that govern their
development.

This is a pragmatic type of research that offers a way to test theory and to
validate practice that has been perpetuated essentially through unchallenged
tradition. In addition, it is a way to establish new procedures, techniques, and
tools based upon a methodical analysis of specific cases.
The design and development of instructional products and programs is con-
sidered by many to be the heart of the instructional design and technology
(lOT) field. Practitioners in the field typically follow well-established system-
atic models and procedures to design and develop instructional and non-

2
2 DESIGN AND DEVELOPMENT RESEARCH

instructional interventions. These approaches are advocated (and have been

used) in a wide range of education and training environments. They include
a set of common characteristics, including a focus on measurable goals and
outcomes derived through an initial analysis phase, the selection of content
and strategies that match these goals, a process of routinely evaluating the
products prior to finalizing the project, and the assessment of the learning and
performance outcomes (Gustafson & Branch, 2002; Seels & Glasgow, 1998).1
The practice of design and development is to a great extent empirical by na-
ture. Design models parallel the scientific problem-solving processes. It would
not be unreasonable, then, to assume that design and development processes
themselves would have robust empirical support. However, historically there
has been a paucity of research on our models, tools, and products.

THE NEED FOR DESIGN

AND DEVELOPMENT RESEARCH

Over the last five decades, many scholars have called for research to
strengthen the fundamental knowledge base of the IDT field. In 1953, Finn as-
serted that the audio-visual field suffered from a dominance of "gadgeteering"
that reflected a "poverty of thought" (p. 13) and suggested that research would
help professionalize the field. A decade later, Markle (1967) called for the de-
velopment of empirically grounded instructional materials through systematic
testing procedures. In 1984, Heinich stressed the importance of field-based
research to inform practice. More recently, Richey (1997) suggested that "our
practice is not sufficiently informed by research and that our research is not
sufficiently attuned to practice" (p. 91).

Design and Developtnent as a Science

Opinions on the role of research on design and development often depend on

one's conception of what it actually is. We take the position that design and de-
velopment is a science. As a science it should be bound by understandings built
upon replicated empirical research. Our models and procedures should be
validated. The solutions to our problems should be supported by data. This,
however, is not a universally accepted position.

1 Readers who arc not familiar with instructional design and development should consult some of
the standard references to gain further knowledge of the field. These books include The Systematic
Design of Instruction by Dick, Carey, and Carey (2005), Designing Effective Instruction by Morrison,
Ross, and !(cmp, (2006), and Instructional Design by Smith and Ragan, (2005).
An Overview o('I)esign and O/,velopmenlllesean'h 3

Oavies (1981) first presented the question of whether this field was an art,
a craft or a science, opting for the artistic orientation. He viewed design and
development as a holistic process, one that cannot be simply analyzed and
dissected. It is a view that emphasizes the systemic over the systematic.
Others have also explored the essential nature of the field. For example,
Visscher-Voerman and Gustafson's (2004) study of practicing instructional
designers shows that some designers follow an instrumental design paradigm
that is rooted in the natural sciences. This approach is akin to the traditional
instructional systems design model (ISO). Others employ an artistic para-
digm. This approach is one that eschews scientific methods and "dissolves the
borders between the domains of the sciences and the arts" (p. 83).
Cl ark and Estes (1998) view many of the solutions to educational prob-
lems as craft. They define craft as skills that are based upon "fortunate acci-
dents, personal experience, insights ... revised through trial and error" (p. 6).
As an alternative, they advocate following a technology orientation in which
problems are addressed through the use of scientific theory.
Like Clark and Estes, we approach design and development (and, in turn,
research on it) with the assumption that science and empiricism provide a
more effective and reliable route to disciplinary integrity than depending on
artistic tactics and craft-based solutions. We believe that our field has not suf-
ficiently employed scientific methods to facilitate our understanding of design
and development processes. The need for research is especially critical with re-
spect to the models and processes employed by designers and developers. Few
models, design strategies, and tools employed in practice have been empirically
tested and validated. This is the gap that design and development research
seeks to address.

The Design and Developtnent Knowledge

Base and Its Foundations

The field, of course, is not devoid of research. A large and comprehensive

base of knowledge informs our work. From our perspective, the design and
development knowledge base has six major components. These six facets focus
on the different elements of the design and development enterprise: (a) learn-
ers and how they learn, (b) the context in which learning and performance oc-
cur, (c) the nature of content and how it is sequenced, (d) the instructional
strategies and activities employed, (e) the media and delivery systems used,
and lastly, (0 the designers themselves and the processes they use. Figure 1-1
portrays the overlapping elements of this knowledge base.
This knowledge base has been shaped over the years by a combination of
the foundational research and theory of other disciplines as well as the research
and theory unique to instructional design and development. There are three
4 DESIGN ANI) DEVELOPMENT RESEARCII

Learningl
Transfer Context

Learner & Content

Learning Structurel
Processes

Designer Instructional
& Design Strategies
Processes

Media &
Delivery
Systems

FIGURE 1-1. The !DD knowledge base.

key lines of research and theory that have had the most influence on the de-
sign and development knowledge base. These are

• Psychological and learning theory and research.

• Instructional theory and teaching-learning research.
• Communication theory and message design research.

Psychological and learning theory and research provides the dominant

foundations for knowledge and practice pertaining to: (a) the learner and the
learning process, (b) the learning and transfer context, and (c) instructional
strategies. Instructional design (ID) is intimately tied to human learning and
today is being extended to organizational learning. Originally, behavioral ex-
planations of the learning process were dominant in instructional design.
While there is still some evidence of behavioral thinking, cognitive principles
are often followed when devising strategies that facilitate motivation for learn-
An Overview ofiJesi~/n and IJeve/opment Research 5

ing, as well as the understanding, retention, and use of learned information.

And today, there is a great deal of interest in the view that "(1) learning is an
active process of constructing rather than acquiring knowledge and (2) in-
struction is a process of supporting that construction rather than communica-
tion of knowledge" (DuffY & Cunningham, 1996, p. 171). In this orientation
emphasis is placed upon the social context of learning, collaboration, and
learner control.
Next, instructional theory and teaching-learning research provides the
foundations for knowledge and practice pertaining to (a) content structure and
sequence, (b) instructional strategies, and (c) media and delivery systems.
While instructional theory is necessarily prescriptive in nature, it is typically in-
tertwined with the orientation of a particular learning theory.
Most strategies for presenting instruction are rooted in instructional theory
(Reigeluth, 1983; Richey, 1986). One example of a theory of instruction is
Gagne's (1985) Events of Instruction. This theory suggests a particular sequence
of instructional activities that can be incorporated into a lesson to facilitate
learning and transfer. Constructivist orientations to learning, which avoid the
"presentation" of content in favor of students controlling their own learning
activities, also rely upon instructional theories and teaching-learning research.
The vast majority of design and development approaches assume that in-
struction should vary depending upon the type of learning task being ad-
dressed. Another important design decision, the selection of media and deliv-
ery systems, is typically guided by instructional principles, such as those
relating to facilitating learner involvement in "real-world" activities. Currently,
the growing use of distance education methodologies is stimulating teaching-
learning research in this area, and this work should be of increasing importance
to designers and developers.
Lastly, communications theory and message design research has provided
the major foundations for knowledge and practice pertaining to media and
delivery systems. When combined with the principles of information process-
ing and perception, communication theory tenets and message-design princi-
ples guide page layout, screen design, graphics, and visual design. Research rel-
evant to gaining and controlling attention has been of particular importance to
the design and development field.
Traditionally, this type of research and theory has been crucial to all media
design. Today, its major impact is on the development of interactive, computer-
based media and Web design. In all cases, however, this type of research is
fundamental to the study and applications of visual thinking, visual learning,
and visual communication.
While psychological research, teaching-learning research and message-design
research all have implications for design and development, this body of litera-
6 DESIGN AND DEVELOPMENT RESEARCI I

ture does not fully address the last component of the 100 knowledge base:
the designer and design processes. It does not explain, for example, the role
of the designer or the context in which design and development takes place.
It does not explain designer problem-solving processes. It does not speak to the
ultimate value of ID models, or of ways of reducing design cycle time, or of
the most effective and efficient ways of using learning objects when designing
instruction using advanced technologies. Design and development research is
the primary source of such knowledge, but, unfortunately, there is only a lim-
ited amount of this type of research. Our position is that this shortage ulti-
mately impedes not only design and development practice, but scholarship in
the field as well.

Design and Developtnent Research

and the Advancetnent of the Field

Instructional design, and hence its practice, can be seen in two ways: (a) the
design of particular lessons, products, or programs, and (b) the implementa-
tion and management of the overall design process. The former is guided by
design principles for selecting and sequencing instructional strategies that
are richly supported by learning theory and teaching-learning research (Ragan
& Smith, 2004). The latter is typically guided by instructional system design
(ISO) models which have not been tested to a great extent using research.
Historically, ISO models were devised as an application of General Systems
Theory (Banathy, 1968), but systems theory derived its support from deduc-
tive logic rather than empirical research. This pattern continues today with
respect not only to new design and development models, but also for newly
devised approaches to producing instruction and other interventions utilizing
advanced technologies. This meager research base, in turn, makes it impos-
sible for scholars to construct sound and comprehensive design and develop-
ment theory.
We posit that design and development research will not only enhance our
knowledge base, but will also provide the empirical basis for the construction
of a comprehensive theory of design and development. This will give the field
a fourth theory base, supplementing the understandings we have already ac-
quired from psychological and learning theory, instructional and teaching-
learning theory, and communication and message-design theory. The research
and theory foundations of the design and development knowledge base would
then be as shown in Figure 1-2.
This research and theory is as vital to the advancement of design and de-
velopment practice as it is to the scholarly advancement of the field.
An Overview o('lJesigl1 ami J)eve/ofJment Ileseanh 7

Psychological
and learning
RHearch &
Theory

DHlon& DHlgn& Instructional

Development Development Theory &
Theory & Knowledge Teaching"
Research Base Learning
Theory

Communica-
tions Theory
& Message
Design
Research

FIGURE 1-2. The research and theory foundations of the design and development
knowledge base.

THE SCOPE OF DESIGN

AND DEVELOPMENT RESEARCH

Design and development research covers a wide spectrum of activities and

interests. In its simplest form, it could be either

• The study of the process and impact of specific design and development
efforts.
• The study of the design and development process as a whole, or of par-
ticular process components.

Such research can involve a situation in which someone is studying the

design and development work of others. However, it can also involve a situation
in which someone is performing design and development activities and studying
8 DESIGN AND DEVELOPMENT RESEARCH

the process at the same time. In either case, there is a distinction between doing
design and development and studying the processes.
Understanding the nature of this research is a matter of understanding the
range of problems to which it can be applied. It is also a process of recogniz-
ing those research interests and endeavors that are not a part of this orienta-
tion. Design and development research, as with all research endeavors, leads
to knowledge production, a more complete understanding of the field, and
the ability to make predictions. Design and development research accom-
plishes these goals through two large categories of research projects:

• Product and tool research.

• Model research.

Understanding these categories can facilitate an appreciation of the breadth

and depth of design and development research.
Within each of these two major types of design and development research
various lines of inquiry have emerged over the years. Table 1-1 shows many
of these lines of study.
Clearly these research clusters are not all that exist or that could fit within
the two major divisions of design and development research. They do, how-

TABLE 1-1.
Representative Clusters of Design and Development Research

Design & Development Research

Product & Tool Research Model Research
Comprehensive Design and Model Development
Development Projects • Comprehensive Model Development
• Instructional Products & Programs • Development of Model
• Non-instruction Products & Programs Component Processes
Specific Project Phases Model Validation
• Analysis • Internal Validation of Model
• Design Components
• Development • External Validation of Model Impact
• Evaluation
Design & Development Tools Model Use
• Tool Development • Study of Conditions Impacting
• Tool Use Model Use
• Designer Decision-Making Research
• Designer Expertise &
Characteristics Research
An Overview of/Jesign and IJevelopment Reseanh 9

ever, show the broad scope of research that is encompassed within the do-
main of design and development research. We will now summarize the kinds of
studies that fit within the two types of design and development research-
product and tool research and model research.

Research on Product and Tool Design

and Development

The most straightforward design and development research falls into the first
category: research conducted during the design and development of a prod-
uct or tool. Often the entire design and development process is documented.
Some research, however, concentrates on one aspect of design and develop-
ment only (such as production) or de-emphasizes some phases (such as needs
assessment). Many recent studies focus on the design and development of
technology-based instruction. In this research there is a tendency to combine
the task of doing design and development and studying the processes.
Product and tool research typically involves situations in which the design
and development process used in a particular situation is described, analyzed,
and a final product is evaluated. Oriscoll (1984) used the term "systems-based
evaluation" to describe a similar research paradigm, while van den Akker (1999)
labeled it as "formative research." van den Akker further defines this as "research
activities performed during the entire development process of a specific inter-
vention, from exploratory studies through (formative and summative) evalua-
tion studies" (p. 6). We previously referred to this category of design and devel-
opment research as Type 1 developmental studies (Richey, Klein, & Nelson, 2004).

Product development research. Product studies originate with the design

and development of an instructional or non-instructional product or program.
They demonstrate a range of design and development principles available to
practitioners. Frequently, the entire design and development process is docu-
mented. Consistent with predominant practice in the field, the procedures em-
ployed usually follow the tenets of ISO, encompassing front-end analysis
through evaluation. For example, Hirumi, Savenye, and Alien (1994) describe
the analysis, design, and development of an interactive videodisc exhibit in a
natural history museum. Furthermore, a field evaluation was conducted to
examine visitors' use of the videodisc by focusing on "the program's ability to
attract and hold viewers' attention, and visitors' interaction with the program"
(p. 51). This study provides evidence that ISO processes can be adapted to
informal educational settings.
Other studies emphasize a particular design and development phase, such
as needs assessment or evaluation. For example, Klein et al. (2000) report on
a needs assessment conducted to determine the optimal instructional content
10 DESIGN AND DEVELOPMENT RESEARCI I

and delivery method for an introductory course in educational technology. The

results of the needs assessment were used to revise an existing course. Fischer,
Savenye, and Sullivan (2002) report on the evaluation of a computer-based
training course for an online financial and purchasing system. The purpose of
the study was to evaluate the effectiveness of the training and to identify ap-
propriate revisions to incorporate into the program.

Tool development research. Instructional designers use a variety of tools in

the course of a project. These tools range from paper-based job aids to elec-
tronic performance support systems (EPSS). In recent years there has been
an effort to develop computer-based tools intended to automate many design
and development processes. These include tools for identifying and structuring
content, for selecting strategies for learners, and for authoring instructional
programs (Li & Merrill, 1991; Merrill & Li, 1989). Design and development
research can focus on the development and the efficacy of these tools.
For example, Nieveen and van den Akker (1999) reported on the devel-
opment and evaluation of five prototypes of a computer system to support de-
signers during formative evaluation. This study exemplifies tool development
research; it includes a careful and extensive documentation of the phases of
ISO-needs analysis, design, development, and formative evaluation of several
prototypes of a performance support tool. Another example of a tool study is
Preese and Foshay's (1999) research on the development, quality assurance,
and impact of a set of object-oriented authoring tools. This study has not only
project-specific conclusions, but it also identifies lessons that can apply to
other large-scale CBT development projects. These two studies show how re-
search techniques have been employed to study the development and valida-
tion of tools for instructional designers.
Recent research has also focused on non-instructional tools. For example,
Nguyen (2005) described a needs assessment conducted to determine the
types of EPSS that training professionals and end users think are valuable to fa-
cilitate performance. In addition, Nguyen, Klein, and Sullivan (2005) con-
ducted an experiment in which employees at a manufacturing company com-
pleted a procedural task and received support from either an intrinsic,
extrinsic, or external EPSS or received no support. The authors suggest how to
improve user performance by integrating support with work tasks. Thus, con-
clusions are aimed at designers.

Research on Design
and Developll1.ent: Models

The second type of design and development research pertains to studies of

the development, validation, and use of design and development models.
All Overview of'/Jesigl1 ul1d IJeve/opmel1t Ilesean'h 11

These studies focus on the models and processes themselves, rather than their
demonstration. While it is possible to conduct model research in conjunction
with the development of a product or program, most model studies concen-
trate on previously developed instruction, and consequently are not project-
specific. Model research may address the validity or effectiveness of an existing
or newly constructed development model, process, or technique. In addition,
these studies often seek to identify and describe the conditions that facilitate
successful design and development.
Model research is the most generalized of design and development research.
The ultimate object of this research is the production of new knowledge, of-
ten in the form of a new (or an enhanced) design or development model. This
research may emphasize comprehensive models or particular design techniques
or processes. It commonly examines design and development as it is practiced
in the workplace. In this section, we will describe a number of studies to clar-
ify the nature of model research.

Model development research. One genre of design and development

research results in the development of new or enhanced models to guide the
ID process. For example, Spec tor, Muraida, and Marlino (1992) proposed an
enhanced ISO model for use in courseware authoring, which is grounded in
cognitive theory. This use of the model was then described and evaluated in a
military training environment. Another example is Carliner's (1998) natural-
istic study of design practices in a museum setting in which an enhanced model
of instructional design for informal learning in museums was proposed as an
outcome of the research. Finally, Plass and Salisbury (2002) described and
evaluated a design model for a Web-based knowledge management system.
This model includes not only the initial development of the knowledge man-
agement system, but provides for further development and maintenance to
accommodate changing needs.
The newly developed models, however, need not relate to the entire de-
sign and development process. Some studies focus only on a part of the process.
For example, Forsyth (1998) addressed only the development of a train-the-
trainer model and she concentrated on one particular setting, non-profit com-
munity-based training environments. Her data describes the specific steps
completed when following her model, the time allocated to each phase of the
model and the lessons learned throughout.

Model validation research. Model construction, however, is only one of

the researchable issues that can be addressed with a design and development
study. Today, many recognize that ID models should be substantiated by sys-
tematic validation rather than relying primarily on user testimonials as evi-
dence of their effectiveness (Gustafson & Branch, 2002). ID model validation
12 DESIGN AND DEVELOPMENT RESEARCH

is an empirical process that demonstrates the effectiveness of a model's use

in the workplace or provides support for the various components of the model
itself (Richey, 2005). This is an important type of design and development
research.
Often model validation efforts are combined with model development re-
search. One such example is that of Sleezer (1991) who developed and vali-
dated a Performance Analysis for Training (PAT) Model using expert review
methods. She used experts in training needs assessment to evaluate the con-
tent and face validity of the PAT model. This is an example of an internal val-
idation study, a validation of the components and processes of an ID model.
Studies such as these focus upon the integrity of the model and its use.
There are also external validation studies that confirm the model not by
verifying its components, but by documenting the impact of the model's use.
These studies address the instructional products produced by following the
model, and the impact of these products on learners, clients, and organizations.
In many respects, these studies can be seen as summative or confirmative eval-
uations of the model. For example, a few studies have validated the ARCS
Model of Motivational Design (Keller, 1987) by studying the impact of the var-
ious model components (e.g., attention, relevance, confidence, and satisfac-
tion) on achievement (Brolin, Milheim, & Viechnicki, 1993-94; Means,
Jonassen, & Dwyer, 1997; Small & Gluck, 1994).

Model use research. It is not unusual for model validation studies to ad-
dress usability issues; however, there is another group of design and develop-
ment studies that address these processes by concentrating on their use. These
studies may focus on the conditions that affect model use in an effort to iden-
tify the impact of the various design and development environments. Other
model use studies focus on the designers themselves in an effort to under-
stand the design and development process as it is actually performed.
Several studies can serve as examples of these model use foci. Roytek (2000)
conducted an extensive examination of two design projects that employed
rapid prototyping procedures in an effort to determine which contextual fac-
tors influence project success. The instructional support factor was most
closely related to the work of the designers and developers. In another study,
Spec tor, Muraido, and Marlino (1992) examined the variables that affect the
ability of designers to effectively author computer-based instruction.
Rowland's (1992) comprehensive study of the use of design models serves
as an example of both designer decision-making research and research on
designer characteristics. Four expert and four novice designers were given a
task to design instruction for an industrial setting involving training employees
to operate two hypothetical machines. The thoughts designers had while com-
pleting the task were coded and analyzed. The results of this study describe
An Overview of/Jesigl1 and IJevelol1mel1t Research 13

how design processes actually work and compare the differences between how
experts and novices design instruction.

The Outcontes of Product and Tool

Research and Model Research

In the contemporary orientation toward research, most accept the premise that
research can have a broader function than the creation of generalizable state-
ments oflaw; context-specific studies also are valued. Design and development
research encompasses studies with conclusions that are both generalizahle
and contextually specific. This reflects the fact that product and tool research
typically involves studies that describe and analyze the design and develop-
ment processes used in particular projects, and are thus to a great extent con-
text-bound. Model research studies, on the other hand, are oriented toward a
general analysis of design and development processes. These studies tend to be
more generalizable than product studies. Table 1-2 portrays the relationships
between the two major types of design and development research.
All of these studies, however, can provide some direction to others in the
field, even those whose conclusions are derived from particular projects and
contexts. The "lessons learned" from these studies can apply to those who are
confronting similar design and development projects. Model studies may gen-
erate new or enhanced models available for general use, but not all have such
a comprehensive goal. They may explain the way in which existing models
are used; they may account for success and failures of model use.

TABLE 1-2.
The Varying Outcomes of Design
and Development Research

Design & Development Research

Product & Tool Research Model Research
Emphasis Study of specific product or Study of model development,
tool design and development validation or use
projects
Outcome Lessons learned from developing New design and development
specific products and analyzing procedures or models, and
the conditions which facilitate conditions which facilitate
their use their use

Context-Specific Generalized
Conclusions Conclusions
14 DESIGN AND DEVELOPMENT RESEARCII

The Unique Role of Design

and Developntent Research

Design and development research-both product and tool research and model
research-covers a broad array of design and development issues. It spans all
phases of the design and development process, the various education and
training settings in which designers work, and encompasses all design philoso-
phies and orientations. Unlike most other research efforts, design and devel-
opment research is in a position to substantially expand the theory base of ID
by reaching beyond the traditional foundations of teaching and learning re-
search. Design and development researchers are also in a position to directly
impact the work of practitioners, especially because of the propensity of these
researchers to situate their studies in natural work settings and to address the
pressing problems of the workplace. These researchers value the importance of
learning and performance, but also view the designer as a key element in the
design and development process. In many respects, design and development
research serves as an important link between theory and practice.

LOOKING AHEAD

We have provided an overview of design and development research in this

chapter. We indicated why this type of research is important and offered a ra-
tionale for further studies of this type. We also focused on what design and
development research is by defining its scope and its categories, but we have
not yet discussed how to conduct design and development research. In the next
chapter, we will begin to tackle these procedures by focusing on how to iden-
tify a research problem for a design and development study. We will examine
the sources, characteristics, and parameters of problems that relate to con-
ducting design and development research.
 References

Adamski, A. J. (1998). The development of a systems design model for job performance
aids: A qualitative developmental study (Doctoral dissertation, Wayne State Univer-
sity, 1998). Dissertation Abstracts International-A, 59(03),789.
Albero-Andres, M. (1983). The use of the Agency for Instructional Television instruc-
tional development model in the design, production, and evaluation of the series of
Give and lake (Doctoral dissertation, Indiana University, 1982). Dissertation Abstracts
International-A, 43(11), 3489.
Alessi, S. M. (1988). Learning interactive videodisc development: A case study. lournal of
Instructional Development, 11 (2),2-7.
Armstrong, A. (2003). Instructional design in the real world: A view from the trenches. Hershey,
PA: The Idea Group.
Aukerman, M. E. (1987). Effectiveness of an interactive video approach for CPR recertifi-
cation of registered nurses (Doctoral dissertation, University of Pittsburgh, 1986).
Dissertation Abstracts International-A, 47(06), 1979.
Baker, E. L. (1973). The technology of instructional development. In R. M. W Travers
(Ed.), Second handbook of research on teaching (pp. 245-285). Chicago: Rand McNally
& Company.
Banathy, B. H. (1968). Instructional systems. Palo Alto, CA: Fearon Publishers.
Beauchamp, M. (1991). The validation of an integrative model of student affect variables
and instructional systems design (Doctoral dissertation, Way ne State University,
1990). Dissertation Abstracts International-A, 51 (6),1885.
Berg, B. L. (2004). Qualitative research methods for the social sciences (5th ed.). Boston, MA:
Pearson Education, Inc.
Bichelmeyer, B. (2004). Rapid prototyping. In A. Kovalchick & K. Dawson (Eds.), Educa-
tion and technology: An encyclopedia, Volume 2:}-Z (pp. 483--489). Santa Barbara, CA:
ABCCLIo.
Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Edu-
cational Researchlournal, 5, 437-474.
Brady, H. E., & Collier, D. (Eds.). (2004). Rethinking social inquiry: Diverse tools, shared stan-
dards. Lanham, MD: Rowman & Littlefield Publishers, Inc.

161
162 DESIGN AND DEVELOPMENT RESEARCH

Brewer, J., & Hunter, A. (2006). Foundations of multimethod research: Synthesizing styles.
Thousand Oaks, CA: Sage Publications.
Briggs, L. J. (Ed.). (1977). Instructional design: Principles and applications. Englewood Cliffs,
NJ: Educational Technology Publications.
Broderick, M. (1963). Logic and scientific method in research on teaching. In N. L. Gage
(Ed.), Handbook of research on teaching (pp. 44-93). Chicago: Rand McNally &
Company.
Brolin, R. M., Milheim, W D., & Viechnicki, K. J. (1993-94). The development of a model
for the design of motivational adult instruction in higher education. Journal of Educa-
tional Technology Systems, 22(1), 3-17.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in
creating complex interventions in classroom settings. Journal of the Learning Sciences, 2,
141-178.
Cambre, M. A. (1979). The development offormative evaluation procedures for instruc-
tional film and television: The first fifty years (Doctoral dissertation, Indiana Univer-
sity, 1978). Dissertation Abstracts International-A, 39(7), 3995.
Campbell, D. T, & Stanley, J. C. (1963). Experimental and quasi-experimental designs for
research. In N. L. Gage (Ed.) , Handbook of research on teaching (pp. 171-246). Chicago:
Rand McNally & Company.
Carliner, S. (1998). How designers make decisions: A descriptive model of instructional design
for informal learning in museums. Performance Improvement Quarterly, 11 (2),72-92.
Carr-Chellman, A., Cuyar, C., & Breman, J. (1998). User-design: A case application in
health care training. Educational Technology Research and Development, 46(4), 97-114.
Chase, C. A. (2003). The effects of gender differences and levels of expertise on instruc-
tional design (Doctoral dissertation, Wayne State University, 2002). Dissertation
Abstracts International-A, 63(11), 3815.
Chomei, T, & Houlihan, R. (1970). Comparative effectiveness of three language laboratory
methods using a new equipment system. Audio-Visual Communication Review, 18(2),
160-168.
Chou, c., & Sun C. (1996). Constructing a cooperative distance learning system: The
CORAL experience. Educational Technology Research and Development, 44(4),71-84.
Clark, R. E., & Estes, E (1998). Technology or craft: What are we doing? Educational Tech-
nology, 42(5), 5-11.
Cobb, R.W, & Elder, C. D. (1983). Participation in American politics: The dynamics of agenda-
building (2nd ed.). Baltimore, MD: The Johns Hopkins University Press.
Cochran, W (1977). Sampling techniques (3rd ed.). New York: John Wiley & Sons.
Collins, A. (1992). Toward a design science of education. In E. Scanlon & T O'Shea (Eds.),
Issues in education research: Problems and possibilities. San Francisco: Jossey-Bass.
Corno, L., & Randi, J. (1999). A design theory for classroom instruction in self-regulated
learning? In C. M. Reigeluth (Ed.) , Instructional design theories and models, Volume II:
A new paradigm of instructional theory (pp. 293-318). Mahwah, NJ: Lawrence Erlbaum
Associates, Publishers.
Corry, M. D., Frick, T W, & Hansen, L. (1997). User-centered design and usability testing
of a web site: An illustrative case study. Educational Technology Research and Develop-
ment, 45(4), 65-76.
Ileferenl'es 163

Cowell, D. M. (2001). Needs assessment activities and techniques of instructional design-

ers: A qualitative study (Doctoral dissertation, Wayne State University, 2000). Dis-
sertation Abstracts International-A, 61 (10),3873.
Creswell, J. W (2003). Research design: Qualitative, quantitative, and mixed methods approaches
(2nd ed.). Thousand Oakes, CA: Sage Publications.
Davies, I. K. (1981). Instructional development as an art: One of the three faces of ID.
Pelformance & Instruction, 20(7), 4-7.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm
for educational inquiry. Educational Researcher, 32(1), 5-8.
Dessinger, J. c., & Moseley, J. L. (2004). Confirmative evaluation: Practical strategies for
valuing continuous improvement. San Francisco, CA: Pfeiffer, A Wiley Imprint.
Dick, W (1981). Instructional design models: Future trends and issues. Educational1ech-
n%gy, 21 (7),29-32.
Dick, W. (1997). A model for the systematic design of instruction. In R. D. Tennyson,
F. Schott, N. M. Sed & S. Dijkstra's (Eds.), Instructional design: International per-
spectives: Volume J- Theory, research and methods (pp. 361-369). Mahwah, NJ.:
Lawrence Erlbaum Associates, Publishers.
Dick, W, & Carey, L. (1996). The systematic design of instruction (4th ed). New York: Harper-
Coil ins Publishers.
Dick, w., Carey, L., & Carey, J. O. (2005). The systematic design of instruction (6th ed.).
New York: Allyn & Bacon/Longman Publishers.
Driscoll, M. P. (1984). Paradigms for research in instructional systems. Journal of Instructional
Development, 7 (4), 2-5.
Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the design
and delivery of instruction. In 0. Jonassen (Ed.), Handbook of research for educational
communications and technology (pp. 170-198). Mahwah, NJ: Lawrence Erlbaum Asso-
ciates, Publishers.
Edmonds, G. S., Branch, R. C, & Mukherjee, P. (1994). A conceptual framework for com-
paring instructional design models. Educational Technology Research and Development,
42(4),55-72.
Elder, Co., & Cobb, R.W (1984). Agenda-building and the politics of aging. Policy Stud-
ies Journal, 13 (1), 115-129.
Ely, D. (1990). Conditions that facilitate the implementation of educational technology
innovations. Journal of Research on Computing in Education, 23(2),298-305.
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert
performance in medicine and related domains. Academic Medicine, 79(10), 70-81.
Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition.
American Psychologist, 49(8), 725-747.
Finn, J. D. (1953). Professionalizing the audio-visual field. Audio-Visual Communication
Review, 1(1), 6-17-
Fischer, K. M., Savenye, W C, & Sullivan, H. J. (2002). Formative evaluation of computer-
based training for a university financial system. Performance Improvement Quarterly,
15(1), 11-24.
Flanagan, J. C. (1988). The critical incident technique. In R. Zemke and T. Kramlinger's
(Eds.), Figuring things out: A trainer's guide to needs and task analysis (pp. 277-317).
164 DESIGN AND DEVELOPMENT RESEARCII

Reading, MA: Addison-Wesley Publishing Company, Inc. (Reprinted from Psychologi-

cal Bulletin, v. 51 [July, 1954). pp. 327-58.)
Forsyth, J. E. (1998). The construction and validation of a model for the design of commu-
nity-based train-the-trainer instruction (Doctoral dissertation, Wayne State Univer-
sity, 1997). Dissertation Abstracts International-A, 58(11), 4242.
Fowler, F. j. (2002). Survey research methods (3rd ed.). Thousand Oaks, CA: Sage Publications.
Fox, E. j., & Klein, j. D. (2003). What should instructional designers and technologists
know about human performance technology? Performance Improvement Quarterly,
16(3),87-98.
Foxon, M., Richey, R. C., Roberts, R., &Spannaus, T. (2003). Training manager competencies:
The standards (3rd ed.). Syracuse, NY: ERIC Clearinghouse on Information and
Technology.
Frankfort-Nachmias, c., & Nachmias, D. (2000). Research methods in the social sciences
(6th ed.). New York: St. Marin's Press.
Freeman, P. (1983). Fundamentals of design. In P. Freeman (Ed.), Software design tutorial
(pp. 2-22). New York: IEEE Computer Society Press.
Gagne, R. M. (1985). The conditions oflearningand theory of instruction (4th ed.). New York:
Holt, Rinehart and Wins ton.
Gagne, R. M., Wager, W W, Golas, K. C., & Keller, j. M. (2005). Principles of instructional
design. (4th ed.) Belmont, CA: Wadsworthffhompson Learning.
Gall, M. D., Gall, J. P., & Borg, W R. (2003). Educational research: An introduction (7th
ed.). Boston: Allyn and Bacon.
Gardner, H. (1993). Frames of mind: The theory of multiple intelligences. New York: Basic
Books.
Gay, G., & Mazur, J. (1993). The utility of computer tracking tools for user-centered design.
Educational Technology, 33 (4),45-59.
George, A. L., & Bennett, A. (2005). Case studies and theory development in the social sciences.
Cambridge, MA: MIT Press.
Gettman, D., McNelly, T., & Muraida, D. (1999). The guided approach to instructional
design advising (GUIDA): A case-based approach to developing instructional design
expertise. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen & T. Plomp's
(Eds.), Design approaches and tools in education and training (pp. 175-182). Dordrecht,
The Netherlands: Kluwer Academic Publishers.
Gilbert, N. j., & Driscoll, M. P. (2002). Collaborative knowledge building: A case study.
Educational Technology Research and Development, 50(1), 59-79.
Greenhill, L. P. (1955). A study of the feasibility of local production of minimum cost sound
motion pictures. (Pennsylvania State University Instructional Film Research Program).
Port Washington, NY: US Naval Training Device Center, Office of Naval Research,
Technical Report No. SDC 269- 7-48.
Gustafson, K. L., & Branch, R. M. (2002). Survey of instructional development models (4th
ed.). Syracuse, NY: ERIC Clearinghouse on Information & Technology.
Gustafson, K. L., & Reeves, T. C. (1990). IDioM: A platform for a course development
expert system. Educational Technology, 30(3), 19-25.
Hallamon, T. C. (2002). A study of factors affecting the use of task analysis in the design
of instruction (Doctoral dissertation, Way ne State University, 2001). Dissertation
Abstracts International-A, 62(12), 4131.
llefi.'rel1feS 165

Harris, M., & Cady, M. (1988). The dynamic process of creating hypertext literature. Edu-
cational Technology, 28(11),33-40.
Heinich, R. (1984). The proper study of instructional technology. Educational Communica-
tion and 'li:chnology Journal, 32(2),67-87.
Hilgard, E. R. (Ed.). (1964). Theories of learning and instruction. The sixty-third yearbook of the
National Society for the Study of Education. Chicago: University of Chicago Press.
Higgins, N., & Reiser, R. (1985). Selecting media for instruction: An exploratory study.
Journal of Instl'llctional Development, 8 (2), 6-10.
Hirumi, A., Savenye, W., & Alien, B. (1994). Designing interactive videodisc-based
museum exhibits: A case study. Educational 'Iechnology Research and Development,
42(1),47-55.
Hoban, C. E (1953). Determinants of audience reaction to a training film. Audio-Visual
Communication Review, I (1),30-.37.
Hoover, K. R., & Donovan, T. (1995). The elements of social scientific thinking (6th ed.).
Belmont, CA: ThompsonlWadsworth.
johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research para-
digm whose time has come. Educational Researcher, 33(7), 14-16.
jonassen, D. H. (Ed.). (1996). Handbook of research for educational communications and tech-
nology. New York: Simon & Schuster Macmillan.
jonassen, D. H. (Ed.). (2004). Handbook of research for educational conlmunications and tech-
nology (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
jones, T S. (1999). Validating the process of designing and developing instructional mate-
rials using the rapid prototyping methodology (Doctoral dissertation, Wayne State
University, 1998). Dissertation Abstracts International-A, 59(12), 4409.
jones, T S., & Richey, R. C. (2000). Rapid prototyping in action: A developmental study.
Educational Technology Research and Development, 48(2), 63-80.
Kaner, j. H., & Rosenstein, A. j. (1960). 'Television in army training: Color vs. black and
white. Audio-Visual Communication Review, 8(6), 243-252.
Keller, j. M. (1987). The systematic process of motivational design. Performance & Instruc-
tion, 26(10), 1-8.
King, G., Keohane, R. 0., & Verba, S. (1994). Designing social inquiry: Scientific inference in
qualitative research. Prince ton, Nj: Princeton University Press.
Klein, j. D., Brinkerhoff, J., Koroghlanian, c., Brewer, S., Ku, H., & MacPherson-Coy, A.
(2000). The foundations of educational technology: A needs assessment. lech-Iimds,
44(6),32-36.
Klein, j. D., Nguyen, E, Bevill, L., Winter, c., Reisslein, j., & Fox, E. (2003, October). Teach-
ing performance improvement: Can we help clients identify and solve their problems? Paper
presented at the annual meeting of the Association for Educational Communications
and 'Iechnology. Anaheim, CA.
Klein, j. D., & Rushby, N. (2007). Professional organizations and publications in instruc-
tional design and technology. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and
issues in instructional design and technology (2nd ed.) (pp. 260-270). Saddle River,
Nj: Pearson Prentice Hall.
Klein, j. D., Spector, j. M., Grabowski, B., & de la Teja, I. (2004). Instructor competencies:
Standards for face-to-face, online, and blended settings. Greenwich, er: Information Age
Publishing.
166 DESIGN AND DEVELOPMENT RESEARCH

Koszalka, T. A. (2001). Designing synchronous distance education: A demonstration pro-

ject. The Quarterly Review of Distance Education, 2(4), 333-345.
Krejcie, R. v., & Morgan, D. W. (1970). Determining sample size for research activities.
Educational and Psychological Measurement, 30, 607-610.
Leedy, P. D. (1985). Practical research: Planning and design (3rd ed.). New York: Macmillan.
Le Maistre, C. (1998). What is an expert instructional designer? Evidence of expert per-
formance during formative evaluation. Educational Technology Research and Develop-
ment, 46(3), 21-36.
Levine, H. O. (1992). Naturalistic inquiry, types oE In M. C. Alkin (Ed.) Encyclopedia of edu-
cational research (6th ed.) (pp. 889-892). New York: Macmillan Publishing Company.
Li, Z., & Merrill, M. D. (1991). ID Expert 2.0: Design theory and process. Educational Tech-
nology Research and Development, 39(2), 53-69.
Link, N., & Cherow-O'Leary, R. (1990). Research and development of print materials at the
Children's Television Workshop. Educational Technology Research and Development,
38(4),34-44.
Littlejohn. S. W. (1978). Theories of human communication. Columbus, OH: Charles E.
Merrill Publishing Company.
Luiz, T. (1983). A comparative study of humanism and pragmatism as they relate to decision
making in instructional development processes (Doctoral dissertation, Michigan State
University, 1982). Dissertation Abstracts International-A, 43(12), 3839.
Lumsdaine, A. A. (1953). Audio-visual research in the U.S. Air Force. Audio-Visual Com-
munication Review, 1 (2), 76-90.
Markle, S. M. (1967). Empirical testing of programs. In P. C. Lange (Ed.) , Programed instruc-
tion: The sixty-sixth yearbook of the National Society for the Study of Education, Part II
(pp. 104-138). Chicago: The University of Chicago Press.
Martin, B., & Bramble, W. (1996). Designing effective video teletraining instruction: The
Florida teletraining project. Educational Technology Research and Development, 44(1),
85-99.
McCracken, O. (1988). The long interview. Newbury Park, CA: Sage Publications.
McKenney, S. (2002). Computer-based support for science education materials develop-
ment in Africa: Exploring potentials (Doctoral dissertation, Universiteit Twente [The
Netherlands], 2001). Dissertation Abstracts International-C, 63(03),355.
McKenney, S., & van den Akker; J. (2005). Computer-based support for curriculum designers:
A case of developmental research. Educational Technology Research and Development,
53(2),41-66.
McLellan, H. (2004). Virtual realities. In D. Jonassen (Ed.), Handbook of research for edu-
cational communications and technology (2nd ed.) (pp. 461-498). Mahwah, NJ:
Lawrence Erlbaum Associates, Publishers.
Means, T. B., Jonassen, D. H., & Dwyer, E M. (1997). Enhancing relevance: Embedded
ARCS strategies vs. purpose. Educational Technology Research and Development, 45 (1),
5-17.
Medley, D. M. (1992). Structured observation. In M. C. Alkin (Ed.), Encyclopedia of educa-
tional research (6th Ed.) (pp. 1310-1315). New York: Macmillan Publishing Company.
Merrill, M. D. (1983). Component display theory. In C. Reigeluth (Ed.), Instructional-design
theories and models: An overview of their current status (pp. 279-333). Hillsdale, NJ:
Lawrence Erlbaum Associates, Publishers.
References 167

Merrill, M. D., Drake, L., Lacy, M. J., Pratt, J. A, & the ID2 Research Group. (1996).
Reclaiming instructional design. Educational Technology, 36(5), 5-7.
Merrill, M. D., & Li, Z. (1989). An instructional design expert system. Journal of Computer-
Based Instruction, 16(3),95-101.
Milrad, M., Spector, J. M., & Davidsen, P. I. (2000). Building and using simulation based
environments for learning about complex domains. In R. Robson (Ed.), MSET/2oo0
conference proceedings (pp. 304-308). Charlottesville, VA: Association for the
Advancement of Computing in Education.
Mooij, T. (2002). Designing a digital instructional management system to optimize early
education. Educational Technology Research and Development, 50(4), 11-23.
Morrison, G. R., Ross, S. M., & Kemp, J. E. (2006). Designing effective instruction (5th ed.)
John Wiley & Sons, Inc.
Nelson, B., Ketelhut, D. J., Clarke, J., Bowman, c., & Dede, C. (2005). Design-based
research strategies for developing a scientific inquiry curriculum in a multi-user vir-
tual environment. Educational Technology, 45(1), 21-28.
Nguyen, E (2005). EPSS needs assessment: Oops, I forgot how to do that! Performance
Improvement, 44(9), 33-39.
Nguyen, E, Klein, J. D., & Sullivan, H. (2005). A comparative study of electronic perfor-
mance support systems. Performance Improvement Quarterly, 18(4),71-86.
Nieveen, N., & van den Akker, J. (1999). Exploring the potential of a computer tool
for instructional developers. Educational Technology Research and Development,
47(3), 77-98.
Norton, T. C. (2004a). Usability. In A Kovalchick & K. Dawson (Eds.), Education and
technology: An encyclopedia, Volume 2:J-2 (pp. 579-584) Santa Barbara, CA: ABC-
CLIO, Inc.
Norton, T. C. (2oo4b). Web-based surveys. In A Kovalchick & K. Dawson (Eds.), Educa-
tion and technology: An encyclopedia, Volume 2:J-2 (pp. 644-650) Santa Barbara, CA:
ABC-CLIO, Inc.
Orr, S. K. (2005). New technology and research: An analysis of Internet survey methodol-
ogy in political science. PS: Politics & Science, 38(2), 263-267.
Ottevanger, W (2001). Teacher support materials as a catalyst for science curriculum imple-
mentation in Namibia (Doctoral dissertation, Universiteit Twente [The Netherlands],
2001).
Patten, M. L. (2002). Proposing empirical research: A guide to the fundamentals (2nd ed.).
Los Angeles: Pyrczak Publishing.
Patten, M. L. (2004). Understanding research methods: An overview of essentials. (4th ed.).
Glendale: Pyrczak Publishing.
Patton, M. Q. (2001). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks,
CA: Sage.
Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park, CA: Sage
Publications, Inc.
Perez, R. S., & Emery, C. D. (1995). Designer thinking: How novices and experts think
about instructional design. Performance Improvement Quarterly, 8(3), 80--95.
Phillips, J. H. (2000). Evaluating training programs for organizational impact (Doctoral
dissertation, Wayne State University, 2000). Dissertation Abstracts Intemational-A
61 (03),840.
168 DESIGN AND DEVELOPMENT RESEARCH

Plass, J. L., & Salisbury, M. W. (2002). A living-systems design model for web-based knowl-
edge management systems. Educational Technology Research and Development, 50(1),
35-57.
Plummer, K. H., Gillis, P. D., Legree, P. J., & Sanders, M. G. (1992). The development and
evaluation of a job aid to support mobile subscriber radio-telephone terminal (MSRT).
Performance Improvement Quarterly, 5 (1), 90-105.
Preese, E, & Foshay, W. (1999). The PLATO courseware development environment. In
J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen & T. Plomp's (Eds.), Design
approaches and tools in education and training (pp. 195-204). Dordrecht, The Nether-
lands: Kluwer Academic Publishers.
Quifiones, M. A, Ford, J. K., Sego, D. J., & Smith, E. M. (1995/1996). The effects of indi-
vidual and transfer environment characteristics on the opportunity to perform trained
tasks. Training Researchlournal, 1 (1),29-49.
Ragan, T. J., & Smith, P. L. (2004). Conditions theory and models for designing instruc-
tion. In D. Jonassen (Ed.), Handbook of research for educational communications and
technology (2nd ed.) (pp. 623-649). Mahwah, NJ: Lawrence Erlbaum Associates,
Publishers.
Raybould, G. (1995). Performance support engineering: An emerging development
methodology for enabling organizational learning. Performance Improvement Quarterly,
8(1), 7-22.
Reeves, T. C. (2005). Design-based research in educational technology: Progress made,
challenges remain. Educational Technology, 45(1), 48-52.
Reigeluth, C. M. (1983). Instruction design: What is it and why is it? In C. M. Reigeluth
(Ed.) , Instructional design theories and models (pp. 3-36). Mahwah, NJ: Lawrence Erlbaum
Associates, Publishers.
Reigeluth, C. M. (Ed.). (1999). Instructional design theories and models, Volume 1I: A new
paradigm of instructional theory. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating
and improving design theories. In C. M. Reigeluth (Ed.), Instructional design theories and
models, Volume 1I: A new paradigm of instructional theory (pp. 633-651). Mahwah, NJ:
Lawrence Erlbaum Associates, Publishers.
Reiser, R. A (2002). A history of instructional design and technology. In R. A. Reiser &
J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 26-
53). Upper Saddle River, NJ: Pearson Prentice Hall.
Reiser, R. A, & Dempsey, J. V. (Eds.). (2007). Trends and issues in instructional design and
technology (2nd ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
Richey, R. C. (1986). Theoretical and conceptual bases of instructional design. London: Kogan
Page.
Richey, R. C. (1992). Designing instruction for the adult learner: Systemic training theory and
practice. London/Bristol, PA: Kogan PageITaylor and Francis.
Richey, R. C. (1997). Agenda-building and its implications for theory construction in
instructional technology. Educational Technology, 37 (1), 5-11.
Richey, R. C. (1998). The pursuit of useable knowledge in instructional technology. Edu-
cational Technology Research and Development, 46(4), 7-22.
Ref'erenC'es 169

Richey, R. C. (2005). Validating instruction design and development models. In J. M.

Spec tor & D. A. WHey (Eds.), Innovations in instructional technology: Essays in honor
of M. David Merrill (pp. 171-185). Mahwah, NJ: Lawrence Erlbaum Associates,
Publishers.
Richey, R., Fields, D., & Foxon, M. (2001). Instructional design competencies: The standards
(3rd ed.). Syracuse, NY: ERIC Clearinghouse on Information & Technology.
Richey, R. C., & Klein, J. D. (2005). Developmental research methods: Creating knowledge
from instructional design and development practice. Journal of Computing in Higher
Education, 16(2),23-38.
Richey, R. c., Klein, J., & Nelson, W (2004). Developmental research: Studies of instruc-
tional design and development. In D. Jonassen (Ed.), Handbook of research for educa-
tional communications and technology (2nd ed.) (pp. 1099-1130). Mahwah, NJ:
Lawrence Erlbaum Associates, Publishers.
Richey, R. c., & Nelson, W (1996). Developmental Research. In D. Jonassen (Ed.) Hand-
book of research for educational communications and technology (pp. 1213-1245). New
York: Simon & Schuster.
Ross, S. M., & Morrison, G. R. (2004). Experimental research methods. In D. Jonassen
(Ed.), Handbook of research for educational communications and technology (2nd ed.) (pp.
1021-1043). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Rossett, A. (1999). First things fast: A handbook for performance analysis. San Francisco:
Jossey-Bass/Pfeifer.
Rowland, G. (1992). What do instructional designers actually do? An initial investigation
of expert practice. Performance Improvement Quarterly, 5(2), 65-86.
Roytek, M. A. (2000). Contextual factors affecting the use of rapid prototyping within the
design and development of instruction (Doctoral dissertation, Wayne State University,
1999). Dissertation Abstracts International-A, 61 (01),76.
Rudestam, K. E., & Newton, R. R. (2001). Surviving your dissertation: A comprehensive guide
to content and process (2nd ed.). Thousand Oaks, CA: Sage Publications.
Russell, C. M. (1990). The development and evaluation of an interactive videodisc system
to train radiation therapy technology students on the use of the linear accelerator
(Doctoral dissertation, University of Pittsburgh, 1988). Dissertation Abstracts Interna-
tional-B, 50(3), 919.
Ryan, M., Hodson-Carlton, K., & Ali, N. S. (2005). A model for faculty teaching online:
Confirmation of a dimensional matrix. Journal of Nursing Education, 44 (8), 357-365.
Saroyan, A. (1993). Differences in expert practice: A case from formative evaluation.
Instructional Science, 21, 451-472.
Seels, B., & Glasgow, Z. (1998). Making instructional design decisions (2nd ed.). Columbus,
OH: Merrill Publishing Company.
Seels, B. B., & Richey, R. C. (1994). Instructional technology: The definition and domains of the
field. Washington, DC: Association for Educational Communications and Technology.
Shellnut, B. J. (1999). The influence of designer and contextual variables on the incorpo-
ration of motivational components to instructional design and the perceived success of
a project (Doctoral dissertation, Wayne State University, 1999). Dissertation Abstracts
International-A, 60(06), 1993.
170 DESIGN AND DEVELOPMENT RESEARCH

Shellnut, B. J., Knowlton, A., & Savage, T. (1999). Applying the ARCS model to the design
and development of computer-based modules for manufacturing engineering courses.
Educational Technology Research and Development, 47(2), 100-110.
Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Science, 61,394-403.
Singleton, R. A., Jr., & Straits, B. C. (2005). Approaches to social research (4th ed.). New
York: Oxford University Press.
Sleezer, C. M. (1991). The development and validation of a performance analysis for train-
ing model. (Vol. I -Ill) (Doctoral dissertation, University of Minnesota, 1990). Disser-
tation Abstracts International- A, 52 (01), 144.
Sloane, F. C., & Gorard, S. (2003). Exploring mode ling aspects of design experiments. Edu-
cational Researcher, 32(1), 29-31.
Smaldino, S. E., Russell, J. D., Heinich, R., & Molenda, M. (2005). Instructional technology
and media for learning (8th ed.). Upper Saddle River, NJ: Pearson/Merrill Prentice Hall.
Small, R. v., & Gluck, M. (1994). The relationship of motivational conditions to effective
instructional attributes: A magnitude scaling approach. Educational Technology, 34(8),
33-40.
Smith, P. L., & Ragan, T. J. (2005). Instructional design (3rd ed.). Hoboken, NJ: John Wiley
& Sons, Inc.
Spector, J. M., Muraida, D. J., & Marlino, M. R. (1992). Cognitively based models of
courseware development. Educational Technology Research and Development, 40(2),
45-54.
Spector, J. M., & Song, D. (1995). Automated instructional design advising. In R. D. Ten-
nyson & A. E. Baron (Eds.), Automating instructional design: Computer-based develop-
ment and delivery tools (pp. 377--402). Brussels, Belgium: Springer-Verlag.
Stolovitch, H. D., & Keeps, E. J. (Eds.). (1999). Handbook of human performance technology
(2nd Ed.). San Francisco: Jossey-Bass/Pfeiffer.
Sullivan, H., Ice, K., & Niedermeyer, F. (2000). Long-term instructional development:
A 20-year ID and implementation project. Educational Technology Research and Devel-
opment, 48(4), 87-99.
Surry, D. w., & Ely, D. P. (2002). Adoption, diffusion, implementation, and institutional-
ization of instructional design and technology. In R. A. Reiser & J. v. Dempsey (Eds.),
Trends and issues in instructional design and technology (pp. 183-193). Upper Saddle
River, NJ: Pearson Prentice Hall.
Taylor, B., & Ellis, J. (1991). An evaluation of instructional systems development in the
Navy. Educational Technology Research and Development, 39(1), 93-103.
Teachout, M. S., Sego, D. J., & Ford, J. K. (199711998). An integrated approach to sum-
mative evaluation for facilitating training course improvement. Training Research}our-
nal, 3, 169-184.
Tessmer, M., McCann, D., & Ludvigsen, M. (1999). Reassessing training programs: A model
for identifying training excesses and deficiencies. Educational Technology Research and
Development, 47(2), 86-99.
Tessmer, M., & Richey, R. C. (1997). The role of context in learning and instructional
design. Educational Technology Research and Development, 45(2),85-115.
Rereren('e.~ J7I

Tracey, M. W. (2002). The construction and validation of an instructional design model

for incorporating multiple intelligences (Doctoral dissertation, Wayne State University,
2001). Dissertation Abstracts International-A, 62 (12), 4135.
Tracey, M. W., & Richey, R. C. (in press). ID model construction and validation: A mul-
tiple intelligences case. Educationallechnology Research & Development.
Tuckman, B. W. (1999). Conducting educational research (5th ed.). Belmont, CA: Wadsworth
Groupffhomson Learning.
lwitchell, S., Holton, E. E, & Trott, J. W. (2000). Technical training evaluation practices
in the United States. Peljormance Improvement Quarterly, 13(3),84-110.
van den Akker, J. (1999). Principles and methods of development research. In J. van den
Akker, R. M. Branch, K. Gustafson, N. Nieveen & T Plo mp's (Eds.) Design approaches
and tools in education and training (pp. 1-14). Dordrecht, The Netherlands: Kluwer
Academic Publishers.
van Merrienboer, J. J. G., & Dijkstra, S. (1997). The four-component instructional
design model for training complex cognitive skills. In R. D. Tennyson, E Schot, N.
M. Seel, & S. Dijkstra (Eds.), Instructional design: International perspectives, Volume
1: Theory research and models (pp. 427-445). Mahwah, NJ: Lawrence Erlbaum Asso-
ciates, Publishers.
Visscher-Voerman, J. I. A. (2000). Design approaches in training and education: A re con-
structive study (Doctoral dissertation, Universiteit Twente [The Netherlands], 1999).
Dissertation Abstracts IntematiOlUlI-C, 61 (03),638.
Visscher-Voerman, I., & Gustafson, K. L. (2004). Paradigms in the theory and practice of
education and training design. Educational-fecllllology Research and Development, 52 (2),
69-89.
Visser, L. (1998). The development of motivational communication in distance education
support. (Doctoral dissertation, University of Twente, NL).
Visser, L., Plomp, T, Amirault, R. J., & Kuiper, W. (2002). Motivating students at a distance:
The case of an international audience. Educational Technology Research and Develop-
ment, 50(2), 94-110.
Wallen, N. E., & Fraenkel, J. R. (2001). Educational research: A guide 10 the process (2nd
ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Wang, E, & Hannafin, M. J. (2005). Design-based research and technology-enhanced
learning environments. Educational Technology Research and Development, 53(4),
5-23.
Wang, E K., Moore, J. L., Wedman, J., & Shyu, C. R. (2003). Developing a case-based rea-
soning knowledge repository to support a learning community-An example from
the technology integration community. Educational Technology Research and Develop-
ment, 51 (3), 45-62.
Weston, C., McAlpine, L., & Bordonaro, T (1995). A model for understanding formative
evaluation in instructional design. Educational Technology Research and Development,
43(3),29-48.
Wiley, D. A. (Ed.). (2000). Instructional use of learning objects. Bloomington, IN: Associa-
tion for Educational Communications and Technology.
172 DESIGN AND DEVELOPMENT RESEARCH

Willis, J. (1995). A recursive, reflective instructional design model based on constructivist-

interpretivist theory. Educational Technology, 35(6), 5-23.
Winn, W (2004). Cognitive perspectives in psychology. In D. Jonassen (Ed.), Handbook
of research for educational communications and technology (2nd ed.) (pp. 79-112). Mah-
wah, NJ: Lawrence Erlbaum Associates, Publishers.
Wolfe, P., Wetzel, M., Harris, G., Mazour, T., & Riplinger, J. (1991). Job task analysis: Guide
to good practice. Englewood Cliffs, NJ: Educational Technology Publications.
Young, A. C., Reiser, R. A., & Dick, W (1998). Do superior teachers employ systematic
instructional planning procedures? A descriptive study. Educational Technology Research
and Development, 46(2), 65-78.
Zielinski, D. (2000). Objects of desire. Training, 37(9), 126-134.