How do differing theoretical and philosophical perspectives shape qualitative inquiry, and why is this diversity important?

Differing theoretical and philosophical perspectives form the foundation of qualitative inquiry, influencing how studies are structured, data is collected, and analyses are conducted. The diversity of perspectives—ranging from ethnography and phenomenology to symbolic interactionism and critical theory—enables researchers to approach research questions from multiple angles, enriching the understanding of complex social phenomena. This variety allows for more nuanced interpretations of data and reflects the complexity of human experiences .

What challenges and considerations are associated with determining sample size in qualitative research, and how should researchers address them?

Determining sample size in qualitative research involves balancing the need for depth and detail with practical constraints such as resources and time. The ambiguity inherent in qualitative design means there's no fixed rule for sample size, which depends on the study's purpose, the richness of the data, and the study's goals for variation. Researchers address these challenges by using purposeful sampling to select cases that maximize learning potential, taking into account the study's context and the importance of comprehensive exploration over broad generalization .

In what ways do the practical applications of qualitative inquiry in professional fields like design demonstrate its value beyond academic research?

In fields like design, qualitative inquiry provides critical insights into consumer interactions with products in real-world settings. Techniques such as ethnography allow designers to observe and understand user behavior and needs, which can inform the development of products that better satisfy consumer requirements. This practical application demonstrates the value of qualitative methods in creating user-centered designs and highlights the potential for qualitative research to drive innovation by grounding it in real user experiences and cultural contexts .

How do Sara Johnson's experiences illustrate the impact of obtaining a General Equivalency Diploma (GED) on personal identity and career development?

Sara Johnson's experiences highlight the profound impact that obtaining a GED can have on personal identity and career development. After dropping out of school in the 10th grade and subsequently fearing professional rejection due to her lack of a diploma, obtaining a GED served as an affirmation for Sara. Her decision to go back to school alleviated her fears of being discovered as a dropout and losing her job. It also allowed her to immediately enter college, showcasing a pivotal shift in her professional trajectory and self-confidence .

What is meant by 'orientational' studies in qualitative research, and how does it differ from 'ideologically based' inquiry?

'Orientational' studies in qualitative research refer to research oriented in a particular direction or framed from a specific perspective, unlike 'ideologically based' inquiries which may imply a bias or agenda. Orientational studies maintain a degree of neutrality by focusing on elucidation rather than discovery. This approach emphasizes understanding social phenomena through theoretically driven frameworks, allowing for cultural and social insights while remaining clear about the implications of their perspectives .

How does the use of case studies complement statistical data in qualitative research, and why are they important?

Case studies complement statistical data by providing depth, detail, and individual meaning to otherwise broad patterns shown by statistics. While statistical data can provide concise summaries of major patterns, case studies reveal complex, detailed personal stories, deepening the understanding of individual variations within those patterns. This combination of quantitative and qualitative approaches offers a richer, more comprehensive picture of the subject matter .

Evaluate how participant observation can enrich qualitative research findings, using the example of a wilderness program.

Participant observation enriches qualitative research by allowing researchers to gather data in a natural setting, directly observing interactions and behaviors. In the wilderness program example, it enabled the researcher to understand participants' reactions and interactions during unstructured program time. Observing an older man, initially resistant to the program, writing intently, demonstrated the potential for personal engagement and reflection, which might not be captured through other methods. This immersive approach provides nuanced insights into participant experiences and shifts in perspective .

How can qualitative research benefit from studying non-events or things that do not happen?

Qualitative research can gain significant insights by studying non-events, as they reveal underlying assumptions, expectations, and routines within a setting. Observing the absence of expected interactions or activities can highlight relational dynamics and systemic structures influencing behavior. For instance, not observing certain expected activities might indicate implicit group norms or reveal the effectiveness of interventions. These observations are crucial for a comprehensive understanding of the social context and can illuminate underlying causal relationships .

Discuss the significance of open-ended group interviews in understanding participant perspectives in qualitative research.

Open-ended group interviews play a crucial role in qualitative research by providing a platform for participants to express their perspectives in their own words. This approach reveals deeper insights than closed-ended surveys by facilitating the emergence of insights through interaction, enhancing understanding of participant experiences. Group interviews allow for a dynamic exchange of ideas, which can generate a rich body of qualitative data and synergistically lead to new insights .

What role does 'purposeful sampling' play in qualitative research, and how can it be strategically employed?

Purposeful sampling is crucial in qualitative research as it aims to select information-rich cases that can offer deep insights into issues of central importance to the study's purpose. This method allows researchers to focus intensively on specific cases that will illuminate the questions under study. Strategies such as extreme or deviant case sampling can produce significant insights by studying unique or special cases, which may not be representative but are richly informative and enhance understanding of the phenomenon .

Open navigation menu

Upload

95% found this document useful (56 votes)

22K views690 pages

Qualitative Research and Evaluation Methods PDF

Uploaded by

albertodarocha

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

95% found this document useful (56 votes)

22K views690 pages

Qualitative Research and Evaluation Methods PDF

Uploaded by

albertodarocha

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 690

rvalua+ion

Metkods

Michael Quinn Patton

]
To the faculty and learners of The Union Institute, Cincinnati, Ohio,
for their friendship and scholarship, and their commitments to
methodological eclecticism, interdisciplinary inquixy, integration of
theory and practice, valuing both reflection and action, scholarship
that is socially relevant and meaningful, individualized professional
and personal development, lifelong learning/ social justice and equity,
human diversty and global community; a scholarly community
governed by principies and processes rather than rules and
regulations, and iimovations in learning-centered, nontraditional
doctoral education, including faculty meetings that are interesting
and important, an indication of innovation of the highest order
KF/^
F A C U L D A D E DE n-wc
DESPORTO t D E E D C a A
U N I V E R S I D A D E DO P O R T C

BBLOTHC.

EDITION

Michael Quinn Patton

F A C U L D A D E PE CINCIAS DO
HPORTO DESPORTO E D- EDUCAO FSiCA
FACULDADE DE DESPORTO

O
UNIVERSIDADE DO PORTO
UNV
fERSD
f ARE 00 PORTO

I
015774
| /Q^
/vSage Publications
4 SC I X I International Educational and ProfessionaI Publisher
Thousand Oaks London New Delhi
Ty/fTM FCDEF
Copyright 2002 by Sage Publications, Inc.

Ali rights reserved. No part of this book may be reproduced or utilize d in any form or by
any means, electronic or mechanical, including photocopying, recording, orby any infor-
ma tion storage and retrieval system, without permission in writing from the publisher.

For information:

Sage Publications, Inc.

2455 Teller Road
Thousand Oaks, Califrnia 91320
E-mail: [email protected]
Sage Publications Ltd.
6 Bonhill Street
London EC2A 4PU
United Kingdom
Sage Publications ndia Pvt. Ltd.
M-32 Market
Greater Kailash I
New Delhi 110 048 ndia
Printed in the United States of America

Library of Congress Cataloging-in-Publication Data

Patton, Michael Quinn.

Qualitative research and evaluation methods / by Michael Quinn
Patton. 3rd ed.
p. cm.
Rev. ed. of: Qualitative evaluation and research methods. 2nd ed. 1990. )
Includes bibliographical references and index.
ISBN 0-7619-1971-6
1. Social sciencesMethodology. 2. Evaluation research (Social
action programs). I. Patton, Michael Quinn. Qualitative evaluation and
research methods. II. Title.
H62.P3218 2001
300'.7'23dc21 2001005181

04 05 06 10 9 8 7 6 5 4

Acquiring Editor: C. Deborah Laughton

Editorial Assistant: Vernica Novak
Production Editor: Diana E. Axelsen
Editorial Assistant: Kathryn Joumey
Copy Editor: Kate Peterson
Typesetter/Desigtier: Janelle LeMaster
Cover Designer: Michelle Lee
BHef (Confervfs

Preface xxi

PjA"RTT 1 . Conceptual Issues in Qualitative Inquiry 1

1. The Nature of Qualitative Inquiry 3

2. Strategic Themes in Qualitative Inquiry 37
3. Varie ty in Qualitative Inquiry: Theoretical Orientations 75

4. Particularly Appropriate Qualitative Applications 143

P A R T 2 . Qualitative Designs and Data Collection 207

5. Designing Qualitative Studies 209
6. Fieldwork Strategies and Observation Methods 259
7. Qualitative Interviewing (^339^

P j A R T 3 . Analysis, Interpretation, and Reporting 429

8. Qualitative Analysis and Interpretation (431^)

9. Enhancing the Quality and Credibility of Qualitative Analysis

References RI
Author Index 11
Subject Index 113

About the Author Al

Defailed (Zov\ie.rs

Preface xxi

P . A R T 1 , Conceptual Issues in Qualitative Inquiry 1

1. The Nature of Qualitative Inquiry 3

The Fruit of Qualitative Methods 3
|ggSj||i Three Kinds of Qualitative Data 4
Recognizing Qualitative Data 4
Qualitative Findirigs: Themes, Patterns/ Concepts, Insights, Understandings 5

E H 1 H Women's Ways of Knowing: An Example of

Qualitative Frndings 7
mHnmBn
Coming-of-Age Paradigms 9
Different Purposes of and Audiences for Qualitative Studies:
Research, Evaluation, Dissertations, and Personal Inquiry 9
Making Methods Decisions 12
Methods Choices: Contrasting Qualitative and Quantitative Emphases 12
nsmnnHn
m U l i ^ l U B i Some Guiding Questions and Options for Methods Decisions 13
Comparing Two Kinds of Data: An Example 14
The Power of Qualitative Data 17
Face Validity and Credibility 20
The Purpose of Open-Ended Responses 20
Inquiry by Observation 21
The Raw Data of Qualitative Inquiry 26
People-Oriented Inquiry 27
The Fruit of Qualitative Methods Revisited 28
^ ^ [ f f l g Internet E-mail Discussion Groups (listservs)
on Qualitative Methods 29

Between-Chapters Interlude:
Top Ten Pieces of Advice to a Graduate Student
Considering a Qualitative Dissertation 31

2. Strategic Themes in Qualitative Inquiry 37

General Principies 37
The Purpose of a Strategic Framework 38
Design Strategies for Qualitative Inquiry 39
Naturalistic Inquiry 39
|X]J|||]||9I Themes of Qualitative Inquiry 40
Emergent Design Flexibility 43
Purposeful Sampling 45
Data Collection and Fieldwork: Strategies for Qualitative Inquiry 47
Qualitative Data 47
Direct Personal Experience and Engagement: Going Into the Field 47
Empathic Neutrality 49
Empathy and Insight 51
A Dynamic, Developmental Perspective 54
Analysis Strategies for Qualitative Inquiry 55
Unique Case Orientation 55
Inductive Analysis and Creative Synthesis 55
Holistic Perspective 58
Context Sensitivity 61
Voice and Perspective: Reflexivity 63
i i Reflexive Questions: Triangulated Inquiry 66
From Strategic Ideais to Practical Choices 66
Beyond Competing Inquiry Paradigms 68
Pragmatism 71
Ideal Conditions for Research: A Cautionary Tale 72
3. Variety in Qualitative Inquiry: Theoretical Orientations 75
Special Gifts 75
From Core Strategies to Rich Diversity 76
Which Approach Is Right? 77
Alternative Ways of Distinguishing Qualitative Traditions 79
Foundational Questions 80
Theoretical Traditions and Orientations 81
Ethnography 81
B J M I Culture, Culture Everywhere: Sample of Media Headlines 83
Autoethnography and Evocative Forms of Inquiry 84
l52jm[|lBS9 Varieties of Autoethnography: A Partial Lexicology 85
Truth and Reality-Oriented Correspondence Theory:
Positivist, Realist, and Analytic Induction Approaches 91
Social Construction and Constructivism 91
Constructivism Versus Constructionism 97
Phenomenology 104
Heuristic Inquiry 107
Qualitative Heuristics: A German Alternative Tradition 109
Ethnomethodology 110
Symbolic Interaction 112
Hermeneutics 113
Narra tology or Narrative Analysis 115
Ecological Psychology 118
A Systems Perspective and Systems Theory 119
Chos and Complexity Theory: Nonlinear Dynamics 123
Grounded Theory 124
Complexity (Chos) Theory Precepts and Qualitative
Inquixy Implications 126
Orientational Qualitative Inquiry: Feminist Inquiry Criticai Theory
and Queer Theory as Examples 129
Variety in Qualitative Inquiry: Different Answers to Core Questions 131
XO^KKI Variety in Qualitative Inquiry: Theoretical Traditions 132
Pragmatism 135

B B Sample Internet E-mail Discussion Groups (listservs) and

Sites Relevant to Qualitative Inquiry and Theory 136
The Apple of Your Eye 137
APPENDIX 3.1. Example of Autoethnographic Writing 138
4. Particularly Appropriate Qualitative Applications 143
Apprenticeship in Pragmatism 143
Practical Purposes and Concrete Questions 145
A Focus on Quality 145
Quality Assurance and Program Evaluation 147
^^^QQOm Comparing Program Evaluation and Quality Assurance 149
Evaluation Applications 151
Outcomes Evaluation 151
Evaluating Individualized Outcomes 152
I I B e h i n d the Numbers of an Employment Program:
The Story of Li 155
Process Studies 159
Implementation Evaluation 161
Logic Models and Theories of Action 162
Evaluability Assessments 163
Comparing Programs: Focus on Diversity 163
155181131111 Types of Teacher Centers 166
Prevention Evaluation 166
Documenting Development Over Time and Investigating
System Changes 167
From Evaluation Issues to Evaluation Models 168
Evaluation Models 169
Goal-Free Evaluation 169
Transaction Models: Responsive and Illuminative Evaluation 171
Connoisseurship Studies 172
Utilization-Focused Evaluation 173
Interactive and Participatory Applications 175
Personalizing and Humanizing Evaluation 175
Harmonizing Program and Evaluation Values 176
Developmental Applications: Action Research, Action Learning,
Reflective Practice, and Learning Organizations 177
j^QQJ^yjQQI Common Principies Undergirding Qualitative Inquiry
and Humanistic Values 177
Matching Program Philosophy and Evaluation Approach:
An Illustration 178
Appreciative Inquiry 181
Participatory Research and Evaluation: Valuing and Facilitating
Collaboration 182
Principies of Fully Participatory and Genuinely
Collaborative Inquiry 185
Supporting Democratic Dialogue and Deliberation 185
Supporting Democracy Through Process Use: Helping the
Citizenry Weigh Evidence and Think Evaluatively 187
Special Applications 191
The Need for Unobtrusive Measures 191
State-of-the-Art Considerations: Lack of Proven Quantitative
Instrumentation 192
Confirmatory and Elucidating Research: Adding Depth, Detail,
and Meaning to Quantitative Analyses 193
Rapid Reconnaissance 194
Capturing and Communicating Stories 195
Example of a "Most Significant Change" Story 197
Legislative Auditing and Monitoring 198
Futuring Applications: Anticipatory Research and Prospective
Policy Analysis 200
Breaking the Routine: Generating New Insights 202
Summary: A Vision of the Utility of Qualitative Methods 202
Qualitative Inquiry Applications: Summary Checklist
of Particularly Appropriate Uses of Qualitative Methods 204
EXHIBIT 4.9 Sample Internet E-mail Discussion Groups (listservs)
and Sites Relevant to Qualitative Applications and Practice 205

P A R T 2 . Qualitative Designs and Data Collection 207

5. Designing Qualitative Studies 209

The First Evaluation 209
A Meta-Evaluation 211
Clarity About Purpose: A Typology 213
Basic Research 215
EXHIBIT 5.1 Fundamental Disciplinary Questions 216
Applied Research 217
Sample Interdisciplinary Applied Research Questions 218
Evaluation Research: Summative and Formative 218
Action-Oriented, Problem-Solving Research 221
The Purpose of Purpose Distinctions 222
Examples of Types of Research Questions: A Family Research Example 223
Criticai Trade-Offs in Design 223

A Typology of Research Purposes 224

^QQQQ^IjQQI Family Research Example: Research Questions

Matched to Research Category 225
Breadth Versus Depth 227
Units of Analysis 228
Purposeful Sampling 230
f ^ j ^ j j j Q Q Examples of Units of Analysis for Case Studies and
Comparisons 231
Information-Rich Cases 242
Sample Size 242
E&UBBB^^BBBSBBB^BBB
B a Sampling Strategies 243
Emergent Designs and Protection of Human Subjects 246
Methodological Mixes 247
Triangulation 247
Mixing Data, Design, and Analysis Approaches 248
The Case of Operation Reach-Out: Variations in Program
Evaluation Design 249
Altemative Pure and Mixed Strategies 251
E 3 J Measurement, Design, and Analysis:
Pure and Mixed Combinations 252
Design and Methods Decisions 253
Design Issues and Options 254
mSBSBiuSfinSfiSEmSSHS ^
Choices 257

6. Fieldwork Strategies and Observation Methods 259

To Understand the World 260
Folk Wisdom About Human Observation 261
The Value of Direct Observations 264
Observation-Based Evaluation and Applied Research in a
Political World 265
Variations in Observational Methods 265
Variations in Observer Involvement: Participant or Onlooker or Both? 265
Insider and Outsider Perspectives: Emic Versus Etic Approaches 267
Who Conducts the Inquiry? Solo and Team Versus Participatory
and Collaborative Approaches 269
Overt Versus Covert Observations 273
Variations in Duration of Observations 273
Variations in Observational Focus 275
Dimensions Along Which Fieldwork Varies: An Overview 276
What to Observe: A Sensitizing Framework 276
KQ^njnnQn Dimensions Showing Fieldwork Variations 277
Sources of Data 279
E P ^ ^ m Examples of Sensitizing Concepts 280
The Setting 280
B H J i n Example of Combining Description and Metaphor
to Provide a Sense of Place 282
The Human, Social Environment 283
Historical Perspectives 284
Planned Program Implementation Activities and Formal Interactions 285
Informal Interactions and Unplanned Activities 285
The Native Language of the Program 288
Nonverbal Communication 290
Unobtrusive Observations 291
Documents 293
Observing What Does Not Happen 295
Nested and Layered Case Studies Dnring Fieldwork 297
Observing Oneself 299
Nested, Layered, and Overlapping Mini-Case Studies
During Fieldwork: Example From the Wilderness
Education Program Evaluation 300
Sources of Data Reviewed 301
Creativity in Fieldwork 302
Doing Fieldwork: The Data-Gathering Process 302
Field Notes 302
EXHIBIT 6.5 Fieldnotes Comparisons 304
Procedurally Speaking 305
Observations, Interviews, and Documentation: Bringing Together
Multiple Perspectives 306
The Technology of Fieldwork and Observation 307
Stages of Fieldwork 310
Entry Into the Field 310
What You Say and What You Do 314
Routinization of Fieldwork: The Dynamics of the Second Stage 317
Key Informants 321
Bringing Fieldwork to a Close 322
Evaluation Feedback 324
The Observer and What Is Observed: Unity and Separation 326
The Personal Experience of Fieldwork 329
A Part of and Apart From the World Observed 329
Summary Guidelines for Fieldwork 330

^ ^ Q H Q Summary Guidelines for Fieldwork 330

Between-Chapters Interlude:
Outside to Inside, Inside to Outside: Shifting Perspectives 333
Preface 335
"Nothing About Us, Without Us" 335
Barbara Lee
Qualitative Interviewing 339
Beyond Silent Observation 339
Rigorous and Skillful Interviewing 340
Inner Perspectives 340
Variations in Qualitative Interviewing 341
The Informal Conversational Interview 342
The Interview Guide 343
The Standardized Open-Ended Interview 344
QJQ^QQH Evaluation Interview Guide for Participants in an
Employment Training Program 345
Combining Approaches 347
Summary of Interviewing Strategies 348
Question Options 348
Experience and Behavior Questions 348
M i Variations in Interview Instrumentation 349
Opinion and Values Questions 350
Feeling Questions 350
Knowledge Questions 350
Sensory Questions 350
Background/Demographic Questions 351
Distinguishing Question Types 351
The Time Frame of Questions 351
HxffiHHHKB A Matrix of Question Options 352
Sequencing Questions 352
Wording Questions 353
Asking Truly Open-Ended Questions 353
The Horns of a Dichotomy 354
Asking Singular Questions 358
Clarity of Questions 361
Why to Take Care Asking "Why?" 363
Rapport and Neutrality 365
Neutral Questions 365
Using Illustrative Examples in Questions 366
Role-Playing and Simulation Questions 367
Presupposition Questions 369
Alternative Question Forma ts 370
Prefatory Statements and Announcements 370
Probes and Follow-Up Questions 371
Process Feedback During the Interview 374
Support and Recognition Responses 375
Maintaining Control and Enhancing the Quality of Responses 375
The One-Shot Question 378
The Final or Closing Question 379
Beyond Technique 379
Mechanics of Gathering Interview Data 380
Recording the Data 380
BSSBFFHHHI Tins fnr Taoe-Rerordine: Interviews: How to Keeo
Transcribers Sane 382
Taking Notes During Interviews 383
After the Interview 383
Special Applications and Issues 385
Think-Aloud Protocol Interviewing 385
Focus Group Interviews 385
Group Interviews 390
Cross-Cultural Interviewing 391
Language Differences 392
Differing Norms and Values 393
Beyond Standard Interviewing: Creative Qualitative Modes of Inquiry 394
Participant Interview Chain 396
Data Collection by Program Staff 397
j M S I U I I I ^ S Training Nonresearchers as Focus Group Interviewers:
Women Leaving Prostitution 399
Interactive Group Interviewing and Dialogues 400
Creativity and Data Quality: Qualitative Bricolage 400
Specialized and Targeted Interview Approaches 402
Ethical Challenges in Qualitative Interviewing 405
Informed Consent and Confidentiality 407
i s a y n n i l i s i Ethical Issues Checklist 408
New Directions in Informed Consent: Confidentiality
Versus People Owning Their Own Stories 411
Reciprocity: Should Interviewees Be Compensated? If So, How? 412
How Hard Should Interviewers Push for Sensitive Information? 415
Be Careful. It's Dangerous Out There. 415
Personal Reflections on Interviewing 416
An Interview With the King of the Monkeys 417
Halcolm on Interviewing 418
APPENDIX 7.1. Sample of a Detailed Interview Guide 419
APPENDIX 7.2. Examples of Standardized Open-Ended Interviews 422

P;A~RT 3 . Analysis, Interpretation, and Reporting 429

8. Qualitative Analysis and Interpretation 431

The Complete Analysis Isn't 431
The Challenge 432
Purpose as Context 434
When Does Analysis Begin? 436
Thick Description 437
Options for Organizing and Reporting Qualitative Data 439
Organizing the Data 440
Protecting Data 441
Computer-Assisted Qualitative Data Management and Analysis 442
KMHHffifli Examples of Software Programs for Qualitative Analysis 444

I M l i l i Internet Resources and E-mail Discussion Groups (listservs)

on Qualitative Analysis 445
Case Studies 447

UJ23QE&I Case Study: Layers of Possible Analysis 448

From Data to Case Study 449
i B i l i i The Process of Constructing Case Studies 450
Pattern, Theme, and Content Analysis 452
Inductive and Deductive Qualitative Analyses 453
Indigenous Concepts and Practices 454
Sensitizing Concepts 456
Indigenous Typologies 457
Analyst-Constructed Typologies 458
||||]||||!gj Qualitative Analysis of Ancestry at the U.S. Census 461
The Intellectual and Mechanical Work of Analysis 462
Coding Data, Finding Patterns, Labeling Themes, and Developing
Category Systems 462
IMUflCT First-Cut Coding Examples: Sample Codes From the
Field Note Margins 464
Convergence and Divergence in Coding and Classifying 465
Determining Substantive Significance 467
Logical Analysis 468
^TQnnQgl An Empirical Typology of Teacher Roles in Dealing With
High School Dropouts 469
A Process/Outcomes Matrix 471
E B E B I Mapping Stakeholders' Stakes 472
Conceptual Guide for Data Collection and Analysis:
Utilization of Planmng, Evaluation, and Reporting 473
^yOQQQIP Matrix of Linkages Between Program Processes and Impacts 474
An Analysis Example: Recognizing Processes, Outcomes, and
Linkages in Qualitative Data 474
Interpreting Findings 477
Interpreting for Meaning 477
Comparisons, Causes, Consequences, and Relationships 478
Theory-Based Analysis Approaches 481
Phenomenological Analysis 482
Grounded Theory 487
Qualitative Comparative Analysis 492
Analytic Induction 493
Special Analytical Issues and Frameworks 494
Reflexivity and Voice 494
Collaborative and Participatory Analyses 496
The Hermeneutic Circle and Interpretation 497
Analyzing Institutional Documents 498
Dramaturgical Analysis 499
Finding Nothing 500
Synthesizing Qualitative Studies 500
Reporting Findings 502
Balance Between Description and Interpretation 503
Communicating With Metaphors and Analogies 504
Drawing Conclusions 506
Special Issues in Evaluation Reporting and an Example 506
Feedback and Analysis 506
Evaluative Feedback Using Indigenous Typologies 507
QJJ^QjJQ Distinguishing Observations From Perceived Impacts 509
To Write a Report or Not to Write a Report? 510
Focus 511
The Executive Summary and Research Abstract 511
I B g M i l l Utilization-Focused Evaluation Reporting 512
Carpe Diem Briefings 512
The Creativity of Qualitative Inquiry 512
The Past and the Future: Deciding in Which Direction to Look 515
APPENDIX 8.1. Excerpts From a Codebook for Use by Multiple Coders 516
APPENDIX 8.2. Mike: An Dlustrative Case Study 518
APPENDIX 8.3. Excerpts From an Illustrative Interview Analysis:
Reflections on Outcomes From Participants in a
Wilderness Education Program 525

Between-Chapters Interlude:
Riddles of Qualitative Inquiry: Who Am I? 537
Gary D. Shank

9. Enhancing the Quality and Credibility of Qualitative Analysis 541

Interpreting Truth 541
Alternative Criteria for Judging Quality 542
^^QQQyQQ Alternative Sets of Criteria for Judging the Quality
and Credibility of Qualitative Inquiry 544
Traditional Scientific Research Criteria 544
Social Construction and Constructivist Criteria 546
Artistic and Evocative Criteria 547
Criticai Change Criteria 548
Evaluation Standards and Principies 549
Clouds and Cotton: Mixing and Changing Perspectives 551
Credibility 552
Rigor: Strategies for Enhancing the Quality of Analysis 553
Integrity in Analysis: Generating and Assessing Rival Conclusions 553
Negative Cases 554
Triangulation 555
ISfflMKEl A Story of Triangulation: Testing Conclusions With
More Fieldwork 559
Design Checks: Keeping Methods and Data in Context 563
High-Quality Lessons Learned 564
I^Sn^QEQI High-Quality Lessons Leamed 565
The Credibility of the Researcher 566
Considering Investiga tor Effects: Varieties of Reactivity 567
Intellectual Rigor 570
The Paradigms Debate and Credibility 570
Beyond the Numbers Game 572
Beyond Objectivity and Subjectivity: New Concepts, New Language 574
Reflections on Truth and Utility as Criteria of Quality 577
From Generalizations to Extrapolations and Transferability 581
The Credibility Issue in Retrospect: Increased Legitimacy for
Qualitative Methods 584
Beyond the Qualitative-Quantitative Debate 584
Matching Claims and Criteria 587
APPENDIX 9.1. Case Study: A Documenter's Perspective 589

Refere nc es RI

Author Index II

Subject Index 113

About the Author Al

Preface.

T he story is told that at the conclusion of a rigorous course in philoso-

phy, one of the students lamented: "Professor, you have knocked a
hole in everything I've ever believed in, but you have given me nothing to
take its place."
To which the philosopher replied: "You will recall that among the labors
of Hercules he was required to clean out the Augean stables. He was not, let
me point out, required to fill them,"

In doing this revision, I reviewed over a

W hile part of the task of this revi-

sion has been to clean out the
qualitative Augean stables, the
truly Herculean task has been deciding
thousand new books on qualitative meth-
ods, program evaluation, case studies,
monographs, and related works published
what to add. Unlike the professor who can in the last decade, as well as hundreds of ar-
be content with getting the stables cleaned, ticles scattered through scores of journals
the author of a revision bears responsibility covering the full range of disciplines and
for restocking the stables with fresh nutri- professions. Two important new qualita-
ents and feed, a task made especially chal- tive journals Qualitative Inquiry and Field
lenging because of the unprecedented Methodsbegan publication, as did spe-
blossoming of qualitative inquiry in recent cialized qualitative journals m a number of
years. professions (e.gv health, nursing, social

m. xxi
xxii 13. QUALITATIVE RESEARCH AND EVALUATION METHODS

work, organizational development) and proaches are needed and credible, that
some devoted to specific approaches (e.g., mixed methods can be especially valuable,
Grounded Theory Reviezu). The Handbook of and that the challenge is to appropriately
Qualitative Research was published (1994), as match methods to questions rather than ad-
was a revision (2000), and the Handbook of hering to some narrow methodological or-
Methods in Cultural Anthropology (1998) thodoxy. With less need to establish the
made its debut. Sophisticated new software value of qualitative inquiry by debating
programs have been developed to support those of quantitative and experimental per-
qualitative analysis. Internet listservs have suasion, qualitative inquirers have turned
emerged to facilitate dialogue. The Hercu- their attention to eacli other, noticing that
lean challenge has been analyzing this geo- they are engaging in different kinds of quali-
metric growth to determine primary trends, tative inquiry from widely different per-
patterns, and themes. The results of that spectives. Qualitative methodologists and
analysis are reflected throughout this new theorists have thus taken to debating each
edition. other. The upshot of ali the developmental
The first edition of this book (1980), enti- work in qualitative methods is that there is
tled Qualitative Evaluation Methods, focused now as much variation among qualitative
on the variety of ways in which qualitative researchers as there is between qualitatively
methods were being applied in the then and quantitatively oriented scholars and
still-emergent profession of program evalu- evaluators. A primary purpose of this new
ation. (The American Evaluation Associa- edition is to sort out the major perspectives
tion was not established until 1984.) That in that debate, portray the diversity of quali-
edition appeared in the midst of the heated tative approaches now available, and exam-
qualitative-quantitative debate about the ine the influences of this diversity on appli-
relative value of different methods and al- cations, especially but not exclusively in
ternative paradigms. The second edition program evaluation, which has experienced
(1990), entitled Qualitative Evaluation and Re- a parallel flowering of diversity and atten-
search Methods, was influenced by maturing dant controversies about new directions.
of the paradigms debate. It included much
more attention to the ways in which differ-
ent theoretical and philosophical perspec-
tives influenced qualitative inquiry, as well !=l Organization of This Edition
as the greater range of applications in eval-
uation as that profession blossomed. This Chapter 1 provides a range of examples of
latest edition involves yet another change qualitative findings. I begin by presenting a
of title, Qualitative Research and Evaluation number of significant illustrations of the
Methods, reflecting the degree to which de- fruit of qualitative inquiry, in order to give a
velopments in qualitative inquiry during taste of what results from qualitative stud-
the la st decade have been driven by a diver- ies and help those new to such inquiry know
sifying research agenda and scholarly dia- where they are headed and what they are
logue, much of which has found its way into trying to produce. Chapter 2 reviews and
evaluation, to be sure. adds to the primary strategic themes that
The classic qualitative-quantitative de- define qualitative inquiry. Chapter 3 exam-
bate has been largely resolved with recogni- ines different qualitative approaches, in-
tion that a variety of methodological ap- cluding several that have emerged dis-
Preface |J. xxiii

tinctly in the last decade. Chapter 4 presents edition and traveled many qualitative
a wide range of qualitative applications, miles, the list of those to whom I am in-
many of them new, in evaluation, action re- debted is too long and the danger of leaving
search, and organizational, community, and out important influences too great for me to
international development. Chapters 5, 6, include such traditional acknowledge-
and 7 cover design and data gathering, of- ments here. I can only refer the reader to the
fering guidance in purposeful sampling, references and stories in the book as a start-
mixed methods, fieldwork, observational ing point.
approaches, and interviewing, with special I must, however, acknowledge cartoonist
attention directed to the skills and compe- Michael Cochran of Tupper Lake, New York,
tencies needed to gather high-quality data. who drew the many new illustrations in-
Chapter 8 provides direction and processes clude d here to lighten the reader's way
for analyzing qualitative data, always the along this journey. Our collaboration began
most challenging aspect of this work. at a Union Institute research methods semi-
Finally, Chapter 9 de ais with paradigms, nar he took while pursuing his doctorate in
politics, and ways of enhancing the credibil- professional psychology. His evaluation of
ity of qualitative inquiry. This chapter also the seminar include d cartoons. I liked his
presents what I consider to be the five dis- wit and style, so I offered ideas for cartoons
tinct and competing frameworks for under- on qualitative inquiry and he turned them
taking and judging the quality of qualitative into art. Doing the cartoons, he told me, was
studies: traditional scientific research crite- a wonderful distraction from writing his dis-
ria; social construction and constructivist sertation. Fm grateful for his humor and tal-
criteria; artistic and evocative criteria; criti- ent.
cai change criteria; and pragmatic, utility- The editorial and production staff at Sage
oriented evaluation standards and princi- Publications deserve special mention. Only
pies. Along the way I've added, as is my fellow authors who have struggled with edi-
wont, hundreds of new stories and exam- tors of limited vision and understanding can
ples. I've also created over 50 new exhibits fully appreciate what it means to work with
that summarize and illuminate major C. Deborah Laughton, an experienced and
points. knowledgeable editor who not only knows
qualitative methods and evaluation as
deeply as any practitioner of these arts, but
also has significantly shaped those fields by
tI. A c k n o w l e d g m e n t s conceptualizing works that she saw a need
for and then nurtuiing authors, which she
I began this preface by noting the Herculean does better than any editor I've ever known,
task of revision given the enormous growth to assure that those works came to fruition.
and increased diversity of qualitative in- That she is also a writer and designer has
quiry. One task proved more than Hercu- made our working together a genuine col-
lean, and I could not complete it. I began to laboration. Kate Peterson's skilled copyedit-
list the many colleagues and evaluation cli- ing added appreciably to the final product as
ents to whom I am indebted and who de- she made many suggestions for improving
serve acknowledgement for their contribu- clarity and readability, and I came to trust
tions to my understanding and writing over and even rely on her unusual eye for detail
the years. Now that I have reached this third Janelle Lemaster's interior design work con-
xxiv s QUALITATIVE RESEARCH AND EVALUATION METHODS

verted the raw maimscript into the carefully newborn child. (Having been left out of that
crafted book you now hold. Diana Axelsen decision, the newborn child subsequently
pulled it ali together as production editor to made it clear he didn't always agree.) The con-
get the book launched on schedule. I came to tribution of Jeanne to the book exemplifies
count on not only her great management why the personal and professional sometimes
competence but also her good humor. cannot and ought not be separated. Jeanne's
Finally, in acknowledging the superb Sage reflections on her own evaluation fieldwork
production team, I should also reprise the and interviewing experiences helped me clar-
preface to the first edition in which I noted ify and break through some particularly diffi-
that my initial foray into qualitative writing cult sections of the book. Her editorial advice
was due entirely to the persuasive powers was invaluable. Those were her tangible con-
of Sara Miller McCune, co-founder of Sage tributions; the intangibles she contributed are
Publications, who had shepherded the first the things that made the book happen.
edition of Utilization-Focused Evaluation (1978)
into print and, based on the perspective in
that book, urged me during a trip to Minne- Those intangibles and Jeanne's ongoing
sota in 1978 to write a qualitative compan- support have remained the mainstay of my
ion. Her vision and follow-through have writing. Meanwhile, the newborn child re-
made Sage Publications the leading pub- ferred to above, Quinn Campbell, has com-
lisher of both evaluation and qualitative in- pleted a master's degree in engineering, and
quiry books. his younger sister, Charmagne Campbell-
With the reader's indulgence, and by way Patton, is about to complete college. As this
of further providing a historical context for was being completed, their older brother,
this third edition, permit me to include an Braxidon Patton, participated in a two-day
excerpt from that first preface so maxty years workshop I conducted on qualitative meth-
ago: ods in preparation for his first evaluation
fieldwork, a sideline he has tumed to as a
As other authors know, there is no way to re- way of supporting his real passion, writing
ally recognize the contribution of one's family and performing rock music. Thus have the
to a book like this, the writing of which was a years passed, love maturing and children
struggle and matter of endurance for both growing, bringing forth the need to revise
family and author. While Sara Miller McCune the old, celebrate the new, and clean out the
was persuading me that the book should be qualitative Augean stables while restocking
written, Jeanne was persuading me that we them with fresh nutrients. It is those nutri-
could nurture together both a new book and a ents that follow.
pyvRT 1

Conceptual Issues
in Qualitative Inquiry

Psychometricians try to measure it

Experimentalists try to control it.
Interviewers ask questions about it.
Observers watch it.
Participant observers do it.
Statisticians count it.
Evaluators value it.
Qualitative inquirers find meaning in it.

When in doubt, observe and ask questions.

When certain, observe at length and ask many more questions.

Gigo's law of deduction: Garbage in,, garbage out.

Halcolm/s law of induction: No new experience, no new insight.

Qualitative inquiry cultivates the most useful of ali human capacities:

The capacity to learn.

m. 1
2 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Innovators are told: "Think outside the box."

Qualitative scholars tell their students: "Study the box. Observe it. Inside. Out-
side. From inside to outside, and outside to inside. Where is it? How did it get
there? What's around it? Who says it's a 'box'? What do they mean? Why does it
matter? Or does it? What is not 'box'? Ask the box questions. Question others
about the box. What's the perspective from inside? From outside? Study diagrams
of the box. Find documents related to the box. What does thinking have to do with
the box anyway? Understand this box. Study another box. And another. Under-
stand box. Understand. Then you can think inside and outside the box. Perhaps.
For awhile. Until it changes. Until you change. Until outside becomes inside
again. Then start over. Study the box."

There is no burden of proof. There is only the world to experience and understand.
Shed the burden of proof to lighten the load for the journey of experience.
From Halcolm's Laws of Inquiry
The Nature of Qualitative Inquiry

TThe F ^ u i t o f Q u a l i t a t i v e j M e ^ o d s

There once lived a man in a country with no fruit trees. A scholar, he spent a
great deal of time reading. He often carne across references to fruit. The descrip-
tions enticed him to undertake a journey to experience fruit for himself.
He went to the marketplace and inquired where he could find the land of
fruit. After much searching he located someone who knew the way. After a long
and arduous journey he came to the end of the directions and found himself at
the entrance to a large apple orchard. It was springtime and the apple trees were
in blossom.
The scholar entered the orchard and, expectantly, pulled off a blossom and
put it in his mouth. He liked neither the texture of the flower nor its taste. He
went quickly to another tree and sampled another blossom, and then another,
and another. Each blossom, though quite beautiful, was distasteful to him. He
left the orchard and returned to his home country, reporting to his fellow villag-
ers that fruit was a much overrated food.
Being unable to recognize the difference between the springblossom and the
summer fruit, the scholar never realized that he had not experienced what he
was looking for.

From Halcolm's Inquiry Parables

S 3
4 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Three Kinds of Qualitative Data

Interviews
Open-ended questions and probes yield in-depth responses about people's experiences, per-
ceptions, opinions, feelings, and knowledge. Data consist of verbatim quotations with sufficient
context to be interpretable.

Observatons
Fieldwork descriptions of activities, behaviors, actions, conversations, interpersonai interac-
tions, organizational or community processes, or any other aspect of observable human experi-
ence. Data consist of field notes: rich, detailed descriptions, including the context within which the
observatons were made.

Documents
Written materiais and other documents from organizational, clinicai, or programs records;
memoranda and correspondence; offcial publications and reports; personal diaries, letters, artis-
tic works, photographs, and memorabilia; and written responses to open-ended surveys. Data con-
sist of excerpts from documents captured in a way that records and preserves context.

Recognizing Qualitative Data processes that are part of observable hu-

man experience. Document analysis includes
This book discusses how to collect, ana- studying excerpts, quotations, or entire pas-
lyze, and use qualitative data. To begin, let's sages from organizational, clinicai, or pro-
examine the fruit of qualitative methods. It is gram records; memoranda and correspon-
important to know what qualitative data dence; official publications and reports;
and findings look like so that you will know personal diaries; and open-ended written
what you are seeking. It will also be impor- responses to questionnaires and surveys.
tant to consider criteria for judging the qual- (See Exhibit 1.1.)
ity of qualitative data. Apples come to mar- The data for qualitative analysis typically
ket sorted by type (Red Delicious/ Golden), come from fieldwork. During fieldwork, the
purpose (e.g., cooking or eating), and qual- researcher spends time in the setting under
ity. Likewise, qualitative studies vary by studya program, an organization, a com-
type, purpose, and quality. munity, or wherever situations of impor-
Qualitative findings grow out of three tance to a study can be observed, people in-
kinds of data collection: (1) in-depth, open- terviewed, and documents analyzed. The
ended interviews; (2) direct observation; researcher makes firsthand observa tions of
and (3) written documents. Interviews yield activities and interactions, sometimes en-
direct quotations from people about their ex- gaging personally in those activities as a par-
periences, opinions, feelings, and knowl- ticipant observer. For example, an evaluator
edge. The data from observatons consist of might participa te in ali or part of the pro-
detailed descriptions of people's activities, gram under study, participating as a regular
behaviors, actions, and the full range of in- program member, client, or student. The
terpersonai interactions and organizational qualitative researcher talks with people
The Nature of Qualitative Inquiri/ [J. 5

about their experiences and perceptions. The quality of qualitative data depends
More formal individual or group interviews to a great extent on the methodological
may be conducted. Relevant records and skill, sensitivity, and integrity of the re-
documents are examined. Extensive field searcher. Systematic and rigorous obser-
notes are collected through these observa- vation involves far more than just being
tions, interviews, and document reviews. present and looking around. Skillful inter-
The voluminous raw data in these field viewing involves much more than just
notes are organized into readable narrative asking questions. Content analysis re-
descriptions with major themes, categories, quires considerably more than just read-
and illustrative case examples extracted ing to see what's there. Generating useful
through content analysis. The themes, pat- and credible qualitative findings through
terns, understandings, and insights that observation, interviewing, and content
emerge from fieldwork and subsequent analysis requires discipline, knowledge,
analysis are the fruit of qualitative inquiry. training, practice, creativity, and hard
Qualitative findings may be presented work.
alone or in combination with quantitative This chapter provides an overview of
data. Research and evaluation studies em- qualitative inquiry. La ter chapters exam-
ploying multiple methods, including combi- ine how to choose among the many op-
nations of qualitative and quantitative data, tions available within the broad range of
are common. At the simplest levei, a ques- qualitative methods, theoretical perspec-
tionnaire or interview that asks both fixed- tives, and applications; how to design a
choice (closed) questions and open-ended qualitative study; how to use observa-
questions is an example of how quantitative tional methods and conduct in-depth,
measurement and qualitative inquiry are of- open-ended interviews; and how to ana-
ten combined. lyze qualitative data to generate findings.

& Qualitative Findings: T h e m e s , Patterns,

Concepts, Insights, U n d e r s t a n d i n g s

ewton and the apple. Freud and anxiety. Jung and dreams. Piaget
and his children. Darwin and Galapagos tortoises. Marx and
England's factories. Whyte and street corners. What are you obsessed with?

Halcolm

Mary Field Belenky and her colleagues gether, informed partly by previous re-
set out to study women's ways of knowing. search but ultimately basing the analysis
They conducted extensive interviews with on their own collective sense of what cate-
135 women from diverse backgrounds prob- gories best captured what they found in
ing how they thought about knowledge, au- the narrative data. They argued with each
thority, truth, themselves, life changes, and other about which responses belonged in
life in general. They worked as a team to which categories. They created and aban-
group similar responses and stories to- doned categories. They looked for com-
6 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Discovery of an early qualitative evaluation report

monalities and differences. They worked irtforming variations in ways of knowing.

hard to honor the diverse points of view they After painstaking analysis, they ended up
found while also seeking patterns across sto- with the five categories of knowing summa-
ries, experiences, and perspectives. One rized in Exhibit 1.2, a framewbrk that be-
theme emerged as particularly powerful: caine very influential in women's studies
"Again and again women spoke of 'gaining and represents one kind of fruit from quali-
voice' " (Belenky et al. 1986:16). Voice versus tative inquiry.
silence emerged as a central metaphor for
The Nature of Qualitative Inquiri/ [J. 7

WomerTs Ways of Knowing:

An Example of Qualitative Findings

Silence: A position in which women experience themselves as mindless and voiceless and
subject to the whims of externai authority.

Received knowledge: Women conceive of themselves as capable of receiving, even reproducing

knowledge from externai authorities, but not capable of creating knowledge on their own.

Subjective knowledge: A perspective from which truth and knowledge are conceived as
persona!, private, and subjectively known or intuited.

Procedural knowledge: Women are invested in learning and apply objective procedures for
obtaining and communicatng knowledge.

Constructedknowledge: Women view ali knowledge as contextual, experience themselves as

creators of knowledge, and value both subjective and objective strategies for knowing.

SOURCE: Belenky et al. (1986:15).

One of the best-known and most influen- Stephen Covey (1990) used this same
tial books in organizational development sampling approach in doing case studies of
and management is In Search of Excellence: "highly effective people." He identified
Lessons From America's Best-Run Companies. seven habits these people practice: (1) being
Peters and Waterman (1982) based the book proactive; (2) beginning with the end in
on case studies of 62 highly regarded com- mind; (3) putting first things first; (4) think-
panies. They visited companies, conducted ing win/ win; (5) seeking first to understand,
extensive interviews, and studied corporate then seeking to be understood; (6) syner-
documents. From that massive amount of gizing, or engaging in creative cooperation;
data they extracted eight attributes of excel- and (7) self-renewal.
lence: (1) abias for action; (2) close to the cus- Both of these best-selling books, In Search
tomer; (3) autonomy and entrepreneurship; of Excellence and The 7 Habits of Highly Effec-
(4) productivity through people; (5) hands- tive People, distill a small number of impor-
on, value-driven; (6) stick to the knitting; (7) tant lessons from a huge amount of data
simple form, lean staff; and (8) simultaneous based on outstanding exemplars. It is com-
loose-tight properties. Their book devotes a mon in qualitative analysis for mounds of
chapter to each theme with case examples field notes and months of work to reduce to a
and implications. Their research helped small number of core themes. The quality of
launch the quality movement that has now the insights generated is what matters, not
moved from the business world to not-for- the number of such insights. For example, in
profit organizations and government. This an evaluation of 34 programs aimed at peo-
study also illustrates a common quaHtative ple in poverty, we found a core theme that
sampling strategy: studying a relatively separated more effective from less effective
small number of special cases that are suc- programs: How people are treated affects
cessful at something and therefore a good how they treat others. If staff members are
source of lessons learned. treated autocratically and insensitively by
8 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

management, with suspicion and disre- tinguish, and elaborate three primary pro-
spect, staff will treat clients the same way. cesses that contribute to the development of
Contrariwise, responsiveness reinforces re- a relationship: "Being-In," "Being-For," and
sponsiveness, and empowerment breeds "Being-With."
empowerment. These insights became the Being-In involves immersingoneself in
centerpiece of subsequent cross-project, col- another's world: listening deeply and atten-
laborative organizational and staff develop- tively so as to enter into the other person's
ment processes. experience and perception. "I do not select,
A different kind of qualitative finding is interpret, advise, or direct. . . . Being-In the
illustrated by Angela Browne's book When world of the other is a way of going wide
Battered Women Kill (1987). Browne con- open, entering in as if for the first time, hear-
ducted in-depth interviews with 42 women ing just what is, leaving out my own
from 15 states who were charged with a thoughts, feelings, theories, biases I enter
crime in the death or serious injury of their with the intention of understanding and ac-
mates. She was often the first to hear these cepting perceptions and not presenting my
women's stories. She used one couple's his- own view or reactions 1 only want to en-
tory and vignettes from nine others, repre- courage and support the other person's ex-
sentative of the entire sample, to illuminate pression, what and how it is, how it came to
the progression of an abusive relationship be, and where it is going." (Moustakas 1995:
from romantic courtship to the onset of 82-83)
abuse through its escalation until it was on- Being-For involves taking a stand in
going and eventually provoked a homicide. support of the other person, being there for
Her work helped lead to legal recognition the other. "I am listening. I am also offering a
of battered women's syndrome as a legiti- position, and that position has an element of
mate defense, especially in offering insight my being on that person's side, against ali
into the common outsider's question: Why others who would minimize, deprecate, or
doesn't the woman just leave? An insider's deny this person's right to be and to grow....
perspective on the debilitating, destructive, I become an advocate of the person with ref-
and all-encompassing brutality of battering erence to his or her frustrations and prob-
reveals that question for what it is: the facile lems in dealing with others." (Moustakas
judgment of one who hasn't been there. The 1995:83)
effectiveness of Browne's careful, detailed, Being-With involves being present as
and straightforward descriptions and quo- one's own person in relation to another per-
tations lies in their capacity to take us inside son, bringing one's own knowledge and ex-
the abusive relationship. Offering that in- perience into the relationship. "This may in-
side perspective powers qualitative report- volve disagreeing with the other's ways of
ing. interpreting or judging or presenting some
Clark Moustakas (1995), a humanistic spect of the world. Being-With means lis-
psychologist and phenomenologist, also tening and hearing the other's feelings,
gives us an insider's perspective: his thoughts, objectives, but it also means offer-
own. An astute and dedicated observer of ing my own perceptions and views. There is,
relationships, especially therapeutic rela- in Being-With, a sense of joint enterprise
tionships, he drew deeply on his own expe- two people fully involved, struggling, ex-
riences and clinicai cases to identify, dis- ploring, sharing." (Moustakas 1995:84)
The Nature of Qualitative Inquiri/ [J. 9

IjfPlIH''"1' Coming-of-Age Paradigms

Dimensions
of Comporiso n Tribal Initiotion Modern Corning ofAg

View of life passages One-time transition from child Multiple passages over a lifetime
to aduit journey
Territory Tribal territory Earth: Global community
Ancestry Creation myth Evolutionary story of humankind
Identity Becoming a man or woman Becoming a complete person
Approach Standardized Individualized
Outcome Tribe-based identity Person a lity identity: Sense of self
Message You are first and foremost a You are first and foremost a person
member of the tribe in your own right

SOURCE: Patton (I999a:333, 335).

Qualitative findings of ten have this sim- sis that leads to a new synthesis. In philoso-
ple yet elegant and insightful character. This phy such contrasts derive from the rumina-
straightforward yet nuanced framework tions of philosophers; in qualitative research
represents a creative synthesis of years of such thematic contrasts emanate from and
participant observation and personal in- are grounded in fieldwork.
quiry. Through cases, dialogues, quotations, This quick sampling of the fruit of quali-
cases, and introspective reflections, Mous- tative inquiry is meant, like a wine tasting, to
takas illuminates the process of moving demonstrate choices toward developing a
froin Being-In to Being-For and ultimately more sophisticated palate, or like appetiz-
Being-With. His work exemplifies the con- ers, as an opening to the fuller feast yet to
tribution of phenomenological inquiry to come. The next section discusses some of the
humanistic psychology. different research and evaluation purposes
Still a different format for capturing and that affect what kind of fruit results from
reporting qualitative findings is illustrated qualitative inquiry and how the quality of
by my own inquiry into alternative coming- that fruit is judged.
of-age approaches. I used the device of con-
structing ideal-typical alternative para-
digms to compare and contrast what I Different Purposes of and
learned (Patton 1997a). Exhibit 1.3 provides Audiences for Qualitative Studies:
a sampling of contrasts between traditional Research, Evaluation, Dissertations,
tribe-centered initiations and modern youth- and Personal Inquiry
centered coming-of-age celebrations. These
kinds of polar contrasts can sometimes set As the title of this book indicates, qualita-
up a Hegelian dialectic of thesis and antithe- tive methods are used in both research and
10 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

evaluation. But because the purposes of re- Qualitative methods are often used in
search and evaluation are different, the crite- evaluations because they tell the program's
ria for judging qualitative studies can vary story by capturing and communicating the
depending on purpose. This point is impor- participants' stories. Evaluation case studies
tant. It means one can't judge the appropri- have ali the elements of a good story. They
ateness of the methods in any study or the tell what happened, when, to whom, and
quality of the resulting findings without with what consequences. Many examples in
knowing the study's purpose, agreed-on this book are drawn from program evalua-
uses, and intended audiences. Evaluation tion, policy analysis, and organizational de-
and research typically have different pur- velopment. The purpose of such studies is to
poses, expected uses, and intended users. gather information and generate findings
Dissertations add yet another layer of com- that are useful. Understanding the pro-
plexity to this mix. Let's begin with evalua- gram^ and participants' stories is useful to
tion. the extent that they illuminate the processes
Program evaluation is the systematic col- and outcomes of the program for those who
lection of information about the activities, must make decisions about the program. In
characteristics, and outcomes of programs Utization-Focused Evaluation (Patton 1997a),
to make judgments about the program, im- I presented a comprehensive approach to
prove program effectiveness, and/or inform doing evaluations that are useful, practical,
decisions about future prograrmning. Pol- ethical, and accurate. The primary criterion
icies, organizations, and personnel can also for judging such evaluations is the extent to
be evaluated. Evaluative research, quite which intended users actually use the find-
broadly, can include any effort to judge or ings for decision making and program im-
enhance human effectiveness through sys- provement. The methodological implication
tematic data-based inquiry. Human beings of this criterion is that the intended users
are engaged in ali kinds of efforts to make must value the findings and find them credi-
the world a better place. These efforts in- ble. They must be interested in the stories,
clude assessing needs, formulating policies, experiences, and perceptions of program
passing laws, delivering programs, manag- participants beyond simply knowing how
ing people and resources, providing ther- many came into the program, how many
apy, developing communities, changing or- completed it, and how many did what after-
ganizational culture, educating students, ward. Qualitative findings in evaluation il-
intervening in conflicts, and solving prob- luminate the people behind the numbers
lema. In these and other efforts to make the and put faces on the statistics, not to make
world a better place, the question of whether hearts bleed, though that may occur, but to
the people involved are accomplishing what deepen understanding.
they want to accomplish arises. When one Research, especially fundamental or basic
examines and judges accomplishments and research, differs from evaluation in that its
effectiveness, one is engaged in evaluation. primary purpose is to generate or test theory
When this examination of effectiveness is and contribute to knowledge for the sake of
conducted systematically and empirically knowledge. Such knowledge, and the theo-
through careful data collection and thought- ries that undergird knowledge, may subse-
ful analysis, one is engaged in evaluation re- quently inform action and evaluation, but
search. action is not the primary purpose of funda-
The Nature of Qualitative Inquiri/ [J. 11

UAA.'-' . ; . j : .:, .,.,V

tfLftt k>$X"- !;=: !;:
^rrtsThil. 8 st! .'.:, .-iV"i L-I !.'i.'! u !:'* . , ' , ! ' ':,:; . 1 o r -vj ? 1 < " ' rr> m ; i

flWJ-fOT-n Ti Vjji:" iVy.Ti ji ': ! 'v!:.' i w! i - u i t :. ' ":!:> V : ; '! , i 1 : " f 11-3-=- [ !<' 1
VI' I. i >
i.i-.iiv.li,::'
;,!=:; j:-n j;! jT!t:! !;= i f t j ? ^ f f c s - ri h ! >n !' i ! ! , ii r 1V1 m \ \ 'i; 'V',iVi!:oi:r."!-. mvi-'' j
.'-=.'= j:" !.!:::-."-.;!!" i-,v:V: , i ijYi v; ,i.v. jjVi!.v i! nJ i iW!.',.;"';,:::;::,:,! yj.y .
V M-V
-.fl Ii-I j:;; iYi;Mi!::!;h!ni:ir!tfktiyr:siri" i n - ii r :.=: !.Vi:. U.i ri t: IV!; ri ,1 ,! ,!.;: j -v. ,, ,-i....,,-j- vi ,,,,.,,,
h:i -inl^r-^r-jii.H^^rh^n^^rr ^i-i-i: !-!." ;. v l Y : i-vkz! q.!.a IY-.h.;
3
11 oIA'', i!* 1 ".: !: -'! ::', m iyi i i = : n 11
V':i!;::: ^ . n a /-.'n, . ..VMV s:-! r.ii.' ::;i.::M;.' r:'.! "-: .: .r- r
1

'.YPOV :;":' rivrjiM.- umIY! t- ms.. : cti11.: 1 :.i:V!!! i:,11 : vr!T! : \ i 'o ! i v, i =i i vi i
i!l'i -!:!' -=:"':'!= \>!l =!'=:.!:liY/rvi|i ! i = " i ! = i ' wrt
:
.-'IA-;!.;!.n.:iiV!..; 'j.'i- I:I.-i "
!: ' ; : i".;-!,!":: : ! v; - S t i " =-:;!-t: n m nVi-iv.ri.i.1 v- i!:>i ! i li!'.!!!!!!!.' ! P ! . ! ! i ' " t ' ! : * 1 " > ' ' i .

X-'.;>::;. :iT= -v;: !:= ! 6 * 7 ' ' r v t V " , i V r - i ,i ; i i :. !:;.'! -. ^ o > !'' :;jxia > .:;>.
i=r.':-i:'!' n r r o '; M t t f i t f . ; d :ni I * 1 !" i i'!l

mental research. Qualitative inquiry is espe- ing the quality of the methodological proce-
cially powerful as a source of grounded the- dures followed and the analysis done. Qual-
ory, theory that is inductively generated itative dissertations, once quite rare, have
from fieldwork, that is, theory that emerges become increasingly common as the criteria
from the researcher's observations and infor judging qualitative contributions to
terviews out in the real world rather than in knowledge have become better understood
the laboratory or the academy. The primary and accepted. But those criteria are not abso-
audiences for research are other researchers lute or universally agreed on. As we shall
and scholars, as well as policymakers and see, there are many varieties of qualitative
others interested in understanding some inquiry and multiple criteria for judging
phenomenon or problem of interest. The re- quality, many of which remain disputed.
search training, methodological prefer- While the precedmg discussion of evalua-
ences, and scientific values of those who use tion, research, and dissertations has empha-
research will affect how valuable and credi- sized taking into account externai audiences
be they find the empirical and theoretical and consumers of qualitative studies, it is
fruit of qualitative studies. also important to acknowledge that you may
Dissertations and graduate theses offer be the primary intended audience for your
special insight into the importance of atten- work. You may study something because
tion to audience. Savvy graduate students you want to understand it. As my children
learn that to complete a degree program, the grew to adulthood, I found myself asking
studenfs committee must approve the questions about coming of age m modern
work. The particular understandings, val- society so I undertook a personal inquiry that
ues, preferences, and biases of committee became a book (Patton 1997a), but I didn't
members come into play in that approval start out to write a book. I started out trying
process. The committee will, in essence, to understand my own experience and the
evaluate the studenfs contribution, includ- experiences of my children. That is a form of
12 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

qualitative inquiry. While doing interviews Making Methods Decisions

with recipients of MacArthur Foundation
Fellowships (popularly called "Genius The implication of thinking about pur-
Awards"),1 was told by a social scientist that pose and audience in designing studies is
her fieldwork was driven by her own search that methods, no less than knowledge, are
for understanding and that she disciplined dependent on context. No rigid rules can
herself to not even think about publication prescribe what data to gather to investigate a
while engaged in interviewing and observ- particular interest or problem. There is no
ing because she didn't want to have her in- recipe or formula in making methods deci-
quiry affected by attention to externai audi- sions. Widely respected psychometrician
ences. She wanted to know because she wanted Lee J. Cronbach has observed that design-
to know, and she had made a series of career ing a study is as much art as science. It is
and professional decisions that allowed her "an exercise of the dramatic imagination"
to focus on her personal inquiry without be- (Cronbach 1982:239). In research as in art,
ing driven by the traditional academic ad- there can be no single, ideal standard.
monition to "publish or perish." She didn't Beauty no less than "truth" is in the eye of
want to subject herself to or have her work the beholder, and the beholders of research
influenced by externai criteria and judg- and evaluation can include a plethora of
ment. stakeholders: scholars, policymakers, fund-
In summary, ali inquiry designs are af- ers, program managers, staff, program par-
fected by intended purpose and targeted au- ticipants, journalists, critics, and the general
dience, but purpose and audience deserve public. Any given design inevitably reflects
special emphasis in the case of qualitative some imperfect interplay of resources, capa-
studies, where the criteria for judging qual- bilities, purposes, possibilities, creativity, and
ity may be poorly understood or in dispute, personal judgments by the people involved.
even among qualitative methodologists. Research, like diplomacy, is the art of the
This book cannot resolve these debates, but possible. Exhibit 1.4 provides a set of ques-
it will illuminate the methodological options to consider in the design process, re-
tions and their implications. (Chapter 9 dis- gardless of type of inquiry. With that back-
cusses alternative criteria for judging the ground, we can tum to consideration of the
quality of qualitative studies.) relative strengths and weaknesses of quali-
tative and quantitative methods.

13. M e t h o d s Choices: Contrasting

Qualitative and Quantitative E m p h a s e s

ot everything that can be counted counts, and not everything that

counts can be counted.

Albert Einstein
The Nature of Qualitative Inquiri/ [J. 13

Some Guiding Questions and Options for

EXHIBIT 1.4 Methods Decisions

1. What are the purposes of the inquiry?

Research: Contribution to knowledge
Evaluation: Program improvement and decision making
Dissertation: Demonstrate doctoral-levei scholarship
Personal inquiry: Find out for oneself

2. Who are the primary audiences for the findings?

Scholars, researchers, academicians
Program funders, administrators, staff, participants
Doctoral committee
Oneself, friends, family, lovers

3. What questions wll guide the inquiry?

Theory-derived, theory-testing, and/or theory-oriented questions
Practical, applied, action-oriented questions and issues
Academic degree or discipine/specialization priorities
Matters of personal interest and concern, even passion

4. What data will answer or illuminate the inquiry questions?

Qualitative: Interviews, field observatons, documents
Quantitative: Surveys, tests, experments, secondary data
Mixed methods: What kind of mix? Which methods are primary?

5. What resources are avaiiable to support the inquiry?

Financial resources
Time
People resources
Access, connections

6. What criteria will be used to judge the quality of the findings?

Traditional research cr/ter/o; Rigor, validty, relabilty, generalizabiity
Evaluatian standards: Utility, feasiblty, propriety, accuracy
Nontraditional criteria: Trustworthiness, diversity of perspectives, clarity of voce, credibility
of the inquirer to primary users of the findings

Thinking about design alternatives and know if they're obese, measure body fat in
methods choices leads directly to consider- relation to height and weight and compare
ation of the relative strengths and weak- the results to population norms. If you want
nesses of qualitative and quantitative data. to know what their weight means to them,
The approach here is pragmatic. Some ques- how it affects them, how they think about it,
tions lend themselves to numerical answers; and what they do about it, you need to ask
some don't. If you want to know how much them questions, find out about their experi-
people weigh, use a scale. If you want to ences, and hear their stories. A comprehen-
14 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

sive and multifaceted understanding of a distraction. Guba and Lincoln (1981) have
weight in people's lives requires both their commented on this aspect of qualitative
numbers and their stories. Doctors who look research:
only at test results and don't also listen to
their patients are making judgments with Fatigue, shifts in knowledge, and cooptation,
inadequate knowledge, and vice versa. as well as variations resulting from differences
Qualitative methods facilitate study of is- in training, skill, and experience among differ-
sues in depth and detail. Approaching field- ent "instruments," easily occur. But this loss in
work without being constrained by prede- rigor is more than offset by the flexibility, in-
termined categories of analysis contributes sight, and ability to build on tacit knowledge
to the depth, openness, and detail of qualita- that is the peculiar province of the human in-
tive inquiry. Quantitative methods, on the strument. (p. 113)
other hand, require the use of standardized
measures so that the varying perspectives Because qualitative and quantitative
and experiences of people can be fit into a methods involve differing strengths and
limited number of predetermined response weaknesses, they constitute alternative, but
categories to which numbers are assigned. not mutually exclusive, strategies for re-
The advantage of a quantitative approach search. Both qualitative and quantitative
is that it's possible to measure the reactions data can be collected in the same study. To
of a great many people to a limited set of further illustrate these contrasting ap-
questions, thus facilitating comparison and proaches and provide concrete examples of
statistical aggregation of the data. This gives the fruit of qualitative inquiry, the rest of this
a broad, generalizable set of findings pre- chapter presents select excerpts from actual
sented succinctly and parsimoniously. By studies.
contrast, qualitative methods typically pro-
duce a wealth of detailed information about
a much smaller number of people and cases. Comparing Two Kinds
This increases the depth of understanding of of Data: An Example
the cases and situations studied but reduces
generalizability. The Technology for Literacy Center was a
Validity in quantitative research depends computer-based adult literacy program in
on careful instrument construction to ensure Saint Paul, Minnesota. It operated out of a
that the instrument measures what it is storefront facility in a lower-socioeconomic
supposed to measure. The instrument must area of the city. In 1988, after three years of
then be administered in an appropriate, pilot operation, a major funding decision
standardized manner according to pre- had to be made about whether to continue
scribed procedures. The focus is on the mea- the program. Anticipating the funding deci-
suring instrumentthe test items, survey sion, a year earlier local foundations and the
questions, or other measurement tools. In public schools had supported a summative
qualitative inquiry, the researcher is the in- evaluation to determine the overall outcomes
strument. The credibility of qualitative and cost-effectiveness of the center. The
methods, therefore, hinges to a great extent evaluation design included both quantita-
on the skill, competence, and rigor of the tive and qualitative data.
person doing fieldworkas well as things The quantitative testing data showed
going on in a person's life that might prove great variation. The statistics on average
The Nature of Qualitative Inquiri/ [J. 15

achievement gains masked great differences death from hepatitis. During the week she
among participants. The report concluded seldom gets more than three hours of sleep
that although testing showed substantial each night. At the time of the case study, she
achievement test gains for the treatment had spent 15 months in the program and
group versus the control group, the more progressed from not reading at ali (second-
important finding concerned the highly in- grade levei) to being a regular library user
dividualized nature of student progress. The (and testing a grade levei higher than where
report concluded, "The data on variation in she began). She developed an interest in
achievement and instructional hours lead to Black history and reported being particu-
a very dramatic, important and significant larly pleased at being able to read the Bible
finding: there is no average student at TLC" on her own. She described what it was like
(Patton and Stockdill 1987:33). not being able to read:
This finding highlights the kind of pro-
gram or treatment situation where qualita- Where do you go for a job? You can't make out
tive data are particularly helpful and appro- an application. You go to a doctor and you
priate. The Technology for Literacy Center can't fill out the forms, and it's very embar-
has a highly individualized program in rassing. You have to depend on other people
which learners proceed at their own pace to do things like this for you. Sometimes you
based on specific needs and interest. The don't even want to ask your own kids because
students come in at very different leveis, it's just like you're depending too much on
with a range of goals, participate in widely people, and sometimes they do it willingly,
varying ways, and make very different and sometimes you have to beg people to
gains. Average gain scores and average help
hours of instruction provide a parsimonious Ali the progress has made me feel lots
overview of aggregate progress, but such better about myself because I can do some of
statistical data do little to help funders un- the things I've been wanting to do and I could-
derstand what the individual variation n't do. It's made me feel more independent to
means. To get at the meaning of the program do things myself instead of depending on
for individual participants, the evaluation other people to do them for me.
included case studies and qualitative data
from interviews. A second contrasting case tells the story
of Sara Johnson, a 42-year-old Caucasian
woman who dropped out of school in the
INDIVIDUAL CASE EXAMPLES lOth grade. She is a clerical office manager.
She tested at 12th-grade levei on entry to the
One case is the story of Barbara Jenkins, a program. After 56 hours of study over 17
65-y ear-old Black grandmother who came to days, she received her general equivalency
Minnesota after a childhood in the deep diploma (GED), making her a high school
South. She works as a custodian and house graduate. She immediately entered college.
cleaner and is proud of never having been on She said that the decision to return for her
welfare. She is the primary breadwinner for GED was
a home with five children spanning three
generations, including her oldest daugh- an affirmation, as not having a diploma had
ter's teenage children for whom she has really hurt me for a long time It was always
cared since her daughter's unexpected scary wondering if somebody actually found
16 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

out that I was not a graduate that they would To get the perspective of students, I con-
fire me or they wouldn't accept me because I ducted group interviews. "Groups are not
hadn't graduated. The hardest thing for me to just a convenient way to accumulate the in-
do was tell my employer. He s very much into dividual knowledge of their members. They
education and our company is education- give rise synergistically to insights and solu-
oriented. So the hardest thing I ever had to do tions that would not come about without
was tell him I was a high school dropout. I them" (Brown, Collins, and Duguid 1989:
needed to tell because I needed time to go and 40). In group interviews I asked students to
take the test. He was just so understanding. describe the program's outcomes in per-
I couldn't believe it. t was just wonderful. sonal terms. I asked, "What difference has
I thought he was going to be disappointed in what you are learning made in your lives?"
me, and he thought it was wonderful that I Here are some responses.
was going back. He came to graduation.
I love the newspaper now, and actually read it.
Yeah, I love to pick up the newspaper now. I
These short excerpts from two contrast-
used to ha te it. Now I love the newspaper.
ing cases illustrate the value of detailed, de-
scriptive data in deepening our understand- I can follow sewing directions. I make a gro-
ing of individual variation. Knowing that cery list now, so I'm a better shopper. I don't
each woman progressed about one grade forget things.
levei on a standardized reading test is only a
small part of a larger, much more complex Yeah, you don't know how einbarrassing it is
picture. Yet, with over 500 people in the pro- to go shopping and not be able to read the
gram, it would be overwhelming for wife's grocery list. It's helped me out so much
funders and decision makers to attempt to in the grocery store.
make sense of 500 detailed case studies
Helps me with my medicine. Now I can read
(about 5,000 double-spaced pages). Statisti-
the bottles and the directions! I was afraid to
cal data provide a succinct and parsimoni-
give the kids medicine before because I wasn't
ous summary of major patterns, while select
sure.
case studies provide depth, detail, and indi-
vidual meaning. I don't get lost anymore. I can find my way
around. I can make out directions, read the
map. I work construction and we change loca-
OPEN-ENDED INTERVIEWS
tions a lot. Now I can find my way around. I
Another instructive contrast is to com- don't get lost anymore!
pare closed-ended questionnaire results
with responses to open-ended group inter- Just getting a driver's license will be wonder-
views. Questionnaire responses to quantita- ful. I'm 50. If I don't get the GED, but if I can
tive, standardized items indicated that 77% get a license...! I can drive well, but I'm scared
of the adult literacy students were "very to death of the written test. Just getting a
happy" with the Technology for Literacy driver's license . . . , a driver's license.
Center program; 74% reported learning "a
Now I read outdoor magazines. I used to just
great deal." These and similar results re-
read the titles of booksnow I read the books!
vealed a general pattern of satisfaction and
progress. But what did the program mean to I was always afraid to read at school and at
students in their own words? church. I'm not afraid to read the Bible now at
The Nature of Qualitative Inquiri/ [J. 17

Bible class. It's really important to me to be achievement tests administered in both fali
able to read the Bible. and spring, criterion-referenced tests devel-
oped by teachers, performance objectives,
I can fill out applications now. You have to teacher peer ratings, student ratings of
know how to fill out an application in this teachers, parent ratings of teachers, princi-
world. I can look in the Yellow Pages. It used to pal ratings of teachers, and teacher self-
be so embarrassing not to be able to fill out ap- ratings.
plications, not to be able to find things in the
The Kalamazoo accountability system be-
Yellow Pages. I feel so much better now. At
gan to attract national attention. For exam-
least my application is filled out right, even if I
ple, the American School Board Journal re-
don't get the job, at least my application is
ported in April 1974 that "Kalamazoo
filled out right.
schools probably will have one of the most
comprehensive computerized systems of
I'm learning just enough to keep ahead of my personnel evaluation and accountability yet
kids. My family is my motivation. Me and my devised" (p. 40). In the first of a three-part se-
family. Once you can read to your kids, it ries on Kalamazoo, the American School
makes ali the difference m the world. It helps Board Journal asserted: "Take it from Kala-
you to want to read and to read more. When I mazoo: a comprehensive, performance-based
can read myself, I can help them read so they system of evaluation and accountability can
can have a better life. The kids love it when I work" ("Kalamazoo Schools" 1974:32).
read to them.
Not everyone agreed with that positive
assessment, however. The Kalamazoo Edu-
These group interview excerpts provide cation Association charged that teachers
some qualitative rnsights into the individ- were being demoralized by the accountabil-
ual, personal experiences of adults learning ity system. Some school officials, on the
to read. The questionnaire results (77% satis- other hand, argued that teachers did not
fied) provided data on statistically general- want to be accountable. In the spring of 1976,
izable patterns, but the standardized ques- the Kalamazoo Education Association, with
tions only tap the surface of what it means assistance from the Michigan Education As-
for the program to have had "great per- sociation and the National Education Asso-
ceived impact." The much smaller sample of ciation, sponsored a survey of teachers to
open-ended interviews adds depth, detail, find out the teachers' perspective on the ac-
and meaning at a very personal levei of ex- countability program (Perrone and Patton
perience. Another example will show that 1976).
qualitative data can yield not only deeper
The education association officials were
understanding but also political action as
interested primarily in a questionnaire con-
the depth of participants' feelings is re-
sisting of standardized items. One part of
vealed.
the closed-ended questionnaire provided
teachers with a set of statements with which
The Power of Qualitative Data they could agree or disagree. The question-
naire results showed that teachers felt the ac-
In the early 1970s, the school system of countability system was largely ineffective
Kalamazoo, Michigan, implemented a new and inadequate. For example, 90% of the
accountability system. It was a complex sys- teachers disagreed with the school adminis-
tem that included using standardized tration's published statement "The Kala-
18 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

mazoo accountability system is designed to 2. Finally, we'd like you to use this space to
personalize and individualize education"; add any additional comments you'd
88% reported that the system does not assist like to make about any part of the
teachers to become more effective; 90% re- Kalamazoo accountability system.
sponded that the accountability system has
not improved educational planning in A total of 373 teachers (70% of those who
Kalamazoo; and 93% believed, "Account- responded to the questionnaire) took the
ability as practiced in Kalamazoo creates an time to respond to one of these open-ended
undesirable atmosphere of anxiety among questions. Ali of the comments nade by
teachers." And 90% asserted, "The account- teachers were typed verbatim and included
ability system is mostly a public relations ef- in the report. These open-ended data filled
fort." Nor did teachers feel that the account- 101 pages. When the school officials and
ability system fairly reflected what they did school board members rejected the ques-
as teachers, since 97% of them agreed, "Ac- tionnaire data, rather than argue with them
countability as practiced in Kalamazoo about the meaningfulness of teacher re-
places too much emphasis on things that can sponses to the standardized items, we asked
be quantified so that it misses the results of them to turn to the pages of open-ended
teaching that are not easily measured." teacher comments and simply read at ran-
It is relatively clear from these statements dom what teachers said. Examples of the
that most teachers who responded to the comments they read, and could read on vir-
questionnaire were negative about the actually any page in the report, are reproduced
countability system. When school officials below in six representative responses from
and school board members reviewed the the middle pages of the report.
questionnaire results, however, many of
Teacher Response No. 284: "I don't feel that
them immediately dismissed those results
fear is necessary in an accountability situation.
by arguing that they had never expected
The person at the head of a school system has
teachers to like the system, teachers didn't
to be human, not a machine. You just don't
really want to be accountable, and the teach-
treat people like they are machines!
ers' unions had told their teachers to re-
"The superintendent used fear in this sys-
spond negatively anyway. In short, many
tem to get what he wanted. That's very hard to
school officials and school board members
explain in a short space. It's something you
dismissed the questionnaire results as bi-
have to live through to appreciate. He lied on
ased, inaccurate, and the results of teacher
many occasions and was very deceitful. Teach-
union leaders telling teachers how to re-
ers need a situation where they feel comfort-
spond in order to discredit the school au-
able. I'm not saying that accountability is not
thorities.
good. I am saying the one we have is lousy. It's
The same questionnaire included two hurting the studentsthe very ones we're
open-ended questions. The first was placed supposed to be working for."
midway through the questionnaire, and the
second came at the end of the questionnaire. Teacher Response No. 257: "This system is
creating an atmosphere of fear and intimida-
1. Please use this space to make any fur- tion. I can only speak for the school I am in, but
ther comments or recommendations people are tense, hostile and losing their hu-
concerning any component of the ac- manity. Gone is the good will and teain spirit
countability system. of administration and staff and I believe this
The Nature of Qualitative Inquiri/ [J. 19

ali begins at the top. One can work in these teachers and administration and the Board of
conditions but why, if it is to 'shape up' a few Education ali had a good working relation-
poor teachers. Instead, it's having disastrous ship. In the past 4 yearsunder the present
results on the whole faculty community." superintendentI find the atmosphere dete-
rioratmg to the point where teachers distrust
Teacher Response No. 244: "In order to fully
each other and teachers do not trust adminis-
understand the oppressive, stifling atmo-
trators at ali! We understand the position the
sphere in Kalamazoo you have to ^be in the
administrators have been forced into and feel
trenches'the classrooms. In 10 years of
compassion for themhoweverwe still
teaching, 1 have never ended a school year as
have no trust! Going to school each morning is
depressed about 'education' as 1 have this
no longer an enjoyable experience."
year. lf things do not improve in the next two
years, I will leave education. The Kalamazoo
Teacher Response No. 261: "A teacher
accountability system must be viewed in its
needs some checks and balances to function
totality and not just the individual component
effectively; it would be ridiculous to think
parts of it. In to to, it is oppressive and stifling.
otherwiseif you are a concerned teacher. But
"In teaching government and history, stu-
in teaching you are not turning out neatly
dents often asked what it was like to live in a
packaged little mechanical products ali alike
dictatorship. I now know firsthand.
and endowed with the same qualities. This
"The superintendent with his accountabil-
nonsensical accountability program we have
ity model and his abrasive condescending
here makes the superintendent look good to
manner has managed in three short years to
the community. But someone who is in the
destroy teacher morale and effective creative
classroom dealing with ali types of kids, some
classroom teaching.
who cannot read, some who hardly ever come
"Last evening my wife and 1 went to an end
to school, some who are in and out of jail, this
of the school year party. The atmosphere there
teacher can see that and the rigid accountabil-
was strangelittle exuberance, laughter or re-
ity model that neglects the above mentioned
lease. People who in previous years laughed,
problems is pur 'BULLSHIT!' "
sang and danced were unnaturally quiet and
somber. Most people went home early. The
Teacher Response No. 251: " 'Fear' is the
key topic was the superintendent, the school
word for 'accountability' as applied in our sys-
board election, and a millage campaign. Peo-
tem. My teaching before 'Accountability' is
ple are still tense and uncertain.
the same as now. 'Accountability' is a political
"While the school board does not 'pay us to
ploy to maintain power. Whatever good there
be happy' it certainly must recognize that
may have been in it in the beginning has been
emotional stability is necessary for effective
destroyed by the awareness that each new ed-
teaching to take place. The involuntary trans-
ucational 'system' has at its base a political
fers, intimidation, coercion and top to bottom
motive. Students get screwed... . The bitter-
'channelized' communication in Kalamazoo
ness and hatred in our system is incredible.
must qualify this school system for the list of
What began as 'noble' has been destroyed.
'least desirable' school systems in the nation."
You wouldn'tbelieve the new layers of admin-
Teacher Response No. 233: "1 have taught in istration that have been created just to keep
Kalamazoo for 15 years and under five super- this monster going.
intendents. Until the present superintendent, I "Our finest compliment around our state is
found working conditions to be enjoyable and that the other school systems know what is go-
20 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

ing on and are having none of it. Lucky people.

Come down and visit in hell sometime."
T H n i i n H T I - i:: r f c = A : ; H n

Face Validity and Credibility V 'ii''..rr i ! . . !' J..I- : = ..!'! h-I S i.

sar i/.w!- i ui-Ji in:-! !Vi- =i ::!.!1
;

What was the impact of the qualitative :' : - i ! i V A ' i- ! ! I U ! i ! '.. i 11 i ! : i" ! i . i - ! i ! 11 .-. L-

data collected from teachers in Kalamazoo? i.-i!. v i : 1 ! * 1 1 i : : ij 5 "i:-!* .. >. !.; . i v : i r.-."i i . ! : i !: j . l \ i -i

You will recall that many of the school board V . ir,!. r i.'i. 1 ' ' I ! :. ! i:' 1 i : ' . ! : .'i' i ! = ! ' : . \\i'-1 i',
members initially dismissed the standard- l*r I l i - ! ' . ! | !.'. ii' i ::!= I ! I ' . ' y ! j M '

ized questionnaire responses as biased, : : ; . ' ! " ' < : !':/i.! . V i i . i i : i ' i i I M ! . - " i i . '
!

rigged, and the predictable result of the un- "iinkl ii!' MIv- ,I..!,:' \l ,\ iij- Ji, M " Vil1

ion's campaign to discredit school officials. I ! ! i ! m i: -|i- .V j.. .'! i' . r m Y ! r l : . i\ : i V I i/.'.'i I T

However, after readmg through a few pages Lnl lv > M -i i'T=" Vi }-, i . ! Vi' i i vl-I l'i '.Vi.'i !,=
of the teachers' own personal comments, af- i !.': !:.= .vil !1
i r r ! =" -."> s* 'ii.r Vi:. ii i 'i ! i'il' i.i \U

ter hearing about teachers' experiences with :!:- ! ri Jal-iiS i' }':-,* IRIRU .*=iJ-: !I .''

the accountability system in their own i -i i T ' N ! ! ! .T = T :; i. 1'Vi 1 ! N 1 ,',T! ' r

words, the tenor of the discussion about the Jt i" i '.ri :'i!' i l/i .'l i '||.
! ' 'i IV! 'i i! A I- i i' ! I VI i1 !.'

evaluation report changed. School board = != >!!' i-i-iiilrvi .:".'!! i'! ^vl' -i iVi-'
members could easily reject what they per- !!: nr-D-V, ''"" - w s ^ ir p * m' /-vy-'!^
ceived as a "loaded" questionnaire. They .v r it i;v! :
! ,'i -i 11; ;,yl i 'i ( ' . i 1 \ i ' ' 'i l' i h ) r
could not so easily dismiss the anguish, fear, H -Vlro" hr !Oi mi'i-rv hi - p r
and depth of concern revealed m the teach- i : "ii-3-:= :: N-V .*'
ers' own reflections. The teachers' words on|i !"i'i .!':;!>"! -.V.?- r l n j l l - i r i " '
had face validity and credibility. Discussion IlIllIllIlISiSSSSiSSiSSiSSSiiSSSIiSSSiS!
of the evaluation results shifted from an at-
tack on the measures used to the question:
"What do you think we should do?"
During the summer of 1976, following teachers' own words became part of the im-
discussion of the evaluation report, the su- petus for change in Kalamazoo.
perintender "resigned." The new superin-
tendent and school board in 1976-1977 used
The Purpose of
the evaluation report as a basis for starting
Open-Ended Responses
fresh with teachers. A year la ter teacher as-
sociation officials reported a new environ- The preceding example illus trates the dif-
ment of teacher-administration cooperation ference between qualitative inquiry based
in developing a mutually acceptable ac- on responses to open-ended questions and
countability system. The evaluation report quantitative measurement based on scales
did not directly cause these changes. Many composed of standardized questionnaire
other factors were involved in Kalamazoo at items. Quantitative measures are succinct,
that time. However, the qualitative informa- parsimonious, and easily aggregated for
tion in the evaluation report revealed the full analysis; quantitative data are systematic,
scope and nature of teachers' feelings about standardized, and easily presented in a
what it was like to work in the atmosphere short space. By contrast, the qualitative find-
created by the accountability system. The ings are longer, more detailed, and variable
depth of those feelings as expressed in the m content; analysis is difficult because re-
The Nature of Qualitative Inquiri/ [J. 2 1

sponses are neither systematic nor standard- the open-ended comments of the Kalama-
ized. Yet, the open-ended responses permit zoo teachers illustrate the fruit of qualitative
one to understand the world as seen by the methods.
respondents. The purpose of gathering re- While the Kalamazoo example illustrates
sponses to open-ended questions is to en- the most elementary form of qualitative in-
able the researcher to understand and cap- quiry, namely, responses from open-ended
ture the points of view of other people questionnaire items, the major way in which
without predetermining those points of qualitative researchers seek to understand
view through prior selection of question- the perceptions, feelings, and knowledge of
naire categories. As Lofland (1971) put it: people is through in-depth, intensive inter-
"To capture participants 'in their own terms' viewing. The chapter on interviewing will
one must learn their categories for rendering discuss ways of gathering high-quality in-
explicable and coherent the flux of raw real- forma tion from peopledata that reveal ex-
ity. That, indeed, is the first principie of qual- periences with program activities and per-
itative analysis" (p. 7, emphasis added). spectives on treatment impacts from the
Direct quotations are a basic source of raw points of view of participants, staff, and oth-
data in qualitative inquiry, revealing respon- ers involved in and knowledgeable about
dents' depth of emotion, the ways they have the program or treatment being evaluated.
organized their world, their thoughts about
what is happening, their experiences, and Inquiry by Observation
their basic perceptions. The task for the qual-
itative researcher is to provide a framework What people say is a major source of qual-
within which people can respond in a way itative data, whether what they say is ob-
that represents accurately and thoroughly tained verbally through an interview or in
their points of view about the world, or that written form through document analysis or
part of the world about which they are talk- survey responses. There are limitations,
ingfor example, their experience with a however, to how much can be learned from
particular program being evaluated. Too of- what people say. To understand fully the
ten social scientists "enter the field with pre- complexities of many situations, direct par-
conceptions that prevent them from allow- ticipation in and observation of the phenom-
ing those studied to 'tell it as they see it' " enon of interest may be the best research
(Denzin 1978b:10). method. Howard S. Becker, one of the lead-
I have included the Kalamazoo evalua- ing practitioners of qualitative methods in
tion findings as an illustration of qualita- the conduct of social science research, ar-
tive inquiry because open-ended responses gues that participant observation is the most
on questionnaires represent the most ele- comprehensive of ali types of research strat-
mentary form of qualitative data. There are egies.
severe limitations to open-ended data col-
lected in writing on questionnaires, limi- The most complete form of the sociological da-
tations related to the writing skills of re- tum, after ali, is the form in which the partici-
spondents, the impossibility of probing or pant observer gathers it: an observation of
extending responses, and the effort required some social event, the events which precede
of the person completing the questionnaire. and follow it, and explana tions of its meaning
Yet, even at this elementary levei of inquiry, by participants andspectators,before, during,
the depth and detail of feelings revealed in and after its occurrence. Such a datum gives us
22 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Certain really discriminating people like nothing better than to relax on the
beach with a good, in-depth, and detailed qualitative study in hand.

moreinformation about the event under study Observational data, especially partici-
than data gathered by any other sociological pant observation, permit the evaluation re-
method. (Becker and Geer 1970:133) searcher to understand a program or treat-
The Nature of Qualitative Inquiri/ [J. 23

ment to an extent not entirely possible using ents so that they could make conscious
only the insights of others obtained through choices about their own parenting styles and
interviews. Of course, not everything can increase their confidence about the choices
be directly observed or experienced, and they make. Parents were also to be treated
participant observation is a highly labor- with respect and to be recognized as the pri-
intensiveand, therefore, relatively expensive mary educators of their childrenin other
research strategy. In a later chapter, strate- words, the early childhood educators were
gies for using observational methods, in- not to impose their expertise upon parents
cluding both participant and nonparticipant but, instead, to make clear that parents are
approaches, will be discussed at length. My the real experts about their own children.
purpose at this point is simply to give the Site visits were made to ali programs, and
reader another taste of the fruit of qualitative parenting discussions were observed on
methods. Before discussing how to collect each site visit. Descriptions of these sessions
observational evaluation data, it is helpful to then became the primary data of the evalua-
know what such data should look like. tion. In short, the evaluators were to be the
The purpose of observational analysis is eyes and ears of the legislature and the state
to take the reader into the setting that was program staff, permitting them to under-
observed. This means that observational stand what was happening in various parent
data must have depth and detail. The data sessions throughout the state. Descriptive
must be descriptivesufficiently descrip- data about the sessions also provided a mir-
tive that the reader can understand what oc- ror for the staff who conducted those ses-
curred and how it occurred. The observer's sions, a way of looking at what they were do-
notes become the eyes, ears, and perceptual ing to see if that was what they wanted to be
senses for the reader. The descriptions must doing.
be factual, accurate, and thorough without What follows is a description from one
being cluttered by irrelevant minutiae and such session. The criterion that should be
trivia. The basic criterion to apply to a re- applied in reading this description is the ex-
corded observation is the extent to which the tent to which sufficient data are provided to
observation permits the reader to enter the take you, the reader, into the setting and per-
situation under study. mit you to make your own judgment about
The observation that follows is meant to the nature and quality of parent education
illustrate what such a descriptive account is being provided.
like. This evaluation excerpt describes a
two-hour observation of mothers discussing
their child rearing in a parent education pro- OBSERVATION DATA ILLUSTRATED:
gram. The purpose of the program, one of 22 A DISCUSSION FOR MOTHERS
such state-supported programs, was to in- OF TWO- YEAR-OLDS
crease the skills, knowledge, and confidence
of parents. The program was also aimed at The group discussion component of this
providing a support group for parents. In parent education program operates out of a
funding the program, legislators empha- small classroom in the basement of a church.
sized that they did not want parents to be The toddler center is directly overhead on
told how to rear their children. Rather, the the first floor so that noises made by the chil-
purpose of the parent education sessions dren these mothers have left upstairs can be
was to increase the options available to par- heard during the discussion. The room is
24 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

just large enough for the 12 mothers, one other mothers will talk about their own ex-
staff person, and me to sit along three sides periences as they want to. For example, one
of the room. The fourth side is used for a of the topics is the problem a mother is hav-
movie screen. Some mothers are smoking. ing with her child urinating in the bathtub.
(The staff person told me afterward that Other mothers share their experiences with
smoking had been negotiated and agreed on this problem, ways of handling it, and
among the mothers.) The seats are padded whether or not to be concerned about it. The
folding chairs plus two couches. A few col- crux of that discussion seems to be that it is
orful posters with pictures of children play- not a big deal and not something that the
ing decorate the walls. Small tables are avail- mother ought to be terribly concerned
able for holding coffee cups and ashtrays about. It is important not to make it a big
during the discussion. The back wall is lined deal for the child; the child will outgrow it.
with brochures on child care and child de- The discussion turns to things that
velopment, and a metal cabinet in the room two-year-olds can do around the house to
holds additional program materiais. help their mothers. This is followed by some
The session begins with mothers watch- discussion of the things that two-year-olds
ing a 20-minute film about preschool chil- can't do and some of their frustrations in try-
dren. The film forms the basis for getting dis- ing to do things. There is a good deal of
cussion started about "what two-year-olds laughing, sharing of funny stories about
do." Louise, a part-time staff person in her children, and sharing of frustrations about
early 30s who has two young children of her children. The atmosphere is informal and
own, one a two-year-old, leads the discus- there is a good deal of intensity in listening.
sion. Louise asks the mothers to begin by Mothers seem especially to pick up on
picking out from the film. things that their things that they share in common about the
own children do, and talking about the way problems they have with their children.
some of the problems with children were Another issue from another mother is the
handled in the film. For the most part, the problem of her child pouring out her milk.
mothers share happy, play activities their She asks, "What does it mean?" This ques-
children like. "My Johnny loves the play- tion elicits some suggestions about using
ground just like the kids in the film." "Yeah, water aprons and cups that don't spill and
mine could live on the playground." other mothers' similar problems, but the dis-
The focus of the discussion turns quickly cussion is not focused and does not really
to what happens as children grow older, come to much closure. The water apron sug-
how they change and develop. Louise com- gestion brings up a question about whether
ments, "Don't worry about what kids do at a or not a plastic bag is okay. The discussion
particular age. Like don't worry that your turns to the safety problems with different
kid has to do a certain thing at age two or else kinds of plastic bags. About 20 minutes of
he's behind in development or ahead of de- discussion have now taken place. (At this
velopment. There's just a lot of variation in point, one mother leaves because she hears
the ages at which kids do things." her child crying upstairs.)
The discussion is free flowing and, once The discussion returns to giving chil-
begun, is not directed much by Louise. dren baths. Louise interjects, "Two-year-
Mothers talk back and forth to each other, olds should not be left alone in the bathtub."
sharing experiences about their children. A With reference to the earlier discussion
mother will bring up a particular point and about urinating in the bathtub, a mother in-
The Nature of Qualitative Inquiri/ [J. 2 5

teijects that water with urine in it is prob- the 11 mothers have participated, most of
ably better than the lake water her kids swim them actively. Four mothers have not partic-
in. The rnother with the child who urinates ipated.)
in the bathtub says again, "It really bugs me Another mother brings up a new prob-
when he urinates in the tub." Louise re- lem. Her child is destroying her plants,
sponds, "It really is your problem, not his. dumping plants out, and tearing them up.
If you can calm yourself down, he'll be "I really get mad." She says that the tech-
okay." nique she has used for punishment is to iso-
At a lull in the discussion, Louise asks, late the child. Then she asks, "How long do
"Did you agree with everything in the you have to punish a two-year-old before it
movie?" The mothers talk a bit about this starts working?" This question is followed
and focus on an incident in the movie where by intense discussion with several mothers
one child bites another. Mothers share sto- making comments. (This discussion is re-
ries about problems they've had with their produced in full to illustrate the type of dis-
childrenbiting. Louise inteijects, "Biting can cussion that occurred.)
be dangerous. It is important to do some-
thing about biting." The discussion turns to Mother No. 2: "Maybe he needs his own
what to do. One mother suggests biting the plant. Sometimes it helps to let a child
child back. Another mother suggests that have his own plant to take care of and
kids will work it out themselves by biting then he comes to appreciate plants."
each other back. Mothers get very agitated,
Mother No. 3: "Maybe he likes to play in the
more than one mother talks at a time. Louise
dirt. Does he have his own sand or dirt to
asks them to "cool it," so that only one per-
play in around the house?"
son talks at a time. (The mother who had left
returns.) Mother No. 4: "Oatmeal is another good
The discussion about biting leads to a dis- thing to play in."
cussion about child conflict and fighting in Louise: "Rice is another thing that children
general, for example, the problem of chil- like to play in and it's clean, good to use
dren hitting each other or hitting their moth- indoors."
ers. Again, the question arises about what to
do. One mother suggests that when her child Mother No. 5: "Some things to play in would
hits her, she hits him back, or when her child be bad or dangerous. For example, pow-
bites her, she bites him back. Louise inter- dered soap isn't a good thing to let kids
jects, "Don't modelbehavior you don't like." play in."
She goes on to expiam that her philosophy is Mother No. 2: "Can you put the plants where
that you should not do things as a model for he can't get at them?"
children that you don't want them to do. She
says that works best for her; however, other Mother with problem: "I have too many plants,
mothers may find other things that work I can't put them ali out of the way."
better for them. Louise comments that hit- Louise: "Can you put the plants somewhere
ting back or biting back is a technique sug- else or provide a place to play with dirt or
gested by Dreikurs. She says she disagrees rice?" (Mother with problem kind of
with that technique, "but you ali have to de- shakes her head no. Louise goes on.) "An-
cide what works for you." (About 40 min- other thing is to tell the kid the plants are
utes have now passed since the film, and 7 of alive, to help him learn respect for living
26 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

things. Tell him that those plants are alive time. Several mothers give their points of
and that it hurts them. Give him his own view.
plant that he can get an investment in." Louise: "The person who owns the house
sets the rales. Two-year-olds can learn to be
Mother with problem: "IT1 try it."
careful. But don't go around ali day long
Mother No. 2: "You've got to be fair about a saying, "No, no.' "
two-year-old. You can't expect them not
to touch things. It's not fair. I try hanging The time had come for the discussion to
ali my plants." end. The mothers stayed around for about 15
minutes, interacting informally and then go-
Louise: "Sometimes just moving a child
ing upstairs to get their children into their
bodily away from the thing you don't
winter coats and hats for the trip home. They
want him to do is the best technique."
seemed to have enjoyed themselves and
Mother No. 4: "They'11 outgrow it anyway." continue d talking informally. One mother
with whom Louise had disagreed about the
Mother with problem: "Now he deliberately issue of whether it was ali right to bite or hit
dumps them and I really get angry." children back stopped to continue the dis-
Louise: "Maybe he feels a rivalry with the cussion. Louise said:
plants if you have so many. Maybe he's
trying to compete." I hope you know that I respect your right to
have your own views on things. I wasn't try-
Mother No. 3: "Let him help with the plants.
ing to tell you what to do. I just disagreed, but I
Do you ever let him help you take care of
definitely feel that everybody has a right to
the plants?"
their own opinion. Part of the purpose of the
Mother No. 6: "Some plants are dangerous to group is for everyone to be able to come to-
help with." gether and appreciate other points of view and
understand what works for different people.
Louise: "Some dangerous house plants are
poison."
The mother said that she certainly didn't
feel bad about the disagreement and she
Louise reaches up and pulls down a bro- knew that some things that worked for other
chure on plants that are dangerous and says people didn't work for her and that she had
she has brochures for everyone. Several peo- her own ways but that she really enjoyed the
ple say that they want brochures and she group.
goes to the cabinet to make them available. Louise cleaned up the room, and the ses-
One mother who has not participated version ended.
bally up to this point specifically requests a
brochure. This is followed by a discussion of
child-proofing a house as a method of child The Raw Data of
rearing versus training the child not to touch Qualitative Inquiry
things, but with less emphasis on child-
proofing, that is, removing temptation ver- The description of this parenting session
sus teaching children to resist temptation. is aimed at permitting the reader to under-
One parent suggests, in this context, that stand what occurred in the session. These
children be taught one valuable thing at a data are descriptive. Pure description and
The Nature of Qualitative Inquiri/ [J. 27

m a w i n L m a i L ^ E S : . q u a o t k &s w e l i h d s c or o m M
i.' v I : V;;-;1.".'!" v i y i i v r YX^Ua rr: r ! . ; i ! i r=,V.V:. n-.f ' i'.v:! vi Vi -i!i!i-j'! ! ii j-- -V
:!!.= ':; i":;-'Vi PTO"! !"" :>"11'! ,;.=:::'!; rw: 1 ? r- ! 'T'-!.!"*
!
: = rv| V- - , v
:".'!! :V: v.ifi'V! ! V! !| jVi!! l'j'i:".":i" " l"l!"| jVi " i= h=' i.M-yLi i.V \:biY. i r!Jri j!i Vili y.if rrifia'.Own
u M ! ! ' ! | ! = : : ! * i i " V | : : ! : ! | ! ! = : : i > i-1 V i ! ; . ; :i:T:I.I::" [rwyf
| "'! '.-] 11 "Ti V !! Vi .V! ! v.- r Vi :i i i i' ::v,'i rs.!
i,=vi..: li
: !!=, ir.M r. :;=:m:i: rt r ="i ! i i." :! SlSllSISSSSSlllilIillilIIillIllIll
! ' :,; \m .'h- v; ;= 1 I! |.-i! >:!iVi
;; '
!"'!!:Yi :;y r r - ! m= : : y : : r . ': r - : i ; . - ' i u - m ,
>-i i'i1:;i:i,:|:'i:!i.i!'n =v'..vi!':i"i.i:'-:i.:!.'::! i.V::=-S.!
!! n:v!. n.;; rw;: !=.: iV: :;=.! :; = ! i ; v S ivy-..i i i: h ,'i
M iV!1- : -|.:"s 3.!.P:V \ i- UK-S '! ii-i:'"'-i:!'1 i!!!
;!.! !==: i i v : - : !: l v !.n'V i " - : ! 1j - d =:.==.!
! !

li/' i'i!.bi:'r:"
!

: r' ==::-!: -:T VT' y^^ii i'

:::;!'i'!'-^"''!:'::-.- rp.SPJi.M " ! ! " M i 1 ;!.!;,:: i^M-iVi n.;;T! j,'! jV i.!:'" >:U 'Vi jJ"i -
:;!i:v::::yi!V!v;j;= !!.;:!: .'|V! : i? w n"-" .s'! :;!'=:: 5.--I=: J- iniratim I ' =V;I!l-5 i s;="="s !'i v:i'iyO U
:;::. i J: zrvkor* .^vir niikip -r-i' i1! il1:'i Vi lVV!:'! S!' /il-.r'!! M Vti? :ii! -.Vin V
; v i : ! .! : ;. =;=-.! : n \ >: r : i f y m -'n l!! r . m v :;=!:, u n .'i-i:!!!;!!: iWiVii1 .vi nu ^.'iiVni! i:M:I.! r-i
ii:11 i-1! .';!;!: 'I! J; =:!: f:!.;: sT!i.!:ij:i VJ;: ^.jivi;' hi;Vi'!;'i 5V; .'!!'"ii.;r !ri;v"
zaxrw, 00 r nrn -i:; ,-cv 1 - i : ; ! a ! v ; V ! :i !=
: ihMi i r : ! i ' ! i ! ! i j k U ! ' ' ! ! Z iJ,:!:!'! i-i
!

i.'i i j i" 1 v ! 1V! !i'ij.'i ". .. m i i K' !.;i.V I i.',1' iV,''i

V!; i v[ i." i " i:! t<i1.? . i.!iVii-ii=: iV:"i:-i (ivu/irrr- rffr, jrirr!";!I1 !V!;;;;'iV-jj! rc!IM;j-vv1 ' i'
Ll:,'i!'.Vi :i.:!! a! !! i l 'V:!-: iVfti:. f ,':i .ti :!'!! i! W !=:!'!!f'M'i'=. i^Ij,L-I,.: j.|iHTV =V

quotations are the raw data of qualitative in- sented as separa te and distinct from each
quiry. other. In practice, they are often fully inte-
The description is meant to take the grated approaches. Becoming a skilled ob-
reader into the setting. The data do not in- server is essential even if you concentrate
clude judgments about whether what oc- primarily on interviewing because every
curred was good or bad, appropriate or in- face-to-face interview also involves and re-
appropriate, or any other interpretive quires observation. The skilled interviewer
judgments. The data simply describe what is thus also a skilled observer, able to read
occurred. State legislators, program staff, nonverbal messages, sensitive to how the in-
parents, and others used this description, terview setting can affect what is said, and
and descriptions like this from other pro- carefully attuned to the nuances of the inter-
gram sites, to discuss what they wanted the viewer-interviewee interaction and relation-
programs to be and do. The descriptions ship.
helped them make explicit their own judg- Likewise, interviewing skills are essential
mental criteria. for the observer because during fieldwork,
In later chapters, guidance on interpret- you will need and want to talk with people,
ing qualitative data will be offered in depth. whether formally or informally. Participant
observers gather a great deal of information
People-Oriented Inquiry through informal, naturally occurring con-
versations. Understanding that interview-
Thus far, the examples of observation and ing and observation are mutually reinforc-
interviewing in this chapter have been pre- ing qualitative techniques is a bridge to
28 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

understanding the fundamentally people- The Fruit of Qualitative

oriented nature of qualitative inquiry. Methods Revisited
Sociologist John Lofland has suggested
that there are four people-oriented man- This chapter began with the parable of the
dates in collecting qualitative data. First, the man who traveled far in search of a widely
qualitative methodologist must get close proclaimed food called "fruit." When finally
enough to the people and situation being directed to a fruit tree, he confused the
studied to personally understand in depth spring blossom of the tree with the fruit of
the details of what goes on. Second, the qual- the tree. Finding the blossom to be tasteless,
itative methodologist must aim at capturing he dismissed ali he had heard about fruit as a
what actually takes place and what people hoax and went on his way. This chapter has
actually say: the perceived facts. Third, qual- described qualitative data so that the person
itative data must include a great deal of pure in search of the fruits of qualitative methods
description of people, activities, interac- will know what to look forand know
tions, and settings. Fourth, qualitative data when the real thing has been attained. Ex-
must include direct quotations from people, hibit 1.5 lists Internet resources for those
both what they speak and what they write who want to carry on this search for qualita-
down. tive fruit in virtual space. To close this chap-
ter, it may be instructive to consider two
other short parables about the search for
The commitment to get close, to be factual, de- fruit.
scriptive and quotive, constitutes a significant While the first seeker after fruit arrived
commitment to represent the participants in too early to experience the ripened delicacy
their own terms. This does not mean that one and tasted only the blossom, a second seeker
becomes an apologist for them, but rather that after fruit arrived at a tree that had been im-
one faithfully depicts what goes on in their properly cultivated, so that its fruit was
lives and what life is like for them, in such a shriveled and bitter. This bad fruit had been
way that one's audience is at least partially left to rot. Not knowing what good fruit
able to project themselves into the point of looked like, he sampled the bad. "Well, I've
view of the people depicted. They can "take seen and tasted fruit," he said, "and I can tell
the role of the other" because the reprter has you for sure that it's terrible. I've had it with
given them a living sense of day-to-day talk, fruit. Forget it. This stuff is awful." He went
day-to-day activities, day-to-day concerns on his way and his journey was wasted.
and p r o b l e m s , . . . One can hope that such a foolish mis take
A major methodological consequence of is less likely today, because early in school
these commitments is that the qualitative students are taught the danger of generaliz-
study of people in situ is a process of discovery. It ing from limited cases. Yet, rumors persist
is of necessity a process of learning what is that some people continue to reject ali quali-
happening. Since a major part of what is hap- tative data as worthless (and "rotten"), hav-
pening is provided by people in their own ing experienced only bad samples produced
terms, one must find out about those terms with poor methods.
rather than impose upon them a preconceived A third seeker after fruit arrived at the
or outsider's scheme of what they are about. It same tree that produced the shriveled and
is the observer's task to find out what is funda- bitter fruit. He picked some of the rotting
mental or central to the people or world under fruit and examined it. He took the fruit to a
observation. (Lofland 1971:4) farmer who cultivated fruit trees with great
The Nature of Qualitative Inquiri/ [J. 29

Internet E-mail Discussion Groups (listservs) on

Qualitative Methods

1. [email protected]: Qualitative Research for the Human Sciences; to subscribe,

send this message to [email protected]: subscribe QUALRS-L yourname

2. [email protected]: Qualitative Research in Management and Organization Studies;

to subscribe, send this message to [email protected]: subscribe qualnet

3. [email protected]: Qualitative Research List, initiated by Penn State, but immediately

attracted a broader audience; to subscribe, send this message to [email protected]:
subscribe QUAL-L firstname lastname

Other resources for qualitative evaluation and research:

4. [email protected]: American Evaluation Association (AEA) Discussion List; to

subscribe, send this message to [email protected]: subscribe evaltalk ourname

AEA home page with links to evaluation organizations, training programs, and
Internet resources: www.eval.org

5. [email protected]: A list for social science research methods instructors; to subscribe,

send this message to [email protected]: join methods yourname

NOTE: Thanks to Judith Preisse, Aderhold Distinguished Professor, Social Foundationsof Education, University
of Gergia, for list subscription details. These sites and subscription details may change, and this list is not ex-
haustive. This list is meant to be suggestive of the qualitative resources availabie through the Internet. See
Chapter3, Exhibit 3.7;Chapter4, Exhibit 4.9; and Chapter8, Exhibit 8.3,for addtional, morespecialized quali-
tative resources through the Internet.

success. The farmer peeled away the rotten the stonelike thing he held in his hand was a
exterior and exposed what looked like a seed, ali he had to do was plant it, tend prop-
stone inside. The farmer told him how to erly the tree's growth, and work for the
plant this hard core, cultivate the resulting eventual harvestthe fruit. Though there
trees, and harvest the desired delicacy. The was much work to be done and there were
farmer also gave him a plump, ripe sample many things to be leamed, the resulting
to taste. Once the seeker after fruit knew high-quality fruit was worth the effort.
what fruit really was, and once he knew that
Between-Chapters Interlude
!

i
Top Ten Pieces of Advice to a
Graduate Student Considering
a Qualitative Dissertation

3. Top Ten Responses

T he following query was posted on

an Internet listserv devoted to dis-
cussing qualitative inquiry:

I am a new graduate student thinking about

1. Be sure that a qualitative approach fits
your research questions: questions about
doing a qualitative dissertation. I know you
people's experiences; inquiry into the
are ali busy, but I would appreciate an an-
meanings people make of their expe-
swer to only one question.
riences; studying a person in the con-
If you could give just one bit of advice to a
text of her or his social/interpersonal
student considering qualitative research for a
environment; and research where not
dissertation, what would it be?
enough is known about a phenomenon
for standardized instruments to have
The responses below carne from differ- been developed (or even to be ready to
ent people. I've combined some responses, be developed).
edited them (while trying to maintain the (Chapter 2 will help with this by
flavor of the postings), and arranged them presenting the primary themes of qual-
for coherence. itative inquiry.)

!J. 33
34 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

2. Study qualitative research. There are 5. Practice interviewing and observation

lots of different approaches and a lot to skills. Practice! Practice! Practice! Do
know. Study carefully a couple of the lots of intervie ws. Spend a lot of time do-
books that provide an overview of dif- ing practice fieldwork observations. Get
ferent approaches, then go to the origi- feedback from someone who's really
nal sources for the design and analysis good at interviewing and observations.
details of the approach you decide to There's an amazing amount to learn.
use. And it's not just head stuff. Qualitative
(Chapter 3 covers different qualita- research takes skill. Don't make the
tive approaches.) mis take of thinking it's easy. The better I
get at it, the more I realize how bad I was
3. Find a dissertation adviser who will when I started.
support your doing qualitative re- (Chapters 6 and 7 cover the skills of
search. Otherwise, it can be a long, qualitative inquiry.)
tough haul. A dissertation is a big com-
mitment. There are other practical ap- 6. Figure out analysis before you gather
proaches to using qualitative methods data. I've talked with lots of advanced
that don't involve ali the constraints of grad students who rushed to collect
doing a dissertation, things like pro- data before they knew anything about
gram evaluation, action research, and analyzing itand lived to regret it, big
organizational development. You can time. This is true for statistical data and
still do lots of great qualitative work quantitative data, but somehow people
without doing a dissertation. But if you seem to think that qualitative data are
can find a supportive adviser and com- easy to analyze. No way. That's a big-
mittee, then, by ali means, go for it. time NO WAY. And don't think that the
(Chapter 4 covers particularly ap- new software will solve the problem.
propriate practical applications of quali- Another big-time NO WAY. You, that's
tative methods.) YOU, still have to analyze the data.
(Chapter 8 covers analysis.)
4. Really work on design. Qualitative de-
signs follow a completely different logic 7. Be sure that you're prepared to deal
from quantitative research. Completely with the controversies of doing qualita-
different. Are you listening? Com- tive research. People on this listserv are
pletely different. Especially sampling. constantly sharing stories about people
This is not the same as questionnaires who don't "get" qualitative research
and tests and experiments. You can and put it down. Don't go into it naively.
combine designs, like quant and qual Understand the paradigms and politics.
approaches, but that gets really compli- (Chapter 9 deals with paradigms,
cated. Either way, you have to figure out politics, and ways of enhancing the
what's unique about qualitative de- credibility of qualitative inquiry.)
signs.
(Chapter 5 covers qualitative de- 8. Do it because you want to and are con-
signs.) vinced it's right for you. Don't do it be-
Top Ten Pieces ofAdvice !}. 35

cause someone told you it would be storm, and problem solve, as well as
easier. It's not. Try as hard as possible share in each other 's successes, ali in a
to pick/negotiate dissertation research more relaxed environment that helps
questions that have to do with some take some of the edge off the stress
passion/interest in your professional (for example, you might have potluck
life. Qualitative research is time-con- meals at different homes?). This can be
suming, intima te, and intenseyou will tremendously liberating (even on a less
need to find your questions interesting than regular basis). Take care of your-
if you want to be at ali sane during the self.
processand still sane at the end.
10. Prepare to be changed. Looking deeply
9. Find a good mentor or support group. at other people's lives will force you to
Or both. In fact, find several of each. If look deeply at yourself.
you can, start a small group of peers in (See the discussions "Voice, Perspec-
the same boat, so to speak, to talk about tive, and Reflexivity" in Chapter 2 and
your research together on a regular "The Observer and What Is Observed:
basisyou can share knowledge, brain- Unity and Separation" in Chapter 6.)
Strategic Themes in
Qualitative Inquiry

rand strategy should guide tactical decisions. Within a grand strat-

egy ali manner of tactical errors may be made, and indeed, are inevi-
table, but can be corrected as long as the strategic vision remains true and fo-
cused. At least that's the theory. In practice . . . ? Try it and see.

Halcolm

ene^al T-Vinciples

Strategos is a Greek word meaning "the thinking and action of a general."

What it means to be strategic is epitomized by that greatest of Greek generais,
Alexander. He conducted his first independent military operation in northern
Macedonia at age 16. He became the ruler of Macedonia after his father, Philip,
was assassinated in 336 B.C. Two years later, he embarked on an invasion of Pr-
sia and conquest of the known world. In the Battle of Arbela, he decisively de-
feated Darius III, King of Kings of the Persian Empire, despite being outnum-
bered 5 to 1 (250,000 Persians against Alexander and fewer than 50,000 Greeks).
Alexander's military conquests are legend. What is less known and little ap-
preciated is that his battlefield victories depended on in-depth knowledge of the
38 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

I psychology and culture of the ordinary people and military leaders in opposing
I armies. He included in his military intelligence information about the beliefs,
| worldview, motivations, and patterns of behavior of those he faced. Moreover,
his conquests and subsequent rule were more economic and political in nature
than military. He used what we would now understand to be psychological, so-
ciological, and anthropological insights. He understood that lasting victory de-
pended on the goodwill of and alliances with non-Greek peoples. He carefully
studied the customs and conditions of people he conquered and adapted his
policiespolitically, econornically, and culturallyto promote good conditions
in each locale so that the people were reasonably well-disposed toward his rule
(Garcia 1984).
In this approach, Alexander had to overcome the arrogance and ethnocen-
trism of his own training, culture, and Greek philosophy. Historian C. A. Robin-
son, Jr. explained that Alexander was brought up in Plato's theory that ali
non-Greeks were barbarians, enemies of the Greeks by nature, and Aristotle
taught that ali barbarians (non-Greeks) were slaves by nature. But
Alexander had been able to test the smugness of the Greeks by actual contact with
the barbarians,. . . and experience had apparently convinced him of the essential
sameness of ali people. (Robinson 1949:136)

In addition to being a great general and enlightened ruler, Alexander appears

to have been an extraordinary ethnographer, a qualitative inquirer par excel-
lence, using observa tions and firsthand experience to systematically study and
understand the peoples he encountered and to challenge his own culture's prej-
udices.
And as Halcolm finished telling the story of Alexander the Great, he re-
minded those assembled that skills in observation and interviewing are life
skills for experiencing the world. "One can say of qualitative inquiry what Mar-
eei Proust said of art, Thanks to this, instead of seeing one world, our own, we
see it multiplied. So many worlds are at our disposal.' "

From Halcolm's Historical Biographies

The Purpose of a Strategic Framework

i P;
a ^ s erception is strong and sight weak. In strategy it is important to see
distant things as if they were close and to take a distanced view of
close things.

Miyamoto Musashi (1584-1645),

Japanese warrior, strategist
Strategic Themes in Qualitative Inquiri/ J. 39

D on't mistake a clear view for a short distance.

Grand Canyon hiking advice

verybody has a plan until they've been hit.

Old boxing saying

A well-conceived strategy, by providing these researchers have in common? They are

overall direction, provides a framework for in the field studying the real world as it un-
decision making and action. It permits folds.
seemingly isolated tasks and activities to fit Qualitative designs are naturalistic to the
together, integra ting separate efforts toward extent that the research takes place in real-
a common purpose. Specific study design world settings and the researcher does not
and methods decisions are best made within attempt to manipulate the phenomenon of
an overall strategic framework. This chap- interest (e.g., a group, event, program, com-
ter offers 12 major themes or principies of munity, relationship, or interaction). The
qualitative inquiry that, taken together, con- phenomenon of interest unfolds naturally in
stiitute a comprehensive and coherent stra- that it has no predetermined course estab-
tegic framework for qualitative inquiry, in- lished by and for the researcher such as
cluding fundamental assumptions and would occur in a labor ator y or other con-
epistemological ideais. Exhibit 2.1 summa- trolled setting. Observations take place in
rizes those themes in three basic categories: real-world settings and people are inter-
design strategies, data collection and field- viewed with open-ended questions in places
work strategies, and analysis strategies. and under conditions that are comfortable
for and familiar to them.
Egon Guba (1978), in his classic treatise on
Design Strategies for naturalistic inquiry, identified two dimen-
Qualitative Inquiry sions along which types of scientific inquiry
can be described: (1) the extent to which the
Naturalistic Inquiry scientist manipulates some phenomenon in
advance in order to study it and (2) the ex-
An anthropologist studies initiation rites tent to which constraints are placed on out-
among the Gourma people of Burkina Faso puts, that is, the extent to which predeter-
in West frica. A sociologist observes inter- mined categories or variables are used to
actions among bowlers in their weekly describe the phenomenon under study. He
league games. An evaluator participates then defined "naturalistic inquiry" as a
fully in a leadership training program she is "discovery-oriented" approach that mini-
documenting. A naturalist studies bighorn mizes investigator manipulation of the
sheep beneath Powell Plateau in the Grand study setting and places no prior constraints
Canyon. A policy analyst interviews people on what the outcomes of the research will be.
living in public housing in their homes. An Naturalistic inquiry contrasts with con-
agronomist observes farmers' spring plant- trolled experimental designs where, ideally,
ing practices in rural Minnesota. What do the investigator controls study conditions
40 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Themes of Qualitative Inquiry

Desiqn Strategies
1. Naturaistic inquiry Studying real-world situations as they unfold naturalfy;
nonmanipulative and noncontrolling; openness to whatever
emerges (lack of predetermined constraints on findings).
2. Emergent design Openness to adapting inquiry as understanding deepens and/or
flexibility situations change; the researcher avoids getting locked into rigid
designs that eliminate responsiveness and pursues new paths of
discovery as they emerge.
3. Purposeful sampling Cases for study (e.g., people, organizations, communities,
cultures, events, criticai incidences) are selected because
they are "information rich" and illuminative, that is, they offer
useful manifestations of the phenomenon of interest; sampling,
then, is aimed at insight about the phenomenon, not empirical
generalization from a sample to a popuation.

Data Coilection and Fieldwork Strategies

4. Qualitative data Observations that yield detailed, thick description; inquiry in
depth; interviews that capture direct quotations about people's
personal perspectives and experiences; case studies; careful
document review.
5. Personal experience The researcher has direct contact with and gets close to the
and engagement people, situation, and phenomenon under study; the researcher's
personal experiences and insights are an important part of
the inquiry and criticai to understanding the phenomenon.
6. Empathic neutrality An empathic stance in interviewing seeks vicarious under-
and mindfulness standing without judgment (neutrality) by showing openness,
sensitivity, respect, awareness, and responsiveness; in
observation it means being fully present (mindfulness).
7. Dynamic systems Attention to process; assumes change as ongoing whether
focus is on an individual, an organization, a community, or an
entire culture; therefore, mindful of and attentive to system
and situation dynamics.

by manipulating, changing, or holding con- ence between asking, "Tell me about your
stant externai influences and where a very experience in the program" and "How satis-
limited set of outcome variables is mea- fied were you? Very, somewhat, little, not at
sured. Open- ended, conversation-like inter- ali."
views as a form of naturalistic inquiry con- In the simplest form of controlled experi-
trast with questionnaires that have prede- mental inquiry the researcher enters the
termined response categories. It's the differ- program at two points in time, pretest and
Strategic Themes in Qualitative Inquiri/ J. 41

Anaiysis Strateqies
8. Unique case Assumes each case is special and unique; the first levei of
orientation analysis is being true to, respecting, and capturing the detas
of the individual cases being studied; cross-case analysis
follows from and depends on the quality of individual case
studies.
9. nductive analysis Immersion in the details and specifics of the data to discover
and creative synthesis important patterns, themes, and interrelationships; begins by
exploring, then confirming; guided by analytical principies
rather than ruies; ends with a creative synthesis.
10. Holistic perspective The whole phenomenon under study is understood as a complex
system that is more than the sum of its parts; focus on complex
interdependencies and system dynamics that cannot meaning-
fully be reduced to a few discrete variables and linear, cause-
effect relationships.
11. Context sensitivity Places findings in a social, historical, and temporal context;
careful about, even dubous of, the possibility or meaningful-
ness of generalizations across time and space; emphasizes
instead careful comparative case anayses and extrapolating
patterns for possible transferability and adaptation in new
settings.
12. Voice, perspective, The qualitative analyst owns and is reflective about her or his
and reflexivity own voice and perspective; a credible voice conveys authentic-
ity and trustworthiness; complete objectivity being impossible
and pure subjectivty undermtning crediblity, the researcher's
focus becomes balanceunderstanding and depicting the world
authenticaliy in ali its complexity while being self-analytical,
politically aware, and reflexive in consciousness.

posttest, and compares the treatment group treatment remains relatively constant and
to some control group on a limited set of unchanging.
standardized measures. Such designs as- While there are some narrow, carefully
sume a single, identifiable, isolated, and controlled, and standardized treatments
measurable treatment. Moreover, such de- that fit this description, in practice human
signs assume that, once introduced, the interventions (programs) are often quite
comprehensive, variable, and dynamic
42 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

changing as practitioners learn what does ments can involve comparing two groups,
and does not work, developing new ap- one of which experiences some change
proaches and realigning priorities. This, of while the other doesn't. What makes this
course, creates considerable difficulty for naturalistic inquiry is that real-world partic-
controlled experimental designs that need ipants direct the change, not the researcher,
specifiable, unchanging treatments to relate as in the laboratory.
to specifiable, predetermined outcomes. However, the distinction is not as simple
Controlled experimental evaluation designs as being in the field versus being in the labo-
work best when it is possible to limit pro- ratory; rather, the degree to which a design is
gram adaptation and improvement so as not naturalistic falls along a continuum with
to interfere with the rigor of the research de- completely open fieldwork on one end and
sign. completely controlled laboratory control on
By contrast, under real-world conditions the other end, but with varying degrees of
where programs are subject to change and researcher control and manipulation be-
redirection, naturalistic inquiry replaces the tween these end points. For example, the
fixed treatment/outcome emphasis of the very presence of the researcher, asking ques-
controlled experiment with a dynamic, pro- tions, or as in the case of formative program
cess orientation that documents actual oper- evaluation, providing feedback, can be an
ations and impacts of a process, program, or intervention that reduces the natural un-
intervention over a period of time. The eval- folding of events. Unobtrusive observations
uator sets out to understand and document are needed as an inquiry strategy when the
the day-to-day reality of participants in the inquirer wants to minimize data collection
program, making no attempt to manipulate, as an intervention. Nor are laboratory condi-
control, or eliminate situational variables or tions found only in buildings. Field experi-
program developments, but accepting the ments are common in agriculture where re-
complexity of a changing program reality. searchers want to introduce a considerable
The data of the evaluation include whatever amount of control, reduce variation in extra-
emerges as important to understanding neous variables, and focus on a limited set of
what participants experience. predetermined measures, as in crop fertil-
Natural experiments occur when the ob- izer studies.
server is present during a real-world change Let me conclude this discussion of natu-
to document a phenomenon before and after ralistic inquiry with two examples to illus-
the change. Durrenberger and Erem (1999) trate variations in this design strategy. In
documented "a natural experiment in evaluating a wilderness-based leadership
thought and structure" when, because of a training program, I participated fully in the
change at a hospital they were studying, 10-day wilderness experience, guided in my
they were able to contrast two different observations by nothing more than the sen-
structures of leadership in a union worksite. sitizing concept "leadership." The only "un-
They had already documented the degree natural" elements of my participation were
and nature of "union consciousness" before that (1) everyone knew I was taking notes to
the change, so by repeating their observa- document what happened and (2) at the end
tions after the change in a hospital structure, of each day I conducted open-ended, con-
they were able to. take advantage of a natu- versational interviews with staff. While this
rally occurring experiment. Natural experi- constitutes a relatively pure naturalistic in-
Strategic Themes in Qualitative Inquiri/ J. 4 3

"4<1d

quiry strategy, my presence, note taking, tivities. For 736 consecutive nights, Strass-
and interviews must be presumed to have mann kept track of ali the women who used
altered somewhat the way the program un- the hut. This allowed her to collect statis-
folded. I know, for example, that the debrief- tics on the frequency and length of menstru-
ing questions I asked staff in the evenings ation among the Dogon women, but with a
got them thinking about things they were completely naturalistic inquiry strategy, il-
doing that led to some changes along the lustrating how both quantitative and quali-
way in how they conducted the training. tative data can be collected within a natural-
The second example comes from the istic design strategy. There's no reason to
fieldwork of Beverly Strassmann among the believe that her presence over this long pe-
Dogon people in the village of Sangui in the riod changed the women's menstruation
Sahel, about 120 miles south of Tombouctou patterns.
in Mali, West frica (Gladwell 2000). Her
study focused on the Dogon tradition of
Emergent Design Flexibility
having menstruating women stay in small,
segregated adobe huts at the edge of the vil-
lage. She observed the comings and goings In the wilderness leadership training pro-
of these women and obtained urine samples gram I evaluated, halfway through the
from them to be sure they were menstruat- 10-day experience the group I was with un-
ing. The women only slept in the huts. Dur- expectedly split into two subgroups. I had to
ing the day, they went about their normal ac- make an in-the-field, on-the-spot decision
44 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

about which group to follow and how to get search. How will they know what will result
interviews with the others at a later time. from the inquiry if the design is only par-
Naturalistic inquiry designs cannot usu- tially specified? The answer is: They won't
ally be completely specified in advance of know with any certainty. Ali they can do is
fieldwork. While the design will specify an look at the results of similar qualitative m-
initial focus, plans for observations, and ini- quiries, inspect the reasonableness of the
tial guiding interview questions, the natu- overall strategies in the proposed design,
ralistic and inductive nature of the inquiry and consider the capacity of the researcher
makes it both impossible and inappropriate to fruitfully undertake the proposed study.
to specify operational variables, state test- As with other strategic themes of qualita-
able hypotheses, or finalize either instru- tive inquiry, the extent to which the design is
mentation or sampling schemes. A natural- specified in advance is a matter of degree.
istic design unfolds or emerges as fieldwork Doctoral students doing qualitative disser-
unfolds. ta tions will usually be expected to present
Lincoln and Guba (1985) made an exten- fairly detailed fieldwork proposals and in-
sive comparison of the design characteristics terview schedules so that the approvmg
of qualitative/naturalistic inquiry in con- doctoral committee can guide the student
trast to quantitative/experimental methods. and be sure that the proposed work will lead
They concluded: to satisfying degree requirements. Many
funders will fund only detailed proposals.
What these considera tions add up to is that the As an ideal, however, the qualitative re-
design of a naturalistic inquiry (whether researcher needs considerable flexibility and
search, evaluation, or policy analysis) cannot openness. The fieldwork approach of an-
be given in advance; it must emerge, develop, thropologist Brackette F. Williams repre-
unfold The call for an emergent design by sents the ideal of emergence in naturalistic
naturalists is not simply an effort on their part mquiry.
to get around the "hard thinking" that is sup- Williams has focused on issues of cultural
posed to precede an inquiry; the desire to per- identity and social relationships. Her work
mit events to unfold is not merely a way of has mcluded in-depth study of ritual and
rationalizing what is at bottom "sloppy in- symbolism in the construction of national
quiry/' The design specifications of the con- identity in Guyana (1991), and the ways that
ventional paradigm form a procrustean bed of race and class function in the national con-
such a nature as to make it impossible for the sciousness of the United States. In 1997, she
naturalist to lie in itnot only uncomfortably, received a five-year MacArthur Fellowship,
but at ali (p. 225) which has allowed her to pursue a truly
emergent, naturalistic design in her current
Design flexibility stems from the open- fieldwork on the phenomenon of killing in
ended nature of naturalistic inquiry as well America. I had the opportunity to interview
as pragmatic considerations. Being open her about her work and am including sev-
and pragmatic requires a high tolerance for eral excerpts from that interview1 through-
ambiguity and uncertainty as well as trust in out this chapter to illustrate actual scholarly
the ultimate value of what inductive analy- implementa tion of some of the strategic ide-
sis will yield. Such tolerance, openness, and ais of qualitative inquiry. Here she describes
trust create special problems for dissertation the necessity of an open-ended approach to
committees and funders of evaluation or re- her fieldwork because her topic is broad and
Strategic Themes in Qualitative Inquiri/ J. 4 5

she needs to follow wherever the phenome- col of questions and issues to pursue. It was
non takes her. general sampling to get a sense of what I
wanted to know. At other times, it's just to get
I'm tracking somethingkillingthat's mov- a general opinion from Jolin Q. Public about a
ing very rapidly in the culture. Every time I question that I've gotten ali kinds of official re-
talk to someone, there's another set of data, an- sponses to, but I want to know what people in
other thing to look at. Anything that happens general think. In an airport, I may get an op-
in America can be relevant, and thafs the ex- portunity to talk to 5 or 10 people. If I have sev-
hausting part of it. It never shuts off. You listen eral stops, I may get 15 or 20 by the time I come
to the radio. You watch television. You pass a home.
billboard with an advertisement on it. There's I fashion the research as I want to fashion it
no such thing as something irrelevant when based on what I think this week as opposed to
you're studying something like this or maybe what I thought Iast week. I don't follow some
just studying the society that you're in. You proposal. I don't have in mind that this has to
don't always kno w exactly how it's going to be be a book that's going to have to come out a
relevant, but somehow it just strikes you and certain way. I'm following where the data take
you say to yourself: I should document the me, where my questions take me.
date of when I saw this and where it was and
what was said because it's data. Few qualitative studies are as fully emergent
I don't follow every possible lead people and open-ended as the fieldwork of Wil-
give me. But generally, it is a matter in some liams. Her work exemplifies the ideal of
sense of opportunity sampling, of serendipity, emergent design flexibility.
whatever you want to call it, I key into things
that turn out to be very important six months Purposeful Sampling
la ter.
I do impromptu interviews. I don't have In 1940, eminent sociologist Kingsley
some target number of interviews in mind or Da vis published what was to become a clas-
predetermined questions. It depends on the sic case study, the story of Anna, a baby kept
person and the situation. Airports, for exam- in nearly total isolation from the time of her
ple, are a good place for impromptu inter- birth until she was discovered at age six. She
views with people. So some times, instead of had been deprived of human contact, had
using airport time to write, I interview people acquired no language skills, and had re-
about the death penalty or about killing or ceived only enough care to keep her barely
about death in their life. It's called opportunity alive. This single case, horrifying as was the
sampling, I begin with a general description. abuse and neglect, offered a natural experi-
You're such and such an age. You come from ment to study socialization effects and the
such and such a place and, by the way, what do relative contributions of nature and nurture
you think about ali this killing? And I sort of to human development. In 1947, Davis pub-
launch into a conversation. Sometimes the in- lished an update on Anna and a comparison
terview goes on for a couple of hours and case of socialization isolation, the story of
sometimes, maybe 10 or 15 minutes. I just say, Isabelle. These two cases offered consider-
"You wouldn't mind if I record this, would able insight into the question of how long a
you?" If they say no, I take notes. human being could remain isolated before
I did a lot of that kind of impromptu inter- "the capacity for full cultural acquisition"
viewing m the first year to formulate a proto- was permanently damaged (Davis 1940,
46 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

1947). The cases of Anna and Isabelle are ex- which one can learn a great deal about issues
treme examples of purposeful case sam- of central importance to the purpose of the
pling. research, thus the term purposeful sampling.
Unusual clinicai cases in medicine and For example, if the purpose of an evaluation
psychology, instructive precisely because is to increase the effectiveness of a program
they are unusual, offer many examples of in reaching lower-socioeconomic groups,
purposeful sampling. Neurologist Oliver one may leam a great deal more by focusing
Sacks (1985) presents a number of such cases in depth on understanding the needs, inter-
in his widely read and influential book The ests, and incentives of a small number of
Man Who Mistook His Wifefor a Hat, the very carefully selected poor families than by
title of which hints at the uniqueness of the gathering standardized information from a
cases examined. While one cannot general- large, statistically significant sample. The
ize from single cases or very small samples, cases sampled can be individual people,
one can leam from themand learn a great families, organizations, cultures, incidents,
deal, often opening up new territory for fur- or activities, to mention examples. But re-
ther research, as was the case with Piagefs gardless of the kind of unit of analysis (e.g.,
detailed and insightful observations of his an athlete or a sports team, a teacher or a
own two children. classroom), the purpose of purposeful sam-
Perhaps nowhere is the difference be- pling is to select information-rich cases
tween quantitative and qualitative methods whose study will illuminate the questions
better captured than in the different strate- under study.
gies, logics, and purposes that distinguish Chapter 5 will review several different
statistical probability sampling from quali- strategies for purposefully selecting infor-
tative purposeful sampling. Qualitative in- mation-rich cases. In my interview with her,
quiry typically focuses on relatively small Brackette F. Williams offered an example of
samples, even single cases (N = 1) such as an information-rich case from her ongoing
Anna or Isabelle, selected purposefully to per- study of killing in America.
mit inquiry into and understanding of a phe-
nomenon in depth. Quantitative methods I've been tracking information on a serial
typically depend on larger samples selected killersomeone who has just been identified
randomly in order to generalize with confi- as a "serial killer" in Louisianawho's killing
dence from the sample to the population young Black men, shooting them up with
that it represents. Not only are the tech- drugs and taking one of their tennis shoes,
niques for sample selection different, but the sometimes both. Now, I'm interested in the
very logic of each approach is distinct be- fact that as society more and more identifies
cause the purpose of each strategy is differ- young Black men as sort of the quintessential
ent. bad guys, this serial killer picks a bad guy. For
The logic and power of probability sam- contrast, look at serial killers who picked
pling derive from its purpose: generaliza- women at a certain period of time, about 15-20
tion. The logic and power of purposeful years ago, because they wore, in his estima-
sampling derive from the emphasis on tion, a size 13. Now, track our obsession with
in-depth understanding. This leads to se- obesity. How a serial killer picks his victims
lecting information-rich cases for study in can tell you something important about
depth. Information-rich cases are those from what's going on in society.
Strategic Themes in Qualitative Inquiri/ J. 4 7

3. Data Collection and I was down in Texas interviewing last March,

Fieldwork: Strategies thinking about my research and interviewing
for Qualitative Inquiry people, and there was a childhood memory
that I had of an electrocution of a man that was
the son of a woman who lived across the field
Qualitative Data from us. Now a rumor about this had always
been in the back of my mind. Whenever I'd

Qualitative data consist of quotations, ob- hear about a death penalty case over the years,

servations, and excerpts from documents. I would thihk about this man having been

The first chapter provided several examples electrocuted. I thought he was electrocuted be-

of qualitative data. Deciding whether to use cause he raped this White woman. So I'm sit-

naturalistic inquiry or an experimental ap- ting in my cousin's kitchen after I had done

proach is a design issue. This is different some of these interviews and another woman,

from deciding what kind of data to collect an older woman who was a relative of hers,

(qualitative, quantitative, or some combina- came in and the conversation goes around. I

tion), although design and data altematives happen to mention this memory of mine. I

are clearly related. Qualitative data can be asked, "Is that just something that I concocted

collected in experimental designs where out of having read a book or something, but it

participants have been randomly divided never happened?" She answered, "Oh, no, it

into treatment and control groups. Likewise, happened. You only have one part of the story

some quantitative data may be collected in wrong. He didn't rape her. He looked at her."

naturalistic inquiry approaches. Neverthe- You know, you read about these things in
less, controlled experimental designs pre- history books and then ali of a sudden, it's Rke
dominantly aim for statistical analyses of a part of a world that you existed in. These
quantitative data, while qualitative data are things happened around you and yet some-
the primary focus in naturalistic inquiry how there was so much of a distance, you
This relationship between design and mea- couldn't touch it. I knew about this man ali my
surement will be explored at greater length life, but in ali the reading and ali the history
in the chapter on design. books, I couldn't touch that. Doing this project

Qualitative data describe. They take us, the way I'm doing it allozvs me to touch things that
as readers, into the time and place of the ob- otherwise I would never touch.
serva tion so that we know what it was like to
have been there. They capture and commu-
Direct Personal Experience and
nicate someone else's experience of the
Engagement: Going Into the Field
world in his or her own words. Qualitative
data tell a story. In the excerpt below, from
my interview with her, Williams tells the The preceding quotation from Williams
story of checking out a childhood memory. exemplifies the personal nature of qualita-
This story gives us insight into the nature of tive fieldwork. Getting close to her subject
her naturalistic inquiry and open-ended in- matter, including using her own experi-
terviewing, shows how a criticai incident ences, both from childhood and day-to-day
can be a purposeful sample, and, in the story in her adult life, illustrates the all-encom-
itself, offers something of the flavor of quali- passing and ultimately personal nature of
tative data. in-depth qualitative inquiry. Traditionally,
48 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

social scientists have been warned to stay other people, assume that in order to know or
distant from those they studied to maintain understand others one is well-advised to give
"objectivity." But that kind of detachment some conscious attention to that effort in
can limit one's openness to and understand- face-to-face contacts. They assume, too, that
ing of the very nature of what one is study- the internai world of sociologyor any other
ing, especially where meaning-making and social worldis not understandable unless
emotion are part of the phenomenon. Look one has been part of it in a face-to- face fashion
closely at what Williams says about the ef- for quite a period of time. How utterly para-
fects of immersing herself personally in her doxical, then, for these same persons to tum
fieldwork, even while visiting relatives: around and make, by implication, precisely
"Doing this project the way Tm doing it al- the opposite claim about people they have
lows me to touch things that otherwise I never encountered face-to-facethose people
would never touch." appearing as numbers in their tables and as
Fieldwork is the central activity of quali- correlations in their matrices! (Lofland 1971:3)
tative inquiry. "Going into the field" means
having direct and personal contact with peo- Qualitative inquiry means going into the
ple under study in their own environments fieldinto the real world of programs, orga-
getting close to the people and situa- nizations, neighborhoods, street corners
tions being studied to personally under- and getting close enough to the people and
stand the realities and minutiae of daily life, circumstances there to capture what is hap-
for example, life as experienced by partici- pening. To immerse oneself in naturally oc-
pants in a welfare-to-work program. The in- curring complexity involves what qualita-
quirer gets close to the people under study tive methodologist Norman Denzin (1978a)
through physical proximity for a period of has called "the studied commitinent to ac-
time as well as through development of tively enter the worlds of interacting in-
closeness in the social sense of shared expe- dividuais" (pp. 8-9). This makes possible
rience, empathy, and confidentiality. That description and understanding of both exter-
many quantitative methodologists fail to nally observable behaviors and internai
ground their findings in personal qualitative states (worldview, opinions, values, atti-
understanding poses what sociologist John tudes, and symbolic constructs). Given the
Lofland (1971) called a major contradiction qualitative emphasis on striving for depth of
between their public insistence on the ade- understanding, in context, attitude surveys
quacy of statistical portrayals of other hu- and psychological tests are inadequate for
mans and their personal everyday dealings revealing inner perspectives. "The inner
with and judgments about other human perspective assumes that understanding
beings. can only be achieved by actively participai-
ing in the life of the observed and gaining in-
In everyday life, statistical sociologists, like sight by means of introspection" (Bruyn
everyone else, assume that they do not know 1963:226).
or understand very well people they do not Actively participating in the life of the ob-
see or associate with very much. They assume served means going where the action is, get-
that knowing and understanding other people ting one's hands dirty, participating where
require that one see them reasonably often and possible in actual program activities, and
in a variety of situations relative to a variety of getting to know program staff and partici-
issues. Moreover, statistical sociologists, like pants on a personal leveiin other words,
Strategic Themes in Qualitative Inquiri/ J. 49

getting personally engaged so as to use ali of It is important to note that the admonition
one's senses and capacities, including the ca- to get close to the data is in no way meant to
pacity to experience affect no less than cog- deny the usefulness of quantitative meth-
nition. Such engagement stands in sharp ods. Rather, it means that statistical portray-
contrast to the professional comportment of als must always be interpreted and given
some in the field, for example, supposedly human meaning. I once interviewed an eval-
objective evaluators, who purposely project uator of federal health programs who ex-
an image of being cool, calm, externai, pressed frustration at trying to make sense
and de tache d. Such detachment is pre- out of statistical data from over 80 projects
sumed to reduce bias. However, qualitative after site visit funds had been cut out of the
methodologists question the necessity and evaluation: "There's no way to evaluate
utility of distance and detachment, asserting something thafs just data. You know, you
that without empathy and sympathetic in- have to go look."
trospection derive d from personal encoun- Going into the field and having personal
ters, the observer cannot fully understand contact with program participants is not the
human behavior. Understanding comes only legitimate way to understand human
from trying to put oneself in the other per- behavior. For certain questions and for situa-
son's shoes, from trying to discern how oth- tions involving large groups, distance is in-
ers think, act, and feel. evitable, perhaps even helpful, but to get at
In a classic study, educational evaluator deeper meanings and preserve context,
Edna Shapiro (1973) studied young chil- face-to-face interaction is both necessary
dren in classrooms in the national Follow and desirable. This returns us to a recurrent
Through program using both quantitative theme of this book: matching research meth-
and qualitative methods. It was her close- ods to the purpose of a study, the questions
ness to the children in those classrooms that being asked, and the resources available.
allowed her to see that something was hap- In thinking about the issue of closeness to
pening that was not captured by standard- the people and situations being studied, it is
ized tests. She could see differences in chil- useful to remember that many major contri-
dren, observe their responses to diverse butions to our understanding of the world
situations, and capture the varying mean- have come from scientists' personal experi-
ings they attached to common events. She ences. One finds many instances where close-
could feel their tension in the testing situa- ness to sources of data made key insights
tion and their spontaneity in the more natu- possible Piagefs closeness to his children,
ral classroom setting. Had she worked solely Freud's proximity to and empathy with his
with data collected by others or only at a dis- patients, Darwin's closeness to nature, and
tance, she would never have discovered the even Newton's intimate encounter with an
crucial differences in the classroom settings apple. In short, closeness does not make bias
she studieddifferences that actually al- and loss of perspective inevitable; distance
lowed her to evaluate the innovative pro- is no guarantee of objectivity.
gram in a meaningful and relevant way.
Where standardized tests showed no differ- Empathic Neutrality
ences between classrooms using different
approaches, her direct observations docu- If, as the previous section has discussed,
mented important and significant program naturalistic inquiry involves fieldwork that
impacts. puts one in close contact with people and
50 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

their problems, what is to be the researcher's of a supposedly value-free social science,

cognitive and emotional stance toward subjectivity is the very antithesis of scientific
those people and problems? No universal inquiry.
prescription can capture the range of possi- Objectivity has been considered the
bilities, for the answer will depend on the strength of the scientific method. The pri-
situation, the nature of the inquiry, and the mary methods for achieving objectivity in
perspective of the researcher. But thinking science have been conducting blind experi-
strategically, I offer the phrase "empathic ments and quantification. "Objective tests"
neutrality" as a point of departure. It sug- gather data through instruments that, in
gests that there is a middle ground between principie, are not dependent on human skill,
becoming too involved, which can cloud perception, or even presence. Yet, it is clear
judgment, and remaining too distant, which that tests and questionnaires are designed
can reduce understanding. What is em- by human beings and therefore are subject to
pathic neutrality? Consider this anecdote by the intrusion of the researcher's biases by
way of illustration. the very questions asked. Unconscious bias
Pragmatst philosopher William James, in the skillful manipula tion of statistics to
also a scholar of anatomy and psychology, prove a hypothesis in which the researcher
had a great capacity for empathy, as dis- believes is hardly absent from hypothetical-
played in his classic study The Varieties o/Re- deductive inquiry.
ligious Experience ([1902] 1999). Editor Clif- Part of the difficulty in thinking about the
ton Fadiman (1985:305) recounts that while fieldwork stance of the qualitative mquirer
he was teaching at Radcliffe, Gertrude Stein is that the terms objectivity and subjectivity
took a course from him in which, having at- have become so loaded with negative con-
tended the opera and then partied into the notations and subject to acrimonious debate
wee hours the night before an exam, she (e.g., Scriven 1972a; Borman and Goetz 1986;
wrote, "Dear Professor James, I am so sorry Krenz and Sax 1986; Guba 1991) that neither
but I do not feel a bit like writing an exami- term any longer provides useful guidance.
nation paper today." James is said to have These terms have been politicized beyond
written back: "Dear Miss Stein, I understand utility. To claim the mantle of "objectivity" in
perfectly. I often feel like that myself." Had the postmodern age is to expose oneself as
he added, but the exam is still due, instead of embarrassingly naive. The ideais of absolute
ordinary sympathy he would have dis- objectivity and value-free science are impos-
played extraordinary empathic neutrality. sible to attam in practice and are of question-
Methodologists and philosophers of sci- able desirability in the first place since they
ence debate what the researcher's stance ignore the intrinsically social nature and hu-
should be vis--vis the people being studied. man purposes of research. On the other
Critics of qualitative inquiry have charged hand, subjectivity has such negative connota-
that the approach is too subjective, in large tions in the public mind that to admit being
part because the researcher is the instrument subjective may undermine one's credibility
of both data collection and data interpreta- with audiences unsophisticated about
tion and because a qualitative strategy in- phenomenological assumptions and nu-
cludes having personal contact with and ances. In short, the terms objectivity and sub-
getting close to the people and situation un- jectivity have become ideological ammuni-
der study. From the perspective of advocates tion in the methodological paradigms
Strategic Themes in Qualitative Inquiri/ J. 51

debate. My pragmatic solution is to avoid plexities and multiple perspectives as they

using either word and to stay out of futile de- emerge, and be balanced in reporting both
bates about subjectivity versus objectivity. confirmatory and disconfirming evidence
Qualitative research in recent years has with regard to any conclusions offered.
moved toward preferring such language as Neutrality is not an easily attainable
trustworthiness and authenticity. Evaluators stance, so ali credible research strategies in-
aim for "balance/' "fairness," and "com- clude techniques for helping the investiga-
pleteness" (Patton 1997a:282). Chapter 9 will tor become aware of and deal with selective
discuss these terms and the stances they im- perception, personal biases, and theoretical
ply at greater length. At this point, I simply predispositions. Qualitative inquiry, be-
want to note the strategic nature of the issue cause the human being is the instrument of
of inquirer stance and add empathic neutral- data collection, requires that the investigator
ity to the emerging lexicon that attempts to carefully reflect on, deal with, and report po-
supersede the hot button term objective and tential sources of bias and error. Systematic
the epithet subjective. data collection procedures, rigorous train-
Any research strategy ultimately needs ing, multiple data sources, triangulation, ex-
credibility to be useful. No credible research ternai reviews, and other techniques to be
strategy advocates biased distortion of data discussed in this book are aimed at produc-
to serve the researcher's vested interests and ing high-quality qualitative data that are
prejudices. Both qualitative/naturalistic incredible, trustworthy, authentic, balanced
quiry and quantitative/experimental in- about the phenomenon under study, and
quiry seek honest, meaningful, credible, and fair to the people studied.
empirically supported findings. Any credi- The livelihood of evaluators and re-
ble research strategy requires that the inves- searchers depends on their integrity and
tigator adopt a stance of neutrality with re- credibility. Independence and neutrality,
gard to the phenomenon under study. This then, are serious issues.
simply means that the investigator does not However, neutrality does not mean de-
set out to prove a particular perspective or tachment. It is on this point that qualitative
manipulate the data to arrive at predisposed inquiry makes a special contribution. Quali-
truths. Theneutral investigator enters the re- tative inquiry depends on, uses, and en-
search arena with no ax to grind, no theory hances the researcher's direct experiences in
to prove (to test but not to prove), and no the world and insights about those ex-
predetermined results to support. Rather, periences. This includes learning through
the investigator's commitment is to under- empathy.
stand the world as it unfolds, be true to com-

Empathy and Insight

T he idea of acquiring an "inside" understandingthe actors' definitions

of the situationis a powerful central concept for understanding the
purpose of qualitative inquiry.

Thomas A. Schwandt (2000:102)

52 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Empathy develops from personal contact

with the people interviewed and observed
during fieldwork. Empathy involves being
able to take and understand the stance, posi- iVi i: to 'n"i'i;;i:;! SFV ::.:;: iid * > u-, kx .VSH'!:';

tion, feelings, experiences, and worldview .ji.;j r&i |;!"!':!i! i! ir.V:! r-h-, V . 'iVn r/iu!
of others. Put metaphorically, empathy is ! ! , !':" ! ' :!! i : -iii' I '! !; ! ; I;'I ! !:!!I;".' ! i.:J" n n l V i ! i'-
"like being able to imagine a life for a spider, A ; \ V J ! ; TF! R. Y:'.-; I V A I : ! -.-JYV. (> RI-N <".'

a maker's life, or just some aliveness in its ':i,i;!!:M;:!'\!!!n:,!:Mi!i:v. irjsYn!;! k\ mlv i ^-i-'
wide abdmen and delicate spinnerets so & !i, >Vi k : T . ??;!!! K I - V i A M - N h : V " I i i W -
!

you take it outside in two paper cups instead W - .Mv hi! : I: H;. OA r' :',!

of stepping on it" (Dunn 2000:62). Empathy 1 P J ! ' I ! " i " !L|: RT-D* L . ' ! . I : i " ! . J ' I T ! ':: ! R I,:;.

combines cognitive understanding with af- :.!'!:' V i V i !.';!;! J I T F ! IV; ! I L I ' ! M' ri 1 !.!. W , NNHN ri

fective connection, and in that sense differs WJS V!:'RVSN1;::NYJ.r, IIIXHRIY''!" W M ! !|

from sympathy, which is primarily emo- ; : ; I R ; 1 ; H ' " ; ! . TH-H V/!';:! F:-I" . T T i.VI W : , I H T !I I;'

tional (Wisp 1986). Ki' TIJ. II.I,\<:!:'i:I a.*:- 1'!!-:' D -A V I.-RI-I*-, !::M;IJ: N 1 ; P;

The value of empathy is emphasized in .! : 'i i 'i '.] i' j!i-ii FRI"1 ! ! !:.'' M S ! I . V ^ ' . , Y Y v ]i ri.;R N ; v i : ! !'

the phenomenological doctrine of Verstehen . V i HL' i V i {! :i i'i.' I! Y L " ' J i.i > J I."I .! Y r M V I ! rvri,

that undergirds much qualitative inquiry. *T i i i ! I ' H \ iii R':I::.I" IV!T; RRN-J.''

Verstehen means "understanding" and refers

to the unique human capacity to make sense
of the world. This capacity has profound im- ences need methods different from those
plications for how one studies human be- used in agricultural experimentation and
ings. The Verstehen doctrine presumes that physical sciences because human beings are
since human beings have a unique type of different from plants and nuclear particles.
consciousness, as distinct from other forms The Verstehen tradition stresses understand-
of life, the study of humanbeings willbe dif- ing that focuses on the meaning of human
ferent from the study of other forms of life behavior, the context of social interaction, an
and nonhuman phenomena. The capacity empathic understanding based on personal
for empathy, then, is one of the major assets experience, and the connections between
available for human inquiry into human af- mental states and behavior. The tradition of
fairs. Verstehen places emphasis on the human ca-
The Verstehen premise asserts that human pacity to know and understand others
beings can and must be understood in a through empathic introspection and reflec-
manner different from other objects of study tion based on direct observation of and in-
because humans have purposes and emo- teraction with people. "Verstehen thus en-
tions; they make plans, construct cultures, tails a kind of empathic identification with
and hold values that affect behavior. Their the actor. It is an act of psychological
feelings and behaviors are influenced by reenactment getting inside the head of an
consciousness, deliberation, and the capac- actor to understand what he or she is up to in
ity to think about the future. Human beings terins of motives, beliefs, desires, thoughts,
live in a world that has special meaning to and so on" (Schwandt 2000:192).
them, and because their behavior has mean- Max Weber brought the term empathy into
ing, "human actions are intelligible in ways social science to emphasize the importance
that the behavior of nonhuman objects is of comprehending the motives and feelings
not" (Strike 1972:28). Human and social sci- of people in a social-cultural context.
Strategic Themes in Qualitative Inquiri/ J. 53

Both Verstehen and empathy depend largely deeply and attentively so as to enter into the
on qualitative data. Verstehen is an attempt to other person's experience and perception.
"crack the code" of the culture, that is, detect
the categories into which a culture codes ac- I do not select, interpret, advise, or d i r e c t . . . .
tions and thoughts Empathy in evaluation Being-In the world of the other is a way of go-
is the detection of emotions manifested in the ing wide open, entering n as if for the first
program participants and staff, achieved by time, hearing just what is, leaving out my own
evaluators' becoming aware of similar or com- thoughts, feelings, theories, b i a s e s . . . . I enter
plementary emotions in themselves. (Meyers with the intention of understanding and ac-
1981:180) cepting perceptions and not presenting my
own view or r e a c t i o n s . . . . I only want to en-
A qualitative strategy of inquiry proposes courage and support the other person's ex-
an active, involved role for the social scien- pression, what and how it is, how it came to be,
tist. "Hence, insight may be regarded as the and where itis going. (Moustakas 1995:82-83)
core of social knowledge. It is arrived at by
being on the inside of the phenomena to be At first, the phrase "empathic neutrality"
observed It is participation in an activity may appear to be an oxymoron, combining
that generates interest, purpose, point of contradictory ideas. Empathy, however, de-
view, value, meaning, and intelligibility, as scribes a stance toward the people one en-
well as bias" (Wirth 1949:xxii). This is a quite countersit communicates understanding,
different scientific process from that envi- interest, and caring. Neutrality suggests a
sioned by the classical, experimental ap- stance toward their thoughts, emotions, and
proach to science, but it is still an empirical, behaviorsit means being nonjudgmental.
(i.e., data-based), scientific perspective. The Neutrality can actually facilitate rapport
qualitative perspective "in no way suggests and help build a relationship that supports
that the researcher lacks the ability to be sci- empathy by disciplining the researcher to be
entific while collecting the data. On the con- open to the other person and nonjudgmental
trary, it merely specifies that it is crucial for in that openness. Rapport and empathy,
validityand, consequently, for reliability however, must not be taken for granted, as
to try to picture the empirical social world Radhika Parameswaran (2001) found in do-
as it actually exists to those under investiga- ing fieldwork among young middle-class
tion, rather than as the researcher imagines it women in urban ndia who read Western ro-
to be" (Filstead 1970:4), thus the importance mance fiction.
of such qualitative approaches as partici-
pant observation, depth interviewing, de- Despite their eventual willingness to share
tailed description, and case studies. their fears and complaints about gendered so-
These qualitative inquiry methods pro- cial pressures, I still wonder whether these
vide opportunities to achieve empathy and young women would have been more open
give the researcher an empirical basis for about their sexuality with a Westerner who
describing the perspectives of others. Chap- might be seen as less likely to judge them
ter 1 cited the framework of humanis tic psy- based on cultural expectations of women's be-
chologist Clark Moustakas, who has de- havior in Indian society. The well-known
scribed this nonjudgmental empathic stance word rapport, which is often used to signify
as "Being-In" another's worldimmers- acceptance and warm relationships between
ing oneself in another's world by listening informants and researchers, was thus some-
54 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

thing I could not take for granted despite be- data interpretation and formulating recom-
ing an insider; ali I could claim was an mendations, but during fieldwork, the focus
imperfect rapport. (Parameswaran 2001:69) should be on rigorously observing and inter-
viewing to understand the people and situa-
Evaluation presents special challenges tion being studied. This nuanced relation-
for rapport and neutrality as well. After ship between neutrality and empathy will
fieldwork, an evaluator may be called on to be discussed further in both the data collec-
render judgments about a program as part of tion and analysis chapters.

A Dynamic, Developmental Perspective

"D here is nothing permanent except change.

Heraclitus (Ancient Greece)

A questionnaire is like a photograph. A in participants' experiences undermine the

qualitative study is like a documentary film. logic of an experimental design because
Both offer images. One, however the pho- these developmentsali natural, even inev-
tographcaptures and freezes a moment in itable, in real-world programscall into
time, like recording a respondenfs answer question what the "treatment" or experi-
to a survey question at a moment in time. ment actually is.
The otherthe film offers a fluid sense of Naturalistic inquiry assumes the ever-
development, movement, and change. changing world posited by the observation
Qualitative evaluation researchers, for in the ancient Chinese proverb that one
example, conceive of programs as dynamic never steps into the same river twice.
and developing, with "treatments" chang- Change is a natural, expected, and inevita-
ing in subtle but important ways as staff ble part of human experience, and docu-
learns what does and doesn't work, as cli- menting change is a natural, expected, and
ents move in and out, and as conditions of intrinsic part of fieldwork. Rather than try-
deli very are altered. A primary challenge, ing to control, limit, or direct change, natu-
then, becomes describing and understand- ralistic inquirers expect change, anticipate
ing these dynamic program processes and the likelihood of the unanticipated, and are
their holistic effects on participants so as to prepared to go with the flow of change. One
provide information for program improve- gets this sense of pursuing change in the
ment. In contrast, an experimental design comment by anthropologist Williams cited
for an evaluation typically conceives of the earlier: 'Tm tracking somethingkilling
program as a fixed thing, like a measured that's moving very rapidly in the culture."
amount of fertilizer applied to a cropa Part of her inquiry task is to track cultural
treatment, an interventionthat has prede- changes the way an epidemiologist tracks a
termined, measurable outcomes. Inconsis- disease. As a result, reading a good qualita-
tency in the treatment, instability in the tive case study gives the sense of reading a
intervention, changes in the program, vari- good story. It has a beginning, a middle, and
ability in program processes, and diversity an endingthough not necessarily an end.
Strategic Themes in Qualitative Inquiri/ J. 5 5

3. Analysis Strategies evaluation. That is what is meant by the

for Qualitative Inquiry "unique case orientation" of qualitative in-
quiry.
Uni que Case Orientation Case studies are particularly valuable in
program evaluation when the program is in-
"Six windows on respect" is how Har- dividualized, so the evaluation needs to be
vard sociologist Sara Lawrence-Lightfoot attentive to and capture individual differ-
(2000:13) describes the six detailed case ences among participants, diverse experi-
studies, each a full chapter, she presents in ences of the program, or unique variations
her book Respect. The cases offer different from one program setting to another. As
angles on the meaning and experience of re- noted earlier, a case can be a person, an
spect in inodern society as illuminated by a event, a program, an organization, a time
nurse-inidwife, a pediatrician, a teacher, an period, a criticai mcident, or a community.
artist, a law school professor, and a pastoral Regardless of the unit of analysis, a qual-
therapist/AIDS activist. Before drawing itative case study seeks to describe that unit
themes and contrasts froin this small, pur- in depth and detail, holistically, and in
poseful sample, and before naming the six context.
angles they represent, Lawrence-Lightfoot
had the task of constructing the unique cases
Inductive Analysis
to tell these distinct stories. Her first task,
and Creative Synthesis
then, was to undertake the "art and science
of portraiture" (Lawrence-Lightfoot and Benjamin Whorf's development of the fa-
Da vis 1997). From these separate portraits, mous Whorf hypothesisthat language
she fashions a stained glass mosaic that de- shapes our experience of the environment
picts and illuminates Respect and that words shape perceptions and ac-
I undertook a study of a national fellow- tions, a kind of linguistic relativity theory
ship award program that had had more than (Schultz 1991)provides an instructive ex-
600 recipients over a 20-year period. A sur- ample of inductive analysis. Whorf was an
vey had been done to get the fellows' opin- insurance investigator assigned to look into
ions about select issues, but the staff wanted explosions in warehouses. He discovered
more depth, richness, and detail to really un- that truck drivers were entering "empty"
derstand patterns of fellowship use and im- warehouses smoking cigarettes and cigars.
pact. With a team of researchers, we con- The warehouses, it turned out, often con-
ducted 40 in-depth, face-to-face interviews tained invisible, but highly flammable
and wrote case studies. Through inductive gases. He interviewed truckers and found
analysis, we subsequently identified dis- that they associated the word empty with
tinct enabling processes and impacts, and harmless and acted accordingly. From these
we created a frainework that depicted rela- specific observations and findings, he induc-
tionships between status at the time of the tively formulated his general theory about
award, enabling processes, and impacts. The language and perception that has informed
heart of the study remained the 40 case stud- a half-century of communications scholar-
ies. To read only the framework analysis ship (Lee 1996).
without reading the case studies would be to Qualitative inquiry is particularly ori-
lose much of the richness, depth, meaning, ented toward exploration, discovery, and in-
and contribution of qualitative research and ductive logic. Inductive analysisbegins with
56 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

ables are important and what relationships

among those variables can be expected.
^biSyjTTVi!!
AMPARA TIVE ANAL/fiiSi The strategy of inductive designs is to al-
low the important analysis dimensions to
vi :::;:::!!= :: I: :'I o emerge from pattems found m the cases un-
V: i 0 r ,;i lyj :.>. .V! "; (Vi. ; ; : der study without presupposing in advance
what the important dimensions will be. The
I.=::i:!-::.!; YI --s:---:i:s rr?-=:-=-.' r c :;' qualitative analyst seeks to understand the
multiple interrelationships among dimen-
i = n C^:;yy:!: :: !=-= :;='= ::=i.:: sions that emerge from the data without
:: m ; !--' n! i'..i -ni .;! i & ' !=.= making prior assumptions or specifying hy-
erO?J blh--n VI !! == :==!'.:! !"J :=: ! ! potheses about the linear or correlative rela-
fei iii-- !=!':=" i !=::! =:: >
: > tionships among narrowly defined, opera-
!"3W.i ri:.:; t u s e = = .lu tionalized variables. For example, an induc-
Hard Wai HHCTIS in !'=::prarr iihtuaitai tive approach to program evaluation means
rn.;;;: ".::.':! iihsi ^(v^ wt that understanding the nature of the "inter-
cra-rto ii i''i!ii=:' ili^ilsi!;'-!' !i:i. vention" emerges from direct observations
.-.1-rtl of program activities and interviews with
.V!IY! 1:iMM- ::::!IttlF!.:= !!!SY^I^ ,'i:!11 participants. In general, theories about what
! - m U ! : n V v ! ! ' c w m w . . - * a ;:! :
i!:;/ YM -::=Y!>= is happening in a setting are grounded in

Y' ^ I Y V I ! ' i Y i k ; ; -! R== i: i l V ^ i h - i f EIIYC-I and emerge from direct field experience
rather than being imposed a priori as is the
!V;:=i !. :;! li M iY ^ fi' .vi .! W.,"^ case in formal hypothesis and theory testing.
The straightforward contrast between
: t ti ti iji : ii [liiiliii closed-ended questionnaires and open-
=:IPC J I T A ^ S :. I:;:.;
! ; : V I Y ! Y G T F P T '

. : ! i [ L i : ! j ! Y ' i ! V i i=i=j.:J: t;i:i:r- r i / f iiitiiiiiiiiiiiiiiiiiiiiiiiiitiiiiiiiiii;

i ^ ^ ended interviews in Chapter 1 illustrated
the difference between deductive and in-
ductive approaches at the simplest levei. A
specific observations and builds toward structured, multiple-choice questionnaire
general pattems. Categories or dimensions requires a deductive approach because
of analysis emerge from open-ended obser- items mustbe predetermined based on some
vations as the inquirer comes to understand theory or preordinate criteria, for example,
pattems that exist in the phenomenon being program goals about what is important to
investigated. measure. An open-ended interview, by way
Inductive analysis contrasts with the hy- of contrast, permits the respondent to de-
pothetical-deductive approach of experi- scribe what is meaningful and salient with-
mental designs that require the specification out being pigeon hole into standardized cat-
of main variables and the statement of spe- egories.
cific research hypotheses before data collec- In practice, these approaches are often
tion begins. A specification of research hy- combined. Some evaluation or research
potheses based on an explicit theoretical questions may be determined deductively,
framework means that general constructs while others are left sufficiently open to
provide the framework for understanding permit inductive analyses based on direct
specific observations or cases. The investiga- observations. While the quantitative/exper-
tor must then decide in advance what vari- imental approach is largely hypothetical-
Strategic Themes in Qualitative Inquiri/ J. 5 7

Pigeon Holing

deductive and the qualitative/naturalistic separate cases. Once that is done, cross-case
approach is largely inductive, a study can in- analysis can begin in search of patterns and
clude elements of both strategies. Indeed, themes that cut across individual experi-
over a period of inquiry, an investigation ences. The initial focus is on full understand-
may flow from inductive approaches, to find ing of individual cases before those unique
out what the important questions and vari- cases are combined or aggregated themati-
ables are (exploratory work), to deductive cally. This helps ensure that emergent cate-
hypothesis-testing or outcome measure- gories and discovered patterns are grounded
ment aimed at confirming and/or generaliz- in specific cases and their contexts (Glaser
ing exploratory findings, then back again to and Strauss 1967).
inductive analysis to look for rival hypo- Just as writers report different creative
theses and unanticipated or unmeasured processes, so too qualitative analysts have
factors. different ways of working. Although soft-
The precise nature of inductive analysis ware programs now exist to facilitate work-
depends, in part, on the purpose of the anal- ing with large amounts of narrative data and
ysis and the number and types of cases in a substantial guidance can be offered about
study. Where there are several cases to be the steps and processes of content analysis,
compared and contrasted, an inductive ap- making sense of multiple interview tran-
proach begins by constructing individual scrip ts and pages of field notes cannot be re-
cases, without pigeon holing or categorizing duced to a formula or even a standard series
those cases. That is, the first task is to do a of steps. There is no equivalent of a statistical
careful job independently writing up the significance test or factor score to tell the an-
58 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

alyst when results are important or what that I should rewrite this part of this chapter. I
quotations fit together under the same had completely forgotten about this tape. It
theme. Finding a way to creatively synthesize was done in early '98 or late '97 and maybe I
and present findings is one of the challenges hadn't listened to it or looked at the transcript
of qualitative analysis, a challenge that will for a while, and I've just finished a chapter or
be explored at length in Part 3 of this book. section of a chapter. I pull that tape off the
For the moment, I can offer a flavor of that shelf. I listen to it. I go back to the transcript
challenge with another excerpt from my in- and I start writing again. I start revising in
terview with anthropologist Williams. Here ways that it seems to me that tape emands.
she describes part of her own unique ana-
lytic process. As Williams describes her analysis and
writing process, she offers insight into what
My current project folio ws up work that I have it means when qualitative researchers say
always done, which is to study categories and they are "working to be true to the data" or
classifications and their implications. Right that their analytical process is "data driven."
now, as I said, the focus of my work is on kill- Williams says, "I start revising in ways that it
ing and the desire to kill and the categories seems to me that tape demands." It is com-
people create in relation to killing. Part of it mon to hear qualitative analysts say that, as
right now focuses on the death penalty, but they write their conclusions, they keep go-
mainly on killing. My fascination is with the ing back to the cases; they reread field notes;
links between category distinctions, commit- and they listen again to interviews. Induc-
ments, and the desire to kill for those comm.it- tive analysis is built on a solid foundation
ments. That's what I study. of specific, concrete, and detailed observa-
I track categories, like "serial killers" or tions, quotations, documents, and cases.
"death row inmates." The business of con- As thematic structures and overarching
stantly transforming people into acts and acts constructs emerge during analysis, the
into people is part of the way loyaltes, com- qualitative analyst keeps returning to field-
mitments, and hatreds are generated. So I'm a work observations and interview tran-
classifier. I study classificationtheories of scripts, working from the bottom up, stay-
classification. A lot of categories have to do ng grounded in the foundation of case
with very abstract things; others have to do write-ups, and thereby examining emergent
with very concrete things like skin color. But themes and constructs in light of what they
ultimately, the classification of a kill is what illuminate about the case descriptions on
I'm focusing on now. I've been asking myself which they are based. That is inductive
lately, for the chapter I've been working on, "Is analysis.
there a fundamental difference, for example,
in the way we classify to kill?" Consider Holistic Perspective
thepercentage of people classified as "death
worthy"the way we classify to justify the Holography is a method of photography
death penalty. in which the wave field of light scattered by
As I write, moving back and forth between an object is captured as an interference pat-
my tapes and my interviews, I don't feel that I tern. When the photographic recordthe
have to follow some fixed outline or that I hologramis illuminated by a laser, a three-
have to code things to come out a certain way. dimensional image appears. Any piece of a
Sometimes I listen to a tape and I start to think hologram will reconstruct the entire image.
Strategic Themes in Qualitative Inquiri/ J. 59

This has become a metaphor for thinking in quires operationalization of independent

new ways about the relationships between and dependent variables with a focus on
parts and wholes. The interdependence of their statistical covariance. Outcomes must
flora, fauna, and the physical environment be identified and measured as specific vari-
in ecological systems offers another meta- ables. Treatments and programs must also
phor for what it means to think and analyze be conceptualized as discrete, independent
holistically. variables. The characteristics of program
Researchers and evaluators analyzing participants are also described by standard-
qualitative data strive to understand a phe- ized, quantified dimensions. Sometimes the
nomenon or program as a whole. This variables of interest are derived from pro-
means that a description and interpretation gram goals, for example, student achieve-
of a person's social environment, or an orga- ment test scores, recidivism statistics for a
nizations externai context, is essential for group of juvenile delinquents, sobriety rates
overall understanding of what has been ob- for participants in chemical dependency
serve d during fieldwork or said in an inter- treatment programs. At other times, the
view. This holistic approach assumes that variables measured are indicators of a larger
the whole is understood as a complex sys- construct. For example, community well-
tem that is greater than the sum of its parts. being may be measured by such rates for de-
The analyst searches for the totality or unify- linquency, infant mortality, divorce, unem-
ing nature of particular settingsthegestalt. ployment, suicide, and poverty (Brock,
Psychotherapist Fritz Perls (1973) made the Schwaller, and Smith 1985). These variables
term gestalt equivalent with a holistic per- are statistically manipulated or added to-
spective inpsychology. He use d the example gether in some linear fashion to test hypoth-
of three sticks that are just three sticks until eses and draw inferences about the relation-
one places them together to form a triangle. ships among separate indicators, or the
Then they are much more than the three sep- statistical significance of differences be-
arate sticks combined: They form a new tween measured leveis of the variables for
whole. different groups. The essential logic of this
approach is as follows: (1) Key program out-
A gestalt may be a tangible thing, such as a tri- comes and processes can be represented by
angle, or it may be a situation. A happening separate independent variables, (2) these
such as a meeting of two people, their conver- variables canbe quantified, and (3) relation-
sation, and their leave-taking would consti- ships among these variables are best por-
tute a completed situation. If there were an trayed statistically.
interruption in the middle of the conversadon, The primary critique of this logic by qual-
it would be an incomplete gestalt. (Brown itative-naturalistic evaluators is that such an
1996:36) approach (1) oversimplifies the complexities
of real-world programs and participants' ex-
The strategy of seeking gestalt units and periences, (2) misses major factors of impor-
holistic understandings in qualitative analy- tance that are not easily quantified, and (3)
sis contrasts with the logic and procedures fails to portray a sense of the program and its
of evaluation studies conducted in the ana- impacts as a whole. To support holistic anal-
lytical tradition of "let's take it apart and see ysis, the qualitative inquirer gathers data on
how it works." The quantitative-experimen- multiple aspects of the setting under study
tal approach to evaluation, for example, re- to assemble a comprehensive and complete
60 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

HaLisric \ j t o b m T A m \ m . !N ^ LTICS

H W Z R K ::;' ; N-RH ^ R - :
: I :'-. I U V U I V R R I I I I ii' i. i'I :'. !: -li vi i, I vi li :"! Or' ri .v j iV.'
U.1" n r f jV:. foitf ! i:v VV. L j !i i! ! i:"i H , ! >..! V' i " T C i'i:;i Vi '>..'.:b Vri!!.'.i. :iIIiJ'i-3'!=!VRRT vM\\\
iV:n liVsy ^;:!!::M |;''!''<: ,> ! r.i ,' ' u iym' .tith- - : rM.V-ii i:! rn iVi." iVll! Q % OA! S I "
T-' !'v W, :\:KhMf " -fritS Vi !I:V.!Vi\
1 . . I'I. '..II"IJ"!ri!V : M, R.'
J P ! ! ! - ! ! I;'R ;'.;.;I:.;:N

hl!! ri!!!!' " I n C:!;V P I iVfl !.'.!!' i:>,\\ v 'nr i.- > r, Iiis' II""rV.:1 tri."rj.! i-Vs?i:;.;! Liki'" Cr.Vii- Imf
[ii|iHtlf || i| l llll Ml^ ill lllll II il l il lllll iil [ Ml[ [ ll [ [ ! o I nli nl INI1 n] Ii.NIi:iV:' III Ni:i!.'.n.i
M il Iii'I i:!!.'l 1HJ" ' lr.Wii
H nl M =.i" ITI ' 11,=.' ri =;. : V
V.T; N ; Y ! , : ) ' I V I .,:!,Vi ..'.I.' --\fJ ;:';! '.V"-
VII" .II!::II, :'I!' .II* I W I
! V J H F ;:!RN>::!.:'. "-VI|T
; :;V. ! hi.- IJ:""I"!.':' \>V>? l:^:1 IVismaiM ,VVM-
rAan Ti':-!! i^";]! n :^-::;'!' ;^" . iMfl-
"':;.;i;;-':'I.=r-.;',ryi;",;!V." .j.;:
tv&it \v !;'':!:;s!i!:!J irnivv: :!.V vh^-^d n .
!
iVi.ji.ii i-,."iH !-V!|-=;iri-^i^-^:1!:^:!1 Ih:' iAf.Vv-nr;,
iiiHUPita.'. |:!.'jj"i'ii'i-U.jiVr" v ! , 1 i^.Ti.Vii

.Vra.i n I !i. I ! "S V;T:; j' i j !'. V i i r . V ! j j =<''! Vi i ' i ! ' ! j : l . ' i . IY
- ! :
? ilVV - i"! T U "UVHP N p"." i !I i 1 ; ; V ! ;i IIIJ-! ! J 1

!!.!:!! iVniV .jf 1

:i i . V ' r ' . ' ' ! r W i
i"r!.'i'ir-iir !iir/iVi:!1 Li =-.i /'!:. .V:iVr ,'irr!:;i'Vii"i'.
O Y O M IYM' ! !
i . " i . i !.' u ! r . ' ; ! ; ! " ; r i ! ! i . ; j ! |;
iV'nhrf nw ": .; ::!!r=:!i;." .;;.:-..;';I:!.I!; .!-; !iv
r i v i ! :". ; n . ; : i r -
! :
1' C P ; ' ! U V i t E J s p V . Y ! ^ . ' i i':i,
;i I:I a i r, \ ';.=r,' !ri I :. I I HV -J R,'! ";.. ;;-RI.;; ri.
:!!' isM ftit nt- i : ".W t^ -vg.^T.Wvi J O ! ! !i?iu-j:' - =--:Ih;:-H sv .is:iis.'!1 > v A ; i-=--
s : !:

j:'iii i'i*Mo - ii Huni<. r r i r!V 'i""r! r r i ! !;: s- : = :

.
i V I' i' i Viri-i I. ;VI'! : ' A R \ \ I I | ;: : . !:.'! L I-I: , Y!-: i'
1

r. !!' rr-ri IVvJr-J "

1 - . . .. J W H C '!' J I - I I ' Y ! !
: I ' I I ' ! ! . : ' I"I'.'IRI>r-v. "VN':!'
.!< :'!:= ii;'!=:;ri': :i :':- .i liIT mriYrg
! ! i'
- ni-.;Y"i:r>.:iV !' ! > Q \ i i;;.},' .. , '.: ! > u vi i Vr
l

i',!i I u ,i!i ! ; Sfr i! i!!!:" f6> k . : "-siT!i.;: .i:.:;.|! J'S:-M:!S il1.1. i'i..' i.r !nr::i;vo!";rri:r .Ji :: .; i\i :j
U"n- I; :'.p y !". TH!:..:;'. n t TD.1 lVU.1 A i rrsn !Y:: .".'Sm.TiV'!!! , t ! 1 IHR ;i.yrv,i::;V
: ! ' : V S : I nhU: VI :r iini.vi . MIM ! I M .VS.V V W
! ! i'i i1"!1'!.-'! :i/!Vi ri rrhi'.!!.") i!; ri.;-. fiiiVri-a l".'i vi
R ' I ! ; : . ' E I K R F ,M:i.!! 'VIR ! > | :I!"<!' v.ijV, ;V!:i a; xiy! -=,:"i. i' !i i- Vi-! '/ij.' :;'.;-
::i'i.::,T! i l'. ! v jVi-!! .;'!! W. \ ^iYDi" !i '!? 11.
1

picture of the social dynamic of the particu- surement), then statistical portrayals can be
lar situation or program. This means that at quite powerful and succinct. The advan-
the time of data collection, each case, event, tages of qualitative portrayals of holistic set-
or setting under study, though treated as a tings and impacts are that greater attention
unique entity with its own particular mean- can be given to nuance, setting, interdepen-
ing and its own constellation of relation- dencies, complexities, idiosyncrasies, and
ships emerging from and related to the con- context. John Dewey (1956) articula ted what
text within which it occurs, is also thought of a holistic approach means for both teaching
as a window into the whole. Thus capturing and research if one wants to gain insight into
and documenting history, interconnections, and understand the world of the child:
and system relationships are part of field-
work. The child's life is an integral, a total one. He
The advantages of using quantitative passes quickly and readily from one topic to
variables and irvdicators are parsimony, pre- another, as from one spot to another, but is not
cision, and ease of analysis. Where key ele- conscious of transition or break. There is no
ments can be quantified with validity, reli- conscious isolation, hardly conscious distinc-
ability, and credibility, and where necessary tion. The things that occupy him are held to-
statistical assumptions can be met (e.g., lin- gether by the unity of the personal and social
earity, normality, and independence of mea- interests which his life carries along.. . . [His]
Strategic Themes in Qualitative Inquiri/ J. 61

universe is fluid and fluent; its contents dis- How many program staffs have complained
solve and reform with amazing rapidity. But of the evaluation research monster?
after ali, it is the child's own world. It has the It is no simple task to undertake holistic
unity and completeness of his own life. analysis. The challenge is "to seek the es-
(pp. 5-6) sence of the life of the observed, to sum up,
to find a central unifying principie" (Bruyn
Qualitative sociologist Irwin Deutscher 1966:316). Again, Shapiro's (1973) work in
(1970) commented that despite the totality of evaluating innovative Follow Through class-
our personal experiences as living, working rooms is instructive. She found that stan-
human beings, social scientists have tended dardized test results could not be inter-
to focus their research on parts to the virtual preted without understanding the larger
exclusion of wholes: cultural and institutional context in which
the individual child is situated. Taking con-
text seriously, the topic of the next section, is
We knew that human behavior was rarely if an important element of holistic analysis.
ever directly influenced or explained by an An illuminative example of holistic think-
isolated variable; we knew that it was imposing came to me from a Portuguese colleague.
sible to assume that any set of such variables He told of driving in a remote area of his
was additive (with or without weighting); we country when he came upon a sizable herd
knew that the complex mathematics of the in- of sheep being driven along the road by a
teraction among any set of variables was in- shepherd. Seeing that he would be delayed
comprehensible to us. In effect, although we until the sheep could be turned off the road,
knew they did not exist, we defined them into he got out of the car and struck up a conver-
being. (p. 33) sation with the shepherd.
"How many sheep do you have?" he
While many would view this intense cri- asked.
tique of variable analysis as too extreme, the "I don't know," responded the young
reaction of many program staff to scientific man.
research is like the reaction of Copernicus to Surprised at this answer, the traveler
the astronomers of his day: "With them/' he asked, "How do you keep track of the flock if
observed, "it is as though an artist were to you don't know how many sheep there are?
gather the hands, feet, head, and other mem- How would you know if one was missing?"
bers for his images from diverse models, The shepherd seemed puzzled by the
each part excellently drawn, but not related question. Then he explained, "I don't need
to a single body, and since they in no way to count them. I know each one and I know
match each other, the result would be mon- the whole flock. I would know if the flock
ster rather than man" (from Kuhn 1970:83). was not whole."

Context Sensitivity

ny single act from any single person, put out of context, is

damnable.

Actor Kevin Spacey accepting the 2000 Academy Award

for Best Performance by an Actor in the film
American Beauty, explaining the film's message
62 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Let's move, now, from sheep to elephants. text and as part of an ecological system in re-
One of the classic tales used to illustrate the la tion to other flora and fauna, in its natural
relationship between parts and wholes is the environment.
story of the nine blind people and the ele- When we say to someone, "You've taken
phant. Each person touches only one part of my comment out of context," we are saying,
the elephant and therefore knows only that You have distorted what I said, changed its
part. The person touching the ears thinks an meaning by omitting criticai context.
elephant is like a large, thin fan. The person In Victor Hugo's great classic Les
touching the tail thinks the elephant is like a Misrables, we first encounter Jean Valjean as
rope. The person touching the truck thinks a hardened criminal and common thief; then
of a snake. The legs feel like tree trunks, the we learn that he was originally sentenced to
elephanfs side like a tall wall. And so it five years in prison for stealing a loaf of
goes. The holistic point is that one must put bread for his sister's starving family. That
ali of these perspectives together to get a full added context for his "crime" changes our
picture of what an elephant actually looks understanding. The battle over standard-
like. ized sentencing guidelines in the criminal
But such a picture will still be limited, justice system is partly a debate about how
even distorted, if the only place one sees the much to allow judges sway in taking into ac-
elephant is in the 200 or at the circus. To un- count context and individual circumstances
derstand the elephanthow it developed, in pronouncing sentences.
how it uses its trunk, why it is so largeone Naturalistic inquhy preserves natural
must see it on the African savanna or in the context. Social psychology experiments un-
Asian jungle. In short, one must see it in con- der laboratory conditions strip observed ac-
Strategic Themes in Qualitative Inquiri/ J. 63

tions from context. But that is the point of were obtained on 120 students from four
such laboratory experimentsto generate classrooms. The extraction of significant pre-
fndings that are context free. The scientific dictor variables is the purpose of the final
ideal of generalizing across time and space analysis. Interviews were conducted with
is the ideal of identifying principies that do teachers and principais to determine how test
not depend on context. In contrast, qual- scores were used. The analysis concludes with
itative inquiry elevates context as criticai to the researcher's interpreta tions. The research-
understanding. Portraitist Sara Lawrence- er wishes to thank those who cooperated in
Lightfoot (1997) explains why she finds con- this study.
text "crucial to the documentation of human
experience and organizational culture": This journal article abstract represents ac-
ademic writing as I was taught to do it in
By context, I mean the settingphysical, geo- graduate school. This writing style still pre-
graphic, temporal, historical cultural, aesthetic dominates in scholarly journals and books.
within which action takes place. Contextbe- No human being is visible in this writing.
comes the framework, the reference point, the The passive voice reigns. Instruments were
map, the ecological sphere; it is used to place selected; decisions were made; a model was
people and action in time and space and as a constructed; records were reviewed and
resource for understanding what they say and coded; data were obtained; predictor vari-
do. The context is rich in clues for interpreting ables were extracted; interviews were con-
the experience of the actors in the setting. We ducted. The warmth of thanks is extended
have no idea how to decipher or decode an by a role, the researcher: "The researcher
action, a gesture, a conversation, or an excla- wishes to thank those who cooperated." The
ma tion unless we see it embedded in context third-person, passive voice communicates a
(p- 41) message: This work is about procedures not
people. This academic style is employed to
project a sense of objectivity, control, and au-
Voice and Perspective:
thority. The overall impression is mechani-
Reflexivity
cal, robotlike, distant, detached, systematic,
and procedural. The research is the object of
ABSTRACT OF A SCHOOL attention. Any real, live human being, sub-
ACHIEVEMENT STUDY ject to ali the usual foibles of being human, is
barely implied, generally disguised, hidden
This study will delineate the major factors that away, and kept in the background.
affect school achievement Instruments were Contrast that academic voice with my
selected to measure achievement based on va- explana tion of how I analyzed a 10-day
lidity and reliability criteria. Decisions were coining-of-age experience with my son in
ma de about administering the tests in con- the Grand Canyon. (I presented part of the
junction with administrators taking into ac- analysis of that experience as Exhibit 1.3 in
count time and resource constraints. A the first chapter.) Here's an excerpt in which
regression model was constructed to test rela- I describe the analytical process.
tionships between various background vari-
ables and demonstrated achievement. School I'm not sure when the notion first took hold of
records were reviewed and coded to ascertain me that articulating altemative coming of age
students' background characteristics. Data paradigms might help elucidate our Canyon
64 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

experience. Before formally conceptualizmg notes, asks interview questions, and inter-
contrasting paradigm dimensions, I experi- prets responses. Self-awareness, then, can be
enced them as conflicting feelings that ema- an asset in both fieldwork and analysis. De-
nated from my struggle to sort out what I veloping appropriate self-awareness can be
wanted my son's initiation to be, while also a form of "sharpening the instrument"
grappling with defining my role in the pro- (Brown 1996:42). The methods section of a
cess. I suppose the idea of alternative para- qualitative study reports on the researcher's
digms first emerged the second night as I training, preparation, fieldwork procedures,
paced the narrow beach where White Creek and analytical processes. This is both the
intersects Shinumo and pondered the Great strength and weakness of qualitative meth-
Unconformity [a geologic reference] as meta- ods, the strength in that a well-trained, expe-
phor for the gap between tribal approaches to rienced, and astute observer adds value and
initiation and coming of age for contemporary credibility to the inquiry, while an ill-pre-
youth. In the weeks and months after our Can- pared, inexperienced, and imperceptive ob-
yon experience, far from languishing in the server casts doubt on what is reported. Judg-
throes of retox as I expected, the idea of con- ments about the significance of findings are
trasting paradgms stayed with me, as did the thus inevitably connected to the researcher's
Canyon experience. I started listing themes credibility, competence, thoroughness, and
and matching them with incidents and turn- integrity. Those judgments, precisely be-
ing points along the way. The sequence of inci- cause they are acknowledged as inevitably
dents became this book and the contrasting personal and perspective dependent, at least
themes became the basis for this closing chap- to some extent, invite response and dia-
ter, a way for me to figure out how what logue, rather than just acceptance or rejec-
started out as an initiation become a humanist tion.
coming of age celebra tion. (Patton 1999a:332) Reflexivity has entered the qualitative lex-
icon as a way of emphasizing the impor-
The contrast between the traditional aca- tance of self-awareness, political/cultural
demic voice and the personal voice of quali- consciousness, and ownership of one's per-
tative analysis recalls philosopher and theo- spective.
logian Martin Buber's (1923) influential
distinction between "I-It" and "I-Thou" re- In the rush of interest in qualitative research in
lationships. An I-It relationship regards the past 15 years, few topics have developed
other human beings from a distance, from a as broad a consensus as the relevance of ana-
superior vantage point of authority, as ob- lytic "reflexivity." By most accounts, reflexivi-
jects or subjects, things in the environment ty is a deconstructive exercise for locating the
to be examined and placed in abstract intersections of author, other, text, and world,
cause-effect chains. An I-Thou perspective, and for penetrating the representational exer-
in contrast, acknowledges the humanity of cise itself. (MacBeth 2001:35)
both self and others and implies relation-
ship, mutuality, and genuine dialogue. Being reflexive involves self-questioning
The perspective that the researcher brings and self-understanding, for "ali under-
to a qualitative inquiry is part of the context standing is self-understanding" (Schwandt
for the findings. A human being is the in- 1997a:xvi). To be reflexive, then, is to under-
strument of qualitative methods. A real, live take an ongoing examination of what I know
person makes observations, takes field and how I know it, "to have an ongoing con-
Strategic Themes in Qualitative Inquiri/ J. 65

versation about experience while simulta- account, and communicate perspective and
neously living in the moment" (Hertz 1997: voice. Balancing criticai and creative analy-
viii). Reflexivity reminds the qualitative in- ses, description and interpretation, or direct
quirer to be attentive to and conscious of the quotation and synopsis also involves issues
cultural, political, social, linguistic, and of perspective, audience, purpose, and
ideological origins of one's own perspective voice. No rules or formula can tell a qualita-
and voice as well as the perspective and tive analyst precisely what balance is right
voices of those one interviews and those to or which voice to use, only that finding both
whom one reports. Exhibit 2.2 depicts this balance and voice is part of the work and
reflexive triangulation. challenge of qualitative inquiry, what Lewis
Writing in the first-person, active voice (2001) has acknowledged as "the difficulty
communicates the inquirer's self-aware role of trying to situate the I in narrative re-
in the inquiry: "I started listing themes and search" (p. 109).
matching them with incidents and turning In addition to finding voice, the criticai and
points along the way." The passive voice creative writing involved in qualitative
does not: "Themes were listed and matched analysis and synthesis challenge the in-
to incidents and turning points along the quirer to own onefs voice and perspective. Here,
way." Judith Brown (1996) captured the im- we owe much to feminist theory for high-
portance of the first-person voice in the title lighting and deepening our understanding
of her book The I in Science: Training to Utilize of the intricate and implicate relationships
Subjectivity in Research. By subjectivity she between language, voice, and consciousness
means "the domain of experiential self- (e.g., Gilgan 1982; Minnich 1990). We are
knowledge" (p. 1). Voice reveals and com- challenged by postmodern critiques of
municates this domain. knowledge to be clear about and own our
But voice is more than grammar. A credi- authorship of whatever we propound, to be
ble, authoritative, authentic, and trustwor- self-reflective, to acknowledge biases and
thy voice engages the reader through rich limitations, and to honor multiple perspec-
description, thoughtful sequencing, appro- tives (Greene 1998a, 1998b; Mabry 1997)
priate use of quotes, and contextual clarity while "accepting incredulity and doubt as
so that the reader joins the inquirer in the modal postmodern responses to ali at-
search for meaning. And there are choices of tempts to explain ourselves to ourselves"
voice: the didactic voice of the teacher; the (Schwandt 1997b:102). From struggles to lo-
searching, logical voice of the sleuth; the nar- cate and acknowledge the mevitably politi-
rator voice of the storyteller; the personal cal and moral nature of evaluative judg-
voice of the autoethnographer; the doubting ments, we are challenged to connect voice
voice of the skeptic; the intimacy of the in- and perspective to praxisacting in the
sider's voice; the detachment of the out- world with an appreciation for and recog-
sider's voice; the searching voice of uncer- nition of how those actions iriherently ex-
tainty; and the excited voice of discovery, to press social, political, and moral values
offer but a few examples. Just as point of (Schwandt 1989, 2000) and to personalize
view and voice have become focai points of evaluation (Kushner 2000), both by owning
writing good fiction and nonfiction, as in our own perspective and by taking seriously
Nancy Mairs's (1997) Voice Lessons: On Be- the responsibility to communicate authenti-
coming a (Woman) Writer, so too qualitative cally the perspectives of those we encounter
analysts are having to learn about, take into during our inquiry. These represent some
66 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

mSjaiMMiim Reflexive Questions: Triangulated Inquiry

Those studied \ / Those receiving the

(participants): \ Reflexive screens: / study (audience):
How do they know what \ Culture, age, gender, class, / How do they make
they know? What shapes and \ social status, education, / make sense of what 1 give
has shaped their woridview? \ family, politicai praxis, / them? What perspectives
How do they perceive me? \ language, vaiues / do they bring to the findings
Why? How do 1 know? 1 offer? How do they perceive
How do 1 perceive them? me? How do 1 perceive them?

Myself:
(as qualitative inquirer):
What do 1 know?
How do 1 know what know?
What shapes and has shaped my perspective?
With what voice do 1 share my perspective?
What do 1 do with what i have found?

of the more prominent contextual forces real-world observations through naturalis-

that have elevated the mportance of own- tic inquiry; openness, responsiveness, and
ing voice and perspective in qualitative flexibility through emergent designs; focus
analysis. through purposeful sampling; richness and
It takes no great self-awareness or self- depth through qualitative data; use of ali of
confidence to report a statistically signifi- one's capacities through personal experi-
cant t test with confidence intervals based on ence and engagement; balancing the criticai
a formula and calculations easily replicated and creative through a stance of empathic
and confirmed. It can take considerable neutrality; sensitivity to dynamic processes
self-awareness and confidence to report: I and systems; appreciation of idiosyncrasies
coded these 40 interviews, these are the through a unique case orienta tion; insight
themes I found, here is what I think they and understanding through inductive anal-
mean, and here is the process I undertook to ysis, contextual sensitivity, and a holistic
arrive at those meanings. The latter state- perspective; and authenticity and trustwor-
ment calls for, even demands, a sense of thiness through ownership of voice and per-
voice and perspective. spective. These are not absolute and univer-
sal characteristics of qualitative inquiry, but
rather strategic ideais that provide a direc-
S From Strategic Ideais tion and framework for developing specific
to Practical Choices designs and concrete data collection tactics.
Ideally, a pure qualitative inquiry strat-
The 12 themes of qualitative inquiry re- egy includes ali the themes and dimensions
viewed in this chapter are strategic ideais: identified in this chapter. For example, in an
Strategic Themes in Qualitative Inquiri/ J. 6 7

ideal naturalistic/inductive inquiry the re- begin to affect the program quite directly
searcher neither manipulates the setting un- and intentionally (given the job of helping
der study nor predetermines what variables improve the program), thus moving away
or categories are worth measuring. In prac- from a purely naturalistic approach. As
tice, however, it is important to recognize evaluative feedback is used to improve the
that actually conducting holistic-inductive program, the evaluator may then move back
analysis and implementing naturalistic in- into a more naturalistic stance to observe
quiry are always a matter of degree. In mak- how the feedback-induced changes in the
ing this point, Guba (1978) has depicted the program unfold.
practice of naturalistic inquiry as a wave on In the same vein, the attempt to under-
which the investigator moves from varying stand a program or treatment as a whole
degrees of a "discovery mode" to varying does not mean that the investigator never
emphasis of a "verification mode" in at- becomes involved in component analysis or
tempting to understand the real world. As in looking at particular variables, dimen-
fieldwork begins, the inquirer is open to sions, and parts of the phenomenon under
whatever emerges from the data, a discov- study. Rather, it means that the qualitative
ery or inductive approach. Then, as the in- inquirer consciously works back and forth
quiry reveals patterns and major dimen- between parts and wholes, separate vari-
sions of interest, the investigator will begin ables, and complex, interwoven constella-
to focus ori verifying and elucidating what tions of variables in a sorting-out then
appears to be emerginga more deductive putting-back-together process. While stay-
approach to data collection and analysis. In ing true to a strategy that emphasizes the im-
essence, what is discovered may be verified portance of a holistic picture of the program,
by going back to the world under study and the qualitative evaluator recognizes that
examining the extent to which the emergent certain periods of fieldwork may focus on
analysis fits the phenomenon and zuorks to component, variable, and less-than-the-
explain what has been observed. Glaser and whole kinds of analysis.
Strauss (1967), in their classic framing of The practice and practicalities of field-
grounded theory, described what it means work also mean that the strategic mandate
for results to fit and work: "By 'fit' we mean to "get close" to the people and setting un-
that the categories must be readily (not forc- der study is neither absolute nor fixed.
ibly) applicable to and indicated by the data Closeness to and involvement with the peo-
under study; by 'work' we mean that they ple under study are most usefully viewed as
must be meaningfully relevant to and be variable dimensions. The personal styles
able to explain the behavior under study" and capabilities of evaluators will permit
(p. 3). Discovery and verification mean mov- and necessitate variance along these dimen-
ing back and forth between induction and sions. Variations in types of programs and
deduction, between experience and reflec- evaluation purposes will affect the extent to
tion on experience, and between greater de- which an evaluator can or ought to get close
grees and lesser degrees of naturalistic in- to the program staff and participants. More-
quiry. over, closeness is likely to vary over the
In program evaluation in particular, the course of an evaluation. At times the eval-
evaluator may, through feedback of initial uator may become totally immersed in the
findings to program participants and staff, program experience. These periods of im-
68 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

mersion may be followed by times of Beyond Competing

withdrawal and distance (for personal as Inquiry Paradigms
well as methodological reasons), to be fol-
lowed still later by new experiences of im- Having presente d the strategic ideais of
mersion in and direct experience with the qualitative inquiry and noted variations in
program. their practical implementation and attain-
Nor is it necessary to be a qualitative ment, before closing this chapter I want to
methods purist. Qualitative data can be acknowledge and comment on the contro-
collected and used in conjunction with versy that sometimes engulfs qualitative
quantitative data. Today's evaluator must methods. Students attempting to do qualita-
be sophisticated about matching research tive dissertations can get caught up in and
methods to the nuances of particular evalua- may have to defend, philosophically as well
tion questions and the idiosyncrasies of spe- as methodologically, the use of qualitative
cific stakeholder needs. Such an evaluator inquiry to skeptical committee members
needs a large repertoire of research methods who define doctoral-level work as rigorous
and techniques to use on a variety of prob- hypothesis testing. Evaluators may encoun-
lems. Thus, an evaluator may be called on to ter policymakers and funders who dismiss
use any and ali social science research meth- qualitative data as mere anecdote. The sta-
ods, including analyses of quantitative data, tistically addicted may poke fun at what
questionnaires, secondary data analysis, they call the "softness" of qualitative data.
cost-benefit and cost-effectiveness analyses, (In Western society, where anything can be
standardized tests, experimental designs, and often is sexualized, the distinction be-
unobtrusive measures, participant observa- tween "hard" data and "soft" data has addi-
tion, and in-depth interviewing. The evalua- tional nuances of meaning and innuendo.)
tion researcher works with intended users of Such encounters derive from a long-stand-
the findings to design an evaluation that ining methodological paradigms war. Though
cludes any and ali data that will help shed many have pronounced the war and even
light on important evaluation questions, the debate over (cf. Cook 1995; Greene
given constraints of resources and time. 1998a:36; Patton 1997a: 290-95), not every-
Such an evaluator is committed to research one has adopted a stance of methodological
designs that are relevant, meaningful, un- enlightenment and tolerance, namely, that
derstandable, and able to produce useful re- methodological orthodoxy, superiority, and
sults that are valid, reliable, and believable. purity should yield to methodological ap-
On many occasions a variety of data collec- propriateness, pragmatisin, and mutual re-
tion techniques and design approaches may spect. Therefore, a brief review of the para-
be used together. Multiple methods and a digms debate is in order. (Elsewhere I have
variety of data types can contribute to meth- provided a more extensive review of the
odological rigor. The ideal in evaluation de- methodological paradigms debate; Patton
signs is methodological appropriateness, 1997a: 265-99,1988c.)
design flexibility, and situational respon-
Philosophers of science and method-
siveness in the service of utility (Patton
ologists have been engaged in a long-stand-
1997a)not absolute allegiance to some
ing epistemological debate about the nature
ideal standard of paradigm purity and
of "reality" and knowledge. Thatphilosoph-
methodological orthodoxy.
ical debate finds its way into research and
Strategic Themes in Qualitative Inquiri/ J. 69

evaluation in arguments over the goals of the variety of inquiry approaches has ex-
empirical studies and differences of opinion panded well beyond the simplistic
about what constitutes "good" research. In dichotomy between quantitative and quali-
its simplest and most strident formula tion, tative paradigms. In contrast to these two
this debate has centered on the relative value classically opposed orthodoxies, this book
of two different and competing inquiry par- offers a pragmatic strategy of matching con-
adigms: (1) using quantitative and experi- crete methods to specific questions, includ-
mental methods to generate and test hypo- ing the option of tactically mixing methods
thetical-deductive generalizations versus as needed and appropriate. My practical
(2) using qualitative and naturalistic ap- (and controversial) view is that one can learn
proaches to inductively and holistically un- to be a good interviewer or observer, and
derstand human experience and con- learn to make sense of the resulting data,
structed meanings in context-specific without first engaging in deep
settings. For example, Taylor and Bogdan epistemological reflection and philosophi-
(1984) contrast the Verstehen tradition, cal study. Such reflection and study can be
rooted in qualitative phenomenology, to
helpful to those so inclined, but it is not a
measurement-oriented positivism as fol-
prerequisite for fieldwork. Indeed, it can be
lows:
a hindrance. Getting some field experience
first, then studying philosophy of science,
Two major theoretical perspectives liave dom-
has much to recommend it as a learning
inated the social science scene. The irst, posi-
strategy. Otherwise, it's ali abstractions.
tivism, traces its origins in the social sciences to
Still, the paradigms debate is part of our
the great theorists of the riineteenth and early
methodological heritage and knowing a bit
twentieth centuries and especially to Auguste
about it, and its distortions (Shadish 1995b,
Comte and Emile Durkheim. The positivist
1995c), may deepen appreciation for the im-
seeks the facts or causes of social phenomena
portance of a strategic approach to methods
apart from the subjective states of individuais.
Durkheim told the social scientist to consider
decision making.
social facts, or social phenomena, as "things" A paradigm is a worldviewa way of
that exercise an externai influence on people. thinking about and making sense of the
The second theoretical perspective, which, complexities of the real world. As such, par-
following the lead of Deutscher, we will de- adigms are deeply embedded in the social-
scribe as phenomenological, has a longhistory in ization of adherents and practitioners. Para-
philosophy and sociology. The phenomen- digms tell us what is important, legitimate,
ologist is committed to understanding social and reasonable. Paradigms are also norma-
phenomena from the actor's own perspective. tive, telling the practitioner what to do with-
He or she examines how the world is experi- out the necessity of long existential or
enced. The important reality is what people epistemological consideration. But it is this
perceive it to be. (pp. 1-2) aspect of paradigms that constitutes both
strength and weaknessa strength in that it
Debate about these contrasting and com- makes action relatively easy, a weakness in
peting perspectives has been an important that the very reason for action is hidden in
part of the history of research and evalua- the unquestioned assumptions of the para-
tion, but, as Chapters 3, 4, and 9 will show, digm.
70 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Now Calm down a bit Fm not criticizing your operation

down here... I'm just asking... .in your entry evaluation process,
do you operate from a qualitative or quantitative paradigm?

Are there paradigms after death?

Scientists work from models acquired through ask or debate what makes a particular prob-
education and through subsequent exposure lem or solution legitimate tempts us to sup-
to the literature often without quite knowing pose that, at least intuitively, they know the
or needing to know what characteristics have answer. But it may only indicate that neither
given these models the status of community the question nor the answer is felt to be rele-
paradigms. . . . That scientists do not usually vant to their research. Paradigms may be prior
Strategic Themes in Qualitative Inquiri/ J. 71

to, more binding, and more complete than any make researchers biased in favor of and
set of rules for research that could be unequiv- against certain approaches.
ocally abstracted from them. (Kuhn 1970:46) While one may still encounter people
who rigidly confess allegiance to only quan-
But what does ali this matter to the stu- titative or qualitative methods, most practi-
dent interested in pursuing some research or tioners appear to have become eclectic and
evaluation question? It matters because pragmatic. Looking back, we can now see
paradigm-derived biases are the source of that the qualitative-quantitative debate
the distinctions mentioned earlier between oversimplified and often confused method-
"hard" data and "soft" data, empirical stud- ological and philosophical issues. For exam-
ies versus "mere anecdotes," and "objec- ple, the notion that some combinations of
tive" research versus "subjective" studies. methods and philosophy ever constituted
These labels reveal value-laden prejudices consistent, coherent, and stable paradigms
about what constitute credible and valuable has proved problematic. Shadish (1995c), for
contributions to knowledge. Such preju- example, in introducing an important set of
dices and paradigmatic blinders limit meth- articles aimed at "de-Kuhnifying" the de-
odological choices, flexibiHty, and creativity. bate, concluded that "there is little empirical
Adherence to a methodological paradigm evidence in support of such a Kuhnian para-
can lock researchers into unconscious pat- digm portrayal. . . . [T]he relevant concep-
terns of perception and behavior that dis- tual and philosophical issues are far more
guise the biased, predetermined nature of complex than the simple quantitative-quali-
their methods "decisions." Methods deci- tative dichotomy implies" (p. 48). Chapter 9
sions tend to stem from disciplinary pre- will revisit the quantitative-qualitative para-
scriptions, concerns about scientific status, digms debate in more depth as part of our
old methodological habits, and comfort examination of issues that affect judgments
with what the researcher knows best. about the quality and credibility of qualita-
Training and academic socialization tend to tive methods.

Pragmatism

/ - very thinker puts some portion an apparently stable world in peril.

John Dewey (1929)

While a paradigm offers a coherent assert that randomized experiments are "the
worldview, an anchor of stability and cer- standard against which other designs for
tainty in the real world sea of chos, operat- impact evaluation are judged" (p. 21). My
ing narrowly within any singular paradigm pragmatic stance aims to supersede one-
can be quite limiting. As a pragmatist, I take sided paradigm allegiance by increasing the
issue as much with the purist, one-sided ad- concrete and practical methodological op-
vocacy of Lincoln and Guba (1985), who be- tions available to researchers and evalua-
lieve that naturalistic inquiry is the only tors. Such pragmatism means judging the
valid and meaningful way to study human quality of a study by its intended purposes,
beings, as I do with the narrow, intolerant available resources, procedures followed,
stance of Boruch and Rindskopf (1984), who and results obtained, ali within a particular
72 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

context and for a specific audience. When a on strategic choices has conveyed, I hope,
new drug is tested before being made avail- the idea that a wide range of possibilities ex-
able to the general population, a dou- ists when selecting methods. The point is to
ble-blind randomized experiment to deter- do what makes sense, report fully on what
mine efficacy is the design of choice, with was done, why it was done, and what the
careful attention to controlled and carefully implications are for findings. Chapter 5 is
measured dosage and outcome interactions, devoted to design issues, including design
including side effects. But if the concern is flexibility, using multiple methods, and
whether people take the new drug appropri- making practical decisions.
ately, and one wants to know what people in A Sufi story about the wise fool Mulla
a group think about the new drug (e.g., an Nasrudin illustrates the importance of un-
antidepressant), how they make sense of derstanding the connections between strate-
taking or not taking it, what they believe gic ideais and practical tactics in real-world
about themselves as a result of experiencing situations. Real-world situations seldom re-
the drug, and how those around them deal semble the theoretical ideais taught in the
with it, then in-depth interviews and ob- classroom.
servations are the place to start. The impor-
tance of understanding alternative research Ideal Conditions for
paradigms is to sensitize researchers and Research: A Cautionary Tale
evaluators to the ways in which their meth-
odological prejudices, derived from their In his youth, Nasrudin received training
disciplinary socialization experiences, may in a small monastery noted for its excellence
reduce their methodological flexibility and in the teaching of martial arts. Nasrudin be-
adaptability. came highly skilled in self-defense and after
I reiterate: Being pragmatic allows one to two years of training both his peers and his
eschew methodological orthodoxy in favor teachers recognized his superior abilities.
of methodological appropriateness as the pri- Each day, it was the responsibility of one
mary criterion for judging methodological of the students to go to the village market to
quality, recognizing that different methods beg for alms and food. It happened that a
are appropriate for different situations. Situ- small band of three thieves moved into the
ational responsiveness means designing a area. They observed how the monastery ob-
study that is appropriate for a specific in- tained food daily and began hiding along
quiry situation or interest. A major purpose the path the students had to take back to the
of this book, and the focus of Chapter 4, is to monastery. As a returning student returned,
identify the kinds of research questions and laden with food and alms, the thieves would
program evaluation situations for which attack. After three days of such losses, the
qualitative inquiry is the appropriate meth- monastery's few supplies were exhausted. It
od of choice. was Nasrudin's turn to go to the village mar-
Paradigms are really about epistemology, ket. His elders and peers were confident that
ontology, and philosophy of science. As Nasrudin's martial arts skills were more
such, paradigms are important theoretical than sufficient to overcome the small band
constructs for illuminating fundamental as- of thieves.
sumptions about the nature of reality. But at At the end of the day, Nasrudin returned
the pragmatic levei of making concrete ragged, beaten, and empty-handed. Every-
methods decisions, this chapter's emphasis one was amazed. Nasrudin was taken im-
Strategic Themes in Qualitative Inquiri/ J. 73

mediatelybefore the Master. "Nasrudin," he and conditions in which qualitative strate-

asked, "how is it that with ali your skill in gies and methods offer advantages. In
our ancient arts of defense, you were over- Chapter 4,1 will present a range and variety
come?" of situations and inquiry problems that par-
"But I did not use the ancient arts," re- ticularly lend theinselves to qualitative in-
plied Nasrudin. quiry. Chapter 5 will then discuss in more
Ali present were dumbfounded. An ex- detail some of the methodological trade-offs
planation was demanded. involved in adapting the strategic ideais of
"Ali of our competitions are preceded by qualitative methods to the practical realities
great and courteous ceremony," Nasrudin of conducting research and evaluation in the
explained. "We have learned that the open- field. Chapters devoted to observation, in-
ing prayers, the ceremonial cleansing, the terviewing, analysis, and enhancing the
bow to the Eastthese are essential to the quality and credibility of qualitative studies
ancient ways. The ruffians seemed not to un- follow that design chapter. To lay the
derstand thenecessity for these things. 1 did- groundwork for in-depth review of applica-
n't find the situa tion ideal enough to use the tions and methods, the next chapter exam-
methods you have taught us, Master." ines alternative theoretical frameworks that
are closely associated with and used to
guide qualitative inquiry.

On more than one occasion, researchers

or evaluators have told me of their belief in
the potential usefulness of qualitative meth- S Note
ods, but they tell me, "I just haven't found
the ideal situation in which to use them." 1. Excerpts in this chapter from the interview
Ideal situations are rare, but we will con- with Brackette F. Williams are used with her per-
sider throughout this book the questions mission.
Variety in Qualitative Inquiry
Theoretical Orientations

S p e c i a l (C\i-f+s

"Tell us again, Master, how it was in the beginning."

"In the beginning special gifts were given to different groups of people. The
caregivers were endowed with compassion for the less fortunate. The engineers
were given the ability to see what was not yet there. The carpenters were given
patience to set straight lines and perfect angles. The technicians were provided
with diligence so that they might conscientiously follow the blueprints and de-
tailed directions of others. The experimental scientists were given the certain be-
lief that the world could be manipulated according to their vision of it. The qual-
itative inquirers were gifted with a passion for depth, detail, and understanding
meanings. And so it went until, finally, there remained one last group and one
last gift. These were the explorers. To them was given the gift of curiosity that
they might forever see new worlds and uncover the many wonders of the
world."
"But what of the evaluators?" the children asked. "You have not mentioned
their special gift."
Halcolm smiled. "The evaluators, dear children, were spread throughout ali
the other groups, each endowed with the special gift of his or her own group,
and each using that gift in a special way."
"But does that not make for much arguing among evaluators about who has
the most special gift of ali?"
Halcolm grinned.
From Halcolm/s Origins of Human Species

13. 75
76 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

From Core Strategies to Rich cussing qualitative research as if it were one

Diversity approach. (Jacob 1988:16)

The last chapter presented 12 primary Major social sciences have drawn on and
threads that are woven through the tapestry contributed to qualitative methods in differ-
of qualitative inquiry. A central point of that ent ways depending on the interests of theo-
chapter was that different purposes, situa- rists and methodologists in a particular dis-
tions, questions, and resources will affect the cipline (cf. Brizuela et ai. 2000; Kuhns and
degree to which such qualitative ideais as Martorana 1982). The language of discourse
naturalistic inquiry, a holistic perspective, also varies. As Schwandt (1997a) has ob-
and inductive analysis can be realized in served in his very useful dictionary of quali-
practice. Yet, despite variation along the sev- tative terminology:
eral dimensions of qualitative inquiry, there
are still core strategies and directions that Qualitative inquiry... is a set of multiple prac-
differentiate a qualitative/naturalistic strat- tices in which words in methodological and
egy from a quantitative/experimental one, philosophical vocabularies acquire different
as well as places where they can usefully be meanings in their use or in particular acts of
combined to complement each other (e.g., speaking about the meariing of the practice.
Tashakkori and Teddlie 1998). This chapter These different ways of speaking form some-
will present the rich menu of alternative pos- thing more like a constellation of contested
sibilities zvithin qualitative research by fo- practices than an integrated, readily survey-
cusing on different theoretical perspectives able order. There are multiple sources and
that are associated with qualitative inquiry. kinds of disputes, but generally they involve
Qualitative inquiry is not a single, mono- different ways of conceiving of the aim of
lithic approach to research and evaluation. qualitative inquiry stemming from different
Discussions such as that in Chapter 2 that fo- traditions of thought. (p. xiv)
cus on differentiating primary strategies of
qualitative/naturalistic methods from those Those coming new to qualitative inquiry
of quantitative/experimental methods can are understandably confused and even
leave the impression that there are only two discombobulated by the diverse terminol-
methodological or paradigmatic alterna- ogy and contested practices they encounter.
tives. In fact, as we "turn inward in qualita- Phenomenology. Hermeneutics. Ethnometh-
tive research," we find "an exhilarating and odology. Semiotics. Heuristics. Phenomen-
at times exhausting proliferation of types ography. Such language! Exhibit 3.1 repro-
zvithin the qualitative paradigm (Page duces a letter of lamentation I received fol-
2000:3). io wing publication of the first edition of this
book, which did not include the current
When one looks more c l o s e l y . . . the apparent chapter.
unity of the qualitative approach vanishes, This chapter sorts through some of the
and one sees considerable diversity. What has major perspectives and traditions that in-
been called "qualitative research" conveys dif- form the rich variety that is qualitative in-
ferent meanings to different people. Needless quiry. We shall look at how varying theoreti-
to say, this has ca used considerable confusion. cal traditions emphasize different questions
. . . A major source of the confusion lies in dis- and how these particular emphases can
Variety in Qualitative Inquiri/ J. 77

E X H B I T 3.1 Which Approach Is Rght?

Helpl
Dear Dr. Patton:

I desperately need your help. I am a graduate student in education, planning to do

my dissertation observing classrooms and teachers identified as innovative and ef-
fective. want to see if they share any common approachesorwisdom that might be
considered "best practices." I took this idea to one professor who asked me if was
proposng a phenomenologicai or grounded theory study. When i asked what the dif-
ference was, he said it was myjob to find out. I've read about both but am stii con-
fused. Another professortold me i could do a qualitative study, but that asking about
"best practices" meant that I was a positivist not a phenomenologist. Another grad
student was tod to "use a hermeneuticframing," butshe'sin a different department
with a differenttopic. I'm a formerschooiteacherand, Ithink, a prettygoodobserver
and interviewer. gotvery excited reading your book about the valueof in-depth ob-
servations and interviewing, and that's where I got the idea for my dissertation, but
now Tm being toid I have to fit into one of these categories. Please tell me which one
is rightformy study. I don'tcare which one itis. Ijustwanttogeton withstudying in-
novative classrooms. I feel lost and am on the verge of just doing a questionnaire
where these phiiosophy questions don't seem to get asked. But if you can tel me
which approach is right, I might still be able to do what l want to do. Help!!!

Dear :

Yourdilemma is common.The distinctionsyoiTre being asked to makeare, indeed,

difficult-and not everyone agrees about what these terms and traditions mean. I
didn't include them in my book in the hopes that the methods of in-depth interview-
ing and observation could stand on their own. As youVe discovered, you don't need a
class in phiiosophy to design good questionnaires, though an argument can be made
that people using questionnaires and statistics would benefit from reflection on their
epistemological (nature and justification of knowledge) and ontological (nature of
reality) assumptions. Unfortunately, a lot of qualitative courses spend more time on
epistemology than methods, which may make students better philosophers than in-
terviewers. Some balance is needed. Your professors are doing you a service by having
you struggle with understanding different qualitative schools of thought because
what approach you take does make a difference-and students of qualitative inquiry
should be expected to know at least the major competing and contrasting traditions,
just as those doing statistical tests need to understand what different tests do. In the
next edition of my book Ml include a chapter reviewing major phlosophical and
methodological traditions. But that won't help you now.
Toansweryourquestion directly, there is no "right" approach any more than there
is a "right" fruit-apples, oranges, passion fruit. What you eat is a matter of personal
taste, availability, prce, history, and preference. Since you are also serving others
(your doctoral committee), their preferences come into play, as you well know. Each
tradition of qualitative inquiry offers a different emphasis, framework.or focus. I am
reluctant to offer a recommendation about which tradition fits your work best, but

(continued)
78 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

EXHIBIT 3.1 Continued

until the nextedition ofmy book is out with a new chapter that sorts through thevari-
ous traditions, i feel obliged to offeryou some guidance. So, here are three alternatives
to consider.
First, because you portray yourself as a pragmatic, experienced practitioner, you
could frameyour5tudy as qualitative, utilization-focused evaluation research [Patton
1997a]. You have to specify intended users for your study (for example, innovative
teachers and curriculum designers) and intended uses (facilitating discussing about
"effective practices"). This puts you in a tradition of generating practical and useful
knowledge for action in the tradition of refiective practice [Schon 1983]. Yourfocus
would be perceived patterns of effectiveness.
If that doesn't work and your committee insists on a more explicitly phiiosophical
or theoretical framework for your inquiry, you might consider either "social
constructionism" or "realism," which are two of the most general (and contradictory)
of the traditions informing qualitative inquiry. I must warn you that there are compet-
ing versions of constructionism and realism (academics without arguments are like
paraders without costumes or sports teams without u n i f o r m s - i f s how the players
differentiate themselves and figure out who to applaud). Either of these traditions will
guide you in thinking about how people in particular contexts (in your case, schools)
individualy and collectiveiy construct meaning and knowledge (in your case, effective
or "best" practices).
The third alternative involves a change of topic, which may sound like bad news.
The good news isthatyouVe already collected a lot ofthe data. You could do a disser-
tation on the social constructions of qualitative paradigms using your professors as
subjects. Obviously, you've already been doing participant observation on this topic.
Or you might do a hermeneutic study of qualitative terminology. Or a phenom-
enological study on the experience of graduate students trying to frame a qualitative
study. Or a heuristic inquiry into your own experience of qualitative design. Or but
thafs where you started out, sn't it.
Best wishes, whateveryou decide.
Michael Quinn Patton

affect the analytical framework that guides This chapter will be of particular interest
fieldwork and interpretation. Understand- to social scientists conducting basic or ap-
ing the divergent theoretical and phiiosoph- plied research, and students doing disserta-
ical traditions that have influenced qualita- tions, because their work is typically based
tive inquiry is especially important in the on and aimed at contributing to theory. The
design stage when the focus of fieldwork next chapter, in contrast, will focus on practi-
and interviewing is determined. Weaving cal and concrete evaluation and action re-
together theory-based inquiry traditions search questions appropriate for qualitative
and qualitative methods will reveal a rich inquiry, though theoretical understandings
tapestry with many threads of differing tex- can be important for practitioners and pol-
ture, color, length, and purpose. icy analysts because "theoretical concep-
Variety in Qualitative Inquiri/ J. 79

tions shape public arguments, giving people their introduction to the Handbook of Qualita-
the concepts they use and shaping the alter- tive Research trace six phases of qualitative
na tives they consider" (Nussbaum 2001:35). research history that help explain the dra-
Taken together, this chapter and the next of- matically varying conceptions of what con-
fer a broad range of goals for and ap- stitutes qualitative research.
proaches to qualitative research. Chapter 5
will then integrate theoretical and practical 1. During the "traditional period" of colo-
concerns by introducing a typology of re- nial research (up to World War II),
search purposes to elucidate the design, ethnographers, influenced by positiv-
methods, and analysis implica tions of vary- ism, strove for objectivity in their field-
ing purposes. work and reports.

2. The "modernist phase" (to the 1970s)

Alternative Ways was a time in which qualitative re-
of Distinguishing searchers emphasized methodological
Qualitative Traditions rigor and procedural formalism as they
sought acceptance within social science
There is no definitive way to categorize and reacted against postpositivism's
the various philosophical and theoretical emergent emphasis on interpretivism.
perspectives that have influenced and that
distinguish types of qualitative inquiry. Lin- 3. During the "blurred genres phase"
coln and Guba (2000) identify five "alterna- (1970-1986), a large number of alterna-
tive inquiry paradigms": positivism, post- tive approaches emerged, creating com-
positivism, criticai theory, constructivism, peti tion and confusion, the legacy of
and participatory. Schwandt (2000) dis- which remains in the daunting jargon
cusses "three epistemological stances for and labels of qualitative perspectives:
qualitative inquiry: interpretivism, herme- structuralism, symbolic interactionism,
neutics, and social constructionism." Crotty phenomenology, ethnomethodology, crit-
(1998) also offers three primary epistemo- icai theory, semiotics, neopositivism,
logical influences: objectivism, construc- micro-macro descriptivism, neo-Marx-
tionism, and subjectivism; these, he posits, ism, poststructuralism, naturalism, con-
have influenced in varying degrees different structionism, and deconstructionism.
theoretical perspectives: positivism (and
postpositivism), interpretivism (symbolic 4. Next carne the "crisis of representation"
interaction, phenomenology, hermeneu- that focused on issues of reflexivity,
tics), criticai inquiry, feminism, and post- power, privilege, race, gender, and so-
modernism. Creswell (1998) distinguishes cioeconomic classali of which under-
"five qualitative traditions of inquiry": biog- mined traditional notions of validity
raphy, phenomenology, grounded theory, and neutrality.
ethnography, and case study.
While there is some overlap among these 5. The "fifth moment" describes recent
frameworks, there are also important differ- history and is characterized as "a triple
ences reflecting varying experiences with crisis of representation, legitimation,
and emphases within the history of qualita- and praxis" (p. 17) in which the inevita-
tive research. Denzin and Lincoln (2000b) in bly creative and interpretive nature of
80 [. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

qualitative writing is put under the mi- plines given birth by the mother of ali dis-
croscope, mcluding the perspective of ciplines, philosophy, can be distinguished
the qualitative writer, and searching by their core burning questions. For sociol-
questions are raised about how to evalu- ogy, the burning question is the Hobbesian
ate the quality of qualitative research question of order: What holds society or so-
and evaluation. During this period, cial groups together? What keeps them from
more activist, explicitly political, and falling apart? Psychology asks: Why do in-
participatory approaches sought legiti- dividuais think, feel, and act as they do? Po-
macy as, for example, in "empower- litical science asks: What is the nature of
ment evaluation" (Fetterman, Kaftar- power, how it is distributed, and with what
ian, and Wandersman 1996) and using consequences? Economics studies how re-
qualitative/interpretive writing "to ad- sources are produced and distributed.
vance the promises of radical demo- Disciplines and subdisciplines reveal lay-
cratic racial justice embodied m the ers of questions. Biologists inquire into the
post-civil rights, Chicana/Chicano and nature and varie ty of life. Botanists ask how
Black Arts Aesthetic movements" (Den- plants grow, while agriculturists investigate
zin 2000a:256). producing food, and agronomists narrow
their focus still further to field crops.
6. In the sixth phase, which Denzin and To be sure, reducing any complex and
Lincoln call "p os texperimental," the multifaceted discipline to a singular burning
boundaries of qualitative inquiry are ex- question oversimplifi.es. But what is gained
panded to include creative nonfiction, are clarity and focus about what distin-
autobiographical ethnography, poetic guishes one lineage of inquiry from another.
representations, and multimedia pre- It is precisely that clarity and focus I shall
sentations. strive for in identifying the burning ques-
tions that distinguish major lineages of
They clearly expect qualitative inquiry to qualitative inquiry. In doing so, I shall dis-
continue developing in new directions for please those who prefer to separate para-
they call the future the "seventh moment" digms from philosophies from theoretical
or perhaps this will be the moment of rest, orientations from design strategies. For ex-
when qualitative researchers cease debating ample, social constructivism may be viewed
their differences and celebrate the marvel- as a paradigm, ethnography may be consid-
ous varie ty of their creations. ered a research strategy, and symbolic
interactionism may be examined as a theo-
Foundational Questions retical framework. However, distinctions
between paradigmatic, strategic, and theo-
This chapter, in contrast with the work of retical dimensions within any particular
qualitative theorists and historians cited approach are both arguable and somewhat
above, distinguishes theoretical perspectives arbitrary. Therefore, I have circumvented
by their foundational questions. A founda- those distinctions by focusing on and distin-
tional or burning question, like the mythical guishing foundational questions as the basis
burning bush of Moses, blazes with heat for understanding and contrasting long-
(controversy) and light (wisdom) but is not standing and emergent qualitative inquiry
consumed (is never fully answered). Disci- approaches.
Variety in Qualitative Inquiri/ J. 81

Theoretical Traditions Anthropologists have traditionally stud-

and Orientations ied nonliterate cultures in remote settings,
what were often thought of as "primitive" or
"exotic" cultures. As a result, anthropology
Ethnography and ethnographers became intertwined
with Western colonialism, sometimes resist-
ing imperialism in efforts to sustam native
Foundatio^ql question: cultures and sometimes as handmaidens to
W h a t is the CM l+U y-e. of +his gfowp conquering empires as their findings were
oj' people? used to overcome resistance to change and
manage subjugated peoples.
Ethnography, the primary method of an- Modern anthropologists apply ethno-
thropology, is the earliest distinct tradition graphic methods to the study of contem-
of qualitative inquiry. The notion of culture porary society and social problems, for ex-
is central to ethnography Ethnos is the Greek ample, technological diffusion, globaliza-
word for "a people" or cultural group. The tion, environmental degradation, poverty,
study of ethnos then, or ethnography, is "de- the gap between rich and poor, and societal
voted to describing ways of life of human- breakdown (Scudder 1999); education (Spind-
kind . . . , a social scientific description of a ler and Hammond 2000); addiction (Agar
people and the cultural basis of their 1986; Agar and Reisinger 1999); child labor
peoplehood" (Vidich and Lyman 2000:38). (Kenny 1999); intercultural understanding
Ethnographic inquiry takes as its central and in schools (Jervis 1999); and international
guiding assumption that any human group border conflicts (Hart 1999), to give but a
of people interacting together for a period of few of many examples. The importance of
time will evolve a culture. Culture is that col- understanding culture, especially in relation
lection of behavior patterns and beliefs that to change efforts of ali kinds, is the corner-
constitutes "standards for deciding what is, stone of "applied ethnography" as it has
standards for deciding what can be, stan- emerged in modern society (Chambers
dards for deciding how one feels about it, 2000). This can be seen in the ongoing re-
standards for deciding what to do about it, ports of members of the Society for Applied
and standards for deciding how to go about Anthropology since its founding in 1941.
doing it" (Goodenough 1971:21-22). The pri- Whyte (1984), for example, has collected a
mary method of ethnographers is partici- number of classic examples of ethnographic
pant observation in the tradition of anthro- fieldwork applied to problems of industrial
pology. This means intensive fieldwork in democracies.
which the investigator is immersed in the Since the 1980s, understanding culture
culture under study. While ethnographers has become central in organizational studies
share an interest in culture, there is debate (Morgan 1986, 1989; Pettigrew 1983) and in
about the nature of its essence (Douglass much organizational development work
2000) as well as several different styles of (Raia and Margulies 1985; Louis 1983), in-
ethnography, including the classic holistic cluding major efforts to change an organiza-
style of Benedict and Mead, the semiotic tion's culture (Schein 1985; Silverzweig and
style of Boas and Geertz, and the behaviorist Allen 1976). Organizational ethnography
style of the Whitings (Sanday 1983). has a distinguished history that can be
82 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

Cross-cultural perspective

traced back to the influential Hawthome tion (Fetterman 1984,1989) and applied edu-
electric plant study that began in 1927 cational research (Dobbert 1982). Programs
(Schwartzman 1993). Ethnography has also develop cultures, just as organizations do.
emerged as an approach to program evalua- The program's culture can be thought of as
Variety in Qualitative Inquiri/ J. 83

Culture, Culture Everywhere: Sample of Media

E X H I B I T 3.2 Headlines

"Stopping the Culture of Violence"

(appied alternatively to gangs, families, neighborhoods, television shows, hockey games,
even politics)
"Inside the Culture of Sports"
(Nike culture, Little League culture, "soccer mom" culture)
"Learn the Culture of Day-Trading Stocks"
(virtual trading culture, online business culture)
"Girls and the Barbie Doll Culture"
(or boys and the G.l. Joe action figure culture, kids and the Stor Wars culture)
"Eat Right: Fast Food Culture"
("Golden Arches Culture Around the World")
"The Culture of Negative Political Campaigning: Why It Wins"
Mu5ic, dance, or art culture
Postmodern culture
Virtual culture (aka Internet or Web culture)

part of the program's treatment. As such, the thinking about ethnographic research
culture affects both program processes and (LeCompte and Schensul 1999), Living the
outcomes. Improving a program, then, may Ethnographic Life (Rose 1990), Selecting
include changing the program's culture. An Ethnographic Informants (Johnson 1990), and
ethnographic evaluation would both facili- how to write ethnographies (Atkinson 1992)
tate and assess such change. or write the methods section in ethno-
Ironically, perhaps, awareness of the im- graphic reports (Stewart 1998). The Ethnog-
portance of culture has found its way into rapher's Toolkit hasbeenpublished (Schensul
popular culture and mass media to such an and LeCompte 1999).
extent that the term shows up nearly ubiqui- While traditionally ethnographers have
tously as an implied explanation for ali used the methods of participant observation
kinds of social problems and phenomena, as and intensive fieldwork to study everything
shown in Exhibit 3.2. from small groups to nation-states, what it
Ethnographic methods continue to de- means to "partcipate" or be in the "field"
velop as new approaches emerge, for exam- or even be a "group" has changed with
ple, Doing Team Ethnography (Erickson and the World Wide Web and the emergence of
Stull 1998), and new issues surface, for ex- the virtual ethnographerstudying people
ample, Ethnographic Decision Tree Modeling connected through distributed electronic
(Gladwin 1989) or Writing the New Ethnogra- environments (Ruhleder 2000). Neverthe-
phy (Goodall 2000). Other ethnographic less, whether doing ethnography in virtual
methodologists continue to delve deeply space, a nonliterate community, a multina-
into classic issues such as paradigms for tional Corporation, or an inner-city school,
84 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

what makes the approach distinct is the mat- uations, the other became the program cli-
ter of interpreting and applying the findings ent, the student, the welfare recipient, the
from a cultural perspective (Wolcott 1980:59; patient, the alcoholic, the homeless person,
Chambers 2000:852). the victim, the perpetrator, or the recidivist.
In organizational studies, the other was the
worker, the manager, the leader, the fol-
Autoethnography and
lower, and/or the board of directors. The
Evocative Forms of Inquiry
others were observed, interviewed, de-
scribed, and their culture conceptualized,
analyzed, and interpreted. Capturing and
PouiAda-Hcmal question;
being true to the perspective of those stud-
How does my own e^cpenence of
ied, what came to be called the emic per-
tkis cw Itu counect witk and offe.?
spective, or the insider's perspective, was
insigkts about+kis culture, situa+ion,
contrasted with the ethnographer's per-
eveiAf> and/ot* way of life?
spective, the etic, or outsider's, view. The etic
viewpoint of the ethnographer implied
We turn now from the earliest qualitative some important degree of detachment or
tradition, ethnography, to the lates t and still "higher" levei of conceptual analysis and
emergent approach: autoethnography. Eth- abstraction. To the extent that ethnog-
nography and autoethnography might be raphers reported on their own experiences
thought of as bookends, or opposite ends of as participant observers, it was primarily
a qualitative continuum, that frame a large methodological reporting related to how
number of distinct qualitative approaches to they collected data and how, or the extent to
be reviewed in this chapter. By considering which, they maintained detachment. To "go
them one after the other throughout this native" was to lose perspective.
chapter, it is hoped you'11 get a sense of the In the new postcolonial and postinodern
range of issues that distinguish qualitative world at the beginning of the 21 st century,
approaches. the relationship between the observed and
Ethnography first emerged as a method the observer has been called into question at
for studying and understanding the other. It every levei. Postcolonial sensitivities raise
was fascination with "exotic otherness" that questions about imbalances of power,
attracted Europeans to study the peoples of wealth, and privilege between ethnog-
frica, Asia, the South Sea Islands, and the raphers and those they would study, includ-
Amricas. "The life world of the 'primitive' ing criticai political questions about how
was thought to be the window through findings will be used. Postmodern critiques
which the prehistoric past could be seen, de- and deconstruction of classic ethnographies
scribed, and understood" (Vidich and have raised fundamental questions about
Lyman 2000:46). In the United States, for ed- how the values and cultural background of
ucated, White, university-based Americans the observer affect what is observed while
the others were Blacks, American Indians, re- also raising doubts about the desirability, in-
cent immigrants, working-class families, deed, the possibility, of detachment. Then
and the inner-city poor (and for that matter, there is the basic question of how an ethnog-
anyone else not well educated, White, and rapher might study her or his own culture.
university based). In recent times, when eth- What if there is no other as the focus of study,
nography began to be used in program eval- but I want to study the culture of my own
Variety in Qualitative Inquiri/ J. 85

BMaiiii^lictf Varieties of Autoethnog raphy: A Partial Lexicology

David Hayano (1979) is credited with originating the term autaethnographyto describe studies
by anthropoogists of their own cultures. In their extensive review, Ellis and Bochner (2000) focus
on studying one's own culture and oneself as part of that culture to understand and illuminate a
way of life. They cite a large number of phrases that have emerged both to support this emergent
frontierof qualitative inquiry and to confuse exactly whatitis. In the end, they conclude, "increas-
ingly, autoethnography has become the term of choice in describing studies and procedures that
connect the personal to the cultural" (p. 740).
Other terms include

Autobiographical ethnography Lived experience

Auto-observation Literary ethnography
Ethnographic poetics Narrative ethnography
Creative analytic practice ethnography Native ethnography
Criticai autobiography Narratives of the self
Ethnic autobiography New ethnography
Ethnographic memoir Personal ethnography
Ethnobiog raphy Personal experience narratives
Ethnographic autobiography Personal narratives
Ethnographic stories Postmodern ethnography
Evocative narratives Reflexive ethnography
Experimental ethnography Self-ethnog raphy
First-person accounts Self-stories
Indigenous ethnography Socioautobiog raphy
Interpretive biography Sociopoetics

group, my own community, my own organi- Autoethnography is an autobiographical

zation, and the way of life of people like me, genre of writing and research that displays
or people I regularly encounter, or my own multiple layers of consciousness, connecting
cultural experiences? the personal to the cultural. Back and forth
These developments have contributed to autoethnographers gaze, first through an
the emergence of autoethnographystudy- ethnographic wide-angle lens, focusing out-
ing one's own culture and oneself as part of ward on social and the cultural aspects of their
that cultureand its many variations. personal experience; then, they look mward,
Goodall (2000) calls this the " n e w ethnogra- exposing a vulnerable self that is moved by
phy": "creative narratives shaped out of a and may move through, refract, and resist cul-
writer's personal experiences within a cultural interpretations. As they zoom backward
ture and addressed to academic and public and forward, inward and outward, distinc-
audiences" (p. 9). Exhibit 3.3 offers a list of tions between the personal and cultural be-
many, but not ali, of the terms that have come blurred, sometimes beyond distinct
emerged to describe variations in this gen- recognition. Usually written in first-person
eral approach. Carolyn Ellis (Ellis and voice, autoethnographic texts appear in a vari-
Bochner 2000) describes it this way: ety of formsshort stories, poetry, fiction,
86 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

novis, photographic essays, personal essays, sional life. The point is that, for an ethnogra-
journals, fragmented and layered writing, and pher, any experienceat home or abroad, of
social science prose. In these texts, concrete ac- self or of otheroffers the potential to become
tion, dialogue, emotion, einbodiment, spiritu- fieldwork. . . . For me, my personal and my
ality, and self-consciousness are featured, ethnographic persona have become so inter-
appearing as relational and institutional sto- twined that it would be impossible to separate
ries affected by history, social structure, and them even if I wanted to do so. (p. 317)
culture, which themselves are dialectically re-
vealed through action, feeling, thought, and
Anthropologist Mary Catherine Bate-
language. (p. 739)
son's (2000) autoethnographic description
of teaching a seminar at Spelman College in
In autoethnography, then, you use your
Atlanta, Gergia, includes detailed attention
own experiences to gamer insights into the
to the personal challenge she experienced in
larger culture or subculture of which you are
trying to decide how to categorize students
a part. Great variability exists in the extent to
of different ages in contemporary American
which autoethnographers make themselves
universities, for example, by calling older
the focus of the analysis, how much they
participants "elders." Aaron Tumer (2000)
keep their role as social scientist in the fore-
of Brunel University in the United Kingdom
ground, the extent to which they use the sen-
has explored using one's own body as a
sitizing notion of culture, at least explicitly,
source of data in ethnography, what he calls
to guide their analysis, and how personal
"embodied ethnography."
the writing is. At the center, however, what
Such personal writing is controversial
distinguishes autoethnography from eth-
among qualitative theorists because of its
nography is self-awareness about and re-
"rampant subjectivism" (Crotty 1998:48).
porting of one's own experiences and intro-
Many social science academics object to the
spections as a primary data source. Ellis
way it blurs the lines between social science
describes this process as follows:
and literary writing. One sociologist told me
angrily that those who want to write creative
I start with my personal life. I pay attention to nonfiction or poetry should find their way to
my physical feelings, thoughts, and emotions. the English Department of the university
I use what I call systematic sociological intro- and leave sociology to sociologists. Richard-
spection and emotional recall to try to under- son (2000b), in contrast, sees the integration
stand an experience I've lived through. Then I of art, literature, and social science as pre-
write my experience as a story. By exploring a cisely the point, bringing together creative
particular life, I hope to understand a way of and criticai aspects of inquiry. She suggests
life. (Ellis and Bochner 2000:737) that what these various new approaches and
emphases share is that "they are produced
In writing about his experiences in a through creative analytic practices," which
"New Age ashram located in Pennsylva- leads her to call "this class of ethnographies
nia," Bruner (1996) confronted the intersec- creative analytic practice ethnography" (Rich-
tion of the ethnographic and the personal: ardson 2000b:929). While the ethnographic
aspect of this work is constructed on a foun-
What started out as part of my personal life dation of careful research and fieldwork
was soon transformed mto part of my profes- (p. 937), the creative element resides primar-
Variety in Qualitative Inquiri/ J. 87

ily in the writing, which she emphasizes is, a credibleaccount of a cultural, so-
itself, "a method of inquiry, a way of finding cial, individual, or communal sense of
out about yourself and your topic" (p. 923). the "real"? (Richardson 2000a:254,
But how is one to judge the quality of such 2000b :937)
nontraditional social scientific approaches
that encourage personal and creative eth- These criteria open up the possibility of
nographic writing? Richardson (2000b) has new writing formats. Elliot Eisner (1996), a
responded to this challenge by asserting that former president of the American Educa-
creative analytic practice ethnography should tional Research Association, has argued that
be held to "high and difficult standards; a novel as a form of qualitative repor ting
mere novelty does not suffice" (p. 937). She could be a legitimate form for a doctoral dis-
offers five criteria of quality drawn from sertation in social science or education. In
both science and creative arts. that vein, he has suggested that in "the new
frontier in qualitative research methodol-
1. Substantive contribution: Does this piece ogy" an artistic qualitative social science
contribute to our understanding of so- contribution canbe assessed by the "number
cial life? Does the writer demonstrate a and quality of the questions that the work
deeply grounded (if embedded) social raises" as much as by any conclusions of-
scientific perspective? How has this per- fered (Eisner 1997:268). In this regard, emi-
spective informed the construction of nent evaluator Emie House (1991) reminds
the text? us that where evaluation reports are con-
cerned, the possibility of fiction is always a
2. Aesthetic merit: Does this piece suc- subtext: "Our evaluation repor t proved to be
ceeded aesthetically? Does the use of so readable many people became enrap-
creative analytic practices open up the tured by it. Some said it read like a novel.
text, invite interpretive responses? Is the Others said it was a novel" (p. 113).
text artistically shaped, satisfying, com- Poetry is another artistic genre that has
plex, and not boring? emerged in ethnographic reporting. Glesne
(1997) converted interview transcripts into
3. Reflexivity: How has the author's sub- poems because she found poetry better cap-
jectivity been both a producer and a tured and communicated what her inter-
product of this text? Is there adequate view with an 86-year-old professor in Puerto
self-awareness and self-exposure for the Rico opened up and revealed. Richardson
reader to make judgments about the (1998) has published a number of field-
point of view? work-based poems, reflecting his view that
poetry offers a language especially
4. Impact: Does this affect me? Emo- well-suited "for those special, strange, even
tionally? Intellectually? Does it generate mysterious moments when bits and pieces
new questions? Move me to write? suddenly coalesce . . . , when the ethnogra-
Move me to try new research practices? pher, away from home and in a strange cul-
Move me to action? ture, has a heightened sense of the frailty of
being human. In such a sense, poetry ap-
5. Expression of a reality: Does this text em- pears to be a way of communicating in-
body a fleshed out, einbodied sense of stances when we feel truth has shown its
lived experience? Does it seem a "true" face" (p. 451). Travisano (1998) included po-
88 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

dictions they experience. Ironically, many

aren't observant enough of the world around
them. The self-questioning autoethnography
demands is extremely difficult. So is confront-
|||S||lfi:i||l:]- =; i' ai !:-.;!::vii;;j-' m ^-HS-I! Y '.; ing things about yourself tliat are less than
flattering. Believe me, honest autoethnog-
! m^Tlfety %S|la iftffi ik >p> raphy exploration generates a lot of fears and
y p ^ ^ ^ k s m m !
M!!: !;': ';!!]'!'irj =!;! | doubtsand einotional pain. Just when you
-/'jfufiifc l^ii'!!!. th - Wffflffijriliil liiiTil !tf i: think you can't stand the pain anymore, well,
that's when the real work has only begun.
lffi^ :; Then there's the vulnerability of revealing
^^jriliiiiiii:^:iij... . ; . . ' ;:. j yourself, not being able to take back what
you've written or having any control over
how readers interpret it. lt's hard not to feel
your life is being critiqued as well as your
etry in his "autobiography of an ethnic iden- work. I t c a n b e humiliating. And the ethical is-
tity" in which he explored his lived experi- sues. Just wait until you've written about fam-
ence of becoming Italian American. ily members and loved ones who are part of
These new frontiers of qualitative inquiry your story. (p. 738)
and reporting, combining art and science,
are also leading to the integration of multi- Part of the challenge in autoethno-
ple forms in a single work. Consider Den- graphic writing is finding and owning one's
zin's (2000b) "Rock Creek History/' which voice. In the last chapter, I contrasted the
he describes as "an experimental, mixed- third-person passive voice of traditional ac-
genre narrative, combining autoethnog- ademic writing with the first-person active
phy with other evocative writing forms, voice of qualitative inquiry. Autoethnog-
including narratives of the self. Using the raphy increases the importance of voice and
techniques of fiction, I tell a story about my- raises the stakes because an authentic voice
self and my experiences with nature, the sa- enhances the authenticity of the work, while
cred, and a small Montana River named an inauthentic voice undermines it. Voice re-
Rock Creek" (p. 71). He also calls his writing veals the author's identity (Ivanic 1998). The
a "performance-based project" that draws tone of voice may be expressive, reflective,
on multiple writing forms and traditions in- searching, academic, or criticai, as in what
cluding, in addition to those already noted, Church (1995) has called the "forbidden
"the ethnographic story, nature writing, lit- narratives" of "criticai autobiography" in
erary nonfiction, the personal memoir, and social science. In voice resides Richardson's
cultural criticism" (p. 79). (2000b) fifth criterion for judging quality
Ellis (Ellis and Bochner 2000) warns that cited earlier, what she called "expression of a
autoethnographic writing is hard to do: reality: Does this text embody a fleshed out,
embodied sense of lived experience? Does it
It's amazingly difficult. lt's certainly not seem a 'true'a credibleaccount of a cul-
something that most people can do well. Most tural, social, individual, or communai sense
social scientists don't write well enough carry of the 'real'?" (p. 937).
it off. Or they're not sufficiently introspective These issues are being raised in a number
about their feelings or motives, or the contra- of disciplinary genres. Historian Edmund
Variety in Qualitative Inquiri/ J. 89

Morris won a Pulitzer Prize for his biogra- doubts, weaknesses, and uncertainties. But
phy of America's 26th president, Theodore once the story was told, the final chapter of
Roosevelt. Partly on this basis, he was cho- the book that contrasts alternative coming-
sen as the "official" biographer of former of-age/initiation paradigms (Exhibit 1.1 in
president Ronald Reagan. The resulting Chapter 1) emerged relatively painlessly.
work (E. Morris 2000) proved highly contro- I've included as Appendix 3.1 at the end of
versial because to tell the story of Reagan's this chapter an excerpt from the book as an
life as he felt it needed to be told, he created a example of autoethnographic writing.
fictional character based on himself and fab- Johnstone (2000) argues that "interest in
ricated encounters with Reagan at various the individual voice" within anthropology
points that led him to first-person reflections can be understood, at least in part, "within
as if he had actually witnessed and partici- the context of a larger shift toward a more
pated in these events and encounters. phenomenological approach to language"
Thus, a traditional and highly respected (p. 405). Autoethnography integrates eth-
historian introduced a form of quasi- nography with personal story, a specifically
autoethnographic literary fiction into a stan- autobiographical manifestation of the more
dard biography in order to have a point general "turn to biographical methods in so-
of view from which to recount his subject's cial science" that strive to "link macro and
life. micro leveis of analysis . . . [and] provide a
In my own major effort at autoeth- sophisticated stock of interpretive proce-
nographic inquiry (Patton 1999a), the strug- dures for relating the personal and the so-
gle to find an authentic voiceauthentic cial" (Chamberlayne, Bornat, and Wengraf
first to me, then to others who know me, and 2000:2-3). Art Bochner (Ellis and Bochner
finally to those who do not know me 2000) has reflected on what this means:
tumed what I thought would be a one-year
effort into seven years of often painful, dis-
couraging writing. And I was only writing What is the point of a storied life? Narrative
about a 10-day period, a Grand Canyon hike truth seeks to keep the past alive in the pres-
with my son in which we explored what it ent. Stories show us that the meanings and sig-
means to come of age, or be initiated into nificance of the past are incomplete, tenta tive,
adulthood, in modern society. My son and revisable according to contingencies of
started and graduated from college while I our present life circumstances, the present
was learning how to tell the story of what we from which we narrate. Doesn't this mean that
experienced together. To make the story the stories we tell always run the risk of dis-
work as a story and make scattered interac- torting the past? Of course, it does. After ali,
tions coherent, I had to rewrite conversa- stories rearrange, redescribe, invent, omit, and
tions that took place over several days into a revise. They can be wrong in numerous
single evening's dialogue, I had to reorder waystone, detail, substance, etc. Does this
the sequence of some conversations to en- attribute of storytelling threaten the project of
hance the plot line, and I had to learn to fol- personal narrative? Not at ali, because a story
low the novelist's mantra to "show don't is not a neutral attempt to mirror the facts of
tell," advice particularly difficult for those of one's life
us who make our living telling. More diffi- Life and narrative are inextricably con-
cult still was revealing my emotions, foibles, nected. Life both anticipates telling and draws
90 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

No, Mom, I didn't say that you're a bad mother.

I know false consciousness sounds negative, but
don't take it personal. That's just the deconstruction
chapter. In the next chapter I do reconstruction and
creative synthesis. I think you'll really resonate to
the synergistic portrayal of an archetypal maternal
figure in the postmodern era of conflicted roles. You
just have to get into the mindset of autoethnography.

Confronting a critic of autoethnography

Variety in Qualitative Inquiri/ J. 91

meaning from it. Narrative is both about liv- What these questions have in common is
ing and part of it. (pp. 745-46) the presumption that there is a real world
with verifiable patterns that can be observed
By opening this chapter with the contrast and predictedthat reality exists and truth
between ethnography and autoethnog- is worth striving for. Reality can be elusive
raphy, we have moved from the beginnings and truth can be difficult to determine, but
of qualitative methods in anthropological describing reality and detennining truth are
fieldwork more than a century ago, where the appropriate goals of scientific inquiry.
the ethnographer was an outsider among Working from this perspective, researchers
exotically distinct nonliterate peoples, to and evaluators seek methods that yield cor-
the most recent manifestation of qualitative respondence with the "real world/' thus this
inquiry in the postmodern age of mass is some times called a correspondence per-
communications, where autoethnographers spective.
struggle to find a distinct voice by docu- Reality-oriented inquiry and the search
menting their own experiences in an increas- for truth have fallen on hard times in this
ingly all-encompassing and commercialized skeptical postmodern age when honoring
global culture. To further sharpen contrasts multiple perspectives and diverse points of
in qualitative approaches, the next two sec- view has gained ascendancy in reaction to
tions illuminate some of the philosophical the oppressive authoritarianism and dog-
underpinnings that have informed and matism that seemed so often to accompany
shaped qualitative methods, including eth- claims of having found "Truth." Yet, many
nography and autoethnography, by con- people, especially policymakers and those
trasting the foundational question of real- who commission evaluation research, find it
ity-oriented research and evaluation (post- difficult to accept the notion that ali explana-
positivist realism) with that of postmodern tions and points of view hold equal merit.
constructivism and social construction. Some people in programs seem to be helped
more than others. Some students seem to
learn more than others. Some claims of effec-
Truth and Reality-Oriented tiveness are more plausible and have more
Correspondence Theory: merit than others. To test a claim of effective-
Positivist, Realist, and Analytic ness by bringing data to bear on it, including
Induction Approaches qualitative data, is to be engaged in a form of
reality testmg that uses evidence to examine
T-QufAclaticmai CjuestioKvss assertions and corroborate claims. In this
Wka+'s êally going on in fea 1
section, we shall examine how to recognize
wo^ld? What ccm we establish with and engage in a reality-testing or reality-ori-
some decj*ee oj- ceH-aitrty? ented approach to qualitative inquiry. In so
Wkat doing, I shall minimize philosophical dis-
ciê plausible expla^cî^ms for course and focus primarily on the practical
veî implications of this orientation, but a brief
fiable pa+terns ? VvWs the foray into philosophical foundations is nec-
tûik msoj-ar os we get at it?
CQKV
essary to provide a context for practice.
"H
Philosophical inquiry into truth and real-
ow can we study o, pke.Kvomenon ity involves examining the nature of knowl-
so that out* f-indmgs coyyespond/
inso-fak1 as it's possible, to tke.
êal world?
92 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

edge itself, how it comes into being and is paradigm debates and routinely used incor-
transmitted through language. Positivism, rectly though persistently. Shadish (1995b)
following Auguste Comte, asserted that argues that one would be hard-pressed to
only verifiable claims based directly on ex- find any contemporary social scientist, phi-
perience could be considered genuine losopher, or evaluator who really adheres to
knowledge. Comte was especially inter- the tenets of logical positivism. Rather, "the
ested in distinguishing the empirically term has become the Iinguistic equivalent
based "positive knowledge" of experience of 'bad/ a rhetorical device aimed at depriv-
from theology and metaphysics, which ing one's opponent of credibility by name-
depended on fallible human reason and calling. This is particularly true m the
belief. Logical positivism, developed by the quantitative-qualitative debate where some
Vienna Circle in ustria and the Berlin qualitative theorists are fond of labeling ali
School in Germany in the early part of the quantitative opponents as logical positiv-
20th century, added to the emphasis on di- ists," a fundamental but common "error"
rect experience from positivism a logic-based (p. 64).
commitinent "to theory development using Logical empiricism and postpositivism,
a rigorous procedural language such as which take into account the criticisms
symbolic logic. Knowledge comes either against and weaknesses of rigid positivism,
from direct experience or indirectly from in- now inform much contemporary social sci-
ferences from experience through the proce- ence research, including reality-oriented
dural language" (Shadish 1995b:64). Logical qualitative inquiry. Logical empiricism, a
positivism subsequently came to be associ- more modera te version of logical positivism
ai ed with philosophical efforts to specify ba- (Schwandt 2001), seeks unity in science,
sic requirements for what could be consid- through both theory formulation and meth-
ered scientific knowledge, which mcluded odological inquiry, and asserts that there are
the search for universal laws through empir- no fundamental methodological differences
ical verification of logically deduced hy- between natural and social sciences.
potheses with key concepts and variables Postpositivism, as articulated by eminent
operationally defined and carefully formu- methodologist Donald T. Campbell in his
lated to permit replication and falsification. collected writings about and vision for an
Thus, real knowledge (as opposed to mere "experimenting society" (Campbell and
beliefs) was limited to what could be logi- Russo 1999), recognizes that discretionary
cally deduced from theory, operationally judgment is unavoidable in science, that
measured, and empirically replica ted. Such proving causality with certainty in explain-
severe, narrow, and rigorous requirements ing social phenomena is problema tic, that
turned out to severely limit what could pass knowledge is inherently embedded in his-
for knowledge and to demand more torically specific paradigms and is therefore
certainty than the complex world of social relative rather than absolute, and that ali
phenomena could yield. Though influential methods are imperfect, so multiple meth-
in the first half of the 20th century, logical ods, both quantitative and qualitative, are
positivism has been "almost urversally re- needed to generate and test theory, improve
jected" as a basis for social science inquiry understanding over time of how the world
(Campbell 1999a:132). The legacy of the opera tes, and support informed policy mak-
fleeting influence of logical positivism is ing and social program decision making.
that the term lives on as an epithet hurled in While modest in asserting what can be
Variety in Qualitative Inquiri/ J. 9 3

known with any certainty, postpositivists do

assert that it is possible, using empirical evi-
COtmUCTED.
dence, to distinguish between more and less fe^feft.4:l^1vN3lM3i flEAUTY
plausible claims, to test and choose between
" T A C T I . T K - R O R N F ^ m T VI ' ' T ' ; , ti&i V ^ I , ! VI"
rival hypotheses, and to distinguish be-
tween "belief and valid belief" (Campbell .STj J:f !! ii!!.! :i'i Ur n?:r! .'!:,-iV. !! h YJ;. ;.!'=! '
1999b:151, emphasis added). : ;!!.!.::> :' R ^ Y ; A - A H -

!''!;; ,; {y RRN !.VY ri .V I !.s" :!I i' i i. u V?

Given this brief philosophical and episte-
!Vi!! ril!^! rh.:s!"!!!T!' h1'!s' f i^Viy t.VV ,'
mological overview, what are the practical
iriXi'!!. M " (!m i ;: a::^;;!;-!;
implications for qualitative inquiry of oper-
ating within a reality-oriented perspective?
It ineans using the language and concepts of
mainstream science to design naturalistic "truth value" and plausibility of findings;
studies, inform data gathering in the field, credibility, impartiality, and independence
analyze results, and judge the quality of of judgment; confirmability, consistency,
qualitative findings. Thus, if you are a re- and dependability of data; and explainable
searcher or evaluator operating from a real- inconsistencies or instabilities (GAO
ity-oriented stance, you worry about valid- 1987:53). You may even generalize case study
ity, reliability, and objectivity (e.g., Perkyl findings, depending on the cases selected
1997). You realize that completely value-free and studied, to generate or test theory (Yin
inquiry is impossible, but you worry about 1989:44, 1999b), establish causality (Ragin
how your values and preconceptions may 1987, 2000), or inform program improve-
affect what you see, hear, and record in the ment and policy decisions from patterns es-
field, so you wrestle with your values, try to tablished and lessons leamed (GAO
make any biases explicit, take steps to miti- 1987:51). In short, you incorporate the lan-
ga te their influence through rigorous field guage and principies of 21st-century science
into naturalistic inquiry and qualitative
procedures, and discuss their possible influ-
analysis to convey a sense that you are dedi-
ence m reporting findings. You may estab-
cated to getting as close as possible to what is
lish an "audit trail" to verify the rigor of your
really going on in whatever setting you are
fieldwork and confirmability of the data col-
studying. Realizing that absolute objectivity
lected because you want to minimize bias,
of the pur positivist variety is impossible to
maximize accuracy, and report impartially
attain, you are prepared to admit and deal
believing that "inaccuracy and bias are un-
with imperfectioris in a phenomenologically
acceptable in any case study" (U.S. General
messy and methodologically imperfect
Accounting Office [GAO] 1987:51). In re-
world, but you still believe that objectivity is
porting, you einphasize the empirical find-
worth striving for. As Kirk and Miller (1986)
ingsgood, solid description and analy-
assert,
sisnot your own personal perspective or
voice, though you acknowledge that some
subjectivity and judgment may enter in. You Objectivity, though the term has been taken by
include triangulation of data sources and some to suggest a naive and inhumane ver-
analytical perspectives to increase the accu- sion of vulgar positivism, is the essential basis
racy and credibility of findings (Patton of ali good research. Without it, the only rea-
1999b). Your criteria for quality include the son the reader of the research might have for
94 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

accepting the conclusions of the investigator rather than logical positivists, further evi-
would be an authoritarian respect for the per- dence that "the weight of criticisms" against
son of the author. Objectivity is a simulta- logical positivism has "caused its internai
neous realization of as much reliability and collapse" (Schwandt 2001:150). Realism as a
validity as possible. Reliability is the degree to qualitative stance is clearly reality oriented,
which the finding is independent of accidental and much of the language quoted above re-
circumstances of the research, and validity is mains in the revised edition. They acknowl-
the degree to which the finding is interpreted edge that knowledge is socially and histori-
in a correct way. (p. 20) cally constructed, and they "affirm the
existence and importance of the subjective,
In the introduction to their widely used the phenomenological, the meaning-mak-
and influential sourcebook Qualitative Data ing at the center of life." Then they return to
Analysis, Miles and Huberman (1984) stated their core reality-oriented stance:
modestly: "We think of ourselves as logical
Our aim is to register and "transcend" these
positivists who recognize and try to atone
processes by buildmg theories to account for a
for the limitations of that approach. Soft-
real world that is both bounded and perceptu-
nosed logical positivists, maybe" (p. 19).
ally laden, and to test these theories in our var-
They went on to explain what this means
ious disciplines.
and, in so doing, provide a succinct sum-
Our tests do not use "covering laws" or the
mary of the reality-oriented approach to
deductive logic of classical positivism. Rather,
qualitative research:
our explanations flow from an account of how
differing structures produced the events we
We believe that social pkenomena exist not observed. We aim to account for events, rather
orily in the mind but also in the objective than simply to document their sequence. We
worldand that there are some lawful and look for an individual or a social process, a
reasonably stable relationships to be found mechanism, a structure at the core of events
among them Given our belief in social reg- that canbe captured to provide a causai descrip-
ularities, there is a corollary: Our task is to ex- tion of the forces at work.
press them as precisely as possible, attending Transcendental realism calls both for
to their range and generality and to the local causai explanation and for the evidence to
and historical contingencies under which they show that each entity or event is an instance of
occur. that explanation. So we need not only an ex-
So, unlike some schools within social phe- planatory structure but also a grasp of the par-
nomenology, we consider it important to ticular configuration at hand. That is one
evolve a set of valid and verifiable methods for reason why we have tilted toward more in-
capturing these social relationships and their ductive methods of study. (Miles and Huber-
causes. We want to interpret and explain these man 1994:4)
phenomena and have confidence that others,
using the same tools, would arrive at analo- Analytic induction offers a specific form of
gous conclusions. (Miles and Huberman inductive analysis that begins deductively,
1984:19-20) by formulating propositions or hypotheses,
and then examines a particular case in depth
Ten years later, in their revised and ex- to determine if the facts of the case support
panded qualitative sourcebook, Miles and the hypothesis. If it fits, another case is stud-
Huberman called themselves "realists" ied, and so forth, in the search for generaliza-
Variety in Qualitative Inquiri/ J. 95

tions. If a case does not support the hypothe- such as logical positivism, postpositivism,
sis, that is, it is a negative case, the hypothesis logical empiricism, realism, transcendental
is revised. The aim is to explain a phenome- realism, and objectivisin are jargon-ish, have
non satisfactorily using qualitative, case- disputed definitions, and carry negative
based inquiry (Schwandt 2001; Vidich and connotations for many, so they come with
Lyman 2000:57-58; Ryan and Bernard 2000: lots of baggage. I have attempted to be de-
786-87; Gilgun 1995; Taylor and Bogdan scriptive about the reality-oriented, corre-
1984: 127-28). Chapter 8 on analysis dis- spondence theory perspective by focusing
cusses the analy tical strategies of analytic in- on its core, foundational questions as articu-
duction in more detail and provides exam- lated at the beginning of this section. While,
ples. as the next section will show, many qualita-
While analytic induction focuses on tive methodologists assert that qualitative
method, realism focuses first on philosophy. inquiry is inherently constructionist or phe-
Realist philosophy (Baert 1998:189-97; Put- nomenological in perspective, the reality-
nam 1987, 1990) has recently has been oriented perspective remains widespread,
adapted by Mark, Henry, and Julnes (2000) even dominant, in those arenas of research
and Pawson and Tilley (1997) as offering the practice where scientific credibility carries a
foundation for a reality-oriented approach premium. These arenas include many dis-
to evaluation research that includes qualita- sertation committees in traditional disci-
tive inquiry. plines where qualitative dissertations are
just beginning to be allowed, in summative
Realism presumes the existence of an externai
evaluation and policy studies where mere
world in which events and experiences are
"anecdotal" evidence is demeaned, and in
triggered by underlying (and often unob-
fields such as medicai research where dou-
servable) mechanisms and structures (Bhas-
ble-blind experimental studies remain the
kar, 1975). Commonsense realism also gives
gold standard. To emphasize this latter
standing to everyday experiences. It is
point, I close this review of reality-oriented
antiformalist in the sense of not expecting log-
qualitative inquiry with an excerpt from a
ical, formal solutions to vexing problems such
medicai journal in which health researchers
as the n ature of truth. And it places a priority
are defending qualitative research to an au-
on practice and the lessons drawn from prac-
dience known to be skep tical. Their ap-
tice. . . . As realists, we see no meaningful
proach is to associate qualitative research
epistemological difference between qualita-
closely with accepted and credible forms of
tive and quantitative methods. Instead we see
experimental research. Such a perspective
both as assisted sensemaking techniques that
epitomizes the reality-testing orientation:
have specific benefits and limitations. And as
commonsense realists, we believe that al-
What, then, does the qualitative researcher do
though there is a world out there to be made
once he or she accomplishes a careful and
sense of, the specific constructions and
trustworthy understanding of the language
construais that individuais make are criticai
and behavior of an individual human being?
and need to be considered. (Mark et al.
Here is where we rely on our positivist skills
2000:15-16)
and methods [OJnce we carefully examine
Throughout this section, I have used the and articulate that which we understand one
term reality-oriented qualitative inquiry to human being to be doing, we attempt to col-
describe this perspective because labels late the language and behavior of many hu-
96 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

man beings, so many that we might be able to

test the relationships, for example, between
fh; c c.;;, K M IVIT
setting andbehavior orbetween age and hope.
Included within the domains of qualitative !''i;: | ijV ;.Ti... ori '-iiviaviVi:;-

science or narrative research, then, are efforts i'i h n-i'! i)s !* jYVi ng y1 r-s:!.:; ;: !'::.I i.1 . ! i1
to generalize, to predict, and to relate initial ;.V!VF H/,!JJ=',:Y;;-.;= H:Z^KF.I/J,; !. A - A : I ' I ' , !

states to outcomes. These efforts require the !:' !i!'ii!!!!l ir!;!!!'i't"i!'""!;!!!::i;l!i!i Ntti ! if! iV-! j<: Tttti
same evidence-based activities that are used J-:!:!.'IWW !Y! :;=.I!\!:: IV-H-Y::':::: W ^ R L<I

in testing any hypotheses. (Charon, Greene, : tii-iir!!! I:!1!^!!:!^!!.:!' ftlM! Ji :!!:!!:];:! !';!:: iVi - 17i M ;1
and Adelman 1998:68) iVi ' . : I ! m n tf != i i * = a z i ' : ; I:=,-i - I;, ! , m l

ro tf:;" !>:;!:v!::;!::';!:!'! :::==h!;= -i*.ii

Social Construction ! " l l t e M T I J - ^ T ' ! IRL^iii. 1 ;ST: /n

and Constructivism Y :R . H ; : : ' I .:= = ' S I , .

I IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII^ ^IIIIIIIIIIIIIIIIII NU IIIIIHIIII III LI II II NII I SIIIHIIIIIIIJ

T-oundational questions: 'Vi' ! !'!:!:V:!! .:!:;'!: ','- !:- ! :'i! n ;i.i V - r ' V " :

How kave +ke people in +his setHng ::!' it i*..".rt; rr-.LVJ AL-V"" i.- /.u';
constructed reality? Wkat are tkeir :I M" IR;:' :;:: I^N-V. v i :. ! ' I1! '.1 |',V .I i V i V ! I. I- N I 'I"

^epo^ted perceptions, "trutks/' ; . T I F T F L C . Y I . ' I:M ! I ' ! I ! T . V \ | ; I = ! : , 1 V L I . - , ! ' . '

explanations, beliefs, and world- /. ^ I I ! ' ' ! * IY.VI.UY. ^ F C L V ! i i I ! ! R ! i.I i . v i IVR^IRM.

view? What ace tke consequences / S n u . " .'fli-i, 1

of tkeir constructions for tkei^

bekaviors and for tkose witk wkom
tkey interact?
constructivists study the multiple realities
constructed by people and the implications
Constructivism begins with the premise of those constructions for their lives and in-
that the human world is different from the teractions with others. Shadish (1995b) re-
natural, physical world and therefore must minds us that social constructionism "refers
be studied differently (Guba and Lincoln to constructing knozvledge about reality, not
1990). Because human beings have evolved constructing reality itself" (p. 67). Construc-
the capacity to interpret and construct re- tionists commonly assume that humans "do
alityindeed, they cannot do otherwise not have direct access to a singular, stable,
the world of human perception is not real and fully knowable externai reality. Ali of
in an absolute sense, as the sun is real, but is our understandings are contextually em-
"made up" and shaped by cultural and lin- bedded, interpersonally forged, and neces-
guistic constructs. To say that the socially sarily limited" (Neimeyer 1993:1-2). Any no-
constructed world of humans is not physi- tion of "truth," then, becomes a matter of
cally real like the sun doesn't mean that it "consensus among informed and sophisti-
isn't perceived and experienced as real by cated constructors, not of correspondence
real people. W. I. Thomas, a distinguished with an objective reality." Likewise, the no-
sociologist and a founding symbolic inter- tion of an objective "fact" has no meaning
actionist, formulated what has become "except within some value framework." It
known as Thomas's theorem: What is defined. follows that "there cannot be an 'objective'
or perceived by people as real is real in its conse- assessment of any proposition" (Guba and
quences (Thomas and Thomas 1928:572). So Lincoln 1989:44). Social construction, or
Variety in Qualitative Inquiri/ J. 97

EXHIBIT 3.4 Constructivism Versus Constructionism

Michael Crotty (1998) makes an important and useful distinction between constructivism and
constructionism, a distinction that illustrates how the process of social construction unfolds
among scholars. It remains to be seen whether this distinction will gain widespread use since the
two terms are so difficult to distinguish and easy to confuse.
"It would appear useful, then, to reserve the term constructivism for the epistemological con-
siderations focusing exclusively on 'the meaning-making activity of the individual mind' and to
use constructionism where the focus includes 'the collective generation [and transmission] of
meaning'" (p. 58}
"Whatever the terminology, the distinction itself is an important one. Constructivism taken in
this sense points out the unique experience of each of us. It suggests that each one's way of making
sense of the world is as valid and worthy of respect as any other, thereby tending to scotch any hint
of a criticai spirit. On the other hand, social constructionism emphasizes the hold our culture has
on us: it shapes the way in which we see things (even in the way in which we feei things!) and gives
us a quite definite view of the world" (p. 58).

constructivist philosophy, is built on the the- tained and reproduced through social life."
sis of ontological relativity, which holds that (pp. 54-55)
ali tenable statements about existence de-
pend on a worldview, and no worldview is Elsewhere Crotty uses the example of a tree:
uniquely determined by empirical or sense
data about the world. Hence, two people can What the "commorisense" view commends to
live in the same empirical world, even us is that the tree standing before us is a tree. It
though one's world is haunted by demons, has ali the ineaning we ascribe to a tree. It
and the other's, by subatomic particles. Ex- would be a tree, with that same meaning,
hibit 3.4 distinguishes the worldviews of whether anyone knew of its existence or not.
constructionism and constructivism, which We need to remind ourselves here that it is hu-
are often used interchangeably. man beings who have constructed it as a tree,
How all-encompassing is the construc- givenit the name, and attributed to it the asso-
tionist view? Michael Crotty (1998) asserts, ciations we make with trees. It may help if we
recall the extent to which those associations
It is not just our thoughts that are constructed differ even within the same overall culture.
for us. We have to reckon with the social con- "Tree" is likely to bear quite different connota-
struction of emotions. Moreover, constructions in a logging town, an artists' settlement
tionism embraces the whole gamut of mean- and a treeless slum. (p. 43)
ingful reality. Ali reality, as meaningful reality,
is socially constructed. There is no exception. How, then, does operating from a con-
. . . The chair may exist as a phenomenal object structionist perspective actually affect quali-
regardless of whether any consciousness is tative inquiry? Let's consider its impact on
aware of its existence. It exists as a chair, how- program evaluation. A constructionist eval-
ever, only if conscious beings construe it as a uator would expect that different stake-
chair. As a chair, it too "is constructed, sus- holders involved in a welfare program (e.g.,
staff, clients, families of clients, administra-
98 !f. CONCEPTUALISSUES IN QUALITATIVE INQUIRY

tors, funders) would have different experi- pacts rather than reaching singular conclu-
ences and perceptions of the program, ali of sions. The mdium of the report carried the
which deserve attention and ali of which are message that multiple voices needed to be
experienced as real. The constructionist heard and valued as a manifestation of di-
evaluator would attempt to capture these versity (Stockdill et ai. 1992). The findings
different perspectives through open-ended were used for both formative and sum-
interviews and observations, and then mative purposes, but the parents and many
would examine the implications of different of the staff were most interested in using the
perceptions (or multiple "realities") but evaluation processes to make themselves
would not pronounce which set of percep- heard by those in power. Being heard was an
tions was "right" or more "true" or more end in itself quite separate from use of find-
"real," as would a reality-oriented (postings.
positivist) evaluator. Constructivist evalua- Guba and Lincoln (1989) included among
tors could compare clients' perceptions and the primary assumptions of constructivism
social constructions with those of funders or the following, whether for evaluation or re-
program staff and could interpret the effects search more generally:
of differences on attainment of stated pro-
gram goals, but they would not value staff "Truth" is a matter of consensus among
perceptions as more real or meaningful. In informed and sophisticated construc-
constructivist evaluation, then, "the claims, tors, not of correspondence with objec-
concerns, and issues of stakeholders serve as tive reality.
organizational foci (the basis for determin- "Facts" have no meaning except within
ing what information is needed)" (Guba and some value framework, hence there can-
Lincoln 1989:50). not be an "objective" assessment of any
Indeed, if constructivist evaluators were proposition.
also operating from a social justice frame-
"Causes" and effects do not exist except
work, they might give added weight to the
by imputation....
perspectives of those with less power and
privilege in order to "give voice" to the dis- Phenomena can only be understood
enfranchised, the underprivileged, the poor, within the context in which they are
and others outside the mainstream (Weiss studied; findings from one context can-
and Greene 1992:145). In the evaluation of a not be generalized to another; neither
diversity project in a school district in Saint problems nor solutions can be general-
Paul, Minnesota, a major part of the design ized from one setting to another. . . .
included capturing and repor ting the expe- b Data derived from constructivist inquiry
riences of people of color. Providing a way have neither special status nor legitima-
for African American, Native American, tion; they represent simply another con-
Chicano-La tino, and Hmong parents to tell struction to be taken into account in the
their stories to mostly White, corporate move toward consensus. (pp. 44-45)
funders was an intentional part of the de-
sign, one approved by those same White cor- Guba and Lincoln (1990:148) summarize the
porate funders. The final report was written constructivist perspective as being ontologi-
as a "multivocal, multicultural" presenta- cally relativist, epistemologically subjectiv-
tion that presented different experiences ist, and methodologically hermeneutic and
with and perceptions of the program's im- dialectic. The thread throughout is the em-
Variety in Qualitative Inquiri/ J. 9 9

phasis on the socially constructed nature of sumption that scientists, rather than being
reality as distinguishing the study of human bound by preconceptions, were open-
beings from the study of other natural phe- minded, value free, and unencumbered by
nomena. inherited ideas. Kuhn applied to science the
The idea that social groups such as street kind of language normally used to describe
gangs or religious adherents construct their confrontatioris between opposing political
own realities has a long history in sociology, and ideological communities, especially
especially the sociology of knowledge (e.g., during revolutions. He argued (and showed
Berger and Luckmann 1967). It wasn't until with natural science examples) that commu-
this idea of socially constructed knowledge nities of scientists, like ideological or reli-
was applied to scientists that construc- gious communities, were organized by cer-
tionism became an influential methodologi- tain traditions that periodically came under
cal paradigm. No work has been more influ- strain when new problems arose that
ential in that regard than Thomas Kuhn's couldn't be explained by old beliefs. New
classic The Structure of Scientific Revolutions explanations and ideas would then compete
(1970). Before Kuhn, most people thought until the old ideas were discarded or re-
that science progressed through heroic indi- vised, sometimes sweepingly. But the com-
vidual discoveries that contributed to an ac- petition was not just intellectual. Power was
cumulating body of knowledge that got involved. The leaders of scientific communi-
closer and closer to the way the world really ties wielded power in support of their posi-
worked. In contrast, Kuhn argued that tions just as political leaders do. The assess-
tightly organized communities of specialists ment of Kuhn's contribution, three decades
were the central forces in scientific develop- after his work first appeared, by Berkeley
ment. Ideas that seemed to derive from bril- historian David Hollinger shows the impor-
liant individual scientific minds were actu- tance of his analysis: "The Structure of Scien-
ally shaped by and dependent on paradigms tific Revolutions presented the strongest case
of knowledge that were socially constructed ever made for the dependence of valid sci-
and enforced through group consensus. ence on distinctly constituted, historically
Rather than seeing scientific inquiry as pro- particular human communities" (Hollinger
gressing steadily toward truth about nature, 2000:23).
he suggested that science is best seen as a se- Scientists constitute a criticai case for so-
ries of power struggles between adherents cial constructionism. If scientific knowledge
of different scientific worldviews. is socially constructed and consensually val-
Kuhn emphasized the power of precon- idated, as opposed to consisting of empirical
ceived and socially constructed ideas to con- truths validated by nature, then surely ali
trol the observations of scientists. He in- knowledge is socially constructed. "Accord-
sisted that without the focusing effect of ingly, not only the social scientistbut equally
agreed-on constructs, investigators would the natural scientist has to deal with realities
notbe able actually to engage in research. A that, as meaningful realities, are socially
fully "open" mind would not be able to fo- constructed. They are on equal footing in
cus on the details necessary to engage in this respect" (Crotty 1998:55).
"normal" science, that is, testing specific Kuhn's analysis, though remaining con-
propositions derived from a theory or "sci- troversial and heavily critiqued (e.g., Fuller
entific paradigm." What made this contribu- 2000), became a cornerstone of the post-
tion so important was the widespread pre- modern skepticism about scientific truth.
100 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Postmodernism, Radavich (2001) asserts,

has become "the most prevalent mode of
M-IAIVI N L L H > T E R J M F V T
thinking in our time.... Postmodernist dis-
W FtJStftilQpii^MISM
course is precisely the discourse that denies
the possibility of ontological grounding" !:'] Y i i V i ! " ii ' i ! V i i,' i V i i h . - i 1 rfw.i. ;.S=r!. : i",*i .'|i

(p. 6). In other words, no truth or "true iW-inVi ! V id: i ,','r,a i-i"! nif;rv I;- -
meaning" about any aspect of existence is t<er|i!' M ^ f L V U l Y i . v l rAjp.' -vM firCT m -

possible, at least not in any absolute sense; it iT!i:.::;i:!: : i'!i.i'i ': m C ! : h . : - i . ! " i !.I ' :;J I rrp: s-!.;s IV

can only be constructed. To understand fcj ;,! iii.j;^ .vi i y-t v. VM* iAvqVn jVln
constructionism and its implications for C13.VC1 i:l:=:i S.'i S I U ! H.U !R>Y'.>'I'!, "I;' T I ! Y ,T] JJ

qualitative inquiry, a brief review of EWUJ!- i:i=:i l.U :':I ! trniv u l m ^ j ":T ^iri:^!!.!!"'
postmodernism may be helpful in that it has SM:'..'! iJ. . I: ' U!! k ! : ' ! s n - i ' l v.xv h }f\ i " .> i j i r i jy/i r

shaped contemporary intellectual discourse !

f M i l Y S : ^ ! i.': li'!:: 1
ftl ^ . ' f i ; V i P J l t f l t (kiy if-H"^!-

in both science and art. in bit ! ri= i' i't! !Vi.!riV! ! o:i'n :>-
Belief in science as generating truth was jjiii':!.'!!1 :r i'ri i'j|iJ:iV sn \v>i,! 5; fri,", ^ ':vm" !! n !"
one of the cornerstones of modernism inher- ! I i.VL II ii H i NVFAR 'i' ;T!." > . ' ! I;; 1 V ! . < ' Y L V I V ;

ited from the Enlightenment. Postmod- i . : r i . i . : R ' n J r i r i | ':'! n r i >} v n k !'ji M i ', <': W E s : j ; v j

ernism attacked this faith in science by ques- ,i,'i.!-=ViiiS"ji!1 S d ^ . ; ; - n :

:
; i i ' 0.

tioning its capacity to generate truth, in part

because, like ali human communications, it
is dependent on language, which is socially
constructed and, as such, distorts reality. knowledge is the norm, and a permanent
Postmodernism asserts that no language, pluralism of cultures is the only real truth
not even that of science, can provide a direct that humans must continually face" (Turner
window through which one can view reality. 1998:599). Constructionism, then, consistent
Language inevitably and inherently is built with postmodernism, is relativistic in
on the assumptions and worldview of the stance, meariing knowledge is viewed as rel-
social group that has constructed it and the ative to time and place, never absolute
culture of which it is a part. Thus, language across time and space, thus the reluctance to
does not and cannot fully capture or repre- generalize and the suspicion of generaliza-
sent reality, a posture called the "crisis of tions asserted by others.
representation" (Denzin and Lincoln 2000b; Power comes into the picture here be-
Turner 1998:598). Translated into Kuhn's cause, as views of reality are socially con-
framework, scientific language and constructed and culturally embedded, those
structs are paradigm based and dependent. views dominant at any time and place will
It follows from this that the continuity of serve the interests and perspectives of those
knowledge over time and across cultures is who exercise the most power in a particular
called into question. Modernism/s faith in culture. By exercising control over language,
science included the assumption that and therefore control over the very catego-
knowledge increased over time and that ries of reality that are opened to conscious-
such accumulation constituted continuous ness, those in power are served.
progress toward deeper and deeper truths. Scientific knowledge, then, is socially
"Postmodernists argue that because there is constructed like ali other knowledge sys-
not a truth that exists apart from the ideolog- tems and, as such, is relative to and contin-
ical interests of humans, discontinuity of gent on the methods and paradigms within
Variety in Qualitative Inquiry |J, 101

which it was generated. "Science, like any tions and positions, from the radical "abso-
knowledge system, is based on incorrigible lutely no reality ever" to a milder "let's
assumptions, is an abstraction fromphysical capture and honor different perspectives
reality, is in need of reification and stabiliza- about reality." These positions share an in-
tion through the processes of institu- terest m the subjective nature of human
tionalization and emotional investment, and perceptions and skepticism about the
is bent on systematically subjugating other possibility of objectivity. Reality-oriented re-
knowledge systems to assertits own reality" searchers, inkind, are skeptical of the subjec-
(Turner 1998:614). Some postmodernists tive knowledge of constructivism. How con-
and constructivists question the possibility tentious is the debate? One gets some sense
of ever finding and expressing true reality, of the gulf that can separate these views
even in the physical world, because lan- from an assessment of postmodernism and
guage creates a screen between human be- constructivism by Rutgers mathematician
ings and physical reality. "Vocabularies are Norman Levitt (1998) in an article titled
useful or useless, good or bad, helpful or "Why Professors Believe Weird Things":
misleadmg, sensitive or coarse, and so on; "Scientific evidencewhich is to say the
but they are not 'more objective' or 'less ob- only meaningful evidencecannot be neu-
jective' nor more or less 'scientific' " (Rorty tralized by 'subjective knowledge,' which is
1994:57). This is because discovermg the to say bullshit" (p. 34). He goes on to com-
"true nature of reality" is not the real pur- ment on constructivism as a particular mani-
pose of language; the purpose of language is festation of postmodernism: "a particular
to communicate the social construction of technique for getting drunk on one's own
the dominant members of the group using words" (p. 35).
the language. Thomas Schwandt (1997a) in his very
The postmodern perspective, and its useful dictionary of qualitative terms strikes
many variationsfor postmodernism is not a more conciliatory tone, recognizing that
a unitary perspective (e.g., Pillow 2000; the rhetoric of constructivism can sound
Constas 1998)has given rise to an empha- radical (and silly) if taken too literally:
sis on deconstruction, which means to take
Although some versions of constructivism do
apart the language of a text to expose its criti-
appear to deny reality, many (if not most, I sus-
cai assumptions and the ideological inter-
pect) qualitative inquirers have a common-
ests being served. Perspective and power oc-
sense realist ontology, that is, they take seri-
cur as hand in glove in postmodern
ously the existence of things, events, struc-
critiques. Social constructions are presumed
tures, people, meanings, and so forth in the
to serve someone's interests, usually those
environment as independent in some way
of the powerful. As Denzin (1991) has as-
from their experience with them. And they re-
serted with reference to deconstructing
gard society, institutions, feelings, intelli-
mass media messages, a criticai analysis
gence, poverty, disability, and so on as being
should "give a voice to the voiceless, as it de-
just as "real" as the toes on their feet and the
constructs those popular culture texts which
sun in the sky. (p. 134)
reproduce stereotypes about the powerless"
(p. 153). Thus, deconstruction constitutes a Further deconstructing the phrase "social
core analytical tool of constructivists. construction," one may find "inescapable
In deconstructing constructionism and connotations of manufacturmg," as if peo-
constructivism, one finds a range of assump- ple sat around and made things up. But
102 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

DUALISTAND MONIST
CONSTRUCTIONISM
iiHLCiil^Mii.iiH.uci iarj .
: UCMftAHi: THF:I:1;!C!3!T ^''V^:^:'! Distinguishing dualist from monist ap-
- W ^JMfflB^SlS^tiÎSfi-l : proaches to social constructionism takes the
deconstruction process through one final
. :ii U iilVlVi.i^.!J):! b. f f ^ j v i filter.
Si.!i':!"':! ' r i i l j i i i^fjjfc.lj
Dualist constructionism distinguishes be-
k* \ riTi lh! n! il-jil'^]!.!^!!^ tween actual states of affairs and perceptions,
. !.i:'j|:i r Vii!.'i: n';lt:-!i:. ItfN
interpretations, or reactions to those affairs
p:=!i U v k w i W , fere! iiiî-.Lî^L^ri^ligjH^r^ i i v i i i h y . ,
When Berger and Luckmann (1967) say that
the Sociology of Knowledge "must concern it-
' . 'iitflji':!ii!:i:vii|'iiiilj'!!'!'!Wl^W
self with whatever passes for 'knowledge' in a
ilfc LfeH !!:i'!V ri/i-iMij = !=
; s1^!^yr!/jr^fl
society" (p. 3), their putting knowledge in
quotation marks demonstrates a commitment
; !!':!-I:,:;.!I; ^ ' J J J T O F C L F R ^ I I ^ ^ P : 5 ^ ^ ^ ! 1 ! ^ ^ ^
to a dualist position. There is knowledge then
: iri- i::i:L'l;i:;:;j:j."j:i:j'j
there is "knowledge." The latter will be
: cfrei ft-! i:ii:J;

:
treated as knowledge by some social group,
i:!> JlTl+jiS^
but judgment can be made on the ultima te va-

!l!!!!!!!!!!!!!!!!!!!!!!!|j||||g| lidity of this group's claims, determining

whether "knowledge" really is knowledge
; 11]|| lliy^^filIltlIfHiHl!^^ i HlfHiflllilplIlIlH^
If the approach to the domain of inquiry is
'.' Slil&fci; j;i|||li!::|i;n|!;j!!||^
dualist this means that the analyst distin-
Illlllllllllllllllllllslliilllisiiiiiis guishes between the objective features of the
domain and members' representations of
those features. The dualist is prepared to judge
the adequacy of the members' representations
to say that people produce the world is not the (beliefs, interpretation). If the approach to the
same as saying that they are sopsists, that domain is monist, then there are only mem-
they are able to fashion the world according to bers' representations, the adequacy of which
their w h i m s . . . . [Ojne cannot ordinarily pro- cannot be raised as an issue; there are no objec-
duce an imaginary or nonsensical phenome- tive features in the domain upon which to base
non and expect to be taken seriously. The a judgment of the adequacy, for example, of a
mistake is to think of the process of production claim of racism. There are only representa-
as one that is free of constraints when in fact it tions of features, for example, the representa-
is a structure of constraints. (Watson and tion/claim of racism. (Heap 1995:54)
Goulet 1998:97)

We can conclude by emphasizing the ba-

Attending to the social construction of real- sic contributions of social construction and
ity, then, points us not only to what is con- constructivist perspectives to qualitative in-
structed but to how it is constructed and the quiry, namely, the emphasis on capturing
very question of what it means to say it is and honoring multiple perspectives; attend-
constructed. For an excellent review of these ing to the ways in which language as a social
issues in both the social and natural sciences, and cultural construction shapes, distorts,
see Hacking (2000). and struetures understandings; how meth-
Variety in Qualitative Inquiry |J, 103

Young lady, the court doesn't need to hear any

more about difficulties with validity & reliability in
qualitative research methods, discrepancies between
different postmodern epistemologies, or this "Great
Paradigm Debate." Please just tell us what you saw.

ods determine findings; and the importance cially the effects of inequitable power dy-
of thinking about the relationship between namicsand how that relationship affects
the investigator and the investigated, espe- what is found.
104 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Phenomenology What these various phenomenological

and phenomenographic approaches share
in common is a focus on exploring how hu-
man beings make sense of experience and
Troundational question: transform experience into consciousness,
Wkat is +ke meaning,, strwc+w^e, both individually and as shared meaning.
and essence oj-tke lived experi- This requires methodologically, carefully,
ence oj- +kis pkenomenon j-or this and thoroughly capturing and describing
p e r s o n or g r o w p oj- people?
how people experience some phenomenon
how they perceive it, describe it, feel about
it, judge it, remember it, make sense of it,
"Phenomenology asks for the very nature and talk about it with others. To gather such
of a phenomenon, for that which makes a data, one must undertake in-depth inter-
some-'thing' what it isand without which views with people who have directly experi-
it could not be what it is" (Van Manen enced the phenomenon of interest; that is,
1990:10). The initial clarity of this definition they have "lived experience" as opposed to
can fade rapidly because the term phenomen- secondhand experience.
ology has become so popular and has been so
widely embraced that its meaning has be- Phenomenology aims at gaining a deeper un-
come confused and diluted. It can refer to a derstanding of the nature or meaning of our
philosophy (Husserl 1967), an inquiry para- everyday experiences....
digm (Lincoln 1990), an interpretive theory Anything that presents itself to conscious-
(Denzin and Lincoln 2000b:14)/ a social sci- ness is potentially of interest to phenomenol-
ence analytical perspective or orientation ogy, whether the object is real or iinagined,
(Harper 2000:727; Schutz 1967, 1970), a ma- empirically measurable or subjectively felt.
jor qualitative tradition (Creswell 1998), or a Consciousness is the only access human be-
research methods framework (Moustakas ings have to the world. Or rather, it is by vir tue
1994). Varying forms complicate the picture of being conscious that we are already related
even more; transcendental, existential, and to the world. Thus ali we can ever know must
hermeneutic phenomenology offer different present itself to consciousness. Whatever falls
nuances of focusthe essential meanings of outside of consciousness therefore falls out-
individual experience, the social construc- side the bounds of our possible lived experi-
tion of group reality, and the language and ence. . . . A person cannot reflect on lived
structure of communication, respectively experience while living through the experi-
(Schwandt 2001:191-94). Phenomenologi- ence. For example, if one tries to reflect on
cal traditions in sociology and psychology one's anger while being angry, one finds that
vary in imit of analysis, group or individual the anger has already changed or dissipa ted.
(Creswell 1998:53). Adding further confu- Thus, phenomenological reflection is not in~
sion to the mix, the term phenornenography trospective but retrospective. Reflection on lived
was coined by Ulrich Sonnemann (1954) to experience is always recollective; it is reflec-
emphasize "a descriptive recording of im- tion on experience that is already passed or
mediate subjective experience as reported" lived through. (Van Manen 1990:9-10).
(p. 344). (For an annotated bibliography of
phenomenographic research, see Bruce and The phenomenon that is the focus of in-
Gerber 1997.) quiry may be an emotionloneliness, jeal-
Variety in Qualitative Inquiry |J, 105

Hi, l'm a graduae student,

and Vm learning about a
new type of research that
focuses on the "lived ex-
perience" of different life
events. 1 just have a few
questions..

Phenomenological abduction

ousy, anger. The phenomenon may be a rela- influences have been Merleau-Ponty (1962),
tionship, a marriage, or a job. The phenome- Whitehead (1958), Giorgi (1971), and Zaner
non may be a program, an organization, or a (1970). More recently, phenomenology has
culture. become an important influence in certain
Phenomenology as a philosophical tradi- approaches to psychotherapy (Moustakas
tion was first used in the development of a 1988,1995).
rigorous science by the German philosopher By phenomenology Husserl (1913) meant
Edmund H. Husserl (1859-1938). The work the study of how people describe things and
of Alfred Schutz (1899-1959) was an impor- experience them through their senses. His
tant influence in applying and establishing most basic philosophical assumption was
phenomenology as a major social science that we can only know what we experience by at-
perspective (Schutz 1977). Other important tending to perceptions and meanings that
106 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

awaken our conscious awareness. Initially, the phenomenon as directly as possible for
ali our understanding comes from sensory ourselves. This leads to the importance of
experience of phenomena, but that experi- participant observation and in-depth inter-
ence must be described, explicated, and in- viewing. In either case, in reporting phe-
terpreted. Yet, descriptions of experience nomenological findings, "the essence or na-
and interpretations are so intertwmed that ture of an experience has been adequately
they often become one. Interpretation is es- described in language if the description re-
sential to an understanding of experience awakens or shows us the lived quality and
and the experience includes the interpreta- significance of the experience in a fuller and
tion. Thus, phenomenologists focus on how deeper manner" (Van Manen 1990:10).
we put together the phenomena we experi- There is one final dimension that differen-
ence in such a way as to make sense of the tiates a phenomenological approach: the as-
world and, in so doing, develop a sumption that there is an essence or essences to
worldview. There is no separate (or objec- shared experience. These essences are the core
tive) reality for people. There is only what meanings mutually understood through a
they know their experience is and means. phenomenon commonly experienced. The
The subjective experience incorporates the experiences of different people are brack-
objective thing and becomes a person's real- eted, analyzed, and compared to identify the
ity, thus the focus on meaning making as the essences of the phenomenon, for example,
essence of human experience. the essence of loneliness, the essence of be-
ing a mother, or the essence of being a partic-
From a phenomenological point of view, we ipant in a particular program. The assump-
are less interested in the factual status of par- tion of essence, like the ethnographer's
ticular rnstances: whether some thing hap- assumption that culture exists and is impor-
pened, how often it tends to happen, or how tant, becomes the defining characteristic of a
the occurrence of an experience is related to purely phenomenological study. "Phenom-
the prevalence of other conditions or events. enological research is the study of essences" (Van
For example, phenomenology does not ask, Manen 1990:10). Phenomenologists are
"How do these children leam this particular
material?" but it asks, "What is the nature or rigorous in their analysis of the experience, so
essence of the experience of learning (so that I that basic elements of the experience that are
can now better understand what this particu- common to members of a specific society, or ali
lar learning experience is like for these chil- human beings, can be identified. This last
dren)?" (Van Manen 1990:10) point is essential to understanding the philo-
sophical basis of phenomenology, yet it is of-
There are two implications of this per- ten misunderstood. On the other hand, each
spective that are often confused in discuss- person has a unique set of experiences which
ing qualitative methods. The first implica- are treated as truth and which determine that
tion is that what is important to know is individual^ behavior. In this sense, truth (and
what people experience and how they inter- associate behavior) is totally unique to each in-
pret the world. This is the subject matter, dividual. Some researchers are misled to think
the focus, of phenomenological inquiry. that they are using a phenomenological per-
The second implication is methodological. spective when they study four teachers and
The only way for us to really know what describe their four unique views. A phenom-
another person experiences is to experience enologist assumes a commonality in those hu-
Variety in Qualitative Inquiry |J, 107

man experiences and must use rigorously the lieved to be essential in invs tigations of hu-
method of bracketing to search for those com- man experience. The root meaning of heuristic
monalities. Results obtained from a phenom- comes from the Greek word heuriskein, mean-
enological study can then be related to and ing to discover or to find. It refers to a process
integrated with those of other phenomen- of internai search through which one discov-
ologists studying the same experience, or phe- ers the nature and meaning of experience and
nomenon. (Eichelberger 1989:6) develops methods and procedures for further
investigation and analysis. The self of the re-
In short, conducting a study with a
searcher is present throughout the process
phenomenological focus (i.e., getting at the
and, while understanding the phenomenon
essence of the experience of some phenome-
with increasing depth, the researcher also ex-
non) is different from using phenomenology
periences growing self-awareness and self-
to philosophically justify the methods of
knowledge. Heuristic processes incorporate
qualitative inquiry as legitimate m social sci-
creative self-processes and self discoveries.
ence research. Both contributions are impor-
(Moustakas 1990b:9)
tant. But a phenomenological study (as op-
posed to a phenomenological perspective) is
one that focuses on descriptions of what
There are two focusing or narrowing ele-
people experience and how it is that they ex-
ments of heuristic inquiry within the larger
perience what they experience. One can em-
framework of phenomenology. First, the re-
ploy a general phenomenological perspec-
searcher must have personal experience
tive to elucidate the importance of using
with and intense interest in the phenomenon
methods that capture people's experience of
under study. Second, others (co researchers)
the world without conducting a phenom-
who are part of the study must share an in-
enological study that focuses on the essence
tensity of experience with the phenomenon.
of shared experience (at least that is my ex-
Heuristics is not inquiry into casual experi-
perience and interpretation of the phenome-
ence. Heuristic inquiry focuses on intense
non of phenomenology).
human experiences, intense from the point
of view of the investigator and coresearch-
Heuristic Inquiry ers. It is the combination of personal experi-
ence and intensity that yields an under-
]~pimdationqi question; standing of the essence of the phenomenon.
What is of this "Heuristics is concerned with meanings, not
pkenomenon cmd +h< essential measurements; with essence, not appear-
e^pefience ance; with quality, not quantity; with ex-
of o+h e.rs who also perience, not behavior" (Douglass and
experience +his pkenomenou Moustakas 1985:42).
m+ensely? The reports of heuristic researchers are
Heuristics is a form of phenomenological filled with the discoveries, personal in-
inquiry that brings to the fore the personal sights, and reflections of the researchers.
experience and insights of the researcher. Discovery comes from being wide open to
the thing itself, a recognition that one must
"Heuristic" research came into my life when I relinquish control and be tumbled about
was searching for a word that would mean- with the newness and drama of a searching
ingfully encompass the processes that I be- focus, "asking questions about phenomena
108 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

that disturb and challenge" (Douglass and understanding is achieved by indwelling.

Moustakas 1985:47). (Polanyi 1967:160)
The uniqueness of heuristic inquiry is the
extent to which it legitimizes and places at The rigor of heuristic inquiry comes from
the fore these personal experiences, reflec- systematic observation of and dialogues
tions, and insights of the researcher. The re- with self and others, as well as depth inter-
searcher, then, comes to understand the es- viewing of coresearchers. This mode of in-
sence of the phenomenon through shared quiry "affirms the possibility that one can
reflection and inquiry with coresearchers as live deeply and passionately in the moment,
they also intensively experience and reflect be fully immersed in mysteries and mira-
on the phenomenon in question. A sense of cles, and still be engaged in meaningful re-
connectedness develops between researcher search experience" (Craig 1978:20).
and research participants in their mutual ef-
forts to elucidate the nature, meaning, and The power of heuristic inquiry lies in its poten-
essence of a significant human experience. tial for disclosing truth. Through exhaustive
The fundamental methods text on heuris- self-search, dialogues with others, and cre-
tic inquiry is by the primary developer of ative depictions of experience, a comprehen-
this approach, Clark Moustakas (1990b). His sive knowledge is generated, beginning as a
classic works in this tradition include stud- series of subjective and developing into a sys-
ies of loneliness (1961, 1972, 1975) and hu- tematic and definitive exposition. (Douglass
manistic therapy (1995). Other examples are and Moustakas 1985:40)
Bernthal (1990), Clark (1988), Hawka (1986),
Weidman (1985), Katz (1987), Cheyne (1988),
Heuristic inquiry is derived from but dif-
Marino (1985), and Craig (1978). Heuristic
ferent from phenomenology in four major
inquiry has strong roots in humanistic psy-
ways (Douglass and Moustakas 1985):
chology (Maslow 1956, 1966; Rogers 1961,
1969, 1977) and Polanyi's (1962) emphasis
1. Heuristics emphasizes connectedness
on personal knowledge, indwelling, and the
and relationship, while phenomenology
tacitdimension (1967). "Tacit knowing oper-
encourages more detachment in analyz-
ates behind the scenes, giving birth to the
ing an experience.
hunches and vague, formless insights that
characterize heuristic discovery" (Doug-
2. Heuristics leads to "depictions of essen-
lass and Moustakas 1985:49). Polanyi ex-
tial meanings and portrayal of the in-
plained tacit knowing as the inner essence of
trigue and personal significance that
human understanding, what we know but
imbue the search to know," while phe-
can't articulate.
nomenology emphasizes definitive der
scriptions of the structures of experi-
Tacit knowing now appears as an act of in- ence.
dwelling by which we gain access to a new
meaning. When exercising a skill we literally 3. Heuristics concludes with a "creative
dwell in the innumerable muscular acts which synthesis" that includes the research-
contribute to its purpose, a purpose which er^ intuition and tacit understand ings,
constitutes their joint meaning. Therefore, while phenomenology presents a distil-
since ali understanding is tacit knowledge, ali lation of the structures of experience.
Variety in Qualitative Inquiry |J, 109

4. "Whereas phenomenology loses the Qualitative Heuristics:

persons in the process of descriptive A German Alternative Tradition
analysis, in heuristics the research par-
ticipants remain visible in the examina- Since no authority exists to monitor and
tion of the data and continue to be por- sort out nomenclature, conflicts in usage oc-
trayed as whole persons. Phenom- cur, contributing to confusion and the im-
enology ends with the essence of experi- portance of reaffirming the admonition to al-
ence; heuristics retains the essence of the ways define one's terms. Heuristic inquiry,
person in experience" (p. 43). la Clark Moustakas and discussed in the pre-
vious section, has a nomenclature rival in
Systematic steps in theheuristic inquiry pro- "qualitative heuristics," an approach devel-
cess lead to the "definitive exposition" of ex- oped at the University of Hamburg, Ger-
periential essence: immersion, incubation, many, which aims to "bring back the quali-
illumination, explication, and creative syn- ties of systematic exploration and discovery
thesis (Moustakas 1990a). into psychological and sociological re-
What is important about heuristics for my search" (Kleining and Witt 2000:1). It is
purpose here, that is, describing variety in based on four rules.
qualitative inquiry, is that heuristic research
epitomizes the phenomenological emphasis Rule 1. The research person should be
on meanings and knowing through per- open to new concepts and change his or
sonal experience; it exemplifies and places at her preconceptions if the data are not in
the fore the way in which the researcher is agreement with them.
the primary instrument in qualitative in-
quiry; and it challenges in the extreme tradi-
Rule 2. The topic of research is preliminary
tional scientific concerns about researcher
and may change during the research
objectivity and detachment, as in autoeth-
process. It is only fully known after be-
nography (described earlier in this chapter).
ing successfully explored.
In essence, it personalizes inquiry and puts
the experience (and voice) of the inquirer
front and center throughout. Rule 3. Data should be collected under the
paradigm of maximum structural varia-
tion of perspectives. Variation of the
If I am investigating the meaning of delight, sample and of research methods avoids
then delight hovers nearby and follows me one-sidedness of representation of the
around. It takes me fully into its confidence topic; variation of questions avoids just
and I take it into mine. Delight becomes a Iin- one answer. If researchers assume that a
gering presence; for awhile, there is only de- variable may influence the data they
light. It opens me to the world in a joyous way should implement variations. Structural
and takes me into a richness, playfulness and variations mean sampling of positions
childlikeness that move freely and effortlessly. in reference to the topic, i.e., when
I'm ready to see, feel, touch or hear whatever studying an emotion, the collection of
opens me to a fuller knowledge and under- data past and present, before and after
standing of the experience of delight. (Mous- its occurrence, in different situations,
takas 1990b:ll) from different respondents, if possible
110 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

from different times and cultures, by dif- unfamiliar sense, has no following and is
ferent methods, etc. tartly reminded that "it isn't in the dictionary"
although down to the time of the first lexi-
Rule 4. The analysis is directed toward dis- cographer (Heaven forgive him!) no author
covery of similarities. It locates similari- ever had used a word that was in the dictio-
ties, accordance, analogies or hom- nary. In the golden prime and high noon of
ologies within these most diverse and English speech; when from the lips of the great
varied data. It tries to overcome differ- Elizabethans fell words .that ma de their own
ences. The rule follows Simmel's fa- meaning and carried it in their very sound;
mous chapter on method saying that when a Shakespeare and a Bacon were possi-
"out of complex phenomena the ho- ble, and the language now rapidly perishing
mogenous will be extracted . . . and the at one and slowly renewed at the other was in
dissimilar paralyzed." (Kleining and vigorous growth and hardy preservation
Witt 2000: online) sweeter than honey and stronger than a lion
the lexicographer was a person unknown,
the dictionary a creation which his Creator
This approach emphasizes "introspec-
had not created him to create. (p. 110)
tion" as a criticai part of the analytical pro-
cess, an element also central to "heuristic in-
quiry" in the tradition of humanistic
Ethnomethodology
psychology. However, neither heuristic in-
quiry as articulated by Moustakas (1990b)
nor this German alternative labeled "quali- Fo unda+ional question:
tative heuristics" can be derived directly -How do people mal<e sense of+k eir
from the common dictionary definition of everyday activities so as to bekave
heuristics, defined as techniques to assist In socJally accep+able ways?
learning or techniques for exploratory prob-
lem solvingthough neither approach con-
flicts explicitly with the dictionary defini- Where heuristic inquiry focuses on issues
tion. Those who lament such variations in of intense personal interest, ethnometh-
meanings, denotations, and connotations odology focuses on the ordinary, the routine,
may find some comfort in Ambrose Bierce's the details of everyday life. Harold
([1906] 1999) DevWs Dictionary definition of Garfinkel (1967) invented the term. While
lexicographer: working with the Yale cross-cultural files,
Garfinkel came across such labels as "eth-
A pestilent fellow who, under the pretense of nobotany," "ethnophysiology," and "ethno-
recording some particular stage in the devel- physics." At the time he was studying jurors.
opment of a language, does what he can to ar- He decided that the deliberation methods
rest its growth, stiffen its flexibility and of the jurors, or for that matter of any
mechanize its methods. For your lexicogra- group, constituted an "ethnomethodol-
pher, having written his dictionary, comes to ogy" wherein ethno refers to the "availability
be considered "as one having authority," to a member of common-sense knowledge
whereas his function is only to make a record, of his society as common-sense knowledge
not to give a l a w . . . . Recognizing the truth that of the 'whatever' " (Turner 1974:16). For the
language must grow by innovation if it grow jurors this was their ordinary, everyday un-
at ali, makes new words and uses the old in an derstanding of what it meant to deliberate as
Variety in Qualitative Inquiry |J, 111

a juror. Such an understanding made jury ing something out of the ordinary. A very
duty possible. simple and well-known such experiment is
Ethnomethodology studies the social or- turning to face the other people on an eleva-
der "by combining a phenomenological sen- tor instead of facing the doors. When they
sibility with a paramount concern for every- conduct such qualitative experiments, "the
day social practice" (Gubrium and Holstein researchers are interested in what the sub-
2000:490). Wallace and Wolf (1980) defined jects do and what they look to in order to
ethnomethodology as follows: "If we trans- give the situation an appearance of order, or
lated the 'ethno' part of the term as 'mem- to 'make sense' of the situation" (Wallace
bers' (of a group) or 'folk' or 'people/ then and Wolf 1980:278). Garfinkel (1967) offered
the term's meaning can be stated as: mem- a number of such experiments (see espe-
bers' methods of making sense of their social cially pp. 38,42,47, 79, and 85).
world" (p. 263). Ethnomethodology gets at Ethnomethodologists also have special
the norms, understandings, and assump- interests in observing naturally occurring
tions that are taken for granted by people in experiments where people are thrust into
a setting because they are so deeply under- new or unexpected situations that require
stood that people don't even think about them to make sense of what is happening,
why they do what they do. It studies "the or- "situations in which meaning is problem-
dinary methods that ordinary people use to atic" (Wallace and Wolf 1980:280). Such situ-
realize their ordinary actions" (Coulon ations include intake into a program, immi-
1995:2). Rooted in phenomenology, ethno- gration clearance centers, the first few weeks
methodology has been particularly impor- in a new school or job, and major transition
tant in sociology. points or criticai incidents in the lives of peo-
ple, programs, and organizations.
Ethnomethodology is, as the name suggests, a
In some respects, ethnomethodologists
study of methods. It asks not why, but how. It
attempt to make explicit what might be
asks how people get things donehow they
called the group's "tacit knowledge," to ex-
transform situations or how they persevere,
tend Polanyi's (1967) idea of tacit knowl-
situation "unchanged," step by step, and mo-
edge from the individual to the group. Heu-
ment to moment. As its name also suggests, it
ristic inquiry reveals tacit knowledge
is interested in ordinary methods, the meth-
through introspection and intersubjective
ods of the people rather than their theorists.
inquiry with coresearchers. Ethnometh-
(Watson and Goulet 1998:97)
odologists get at a group's tacit knowledge
Ethnomethodologists elucidate what a by forcing it to the surface through disrupt-
complete stranger would have to learn to being violations of ordinary experience, since
come a routinely functioning member of a ordinary routines are what keep tacit knowl-
group, a program, or a culture. To do this, edge at an unconscious, tacit levei.
ethnomethodologists conduct depth inter- In short, ethnomethodologists "bracket
views and undertake participant observa- or suspend their own belief in reality to
tion. They stray from the nonmanipulative study the reality of everyday life" (Taylor
and unobtrusive strategies of most qualita- and Bogdan 1984:11). Elucidating the
tive inquiry in employing "ethnometh- taken-for-granted realities of everyday life
odological experiments." During these ex- in a program or organization can becoine a
periments, the researcher "violates the force for understanding, change, and estab-
scene" and disrupts ordinary activity by do- lishing a new reality based on the kind of ev-
112 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

eryday environment desired by people in interpret the world. Only through close con-
the setting being studied. The findings of tact and direct interaction with people in
an ethnomethodological evaluation study open-minded, naturalistic inquiry and in-
would create a programmatic self-aware- ductive analysis could the symbolic
ness that would, in principie at least, facili- interactionist come to understand the sym-
tate program change and improvement. bolic world of the people being studied.
Blumer was also one of the first to use group
Symbolic Interaction discussion and interview methods with key
informants. He considered a carefully se-
Poundationa questions lected group of naturally acute observers
and weH-informed people to be a real "panei
W K Q + common set of symbols
of experts" about a setting or situation, ex-
and understandings \\cxs emefged
perts who would take the researcher inside
+o give me.av\'iv\) to people's
the phenomenon of interest, for example,
interactions?
drug use. As we shall see in the chapter on
Symbolic interaction is a social-psycho- interviewing, group interviews and focus
logical approach most closely associated groups have now become highly valued and
with George Herbert Mead (1934) and Her- widely used qualitative methods.
bert Blumer (1969). It is a perspective that Labeling theorythe proposition that
places great emphasis on the importance of what people are called has major conse-
meaning and interpretation as essential hu- quences for social interactionhas been a
man processes in reaction against behavior- primary focus of inquiry in symbolic inter-
ism and mechanical stimulus-response psy- action. For example, using a sample of 46
chology. People create shared meanings participants in a 12-step group, Debtors
through their interactions, and those mean- Anonymous, Hayes (2000) studied how
ings become their reality. Blumer articulated people who are unable to manage their fi-
three major premises as fundamental to nances responsibly come to feel shame. In
symbolic interactionism: program evaluation, labeling theory can be
applied to such terms as dropouts and at-risk
1. Human beings act toward things on the you th because language matters to staff and
basis of the meanings that the things participants and can affect how they ap-
have for them. proach attaining desired outcomes (Hopson
2000; Patton 2000).
2. The meaning of things arises out of the Though this theoretical perspective
social interaction one has with one's fel- emerged in the 1930s, symbolic interac-
lows. tionists are showing that they can keep up
with the times, for example, by applying
3. The meanings of things are handled in their perspective to "cybersex" on the
and modified through an interpretative Internet. Waskul, Douglass, and Edgley
process used by the person in dealing (2000) have suggested that the new technol-
with the things he or she encounters. ogies of computer-mediated communica-
tion
These premises led Blumer to qualitative
inquiry as the only real way of understand- allow us to examine the nature of human inter-
ing how people perceive, understand, and action in a uniquely disembodied environ-
Variety in Qualitative Inquiry |J, 113

ment that potentially transforms the nature of tific study of a space-time event like a solar
self, body, and situation. Sexfundamentally eclipse or rat behavior," Walker Percy
a bodily activityprovides an ideal situation (1990:150) has explained, "is that as soon as
for examrng these kinds of potential trans- one scratches the surface of the familiar and
formations. In the disembodied context of comes face to face with the nature of lan-
on-line interaction both bodies and selves are guage," one also finds oneself face to face
fluid symbolic constructs emergent in com- with the nature and essence of being human.
munication and are defined by sociocultural This is so because semiotics, in working to
standards. Situations such as these are sugges- "unite logical analysis with the explanatory
tive of issues related to contemporary trans- enterprise of science" (p. 243), has hit upon
gressions of the empirical shell of the body, the fruitful insight that humans are distinc-
potentially reshaping body-to-self-to-social- tively sign-using and symbol-generating an-
world relationships. (p. 375) imais. Thus, semiotics offers a framework
for "analyzing talk and text" (Silverman
2000:826) or studying "organizational sym-
For our purposes, the importance of symbolism" (Jones 1996). The foundational
bolic interactionism to qualitative inquiry is question of semiotics is: How do signs
its distinct emphasis on the importance of (words, symbols) carry and convey meaning
symbols and the interpretative processes in particular contexts?
that undergird interactions as fundamental
to understanding human behavior. For pro-
gram evaluation, organizational develop- Hermeneutics
ment, and other applied research, the study
of the original meaning and influence of Houndafio^at question:
symbols and shared meanings can shed Wkat condi+ions u n d e r
light on what is most important to people, wki^k Q Kuman a c t took p t a c e
what will be most resistant to ehange, and or a produci voas pyoduae.^
what will be most necessary to change if the m a k e i+ p o s s i b l e t o i n t e r p r e t i+s
program or organization is to move in new meanings?
directions. The subject matter and methods
of symbol interactionism also emphasize the In this brief (or not-so-brief, depending on
importance of paying attention to how par- your perspective) excursion through the vari-
ticular interactions give rise to symbolic un- ety of qualitative inquiry, we depart now
derstandings when one is engaged in chang- from phenomenology and its derivative
ing symbols as part of a program im- approaches: heuristic research, ethnometh-
provement or organizational development odology, and symbolic interactionism. Her-
process. meneutics is yet a different theoretical ap-
A related theoretical tradition informing proach that can inform qualitative inquiry
some qualitative inquiry is semiotics, a blend and also help put ali the other theoretical ori-
of linguistics and social science, which fo- entations in this chapter in perspective in
cuss on the analysis of signs by studying that it reminds us that what something
the rules or forms of language as well as the means depends on the cultural context in
relationship between language and human which it was originally created as well as the
behavior (Manning 1987). "The importance cultural context within which it is subse-
of a study of language, as opposed to a scien- quently interpreted. This is a reminder that
114 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

each of the theoretical perspectives pre- cism worked from a conception of knowl-
senteei in this chapter emerged from a partic- edge as correct representation of an inde-
ular context to address specific concerns at pendent reality and was (is) almost exclu-
that time. As we adopt and adapt those per- sively interested in the issue of establishing
spectives to current inquiries, we do so in a the validity of scientific knowledge claims"
different historical, scholarly, and cultural (Schwandt 2000:196). In other words, her-
context. meneutics challenged the assertion that an
Hermeneutic philosophy, first developed interpretation can ever be absolutely correct
by Frederich Schleiermacher (1768-1834) or true. It must remain only and always an
and applied to human science research by interpretation. The meaning of a text, then, is
Wilhelm Dilthey (1833-1911) and other Ger- negotiated among a community of interpret-
man philosophers, focuses on the problem ers, and to the extent that some agreement is
of interpretation. Hermeneutics provides a reached about meaning at a particular time
theoretical framework for interpretive un- and place, that meaning can only be based
derstanding, or meaning, with special atten- on consensual community validation. Texts,
tion to context and original purpose. The then, must be "situated" within some liter-
term hermeneutics derives from the Greek acy context (Barton, Hamilton, and Ivanic
word hermeneuein, meaning to understand 1999).
or interpret. Kvale (1987) has suggested,

There is an obvious link between hermeneuein The attempts to develop a logic of validation
and the god Hermes. Hermes is the within the hermeneutical tradition are rele-
fleet-footed divine messenger (he has wings vant for clarifying the validity of interpreta-
on his feet!). As a messenger, he is the bearer of tion in the qualitative research interview.
knowledge and understanding. His task is to The interpretation of meaning is character-
explain to humans the decisions of the gods. ized by a hermeneutical circle, or spiral. The un-
Whether hermeneuein derives from Hermes or derstanding of a text takes place through a
the other way round is not certain. (Crotty process where the meaning of the separate
1998:88) parts is determined by the global meaning of
text. In principie, such a hermeneutical expli-
In modern usage, hermeneutics offers a cation of the text is an infinite process while it
perspective for interpreting legends, stories, ends in practice when a sensible meaning, a
and other texts, especially biblical and legal coherent understanding, free of inner contra-
texts. To make sense of and interpret a text, it dictions has been reached. (p. 62)
is important to know what the author
wanted to communicate, to understand in- Kneller (1984) has offered four principies
tended meanings, and to place documents in for hermeneutic inquiry and analysis that
a historical and cultural context (Palmer can be applied beyond the interpretation of
1969). Following that principie, hermeneu- legends, literature, and historical docu-
tics itself must be understood as part of a ments:
19th- and 20th-century "broad movement
away from an empiricist, logical atomistic, 1. Understanding a human act or product,
designative, representational account of and hence ali learning, is like interpret-
meaning and knowledge Logical empiri- ing a text.
Variety in Qualitative Inquiry |J, 115

2. Ali interpretation occurs within a tradi- review of the historical development of her-
tion. meneutics and its influence on qualitative
theory, "Our debt to the hermeneutic tradi-
3. Interpretation involves opening myself tion is large" (p. 111).
to a text (or its analogue) and question-
ing it. Narratology or Narrative Analysis

4. I must interpret a text in the light of my Foundational questions:

situation (p. 68). What does this narrative or story
reveal about the person and wo rld
Hermeneutic researchers use qualitative -from which it carne? How can th is
methods to establish context and meaning narrative be interpreted so that it
for what people do. Hermeneutists "are provides an understanding of cxv\<^
much clearer about the fact that they are con- illuminates the life and culture that
structing the 'reality' on the basis of their in- created it?
terpre tations of data with the help of the par-
ticipants who provided the data in the study Hermeneutics originated in the study of
. . . If other researchers had different back- written texts. Narratology, or narrative anal-
grounds, used different methods, or had dif- ysis, extends the idea of text to include
ferent purposes, they would likely develop in-depth interview transcripts, life history
different types of reactions, focus on differ- narratives, historical memoirs, and creative
ent aspects of the setting, and develop some- nonfiction. The hermeneutical perspective,
what different scenarios" (Eichelberger with its emphasis on interpretation and
1989:9). For concrete examples of hermeneu- context, informs narrative studies, as do
tic investigations in psychology, see Packer interpretivist social science, literary nonfic-
and Addison (1989). tion, and literary criticism. Narrative studies
Thus, one must know about the re- are also influenced by phenomenology's
searcher as well as the researched to place emphasis on understanding lived experi-
any qualitative study in a proper, hermeneu- ence and perceptions of experience. "Todo-
tic context. Hermeneutic theory argues that rov coined the term narratology in 1969 in an
one can only interpret the meaning of some- effort to elevate the form 'to the status of an
thing from some perspective, a certain object of knowledge for a new science'"
standpoint, a praxis, or a situational context, (Riessman 1993:1).
whether one is reporting on one's own find- Personal narratives, family stories, sui-
ings or reporting the perspectives of people cide notes, graffiti, literary nonfiction, and
being studied (and thus reporting their life histories reveal cultural and social pat-
standpoint or perspective). These ideas have terns through the lens of individual experi-
become commonplace in much contempo- ences. Rhetoric of ali kinds can be fodder for
rary social science and are now fundamen- narrative analysis, for example, the rhetoric
tal, even basic, in qualitative inquiry, but of politicians or teachers (Graham 1993).
such was not always the case. Two centuries The "biographical turn in social science"
of philosophical dialogue provide our cur- (Chamberlayne et al. 2000) or the "narrative
rent foundation for understanding the cen- turn" in qualitative inquiry (Bochner 2001)
trality of interpretivism m qualitative re- honors people's stories as data that can
search. As Crotty (1998) concluded after his stand on their own as pure description of ex-
116 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

perience, worthy as narrative documentary Ali great literature, I think, lures those who ex-
of experience (the core of phenomenology) or perience it away from the shores of literal truth
analyzed for connections between the psy- and out into uncharted waters where meaning
chological, sociological, cultural, political, is more ambiguous.. . .
and dramatic dimensions of human experi- Ultimately, I erased the boundary between
ence. Robert Coles, Harvard professor of the realm of text which purports to give only
psychiatry and medicai humanities (his title the facts and that of the metaphor-laden story
offers interesting narratological fodder), has which dares (as Sartre once put it) to lie in or-
written The Call ofStories (1989) as a basis for der to tell the truth. But I did so haltingly, and
teaching, learning, and moral reflection. Mi- not in a single confident stroke of understand-
chael White and David Epston in Narrative ing. Indeed, my insight carne only gradually,
Means to Therapeutic Ends (1990) look at the after confronting a form of writing that aims to
power of stories in the lives of individuais straddle the boundary between actual and vir-
and families and the connection between tual worlds, one foot firmly planted in each.
storytelling and therapy. They suggest that These works are hybrids of textual species,
people have adjustment difficulties because essays/stories written in a literary style but
the story of their life, as created by them- shelved (curiously) m the nonfiction section
selves or others, does not match their lived of the library. (pp. 61-62)
experience. They propose that therapists can
help their patients by guiding them in re- Here we have an example of personal nar-
writing their life stories. rative in the form of the narrative re-
The idea of "story," of personal narrative, searcher^ report of his joumey into cross-
intersects with our earlier look at autoeth- genre exploration of the nature of textual in-
nography in which the researcher's story be- terpretation. Later he uses narrative as a
comes part of the inquiry into a cultural phe- method for exploring what it means to be a
nomenon of interest. The language of story professional educational researcher, explor-
carries a connotation different from that of ing the narratives researchers construct
case study. For example, in program evalua- about themselves and implications of those
tions, people may be invited to share their narratives for their relationships with non-
stories instead of being asked to participate researchers (Barone 2000:201-28).
in case studies. The central idea of narrative Tierney (2000), in contrast, examines his-
analysis is that stories and narratives offer torical biographies and testimonios to ex-
especially translucent windows into cul- plore interpretive challenges in using life
tural and social meanings. histories in the postmodern age. His narra-
Much of the methodological focus in nar- tive analysis looks at the intersection of the
rative studies concerns the nature of inter- interpreted purpose of a text, the con-
pretation, as in Norman Denzin's seminal structed and interpreted "truth" of a text,
qualitative works Interpretive Biography and the persona of the author in text creation,
(1989a), Interpretive Interactionism (1989b), ali of which are called into interpretive ques-
and Interpretive Ethnography (1997b). Inter- tion in the postmodern age.
pretation of narrative poses the problem of Tedlock (2000) examines different genres
how to analyze "talk and text" (Silverman of ethnography as constituting varying
2000). Tom Barone (2000) has entered into lit- forms of narrative. She distinguishes life
erary nonfiction to hone his interpretive aes- histories and memoirs from "narrative eth-
thetic: nography," a hybrid form that was created
Variety in Qualitative Inquiry |J, 117

Today would you read

us a story that works on
many leveis?

Budding narratologist

in and attempts to portray accurately the bi- tion" because it unsettled "the boundaries
ographies of people in the culture studied that had been central to the notion of a self
but also to include ethnographers' own ex- studying an other" and replaced it with an
periences in their texts. She assesses this as a "ethnographic interchange" between self and
"sea change in ethnographic representa- other within a single text (pp. 460-61).
118 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Narrative analysis has also now emerged oped out of sociology. Heuristic inquiry is
as a specific approach to studying organiza- grounded in humanistic psychology. A dif-
tions. As such, it takes at least four forms: ferent psychology-based perspective is eco-
logical psychology, which represents a differ-
1. organizational research that is written in ent tradition and theoretical orientation
storylike fashion (tales from the field); because it makes different assumptions
about what is important to understand
2. organizational research that collects or-
about the human experience (Jacob 1987).
ganizational stories (tales of the field);
Robert Barker (1968) and Herbert Wright
3. organizational research that conceptual- (1967) of the University of Kansas devel-
izes organizational life as story making oped ecological psychology drawing
and organizational theory as story read- heavily on natural history field studies.
ing (nterpretative approaches); and They see individuais and the environment
4. a disciplinary reflection that takes the as interdependent (Barker and Wright 1955;
form of literary critique (Czarniawska Barker et al. 1978; Schoggen 1978). They be-
1998:13-14). gin with pure, detailed descriptions of an in-
dividual in an environment. They observe
Stories are at the center of narrative analy- (as spectators, not participant observers)
sis, whether they be stories of teaching "streams of behavior" that are subsequent-
(Preskill and Jacobvitz 2000), stories of and ly analyzed in terms of presumed goal-
by students (Barone 2000:119-31), stories of directed actions. "Coders draw upon their
participants in programs (Kushner 2000), ordinary knowledge and perceptions to in-
stories of fieldwork (Van Maanen 1988), sto- fer the goals that actors intend to achieve,
ries of relationships (Bochner, Ellis, and marking off segments of narrative descrip-
Tillman-Healy 1997), or stories of illness tions into segments leading toward specific
(Frank 1995,2000). How to interpret stories goals" (Jacob 1988:17). The ecological meta-
and, more specifically, the texts that tell the phor can also inform psychological clinicai
stories, is at the heart of narrative analysis. research by seeking "to understand the pa-
tienfs concern within the context of his or
her life worldthe patienfs personal, fam-
Ecological Psychology
ily, community, and ecological stories"
(Miller and Crabtree 2000:617).
T~oundationql question: The unit of analysis in ecological psychol-
Wkat is tke rela+ionskip ogy is primarily the individual, but Barker
between kuman bekaviou and and Schoggen (1973) have also applied this
tke environment? approach in delineating Qualities of Commu-
nity Life. What makes this approach of po-
tential interest for program evaluation and
Several theoretical perspectives that in- organizational or community development
form qualitative inquiry are associated with is the focus on goal-directed behavior.
particular disciplines. For example, herme-
neutics is derived from linguistics and phi- They assume that there are subjective aspects
losophy. Ethnography is the primary to behavior which they examine in terms of
method of anthropology, while ethnometh- the goals of human behavior. They also as-
odology and symbolic interaction devel- sume that there is a subjective aspect to the en-
Variety in Qualitative Inquiry |J, 119

viionments which they usually discuss in

terms of a person's emotional reactions to the
j j r i ; ; ! w y EcLcnar
environment. For example, they might be con-
cerned whether a boy does an activity unwill- IWftyClS-Eiyli 1 !! 1 'm!.' i"h.t-riij-i i"iiY
ingly or unhappily. (Jacob 1988:17) :'JHTLI/!: N."I.I M'iii::IRI!iJ:Vi:':i HFO,'[fi -PJIIS:;V.VI

fj't!:Mj! iVCVd:. it ij: :f!WLv^i.'.Pn."

Ecological psychologists also focus on de- PI.D tO yni'--; ! SiiriM!! i ;.hj-rf bii-iV
lineating the central features of behavior set- jtl-rj'r]ri;!y1'hrdfuf:i?:iVii':tiitioqyw !!'i!i!,'
tings, the particular constellations of places, iT!.l!.'' jyEEfeS. iHilPV mlsViT" ..r!..':i."i):!i'ii'!i;;!
things, and times that constitute a definitive !'"!:> :' airAiWi :; !V! C 'utfj.'m' i:!:i"n:!;'
environment. Such an approach can help iHi"i'!irjj-i,' LU-''!.Vv FIRIS :'i'!i,;!i!' il-!!i!;='!:i"
make explicit what variety of environments I.HW:1 Mik it* fft";!.:: irn-' j-iVt-y"?.! r-
program participants or organization mem- ! il l&afi IS .1 bi!V :!!'!!' !*!! !:i 'nY iij.1 i"n i/iij i r i
: iu,' IVi j |, fi.<i' iilsiriV! h? n!*,!i.l !! i; k i.!.'; i ,Y! .;! r
bers experience.
While ecological psychologists begin ! iVau! '' !-::;J y -I-J J i j; p. LS1? ;: hm
,"p .feirt.aT .'hM-j.1 nMh ,h rVi jiVj='? li i'i! i'1'.'s
with detailed, qualitative descriptionsbased
d r;.^!:':I .MIJM^iftlli?] h?:!- '' Jll:' h.:iM:i:!.';
on observations in natural environments,
' tmtiH .:ii".:r V:!- M-to-iUa.'' ^ri-liiv!!;!1
their coding schemes and analysis proce-
' !! h: n : |:i!,:i! JT.YIH" n ' >
" -:i*i ri! i.
dures are quantitative. Segments of goal-
ns i.!"vJi!i":r Lw i:'i:.IVvi.XViy &\).h!:!iji:i
directed behavior and characteristics of be- i' !'M!' Im!!i' -ii'i ii!'''ii i! dfiSir-jJ
:i ! ! v^y-^i--
havior settings are coded numerically and ififa MMi!' hoWi ^ Nl!i.S.-ii n't ;' iV-^':
analyzed statistically. This illustrates a point ^prfh.JiW r 1 iVv-jiCi"!.! i.'i .'ilMij:' ViiVi! !l. !!:'
to which we shall return later: One can go s!J.!i! i!!fcfd'Yli!i'iiiri'i:i-T''.
from the thick description of qualitative data :T!;ii' tit n^&itf /irwni ;; rn,^ v r !i, ::Vi!r
to quantitative analysis, but not vice versa. ,'i M:; i-rii1!:'!!A'RTJ v.H t' ru!! i.V, ! : I"I \!v. ! i1 ii wA! ! T

One cannot generate thick description and AEW i::=I!T'..i,S!,i:! ia b*'Mr- l^Vi ::'
qualitative narrative from original quantita- ^ M N TI!:' J ! : I 5 : N ! ' : N I ' Y IFCNL

tive data. !" JIL'!?:!: \ I;!:!; i ;: !'J:T i.= i. !i.< ,ii,'ilj' AUi'
!:'I ! I'-

The focus in ecological psychology on the ^'idVOii :ii!'i!!!!1Y ii-.i :ii!i ,'i' .nH-.!!!'!ii!'!!.!!
relationship between human behavior and U.TdMI :1! * y r f p , l Jpi'!'.!:' !!!!>>"!!"
the environment provides a good transition 1111111111111
to the next perspective, systems theory,
which is much more comprehensive and m-
terdisciplinary in examining the context for Parallel to the historical philosophical
human actions in programs, organizations, and methodological paradigms debate be-
and communities. tween positivists and constructivists, there
has been another and corresponding para-
A Systems Perspective digms debate about mechanistic/linear con-
and Systems Theory structions of the world versus organic/sys-
tems constructions. This debate has been
particularly intense among classic organiza-
T^oundational question: tional theorists (Burns and Stalker 1972;
"How and why does this sys+em Azumi and Hage 1972; Lincoln 1985;
a s a whole fwnction a s it does? Gharajedaghi 1985; Morgan 1986, 1989). It
includes concern about definitions of closed
120 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

systems versus open systems, and the im- into independent parts as discrete entities of
plications of such boundary definitions for inquiry because the effects of the behavior of
research, theory and practice in understand- the parts on the whole depend on what is
ing programs, organizations, entire soci- happening to the other parts. The parts are
eties, and even the whole world (Wallerstein so interconnected and interdependent that
1980). any simple cause-effect analysis distorts
It is important to note at the outset that more than it illuminates. Changes in one
the term systems has many and varied mean- part lead to changes among ali parts and the
ings. In the digital age, systems analysis of- system itself. Nor can one simply add the
ten means looking at the interface between parts in some linear fashion and get a useful
hardware and software, or the connectivity sense of the whole.
of various networks. The idea of "systems Gharajedaghi and Ackoff (1985:23) are
thinking" was popularized as the crucial quite insistent that a system as a whole can-
"fifth discipline" of organizational leaming not be understood by analysis of separate
in Peter Senge's (1990) best-selling book. A parts. They argue that "the essential proper-
number of management consultants have ties of a system are lost when it is taken
made systems thinking and analysis the cen- apart; for example, a disassembled automo-
terpiece of their organizational develop- bile does not transport and a disassembled
ment work (e.g., Ackoff 1987, 1999a, 1999b; person does not live." Furthermore, the
Kim 1993, 1994, 1999; Anderson and John- function and meaning of the parts are lost
son 1997). Indeed, over the past 30 years, when separated from the whole. Instead of
since publication of Ludwig Von Berta- taking things apart, they insist that a sys-
lanffy's classic General System Theory (1976), tems approach requires "synthetic think-
a vast literature has developed about sys- ing":
tems theory and applied systems research
(e.g., Checkland 1999). Some of it is highly
quantitative and involves complex coin- Synthetic thinking is required to explain system
puter applications and simulations. Given behavior. It differs significantly from analysis.
this broad and multifaceted context, my In the first step of analysis the thing to be ex-
purpose is quite modest. I want to call to the plained is taken apart: in synthetic thinking it
reader's attention three points: (1) A systems is taken to be a part of a larger whole. In the
perspective is becoming increasingly impor- second step of analysis, the contained parts
tant in dealing with and imderstanding are explained: in synthetic thinking, the con-
real-world complexities, viewing things as taining whole is explained. In the final step of
whole entities embedded in context and still analysis, knowledge of the parts is aggregated
larger wholes; (2) some approaches to sys- into knowledge of the whole: in synthetic
tems research lead directly to and depend thinking understanding of the containing
heavily on qualitative inquiry; and (3) a sys- whole is disaggregated to explain the parts. It
tems orientation can be very helpful in fram- does so by revealing their role or fitnction in
ing questions and, later, making sense out of that whole. Synthetic thinking reveals func-
qualitative data. tion rather than structure: it reveals why a sys-
Holistic thinking is central to a systems tem works the way it does, but not how it does
perspective. A system is a whole that is both so. Analysis and synthesis are compleinen-
greater than and different from its parts. In- tary: neither replaces the other. Systems think-
deed, a system cannot validly be divided ing incorporates both.
Variety in Qualitative Inquiry |J, 121

Because the effects of the behavior of the have become very important in family re-
parts of a system are interdependent, it can be search and therapy (Schultz 1984; Mont-
shown that if each part taken separately is gomery and Fewer 1988; Rosenblatt 1985;
made to perform as efficiently as possible, the Miller and Winstead-Fry 1982; Hoffman
system as a whole will not function as effec- 1981). A systems approach has also become
tively as possible. For example, if we select one of the central orientations to interna-
from ali the automobiles available thebestcar- tional development efforts in recent years.
buretor, the best distributor, and so on for each Specifically, the farming systems approach
part required for an automobile, and then try to development (Farming Systems Support
to assemble them, we will not even obtain an Project [FSSP] 1986) illustrates some unique
automobile, let alone the best one, because the ways of engaging in qualitative inquiry to
parts zoill not fit together. The performance of a support development, intervention, and
system is not the sum of the independent ef- evaluation from a systems perspective. The
fects of its parts; it is the product of their inter- farming systems approach to evaluation
actions. Therefore, effective management of a and research is worth examining in detail
system requires managing the interactions of because it has developed as a theory-based
its parts, not the actions of its parts taken sepa- yet practical solution to agricultural devel-
rately. (Gharajedaghi and Ackoff 1985:23-24) opment problems.
In the first three decades following World
War II, much international development
This kind of systems thinking has pro-
was conceived as direct technology transfer
found implica tions for program evaluation
from more developed to less developed
and policy analysis where the parts are often
countries. Scientists and change agents
evaluated in terms of strengths, weaknesses,
made technology transfer recommenda-
and impacts with little regard for how the
tions within their disciplinary areas of spe-
parts are embedded in and interdependent
cialization, for example, crops, livestock,
with the whole program or policy (Patton
water, and so on. This approach to develop-
1999c). For example, Benko and Sarvimaki
ment epitomized a mechanistic orientation.
(2000) applied systems theory as a frame-
In reaction to the dismal failures of the
work for patient-focused evaluation in nurs-
mechanistic, specialized technology transfer
ing and other health care areas. Such a
approach to development, a farming sys-
framework, they found, allowed complex
tems approach emerged (Shaner, Philipp,
features of processes in health care to appear
and Schmehl 1982b). Several elements are
by conducting simultaneous analyses of re-
central to a farming systems perspective, el-
latioriships on different leveis and with dif-
ements that lead directly to qualitative
ferent methods. This contrasts with the
methods of research.
mostly one-level, reductionist designs that
have usually been employed in nursing and
1. Farming systems research and develop-
health care research. Their "systemic
ment (FSRD) is a team effort (Shaner,
model" offered insights into system dy-
Philipp, and Schmehl 1982a).
namics in both "downward" and "upward"
directionsand the interconnections of 2. FSRD is interdisciplinary. The team con-
these systems dynamics in affecting patient sists of representatives from a mix of
care and outcomes. both agricultural and social science
In addition to their influence in organiza- disciplines (Cernea and Guggenheim
tional development, systems approaches 1985).
122 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

3. FSRD takes place in the field, on real A farming systems approach includes
farms, not at a university or goverrunent both qualitative and quantitative forms of
experiment station (Simmons 1985). inquiry. It includes direct observations, in-
4. FSRD is collaborativescientists and formal interviews, naturalistic fieldwork,
farmers work together on agricultural and inductive analysis, ali within a systems
productivity within the goals, values, framework. Well over 100 such projects in
and situation of participating farmers FSRD have been undertaken worldwide
(Galt and Mathema 1987). (FSSP 1987). There maybe no larger-scale ex-
ample of efforts to integra te naturalistic in-
5. FSRD is comprehensive, including atten- quiry, quantitative methods, and a systems
tion to ali farm family members; ali perspective through interdisciplinary eval-
farming operations, both crops and live- uation and research teamwork for the pur-
stock; ali labor sources; ali income pose of promoting long-term social and eco-
sources; and ali other factors that affect nomic developments.
small farm development (Harwood
FSRD is just one example of a systems ap-
1979).
proach to intervention, research, and evalu-
6. FSRD is inductive and exploratory, begin- ation. What this and other systems ap-
ning by open-ended inquiry into the proaches illustrate is that the complex world
nature of the farming system from the of human beings cannot be fully captured
perspective of those in the system and understood by simply adding up care-
(Holtzman 1986). fully measured and fully analyzed parts. At
7. FSRD begins with qualitative description. the system levei (the whole program, the
The first team task is fieldwork to quali- whole farm, the whole family, the whole or-
tatively describe the system (Sands ganiza tion, the whole community), there is a
1986). qualitative difference in the kind of thinking
that is required to make sense of what is hap-
8. FSRD is sensitive to context, placing the pening. Qualitative inquiry facilitates that
farming system in the larger agro- qualitative difference in understanding hu-
ecological, cultural, political, economic, man or "purposeful systems" (Ackoff and
and policy environments of which it is a Emery 1982).
part (Shaner et al. 1982a). A final story will reinforce this point, the
9. FSRD is interactive, dynamic, and process fable of the nine blind people and the ele-
oriented. The interdisciplinary team be- phant, which I used in the second chapter to
gins with inductive exploration, then illustrate the importance of context, and
moves to trying out system changes, ob- which I repeat here because it illustrates so
serving the effects, and adapting to well the real challenge of systems thinking.
emergent findings. The work is ongoing Besides, good stories have layers of mean-
and developmental (FSSP 1986). ing, and this one has phenomenological,
hermeneutic, and even ethnographic impli-
10. FSRD is situationalh/ responsive and adap- ca tions, which the reader may want to reflect
tive. There are many variations in FSRD on, but TIl simply reintroduce it as a systems
projects depending on priority prob- tale. Ironically, it is of ten offered as an exam-
lems, available resources, team member ple of systems thinking, but is, in its usual
preferences, and situation-specific pos- Western telling, actually quite linear and
sibilities (Sands 1986; FSSP 1987). mechanical.
Variety in Qualitative Inquiry |J, 123

As the story goes, nine blind people en- groups, programs. Chos or complexity
counter an elephant. One touches the ear theorists and researchers are primarily theo-
and proclaims that an elephant is like a fan. retical physicists, meteorologista, biologists,
Another touches the trunk and says the ele- and other natural scientists. Chos research
phant most surely resembles a snake. The has developed as a highly quantitative spe-
third feels the elephanfs massive side and cialty requiring supercomputer calculations
insists that it is like a wall. Yet, a fourth, feel- (Cambei 1992). But the assumptions that un-
ing a solidly planted leg, counters that it dergird chos theory pose challenges to so-
more resembles a tree trunk. The fifth grabs cial science research at the most fundamen-
hold of the tail and experiences the elephant tal leveis of basic conceptualization.
as a rope. And so it goes, each blindly touch- Complexity theory is already being
ing only a part and generalizing inappropri- viewed as a new paradigm of natural science
ately to the whole. The usual moral of the (Nadei and Stein 1995; Murali 1995; Hall
story is that only by putting ali the parts to- 1993; Holte 1993; Waldrop 1992; Gleick 1987;
gether in right relation to each can one get a Cronbach 1988). At least at the levei of meta-
complete and whole picture of the elephant. phor, chos and complexity notions are be-
Yet, from a systems perspective, such a ing used to inf orm approaches to economics
picture yields little real understanding of the (Ormerod 2001), anthropology (Agar 1999),
elephant. To understand the elephant, it organizational development (Eoyang 1997;
must be seen and understood in its natural Allison 2000), and leadership (Wheatley
ecosystem, whether in frica or Asia, as one 1992). The concepts of system and complex-
element in a complex system of flora and ity are often closely related. For example, the
fauna. Only m viewing the movement of a self-organization of systems, as premised by
herd of elephants across a real terrain, over complexity theory, implies the maintenance
time and across seasons, in interaction with of a certain levei of organization or the im-
plants, trees, and other animais will one be- provement of the systems (Rhee 2000). As
gin to understand the evolution and nature social scientists begin to understand its as-
of elephants and the system of which ele- sumptions, complexity theory about nonlin-
phants are a part. That understanding can ear dynamics may become a new paradigm
never come at a zoo. for approaching human complexities. In the
Thus, are we reminded of the challenge meantime, theory and research about non-
and importanceof bringing a systems linear dynamics (complexity) raise ques-
perspective into qualitative inquiry. tions about how we bring order to what we
observea fundamental epistemological
problem for ali forms of inquiry, including
Chos and Complexity Theory: qualitative inquiry. In Exhibit 3.5 (p. 126), I
Nonlinear Dynamics offer some teasers from Gleick's (1987) pop-
ularization of chos theory to suggest impli-
T-pu^daticmal question: cations for qualitative inquiry.
What is the- unde^iymg order, if At this point, complexity theory offers,
cmy, oj- disorderly phenomena? perhaps more than anything else, a new set
of metaphors for thinking about what we
What are disorderly phenomena? The observe, how we observe, and what we
weather, waterfalls, fluids in motion, volca- know as a result of our observations. Chos
noes, galaxiesand human beings, human theory challenges our need for order and
124 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

prediction, even as it offers new ways to ful- Michael Agar, a distinguished anthropol-
fill those needs. While much chos research ogist, used complexity theory, especially the
is highly mathematical, making sense of re- work of John Holland (1995, 1998), to inter-
sults seems to depend heavily on meta- pret fieldwork findings in his study of a her-
phors. Here is an intersection with qualita- oin epidemic among suburban youth in Bal-
tive inquiry that holds particular promise timore County, Maryland. He concluded:
because much work in qualitative analysis,
organiza tional development, and programs Complexity [theory] served, at least at the
includes resort to metaphor (Patton 2000; metaphorical levei, to better define a research
Ronai 1999; Brady 1998). Indeed, Gleick problemexplaining heroin trendsand it
(1987) offers a metaphor to explain the very helps articulate why traditional social re-
nature of inquiry into chos: "It's like walk- search has not answered this most basic ques-
ing through a maze whose walls rearrange tion of drug research: How and why do trends
themselves with every step you take" (p. 24). occur? It also points at the kind of data we
This metaphor fits a great deal of field- need to obtain and organize to do just that,
work in real-world settings, but the implica- however difficult that data might be to obtain.
tions can be so threatening to our need for Furthermore, complexity handles some cur-
order that we ignore the rearranging walls rent anthropological research issueslike the
and describe the maze with a single, static inclusion of the researcher, broadening histor-
diagram. If nothing else, the history and ical and political context, and the issue of pre-
emergent ideas of chos theory may give us dictionas part of its central themes. With
the comfort and courage to describe nonlin- characteristics like holism, emergence, and
ear dynamics (chos) when we find it, with- feedback that map onto anthropological as-
out imposing false order to fulfill the pre- sump tions more so than any previous formal
sumed traditional purpose of analysis. models, complexity is clearly worth a closer
Chos theory challenges us to deal with un- look. (Agar 1999:119)
predictability and indeterminism in human
behavior (Cziko 1989)and therefore in the The metaphors of chos, complexity, and
interventions (programs) we devise to alter nonlinear dynamics open up new possibili-
human behavior as well as the unpredict- ties for doing fieldwork in and understand-
ability and indeterminism of the methods ing those settings that feel like walking
we use to study and evaluate those interven- through a maze whose walls rearrange
tions. themselves with every step you take.

Grounded Theory

T he grounded theory approach is the most influential paradigm for quali-

tative research in the social sciences today.

Norman K. Denzin (1997a:18)

Variety in Qualitative Inquiry |J, 125

gies of theory development in contrast to theory

T^oundational question: generated by logical deduction from a priori
Wka+ tkeory emerges from assumptions.
systematic comparative analysis
and is grounded in fieldwork so In contrasting grounded theory with
as to expl^in wkat kas been and logico-deductive theory and discussing and
is observed? assessing their rela tive merits in ability to fit
and work (predict, explain, and be relevant),
we have taken the position that the adequacy
Now we tum from the fluidity of chos of a theory for sociology today cannot be di-
to the solidity of the ground, specifically, vorced from the process by which it is gener-
grounded theory. Most of the theoretical ated. Thus one canon for judging the use-
perspectives examined thus far focus on a fulness of a theory is how it was generated
particular aspect of human experience: Eth- and we suggest that it is likely to be a better
nography focuses on culture, ethnometh- theory to the degree that it has been induc-
odology on everyday life, symbolic inter- tively developed froin social r e s e a r c h Gen-
actionism on symbolic meanings in be- erating a theory from data means that most
havior, semiotics on signs, hermeneutics on hypotheses and concepts not only come from
interpretations, and phenomenology on the data, but are systematically worked out in
lived experience. Their theoretical frame- relation to the data during the course of the re-
works direct us to particular aspects of hu- search. Generating a theory involves a process of
man experience as especially deserving of research. (Glaser and Strauss 1967:5-6)
attention in our attempt to make sense of the
social world. In contrast, grounded theory This theory-method linkage is of great
focuses on the process of generating theory concem in many of the orientations exam-
rather than a particular theoretical content. ined in this chapter. The idea of a the-
It emphasizes steps and procedures for con- ory-method linkage means that how you
necting induction and deduction through study the world determines what you leam
the constant comparative method, compar- about the world. Grounded theory depends
ing research sites, doing theoretical sam- on methods that take the researcher into and
pling, and testing emergent concepts with close to the real world so that the results and
additional fieldwork. findings are grounded in the empirical
Concern for theory development is often world. Herbert Blumer (1978) has offered a
quite marked in the literature on qualitative metaphor for explaining what it means to
methods. The writings of Glaser (1978, generate grounded theory by being im-
2000), Strauss and Corbin (1998), Denzin mersed in the empirical world:
(1978b), Lofland and Lofland (1984), Blumer
(1969), Whyte (1984), and Becker (1970), to The empirical social world consists of on-go-
name but a few well-known qualitative ing group life and one has to get close to this
methodologists, take as a major focus the life to know what is going on in it. The meta-
task of theory construction and verification. phor that I like is that of lifting the veils that
What distinguishes the discussion of theory obscure or hide what is going on. The task of
in much of the literature on qualitative scientific study is to lift the veils that cover the
methods is the emphasis on inductive strate- area of group life that one purposes to study.
126 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Complexity (Chos) Theory Precepts and

EXHIBIT 3.5 Qualitative Inquiry Implications

Chos Precepts and Assumptions Implications for Qualitative

(Gltick 1987) Inquiry an Human Systems

1. "Nonlinearity means that the act of 1. The entry of the researcher into a setting
piaying the game has a way of changing may do more than create problems of
the rules" (p. 24). validity and reactivity. The researcher's
entry may make it a different setting
altogether-and forever.
2. A butterfly in Beijing flapping its wings 2. Small, minute events can make criticai
may affect the weather in New York differences. Qualitative importance is not
next month or next year. "The butterfly dependent on quantitative magnitude.
effect" has a technical na me: Sensitive For want of a n a i l . . . , the war was lost
dependence on initial conditions (p. 23).
3. A determinstic system can produce much 3. Much qualitative analysis attempts to
more than just periodic behavior. There bring orderfrom chos, identifying
can be "wild disorder" among "islands of patterns in the noise of human complexity.
structure." "A complex system can gve Chos theory suggests we need to learn to
rise to turbulence and coherence at the observe, describe, and value disorder and
same time," each of which is important turbulence without forcing patterns onto
(p. 56). genuine, meaningful chos.

4. "Simple systems can do complicated 4. What presumptions do we bring to field-

things" (p. 167). work and analysis about simplicityand
complexity? These are not neutral terms.
5. "A healthy body is a chaotic one; when 5. How do we observe and describe
you reach an equiibrium in biology you dynamic, constantly changing phenomena
are dead" (p. 298). without imposing a static structure by the
very boundaries we create in seeking to
define and understand?
6. "On the collective scale and on the 6. Chos theory's meanings and implications
personal scale, the ideas of chos for qualitative inquiry in human settings
advance in different ways and for remain to be developed.
different reasons" (p. 316).

The veils are not lifted by substituting, in odology that do not encourage or allow this
whatever degree, preformed images for betray the cardinal principie of respecting the
first-hand knowledge. The veils are lifted by nature of one's empirical world [T]he merit
getting close to the area and by digging deep of naturalistic study is that it respects and
in it through careful study. Schemes of meth- stays close to the empirical domain. (p. 38)
Variety in Qualitative Inquiry |J, 127

Ali of the approaches to theory and re- Let me be clear. Grounded theory is a general
search in this chapter use qualitative meth- method. It can be used on any data or combi-
ods to stay grounded in the empirical world. nation of data. It was developed partially by
Yet, they vary considerably in their concep- me with quantitative data. It is expensive and
tualizations of what is important to ask and somewhat hard to obtain quantitative data, es-
consider in elucidating and understanding pecially m comparison to qualitative data.
the empirical world. While the phrase Qualitative data are inexpensive to collect,
"grounded theory" is often used as a general very rich in meaning and observation, and
reference to inductive, qualitative analysis, very rewarding to collect and analyze. So, by
as an identifxable approach to qualitative in- default, due to ease and growing use,
quiry it consists of quite specific methods grounded theory is being linked to qualitative
and systematic procedures (Glaser 2000, data and is seen as a qualitative method, using
2001). In their book on techniques and proce- symbolic interaction, by many. Qualitative
dures for developing grounded theory, grounded theory accounts for the global
Strauss and Corbin (1998:13) emphasized spread of its use.
that analysis is the interplay between re- I can only caution the reader not to confuse
searchers and data, so what grounded the- this empirical use and the spread of its use
ory offers as a framework is a set of "coding with the fact that it is a general method. In
procedures" to "help provide some stan- some quarters of research, grounded theory is
dardization and rigor" to the analytical pro- considered qualitative, symbolic interaction
cess. Grounded theory is meant to "build research. It is a kind of takeover that makes
theory rather than test theory." It strives to routine qualitative research sound good by
"provide researchers with analytical tools positive stigma. Only highly trained grounded
for handling masses of raw data." It seeks to theory researchers can see the difference and
help qualitative analysts "consider alter- the confusion. Much of it revolves around the
native meanings of phenomenon." It em- notion of emergence versus forcing and the
phasizes being "systematic and creative si- lack of use of ali the grounded theory method-
multaneously." Finally, it elucidates "the ological steps. Any kind of data can be con-
concepts that are the building blocks of the- stantly compared. However, it is prudent for
ory." Glaser (1993) and Strauss and Corbin researchers to go with qualitative grounded
(1997) have collected together in edited vol- theory when that is where the resources are to
umes a range of grounded theory exemplars do it and when that is where researchers can
that include several studies of health (life af- reap career and personal rewards. (p. 7)
ter heart attacks, emphysema, chronic renal
failure, chronically ill men, tuberculosis,
Grounded theory has opened the door to
Alzheimer's disease), organizational head-
qualitative inquiry in many traditional aca-
hunting, abusive relationships, women
demic social science and education depart-
alone in public places, selfhood in women,
ments, especially as a basis for doctoral dis-
prison time, and characteristics of contem-
sertations, in part, I believe, because of its
porary Japanese society.
overt emphasis on the importance of and
While grounded theory has become specific procedures for generating theory. In
widely thought of as an approach specific to addition, I suspect its popularity may owe
qualitative inquiry, Glaser (2000) does not much to the fact that it unabashedly admon-
limit it in that way: ishes the researcher to strive for "objectiv-
128 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

ity." As discussed earlier in this chapter, the She believes that the guidelines for
postmodern attack on objectivity has found grounded theory offered by Strauss and
its way into qualitative inquiry through Corbin (1990, 1998) "structure objectivist
constructivism, hermeneutic interpretivism, grounded theorists' work. These guidelines
and the emphasis on subjective experience are didactic and prescriptive rather than
in phenomenology. Emergent autoethno- emergent and interactive" (Charmaz 2000:
graphic and heuristic approaches to qualita- 524). In contrast, she believes that in a
tive inquiry place even greater emphasis on constructivist grounded theory, "causality is
the researcher's personal and subjective ex- suggestive, incomplete, and indeterminate.
perience. Those social scientists and aca- . . . It looks at how 'variables' are grounded
demics who find some value in the methods given meaning and played out in subjects'
of qualitative inquiry, namely, in-depth in- lives. . . . Their meanings and actions take
terviewing and observation, but who es- priority over researchers' analytic interests
chew the philosophical undeipinnings of and methodological technology" (p. 524). To
constructivism and interpretivism can find illustrate a constructivist approach to
comfort in the attention paid to objectivity in grounded theory, she presents to the reader
grounded theory. the kinds of questions she would ask to
study a topic such as pain:
It is important to maintain a balance between
the qualities of objectivity and sensitivity I start by viewing the topic of pain subjectively

when doing analysis. Objectivity enables the as a feeling, an experience that may take a vari-

researcher to have confidence that his or her ety of forms. Then I ask these questions: What

findings are a reasonable, impartial represen- makes pain, pain? (That is, what is essential to

tation of a problem under investigation, the phenomenon as defined by those who ex-

whereas sensitivity enables creativity and the perience it?) What defining properties or char-

discovery of new theory from data, (Strauss acteristics do ill people attribute to it? When

and Corbin 1998:53) do they do so? How does the person expe-
rience this pain, and what, if any thing, does he
or she do about it? My questions aim to get at
At the same time, the language of meaning, not at truth. As a result, a con-
"grounded theory" has found its way into structivist grounded theory may remain at a
the constructivist literature. Charmaz (2000) more intuitive, impressionistic levei than an
compares "objectivist" (reality-oriented) and objectivist approach. (Charmaz 2000:526)
constructivist approaches to grounded the-
ory and, though she finds examples of both,
Beyond drawing on the inductive and
believes that the majority of grounded theo-
layered emphases in grounded theory la
rists are objectivist in orientation.
Strauss and Corbin, it is hard to see how
what Charmaz describes is different from
Objectivist grounded theory accepts the basic phenomenological inquiry. As a mat-
positivistic assumption of an externai world ter of philosophical distinctness, then,
that can be described, analyzed, explained, grounded theory is best understood as fun-
and predicted: truth, but with a small t . . . It damentally realist and objectivist in orienta-
assumes that different observers will discover tion, emphasizing disciplined and proce-
this world and describe it in similar ways. dural ways of getting the researcher 's biases
(p. 524) out of the way but adding healthy doses of
Variety in Qualitative Inquiry |J, 129

creativity to the analytic process. We shall what conceptual framework will direct
consider the analytic procedures of fieldwork and the interpretation of findings.
grounded theory in more detail in the chap- For example, one can undertake a study
ter on analyzing qualitative data. As a theo- from a feminist perspective, a Marxist per-
retical framework, I have included it in this spective, a capitalist perspective, or a Freud-
chapter because of its emphasis on generat- ian perspective, among others. In these in-
ing theory as the primary purpose of quali- stances, the ideological orientation or
tative social science and its overt embrace of perspective of the researcher determines the
objectivity as a research stance. focus of inquiry.
A feminist perspective presumes the im-
portance of gender in human relationships
Orientational Qualitative and societal processes and orients the study
Inquiry: Feminist Inquiry, in that direction (Guerrero 1999b; Ribbens
Criticai Theory, and Queer and Edwards 1998; Maguire 1996; Reinharz
Theory as Examples 1992; Glennon 1983; Smith 1979). Principies
of feminist inquiry (Guerrero 1999a:15-22;
One of the strengths of qualitative meth- Thompson 1992) can include
ods is the inductive, naturalistic inquiry
strategy of approaching a setting without o a sense of connectedness and equality
predetermined hypotheses. Rather, under- between researcher and researched;
standing and theory emerge from fieldwork n explicitly acknowledging and valuing
experiences and are grounded in the data. "women's ways of knowingj' including
The problem is how to approach the field integrating reason, emotion, intuition,
with an open mind. Phenomenology in- experience, and analytic tfiought;
cludes recommended procedures for be-
n participatory processei that support
coming clear about and taking into account
consciousness-raisiiig and researcher re-
biases and predispositions during both
flexivity; and /
fieldwork and analysis so as to get at the true
essence of the phenomenon under study. going beyond knowledge generation,
Hermeneutics takes the position that noth- beyond "knowledge for its own sake," to
ing can be interpreted free of some perspec- engage in using knowledge for change,
tive, so the first priority is to capture the per- especially "knowledge about women
spective and elucidate the context of the that will coritribute to women's libera-
people being studied. The researcher's own tion and einancipation" (Guerrero 1999a:
perspective must also be made explicit, as 16-17).
must any other tradition or perspective
brought to bear when interpreting mean- How does the lens of gender shape and
ings. affect our understandings and actions?
Orientational qualitative inquiry goes Philosopher Elizabeth Minnich has m-
one step farther. Orientational qualitative vestigated the ways in which conceptual
inquiry eschews any pretense of open- approaches to classifying human beings,
mindedness in the search for grounded or embedded historically, culturally, and po-
emergent theory. Orientational qualitative litically, continue to shape our thinking
inquiry begins with an explicit theoretical or through the very language and categories
ideological perspective that determines available to us. Her book on the subject,
130 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Transforming Knowledge (1990, forthcoming), inquiry challenges the phenomenological

speaks precisely and insightfully to the ori- notion that one can cleanse oneself of such
entation of feminist inquiry. fundamental language-based conceptions
when doing fieldwork and data analysis.
The root problem reappears in different guises Moreover, feminist inquiry provides not
in ali fields and throughout the dominant tra- only conceptual and analytical direction but
dition. It is, simply, that while the majority of also methodological orientation in empha-
humankind was excluded from education and sizing participatory, collaborative, change-
the making of what has been called knowl- oriented, and empoweringforms of inquiry.
edge, the dominant few not only defined them- A quite different theoretical framing for
selves as the inclusive kind of human but also as the inquiry would be a Freudian orientation that
norrn and the ideal. A few privileged men de- assumes that individual behavior must be
fined themselves as constituting mankind/ understood as a manifestation of the strug-
humankind and simultaneously saw them- gle between id, ego, and superego as influ-
selves as akin to what mankind/humankind enced by very early childhood relationships
ought to be in fundamental ways that distin- and sexual experiences that have left their
guished them from ali others. Thus, at the mark on the unconscious. Orientations can
same time they removed women and non- be combined, as in a feminist psychoana-
privileged men within their culture and other lytical framework (Eichenbaum and Orbach
cultures from "mankind," they justified that 1983).
exclusion on the grounds that the excluded Racism and ethnicity can be another de-
were by nature and culture "lesser" people (if fining lensor orientationfor qualitative
they even thought of the others as having "cul- inquiry in research and evaluation (Ladson-
tures"). Their notion of who was properly hu- Billings 2000; Stanfield 1999; Patton 1999d),
man was both exclusive and hierarchical with as can inclusiveness (Mertens 1998, 1999).
regard to those they took to be properly sub- "Queer theory," an orienta tio nal approach
ject to themwomen in ali roles; men who focused on sexual orientation, "took social
worked with their hands; male servants and constructionist insights and added a post-
slaves; women and men from many other cul- structuralist critique of the unified, autono-
tures. mous self," so a lesbian, gay, bisexual, and
Thus, they created root definitions of what
transgender orientation informs inquiry as
it means to be human that, with the concepts
"a deconstructive enterprise, taking apart
and theories that flowed from and reinforced
the view of a self defined by something at its
those definitions, made it difficult to think
core, be it sexual desire, race, gender, nation,
well about, or in the mode of, anyone other
or class" (Gamson 2000:348).
than themselves, just as they made it difficult One of the most influential orientational
to think honestly about the defining few. frameworks is "criticai theory," which fo-
(Minnich 1990:37-38) cuses on how injustice and subjugation
shape people's experiences and understand-
ings of the world.
The concepts and conceptual frameworks
we use, whether unconsciously as a matter A criticai social theory is concerned in particu-
of tradition and training or intentionally as a lar with issues of power and justice and the
matter of choice, carry embedded messages ways that the economy, matters of race, class,
about what and who is important. Feminist and gender, ideologies, discourses, education,
Variety in Qualitative Inquiry |J, 131

religion and other social institutions and cul- term orientational to describe such studies
tural dynamics interact to construct a social because they are oriented in a particular di-
system. . . . Inquiry that aspires to the name rection or framed from a specific perspec-
criticai mustbe connected to an atteinpt to con- tive. Orientational is a more neutral term
front the injustice of a particular society.... than ideologically based inquiry
Research thus becomes a transformative en- The extent to which any particular study
deavor unembarrassed by the label political is orientational is a matter of degree.
and unafraid to consummate a relationship Ethnographic studies can be viewed as
with emancipatory consciousness. (Kincheloe orientational to the extent that they presume
and McLaren 2000:281,291) the centrality of culture in explaining hu-
man experience. "Criticai ethnography"
Thus, what gives criticai theory its name (Thomas 1993) combines the focus on cul-
what makes it criticaiis that it seeks not ture with the commitinent to use findings
just to study and understand society but for change. Symbolic interactionism is
rather to critique and change society. Influ- orientational in focusing on the importance
enced by Marxism, informed by the pre- of the meanings that emerge as people de-
sumption of the centrality of class conflict in fine situations through interpersonal inter-
understanding community and societal actions. Orientational qualitative inquiry is
structures (Crotty 1998; Heydebrand 1983; a legitimate and important approach to the-
Carchedi 1983), and updated in the radical oretical or ideological elaboration, confir-
struggles of the 1960s, criticai theory pro- mation, and elucidation. What is required is
vides a frameworkboth philosophy and that the researcher be very clear about the
methodsfor approaching research and theoretical framework being used and the
evaluation as fundamentally and explicitly implications of that perspective on study
political, and as change-oriented forms of focus, data collection, fieldwork, and anal-
engagement. Fonte (2001) offers an example ysis.
of criticai theory applied to public policy.
Fonte applies the perspective of Marxist in-
Variety in Qualitative
tellectual Antonio Gramsci to contemporary
American politics, considering how domi- Inquiry: Different Answers
nant and subordinate groups based on race to Core Questions
and gender struggle over power in ways
that make every aspect of life political. Exhibit 3.6 summarizes the theoretical
Within any of these theoretical or ideolog- and philosophical perspectives presented in
ical orientations one can undertake qualita- this chapter. This is not an exhaustive list of
tive inquiry, but the focus of inquiry is deter- theoretical possibilities, but it does include
mined by the framework within which one the most common conceptual and philo-
is operating and findings are interpreted sophical frameworksand it certainly doc-
and given meaning from the perspective of uments the variety of perspectives that can
that preordinate theory. Such qualitative in- inform qualitative inquiry.
quiry, therefore, aims to describe and ex- No consensus exists about how to classify
plain specific manifestations of already-pre- the varieties of qualitative research. As
sumed general patterns. Such inquiry is noted in the opening of this chapter, but
aimed at confirmation and elucidation worth repeating as a review of variety in
rather than discovery. I have chosen the qualitative inquiry, Crotty (1998:5) elabo-
132 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

EXHIBIT 3.6 Variety in Qualitative Inquiry: Theoretical Traditions

Perspective Disciplinary Roots Central Questions

1. Ethnography Anthropology What is the culture of this group of people?

2. Autoethnography Literary arts How does my own experience of this culture
connect with and offer insights about this
culture, situation, event, and/or way of life?
3. Reality testing: Philosophy, social What's really going on in the real worid?
Positivist and realist sciences, and What can we establish with some degree of
approaches evaluation certainty? What are plausible explanations for
verifiable patterns? What's the truth insofar
as we can get at it? How can we study a
phenomenon so that our findings correspond,
as much as possible, to the real world?
4. Constructionl5m/ Socoogy How have the people in this setting
constructivism constructed reality? What are their reported
perceptions, "truths," explanations, beliefs, and
worldview? What are the consequences
of their constructions for their behaviors and
for those with whom they interact?
5. Phenomenology Philosophy What is the meaning, structure, and essence of
the lived experience of this phenomenon
for this person or group of people?
6. Heuristic inquiry Humanistic What is my experience of this phenomenon and
psychology the essential experience of others who also
experience this phenomenon intensely?
7. Ethnomethodoiogy Sociology How do people make sense of their everyday
activities so as to behave in socially
acceptable ways?
8. Symbolic interaction Social psychology What comrnon set of symbols and understand-
ings has emerged to give meaning to people's
interactions?

rated five major theoretical perspectives as tions of qualitative inquiry, but a different
the foundations of social research: positiv- five: biography, phenomenology, grounded
ism (and postpositivism), interpretivism theory, ethnography, and case study. Jacob
(which includes phenomenology, herme- (1987) chose yet a different five for a qual-
neutics, and symbolic interactionism), criti- itative taxonomy: ecological psychology,
cai inquiry, feminism, and postmodernism holistic ethnography, ethnography of com-
(to which he adds an "etc." to suggest the munication, cognitive anthropology, and
open-ended nature of such a classification). symbolic interactionism. Schwandt (2000)
Creswell (1998) also settled on five tradi- highlighted "three epistemological stances
Variety in Qualitative Inquiry |J, 133

Perspective Disciplinary Roots CentroI Questions

9. Semiotics Linguistics How do signs (words, symbols) carry and

convey meanng in particular contexts?
10. Hermeneutics Linguistics, philosophy, What are the conditions under which a
iiterary crticism, human act took place or a product was
theoiogy produced that makes it possible to interpret
its meanings?
11. Narratology/ Social sciences What does this narrative or story reveal about
narrative analysis (interpretive): the person and world from which it carne?
Literary crticism, How can this narrative be interpreted to
Iiterary nonfiction understand and illuminate the life and culture
that created it?
12. Ecoiogical psychology Ecology, psychology How do individuais attempt to accomplish
their goals through specific behaviors in
specific environments?
13. Systems theory Interdisciplinary How and why does this system as a whole
function as it does?
14. Chos theory: Theoretical physics, What is the underlying order, if any, of
Nonlinear dynamics natural sciences disorderly phenomenon?
15. Grounded theory Social sciences, What theory emerges from systematic
methodology comparative analysis and is grounded in
fieldwork so as to explain what has been
and is observed?
16. Orientational: ideologies: Poltica!, How is X perspective manifest in this
Feminist inquiry, cultural, and economic phenomenon?
criticai theory, queer
theory, among others

for qualitative inquiry": interpretivism, her- branches showing different "qualitative

meneutics, and social constructivism. strategies." Tesch (1990) identified 27 variet-
Denzin and Lincoln (2000a) organized their ies. Having examined some of the various
reviewof qualitative variety around seven attempts to classify qualitative approaches,
historical periods and seven "para- Miles and Huberman (1994) concluded, "As
digms/ theories": positivist/postpositivist, comprehensive and clarifying as these cata-
constructivist, feminist, ethnic, Marxist, cul- logs and taxonomies may be, they tum out
tural studies, and queer theory. Wolcott to be basically incommensurate, both in the
(1992) created a family tree of 20 distinct way different qualitative strands are defined
134 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

and m the criteria used to distinguish them. a How do we know what we know? (episte-
The mind boggles in trying to get from one mological debates about the possibility
to another" (p. 5). and desirability of objectivity, subjectiv-
Adding to this complexity is the practice ity, causality, validity, generalizability)
of combining some perspectives. For exam- a How should we study the world? (method-
ple, one can do a heuristic feminist (orien- ological debates about what kinds of
tational) study, that is, undertake a heuristic data and design to emphasize for what
inquiry from a feminist perspective. Or do purposes and with what consequences)
"criticai ethnography" (Thomas 1993),
What is worth knowing? (philosophical
combing elements of criticai theory and eth-
debates about what matters and why)
nography. Bentz and Shapiro (1998) have of-
fered what they call "mindful inquiry" as a o What questions should we ask? (disciplin-
synthesis of phenomenology, hermeneutics, ary and interdisciplinary debates about
criticai theory, and Buddhism. From phe- the importance of various burning ques-
nomenology they take the focus on experi- tions, inquiry traditions, and areas of in-
ence and consciousness. From hermeneutics quiry)
they take the focus on texts, on the process of How do we personally engage in inquiry?
understanding, and on letting new mean- (praxis debates about interjecting per-
ings emerge from the research process. From sonal experiences and values into the
criticai theory they direct attention to the so- inquiry, including issues of voice and
cial and historical context of both the re- political action)
searcher and the research topic, including
attention to domination, injustice, and op- The same program, organization, or com-
pression. From Buddhism they take the fo- munity studied by researchers from differ-
cus onbecoming aware of one's own "addic- ent perspectives will lead to quite different
tions" and attachments and on practicing studies even though they might ali under-
compassion. In positing this synthesis, they take observations, interviews, and docu-
aim to place the researcher, rather than re- ment analysis. Nor would it necessarily be
search techniques, at the center of the re- possible to synthesize the descriptions and
search process. This adds something of a re- findings of such different studies even
flexive, autoethnographic orientation as though they took place in the same setting.
another foundation of mindful inquiry be- When researchers operate from different
cause the mindful inquirer uses awareness frameworks, their results will not be readily
of personal, social, and historical context, interpretable by or meaningful to each other.
and personal ways of knowing, to shape the While the frameworks provide guidance
research. and a basis for interaction among research-
The variety of qualitative frameworks is ers operating within the same fr ame work,
distinguished by answers to six core ques- the different theoretical frameworks consti-
tions (one for each day of the week plus a tute barriers that impede interaction across
day left over to integrate your answers): and among different perspectives. In effect,
each theoretical framework is a minipara-
What do we believe about the nature ofreal- digm with its own internai logic and as-
ity? (ontological debates concerning the sumptions.
possibility of a singular, verifiable reality This means one cannot reasonably ask
and truth vs. the inevitability of socially which theoretical framework is "right," best,
constructed multiple realities) or most useful. It depends on what one
Variety in Qualitative Inquiry |J, 135

wants to do and which assumptions one Finally, a caution would seem in order
shares. Gareth Morgan (1983) stated the about the danger of reifying the theoretical
problem quite succinctly after presenting a distinctions offered in this chapter. Take a
variety of research perspectives: look again at Exhibit 3.1, my reply to a letter
from a graduate student desperate to figure
There was the question as to how the reader
out what category of inquiry she fit into. The
could come to some conclusion regarding the
boundaries between perspectives remain
contrary nature, significance, and claims of
fuzzy. Adherents within each perspective
the different perspectives. . . . I realized that
canbe found arguing about what is essential
there was a major problem h e r e . . . . There is a
to that perspective. Tom Schwandt, who has
fallacy in the idea that the proposi tions of a
studied these distinctions as much as any-
system can be proved, disproved, or evalu-
one and is the lexicographer of the Dictio-
ated on the basis of axioms within that system.
nary of Qualitative Inquiry (2001), offers this
. . . This means that it is not possible to judge
reflection on theoretical distinctions:
the validity or contribution of different re-
search perspectives in terms of the ground as- It seems to be a uniquely American tendency
sumptions of any one set of perspectives, since to categorize and label complicated theoretical
the process is self-justifying. Hence the at- perspectives as either this or that. Such label-
tempts in much social science debate to judge ing is dangerous, for it blinds us to enduring
the utility of different research strategies in issues, shared concerns, and pomts of tension
terms of universal criteria based on the impor- thatcut across the landscape of the movement,
tance of generalizability, predictability and issues that each inquirer must come to terms
control, explanation of variance, meaningful with in developing an identity as a social in-
understanding, or whatever are inevitably quirer. In wrestling with the ways in which
flawe: These criteria inevitably favor research these philosophies forestructure our efforts to
strategies consistent with the assumptions that understand what it means to " d o " qualitative
generatesuch criteria as meaningfulgitidelines for inquiry, what we face is not a choice of which
the evaluation of research. It is simply inade- labelinterpretivist, constructivist, herme-
quate to attempt to justify a particular style of neuticist, or something elsebest suits us.
research in terms of assumptions that give rise Rather, we are confronted with choices about
to that style of research Different research how each of us wants to live the life of a social
perspectives make different kinds of knowl- inquirer. (Schwandt 2000:205)
edge claims, and the criteria as to what counts
as significant knowledge vary from one to an-
other. (pp. 14-15)
li. P r a g m a t i s m
In other words, readers must make their
own decisions about the relative value of Having documented the variety of theoreti-
any given perspective. Each has strengths. cal perspectives that inform qualitative in-
Each has limitations. There is no universal quiry, we now leave the world of theory and
standard that can be applied to choose enter the world of practice and pragmatism.
among these different frameworks. Quite Not ali questions are theory based. Indeed,
the contrary, the diversity itself is a good in- the quite concrete and practical questions of
dicator of the complexity of human phe- people working to make the world a better
nomena and the challenges involved in con- place (and wondering if what they're doing
ducting research. is working) can be addressed without plac-
136 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Sample Internet E-mail Discussion Groups (listservs)

E X H I B I T 3.7 and Sites Relevant to Qualitative Inquiry and Theory

1. [email protected]: Use of ethnographic research methods in education; to

subscribe, send this message to [email protected]: join ethnography-in-education firstname
lastname
2. [email protected]: Ethnomethodology/conversation analysis; to subscribe, send this message to
[email protected]: join ethno ourname
3. http://www.tgsa.edu/online/cybrary/phenom.html: Phenomenology
4. http://www.ped.gu.se/biorn/phgraph/home.html: Phenomenography
5. www.groundedtheory.com/vidserie5l.html: Grounded theory Web site
6. [email protected]: Q Methodology discussion list on this broad approach to the study
of subjectivity; to subscribe, send this message to [email protected]: subscribe Q-METHOD
ourname; for help contact [email protected]
7. [email protected]: Biographicai Methods for the Social Sciences; to subscribe, send
this message to [email protected]: join BIOG-METHODS
LASTNAME
8. [email protected]: A discussion of narrative in everyday life; to subscribe, send this
message to [email protected],nz: subscribe psych-narrative
9. www.chass.utoronto.ca/epc/srb/cyber/cyber.html: Cyber Semiotic Institute
10. www.shop.affinia.com/ppsesystem5theory/5tore1/: Systems theory site
11. [email protected]: H-Net/Oral History Association Discussion List on Oral History; to
subscribe, send this message to [email protected] subscribe H-ORALHIST
LASTNAME AFFILIATION

NOTE: Thanks to Judith Preissle, Aderhold Distinguished Professor, Social Foundations of Education, University
of Gergia, for list subscription details. These sites and subscription detaJs may change, and this list is not ex-
haustive.This list is meantto be suggestive of the qualitative analysis resources available through the Internet.
See Chapter 1, Exhibit 1.5; Chapter 4, Exhbit 4.9; and Chapter 8, Exhibit 8.3, for additional qualitative re-
sources through the Internet.

ing the study in one of the theoretical frame- students writing dissertations and academic
works in this chapter. While these intellec- scholars will necessarily be concerned with
tual, philosophical, and theoretical tradi- theoretical frameworks and theory genera-
tions have greatly influenced the debate tion, there is a very practical side to qualita-
about the value and legitimacy of qualitative methods that simply involves asking
tive inquiry, it is not necessary, in m y opin- open-ended questions of people and observ-
ion, to swear vows of allegiance to any sin- ing matters of interest in real-world settings
gle epistemological perspective to use in order to solve problems, improve pro-
qualitative methods. grams, or develop policies. In short, in
Indeed, I would go farther (at the risk of real-world practice, methods can b e sepa-
being heretical) and suggest that one need rated jfrom the epistemology out of which
not even be concerned about theory. While they have emerged.
Variety in Qualitative Inquiry |J, 137

One can use statistics in straightforward able ways to f ind out what is happening in
ways without doing a philosophical litera- programs and other human settings.
ture review of logical empiricism or realism. The next chapter explores some of the
One can make an interpretation without ways in which qualitative inquiry can con-
studying hermeneutics. And one can contribute to practical knowledge and prag-
duct open-ended interviews or make obser- matic understandings. To help make that
vations without reading treatises on phe- transition, this chapter ends with a practical,
nomenology. The methods of qualitative cautionary tale from Halcolm.
inquiry now stand on their own as reason-

X k e . y V p p I e of \Souv-

After Halcolm had completed explaining to a scholarly assembly the many dif-
fering perspectives one could use in looking at the world, he was hungry. While he
answered questions and continued the discussion, he sent a listener to inquire if
the midday meai was ready. The messenger did not return, so Halcolm sent a sec-
ond messenger. The second messenger did not return. So Halcolm wenthimself.
He found the two messengers, the chef, and three visiting scholars engaged in
heated debate. Ignoring the debate, Halcolm asked, "Is the midday meai ready?"
The first and eldest visiting scholar responded, "I have been explaining to these
young men that the state of the food is not the only issue in determining readiness.
A meai is not just food. The meai must include those who would par take of the
food, so the meai is not ready until everything is in order and those who would eat
are assembled."
The second visiting scholar said, "I dared to taste the meai. From the perspec-
tive of a gourmet chef, this meai will never be ready. It is hopeless; let us return to
the city."
The third visiting scholar said, "Readiness is a state of mmd, not a physical
state. Since the food has no mind, the food cannot be ready. Only people can be
ready."
The chef added, "The midday meai is at midday every day. At midday the meai
is ready. Why ask if the meai is ready? It is midday. This is the meai. Therefore the
midday meai is ready."
With that, they ali began talking at once making ever finer points, drawing ever
narrower distinctions.
Halcolm, meanwhile, sat down and ate his midday meai.
A student asked why he had not joined the debate to clarify these important is-
sues. Halcolm took another bite and replied, "The apple of your eye won't satisfy
the emptiness in your stomach. There is a time to talk about the nature of eat-
ingand there is a time to eat."
From Halcolm's Guidefor Gourmands
APPENDIX 3.1
S S S
Example of Autoethnographic Writing

Introduction. This excerpt is from the first chapter of Grand Canyon Celebration:
A Father-Son Journey ofDiscovery (Patton 1999a). The excerpt combines inquiry
into a cultural phenomenon of interest with personal reflection on and experi-
ence of that phenomenon, in this case, male coming of age in modern society.
My son Brandon and I were joined by Malcolm, a friend and our Grand Can-
yon guide.

Vishnu Metamorphism

To see the enormity of the Grand Canyon you have to be orbiting the Earth.
To feel it, you have to descend within. To learn from it, you have only to stay
awhile and be present. At least thafs what Malcolm had claimed when he
first urged me to hike with him from Apache Point to Eives Chasm years ear-
lier. And learn I had, about bloody blisters, debilitating thirst, and the impor-
tance of moving quickly when a rock ledge gives way a thousand feet above
the canyon floor, especially if you're standing on it at the time. Modest
learnings. But they left an impression. As did the depth and beauty of the
Canyon.
Malcolm had been bringing questions about his life to the Canyon for
years. And getting answers. I had gotten no answers on that first trip. But
that, Malcolm explained, was because I had brought no questions. Fair
enough. I had come for the hike and a chance to walk among the oldest ex-
posed rocks on the Earth's surface.
But I did get an idea. Standing atop Mount Huethawali and staring across
the Colorado Ri ver at Holy Grail Temple, I imagined someday hiking with
my son, then just entering toddlerhood, and initiating him into manhood
there amidst buttes named King Arthur Castle, Guinevere Castle, and
Excalibur, and gorges named Merlin Abyss and Modred Abyss. Malcolm
called it a vision, which beguilingly transformed a passing notion into a
quest, like framing a telephone doodle and calling it art. What better place for
grandiosity than the Grand Canyon?
The gilt frame, however, didn't quite make it back with me to Minnesota. I
realized that I lacked a few of the basic necessities for conducting an initia-
tion. Tribal elders, for example. Hard to come by if you don't have a tribe. As
are other essentials, like tradition, a sacred place, ritual, terrifying gods to ap-
pease, wisdom to pass on, and life-threatening tests for the initiate to pass
(preferably ones that the initiator has successfully survived). From what I re-
Variety in Qualitative Inquiry |J.

called of anthropology, strong gender identity would also be a prerequisite.

That, however, might be conjured up. I had felt a vague sense of something
while gazing toward Lancelot Point. Malcolm suggested that the Canyon was
putting me in touch with my masculine collective unconscious. After ten days
in the Canyon such things could be said without soundmg absurd. Like eating
freeze-dried food. It can taste gourmet scrumptious after a hard day hiking, but
cooked at home, it's ghastly. So I found that my Canyon initiation vision didn't
reconstitute well mixed with urban fluoridated water.
But it also didn't evaporate.
Malcolm now smiles and says he never doubted. I, on the other hand, still
find myself amazed that we actually did return with Brandon for an initiation
experience. And, berng a social science researcher, I kept field notes. Not, I
should add, because I had any premonition that they might reveal something
important about a humanist approach to coming of age in contemporary soci-
ety. I did it for family history and, I concede, out of habit. I had spent too many
years in sociological observation to turn off that part of myself just because I
had brought my eldest son into one of the most magnificent landscapes on
earth after many years of anticipation. I considered leaving my scientific side
behind. I even tried .Justbe a father, I told myself. Justbe in the Canyon. Be pres-
ent with Brandon. Don't analyze it while it's happening. Stay with the experi-
ence.
Or were those Malcolm's admonitions? Certainly, some part of me was in-
trigued by Malcolm/s belief that he got answers from the Canyon. And, unlike
our first hike years earlier, this time I found I had come with a question, though
I wasn't fully aware of it until our second night.
We were campe d within the inner gorge, just short of the Colorado River,
where White Creek flows out of Muav Canyon into Shinumo Creek. My aching
body craved rest after two hard days hiking, but Brandon's after-dinner ques-
tions about how different cultures define manhood had left me tossing and
turning. He slept near enough for me to hear his slow, evenbreathing. As I stud-
ied him, he rolled from his back onto his side, pulling his knees up fetus-like, al-
most, but not quite, transforming his gangling, 18-year-old frame into a picture
of innocence. He looked like the question he had asked over dinner.
He had begun with a mocking tone: "So, this is my initiation. When do I find
out about the manhood thing? I'm sure you two have come prepared with im-
portant insights. Might as well get on with it. I, your humble initiate, am ali
ears."
Our subsequent anthropological discussion about how different societies
define manhood was rooted in cultural relativism as solidly as the large cotton-
wood that sheltered our campsite. The discussion had been serious, intense
and surprisingly lacking in satisfaction. Not for Brandon. For me.
As I gazed at Brandon sleeping, voices argued with each other in my head.
What does a modern father tell his son about being a man? Some voices, re-
corded in my memory long ago, rasped repeatedly like a worn needle stuck in
scratched grooves from the waxen days of graduate school. Others, more re-
140 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

cently entered, played intermittently through the scratches. The messages from
different eras competed to be heard, rising to a discordant crescendo, like being
caught in a small gym between opposing fans and their blaring pep bands at a
championship basketball gameexhilarating only if you know which side to
cheer for.
Such imagery being incongruent with my peaceful environs, though I enjoy
both debate and athletic competition, I redirected my inner musings to the
steady gurgle of nearby rapids and the chirping melodies of the canyon night. I
quietly got up to strollback and forth along the creek, pondering what I wanted
to pass on to Brandon about the nature of manhood. I paused in the shadow of
ancient rock and listened as Shinumo's rapids asserted the constant flow of the
present. I tried out possibilities on a disinterested moon: metaphors of male in-
candescence and female florescence.
What was left to tell Brandon that he hadn't already heard from me ad nau-
seam? I could affirm that the moon is disinterested, that the Canyon is rock, and
that life offers many pathways for being a man and developing as a person,
none of them certain. I could offer perspective and Canyon-inspired meta-
phors. . . .
This trip, this "initiation," felt like a last chance. When, if ever again, would I
have Brandon's undivided attention? Or at least some part of it? I was not quite
so delusional as to believe I could attain the impossibly high standard of "undi-
vided attention" with a member in good standing of the generation that grew
up on channel-surfing, But I did have ten uninterrupted days and nights with
Brandon. No outside influences. No competition from television, telephones,
friends or work.
Ten whole days with my son in the Grand Canyon. Ten days before he left homefor
college and the rest ofhis adult life. Ten final days. A last chance.
I returned to where Brandon slept and, gazing at him, considered whether it
much mattered what I had to saywords, after ali, being only words. But
words matter in my world, as do answers. Thought matters. And so I thought
some more until, under the influence of that elixir unique to the small hours
when the body is exhausted and the internai dialogue worn down, I experi-
enced at last a euphoria of analogical clarity. It came as I tumed and peered into
the dark gorge through which we had descended. That very afternoon, we had
traversed the Canyon's Great Unconformity, in one step passing through a gap
of 250 million years across a space that had once been filled with massive
mountains. Recalling that moment took me through what felt like a par aliei un-
conformity, insignificant by standards of Canyon time, but huge when mea-
sured on the modest scale of human evolution. Canyon metaphor offered so-
ciological insight. Malcolm would later say the Canyon had answered my
question.
The Canyon's Great Unconformity had once been filled with towering Pre-
cambrian formations of Bass Limestone, Hakatai Shale, Shinumo Quartzite,
Dox Sandstone, and Cardenas Basalts 800 million to one-and-a-quarter billion
Variety in Qualitative Inquiry |J, 141

years old. They had been turned sideways and thrust up higher than the Rock-
ies by monumental tectonic movements. During this churning, twisting and
thrusting, even more ancient rocks were exposed in places: hardened magma of
Zoroaster Granites and the oldest rock in the Grand Canyon, the metamor-
phosed lava-black Vishnu Schist, 1.7 billion years old. O ver millions of years
these mountains were eroded until the space they once occupied was filled
with sandstone deposited by encroaching seas.
When we arrived at the Great Unconformity, we joked about what it meant
to arrive some place that isn't there. As we hiked on within the depths of the in-
ner canyon, we marveled at the dramatic transition from sand and gravei to
sculptured stone, its significance gradually penetrating with the cold feel of the
marble-like rock. Now, inspired by the memory of that geologic gap, I contem-
plated the chasm that exists between modern society and ancient times. Many
experience the gap as a painful loss. Lately, contemporary male elders have
been trying to fill in the gap, build a bridge back or at least make a connection.
They hope a return to ancient initiation rites will help close the gap. I had been
attracted by that possibility myself, but Brandon's reactions during our hike
down said it wouldn't work, at least not for modern young people who have
tasted choice, experienced the power of intellect, learned to value individuality
and abhor control. The Great Unconformity impressed on me the gap between
past and present when societal customs have been eroded to the point of van-
ishing. Our ancestral past will necessarily and inevitably remain a foundation,
like the ancient Vishnu Schist, formed by 75,000 pounds per square inch of tec-
tonic pressure and named for the Hindu god, the Preserver. The Tapeats forma-
tion now rises atop that preserved foundation, but is neither part of it nor con-
tinuous in time.
I imagined a contemporary coming of age journey that recognizes ancient
foundatons of human experience, but is separate and distinct in accordance
with modern discontinuities and the great unconformity of human potential in
our timesa coming of age process that does not require the societal equivalent
of 75,000 pounds per square inch of pressure to assure conformity. Indeed, a
coming of age process that does not even have conformity as its goal. That
would be the greatest unconformity.
In elucidating the role of traditional initiation for Brandon, Malcolm, my
longtime friend and Canyon guide, also an anthropologist and family thera-
pist, had explained that initiation rites functioned to psychologically separate
sons from parental domination in tribal societies with extended families where
generations would live together in a confined village space. But in modern soci-
ety, just the opposite is the case. Our children are separated from us by daycare,
schools, music, television, peer groups and easy geographic mobility. The chal-
lenge of contemporary times is not to provide for the physical and psychologi-
cal separation of children from parents. Society has evolved multiple mecha-
nisms for detachment. Parents and children today are subjected to
unprecedented centrifugai forces. The challenge now is to bond.
142 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

I thought I had come to the Grand Canyon for a ritual of initiationrecog-

nizing and celebrating Brandon's manhood. But as we had descended into the
inner Canyon, the focus shifted for me. There, in the moonlight, I admitted why.
He was leaving home and going off to college. We needed no ceremony to rec-
ognize his independence. It was not in doubt. Nor was his manhood. What I
craved, that ancient rituais could not provide and had not been designed to ar-
range, was connection.
Abruptly, propelled by the force of illusory insight, I turned again away from
the rapids toward Brandon and sleep. A piercing pain in my leg stopped me. I
had connected with a Prickly Pear cactus. Examining the offending thom, I
heard my voice say: "Reality-check." Suddenly self-conscious, I looked
around, then laughed out loud at the ridiculous figure I presented: pacing the
canyon floor dressed only in the ephemeral threads of an emerging sociological
paradigm shift.

SOURCE: Patton (1999a:21-27). Copyright 1999 by Michael Quinn Patton.

Particularly Appropriate
Qualitative Applications

; p p ^ e k v H c e s k i p in T - V a g m a + i s m

A young carp enter, at the beginning of his career, carne to Halcolm in distress.
He had studied diligently to master carpentry. At the completion of his appren-
ticeship, the master carpenters said that his technical competence and skill were
unmatched for one so young.
Halcolm knew ali this, for word of the young man's mastery had reached
even the great one. Yet, Halcolm could also see that the young carpenter was in
great distress. "What troubles you?" Halcolm asked gently.
"My parents, my townspeople, my master teachers have been most generous.
Upon completion of my apprenticeship, they joined together to give me a fine
set of tools. Ihave been trained by the best. I am told that my skills arewhat can
I say without being immodest?my skills are adequate." The young man
paused, his distress obvious and growing even as he spoke.
"Then what is the problem?" asked Halcolm. The young man looked down,
embarrassed in the presence of the great one. It was a long time before he spoke,
and then only in a whisper. "I have no thing to build."
"Ah, I see," said Halcolm.
"No one will give me any orders," continued the young man.

13. 143
170LEI.CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

"Let me make some inquiries," offered Halcolm. "Return in a week and IT1 tell
you what I have learned."
The seven days were as an agonizing eternity for the young man. At last it was
time to find out what Halcolm had learned through his discrete interviews with a
few knowledgeable and well-connected people.
"I have confirmed ali that you told me," Halcolm began. "Your skill is much re-
spected. Your tools are the finest, given with much affection. Your competence is
not in doubt. And yet, you have nothing to build."
The young man waited for Halcolm to continue.
"During your apprenticeship you did the latticework on the new cathedral.
The craftsmanship you displayed is admire d by ali. You designe d and constructed
the intricate woodwork of the new town hall directrixanother work of great art
admire d even by your masters. You carved and installed the elabora te wine racks
in the guardian's estate. In ali these efforts you have distinguished yourself and
pleased those for whom you built."
The young man was pleased but perplexed as he heard Halcolm affirm the
quality of his work. Indeed, hearing the affirmations deepened his distress atfind-
ing himself now with nothing to build. Halcolm continued.
"You now have nothing to build because the townspeople believe your artistry
and craftsmanship are far superior to their simple needs. They need simple chairs,
tables, and doors. You work on cathedrals, town halls, and estates. You have de-
signed objects of great beauty and complexity. You have not designed and built
objects of great simplicity and practicality. To do the latter only looks easier, but
takes no less skill.
"Build me a simple, functional, and practical bookcase at reasonable cost, and
let us see what the townspeople think. Apply your skills to the everyday needs of
the people and you shall not lack for work."
Halcolm expected the young man to be delighted at the prospect of a solution to
his problemand regular employment. Instead, he saw the distress deepen into
despair.
"I do not know how to build simple, practical, and functional things," lamented
the young man. "I have never applied my skills and my tools to such things."
"Then your apprenticeship is not over," said Halcolm.
And they went down to Halcolm's workshop where the young man began to
learn anew.

From Halcolm's Applied Arts and Sciences

Particular!}/ Appropriate Qualitative Applications j. 145

J. Practical Purposes and that this review, while including a great vari-
Concrete Questions ety of applications, is far from exhaustive.
My purpose is to expand the horizons of
Qualitative methods are first and foremost what is possible and appropriate for both
research methods. They are ways of finding practitioners and decision makers. Because
out whatpeople do, know, think, and feelby the opportunities for qualitative inquiry are
observing, interviewing, and analyzing so vast, the examples offered here can be no
documents. The last chapter reviewed how more than teasers, merely hinting at the
qualitative methods contribute to generat- enormous array of qualitative applications
ing and confirming social science theory that are possible.
This chapter reviews how qualitative meth- Moreover, I have not examined in this
ods can contribute to useful evaluation, prac- chapter how the varying theoretical and
tical problem solving, real-world decision paradigmatic approaches discussed in the
making, action research, policy analysis, previous chapter might affect inquiry into
and organizational or community develop- any of these practical issues. For example,
ment. This chapter offers examples of how one might examine program quality, the first
qualitative methods can help answer con- topic below, phenomenologically, ethno-
crete questions, support development, and graphically, or heuristicallyto cite but
improve programs. Ali of these are ways of three possibilities. Or one might simply con-
contributing to what is sometimes called duct interviews and gather observa tion data
"action science" (Argyris, Putnam, and to answer concrete program and organiza-
Smith 1985). tional questions without working explicitly
Qualitative methods are not appropriate with a particular theoretical, paradigmatic,
for every inquiry situation. The aim of this or philosophical perspective. Well-trained
chapter is to illustrate when it may be partic- and thoughtful interviewers can get mean-
ularly appropriate to use qualitative meth- ingful answers to practical questions with-
ods. Certain purposes, questions, problems, out making a paradigmatic or philosophical
and situations are more consonant with pledge of allegiance. Pragmatic and utilitar-
qualitative methods than others. This chap- ian frameworks can guide qualitative in-
ter samples some of the research and evalua- quiry on their practical and applied under-
tion questions for which qualitative inquiry pinnings without having to be attached to or
strategies are especially appropriate and derived from a theoretical tradition. Prag-
powerful. The actual and potential applica- matism, then, is the foundational orienta-
tions of qualitative methods are so diverse tion of this chapter.

A Focus on Quality

^ ^ o m e , give us a taste of your quality.

William Shakespeare, Hamlet, Act II, scene ii

Quality care. Quality education. Quality agement. Continuous quality improvement.

parenting. Quality time. Total quality man- Quality control. Quality assurance. Malcolm
146 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Baldridge National Quality Award. Quality quality movementW. Edwards Deming

is the watchword of our times. People in and Joseph M. Juranwere preaching qual-
"knowle dge-intensive societies . . . prefer ity in manufacturing before World War II. In
'better' to 'more'" (Cleveland 1989:157). the 1930s, for example, Juran was applying
More requires quantitative dimensions; concepts of empowered worker teams and
better evokes qualitative criteria. continuous quality improvement to reduce
At the most fundamental levei, the de- defects at Western Electric's Hawthorne
bates about abortion, on the one hand, and Works in Chicago (Deutsch 1998; Juran
death with dignity and physician-assisted 1951). Deming and his disciples have long
suicide, on the other, concern, in part, what viewed quality from the customer's per-
is meant by "quality of life" (Humphrey spective, defining quality as meeting or ex-
1991). Kenneth E. Boulding (1985), one of the ceeding customer expectations. More than
most prominent intellectuals of the modern 20 years ago, Philip B. Crosby (1979) wrote a
era, devoted a book to the grand and grandi- best-selling book on "the art of making qual-
ose topic of human betterment. In that book, ity certain." Some of his assertions have be-
he defined development as "the learning of come classic:
quality" (Chapter 8). He struggled, ulti-
ma tely in vain, to define "quality of life." He o The first struggle, and it is never over, is
found the ide a beyond determina tive expli- to overcome the "conventional wisdom"
cation and certainly beyond numerical mea- regarding quality (p. 7).
surement. It is a subject particularly well
o The cost of quality is the expense of do-
suited for in-depth, holistic qualitative in-
ing things wrong (p. 11).
quiry.
Concern for quality surrounds us in the H Quality is ballet, not hockey (p. 13).
postmodern age. Quality has become the H The problem of quality management is
primary marketing theme of our time, for not what people don't know about. The
example, "Quality Is Job One" (Ford Motor problem is what they think they do know
advertising slogan). Customers demand (p. 13).
quality. This may stem, at least in part, from
H Quality management is a systematic way
the fact that in the busy lives we now lead, at
of guaranteeing that organized activities
least in postindustrial society, we simply
happen the way they are planned. . . .
don't have time for things to break down.
Quality management is needed because
We don't have time to wait for repairs. We
nothing is simple anymore, if indeed it
can't afford the lost productivity of not hav-
ever was (p. 19).
ing things work (either products or pro-
grams). We have taken to heart the admoni-
tion that, in the long rim, it is cheaper do it Efforts to implement these and other
right the first timewhatever "it" or "right" principies swelled crescendo-like into a
may refer to. It is within this larger societal national movement as management con-
context that we shall examine what qualita- sultants everywhere sang of total quality
tive inquiry brings to the challenge of study- management and continuous quality im-
ing and evaluating quality. provement. The music began in the corpo-
The current mania spotlighting quality rate sector, but by the early 1990s the "cult of
can give the impression that this is a rela- total quality" had permeated deeply into the
tively recent concern, but the founders of the government and nonprofit sectors (Walters
Particular!}/ Appropriate Qualitative Applications j. 147

1992). The Malcolm Baldridge National school's documented history of high-quality

Quality Awards became, and remain, the education.
pinnacle of recognition that the mountain- Understanding what people value and
top of quality has been reached. the meanings they attach to experiences,
Nor was concern about quality limited to from their own personal and cultural per-
management books. Robert Pirsig's classic spectives, are major inquiry arenas for quali-
Zen and the Art of Motorcycle Maintenance tative inquiry. This is especially true when
(1984) was an investigation into quality and making judgments about quality, or valuing,
excellence, themes he revisited in Lila (1991) which brings us to evaluation and quality
ashe explored the "metaphysics of quality/' assurance.

What the Metaphysics of Quality adds to Quality Assurance

James' pragmatism and his radical empiricism is and Program Evaluation
the idea that the primai reality from which
subjects and objects spring is value. By doing
Program evaluation and quality assur-
so it seems to unite pragmatism and radical
ance have developed as separate functions
empiricism into a single fabric. Value, the
with distinct purposes, largely separate lit-
pragmatic test of truth, is also the primary em-
eratures, different practitioners, and vary-
pirical experience. The metaphysics of quality
ing jargon. Each began quite narrowly, but
says pure experience is value. Experience
each has broadened its scope, purposes,
which is not valued is not experienced. The
methods, and applications to the pomt
two are the same. This is where value fits.
where there is a great deal of overlap and,
Value is not at the tail-end of a series of super-
most important, both functions can now be
ficial scientific deductions that puts it some-
built on a single, comprehensive program
where in a mysterious undetermined loca tion
information system.
in the cortex of the brain. Value is at the very
front of the empirical procession. (Pirsig
Program evaluation traces its modem be-
1991:365) ginnings to the educational testing work of
Thorndike and colleagues in the early 1900s.
Program evaluation was originally focused
An important policy question of our time on measuring attainment of goals and objec-
is whether educational quality can be tives, that is, finding out if a program
achieved through state-imposed standards "works," that is, if it's effective. This came to
and mandates. One example of the differ- be called summative evaluation, which origi-
ence between theory and practice in this re- nally relied heavily on experimental designs
gard is captured astutely in a case study by and quantitative measurement of outcomes.
Goodson and Foote (2001) that describes In recent years, program improvement (for-
what happened when a highly successful, mative) evaluation has become at least as
innovative, and creative alternative to tradi- important and pervasive as summative
tional education was confronted by the de- evaluation (Patton 1997a).
mands of contemporary standardized ac- Quality assurance (QA) in the United
countability. They chronicle the resistance of States had its official birth with the passage
a particular school, the Durant School, to the of the Community Mental Health Act
imposition of state standards and mandated Amendments of 1975 (Public Law 94-63).
tests in an effort, ironically, to maintain the This law required federally funded commu-
148 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Important methods of QA include clinicai

case investigations and peer reviews. Ali
mM:m T A
cases that fail to meet certain standards are
PFRWIVf rriNC- :;;'v
reviewed in depth and detail. For example,
"Tii ! ;'"i!Vi:! ir i:!;M:=i;!; iT!.! lH,iT,rilji . i1! i patients who remain hospitalized beyond
! ..-ii i:ij-i frriiiVi-W sii !j/jftiJi Ys- ;i l l ^ j f ;ii|-T! . an accepted or expected period may trigger
j. i^^riii:'!5!!!.!!, "liri.1!1! ; a review. An original difference between
:. , iT:jivi|i MÎV:! | : ; ' r i t ^ f ; . i' i! li !;'i!'i; jHrtftj . :
;
program evaluation and QA was that QA fo-
: . . : r Y i ! ' i ; i . !:h!i iiii h : i ; i ! ! ^ ' j ^ f e i i f l I. !V! .!<h:i i.j;
: ;
:
:
; !
: cused on individuais on a case-by-case basis,
M ftfM -M. i i - j f o l i i ^ ^: i . ! V y . V r l . ; while program evaluation focused on the
âWd, }d|l;i;vi:i]:iii:!Jfcri:^^îrifii'!!|!:< overall ! program. The traditional concems
with unique individual cases and quality in
ftalAY&rttl11:!.^^ Wtf^^ : QA systems continue to be quite consonant
!:iT 'rn-!Yii' :ti"l'r! :!iJi:iIM-PI!-'! r u
!
: \ with qualitative methods. Moreover, QA ef-
H*!'. !:;i:i- ji;!V: m & i . forts have now moved beyond health care to
V :-,' Vji^l rh!i-i:;!: ii!i the full spectrum of human service pro-
grams and government services (Human
rnriptu-ktA$ r ; j ; ' { f j & ^ t . :,v; Services Research Institute 1984).
jinap-nsT!!! ^ f g f l , S ^ ^ f l i : In-depth reviews of the quality of care for
participants m programs can draw heavily
on clinicai case files only if the files contain
approprate and valid information. When
nity mental health centers to have QA efforts files are to be used for research and evalua-
with utilization and peer review systems. Its tion purposes, clinicians need special train-
purpose was to assure funders, including ining and support in how to gather and report
surers and consumers, thatestablished stan- highly descriptive qualitative data in clinicai
dards of care were being provided. QA sys- case files (Cox 1982). Because there can be
tems involve data collection and evaluation great variation in the quality of clinicai case
procedures to document and support the records, particularly the descriptive quality,
promise made by health and mental health a program of QA must include evaluation of
care providers to funding sources, including and attention to the quality of qualitative
third-party insurance carriers and consum- data available for QA purposes.
ers, "that certain standards of excellence are Moreover, it is useful to distinguish qual-
being met. It usually involves measuring the ity control from quality enhancement. Qual-
quality of care given to individual clients in ity control efforts identify and measure
order to improve the appropriateness, ade- minimum acceptable results, for example,
quacy, and effectiveness of care" (Lalonde minimum competency testing in schools or
1982:352-53). As QA systems have devel- maximum acceptable waiting times before
oped, special emphasis has been placed on seeing a physician in an emergency room.
detecting problems, correcting deficiencies, Quality enhancement, in contrast, focuses
reducing errors, and protecting individual on excellence, that is, leveis of attainment
patients. In addition, QA aims to control well beyond minimums. Quality control re-
costs of health care by preventing over- quires clear, specific, standardized, and
utilization of services and overbilling by measurable leveis of acceptable results. Ex-
providers. cellence, however, often involves individu-
Particular!}/ Appropriate Qualitative Applications j. 149

B38njlsl''iW Comparing Program Evaluation and Quality Assurance

Progrom Evaluation Quality Assuronce

1. Focus on program processes and 1. Focus on individual processes and

outcomes outcomes
2. Aggregate data 2. Individual clinicai cases
3. Goals-based judgment 3. Professional-based judgment
4. intended for decision makers 4. Intended for clinicai staff

alization and professional judgment that control, but attention has shifted to concern
cannot and should not be standardized. Ex- for quality enhancement. Program evalua-
cellence is manifest in quality responses to tion began with an emphasis on summative
special cases or especially challenging cir- judgments about whether a program was ef-
cumstances. Thus, while quality control re- fective or not but has shifted to improving
lies on standardized statistical measures, program effectiveness. In their shared con-
comparisons, and benchmarks, quality en- cern for gathering useful information to sup-
hancement relies more on nuances of judg- port program improvement, program eval-
ment that are often bst captured qualita- uation and QA now overlap and find
tively through case studies and cross-case common ground. Accountability demands
comparisons. can be well served, in part, by evidence that
Traditionally, given their different ori- programs are improving.
gins, program evaluation and QA have had Both accountability and program im-
different emphases. These differences are provement require comprehensive program
summarized in Exhibit 4.1. information systems. We've learned that
The distinctions between QA and pro- such systems should be designed with the
gram evaluation have lost much of their im- direct involvement of intended users; that
portance as both functions have expanded. information systems should be focused on
Program evaluation has come to pay much criticai success factors (not data on every
more attention to program processes, imple- variable a software expert can dream up);
mentation issues, and qualitative data. QA that systems should be streamlined with
has come to pay much more attention to out- utility in mind; and that program improve-
comes, aggregate data, and cumulative in- ment systems benefit from both qualitative
formation over time. What has driven the and quantitative information, both case data
developments in both program evaluation and aggregate data. Indeed, harking back to
and QAand what now makes them more the opening discussion about total quality
similar than differentis concern about pro- management and continuous quality im-
gram improvement and gathering really provement, the systems that support such
useful information. Both functions had their efforts began with a heavy emphasis on sta-
origins in demands for accountability. QA tistical process control and "objective" indi-
began with a heavy emphasis on quality cators of performance, but have come in-
150 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

creasingly to value qualitative perspectives on focuses on what spelling means to the stu-
quality. It turns out that one of the particular dent. How is spelling integrated into the
strengths of qualitative inquiry, perhaps student's approach to writing? How does
commonsensically, is illuminating the na- the student think about spelling, approach
ture and meaning of quality in particular spelling, feel about spelling? The answer to
contexts. This takes on added significance such questions requires description of indi-
since "quality of life has become a com- vidual students' perspectives and situations
monly used concept and is showing grow- such that the meaning of the experience for
ing significance in economic and political the students is elucidated.
terms . . . [and] has two aspects, psychologi- The same distinction holds with regard to
cal and environmental, [yet] some research- programs that emphasize deinstitution-
ers have totally neglected the perception of alizationfor example, community mental
the people" (Turksever and Atalik 2001:163). health programs, community corrections,
There are many aspects of program oper- and community-based programs for the el-
ations, includmg implementation activities derly. It is possible to count the number of
and client outcomes, that can be measured in people placed in the community. It is possi-
terms of relative quantity. It makes sense to ble even to measure on standardized scales
count the number of people who enter a pro- certain attributes of their lives and liveli-
gram, the number who leave the program, hoods. It is possible to have them subjec-
and the number who receive or report some tively rate various aspects and dimensions
concrete benefit from the program. How- of quality of life. However, to fully grasp the
ever, many attributes of programs do not meaning of a change in life for particular
lend themselves to counting. Even the quan- persons it is necessary to develop a descrip-
titative scaling of quality attributes is an in- tion of life quality that integrates interde-
adequate way of capturing either program pendent dimensions of quality into a whole
quality or the effect of a program on the that is placed in context: What is their daily
quality of life experienced by participants life like? Who do they interact with? How do
during and after the program. they perceive their lives? How do they make
For example, school outcomes can be sense of what they experience? What do they
looked at both in terms of quantity of change say about the path they are on? How do they
and quality of change. Quantity of change talk about their quality of life? What do they
may involve the number of books read; a compare themselves to when deciding how
score on a standardized achievement test; well they're doing? These are areas of quali-
the number of words spelled correctly; and tative inquiry that support quality enhance-
the number of interactions with other stu- ment efforts and insights.
dents, the teacher, or people of a different Quality has to do with nuance, with de-
race. Each of these outcomes has a corre- tail, and with the subtle and unique things
sponding quality dimension that requires that make a difference between the points on
description rather than scaling. Thus, to find a standardized scale. In-depth quality de-
out what it means to a student to have read a scriptions can illuminate what the lives and
certain number of books is an issue of qual- perspectives of two different people are like,
ity. How those books affected the student one of whom responded on a scale of 5
personally and intellectually is a question of pomts that an experience was "highly satis-
quality. In contrast to counting the correct factory," while the other responded that it
number of words spelled, the quality issue was an "extremely satisfying" experience.
Particular!}/ Appropriate Qualitative Applications j. 151

This is not a question of interval versus ordi- ability-driven evaluation. The accountabil-
nal scaling, but one of meanings. What do ity movement is not so much about achiev-
programs mean to participants? What is the ing quality (the previous section) as it is
quality of their experiences? Answers to about demonstrating responsible use of
such questions require detailed, in-depth, public funds to achieve politically desired
and holistic descriptions that represent peo- results. The U.S. Government Performance
ple in their own terms and that get close and Results Act (GPRA) of 1994 mandates
enough to the situation being studied to un- outcomes reporting by government agen-
derstand firsthand the nuances of quality. cies. Philanthropic foundations are de-
The failure to find statistically significant manding outcomes evaluation (Porter and
differences in comparing people on some Kramer 1999) as are health care systems
outcome measure does not mean that there (Morse, Penrod, and Hupcey 2000). Indeed,
are no important differences among those in every arena of actionhealth, education,
people on those outcomes. The differences criminal justice, employment, mtemational
may simply be qualitative rather than quan- developmentemphasis has shifted from
titative. A carpenter is reported to have ex- providing services to attaining priority out-
plained this point to William James. The car- comes. A good example of this emphasis is
penter, having worked for many different the widely used United Way (1996) manual
people, observed, "There is very little differ- Measuring Program Outcomes:
ence between one man and another; but
what little there is, is very important." Those In growing numbers, service providers, gov-
differences are differences of quality. ernments, other funders, and the public are
calling for clearer evidence that the resources
they expanded actually produce benefits for
Evaluation Applications people. Consumers of services and volunteers
who provide services want to know that pro-
Outcomes Evaluation grams to which they devote their time really
make a difference. That is, they want better ac-
For programs engaged in healing, transforma- countability for the use of resources. One clear
tion, and prevention, the best source and form and compelling answer to the question of
of information are client stories. It is through "Why measure outcomes?" is: To see if pro-
these stories that we discover how program grams really make a difference in the lives of
staff interact with clients, with other service people. (p. 4)
providers, and with family and friends of their
clients to contribute to outcomes, and how the However, reading this manual one would
clients, themselves, grow and change in re- think that the only way to docurnent out-
sponse to program inputs and other forces and comes attainment is with numbers. The fo-
factors in their lives. There is a ricliness here cus is entirely on numerical indicators of
that numbers alone cannot capture. It is only outcomes and statistics in accomplishments.
for a story not worth telling, due to its inherent Percentage increases in desired outcomes
simplicity, that numbers will suffice. (Kibel (e.g., higher achievement test scores) and
1999:13) percentage decreases in undesirable out-
comes (e.g., reductions in rates of child
Outcomes evaluation has become a cen- abuse and neglect) are important to provide
tral focus, if not the central focus, of account- concrete evidence of overall patterns of ef-
152 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

fectiveness. What such statistics cannot do, behind the numbers, capture unintended
however, is show the human faces behind impacts and ripple effects, and illuminate
the numbers. This is important to provide dimensions of desired outcomes that are dif-
criticai context when interpreting statistical ficult to quantify (e.g., what it means for
outcomes as well as to make sure that the someone to become "self-sufficient"). Such
numbers can be understood as representing qualitative data can add significantly to sta-
meaningful changes in the lives of real peo- tistical reporting to create a more compre-
ple. hensive accountability system.
In an adult literacy program, the test re- Detailed case studies can be even more
sults showed an average increase of 2.7 important when evaluating outcomes at-
grade leveis over a three-month period. The tainment for program improvement (as op-
people in this sample included posed to externai accountability reporting).
To simply know that a targeted indicator has
a Puerto Rican man who was learning to been met (or not met) provides little infor-
read English so that he could help his ma tion for program improvement. Getting
young daughter with schoolwork; into case details better illuminates what
worked and didn't work along the journey
an 87-year-old African American grand-
to outcomesthe kind of understanding a
mother who, having worked hard
program needs to undertake improvement
throughout her life to make sure that her
initiatives.
children and grandchildren completed
Exhibit 4.2 (p. 155) presents highlights of
school, was now attending to her own
a case study from an employment training
education so that she could read the
program. In addition to illuminating what
Bible herself; and
the outcome of a "job placement" actually
a manager in a local corporation who meant to a particular participant, the case
years earlier had lied on her job applica- documents attainment of hard-to-measure
tion about having a high school diploma outcomes such as "understanding the
and was now studying at night to attain a American workplace culture" and "speak-
general equivalency diploma (GED). ing up for oneself" that can be criticai to
long-term job success for an emigrant like
the woman in the story.
In judging the effectiveness of this pro-
We'11 return to the theme of documenting
gram and making decisions about its future,
outcomes through stories near the end of
it can be as important to understand the sto-
this chapter in the Capturing and Communi-
ries behind the numbers as to have the statis-
cating Stories section. The next section, how-
tics themselves. One can justifiably criticize
ever, looks at the special capacity of qualita-
the past reporting practices of many human
tive inquiry to document individualized
service agencies for having been limited to
outcomes in programs that especially value
successful anecdotes with no accountability
individualization.
reporting on overall patterns of the effective-
ness. However, swinging the pendulum to
the other extreme of only reporting aggre- Evaluating Individualized
gate statistics presents its own problems and Outcomes
limitations. Numbers are subject to selection
and distortion no less than anecdotes. Individualization means matching pro-
Well-crafted case studies can tell the stories gram services and treatments to the needs of
Particularhj Appropriate Qualitative Applications lj. 153

o o r c o t f e MARFINS AND rhi;; o u m j T j

IMPACT 0LUJ5
:
: " :!;. ;; i!! {:!: !.! ! i':i i! i!i !!!:: $ i: ! :i ' i ! :>:! !. =-T , .::'i,-.1 i -r - I 'II ! I 7: U ... |.:: !'! = ! ' T 1 = " ! VI KTTi
;V ! !-: :*i! \! .vikw;o!K;!!:!i !.'::!=::i-:;\MV"" I:,"VI:'ALI! i' 'I I" I 'I J !.VI I

il^lVirrY! !>:-=.: Co:M j. : 'ir* s-11 - -> : .".!' 1;V. !' :-!:!;V'V Vi
i ;; r ,vni- s V;:i Y! WttQ !! ; m i " ! ! I I - - . ' ! ! ! . - I . ' . , 5 ; V : : I V - : ' " ; : ^
! V R I > T I

; I: !". " : ! V| :VL I V ,=" I I.VI . 1 F LV-I; J I'-:I . F , I; V (I U i i vi i" i .'"! 11" i. i!' i! !i i'<" i!! '"'" " ; !':'-:Vm n^ :,:., -
! 1 |- 1

.;i !::;. !: !"! f.\ :=.! ,!i ,tv ! i!- KQ '! i:: !i O .,:." ' : ! ,"" -. ' ' I ! I ' . ' ! J / . ' I . ' i I'O!:."'' N I > . " H ' V
jr;vi:;! n''lirn m u ' I.; k\ n\i h" l5 hW""
! v:;.". :.v!' -:::!/' !' = - >J!^ :ivivj;vinl'1'
' VR !:!::" j,V:i r-: vi!,:. !
- : Vi II ! iti- [il Ti >'V\
! ' T I ' ! ' ! ! ! ' . I ' L " P "
! I I ' I ! '
!!<, I I : !'C:!

!::!.-. vS!; l i':." "" 11! :>!.i' /'I ='!':l li1 i/!'';!}!!' i'i!,.!'!"!i i-1ri i"vi " "ii"i! ]. i".;i:- ':i!!'.'i.;r!."!vi'i i'i'i
1 1 !

=" s.'!:: M N V Y ; : : -= - . : : ::V I; .'VIIY

5
: !:: - " - ,'i i',i ii.',f !".'.j i.'Vi; y\IVI"i.i;ri'!V|'
is::.-.: ;n;:,;v;.::';:r !';:ir\ ,:vT ::v.:ri\rfn'*?-KTG!!.!: |ri."!,I ki: Vilu!r-i iiYli,;Yiji jnri<'rirrijIV:!.
.V ! | |.;i |*.s.*i":i IVi I;- :;;: ir;! vi" 'I ::.;/T!:V! ; ;'r!C:;.!:;!! -'*i:ri/ii hivij * 1 !::.';n::.!;:;
: 5 ! : : !;VV! r:1.:r"
i ii- :ft!j:I !h o v !'I:T;:::V vfn I i >'" ^ \ I" W W- - l.'!!/'M'.'.! V, !ii.!,il'h!i; I'! '!'i1"!.' i|11''11' 1

5 ^a:-a:-a:=='s--:i !"!:::i i.Yi1:!'; ! ri1.11':.i." M. !' 'p >'i''0!i: I.

i!i ! ' !'" ! i ' ' !! ! "''T" i.'ir:r .Vi i i=yli iVi ll i: :;V;:
! 5 : Vi|i" rVr.i !i'!i1 1 .'!i.'. !|,-:>ri1 iV111'1 /1!l'1!1'!'i iMi'''"
1

:; :; I V : " : ! I:S I. 'H.: .:.I:

=
S ! !:I.-:" V;-. I^VI:::; !';:-I:J: >. \ v ' -
: 1
M
lIl IMMIMMMMIMMMMMMMMMMMMMBSSSSSSSSSiSSiSSSii
m

irei!" i'=:=.'"hi-iVii;Vj;^"ri:" |':V !;:< h i i i - I I I I Y S : . ' ; I : I Ii:. :'i1'i.vii.irv, U.1 L| ! !.Vi i'l 'Oii lM>r.i
:
! rlij'
O !TR."P-FI :i : . : V : V I : " * I T V I T I > >i .;::= vi-LH/iVI-:! i'i| !::i'CO =:!.:-> i: :>'i I ' I: 'i n.vi i' s.-i I.V.Vi , !< l l. l!Vj i!'!! ",1 il' 1 < i -
II!!!=!.:; LI" ! T V ; ! ! I N V I " I = U !!:.= S wwqvtt ,::;I::'.1 .V,1!j=- .r:-i il 'liH-i IVili' J-iv!''V'
X.- Liivii,:: I/,":! :>:'.!:!.! S T T O !=:I TI 1 IVTI
=;.i iUm !"i i:y-.'ri A V I ; ; " i; I . ' , I i vi iri:i..! i!i i 1.1:1;!. i !:i;i ] i!
!

i5:: !::-!::=!:!!!=:;! jn VZ wvwr =! == :=: t-!:! ;; . = :i w ;;v -i i rj Ir'. 111,1.VI.V/11 Ij - i i il f L! !! i'! li

; - .nsiilkipy V-; ik =: -ri s1 i; = '. V rp ' i - / 1 l-!V I I ! , ' 11! i! m l li' TI I" iVi l!

!!:!!! !.!.!:!.';,!V!|;!. =!Vj!==D;m:!.!i.i,:ii!='i:vi ! IVII::' !':: iV/i -i. I'I I-VIAIIVI:','! Ailmi!'
!

.VIi ! i:i:i'.:; === :=:Ii,VI:; m ^ ^ | ! i i : : h::,! ir =,=;=,rdSi. l:.:.!,.-! vi ; l':!i'JiVIlJ.|'g

.VI - ! ! . . . ! :'.:I !'I I t f a ';!, -: r i ' R V - ; I : v t : ! : " 1 h ? r i ^ w - I.-;I: , I ; I / I V I ! : : = : > = 13:= N : , R ; : . ! - , I R IF , ! IL ,', ! ;I-

i"i ! I" i !i iii\ ^ -I-s^ l J D U=! =!i ! != =" :; ! :V:.VI ! V ! I,- Tu't. 1=1=: Vi u.rrtiirthrriV -- --1 ! V i^Mh:'
m i.v::ii:vii.;ii:i!Vi:i,:i- :':'.ii.i\.:: Vi'!i:!':! s"ri-ni;:! i.'= 11.1. !=: : !:i ;:-':;. hu i" ivi i n -n .tivv r
' }, i" 11< n r:;
IVI : I ':V.' i-ii I. ! i i i i I : = | : : O! >\\FFTL I^'!: !! !: I!"''!" .W.t .'ii:.:;!":; u : : : :
iVi:0 li.! s/iViV^V:!:,- !! = :! IVVI.: ^ ^S!in'r^ih>.

" !" 1p;.i 1; :"i I.i! I. L. 11 ! ti D A 11M 1'I ^ TI I -! i=

: =a:-=-:I IJ!
.I j=
: .!, !v=:i. .31.-;:! rv fvi =.I.!=: I I m
! \I ! - i -u n :-I I-.I I
i=-i !.T! =!
:
: ! 1" i^i =:=i ^'ilT! ] ! :=. H1 :; ===
! ! ir, r ri/i dp

!!!.: :;:'.;..! I rldi :: ! i' ! 1= Vd "1

ir rvrinsiiTi:i:I :i"II-T:.
iiii m.!:; |yo!:! ;; ::i:!'
!.'! =.:""!.;: rl: -; I! =!
M:-iv. Vi i=' = ! II li :'. I!:r i-;;:-!.
iis: ;=.! wpert*
!!n i h:-:!. E !.;:> vr :!= ;=;:.!

(continued)
154 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

=
: !.::!: i 1 birrl !'!:! IviVj." ::=!: !: I !'!'! ii ;"i
. " I. j :j r ' - : !-:->> :.:
'!="-=5 = = i.j- \> == :! =
' :: !=|
: !: ! !' : i: ! !>1 !' ::.-r i. li i
; ! : : M M N T !.W=! M ;:;:= llllllllllllJIIIIIIIIIJSlIIJIII
i:. I! , =:":":- .Vi:' ii! i = ===
.=.! i.=! = :.: v:ii!::-s \-r I"-.! Ldlsn.
V:'-:,- !' " ij i'-: " Ij iV ."i:-":" ;!' =.! i r n:.'"
.^ac jv!:0= i: i'k=
' !!::! j . ! = 1 i! =ii-
'rttnrl l::, ;,!>. !ih,il !!!:!.-; i" :j
j- i- -j i i. j:-,.-:i. ! ; i :,!;. i !.:!=:;! !ii=y !i=vi
rim ""i: lri li" . ==
! : :;!:=: i!!!'"!!' ::.;!: .=.: -. s
. !v ! il:- >
: S :: :-. !!"!:! v ! :! -V ! I :' : : .-'I

\Y S j":1; I !': ;:! "^y . i ' i ! ! ::.=!' ':*!; " " ! vi.

Aud ! i:! !:'i V-0 ftN" =:!ii =

!:V::..I -.! =J::.=! :!,--::-.-!: = :.: I :I v i ! :I

lj ;.i; A : I ! " !rr j ::i" i" i 'v I" i i-:ri":i' i !i ! h:= |i' 'i I j " 1
'flil-Hul "V:!I !;! =! 11'::!;!'. = =.=" :! :' j.'!:.j.i"- .i..
1S
!! "IE*:*E:"5 ':S:V 'i:: i h - :i <:> . :
M I:-:; I IVJ: I l-:j;i:;i|; ' ij l" ii '
::!':! ::!. !!!:::: ar i-i .:.! U ;! '.i' i,' \
i' i :i'i:'i.':i"! . V .1 i i ' 1 : ! ' il "-I

"o-.! &arfr \>k ':=:=! = !: Ij: l ; I i,'.

.JHJ! ;:Ij.: i.:. !,ii::: 11: :==:-!.=!i Iln i":i ='!. ii '!' '.'li" 1.1

! " r :! : ri" :! .i Y ;i ; i :! -:j :: i"i:'"i:i * IY i ' <" r r r : r ' ! !
"Ov ;i-!!. r. .;!:!. i i.Y jji;! i:|B lirh.y; VHj;; >y-, K i- 'h':- ! :'

I J I T I 'MSV;! :! =: .L""i="I! !v. " , :-. I I '.!= :I V : H JW I " ISI":

individual clients. Successful social and ed- staff are justifiably reluctant to generate
ucational programs adapt their interven- standardized criteria and scales against
tions to the needs and circumstances of spe- which ali clients are compared. They argue
cific individuais and families (Schorr that their evaluation needs are for documen-
1988:257). Flexibility, adaptability, and indi- tation of the unique outcomes of individual
vidualization can be important to the effec- clients rather than for measures of outcomes
tiveness of educational and human service standardized across ali clients.
programs. Highly individualized programs There are numerous examples of indi-
operate under the assumption that out- vidualized programs or treatments. Open
comes will be different for different clients. education, for example, is partly a model
Not only will outcomes vary along specific of educational processes that assumes that
common dimensions, but outcomes will be the outcomes of education for each child
qualitatively different and will involve qual- are unique. Open and experiential ap-
itatively different dimensions for different proaches to education offer diverse activi-
clients. Under such conditions, program ties to achieve diverse and individualized
Particularhj Appropriate Qualitative Applications lj. 155

Behind the Numbers of an Employment Program:

The Story of Li

Outcome Statistics
Completed WORK program, stayed in job placement one year.
Highest wage before WORK program, $8.25 without benefits; wage following WORK
program completion, $11.75 with benefits, more than a 42% increase.
Graduated from technical school with a 3.66 GPA (out of 4.0)
TABE test (math and language skills): reading, 7th grade equivaient; language,
4th grade; spelling, 10.6; math computation, 12.9highest score possible, and applied
math, 9.9. Average increase: 5.4 grade leveis.
Participation data: attended 89 classes at WORK, missed 6 and was late to 1.

Outcome Story
Li entered the employment program called deserted her. She found a cleaning job at a
"WORK" two years after she arrived in the bakery and iater at a hotel. Li had always be-
United States from Vietn m. As a recentimmi- lieved that through education she would get
grant, Li faced language and cultural barriers ahead, so she again took up the study of ac-
at school, at work, and in her day-to-day living. counting, begun in Vietnam, by enrolling in a
She was originaUy from Saigon City, where she local technical coliege four months after arriv-
tookthe nationaltestto study atthe university ing in the United States. After two years in the
but failed both times she took it. She gained United States, she found a data entry job for a
entrance to the Vietnmese Technical Coliege, retail business where she received $8.25 per
where she completed a degree in payroll and hour without benefits. She ost that job when
human resources when she was 23. She soon she had to stay home for a week to care for her
went to work for a Vietnamese company as an very sick daughter. She then went on welfare.
accountant. Li first heard about the employment pro-
Li married a man from North Vietnam de- gram, WORK, from some friends at the techni-
spite her family's opposition. Li and her hus- cal coliege. She entered and concentrated on
band were not accepted in either of their par- improving her English, technical skills, and as-
ents' homes, even though tradition calls for sertiveness with support from program staff.
the married couple to live in the home of the WORK provided tuition assistance, cost-of-
husband's parents. As a result, the couple living support, and bus passes. She also re-
rented a small room where they struggled to ceived tutoring in accounting (1.5 hours per
improve their lives. Shortly after the birth of week) from the program's accountant. The
their daughter, the couple discovered thatthey program secured for her a work experience
might be eligible to go to the United States be- placement ata local bankata rate of $7.25 per
cause the U.S. government was granting visas hour. This work environment proved very
to former officers of the Vietnamese army. Af- stressful because her language skills were in-
ter successfuliy negotiating the difficultappli- adequate and she was teased and harassed by
cation and emigration process, they arrived in other workers.
Minnesota where they knew no one. Accord- The WORK program supported her leaving
ing to Li, her husband began treating her very the bank job so that she could more intensively
badly. He immediately got a job but would not study business English, refine her workplace
give her any financial support Within a year he communications skills, and enhance her ac-

(continued)
156 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

m#4nmm*M contmued

counting software skills. The program case re- found it important to learn interviewing
cords show that at times she expressed feel- skills, since the interview is not common in
ings of hopelessness. Her staff worker Vietnam. She emphasized that WORK played
counseied her to focus on ali the barriers that a criticai role in helping her find a profes-
she had overcome by moving to this country sional job in "American corporate culture,"
and her hopes for a positive future for her explaining, "I compare myself with my friends
daughter. The program supported her return to at the Tech College and even now they don't
the technical college where she graduated have a job. If I didn't have WORK, I probably
with dual diplomas n accounting and data en- wouldn't have a job either." The program staff
try. Shortly thereafter, she began interviewing emphasized the challenges Li faced in gain-
for jobs. She went through more than seven ining confidence and learning how to speak up
terviews with different companies before she for herself, issues that the staff continued to
received a final job offerof an accounting posi- work on with her after her job placement.
tion at a retail firm. She said that she had ex- Li believes that WORK played a vital role in
pected more help with placement from the supporting her to overcome the challenges in
program than she got. According to WORK her life. She said, "I can talk to staff about any
staff, there was some confusion about their problem-about my job, money, and my
role in finding a placement for participants. At daughter. I talk to them to figure things out
any rate, Li persevered and felt good about the and solve problems. Even now that I have a
result: "Even though I failed many interviews job, I call and get help from staff." She re-
and I thought I might never get a job, ali the counted getting help dealing with a fellow
staff encouraged me to keep trying. They worker who was making life difficult for her.
talked to me about my good qualities. They Program staff talked her through the process
were ali positive, not negative. In the end, they of discussing the situation with her supervi-
really helped me get a good job even though I sor and getting help to resolve the situation.
didn't understand the mitsof their role at the Li said, "Even though I wasn't successful at
time." home with my husband, I feel like WORK is
While her language and math tests showed my family now. It makes people feel safe here.
great improvement (from elementary-ievel re- The staff encourages us to go forward. If
sults to high school-level results over the there is a problem, they help us solve it. The
two-year period in WORK), Li says that what important thing is that we treat each other
was most important was what she learned at like a family. I iearned so many things here.
WORK about how to be professional in the Things I can't get n school. I learned inter-
workplace and how "to mesh with American view skills, workplace skills, empowerment
culture." Li specifically mentioned that she skills. I learned English that is more effective

outcomes. Moreover, the outcomes of hav- a field trip, follow ed by dictating stories to
ing engaged in a single activity may be quite the teachers and volunteers about that field
different for different students. For example, trip, and then learning to read their stories.
a group of primary school students may take For some students such a process may in-
Particular!}/ Appropriate Qualitative Applications j. 157

on a professional levei for communication. Li feels that her primary challenge is the
WORK helps us not be afraid. They gave me a fact that she is a "foreigner." She says, "An
computer and some cost-of-living support, and American learns one thing and I have to learn
they paid fora pronunciation class. They heped double. I'm slower than another American
me so much." worker, because it's ali new for me."
A friend of Li's interviewed for this case According to Li, her "life has changed a lot
study reported that WORK was "especiaily because of WORK." She explained that before
helpful to Li in making the adjustmentfrom her she received WORK support, it was hard to
culture to this cuture. It was a touchstone for imagine herseif in a professional position in
her. It helped her in her adapting. It was vital." an office environment in this country. Li
Li now works full time. She wakes at 5:30 a.m., noted, "I was so excited to get an office job. It
catches the bus at 6:30, arrives at work at is the luckiest thing in my life, and the biggest
7:30, and begins work at 8:00. She works until challenge." According to Li, one resut of her
4:30 p.m. and immediately takes the bus,arriv- effort with WORK is that she feels she can talk
ing home at 5:30. Several times a week, Li and interact with anyone at work or in the
swims after work at a local community pool. community.She's moresecurein dealing with
Each evening she prepares dinner for her others and has more confidence in herseif. in
daughterand helps her with her homework. Af- addition, Li is abie to manage her own finan-
ter her daughter goes to bed, Li frequently ciai situation and make confident decisions
works on her computer to improve her skills, or about raising her daughter. Her interactions
does other self-study. with her daughter have improved and she says
Li says that she is still working on communi- thatsheis no longerashamed of herdivorce.
cating better in the workplace. Her supervisor Li's future includes short-term and long-
said in an interview:"Weencouraged hertobe- term goals. In the short term, she would like
come more assertive. We wanted her to know to save money to buy the software for Pro-
that it'5 okay to stand up for herseif. It's okay to fessional Payroli Accounting, complete the
be more assertive. It's the American way. She's training, and move nto payroli accounting.
learning." Her job supervisor continued: "WORK She doesn't see herseif staying at her cur-
has been here a couple of times to sit and talk rent company for more than several years.
with myself and the personnel manager when Eventually, she would like to go back to
we were having difficulty with Li and one school to become a CPA and increase her in-
coworker. We wanted their input on what to do. come to support herseif and her daughter.
So we ali kind of worked together to try and One day, Li hopes to travei to Vietnam to visit
solve the problem. They worked really well with her mother.
us and are continuing to do so."

volve learning about the mechanics of lan- cess may be learning how to spell certain
guage: sentence structure, parts of speech, words. For other students the important out-
and verb conjugation, for example. For other come may be having generated an idea from
students the major outcome of such a pro- a particular experience. For yet other stu-
158 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

dents the important outcome may have been have. However, in programs that empha-
something that was learned in the exercise size individualization of treatment and out-
or experience itself, such as knowledge eomes, program staff may argue, quite justi-
about the firehouse or the farm that was vis- fiably, that independence has a different
ited. Other students may become more artic- meaning for different people under differ-
ulate as a result of the dietation exercise. Still ent life condi tions. Thus, for example, for
other students may have learned to read one person independence may have to do
better as a result of the reading part of the ex- with a changing family dynamic and
ercise. The criticai point is that a common ac- changed relationships with parents. For an-
tivity for ali students can result in drasti- other person independence may have to do
cally different outeomes for different with nonfamily relationshipsthat is, inter-
students depending on how they approach actions with persons of the opposite sex, so-
the experience, what their unique needs cial activities, and friendships. For still other
were, and which part of the activity they clients the dominant motif in independence
found most stimulating. Educators involved may have to do with employment and eco-
in individualized approaches, then, need nomic factors. For still others it has to do
evaluation methods thatpermit documenta- with learning to live alone. While clients in
tion of a variety of outeomes, and they resist each case may experience a similar psycho-
measuring the success of complex, individu- therapeutic intervention process, the mean-
alized learning experiences by any limited ing of the outeomes for their personal lives
set of standardized outcome measures (e.g., will be quite different. What program staff
improved reading scores, better spelling, or wants to document under such conditions is
more knowledge about some specific sub- the unique meaning of the outeomes for
ject). Qualitative case studies offer a method each client. Staff members need descriptive
for capturing and reporting individualized information about what a client's life was
outeomes. like on entering treatment, the client's re-
A similar case can be made with regard to sponse to treatment, and what the clienfs
the individualization of leadership develop- life was like following treatment. They also
ment, criminal justice, community mental want to report documented outeomes
health, job training, welfare, and health pro- within the context of a clienfs life for "suc-
grams. Take, for example, the goal of in- cessful programs see the child in the context ofthe
creased independence among a group of cli- family and the family in the context of its sur-
ents receiving treatment in a community roundings" (Schorr 1988:257). Such descrip-
mental health center. It is possible to con- tive information results in a set of individual
struct a test or checklist that can be adminis- case studies. By combining these case histo-
tered to a large group of people measuring ries, it is possible to construct an overview of
their relative degrees of independence. In- the patterns of outeomes for a particular
deed, such tests exist. They typically involve treatment facility or modality.
checking off what kind of activities a person The more a program moves beyond train-
takes responsibility for, such as personal hy- ing in standard basic competencies to more
giene, transportation, initiatives in social in- individualized development, the more qual-
teraction, food preparation, and so on. In itative case studies willbe needed to capture
many programs, measuring such criteria in the range of outeomes attained. A leader-
a standardized fashion provides the infor- ship program that focuses on basic concepts
mation that program staff would like to of planning, budgeting, and communica-
Particular!}/ Appropriate Qualitative Applications j. 159

tions skills may be able to measure outcomes means that actively involving people in the
with a standardized instrument. But a lead- development process is an end in itself, not
ership program that engages in helping par- just a means to some more concrete end; the
ticipants think in systems terms about how process is the point rather than simply the
to find leverage points and intervention means of arriving at some other point. The
strategies to transform their own organiza- journey, not the destina tion, is what matters.
tions will need case studies of the actual For example, a planning process for a com-
transforma tion efforts undertaken by partic- munity or organization may be carried out
ipants, for their individual endeavors are with a heavy emphasis on participation and
likely to vary significantly. One may be the involvement such that building relation-
director of a small community-based non- ships and mutual understandings along the
profit organization. Another may be a midway is at least as important as the focus of
dle-level government manager. Still another the actual plan produced. The process, in
may be part of a large national organization. such a case, becomes the outcome. That is,
"Transforma tion" will mean very different producing a plan (the apparent intended
things in these different settings. Under such outcome) actually becomes a means to
circumstances, qualitative case study meth- building community (the real desired out-
ods and design strategies canbe particularly come).
useful for evaluation of individualized par- In contrast, other interventions and pro-
ticipant outcomes and organization-level grams play down process. The emphasis is
impacts. on results and outcomes. Even in these
cases, however, some process is undertaken
Process Studies to achieve results and understanding the
A focus on process involves looking at process-outcomes relationship necessitates
how something happens rather than or in ad- documenting and understanding processes.
dition to examining outputs and outcomes.
Evaluations vary in their emphasis on pro- Process evaluations study process. Qualitative
cess in part because programs vary in their inquiry is highly appropriate for studying
attention to process. Some therapy ap- process because (1) depicting process re-
proaches in psychology are highly process quires detailed descriptions of how people
oriented in that they focus on the relation- engage with each other, (2) the experience of
ship between the client and therapist, how process typically varies for different people
the client is approaching issues, how the cli- so their experiences need to be captured in
ent feels about the process, and the nature of their own words, (3) process is fluid and dy-
the interactions that occur during therapy, namic so it can't be fairly summarized on a
rather than focusing only or primarily on be- single rating sale at one point in time, and
havioral outcomes. Groups, programs, even (4) participants' perceptions are a key pro-
entire organizations may be characterized as cess consideration.
highly "process oriented" if how members Process evaluations aim at elucidating
and participants feel about what is happen- and understanding the internai dynamics of
ing is given as much attention as the results how a program, organization, or relation-
achieved. There are styles of community and ship operates. Process studies focus on the
organizational development that operate on following kinds of questions: What are the
the premise "What we do is no more impor- things people experience that make this pro-
tant than how we do it." This statement gram what it is? How are clients brought
160 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

into the program, and how do they move A process study is especially approprate
through the program once they are partici- when the following kinds of statements are
pants? How is what people do related to made about some intervention, relationship,
what they're trying to (or actually do) ac- organization, or program:
complish? What are the strengths and weak-
nesses of the program from the perspective We take people through a developmental pro-
of participants and staff? What is the nature cess made up of a series of steps or phases.
of staff-client interactions?
The nature of our process is what makes us
A process evaluation requires sensitivity
unique.
to both qualitative and quantitative changes
in programs throughout their development, We are a very process-oriented place.
which typically means monitoring and de-
scribing the details of the program's imple- We need to spend more time processing
mentation. Process evaluations not only whafs going on.
look at formal activities and anticipated out- I'm having trouble getting a handle on the pro-
comes, but they also investigate informal cess.
patterns and unanticipated interactions. A
variety of perspectives may be sought from What is the process? Is it the same for every-
people with dissimilar relationships to the one? Is the process working for people?
program, that is, inside and outside sources.
Process data permit judgments about the A good example of what can emerge from
extent to which the program or organization a process study comes from an evaluation of
is operating the way it is supposed to be op- the efforts of outreach workers at a prenatal
erating, revealing areas in which relation- clinic in a low-income neighborhood. The
ships can be improved as well as highlight- outreach workers were going door to door
ing strengths of the program that should be identifying women, especially teenagers, in
preserved. Process descriptions are also use- need of prenatal care in order to get them
ful in permitting people not intimately in- into the community prenatal clinic. Instead
volved in a programfor example, externai of primarily doing recruiting, however, the
funders, public officials, and externai agen- process evaluation found that the outreach
ciesto understand how a program oper- workers were spending a great deal of time
ates. This permits such externai persons to responding to immediate problems they
make more intelhgent decisions about the were encountering, for example, need for rat
program. Formative evaluations aimed at control, need for EngHsh as a second lan-
program improvement often rely heavily on guage classes, and protection from neglect,
process data. Finally, process evaluations are abuse, or violence (Philliber 1989). The ac-
particularly useful for dissemination and tual interactions that resulted from the
replication of model interventions where a door-to-door contacts turned out to be sig-
program has served as a demonstration pro- nificantiy different from the way the
ject or is considered to be a model worthy of door-to-door process was designed and con-
replication at other sites. By describing and ceptualized. These findings, which emerged
understanding the details and dynamics of from interviews and observations, had im-
program processes, it is possible to isolate portant implications for staff recruitment
criticai elements that have contributed to and training, and for how much time needed
program successes and failures. to be allocated to cover a neighborhood.
Particular!}/ Appropriate Qualitative Applications j. 161

Implementation Evaluation evaluating outcomes. Where outcomes are

evaluated without knowledge of implemen-
A prominent theme running through the tation, the results seldom provide a direction
preceding sections is that qualitative meth- for action because the decision maker lacks
ods are particularly useful for capturing information about what produced the ob-
differences among people and programs. served outcomes (or lack of outcomes). Pure
Evaluating mdividualized outcomes, devel- pre-post outcomes evaluation is the "empty
oping unique case studies of people and pro- box" approach to evaluation.
grams, and documenting the local diversity One important way of studying program
within national or statewide programs implementation is to gather detailed, de-
these are evaluation research issues for scriptive information about what the pro-
which qualitative strategies are particularly gram is doing. Implementation evaluations
appropriate. This section looks more closely answer the following kinds of questions:
at the appropriateness of qualitative meth- What do clients in the program experience?
ods for evaluating program implementa- What services are provided to clients? What
tion. does staff do? What is it like to be m the pro-
It is important to know the extent to gram? How is the program organized? As
which a program is effective after it is fully these questions indicate, implementation
implemented, but to answer that question it evaluation includes attention to inputs, ac-
is important to learn the extent to which the tivities, processes, and structures.
program was actually implemented. In his Implementation evaluations tell decision
seminal study Social Program Implementation, makers what is going on in the program,
Walter Williams (1976) concluded, "The lack how the program has developed, and how
of concern for implementation is currently and why programs deviate from initial
the crucial impediment to improving com- plans and expectations. Such deviations are
plex operating programs, policy analysis, quite common and natural, as demon-
and experimenta tion in social policy areas" strated in the findings of RAND's classic
(p. 267). "Change Agent Study" of 293 federal pro-
In Utilization-Focused Evaluation (Patton grams supporting educational change
1997a), I suggested that if one had to choose (McLaughlin 1976). That study found that
between implementation information and national programs are implemented incre-
outcomes information because of limited mentally by adapting to local conditions, or-
evaluation resources, there are many in- ganizational dynamics, and programmatic
stances in which implementation informa- uncertainties.
tion would be of greater value. A decision
maker can use implementation information Where implementation was successful, and
to make sure that a policy is being put into where significant change in participant atti-
opera tion according to designor to test the tudes, skills, and behavior occurred, imple-
very feasibility of the policy. Unless one mentation was characterized by a process of
knows that a program is operating accord- mutual adaptation in which project goals and
ing to design, there may be little reason to ex- methods were modified to suit the needs and
pect it to produce the desired outcomes. Fur- interests of the local staff and in which the staff
thermore, until the program is implemented changed to meet the requirements of the pro-
and a "treatment" is believed to be in opera- ject. This finding was true even for highly
tion, there may be little reason even to bother technological and initially well-specified pro-
162 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

jects; unless adaptations were rnade in the The Follow Through data analysis
original plans or technologies, implementa- showed greater within-group variation than
tion tended to be superficial or symbolic, and between-group variation; that is, the 22
significant change in participants did not oc- models failed to show treatment effects as
cur. (McLaughlin 1976:169) such. Most effects were null, some were neg-
ative, but "of ali our findings, the most per-
If a process of ongoing adaptation to local vasive, consistent, and suggestive is proba-
conditions characterizes program imple- bly this: The effectiveness of each Follow
mentation, then the methods used to study Through model depended more on local circum-
implementation should correspondingly be stances than on the nature ofthe model" (Ander-
open-ended, discovery oriented, and capa- son 1977:13). The evaluators, however,
ble of describing developmental processes failed to study the local circumstances that
and program changes. Qualitative methods affected variations in program implementa-
are ideally suited to the task of describing tion and outcomes. "Little remains in the ex-
such program implementation. isting Follow Through evaluation that spe-
Failure to monitor and describe the cifically addresses the problem of how well,
nature of implementation, case by case, pro- and by what process, program models are
gram by program, can render useless stan- implemented" (Elmore 1976:119).
dardized, quantitative measures of program
The study of these important program
outcomes. The national evaluation of Follow
implementation questions requires case
Through was a prime example of this point.
data rich with the details of program content
Follow Through was a planned variation
and context. Because it is impossible to an-
"experiment" in compensatory education
ticipate in advance how programs will
featuring 22 different models of education to
adapt to local conditions, needs, and inter-
be tested in 158 school districts on 70,00 chil-
ests, it is impossible to anticipate what stan-
dren throughout the nation. The evalua-
dardized quantities could be used to capture
tion alone employed 3,000 people to collect
the essence of each program's implementa-
data on program effectiveness. The
tion. Under these evaluation conditions, a
multimillion-dollar evaluation focused al-
strategy of naturalistic inquiry is particu-
most entirely on standardized outcomes
larly appropriate. For a more extensive dis-
aimed at making possible comparisons of
cussion of evaluating program implementa-
the effectiveness of the 22 models. It was as-
tion, see King, Morris, and Fitz-Gibbon
sumed in the evaluation plan that models
(1987) and Patton (1997a: Chapter 9).
could be and would be implemented in
some systematic, uniform fashion. Eugene
Tucker (1977) of the U.S. Office of Education, Logic Models and
however, has poignantly described the error Theories of Action
of this assumption:

It is safe to say that evaluators did not know A logic model or theory of action depicts,
what was implemented in the various sites. usually in graphic form, the connections be-
Without knowing what was implemented it is tween program inputs, activities and pro-
virtually impossible to select valid effective- cesses (implementation), outputs, immedi-
ness measures. . . . Hindsight is a marvelous ate outcomes, and long-term impacts. For
teacher and in large scale experimentations an example, the classic educational model of
expensive one. (pp. 11-12) the popular DARE (Drug Abuse Resistance
Particular!}/ Appropriate Qualitative Applications j. 163

Education) program in schools followed the

following simple logic model: (1) Recruit rNE \M\C M . ! CHAUNiat
and train select police officers (inputs) to
teach children the dangers of drug use; (2) " M ii!?:-!-.)!!-.'!- W J + L V C T I i'n:i":=::=:i'i-:i)-': ! ME"

have the police, in uniform, teach children in . ^i^ !;!!;'!^ !:*!'! ntf'rf"tCC!
special classes in school (implementation); LHI=.II'!;!:!!-!.!! s"ii i i I . i ! .Vi i V i : : ! " i ! A ? . : : $::-:R? T

and, as a result, the children will (3) find the ; liVri'!! !.^! ^i-i...: ' \ !i::: i !=
j IV! !! rc - i fr~
teaching credible (process evaluation) and i !1 !: ! !l=!i'!:i>.!.!i C L T I !
-ctMij,: K V < : v : s * c

(4) learn facts about drugs (cognitive out- = HtHn! jiH;!"' li 1i n t !:-i strt. d: =.<. ! '=.
come), which will (5) convince them not to I s*i-5|f=ia "tf SI'-!! a mn
j mmh i'i.!.' i! tiiv' rs ! -Vr -
use drugs (attitude change outcome), which iLVi-if rn ii;'iVi'M!riLvE i l ! M k ftV!."'i :ij-i!!;:v

will (6) result in students not using drugs iM M ii r !:i.'i! i' , r ' i ; V ! i ' ,'i Vift.":.! k . l . ' ! 1 ; IJ! : 1.-S-." i

(behavior change outcome), which will ulti- : tWWiMU" !"Su^iiU- 1!:''!:Y.

mately show up in community indicators

showing less drug use (impact). At least that
was the model, or theory. In practice, evalua-
tions of DARE consistently showed that the gram staff, administrators, and evaluation
theory didn't work in practice. researchers spent a day together working on
Attention to program theory has become a logic model for a program, Road to Recov-
a major focus of evaluation research (see ery, that transported cncer patients to treat-
Rogers et al. 2000), and with that attention ment. As a result of thorough contextual
has come some confusion about terminol- analysis and interviews with key infor-
ogy. I distinguish a logic model from a the- mants, including patients and volunteer
ory of change. The only criterion for a logic drivers, the program was reconceptualized
model is that it be, well, logical, that is, that it as a treatment compliance strategy aimed at
portrays a reasonable, defensible, and se- cncer control rather than just a transpor ta-
quential order from inputs through activi- tion program. The reason for transporting
ties to outputs, outeomes, and impacts. A patients was to ensure complete and consis-
theory of change or theory of action, in tent treatment. This reconceptualization had
contrast, bears the burden of specifying and significant implications for how the pro-
explaining assumed, hypothesized, or gram was implemented, the outeomes that
tested causai linkages. Logic models are de- were measured, and the importance of the
scriptive. Theory of change and theory of ac- program in relation to American Cncer So-
tion models are explanaiory and predictive. ciety priorities.
The connotative difference between a "the- Organizational theorist Chris Argyris
ory of change" and "theory of action," (1982) introduced what has become a elassie
though the phrases are often used inter- distinetion between "espoused theories"
changeably, is that theory of change is more and "theories-in-use." The espoused theory
research based and scholarly in orientation, is what people say they do; it's the official
whereas a theory of action is practitioner de- version of how the program or organization
rived and practice based. operates. The theory-in-use is what really
The results of a logic modeling or the- happens. Interviewing supervisory or man-
ory-of-action exercise can make a dramatic agerial staff and administrators, and analyz-
difference in how people understand a pro- ing official documents, reveals the espoused
gram. A group of national and local pro- theory. Interviewing participants and front-
164 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

line staff, and directly observing the pro- mon need for a period of time for evaluators
gram, reveals the theory-in-use. The result- to work with program staff, administrators,
ing analysis can include comparing the funders, and participants on clarifying goals
stated ideais (espoused theory) with real pri- and strategiesmaking them realistic,
orities (theory-in-use) to help ali concerned meaningful, agreed on, and evaluable.
understand the reasons for and implications Evaluability assessments often include in-
of discrepancies. terviews and focus groups with diverse pro-
This ideal-actual comparison can support gram constituencies to determine how much
organizational development to improve ef- consensus there is among various stake-
fectiveness. It also helps move toward a rea- holders about a program's goals and inter-
sonably realistic depiction of lhe program vention strategies and to identify where dif-
that can be put to a summative te st, that is, ferences lie. Based on this kind of contextual
one can study the extent to which the model analysis, an evaluator can work with pri-
or treatment actually accomplishes the hy- mary intended users of the evaluation to
pothesized and desired outcomes and im- plan a strategy for goals clarification and
pacts. But such a study can take place only logic model development.
when the model has been described in realis- Studies of evaluability assessments (Rog
tic terms. Qualitative inquiry is especially 1985; Smith 1989) have found that the pro-
appropriate for achieving that description. cess of qualitative inquiry often becomes a
forma tive evaluation as program staff mem-
Evaluability Assessments bers learn about the strengths and weak-
nesses of their program conceptualizations.
Evaluability assessments (Wholey 1979, The evaluability assessments become im-
1994; Smith 1989) are conducted through in- provement-oriented experiences that lead to
terviews, document analysis, and observa- significant program changes rather than just
tions to determine whether a program is a planning exercise preparing for summa-
sufficiently well conceptualized and consis- tive evaluation.
tently implemented to undertake a formal
and rigorous evaluation, especially a
summative evaluation aimed at determm- Comparing Programs:
ing overall effectiveness. Clarifying the pro- Focus on Diversity
gram logic model or theory of action (previ-
ous section) is also an important purpose of
many evaluability assessments. In essence, We have frequently encountered the idea that
an evaluability assessment involves making a program is a fixed, unchanging object, ob-
sure that the program treatment or model is servable at various times and places. A com-
clearly identifiable and logical; that out- mon administra tive fiction, especially in
comes are clear, specific, and evaluable; and Washington, is that because some money asso-
that implementation strategies are reason- ciated with an administrative label (e.g., Head
ably and logically related to expected out- Star t) has been spent at several places and o ver
comes. a period of time, entities spending the money
One reason evaluability assessment has are comparable from time to time and from
become an important preevaluation tool is place to place. Such assumptions can easily
that by helping programs get ready for eval- lead to evaluation-research disasters. Pro-
uation, the process acknowledges the com- grams differ from place to place because
Particular!}/ Appropriate Qualitative Applications j. 165

places differ. (Edwards, Guttentag, and Snap- What these data did not reveal was that
per 1975:142) some of the programs with large student-
staff ratios made extensive use of volun-
Individualizing services to clients has teers. These regularly participating addi-
been one major theme of social action and tional volunteer staff made the real stu-
educational programs in recent years. An- dent-adult ratios much smaller. The global
other closely related theme has been the im- and uniform reporting of the data, how-
portance of adapting programs to local ever, did not allow for that nuance to be
community needs and circumstances as dis- recorded.
cussed earlier in the Implementation Evalu- A good example of the diversity that can
ation section of this chapter. While some ba- emerge from attention to the qualitative dif-
sic framework of how programs should ferences among programs is Sharon Fei-
function may originate in Washington, D.C., man's (1977) classic study of national teacher
Ottawa, Brussels, or Canberra or some state center programs. Although funded as a sin-
or provincial capital, it is clear that program gle national program with common core
implementation at the local levei seldom foi- goals and the shared label "teacher centers,"
lows exactly the proposed design. When an Feiman found that three quite different types
evaluation requires gathering data from sev- of center programs had emerged, what she
eral local sites, quantitative measures may called "behavioral" centers, "humanistic"
be appropriate for comparing local pro- centers, and "developmental" centers. Dif-
grams along standardized dimensions, but ferences among the three types are summa-
qualitative descriptions are necessary to rized in Exhibit 4.3.
capture the unique diversities and contrasts Feiman^ analysis highlights the ways in
that inevitably emerge as local programs which different teacher centers were trying
adapt to local needs and circumstances. Lo- to accomplish different outcomes through
cal sites that are part of national or even in- distinct approaches to teacher center pro-
ternational programs show corisiderable gramming. Uniform, quantitative measures
variation in implementation and outcomes. applied across ali programs might capture
These variations are not such that they can some of these criticai differences, and such
be fully captured and measured along stan- measures have the advantage of facilitating
dardized scales; they are differences in direct comparisons. However, qualitative
kind differences in content, process, goals, descriptions permit documentation of
implementation, politics, context, outcomes, deeper and unanticipated program differ-
and program quality. To understand these ences, idiosyncrasies, and uniquenesses. If
differences, a holistic picture of each unique decision makers want to understand varia-
site is needed. tions in program implementation and out-
Using only standardized measures to comes, qualitative case studies of local pro-
compare programs can seriously distort grams can provide such detailed informa-
what is actually occurring in diverse sites. tion, as with the Camegie Foundation's
Consider data from a national educational excellent Portraits of High Schools (Perrone
program that measured staff-student ratios 1985). Thus, qualitative data are necessary to
across the country. A few programs had stu- give a complete evaluation picture of local
dent-staff ratios as high as 75:1 according to variations within national programs, a pic-
the uniform measures used; other programs ture that is necessarily incomplete so long as
had student-staff ratios as small as 15:1. the only data available are aggregated and
166 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

I4HIIMMI Types of Teacher Centers

Prmary Process far

Types of Centers Working With Teachers Primary Outcomes afthe Process

Behavioral centers Curriculum specialists directly Teachers adopt comprehensive curriculum

and formally nstruct teachers. systems and methods.
Humanistic centers Informal, undirected teacher Teachers feel supported and important: pick
exploration; "teachers select up concrete and practical ideas and materi-
their own treatments." ais for immediate use in classes.
Developmental centers Advisers establish warm, Teachers think in new ways about what
nterpersonal, and trusting they do and why they do it, developing new
relationships with teachers insights and fundamental skilis and capabil-
over time. ities.

SOURCE: Feiman (1977).

standardized statistics about these diverse moved toward healthier lifestyle practices
programs. during the 1980s, essentially wiping out the
control aspect of the intended control group.
Prevention Evaluation Time-series designs examine drops in indi-
cators of interest, for example, teenage sui-
There may be nothing more difficult to cide or alcoholism, but, again, there are seri-
evaluate than preventionthe nonoccur- ous problems determining the causes of
rence of some problem. Yet, there is no more changes in indicators.
important direction for the long-terin solu- Qualitative data can add an important di-
tion of health and social problems than pre- mension to prevention evaluations by find-
vention efforts (Sociometrics 1989). The ing out the extent to which desired attitude
usual designs for evaluating prevention pro- and behavior changes linked to prevention
grams use experimental and control groups actually occur. Interviews with teenagers
or time-series designs. In an experimental about their decisions regarding sexual activ-
design, a sample targeted for prevention is ity will reveal if teenagers are using preven-
compared with one that is not. There are of- tion practices and ideas. This helps get at
ten ethical problems with such designs (e.g., more than whether the desired outcome was
withholding needed services from the con- achieved. It helps illuminate how those in
trol group), and there can be problems in the targeted group think about and under-
control, that is, the control group may be stand what is being attempted.
subject to some other mtervention. A An excellent example of prevention-ori-
10-year study of heart disease prevention ented qualitative inquiry is Agar's (1999)
found no differences between treatment and study of heroin use among suburban youth
control groups because the whole society in Baltimore County, Maryland. His field-
Particular!}/ Appropriate Qualitative Applications j. 167

work looked at how young people begin to tinue living m their homes, sometimes years
experiment with heroin and the stories that beyond what would otherwise have been
are generated and disseminated based on possible. While statistical data could docu-
initial experimentation. Those early stories ment comparative death rates, hospitaliza-
tend to be positive: "Heroin puts one into a tion rates, and costs, only direct interviews
blissful, relaxe d, dreamy state; the stresses, and observations could reveal quality-of-life
strains, and worries of everyday life fali differenceswhat it meant to these elderly
away." Over time, positive stories are fol- people to stay in their own homes. I remem-
lowed by negative stories as some youth be- ber in particular the story of one very frail
come addicted and a few overdose. "The im- woman in her 80s who took the interviewer
age of the addict, as opposed to the to into her bathroom, opened her medicine
experimenter, was uniformly and strongly cabinet, pointed to a bottle of capsules and
negative among the youth we mterviewed, said, "lf they come to take me away, Fll ex-
mostly based on their own observations or cuse myself for a moment, come in here, and
stories they had heard." As a result of his take these pills. I'm never leaving here.
in-depth work on how heroin addiction Never." This scenario was ali clearly worked
spreads through the youth community, out in her mind and routinely practiced
Agar (1999) learned how to be more effective mentally. Understanding prevention in-
in prevention education: cludes understanding what people think
and do as a result of prevention efforts.
I learned in the heroin lecture to emphasize the
dangers of sliding into physical dependency
Documenting Development
true addictionand what post-addicted
life was like. The theme contrasts with the nor-
Over Time and Investigating
mal approach, which conveys to youth that System Changes
addiction and death may occur even with one
experiment. Since youth rely on stories to The final practical application in this sec-
evaluate drug effects and since so many sto- tion focuses on the utility and appropriate-
ries contradict the normal premise, drug edu- ness of using qualitative methods to follow
cation loses its credibility. (pp. 115-16) and document development changes. This
returns us to the value of qualitative inquiry
His fieldwork also included looking at sup- for process studies, discussed earlier in this
ply and demand of heroin, and the larger chapter, for development is best understood
system of which drug use was apart, ali us- as a process. Organizational development,
ing qualitative methods to enhance efforts to community development, human develop-
prevent heroin use. ment, leadership development, professional
We evaluated a prevention program developmentthese are process-oriented
aimed at helping elderly people remain liv- approaches to facilitating change. Pre- and
ing in their homes, that is, preventing posttests do not do justice to dynamic devel-
institutionalization. In some cases, the delay opment processes. Pre- and postmeasures
in institutionalization was only six months imply a kind of linear, ever upward,
to a year. Through volunteer visits, home less-to-more image of growth and develop-
nursing care, and meals on wheels pro- ment. In reality, development usually occurs
grams, however, elderly people could con- in fits and starts, some upward or forward
168 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

progress, then backsliding or consolidation. nace. Just monitoring the temperature

Pre- and posttesting tell you where you gauge won't solve the problem. So it is with
started and where you ended up, but not management information systems. Blips or
what happened along the way. Quantitative changes in indicators are a signal that field-
measures can parsimoniously capture snap- work is needed to find out what's really go-
shots of pre- and poststates, even some n- ing on where the action is.
terim steps, but qualitative methods are
more appropriate for capturing evolution-
From Evaluation Issues
ary and transforrnational developmental
dynamics. For example, I worked with a to Evaluation Models
new and innovative employment training
program that was constantly reorganizing Thus far in this chapter, we have re-
staff and participant teams, realigning viewed the appropriateness of qualitative
courses, trying new mixes of activities, and inquiry for dealing with fairly common
restructuring as growth occurred. We found evaluation challenges: quality assurance
that even quarterly reflective practice ses- and enhancement, documenting outcomes
sions were insufficient to capture ali the through case studies, capturing individual-
changes occurring. To study this highly dy- ized outcomes, process studies and imple-
namic program required ongoing fieldwork menta tion evaluation, clarifying a pro-
framed against a chos theory metaphor (see gram^ theory or logic model by linking
Chapter 3) because the program develop- processes and implementation to outcomes,
ment really was like walking through a evaluability assessments, comparing di-
maze whose walls rearrange themselves verse programs, evaluating prevention pro-
with every step you takeand it was a ma- grams, documenting development over
jor challenge to develop a data collection time, and investigating sudden changes in
process that could capture those rearrange- management information system indica-
ments. tors. This is by no means an exhaustive list of
Even routine monitoring using a stan- evaluation applications, but each in some
dardized management informa tion system way illustrates a practice-oriented application
can provide only an overview of up-and- of qualitative methods, the focus of this
down patterns. A break in a trend line or chapter, in the sense that qualitative findings
sudden blips in a time series can indicate are used to improve programs, deal with
that some qualitative difference may have real problems, or support concrete decision
occurred. Statistical data from monitoring making rather than contribute to social sci-
and information systems canbe used to trig- ence theory or generate knowledge as an
ger more in-depth qualitative studies that end in itself. The next section reviews some
focus on finding out what the changes in the of the major models of evaluation research
statistical indicators mean. Consider this that are most closely associated with qualita-
analogy from the northern climes: The ther- tive methods: goal-free evaluation, respon-
mometer on a fumace thermostat shows the sive evaluation, illuminative evaluation,
temperature throughout winter; if the tem- the transaction model, the connoisseurship
perature starts falling quickly in a house, or art criticisrn approach, and utilization-
someone had better have a look at the fur- focused evaluation.
Particular!}/ Appropriate Qualitative Applications j. 169

!=J. Evaluation Models

H olopis kuntal baris: Indonesian phrase uttered to gain extra strength

when carrying heavy objects.

Howard Rheingold (1988:49)

Conducting an evaluation can be a heavy opher-evaluator Michael Scriven (1972b)

load. Evaluation models help with the first proposed the idea of goal-free evalua-
heavy lifting. Models provide frameworks tion. Essentially, goal-free evaluation means
17 like the metal frame on a backpack that gives doing fieldwork and gathering data on a
support and shape to the load on a hiker's broad array of actual effects or outcomes,
back. Models offer evaluators structure and then comparing the observed outcomes
support. They structure certain method- with the actual needs of program partici-
ological decisions, offer guidance about the pants. The evaluator makes a delibera te at-
appropriate steps to follow in design, pro- tempt to avoid ali rhetoric rela ted to pro-
vide direction in ways of dealing with stake- gram goals; no discussion about goals is
holders (Alkin 1997), and identify the im- held with staff; no program brochures or
portant issues to consider in undertaking a proposals are read; only the program's ob-
study. Models provide frameworks rather servable outcomes and documentable ef-
than recipes, helping evaluators and evalua- fects are studied in relation to participant
tion users identify and distinguish among needs. There are four primary reasons for
alternative approaches. For example, the doing goal-free evaluation:
classic model of evaluation is goals-based
evaluation, that is, measuring the extent to 1. to avoid the risk of narrowly studying
which a program or intervention has at- stated program objectives and thereby
tained clear and specific objectives. One al- missing important unanticipated out-
ternative to goals-based evaluation, consid- comes;
ered below, is "goal-free" evaluation. The
2. to remove the negative connotations at-
models briefly presented here have in com-
tached to the discovery of unanticipated
mon that they have been closely associated
effects: "The whole language of 'side-
with or heavily rely on qualitative methods.
effect' or 'secondary effect' or even 'un-
anticipated effect' tended to be a put-
Goal-Free Evaluation down of what might well be the crucial
achievement, especially in terms of new
In the classic model of evaluation, the fo- priorities" (Scriven 1972b:l-2);
cus is on the intended services and outcomes
3. to eliminate the perceptual biases intro-
of a programits goals. However, an evalu-
duced into an evaluation by knowledge
ator can turn up some very interesting re-
of goals; and
sults by undertaking fieldwork in a program
without knowing the goals of the program, or 4. to maintain evaluator independence by
at least without designing the study with avoiding dependence on goalsa staff
goal attainment as the primary f ocus. Philos- or administrative creation and, often,
170 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

fictionthat can limit the evaluator's quiry is especially compatible with goal-
range and freedom of inquiry. free evaluation because it requires capturing
directly the actual experiences of program
In Scriven's (1972b) own words: participants in their own terms. Moreover,
and in particular, goal-free evaluation re-
It seemed to me, in short, that consideration quires the evaluator to suspend judgment
and evaluation of goals was an unnecessary about what the program is trying to do and
but also a possibly contaminating s t e p The to focus instead on finding out what is actu-
less the externai evaluator hears about the ally occurring in and as a result of the pro-
goals of the project, the less tunnel-vision will gram. The evaluator can thus be open to
develop, the more attention will be paid to whatever data emerge from the phenomena
looking for actual effects (rather than checking of the program itself and participants' expe-f
on alleged effects). (p. 2) riences of the program. But, as Ernie House
(1991) has reported, somewhat tongue-
Goal-free evaluation, in its search for "ac- in-cheek, actually conducting a goal-free
tual effects," employs an inductive and ho- evaluation can be challenging for ali con-
listic strategy aimed at countermg the logi- cerned because during on-site fieldwork
cal-deductive limitations inherent in the staff keep dropping hints about program
usual goals-based approach to evaluation, goals:
Goal-free evaluation was a radical departure
from virtually ali traditional evaluation Many are indignant that you do not want to
thinking and practice. For example, promi- know their objectives and incensed at the idea
nent evaluation researcher Peter Rossi of your looking at what they are domg rather
(Rossi and Williams 1972) asserted that "a than what they are professmg. As they chauf-
social welfare program (or for that matter feur you around m their car, some will blurt
any program) which does not have clearly out the goals as if accidentally, with sly apolo-
specified goals cannot be evaluated without gies for their indiscretion. Others will write
specifying some measurable goals. This them on restroom wallsanonymously.
statement is obvious enough to be a truism" (p-112)
(p. 18). Carol Weiss (1972) emphasized the
centrality of goals in evaluation when she It is important to note that goal-free eval-
stated, "The goals must be clear so that the uations can employ both quantitative and
evaluator knows what to look for. . . . Thus qualitative methods. Moreover, Scriven
begins the long, often painful process of get- (1972b) has proposed that goal-free evalua-
ting people to state goals in terms that are tions might be conducted in parallel with
clear, specific, and measurable" (pp. 24-26). goals-based evaluations, but with separate
In contrast to the predominant goals- evaluators using each approach to maxi-
based approach to evaluation, goal-free mize the strengths and minimize the weak-
evaluation opens up the option of gathering nesses of each approach. (For a more de-
data directly on program effects and effec- tailed discussion of goal-free evaluation,
tiveness without being constrained by a nar- and critiques of this idea, see Alkin 1972;
row focus on stated goals. Qualitative in- Patton 1997a.)
Particular!}/ Appropriate Qualitative Applications j. 171

Transaction Models: Responsive and Illuminative Evaluation

R esponsive evaluation enables an alert reprter to capture the deep-felt

opinions of those most affected by the program, a product of grass-
roots populism.
Ernest House (1991:113)

Robert Stake's (1975) "responsive ap- This sensitivity allows the evaluator to
proach to evaluation" places particular em- collect data and report findings with those
phasis on the importance of personalizing differing perspectives clearly in mind, but
and humanizing the evaluation process. Be- with special attention to those whose per-
ing responsive requires having face-to-face spectives are less often heard. Subsequently,
contact with people in the program and Guba and Lincoln (1989) added an explic-
learning Krsthand about diverse stake- itly constructivist perspective to responsive
holders' perspectives, experiences, and con- evaluation in proposing "responsive con-
cerns. structivist evaluation" as the "fourth gener-
ation" of evaluation. The first generation
focused on measurement; the second on de-
Responsive evaluation is an alternative, an old
scription; the third on judgment; and the
alternative, based on what people do natu-
fourth generation of evaluation focuses on
rally to evaluate things, they observe and re-
issues-derived, values-based perspectives.
act. . . . To do a responsive evaluation, the
(See Chapter 3 for a discussion of construc-
evaluator conceives of a plan of observations
tivism.)
and negotiations. He arranges for various per-
Responsive evaluation includes the fol-
sons to observe the program, and with their
lowing primary emphases:
help prepares brief narrative portrayals, prod-
uct displays, graphs, etc. He finds out what is
1. identification of issues and concerns
of value to his audiences, and gathers expres-
based on direct, face-to-face contact with
sions of worth from various individuais
people in and around the program;
whose points of view differ. Of course, he
checks the quality of his records: he gets pro- 2. use of program documents to further
gram personnel to react to the accuracy of his identify important issues;
portrayals; and audience members to react to
the relevance of his findings. He does most of 3. direct, personal observations of pro-
this informallyiterating and keeping a re- gram activities before formally design-
cord of action and reaction. (Stake 1975:14) ing the evaluation to increase the evalu-
ator^ understanding of what is im-
portant in the program, and what can/
Guba and Lincoln (1981) have integrated should be evaluated;
naturalistic inquiry and responsive evalua-
tion into an overall framework for improv- 4. designing the evaluation based on is-
ing the usefulness of evaluation results. The sues that emerged in the preceding three
openness of naturalistic inquiry permits the steps, with the design to include contin-
evaluator to be especially sensitive to the dif- uing direct qualitative observations in
fering perspectives of various stakeholders. the naturalistic program setting;
172 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

5. reporting information in direct, per- tasks and academic experiences are most af-
sonal contact through themes and por- fected. It aims to discover and document what
trayals that are easily understandable it is like to be participating in the scheme,
and rich with description; and whether as teacher or pupil, and, m addition,
to discern and discuss the innovation's most
6. matching information reports and re-
significant features, recurrmg, concomitant,
porting formats to specific audiences
and criticai processes. In short, it seeks to ad-
with different reports and different for-
dress and to illuminate a complex array of
mats for different audiences.
questions. (Parlett and Hamilton 1976:144)
Responsive evaluation is a form of what
evaluation theorist Ernie House (1978) has Transaction evaluation approaches are
called the "transaction model" of evaluation based on the same assumptions that under-
in that it "concentrates on the educa tional gird qualitative research: the importance of
(or program) processes themselves. . . . It understanding people and programs in con-
uses various informal methods of investiga- text; a commitment to study naturally occur-
tion and has been drawn increasingly to the ring phenomena without introducing exter-
case study as the major methodology . . . , nai controls or manipulation; and the
[derived from] a subjectivist epistemology assumption that understanding emerges
[that] tends to be naturalistic" (p. 5). It treats most meaningfully from an inductive analy-
each case as unique and places prime em- sis of open-ended, detailed, descriptive data
phasis on perception and knowing as a gathered through direct interactions and
transactional process between researcher and transactions with the program and its partic-
research participant, thus the label for this ipants.
model.
Connoisseurship Studies
One can study perceptions only by studymg
particular transactions in which the percep-
While responsive evaluation places the
tions can be observed. Ali parties of the situa-
program's stakeholders at the center of the
tion enter into the transaction as "active
evaluation process, connoisseurship evalua-
participants," and do not appear as separa te
tion places the evaluator's perceptions and
already-existing entities. . . . [The evaluator]
expertise at the center of the evaluation pro-
affects and is affected by the situation, thus he
cess. The researcher as connoisseur or expert
is part of the transaction. (House 1978:9)
uses qualitative methods to study a program
Another variation on the transaction or organization, but does so from a particu-
model is "illuminative evaluation," origi- lar perspective drawing heavily on his or her
nally developed as an educa tional evalua- own judgments about what constitutes ex-
tion approach that emphasized context and cellence, thus the term connoisseur. This is a
interpretation. practical and personalistic version of what I
called "orientational qualitative inquiry" in
The aims of illummative evaluation are to the last chapter and what Eisner (1985:184)
study the innovative program: how it oper- has called "prefigured," that is, the terms
ates; how it is influenced by the various school and focus of the "educational criticism" are
si tuations in which it is applied; what those di- determined in advance by the evaluator and
rectly concerned regard as its advantages and those seeking the evaluation. Prefiguring,
disadvantages; and how students' intellectual however, does not mean that the observer
Particular!}/ Appropriate Qualitative Applications j. 173

cannot be open to a new, emergent focus literary and the factual complement each
during fieldwork. other" (Eisner 1985:182).
This means that evaluative criticism or
A critic might be invited to a school or class-
connoisseurship is also highly interpretive
room without a prefgured focus and after sev-
and makes value judgments about the mer-
eral days or weeks perceive an aspect of the
its of what has been described and inter-
school or classroom that is of considerable sig-
preted, using criteria that are appropriate to
nificance but which could not have been antic-
the situation based on the expertise of the
ipated. For example, one of my students
evaluator (the connoisseur) and the agree-
received permission from a secondary school
nents struck with those who are a party to
English teacher to observe and to write an ed-
the study. In that regard, design and data
ucational criticism about her class. What
collection decisions are explicitly political
emerged during my student's observation
and subjective. While there is a qualitative
was the extraordinary way in which the
research foundation to this kind of evalu-
teacher used satire in her teaching. It was not
ative connoisseurship, the method also "re-
the case that the teacher was herself teaching
quires no small degree of artistry" (Eisner
satire; it was that she was satirical in her teach-
1985:187). The factual aspects of this ap-
ing. Such a process or an approach could
proach to evaluation communicate knowl-
hardly have been prefgured. The point here is
edge. The artistic aspects convey not only
that the focus of criticism can be either prefg-
knowledge of facts "but also knowledge of
ured as a part of the research bargain between
feeling. Art can be said to be that activity
the critic and the teacher, or it can be emergent
concerned with the creation of images of
or it can be both. (Eisner 1985:184-85)
feeling. The situations, people, and objects
The art criticism model as a "con- we encounter are never without affect"
noisseurship" approach has been most fully (Eisner 1988:17). The connoisseur, then, is
articulated by Elliot W. Eisner (1985). The explicitly and purposefully a qualitative re-
imagery of evaluators as "connoisseurs" searcher and an artistic critic of the phenom-
making criticai appraisals of programs is enon being studied.
analogous to the traditional way in which
literary and artistic connoisseurs and critics Utilization-Focused Evaluation
work, Eisner having had considerable expe-
rience as an art educator. Utilization-focused evaluation (Patton
The connoisseur approach has many ele- 1997a) offers an evaluative process, strategy,
ments that relate to naturalistic inquiry. Cen- and framework for making decisions about
tral among these are direct observation and the content, focus, and methods of an evalu-
immersion in the setting under study. Edu- ation. Utilization-focused evaluation begins
cational criticism or connoisseurship is, ac- with identification and organization of spe-
cording to Eisner, first and foremost descrip- cific, relevant decision makers and informa-
tive, both factually and artistically. The tion users (not vague, passive audiences)
factual component reports direct observa- who will use the information that the evalu-
tions and interviews. "The artistic aspect of ation produces. A focus on intended use by in-
description is literary and metaphorical; intended users undergirds and informs every
deed, it can evenbe poetic in places In or- design decision in the evaluation. The evalu-
der to optimize communication, the poten- ator works with these intended users (often
tial of language is exploited so that the an evaluation task force representing several
174 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

constituencies, e.g., program staff, clients, best techniques. Researchers and decision
funders, administrators, board members, makers operate within quite narrow meth-
and community representatives) to focus odological paradigms about what consti-
relevant evaluation questions. From these tutes valid and reliable data, rigorous and
questions flow the appropriate research scientific design, and personal or imper-
methods and data analysis techniques. sonal research methods. As I noted in dis-
Utilization-focused evaluation plans for cussing methodological paradigms (Chap-
use before data are ever collected. The ques- ter 2), most social scientists routinely apply
tion that underlies the ongoing interactions those methods in which they have been
between evaluators and intended users is, trained with little sensitivity to the biases in-
"What difference will this study make?" The troduced by a particular data collection
evaluator asks, "What would you do if you scheme. Social and behavioral scientists
had answers to the questions you're ask- experts in the ultimate subjectivity and arbi-
ing?" In answering the evaluation questions trariness of ali human perceptionare often
of specific intended evaluation users, utili- least aware of their own sociomethod-
zation-focused evaluation does not pre- ological biases and how these biases affect
clude the use of any of the full variety of their view of the social program world. Yet,
methodological options available. Qualita- to be sure, social scientists are not the only
tive inquiry strategies may emerge as appro- participants in the evaluation process oper-
priate in a particular utilization-focused ating on the basis of selective perception; de-
evaluation as a result of defining the infor- cision makers and program staff also hold
mation needs of the specific intended evalu- conditioned views about research and meth-
ation needs. ods. One of the tasks to be accoinplished
Creative, practical evaluators need a full during the interactions between evaluators
repertoire of methods to use in studying a and intended evaluation users is to mutu-
variety of issues. This repertoire should in- ally explore design and data biases so that
clude but not be limited to qualitative meth- the evaluation generates information that is
ods. By offering intended users method- useful and believable to ali concerned.
ological options, utilization-focused evalu- Utilization-focused evaluation was de-
ators collaborate in making criticai design veloped from a study of the factors that
and data collection decisions so as to in- seemed to explain variations in the actual
crease the intended users' understanding use of evaluations. That study used qualita-
and buy-in thereby facilitating increased tive methods to study the uses of 20 federal
commitment to use fmdings. The evalua- health evaluations. Our study team inter-
tor^ responsibility is to interact with deci- viewed evaluators, funders, and program
sion makers about the strengths, weak- managers to find out how evaluation find-
nesses, and relative merits of various meth- ings were used (Patton 1997a). Many others
ods so that mutually agreed-on, informed have since confirmed and elaborated the
methods decisions can be made. The evalua- major elements of utilization-focused evalu-
tor may well challenge entrenched method- ation, again using qualitative methods
ological biases while remaining ultimately (Alkin, Daillak, and White 1979; King and
respectful of the importance of users getting Pechman 1982; Campbell 1983; Holley and
something they will believe m and use. Arboleda-Florez 1988; Ferguson 1989). The
Measurement and methods decisions are utilization-focused approach was further re-
not simply a matter of expertly selecting the fined through experience and practice and
Particular!}/ Appropriate Qualitative Applications j. 175

closely associated with and rely heavily on

qualitative methods. The next sections in
: IIH-S MI I^I OF this chapter review interactive applications
\ !=Alt I"1 -Xii i:: Ir .RH D G 1'i.iVM ! . . .
of qualitative methodspractical and prag-
; ; >..." ;
matic forms of inquiry in which the re-
j 'V'Ti .!!=:!'!:; ttaP! !:f!! fpr8 W&W ^ I-
searcher is especially sensitive to the per-
spectives of others and interacts closely
'Si&siiS ,: iWi i! i-:"; with them in designing and/or implement-
ing the study. Such interactive approaches
: -:r^raJamiyfflW * ^ [ ~ that are especially congruent with qualita-
tive methods include efforts to personalize
and humanize research and evaluation,
working with stakeholders to harmonize
program and evaluation values, action
I. S!:i T I : I== ; i l h i l j i i i : ; ! ; : ! ! : ! ) ; ' . :.!:'!.
! learning and reflective practice, apprecia-
= : :=T! \
: I I I I ^ ^ ^ ^ J - J Y ^ ^ H V ' ! I : : : L : ! L ' 1 $ : - ^ J L L L L - C : :;j tive inquiry, facilitating collaboration with
; ; :=::!!!': " ji;n:i:;jrfii^VV! coresearchers through participatory re-
search, supporting democratic dialogue
nill: lUniTITli HitJi Eili E i Ti: iTI :t Tti H H IIIinnHllTiHlHlHHl^
and de libera tion, and supporting democ-
[ . :
V v : : ; f ! ; = . | ; T ! i:".'i:Ji i 1:
tf, n ^ racy through process use.
; .' jrrn?:;:r ;'!:; i'j i
; ' ; TFNJ S?FI ^ IIRR R R

: Mrhhyrjn;:=: !=:i;|;!. j.-n:;.:;i^iflfF^^i^.; Personalizing and

' l & H f t l t l i =:;";*!"!l:-!.^:!"::-^;^^
Humanizing Evaluation

One reason for using qualitative methods

in evaluation and organizational develop-
ment processes is that such strategies maybe
by studying how well-known and effective perceived and experienced by program staff
evaluators actually consult with clients and and participants as personal in nature, what
conduct themselves in doing evaluation re- Michael Cernea (1991) has called "putting
search. Utilization-focused evaluation, then, people first." Programs that are based on hu-
is a process for creatively and flexibly inter- manistic concerns and principies often resist
acting with intended evaluation users about any kind of quantification because of the
their information needs and alterna tive perception that numbers and standardized
methodological options, taking into account categorization are cold, hard, and imper-
the decision context in which an evaluation sonal. The issue here is not whether such ob-
is undertaken. jections are reasonable. The point is that
such objectification feels real to those who
hold such views. In programs where staff,
S Interactive and
funders, and/or participants feel this way,
Participatory Applications evaluations that rely on quantitative mea-
Thus far, this chapter has been describing surement may be rejected out of hand.
various evaluation issues that are especially The personal nature of qualitative inquiry
appropriate for qualitative inquiry and derives from its openness, the evaluator's
some specific evaluation models that are close contact with the program, and the pro-
176 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

cedures of observation and in-depth inter- tative methods may be perceived as more
viewing, particularly the latter, that commu- humanistic and personal simply by avoid-
nicate respect to respondents by making ing numbers.
their ideas and opinions (stated in their own Personalizing and humanizing evalua-
terms) the important data source for the tion are particularly important for educa-
evaluation. Kushner's book Personalizing tion, therapy, and development efforts that
Evaluation (2000) epitomizes this emphasis are based on humanistic values (Patton
especially in advocating that the perspective 1990). Humanistic values that undergird
of participants be given primacy: both qualitative inquiry and humanistic ap-
proaches to intervention and change include
I will be arguing for evaluators approaching the core principies listed in Exhibit 4.4.
programs through the experience of individu- Where people in a program, organization, or
ais rather than through the rhetoric of pro- community hold these kinds of values, qual-
gram sponsors and managers. I want to itative inquiry is likely to feel particularly ap-
emphasize what we can learn about programs propriate.
from Lucy and Ann. This does not mean ig-
noring the rights of program managers and
Harmonizing Program
sponsors with access to evaluation. There is no
case for using evaluation against any stake-
and Evaluation Values
holder group; though there is a case for assert- The suggestion that one reason for using
ing a compensatory principie in favor of those qualitative methods is that such strategies
who start out with relatively lower leveis of may be perceived by program staff and pro-
access to evaluation. I don't think there is a se- gram clients as more personal in nature
rious risk of evaluators losing touch with their opens up a whole range of potential philo-
contractual obligations to report on programs sophical, political, and value orienta tions
and to support program management and im- that can influence methods decisions. The
provement; I don't think there is a danger that argument here is that it can be appropriate
evaluators will ever lose their preoccupation and desirable to include among the criteria
with program effects. There is always a risk, for making methods decisions the value ori-
however, that evaluators lose contact with entations of the intended users of the study.
people; and a danger that in our concern to re- One example of a framework for supporting
port on programs and their effects we lose harmony between a program philosophy
sight of the pluralism of programs. So my ar- and evaluation approach is presented in Ex-
guments will robustly assert the need to ad- hibit 4.5. This framework illustrates how the
dress "the person" in the program. (pp. 9-10) decision to use qualitative methods in evalu-
ation can flow from the values of the people
Qualitative methods may also be per- who will use the evaluation information.
ceived as more personal because of their in- Understanding, relevance, interest, and
ductive nature. This means that, again, use are ali increased when evaluators and
rather than imposing on people or a pro- users share values about methods. The final
gram some predetermined model or hy- design of an evaluation depends on calcu-
potheses, the results unfold in a way that lated trade-offs and weighing options, in-
takes into account idiosyncrasies, unique- cluding political/philosophic/value con-
ness, and complex dynainics. Finally, quali- siderations. The design also depends on
Particular!}/ Appropriate Qualitative Applications j. 177

Common Principies Undergirding Qualitative Inquiry

EXHIBIT 4.4 and Humanistic Values

1. Each person or community is unique.

2. Each person or community deserves respect.
3. Equity, fairness, and mutual respect should be foundations of human interactions.

4. Change processes (and research) should be negotiated, agreed to, and mutually understoodnot im-
posed, forced, or required.

5. One expresses respect for and concern about others by learning about them, their perspective, and
their world-and by being personally involved.

6. Change processes shouid be person centered, attentive to the effects on real people as individuais
with their own unique needs and interests.
7. Emotion, feeling, and affect are natural, healthy dimensions of human experience.

8. The change agent, therapist, or researcher is nonjudgmental, accepting, and supportive in respecting
others' right to make their own decisions and live as they choose. The point is empowerment of oth-
ers, not contrai or judgment.

9. People and communities should be understood in context and holistically.

10. The process (how things are done) is as important as the outcome (what is achieved).

11. Action and responsibility are shared; unilateral action ts avoided.

12. Information should be openly shared and honestly communicated as a matter of mutual respect and
in support of openness as a value.

opportunity, resources, time constraints, nesses in relation to this complex constella-

and commitment. What is to be avoided is tion of both values and technical factors in
the routine selection of a design without an effort to make methods and values rela-
consideration of its strengths and weak- tively harmonious.

Developmental Applications: Action Research, Action Learning,

R e f l e c t i v e Practice, and L e a r n i n g O r g a n i z a t i o n s

V ou leam something new every day. Actually, you learn something old
every day. Just because you just learned it, doesn't mean it's new.
Other people already knew it. Columbus is a good example.

George Carlin (1997:135)

A variety of organizational, program, emerged that involve inquiry within orgarri-

and staff development approaches have zations aimed at learning, improvement,

FCDEF
178 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Matching Program Philosophy and Evaluation

Approach: An lliustration

What are the values of the school? And what are the values of the evaluation? Throughout
the life of the school, three strong themes have emerged: (a) personalized curriculum, (b) the
experiential nature of learning, and (c) the holistic nature of learning. The structures of the
school organization, the formal goals of the school, emphases in classrooms, and investments
of time and money in staff development activities have ali placed strong value on these aspects
of learning. These are not mutually exclusive, but are intricately intertwined. Their meaning for
the school and for the evaluation of the school are described in the following excerpts from the
Marcy Open School evaluation document.

Marcy Open School Philosophy Evaluation of Marcy Open School

1. Personalized curriculum 1. Personalized evaluation

Curriculum will vary for each child as teaching The determination of the success of the school
extends from the interests, needs, and abilities will vary depending upon the values and
of each child or group of children. The school perspectives of interested people. This evaluation
personnei seek to be aware of each chd as an will present a statement of what was made
individual, and of potential learning activities available by the school and of what was
and materiais. The individual chd, the teacher, accomplished by children in the school. Decisions
and the parents make decisions on the as to the success of the school and as to the
curriculum. validity of those activities must be left to the
individual reading the evaluation report,
according to his or her own perspective.

2. Experiential nature of learning 2. Experiential nature of evaluation

The school seeks to have the children experience This evaluation will attempt to provide an
language rather than only learning to read, to opportunity for the reader to experience the
experience computation rather than only to learn school and its children. It will provide not only
math, to be in and to learn from the community charts and statistics, but photographs, drawings,
rather than only to learn about social studies. and works of children and adults. Even so, it is
Participants in the school believe that experience acknowledged that, at best, any such report can
is the best transmitter of knowledge. Further- only be an imperfect representation of the school
more, the child is expected to interact with his and its processes. Furthermore, this report is not
[or her] environment-to have an effect upon it presented as a final document representing the
in the process of experiencing itto change it aecompiishments of the school, It s, instead,
or to recognize ways in which he [or she] seeks presented as a report-in-process, to be reacted
to move toward change. to and sent back for new descriptions, new data,
about which its readers may be concerned.
Particular!}/ Appropriate Qualitative Applications j. 179

Marcy Open School Phiiosophy Evaluation of Marcy Open School

3. Holistic nature of learninq 3. Holistic evidence for evaluation

Much emphasis has been placed on the Three of the schoofs goais for children have
interrelatedness of learning. Organizationa! been chosen for special attention in this
structures, activities, and materiais are evaluation. They include a range of process,
considered in terms of their multidimensionality content, and context typicai of the goal
of goals. Goa! statements and staff development statement as a whole. The evidence presented
activities give conscious attention to children's attempts to observe a natural order of events
feelings about themselves and their world, how as they happen in the school with a minimum
those relate to the child's relationships with of distortion through departmentalization.
others and how those relate to the child's Both objective figures and subjectiv judgments
interest and ability in learning. The staff seeks are included and are considered valid. The
activities that allow the child to experience the provisioning of the school and the activities
relationship between language, computation, and and products of the children are viewed,
other knowledge rather than compartmentaizing insofar as it's possible, in terms of their
them into separate content areas. mutidimen- sionality of effect.

SOURCE: Olson 1974.

and development (e.g., Argyris and Schon ing the program. In an ongoing "reflective
1978; Senge 1990; Watkins and Marsick 1993; practice group," the teachers shared their
Aubrey and Cohen 1995; Torres, Preskill, cases and interpreted the results. Based on
and Piontek 1996). These efforts are called these reflections and thematic cross-case
various things: "action learning" (Pedler analysis, staff members revised their em-
1991; McNamara 1996), "team learning" phasis on goal setting as the priority first
(Jarvis 2000), "reflective practice" (Schon step in the program. The interviews showed
1983,1987; Tremmel 1993), "action research" that new immigrants lacked sufficient expe-
(Whyte 1989; Gore and Zeichner 1995; rience with and knowledge of American ed-
Stringer 1996), internai evaluation (Sonnich- ucational and employment opportunities to
sen 2000; Love 1991), or organizational de- set meaningful and realistic goals. As a re-
velopment (Patton 1999c). These problem- sult, students typically acquiesced to coun-
solving and learning-oriented processes selor suggestions just to get the mandated
often use qualitative inquiry and case study goal-setting f orms completed, but they carne
approaches to help a group of people reflect out of the orientation without any substan-
on ways of improving what they are doing tial commitment to or ownership of what
or understand it in new ways. were supposedly their own goals. Here we
For example, the teaching staff of an eve- have an example of a short-term, rapid tum-
ning basic skills adult education program around qualitative inquiry for reflective
undertook an action learning inquiry into practice or action learning aimed at program
the experiences of students who had re- improvement. By participating in the pro-
cently immigrated to the United States. Each cess, staff members also deepened their sen-
staff person interviewed three students and sitivity to the perspectives and needs of new
wrote short case studies based on the stu- immigrant students, thereby developing
dents' reports about their experiences enter- professionally.
180 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

I'm at that point where mixed No probiem.

findings would be reassuring so We'll just use mixed methods.
we don't have to act.

Inaction research

The leaming that occurs as a result of succeed if they are carefully designed and
these processes is twofold: (1) The inquiry implemented systemically to deal with or-
can yield specific insights and findings that ganic, cultural, and power complexities. In
can change practice, and (2) those who par- qualitative terms, this means action learning
ticipate in the inquiry learn to think more inquiry that is holistic, grounded, and sensi-
systematically about what they are doing tive to context.
and their own relationship to those with Harvey and Denton (1999) examined the
whom they work, what Bawden and twin and interrelated themes of "organiza-
Packham (1998) have called "systemic tional learning" and the "learning organiza-
praxis." In many cases, the specific findings tion" in the business sector. The qualitative
are secondary to the more general learnings research underpinning their study was con-
that result from being involved in the pro- ducted over a three-year period (1994-1997)
cess, what I have called "process use" as and involved detailed examina tion of orga-
opposed to findings use (Patton 1997a). Pro- nizational learning aspira tions and practices
cess use is greatly enhanced in "develop- within the British operations of five major
mental evaluation" (Patton 1994; 1997a) in manufacturing companies. Sixty-six inter-
which the purpose of the evaluation is ongo- viewees were classified into three groups:
ing learning, internai improvement, and strategy, human resources, and research and
program development rather than generat- development (R&D). They identified a set of
ing reports and summative judgments for six antecedents that together explain the rise
externai audiences or accountability. to prominence of emphasis on organiza-
A lot of attention in recent years has been tional learning: (1) the shift in the rela tive
paid to action learning as a way of helping impor tance of factors of production away
people in organizations cope with change. from capital toward labor, particularly intel-
Mwaluko and Ryan (2000) offer a case study lectual labor; (2) the ever more rapid pace of
showing how action learning programs can change in the business environment; (3)
Particular!}/ Appropriate Qualitative Applications j. 181

widespread acceptance of knowledge as Sorensen, Yaeger, and Nicoll (2000:4) in in-

a prime source of competitive advantage; troducing a special issue of the journal OD
(4) the greater demands being place d on Practitioner devoted to AI.
ali businesses by customers; (5) increasing What interests us here is that AI is
dissatisfaction, among managers and em- grounded in qualitative understandings
ployees, with the traditional, command- and prescribes a particular process of quali-
and-control management paradigm; and (6) tative inquiry within an organization that in-
the intensely competitive nature of global cludes a dialogue process among partici-
business. Their analysis of qualitative inter- pants based on their interviewing each
views featured the interplay of thoughts and other. They ask each other questions that
feelings between management practitioners "elicit the creative and life-giving events ex-
and organizational theorists in getting in- perienced in the workplace" (Watkins and
side the idea of organizational learning. Cooperrider 2000:9):
While they focused on business applica-
tions, the idea of being a learning organiza- 1. Looking at your entire experience with
tion has also become a prominent theme in the organization, remember a time
the nonprofit, government, and philan- when you felt most alive, most fulfilled,
thropic sectors (Sonnichsen 1993; Preskill or most excited about your involvement
and Torres 1999; Preskill and Preskill 1997). in the organization....
Qualitative inquiry as practiced through the
2. Let's talk for a moment about some
lenses of action learning and reflective prac-
things you value deeply; specifically, the
tice can be one of the foundations of a learn-
things you value about yourself, about
ing organization.
the nature of your work, and about this
organization
Appreciative Inquiry
3. What do you experience as the core fac-
tors that give life to this organization?
Appreciative inquiry (AI) has emerged as
Give some examples of how you experi-
a popular organizational development ap-
ence those factors.
proach that emphasizes building on an orga-
nization^ assets rather than focusing on 4. What three wishes would you make to
problems, or even problem solving. Con- heighten the vitality and health of this
ceived and described in the work of David organization? (p. 9)
Cooperrider and his colleagues at Case
Western Reserve's School of Organization These questions aim at generating spe-
Behavior (Watkins and Cooperrider 2000), cific examples, stories, and metaphors about
AI is being offered by its advocates as "a positive aspects of organizational life. Par-
worldview, a paradigm of thought and un- ticipants in the process analyze the results in
derstanding that holds organizations to be groups looking for the themes and topics
affirmative systems created by humankind that can become the foundation for positive
as solutions to problems. It is a theory, a organizational development going forward.
mind-set, and an approach to analysis that
leads to organizational learning and creativ- For example, if the original data suggest that
ity" (Watkins and Cooperrider 2000:6). As COMMITMENT is an important factor in
such, "AI reflects the core values of OD [or- many of the stories about the best of times in
ganizational development]/' stated editors the organization, then the workgroup might
182 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

choose to ask more questions from others in In the film Awakenings, based on Dr. Oli-
the workplace about their experiences with ver Sacks's real-life experiences in a New
commitment. This second round of interviews York institution for mentally ill people
produces informa tion about four to six topics (Sacks 1973), Dr. Sacks (played by Robin Wil-
that become the basis for building "possibility liams as the character Dr. Malcolm Sayer)
propositions" that describe how the organiza- engages the assistance of nurses, orderlies,
tion will be in the future. Each topic or theme janitorial staff, and even patients in joint ef-
can be fashioned into a future statement. And forts to discover what might reach certain
these statements become an integral part of patients suffering from a rare form of catato-
the vision for the organization. Often this pro- nia. The film powerfully displays a form of
cess is completed with a future search confer- collaborative research in which trained re-
ence that uses the appreciative inquiry data as searchers and nonresearchers undertake an
a basis for imaging a positive and creative fu- inquiry together. One orderly tries music. A
ture for the organization. (Watkins and nurse tries reading to patients. A volunteer
Cooperrder 2000:10) tries card games. Together they figure out
what works.
AI has been criticized for being unbal- In an African village, the women were
anced and uncritical in its emphasis (critics skeptical of public health workers' admoni-
say oueremphasis) on accentuating the positions to use well water rather than surface
tive. It may even, ironically, discourage in- water for drinking during the rainy season.
quiry by discouraging constructive criticism Going to the well was more work. Instead of
(Golembiewski 2000). Whether it endures as simply trying to convince the women of the
a viable and popular approach to organiza- wisdom of this public health advice, the ex-
tional development remains to be seen, but tension educators created an experiment in
its questioning strategies could be incorpo- which half the village used well water and
rated into more balanced approaches. AI in- half used surface water. They kept track of
tegrates inquiry and action within a particu- illnesses with matchsticks. At the end of
lar developmental framework that guides three months, the villagers could see for
analysis and processes of group interaction. themselves that there were many more
The qualitative questioning and thematic matchsticks in the surface water group. By
analysis processes constitute a form of inter- participating in the study rather than just re-
vention by the very nature of the questions ceiving results secondhand, the findings be-
asked and the assets-oriented framework came more meaningful to them and more
used to guide analysis. In this way, inquiry useful.
and action are completely integrated. Other
Early in my career, I was commissioned
forms of participatory inquiry also seek inte-
by a provincial deputy minister in Canada to
gration of inquiry and action.
undertake an evaluation in a school division
he considered medocre. I asked what he
wanted the evaluation to focus on. "I don't
Participatory Research
care what the focus is," he replied. "I just
and Evaluation: Valuing
want to get people engaged in some way.
and Facilitating Collaboration Educa tion has no life there. Parents aren't in-
volved. Teachers are just putting in time.
Let's start with three quite different ex- Administrators aren't leading. Kids are
amples of participatory qualitative inquiry. bored. I'm hoping that having them partici-
Particular!}/ Appropriate Qualitative Applications j. 183

pate in an evaluation will stir things up and history. John Elliott (1976) worked with
get people involved again." classroom teachers as coresearchers doing
When conducting research in a collabora- action research "developing hypotheses
tive mode, professionals and nonpro- about classrooms from teachers' practical
fessionals become coresearchers. Humanis- constructs." Bill Hull (1978) worked with
tic research and heuristic inquiry (Douglass teachers in a reflective, research-oriented
and Moustakas 1985) value collaboration, as process to study children's thinking. The
does "cooperative inquiry" (Heron 1996). Boston Woinen's Teachers' Group (1986)
Participatory action research (Wadsworth was organized as a research collaborative for
1993a, 1993b; King and Lonnquist 1994a, studying the effects of teaching on teachers.
1994b) encourages joint collaboration within Eleanor Duckworth's (1978) classic evalua-
a mutually acceptable ethical framework to tion of an African primary science program
understand and/or solve organizational or was collaborative in approach.
community problems. Feminist methods are Genuinely collaborative approaches to
participatory in that "the researcher invites research and evaluation require power shar-
members of the setting to join her in creating ing. One of the negative connotations often
the study" (Reiriharz 1992:184). "Empower- associated with evaluation is that it is some-
ment evaluation" aims to foster "self-deter- thing done to people. Participatory evalua-
mination" among those who participate in tion, in contrast, involves working with peo-
the inquiry process (Fetterman 2000a; ple. Instead of being research subjects, the
Fetterman, Kaftarian, and Wandersman people in the research setting become
1996). This can involve forming "empower- "co-investigators." The process is facilitated
ment partnerships" between researchers by the researcher, but is controlled by the
and participants (Weiss and Greene 1992) people in the program or community. They
and teaching participants to do research undertake a formal, reflective process for
themselves (Wadsworth 1984). In-depth in- their own development and empowerment.
terviewing and description-oriented obser- Participatory evaluation has been used
vations are especially useful methods for with great success as part of intemational
supporting collaborative inquiry because and community development efforts by a
the methods are accessible to and under- number of nongovernmental organizations
standable by people without much technical (NGOs) and private voluntary organiza-
expertise. tions (PVOs) in the Third World (e.g., Aubel
Interest in participatory research has ex- 1993). A collaborative group called Private
ploded in recent years, especially as an ele- Agencies Collaborating Together (PACT)
ment of larger community change efforts published the excellent users guide Partici-
(Stoecker 1999). The principal researcher patory Evaluation (1986) as well as the more
trains the coresearchers to observe, inter- general Evaluation Sourcebook (Pietro 1983).
view, reflect, and / or keep careful records or The guide includes techniques for actively
diaries. Those involved come together peri- involving nonliterate people as active par-
odically to share in the data analysis process. ticipants in evaluating the development ef-
The purpose of such shared inquiry is typi- forts they experience, often using qualita-
cally to elucidate and improve the nature of tive methods.
practice in some arena of action. The processes of participation and collab-
Qualitative, collaborative research efforts oration have an impact on participants and
in educational settings have a distinguished collaborators quite beyond whatever find-
184 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

ings or report they may produce by working themselves not so important as what is
together. In the process of participating in re- learned from the discussion and from the pro-
search, participants are exposed to and have cess of reaching consensus on what questions
the opportunity to learn the logic of evi- should be used to evaluate group perfor-
dence-based inquiry and the discipline of mance and capacity, and on what answers best
evidentiary reasoning. Skills are acquired in describe their group's present status. (p. 272)
problem identification, criteria specifica-
tion, and data collection, analysis, and inter- It was not a group's specific questions or
pretation. Through acquisition of inquiry answers that Uphoff found most affected the
skills and ways of thinking, a collaborative groups he observed. It was the process of
inquiry process can have an impact beyond reaching consensus about questions and en-
the findings generated from a particular gaging with each other in the meaning of the
study. answers turned up. The process of participa-
Moreover, people who participate in cre- tory self-evaluation, in and of itself, pro-
ating something tend to feel more owner- vided useful learning experiences for partic-
ship of what they have created and make ipants.
more use of it. Active participants in re- Viewing participatory inquiry as a means
search and evaluation, therefore, are more of creating an organizational culture com-
likely to feel ownership not only of their find- mitted to ongoing learning, as discussed in
ings but also of the inquiry process itself. the previous section, has become an impor-
Properly, sensitively, and authentically tant theme m recent literature linking pro-
done, it becomes their process. Participants gram evaluation to "learning organizations"
and collaborators can be community mein- (e.g., King 1995; Leeuw, Rist, and Son-
bers, villagers, organizational workers, pro- nichsen 1993). "The goal of a participatory
gram staff, and/or program participants evaluator is eventually to put him- or herself
(e.g., chents, students, farmers). Some times out of work when the research capacity of
administrators, funders, and others also the organization is self-sustaining" (King
participate, but the usual connotation is that 1995:89). Indeed, the "self-evaluating orga-
the primary participants are "lower down" nization" (Wildavsky 1985) constitutes an
in the hierarchy. Participatoiy evaluation is important direction in the institution-
bottom-up. The trick is to make sure that alization of evaluation logic and processes.
participation is genuine and authentic, not I advise caution and care in using the la-
just token or rhetorical, especially in bel "participatory inquiry," "empowerment
participative evaluation, where differing evaluation," or "collaborative research," for
political and stakeholder agendas often these terms mean different things to differ-
compete (Fricke and Gill 1989). ent peopleand serve different purposes.
Norman Uphoff (1991) has published "A Some use these terms interchangeably or as
Field Guide for Participatory Self-Evalua- mutually reinforcing concepts. Wadsworth
tion" aimed at grassroots community devel- (1993a) distinguished "research on people,
opment projects. After reviewing a number for people or with people" (p. 1). Levin (1993)
of such efforts, he concluded: distinguished three purposes for collabora-
tive research: (1) the pragmatic purpose of
If the process of self-evaluation is carried out increasing use of findings by those involved
regularly and openly, with ali group members as emphasized by, for example, Cousins and
participating, the answers they arrive at are in Earl (1992); (2) the philosophical or method-
Particular!}/ Appropriate Qualitative Applications j. 185

Principies of Fully Participatory and Genuinely

EXHIBIT 4.6 Coliaborative Inquiry

The inquiry process involves participants in learning inquiry logic and skills, for example, the na-
ture of evidence, establishing priorities, focusing questions, interpreting data, data-based deci-
sion making, and connecting processes to outcomes.
Participants in the process own the inquiry. They are invoived authentically in making majorfo-
cus and design decisions. They draw and apply conclusions. Participation is real, not token.
Participants work together as a group and the inquiry facilitator supports group cohesion and
collective inquiry.
Ali aspects of the inquiry, from research focus to data analysis, are undertaken in ways that are
understandable and meaningful to participants.
The researcherorevaluatoracts as a facilitator, collaborator.and learning resource; participants
are coequa!.
The inquiry facilitator recognizes and values participants' perspectives and expertise and works
to help participants recognize and value their own and each other's expertise.
Statusand power differences between the inquiry facilitator and participants are minimized, as
much as possible, practical, and authentic, without patronizing or game playing.

ological purpose of grounding data in par- setting being studied as co-inquirers, at least
ticipants' perspectives; and (3) the political to some important extent, though the de-
purpose of mobilizing for social action, for gree and nature of involvement vary widely.
example, empowerment evaluation or what My purpose here has been to point out that
is sometimes called "emancipatory" re- these participatory approaches often em-
search (Cousins and Earl 1995:10). A fourth ploy quahtative methods because those
purpose, identified here, is teaching inquiry methods are understandable, teachable, and
logic and skills. Since no definitive defini- usable by people without extensive research
tions exist for "participatory" and "coliabo- training.
rative evaluation," these phrases must be
defined and given meaning in each setting
where they're used. Exhibit 4.6 presents pri- Supporting Democratic
mary principies for fully participatory and Dialogue and Deliberation
genuinely coliaborative inquiry. This list can
be a starting point for working with partici- Most of this chapter has examined rela-
pants in a research or evaluation setting to tively small-scale applications of quali-
decide what principies they want to adopt tative methods in evaluating programs,
for their own process. developing organizations, supporting plan-
Regardless of the terminologypartici- ning processes and needs assessments, and
patory, coliaborative, cooperative, or em- providing insights into communities. This
powermentthese approaches share a com- section considers a much larger agenda, that
mitment to involving the people in the of strengthening democracy. House and
186 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Howe (2000) have articulated three require- sequence that the evaluator should seek to
ments for evaluation done in a way that sup- represent the full range of interests in the
ports democracy: inchision, dialogue, and de- course of designing an evaluation. In that
liberation. They worry about the power that way, an evaluator can support an informed
derives from access to evaluation and the citizenry, the sine qua non of strong democ-
implications for society if only the powerful racy, by acting as information broker be-
have such access. tween groups who want and need knowl-
edge of each other. The democratic
We believe that the background conditions for evaluator must make the methods and tech-
evaluation should be explicitly democratic so niques of evaluation accessible to nonspe-
that evaluation is tied to larger society by dem- cialists, that is, the general citizenry. Mac-
ocratic principies argued, debated, and ac- Donald^ democratic evaluator seeks to
cepted by the evaluation community. Evalua- survey a range of interests by assuring confi-
tion is too important to society to bepurchased dentiality to sources, engaging in negotia-
by the highest bidder or appropriated by the tion between mterest groups, and making
most powerful interest. Evaluators should evaluation findings widely accessible. The
be self-conscious and deliberate about such guiding ethic is the public's right to know.
matters.... Saville Kushner (2000) has carried for-
If we look beyond the conduct of individ- ward, deepened, and updated MacDonald's
ual studies by individual evaluators, we can democratic evaluation model. He sees eval-
see the outlines of evaluation as an influential uation as a form of personal expression and
societal institution, one that can be vital to the political action with a special obligation to
realization of democratic societies. Amid the be critics of those in power. He uses qualita-
claims and counterclaims of the mass media, tive methods to place at the center of evalua-
amid public relations and advertising, amid tion the experiences of people in programs.
the legions of those in our society who repre- The experiences and perceptions of the peo-
sent particular interests for pay, evaluation ple in programs, the supposed beneficiaries,
can be an institution that stands apart, reliable are where, for Kushner, we will find the in-
in the accuracy and integrity of its claims. But tersection of Politics (big PPolicy) and pol-
it needs a set of explicit democratic principies itics (small ppeople). He uses qualitative
to guide its practices and test its intuitions. case studies to capture the perspectives of
(House and Howe 2000:4) real peoplechildren and teachers and par-
entsand the realities of their lives in pro-
Qualitative inquiry figures into this dem- gram settings as they experience those reali-
ocratic approach to evaluation because, as ties. He feels a special obligation to focus on,
discussed in the section on participatory re- capture, report, and therefore honor the
search, qualitative methods are especially views of marginalized peoples. He calls this
accessible to and understandable by non- "personalizing evaluation," but the larger
researchers, and because case studies can be agenda is strengthening democracy. Con-
an excellent resource for supporting inclu- sider these reflections on the need for evalu-
sion and dialogue. In Europe, the demo- ators and evaluations to address questions
cratic evaluation model of Barry MacDonald of social justice and the democratic contract:
(1987) illustrates these emphases. He argued
that "the democratic evaluator" recognizes Where each social and educational program
and supports value pluralism with the con- can be seen as a reaffirmation of the broad so-
Particular!}/ Appropriate Qualitative Applications j. 187

ciai contract (that is, a re-confirmation of the of findings for enhanced decision making
bases of power, authority, social structure, and program improvement and, therefore,
etc.), each program evaluation is an opportu- making sure that findings reflected the di-
nity to review its assumptions and conse- verse perspectives of multiple stakeholders,
quences. This is commonly what we do at including the less powerful and participants
some levei or another. Ali programs expose de- in programs (instead of just staff, adminis-
mocracy and itsfailings; each program evaluation tra tors, and funde rs). While this thrust re-
is an assessment ofthe effectiveness ofdemocracy mains important, a parallel and reinforcing
in tackling issues in the distribution ofwealth and use of evaluation focuses on helping people
power and social goods. Within the terms of the learn to think and reason evaluatively,, and how
evaluation agreement, taking this levei of rendering such help can contribute to
analysis into some account, that is, renewing strengthening democracy over the long
part of the social contract, is to act more au- term. I turn now to elaborate that contribu-
thentically; to set aside the opportunity is to tion.
act more mauthentically, that is, to accept the
fictions. (Kushner 2000:32-33, emphasis added)
Supporting Democracy
Through Process Use: Helping
While MacDonald, Kushner, and House
the Citizenry Weigh Evidence
and Howe make explicit linkages between
evaluation and democracy, a number of and Think Evaluatively
other evaluation approaches imply such
linkages by emphasizing various degrees Let me begin by offering some context for
and types of stakeholder participation and reflecting on the role of knowledge creation
involvement and, correspondingly, evalua- in relation to democracy In the autumn of
tor responsiveness. For reviews of the vari- 2000,1 had the opportunity to participate in
ety of such approaches and distinctions a seminar sponsored by the Italian Evalua-
among them, see Cousins and Earl (1995) tion Society. While in Rome I visited the F-
and Alkin (1997). The work of Mertens rum, wandered among the ruins of ancient
(1998, 1999) on "inclusive evaluation" and Rome, and spent some meditative time in
the "empowerment evaluation" model of the remains of the Senate seeking inspiration
Fetterman et al. (1996) offer additional ex- about what I might say regarding evalua-
amples of evaluation approaches that em- tion's potential contributions to democracy,
phasize qualitative inquiry and support the theme of the European Evaluation Soci-
democratic principies, social justice, and ex- ety conference in Lausanne, Switzerland,
plicitly political foundations of evaluation where I was headed after Rome. Nothing
in support of those whose stakes tend to be came to me in the Frum, at least nothing
underrepresented in policy discussions be- about evaluation. I couldn't get past vivid
cause they are marginalized economically, images of Caesar's death in that place as
socially, and politically. portrayed by Shakespeare. After leaving the
Taken together, these writings on evalua- Frum, I walked across to the Coliseum,
tion^ role in supporting democratic pro- where gladiators did battle before emperors
cesses reflect a significant shift in the nature and citizens of Rome. There, standing in the
of evaluation^ real and potential contribu- platform area reserved for the senators, I got
tion to strengthening democracy. A decade a distinct image. I imagined an evaluator
ago, the emphasis was ali on increasing use presenting important policy findings to the
188 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

citizens of Rome in the Coliseum. At the end racy. Having experienced totalitarianism,
of the report, the emperor would invite the then having fled it, she devoted much of her
crowd to render a thumbs-up or thumbs- life to studying it and its opposite, democ-
down on the evaluation. Thumbs- up would racy. She believed that thinking thought-
mean a laurel wreath, coin of the realm, and fully in public deliberations and acting
an all-expenses-paid trip to present at the democratically were intertwined. Totalitar-
annual meeting of the Imperial Evaluation ian- ism is built on and sustained by deceit
Society. Thumbs-down would mean the and thought control. To resist efforts by the
lions got fed. I left the Coliseum quickly powerful to deceive and control thinking,
thinking how fortunate I was to be engaged Arendt believed that people needed to prac-
in evaluation at a time when the stakes, tice thinking. Toward that end she devel-
though high, are not quite so high as my Col- oped "eight exercises m political thought"
iseum vision and when, instead, in the beau- (Arendt 1968). She wrote that "experience in
tiful city of Lausanne, in the year 2000, an in- thinking... canbe won, like ali experience in
ternational community of professionals met doing something, only through practice,
together to spin visions of strengthening de- through exercises" (p. 4). From this point of
mocracy by the ways in which we engage in view, might we consider every evaluation an
evaluation and applied research. opportunity for those involved to practice
So what is the connection between quali- thinking? This would mean that every eval-
tative inquiry and democracy? uation is an opportunity to strengthen de-
Start with the premise that a healthy and mocracy by teaching people how to think
strong democracy depends on an informed evaluatively. In this regard, we might aspire
citizenry. A central contribution of policy re- to have policy research, action research, par-
search and evaluation, then, is to help ensure ticipatory research, and collaborative evalu-
an informed electorate by disseminating ation do what Arendt hoped her exercises in
findings as well as to help the citizenry political thought would do, namely, give us
weigh evidence and think evaluatively. This "experience in how to think." Her exercises
involves thinking processes that must be do not prescribe "what to think or which
learned. It is not enough to have trustworthy truths to hold"; rather, they focus on the act
and accurate informa tion (the informed part and process of thinking. For example, she
of the informed citizenry). People must also thought it important to help people think
know how to use information, that is, to conceptually, to "discover the real origins of
weigh evidence, consider inevitable contra- original concepts in order to distili from
dictions and inconsistencies, articulate val- them anew their original spirit which has so
ues, interpret findings, deal with complex- sadly evapora ted from the very keywords of
ity, and examine assumptions, to note but political languagesuch as freedom and
a few of the things meant by "thinking justice, authority and reason, responsibility
evaluatively." Moreover, in-depth demo- and virtue, power and gloryleaving be-
cratic thinking includes political sophistica- hind empty shells" (Arendt 1968:14-15).
tion about the origins and implications of Might we add to her conceptual agenda for
the categories, constructs, and concepts that examination and public dialogue such terms
shape what we experience as information as outcomes and performance indicators, inter-
and "knowledge" (Minnich forthcoming). pretation and judgment, and beneficiary and
Philosopher Hannah Arendt was espe- stakeholder, among many evaluative possi-
cially attuned to this foundation of democ- bilities?
Particular!}/ Appropriate Qualitative Applications j. 189

Helping people learn to think evalu- viewing a variety of people and under-
atively by participating in real evaluation standing in-depth case examples are espe-
exercises is what I've come to call "process cially effective ways of enhancing nonre-
use" (Patton 1997a, 1998). I have defined searcher involvement in evaluation and
process use as relating to and being indi- research to help them increase their capacity
cated by individual changes in thinking and to think about evidence and draw appropri-
behaving that occur among those involved ate conclusions from data.
in evaluation as a result of the learning that Helping people learn to think in these
occurs during the evaluation process. (Changes ways can have a more enduring impact from
in program or organizational procedures a study than use of specific findings gener-
and culture may also be manifestations of ated in that same study. Findings have a very
process impacts, but that is not our focus short half-lifeto use a physical science
here.) This means an evaluation can have metaphor. They deteriorate very quickly as
dual tracks of impact in strengthening de- the world changes rapidly. Specific findings
mocracy: (1) a more informed electorate typically have a small window of relevance.
through use of findings and (2) a more In contrast, learning to think and act
thoughtful and deliberative citizenry evaluatively can have an ongoing impact.
though helping people learn to think and en- The experience of being involved in an eval-
gage each other evaluatively. uation, then, for those stakeholders actually
One way of thinking about process use is involved can have a lasting impact on how
to recognize that evaluation constitutes a they think, on their openness to reality test-
cultural perspective of sorts. When we en- ing, on how they view the things they do,
gage other people in the evaluation process, and on their capacity to engage thoughtfully
we are providing them with a cross-cultural in democratic processes.
experience. This culture of evaluation, Democratic evaluations debunk the myth
which we as evaluators take for granted in that methods and measurement decisions
our own way of thinking, is quite alien to are purely technical. Nonresearchers then
many of the people with whom we work at become savvier about both the technical and
program leveis. Examples of the values of nontechnical dimensions of evaluation.
evaluation include clarity, specificity, and Moreover, we know that use is enhanced
focusing; being systematic and making as- when practitioners, decision makers, and
sumptions explicit; operationalizing pro- other users fully understand the strengths
gram concepts, ideas, and goals; distin- and weaknesses of evaluation data and that
guishing inputs and processes from out- such understanding is increased by being in-
eomes; valuing empirical evidence; and volved in making methods decisions. We
separating statements of fact from interpre- know that use is enhanced when intended
tations and judgments. These values consti- users participate in making sure that as
tute ways of thinking that are not natural to trade-offs are considered, as they nevitably
people and, indeed, quite alien to many. are because of limited resources and time,
When we take people through a process of the path chosen is informed by relevance.
participatory research or evaluationat We know that use is enhanced when users
least in any kind of stakeholder involvement buy into the design and find it credible and
or collaborative process, they are in fact valid within the scope of its intended pur-
learning how to think in these ways. Inter- poses as determined by them. And we know
190 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

that when evaluation findings are pre- blandishments, I for one will have none of it.
s ente d, the substance is less likely to be un- (p-1)
dercut by debates about methods if users
have been in volve d in those decisions prior For others who will have none of it, one
to data collection (Patton 1997a). way to address the issue of methodological
At its roots, participatory evaluations are quality in democratic evaluations is to
informed by a fundamental confidence in reframe the policy analysfs function from
the wisdom of an informed citizenry and a an emphasis on generating expert judg-
willingness to engage ordinary citizens re- ments to an emphasis on supporting in-
spectfully in ali aspects of evaluation, informed dialogue, including methodological
cludingmethodological discussions and de- dialogue. The traditional expert-based sta-
cisions. This point is worth emphasizing tus of researchers, scholars, and evaluators
because somenot ali, to be sure, but has fueled the notion that we provide scien-
someresistance to participatory evalua- tifically based answers and judgments to
tion derives from the status associated with policymakers while, by our independence,
research expertise and an elitist or patroniz- we ensure accountability to the general pub-
ing attitude toward nonresearchers (they lic. Playing such a role depends on a knowl-
are, after ali, "subjects"). Egon Guba (1978) edge paradigm in which correct answers
has described in powerful language this ar- and independent judgments can be con-
chetype: ceived of existing. However, postinod-
ernism, deconstruction, criticai theory, femi-
nist theory, empowerment evaluation, and
It is my experience that evaluators sometimes constructivism, among other perspectives,
adopt a very supercilious attitude with respect share skepticism about the traditional
to their clients; their presumptuousness and truth-oriented knowledge paradigm. They
arrogance are sometimes overwheLming. We offer, in contrast, an emphasis on interest-
treat the client as a "child-like" person who acknowledged interpretations articulated
needs to be taken in hand; as an ignoramus and discussed within an explicit context (po-
who cannot possibly understand the tactics litical, social, historical, economic, and cul-
and strategies that we will bring to bear; as tural). Constructivist orienta tions to qualita-
someone who doesn't appreciate the ques- tive inquiry have played a criticai role in the
tions he ought to ask until we tell himand emergence of dialogical forms of inquiry
what we tell him often reflects our own biases and analysis. Participatory methods have
and interests rather than the problems with increased the access of nonresearchers to
which the client is actually beset. The phrase both research findings and processes. In
"Ugly American" has emerged in interna- combination, constructivist, dialogical, and
tional settings to describe the person who en- participatory approaches offer a vision of re-
ters into a new culture, immediately knows search and evaluation that can support de-
what is wrong with it, and proceeds to foist his liberative democracy in the postmodern
own solutions onto the locais. In some ways I knowledge age. Such a grandiose, even
have come to think of evaluators as "Ugly bombastic, vision derives from recognition
Americans." And if what we are looking for that in this emergent knowledge age, re-
are ways to manipula te clients so that they will searchers have larger responsibilities than
fali in with our wishes and cease to resist our just publishing in academic journals.
Particular!}/ Appropriate Qualitative Applications j. 191

lL Special Applications

f you come to a fork in the road, take it.

Yogi Berra

These final sections examine some special questionnaire. Indeed, administration of

applications and situations where qualita- such instruments may produce artificial re-
tive strategies are especially appropriate: the sults because respondents are affected by
need for unobtrusive measures; state-of-the- the process, what measurement specialists
art considerations; adding depth, detail, and call "reactivity/' a source of invalidity in
meaning to quantitative analyses; rapid re- measurement. Educational researcher Edna
connaissance; capturing and communicat- Shapiro (1973), to her surprse, found this to
ing stories; legislative auditing and monitor- be precisely the case in her study of innova-
ing; futuring applications; and breaking tive Follow Through classrooms. She found
routines. that standardized tests biased evaluation re-
sults by imposing an obtrusive and con-
The Need for trolled stimulus in an environment where
spontaneity, creativity, and freedom of ex-
Unobtrusive Measures
pression were valued and encouraged.
Shapiro found that the results of the test
Qualitative strategies can be particularly
measured response to a stimulus (the test)
appropriate where the administration of
that was essentially alien to the experience
standardized instruments, assigning people
of the children. Because the classrooms she
to comparison groups, and/or the collection
studied relied substantially less on paper-
of quantitative data would affect program
and-pencil skills than traditional schools,
operations by being overly intrusive. Exam-
and because student progress was moni-
ples of unobtrusive measures include ob-
tored daily on a personal basis without the
serving and measuring wear and tear on
use of written examinations, student out-
carpets placed in front of different museum
comes in these classrooms could notbe "ob-
exhibits to evaluate visitor interest in differ-
jectively" measured by the sudden introduc-
ent exhibits; observing people in a public
tion of standardized tests.
place, as a participant in whatever is going
on, without taking notes; and using docu-
ments or reports prepared for other pur- I assumed that the internalized effects of dif-
poses (e.g., clinicai case notes) to generate re- ferent kinds of school experience could be ob-
search case data in situations where no served and inferred only from responses in
additional human subject protection per- test situations, and that the observation of
mission is required because the data are rou- teaching and learning iri the classroom should
tinely collected and findings will be re- be considered auxiliary information, useful
ported only in the aggregate. chiefly to doeument the differences in the chil-
Observations of program activities and dren^ group learning experiences. . . . The
informal interviews with participants can findings of this study, with the marked dispar-
often be carried out in a less obtrusive fash- ity between classroom responses and test re-
ion than having everyone complete a test or sponses, have led me to reevaluate this
192 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

rationale. This requires reconsidera tion of the outcomes, no acceptable, valid, and reliable
role of classroom data, individual test situa- measures exist. The extent to which one be-
tion data, and the relation between them. . . . lieves that particular instruments, such as
The individual's responses in the test situation personality tests, are useful, valid, and reli-
have conventionally been considered the pri- able can be a matter of debate and judgment.
mary means to truth about psychological Moreover, for desired program outcomes
functioning. Test behavior, whether consid- where measures have not been developed
ered as a sign or sample of underlying func- and tested, it can be more appropriate to
tion, is treated as a pure measure. Yet, the test gather descriptive information about what
situation is a unique interpersonal context in happens as a result of program activities
which what is permitted and encouraged, ac- than to use some scale that has the merit of
ceptable and unacceptable, is carefully de- being quantitative but whose validity and
fined, explicitly and implicitly. Responses to reliability are suspect.
tests are therefore made under very special circum- Creativity is a prime example. While
slances. The variables that influence the outcome there are some instruments that purport to
are different from those which operate in the class- measure creativity, the applicability of those
room. (Shapiro 1973:532-34, emphasis added) instruments in diverse situations is at least
open to question. Thus, a program that aims
In their imagina tive classic, Unobtrusive to support students or participants in being
Measures, Webb et ai. (1966) discussed at more creative might do better to document
length the problems of "reactive measure- in detail the activities, products, behaviors,
ment effects." A basic theme of their work feelings, and actual creations of participants
was that research participants' awareness instead of administering some standardized
that they are part of a study (as they com- instrument of marginal or dubious rele-
plete questionnaires or take tests) might dis- vance. Qualitative documenta tion can be in-
tort and confound the study's findings. spected and judged by interested evaluation
Their documenta tion of the sources and na- users to make their own interpretations of
ture of reactivity problems in scholarly so- the extent to which creativity was exhibited
cial science research makes it highly likely by the products produced.
that such problems are magnified in evalu- Even such hallowed concepts as self-es-
ation research (see Holley and Arboleda- teem are open to considerable controversy
Florez 1988). While qualitative methods are
when it comes to specifying measurement
also subject to certain reactivity problems (to
criteria. In addition, for people whose
be discussed in later chapters), the less for-
self-esteem is already quite high, instru-
mal and less obtrusive nature of some quali-
ments that measure self-esteem will not be
tative strategies can reduce or even elimina te
very sensitive to incrementai changes that
distorting reactivity.
may be important to the people involved.
For staff development or leadership training
State-of-the-Art programs that include enhanced self-esteem
Considerations: Lack of Proven as an outcome goal, it may be more useful to
Quantitative Instrumentation do case studies to document changes experi-
enced by participants rather than rely on a
Another reason for using qualitative standardized measurement scale of prob-
methods is that for particular phenomena or lematic relevance and sensitivity.
Particular!}/ Appropriate Qualitative Applications j. 193

The same point can be made with regard curred at a time when farnily violence and
to controversy surrounding even long- child sexual abuse were just emerging into
standing measureinent instruments. The societal consciousness and as a focus of
use of standardized achievement tests to scholarly inquiry. Exploratory work of this
measure student learning is a prime exam- kind is the way that new fields of inquiry are
ple. Strong arguments have been made at- developed, especially in the policy arena.
tacking the relevance of universal, standard-
ized achievement tests for the evaluation of Confirmatory and
particular local programs (Perrone 1977). Elucidating Research: Adding
The way in which norm-referenced, stan- Depth, Detail, and Meaning
dardized achievement tests are constructed to Quantitative Analyses
reduces their relevance and validity for par-
ticular local programs, especially those that At the opposite end of the continuum
serve populations where scores are likely to from exploratory research is the use of quali-
cluster at the lower or higher extremes of the tative methods to add depth and detail to
normal curve. For such programs, more ac- completed studies that used quantitative
curate evaluation results can be produced data where the statistical results indicate
through documentation of actual student global patterns generalizable across settings
portfolios, that is, developing case histories or populations. For example, when a
of what students can do and have done over large-scale survey has revealed certain
time rather than relying on their responses marked and significant patterns of re-
to a standardized instrument administered sponses, it is often helpful to fill out the
under artificial conditions at a moment in meaning of those patterns through in-depth
time (Carrni 1975,1979; Buxton 1982). study using qualitative methods. The quan-
A related state-of-the-art consideration is titative data identify areas of focus; the qual-
explora tory research. In new fields of study itative data give substance to those areas of
where little work has been done, few defini- focus. Consider: What did people really
tive hypotheses exist and little is known mean when they marked that answer on the
about the nature of the phenomenon, quali- questionnaire? What elaborations can re-
tative inquiry is a reasonable beginning spondents provide to clarify responses?
point for research. Excellent examples of How do the various dimensions of analysis
such exploratory research are Angela fit together as a whole from the perspective
Browne's (1987) study When Battered Women of respondents? Follow-up interviews with
Kill; a qualitative study of female child sex- a subsample of respondents can provide
ual offenders in a Minnesota treatment pro- meaningful additional detail to help make
gram (Mathews, Matthews, and Speltz sense out of and interpret survey results.
1989); follow-up interviews documenting Qualitative data can put flesh on the bons
the effects of reunification on sexually abu- of quantitative results, bringing the results
sive families (Matthews, Raymaker, and to life through in-depth case elaboration.
Speltz 1991); Jane Gilgun's (1991) work on Moreover, while the role of qualitative re-
the resilience of and intergenerational trans- search in exploratory inquiry is relatively
mission of child sexual abuse; and related well understood, the confirmatory and elu-
frontline, sinall- scale studies of farnily sex- cidating roles of qualitative data are less well
ual abuse (Patton 1991). These studies oc- appreciated. Adding depth and detail to sta-
194 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

tistical findings is one aspect of confirmation spend less time planning and more time out
and elucidation. Within major traditions of in the trenches talking to people and watch-
theory-oriented qualitative inquiry (Chap- ing what's going on.
ter 3), qualitative methods are also the meth- In crisis epidemiological work, as in the
ods of choice in extending and deepening outbreak of highly contagious diseases (e.g.,
the theoretical propositions and under- the Ebola virus in frica) or the emergence
standings that have emerged from previous of AIDS, rapid reconnaissance teams made
field studies. In short, qualitative inquiry is up of medicai personnel, public health re-
not just for exploratory purposes. searchers, and social scientists are deployed
to investigate the crisis and determine im-
Rapid Reconnaissance mediate interventions and longer-term ac-
tions needed. The film And the Band Played
Sometimes information is needed On, about the early incidences of the HIV vi-
quickly. Indeed, this is increasingly the case rus before it had been identified and named,
in our rapidly changing world. There may shows the intensity of this kind of inter-
be no time to search the literature, develop view-based, snowball-sampling (getting
hypotheses, select probabilistic samples new contacts from each person inter-
based on definitive population enumera- viewed), and field-based crisis reconnais-
tions, and develop, pilot test, and administer sance inquiry.
new instruments. One major advantage of In the late 1980s, cooperative extension
qualitative methods is that you can get into services in the United States adopted a new
the field quickly. approach to programming based on rapid
Experimental, deductiv, hypothesis- response to emergent issues (Patton 1987b,
testing strategies can require a lot of 1988a; Dalgaard et al. 1988). This meant do-
front-end work. You've got to be quite cer- ing ongoing "environmental scanning,"
tain about design and instrumentation before which included content analysis of newspa-
data collection because once the study is un- pers and periodicals, conducting focus
der way, changes in design and measure- groups with emergingnew client groups, in-
ment undermine both internai and externai terviewing key informants well placed to
validity. Naturalistic inquiry, in contrast, identify cutting-edge issues, and making
permits the researcher to enter the field with systematic observations of what is happen-
relatively little advance conceptualization, ing in counties and regions throughout the
allowing the inquirer to be open to whatever United States. Issues teams worked together
becomes salient to pursue. The design is using both quantitative and qualitative in-
emergent and flexible. The questions unfold formation to identify trends, scan the envi-
as the researcher pursues what makes sense. ronment, and formulate new programs
Rapid reconnaissance or "rapid assess- based on emergent issues. These teams
ment" (Beebe 2001) connotes doing field- sometimes undertake rapid reconnaissance
work quickly, as does "quick ethnography" fieldwork to get detailed, descriptive infor-
(Handwerker 2001). In our highly dynamic mation about a developing situation, for ex-
world, it's important to stay close to the ac- ample, an influx or out-migration of new
tion. Best-selling author Tom Peters (1987) people in an area; the impact of sudden eco-
called his field-based, close-to-the-action nomic changes in a county; the sudden ap-
management approach "thriving on chos." pearance of a crop disease or pest; or a rapid
He included the admonition to managers to increase in some problem such as teenage
Particular!}/ Appropriate Qualitative Applications j. 195

pregnancies, crack cocaine-addicted new- nomena and assess quickly developing situ-
borns, homelessness, or elderly persons in ations in a world of rapid change.
need of long-term care. Action research
teams can do rapid reconnaissance on these Capturing and
issues quickly, going where the action is, Communicating Stories
talking to people and observing what is hap-
pening. To keep my head straight I reserve certain
The farming systems approach to interna- words for specialized meaningscase study
tional development was developed using and ethnography are two such labels. I write
rapid reconnaissance teams (Shaner, Phil- stories, not case studies, although readers of
ipp, and Schmehl 1982a, 1982b). Interdisci- my stories may mistakenly call them ethno-
plinary teams conduct fieldwork and in- graphic research or case studies. Were I doing
formal interviews to construct an initial, ethnography or ethnology, which I never do, I
exploratory, and qualitative portrayal of the would have a much heavier burden. I would
farming system in a particular agroeco- have to address questions of validity, of theory
logical area. Through the fieldwork, which contribution, of completeness, of generality,
may last from one to three weeks, the teams of replicability.... A story documents a given
are able to identify system characteristics, milieu in an attempt to communicate the gen-
farmers' needs, extension possibilities, and eral spirit of things. The story need not test
new applied agricultural research priori- theory; need not be complete; and need not be
ties. The Caribbean Agricultural Extension robust in either time or depth. (Denny 1978:1)
Project, for example, used 10-day rapid re-
connaissance studies in each of 10 different In The Springboard, Stephen Denning
islands to assess needs and develop inter- (2001) explams "how storytelling ignites ac-
ventions for the extension services in those tion in knowledge-era organizations." He
countries (Alkin and Patton 1987; Alkin et al. teaches storytelling as a powerful and for-
1989; Patton 1988b). mal discipline for organizational change
In the farming systems literature, these and knowledge management. What he calls
rapid reconnaissance surveys are often "springboard" stories are those that com-
called sondeos after the Spanish word mean- municate new or envisioned strategies,
ing "sounding." A sondeo is a qualitative structures, identities, goals, and values to
"sounding out" of what is happening. Infor- employees, partners, and even customers.
mal interviews and observations are done He argues that storytelling has the power to
on farms and in homes to document and un- transform individuais and organizations.
derstand variations within some defined He offers as an example his frustrated ef-
geographical area. Once interventions be- forts, as director of knowledge management
gin, either research or extension interven- of the World Bank, to convince colleagues to
tions, the sondeos may be periodically re- share information throughout the organiza-
peated to monitor system changes and tion. His reports and graphs were not prov-
development, as well as to evaluate the ining effective in making his case, so he told
terventions. the staff a story. In 1995, a health care worker
The point here is that the very nature of in Zambia was searching for a treatment for
qualitative inquiry makes it possible to get malaria. He found what he needed at the
into the field quickly to study emerging phe- Web site of the Centers for Disease Control.
196 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

The story communicated what his memos Davies 1996) that involves several steps.
had not: the potential life-and-death impor- First, key program stakeholders and partici-
tance of having criticai knowledge available pants (e.g., farmers in an extension pro-
and easily accessible to any World Bank gram) come to an agreement on which "do-
worker in any remote place in the world. mains of change" to monitor with stories.
In another management book, Managing Second, monthly stories of change written
By Storying Around, David Armstrong (1992) by farmers and field staff are collected.
turns the noun story into a verb, storying, to Third, volunteer reviewers and evaluators
emphasize the direct and active impact of using agreed-on criteria select the "most sig-
constructing stories to influence organiza- nificant stories" during regional and state-
tional values and culture. Shaw, Brown, and wide committee meetings. Last, at the end of
Bromiley (1998) reported how capturing the year a document is produced containing
and using "strategic stories" helped the ali the "winning" stories. This document
multinational 3M company improve their forms the basis for a round-table discussion
business planning both interna lly, for those with "key influentials" and funders of the
involved, and externally, for those to whom project, who then also select the most signifi-
strategic results were reported: "A good cant stories according to their views. "This
story (and a good strategic plan) defines re- approach goes beyond merely capturing
lationships, a sequence of events, cause and and documenting client stories; each story is
effect, and a priority among itemsand those accompanied by the storyteller's interpreta-
elements are likey to be remembered as a complex tion, and after review the stories are also ac-
whole" (p. 42). companied by the reviewer 's interpretation.
Qualitative inquiry can be used to dis- One of the ideas behind the process is that it
cover, capture, present, and preserve the sto- promotes a slow but extensive dialog up and
ries of organizations, programs, communi- down the project hierarchy each month"
ties, and families. Barry Kibel (1999) (Dart 2000). See Exhibit 4.7 for an example of
developed a process for capturing the a "most significant change" story. This story
"success stories" of clients in programs also illustrates a qualitative approach to out-
and aggregating them in a method he comes documentation discussed earlier in
called "results mapping." His approach in- this chapter.
volves an arduous and rigorous coding pro- Cognitive scientists have found that sto-
cess that can be challenging to manage, but ries are more memorable and better support
the fundamental idea is that "for programs learning and understanding than nonstory
engaged in healing, transformation, and narratives (Shaw et al. 1998:42). Language
prevention, the best source and form of in- scholar Richard Mitchell (1979) has ob-
formation are client stories" (Kibel 1999:13). served, "Our knowledge is made up of the
Story collecting can be integrated into on- stories we can tell, stories that must be told
going program evaluation, monitoring, and in the language that we know.... Where we
development (organizational learning) pro- can tell no story, we have no knowledge"
cesses. For example, the Institute of Land (p- 34).
and Food Resources at the University of As I noted in the last chapter in discus-
Melbourne in Australia developed a story- sing narrative analysis, which focuses on
based change-monitoring approach called stories as a particular form of qualitative in-
"most significant changes" (Dart et al. 2000; quiry, the language of "storytelling" is less
Particularly Appropriate Qualitative Applications L3. 197

Example of a "Most Signficant Change" Story

Title: 'TH Not Be Miking Cows When l'm 55'

Name of person recordng story: Mark Saddington, dairy farmer
Region: Gippsland
Date of narration: Round 2August 21, 1998
Who was involved: Farmer and family
When did it happen: 1998

What happened? We did the pilot Dairy Business Focus, Program in March, and for the first
time, my wife carne along. We were abfe to look at our farm as a business, not just as a farm.
As a consequence of doing the program, we did a few sums and made a few decisions. We
worked out that we can afford to have her on the farm, and she has ieft her job at the bank.
We will generate enough tncome on the farm to make it more profitable for her to be here. The
kids will benefit from seeing her a lot more, and they won't be in day care. So far this year, this
has made the calving so much easier, we have a joint input, and it has been such a turn around
in my lifestyle. It has been so good.
We actually went to the accountantyesterdaytogetsomefinancial adviceon how weshould be
investing off-farm. He was amazed that what we are doing is treating the farm as a business.
I said: "Now everything that we earn on this farm is going to be put away so that I am not milking
cows when I am 55 years old!"
We have gota debt-reduction program runningforthenext 12 months, but after that the money
will be channeled to off-farm investment. I want to retire young enough to enjoy what we have
been working towards for the last 20 or 30 years. My boss is 77 and is still working on the farm. If I
am that fit when I am his age,! want to be touring around the world.
It has opened up our ves. We are now lookng at off-farm investment, as capita! investment
on-farm is not that great. We are not going to invest in new machinery but are going to invest in
contractors to do any work we can't do. There is no point buying new machinery, as it depreciates.
Instead, we wil! buy shares and invest off the farm. This proves that you can farm on 120 cows, you
don'thavetogetbig, andyoudon't have to milka lot of cows. Itjustdepends whatyou do with your
money. If only we could educate the younger farmers to think ahead instead buying the largest SS
Commodore or the latest dual cab. I followed the same track for a few years until we sat down and
worked out where we were going and where we could be. We made a few mistakes in the past, but
the past is the past.

Feedback from the statewide committee:

9
This story generated lots of discussion. But is it really about profitability or quality of life or
changes in farm practice?
The general consensus was that there needed to be more detail in the story for it to be clearly
about profitability.
B
t is a really powerful story that shows considerable change.

(continued)
198 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

E X H I B I T 4.7 Continued

Feedback from the round-tablc mtina:

The story showed strong evidence of attitudinal change, leading to self-improvement and
goal setting. These people will be high achievers and reap the rewards. They will be good role
models for others who desire similar rewards.
This approach is okay, but it isn't necessarily a prescription for others.
It has some good messages, but it hasn't got ali the answers.
This is a very good example of achieving the goal of the DBF Program: i.e., getting strategic
thinking/planning followed by farmer action.
liked this story as it highlights the diversity in personal goals and ways to get there.

SOURCE: Dart et al. (2000:8-9). Used with permission.

intimidating to nonresearchers than the to go back to the drawing board to search

language of "case studies" or "ethnogra- still deeper for the breakthrough that would
phy," words that sound heavy and aca- lead to a computer that could truly think.
demic. It makes quite a difference when talk- They made revisions and adaptations based
ing to people in a community to say, "We'd on ali their prior failures until finally they
like to hear and record your stories" versus felt more hopeful than ever that they had
"We'd like to do a case study of you." succeeded. The computer program passed
One of my favorite stories was told by an- ali the preliminary tests. They became in-
thropologist Gregory Bateson (1978), the creasingly excited. The moment arrived for
story of attempts to create a computer that the penultimate test. They asked the com-
could think like human beings. Two scien- puter: "Can you think like a human being?"
tists became particularly enthralled with The computer processed the question and
this idea and devoted their lives to the quest. on the screen appeared the answer: "That
Each time they thought they had succeeded, question reminds me of a story." They knew
their efforts failed criticai tests and they had then they had succeeded.

Legislative Auditing and Monitoring

W e've funded nine of these so-called regional transportation

coordinating boards around the state and we have no real idea who's
involved, how well they're workmg, or even what they're really doing. We
need an mdependent perspective on what the hell's going on. Think you can
find out?

Phone inquiry from a state legislator

Particular!}/ Appropriate Qualitative Applications j. 199

On occasion some legislative body or generally, legislative staff members who are
board that has mandated and appropriated particularly interested in the program can be
funds to a new program wants information expected to read such case histories with
about the extent to which the program is op- some care. From a political point of view,
erating in accordance with legislative intent. programs are more likely to be in trouble or
Legislative intent may involve achieving cause trouble for legislators because they
certain outcomes or may focus on whether failed to follow legislative intent in imple-
some mandated delivery specifications are mentation rather than because they failed to
being followed. Sometimes the precise na- achieve desired outcomes. In this case, the
ture of the legislated delivery system is only purpose of legislative monitoring or audit-
vaguely articulated. For example, suchman- ing is to become the eyes and ears of the
dates as "deinstitutionalization," "decen- legislature or board. This means providing
tralization," "services integration," and program descriptions that are sufficiently
"community-based programming" involve detailed and evocative that the legislator or
varied conceptualizations of legislative in- legislative staff can read such descriptions
tent that do not lend themselves easily to and have a good idea of what that program
quantitative specification. Indeed, for the is really like. Having such descriptions en-
evaluator to unilaterally establish some ables legislators to decide whether their own
quantitative measure of deinstitutional- interpretations of legislative intent are being
ization that provides a global, numerical met. The observation of a parent education
summary of the nature of program opera- program reported in the first chapter is an
tions may hide more than it reveals. example of fieldwork done for the purpose
To monitor the complexities of program of monitoring legislative intent. There are
implementation in the delivery of govern- excellent program evaluation units within a
ment services, it can be particularly helpful number of state legislative audit commis-
to decision makers to have detailed case de- sions that use fieldwork to do policy re-
scriptions of how programs are operating search and evaluation for legislators. When
and what they're accomplishing. Such legis- done well, such fieldwork goes beyond sim-
lative monitoring would include descrip- ple compliance audits by using qualitative
tions of program facilities, decision making, methods to get at program processes, imple-
outreach efforts, staff selection procedures, mentation details, and variations in impacts.
the nature of services offered, and descrip- Detailed case histories may also be of con-
tions from clients about the nature and re- siderable service to the programs being
sults of their experiences. An exemplar of monitored because a case history permits
such an effort is an in-depth study of "trans- them to tell their own story in some detail.
portation partnerships" throughout Minne- Thus, where they have deviated from legis-
sota that included case studies in each relative intent, such case histories would be
gion and an extensive cross-case analysis of expected to include information from pro-
patterns and variations (DeCramer 1997). gram administra tors and staff about con-
Busy legislators cannot be expected to straints under which the program operates
read in detail a large number of such histo- and the decisions staff has made that give
ries. Rather, legislators or funders are likely the program its character. At the same time,
to be particularly interested in the case histo- the collection of such case histories through
ries of those programs that are within their site visits and program monitoring need not
own jurisdiction or legislative district. More neglect the possibility for including more
200 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

global statements about statewide patterns cases. Thus, qualitative methods used for
in programs, or even nationwide patterns. It legislative monitoring allow one to docu-
is quite possible through content analysis to ment common patterns across programs as
identify major patterns of program opera- well as unique developments within specific
tions and outcomes for a number of separate programs.

Futuring Applications: Anticipatory Research

and Prospective Policy Analysis

am often surprised, but I am never taken by surprise.

Robert E. Lee, U.S. Civil War general

Lee's statement captures, in essence, eas called Project Future. Part of the devel-
what it means to anticipate the future. Our opment effort involved teams of community
rapidly changing world has increased inter- members interviewing people in their own
est in and need for futures studies. Such communities about their visions, expecta-
work has moved beyond the supermarket tions, hopes, and fears for the future. The
tabloids to become a focus of scholarly in- community teams then analyzed the results
quiry (e.g., Helmer 1983). Much futures and used them to construct alternative fu-
work involves statistical forecasting and ture scenarios for their community. The
computer simulations. There are, however, community next reviewed these scenarios,
also qualitative inquiry futuring research changed them through discussion, and se-
strategies. le cted a future to begin creating. (This is also
One important futuring tool is scenario a form of participatory research discussed
construction (Edmunds 1978; Godet 1987; earlier in this chapter.) Ethnographic futures
Fitzsimmons 1989). Scenarios are narrative research can be part of a community devel-
portrayals of future systems. A scenario can opment process (Domaingue 1989; Textor
be constructed for an organization, a com- 1980).
munity, a farming system, a society, or any The Evaluation Unit of the U.S. General
other unit of interest. Useful scenarios are Accounting Office (GAO) developed meth-
highly descriptxve. One technique for writ- ods for "prospective studies" to help policy-
ing scenarios is to base the scenario on imag- makers anticipate the implications and con-
ined future fieldwork. The scenario is writ- sequences of proposed laws. Prospective
ten as if it were a qualitative study of that studies can include interviewing key knowl-
future system. As such, it would include in- edgeables in a field to solicit the latest and
terview results, observational findings, and best thinking about a proposal, sometimes
detailed descriptions of the phenomenon of f eeding back the findings for a second round
interest. of interviews (a qualitative Delphi tech-
Qualitative methods can also be used to nique). Prospective methods can also in-
gather the data for scenario development. clude doing a synthesis of existing knowl-
The Minnesota Extension Service undertook edge to pull together a research base that
a community development effort in rural ar- will help inform policy making. The GAO
Particular!}/ Appropriate Qualitative Applications j. 201

Somebody sent some

researchers here think-
ing they could get some
interviews from people
interested in the long-
term future. But I get off
work in just 15 minutes.
Are you interested?

handbook Prospective Methods (1989) includes futuring research efforts. Constructing fu-
attention to qualitative and quantitative syn- tures scenarios can also be an effective way
thesis techniques, with particular attention to contextualize evaluation findings and rec-
to the problem of drawing together diverse ommendations, to help decision makers
data sources, including case studies. Indeed, think about the varying future conditions
the Prospective Methods guidebook presents that could affect implementation of alterna-
much of the material in a case study format. tive recommendations (Patton 1988d, 1997a:
Rapid reconnaissance fieldwork can also 328-29).
be used for anticipatory or futuring re- In summary, while most evaluation work
search. Being able to get into the field involves looking at the effectiveness of past
quickly to get a sense of emerging develop- efforts in order to improve the future effec-
ments can be criticai to futures-oriented tiveness of interventions, a futuring per-
needs assessment techniques and forward- spective involves anticipatory research and
looking planning processes. The content forward thinking in order to affect current
analysis techniques of qualitative inquiry, actions toward creating desirable futures.
especially media analysis (Naisbitt and Qualitative inquiry can play a role in both
Aburdene 1990; Merriam and Makower studies of the past and anticipatory research
1988; Naisbitt 1982), are central to many on the future.
202 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Breaking the Routine: find that the collection of qualitative data

Generating New Insights has become a routine and that new insights
can be gained through even the temporary
Being futures oriented includes being use of some quantitative measures. This
open and flexible. Both individuais and or- suggestion for periodically changing meth-
ganizations can easily become trapped in ods derives from a concern with enhancing
routine ways of doing things, including rou- the use of evaluation research. Given the
tine ways of thinking about and conducting ease with which human beings and social
evaluations or engaging in research. Thus, I systems settle into inertia and routine, eval-
end this chapter with one final rationale for uators who want their results to make a dif-
using qualitative methods: breaking the rou- ference need to find creative ways to get peo-
tine or "making the familiar strange" ple to deal with the empirical world.
(Erickson 1973:10). Programs and organiza- Exploring methodological variations may
tions that have established ongoing evalua- be one such approach.
tion systems or management information It is also worth noting that evaluators can
approaches may have become lulled into a settle into routines and inertia. Evaluators
routine of producing statistical tables that who have been using the same methods
are no longer studied with any care. Inertia over and over may have lost the cutting
and boredom can seriously reduce the use- edge of their own creativity. Utilization-
fulness of program evaluation results. After focused evaluators need to have at their dis-
program staff or other decision makers have posal a large repertoire of possible data
seen the same statistical results used in the collection techniques and approaches. Eval-
same kinds of statistical tables year after uators can be more helpful to programs if
year, those results can begin to have a numb- they themselves are staying alert to the
ing effect. Even though the implications of many possibilities available for looking at
those results may vary somewhat from year the world. Indeed, a change in methods may
to year, the very format used to report the do as much or more to reenergize the evalua-
data can reduce the impact of the reports. tor as it does to renew the program evalua-
Mao Tse-tung commented on the tention process.
dency of human beings to settle into numb-
ing routines when he said that a revolution is
! Summary: A Vision of the
needed in any system every 20 years. Revo-
Utility of Qualitative Methods
lutions in the collection of evaluation data
may be needed much more often. One such As I was completing this book, my local cof-
revolution may be to introduce a totally new fee shop went out of business. It had been
approach to evaluation simply for the pur- open only two years, having been part of a
pose of attracting renewed attention to the surge of new coffee shops that sprang up
evaluation process. At the time, changing throughout the United States as Americans
the method may produce new insights or at fell in love with cappuccino. This particular
least force people to deal with the old in- coffee shop was unique because it was also
sights in a new way. a biker hangout. That is, it catered to aficio-
Of course, collection of qualitative data nados of large Harley-Davidson motorcy-
can also become routine. Programs based on cles affectionately called "Hawgs." The
a humanistic philosophy and/or programs owners, Scott and Connie, a young hus-
with an emphasis on individualization may band-and-wife team, were also bikers. Be-
Particular!}/ Appropriate Qualitative Applications j. 203

cause the coffee shop was only a block from tive methods, especially ethnography, "be-
my office, I had become a regular, despite cause it provides a window onto the ways
my lacking the appropriate leather attire consumers interact with products in their
and loud steel machine. One morning, Con- everyday lives" (p. 377). Designers, in devel-
nie mentioned that she had decided to have oping innovative products and services,
laser surgery to correct her nearsighted vi- must be concerned with satisfying the needs
sion. I left for a trip wishing her a positive of users of their products. Qualitative meth-
outcome from the surgery. When I returned ods, including extensive use of videotape to
two weeks later the coffee shop was closed capture people using products and services,
and I found a barely legible, handwritten offer designers insights into the cultural and
note on the door: environmental contexts within which real
consumers use real products. Exhibit 4.9 of-
Closed indefinitelyvision problem fers examples of Intemet-based resources
that can help you monitor new develop-
I made inquiries at a nearby gasoline sta- ments in qualitative applications.
tion, but ali I learned was that the shop had The emphasis in this chapter on practical
closed very suddenly without notice. Some and useful applications stands in contrast to
three weeks later, I happened to see Scott the philosophical and theoretical focus of
riding his motorcycle on the street and the previous chapter. Taken together, these
waved him over to ask how Connie was do- two chapters demonstrate the importance of
ing. He said she was fine. What about the qualitative inquiry to both social science the-
sign on the coffee shop door? That had noth- ory and practice-oriented inquiry, especially
ing to do with Connie's surgery, he ex- in evaluation and organizational develop-
plained. "I just couldn't vision myself serv- ment.
ing coffee the rest of my life." Practical applications of qualitative meth-
This chapter has been aimed at helping ods emerge from the power of observation,
you decide if you can envision yourself in- openness to what the world has to teach, and
terviewing people, doing fieldwork, con- inductive analysis to make sense out of the
structing case studies, and otherwise using world's lessons. While there are elegant
qualitative methods in practical applica- philosophical rationales and theoretical un-
tions. Exhibit 4.8 lists the applications of- derpinnings to qualitative inquiry, the prac-
fered in this chapter, situations and ques- tical applications come down to a few very
tions for which qualitative methods are basic and simple ideas: Pay attention, listen
particularly appropriate. These are by no and watch, be open, think about what you
means exhaustive of the possibilities for ap- hear and see, document systeinatically
plying qualitative approaches, but they sug- (memory is selective and unreliable), and
gest the wide range of arenas in which quali- apply what you learn.
tative methods can be and are being used. The story of the invention of modern run-
New applications of qualitative methods ning shoes illustrates these principies and
continue to emerge as people in a variety of provides a helpful metaphor to close this
endeavors discover the value of in-depth, chapter. The design of sneakers varied little
open-ended inquiry into people's perspec- until the 1960s when competitive runners
tives and experiences. For example, Wasson began to turn to lighter-weight shoes. Re-
(2000) has reported that members of the de- ducing the weight of shoes clearly improved
sign profession have begun using qualita- performance, but problems of traction re-
204 LEI. CONCEPTUAL ISSUES IN QUALITATIVE INQUIRY

Qualitative Inquiry Applications: Summary Checkst of

Particularly Appropriate Uses of Qualitative Methods

Studies Focusing on Quality

Understanding and illuminating quality
Quality assurance
Evaluation Applications
Outcomes evaluation
Evaluating individualized outcomes
Process studies
Implementation evaluation
Logic models and theories of action
Evaluability assessments
Comparing programs: Focus on diversity
Prevention evaluation
Documenting development over time and investigating system changes
Evaluation Models
Goal-free evaluation
Transaction models: Responsive and illuminative evaluation
Connoisseurship studies
Utilization-focused evaluation
Interactive and Participatory Applications
Personalizing and humanizing evaluation
Harmonizing program and evaluation values
Developmental applications: Action research, action learning, reflective practice, and
learning organizations
Appreciative inquiry
Participatory research and evaluation: Valuing and facilitating collaboration
Supporting democratic dialogue and deliberation
Supporting democracy through process use: Helping the citizenry weigh evidence and think
evaluatively
Special Applications
Unobtrusive measures
State-of-the-art considerations: Lack of proven quantitative instrumentation
Confirmatory and elucidating research: Adding depth, detail, and meaning to quantitative
analyses
Rapid reconnaissance
Capturing and communicating stories
Legislative monitoring and auditing
Futuring applications: Anticipatory research and prospective policy analysis
Breaking the routine: Generating new insights

mained. A running coach, Bill Bowerman, One morning while he was making waf-
went into the sneaker business in 1962. He fles, he had an idea. He heated a piece of rub-
paid close attention to the interest in ber in the waffle iron to produce the first
lighter-weight shoes and the problems of waffle-shaped sole pattern that became the
traction. world standard for running shoes (Panati
Particular!}/ Appropriate Qualitative Applications j. 205

Sample Internet E-mail Discussion Groups (listservs) and

EXH1BIT 4.9 Sites Relevant to Qualitative Applications and Practice

1. EVALTALK@bama,ua.edu: American Evaluation Association (AEA) Discussion List; to

subscribe, send this message to [email protected]: subscribe evaltalk ourname
AEA home page with links to evaluation organizations, training programs, and
Internet resources: www.evai.org

2. [email protected]: Action Research Mailing List; to subscribe, send this message to

[email protected]: subscribe ARLIST-L
LASTNAME

3. [email protected]: Action research methodology network; to subscribe, send this

message to [email protected]: subscribe ARMNET-L firstname lastname

4. Organizational development Web site: www.mnodn.org

5. Organizational Development Network: www.ODNetwork.org

6. Evaluation Center at Western Michigan University: www.wmich.edu/evalctr/

7. World Bank Evaluation Unit Web site: www.wbln0018.worldbank.org/wbies/wbievalu.nsf

8. Empowerment Evaluation:
www.stanord.edu/~davidf/EmpowermentWorkshopCSAP/sld001.htm

9. [email protected]: International Visual Sociology Association; to subscribe,

send this message to [email protected]: subscribe ivsa ourname

NOTE: These sites and subscription details may change, noris this list exhaustive.This list ismcant to besug-
gestive of the qualitative analysis resources available through the internet. For other Internet resources, see
Chapter 1, Exhibit 1.5; Chapter 3, Exhibit 3.7; and Chapter 8, Exhibt 8.3.

1987:298-99). Subsequently, engineers and fieldwork and highly skilled interviewing,

computers would be used to design and test the essence of qualitative inquiry is paying
the best waffle patterns for different athletic attention, being open to what the world has
purposes. But the initial discovery came to show us, and thinking about what it
from Bowerman's paying attention, being means. C. Wright Mills (1961) quoted Nobel
open, making connections, drawing on per- Prize winning physicist Percy Bridgman in
sonal experience, getting a feel for what was this regard:
possible, exploration, documenting his ini-
tial results, and applying what he learned. There is no scientific method as such, but the
While later chapters will present in detail vital feature of the scientists procedures has
techniques for increasing the validity and re- been merely to do his utmost with his mind, no
liability of qualitative data through rigorous holds barred. (p. 58)
PART 2

Qualitative Designs
and Data Collection

Always be suspicious of data collection that goes according to plan.

Research subjects have also been known to be people.
The evaluator 's scientific observation is some person's real-life experience.
Respect for the latter must precede respect for the former.
Total trust and complete skepticism are twin losers in the field. Ali things in
moderation, especially trust and skepticism.
Evaluators are presumed guilty until proven innocent.
Make sure when you yield to temptation in the field that it appears to have
something to do with what you are studying.
A fieldworker should be able to sweep the floor, carry out the garbage, carry in
the laundry, cook for large groups, go without food and sleep, read and write by
candlelight, see in the dark, see in the light, cooper ate without offending, sup-
press sarcastic remarks, smile to expressboth pain and hurt, experiencebothpain
and hurt, spend time alone, respond to orders, take sides, stay neutral, take risks,
avoid harm, be confused, seem confused, care terribly, become attached to noth-
ing The nine-to-five set need not apply.
Always carry extra batteries and getaway money.
From Halcolm's Fieldivork Lazus

!J. 207
Designing Qualitative Studies

~Uhe F i ^ s t S v a l u a t i o n

The young people gathered around Halcolm. "Tell us again, Teacher of Many
Things, about the first evaluation."
"The first evaluation," he began, "was conducted a long, long time ago, in an-
cient Babylon when Nebuchadnezzar was king. Nebuchadnezzar had just con-
quered Jerusalem in the third year of the reign of Jehoiakim, King of Judah. Now
Nebuchadnezzar was a shrewd ruler. He decided to bring carefully selected
children of Israel into the palace for special training so that they might be more
easily integrated into Chaldean culture. This special program was the forerun-
ner of the compensatory education programs that would become so popular in
the 20 th century. The three-year program was royally funded with special allo-
cations and scholarships provided by Nebuchadnezzar. The ancient text from
the Great Book records that
the king spake unto Ashpenaz the master of his eunuchs that he shoud bring cer-
tain of the children of Israel, and of the king's seed, and of the princes; children in
whom was no blemish, but well-favored and skillful in ali wisdom, and cunning in
knowledge, and understanding science, and such as had ability m them to stand in
the king's palace, and whom they might teach the learning and the tongue of the
Chaldeans.
And the king appointed them a daily provision of the king's meat, and of the
wine which he drank; so nourishing them three years, that at the end thereof they
might stand before the king. (Daniel 1:3-5)

!5. 209
210 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

"Now this program had scarcely been established when the program director,
Ashpenaz, who happened also to be prince of the eunuchs, found himself faced
with a student rebellion led by a radical named Daniel, who decided for religious
reasons that he would not consume the king's meat and wine. This created a seri-
ous problem for the director. If Daniel and his coconspirators did not eat their dor-
mitory food, they might fare poorly in the program and endanger not only future
program funding but also the program director's head! The Great Book says:

But Daniel purposed in his heart that he would not defile himself with the portion of
the king's meat, nor with the wine which he drank; therefore he requested of the
prince of the eunuchs that he might not defile himself.
And the prince of the eunuchs said unto Daniel, I fear my lord the king, who hath
appointed your meat and your drink; for why should he see your faces worse liking
than the children which are of your sort? Then shall ye make me endanger my head to
the king. (Daniel 1:8,10)

"At this point, Daniel proposed history's first educational experiment and pro-
gram evaluation. He and three friends (Hananiah, Mishael, and Azariah) would
be placed on a strict vegetarian diet for ten days, while other students continued
on the king's rich diet of meat and wine. At the end of ten days the program direc-
tor would inspect the treatment group for any signs of physical deterioration and
judge the productivity of Daniel's alternative diet plan. Daniel proposed the ex-
periment thusly:

Prove thy servants, I beseech thee, ten days; and let them give us pulse to eat, and wa-
ter to drink. Then let our countenances be looked upon before thee, and the counte-
nance of the children that eat of the portion of the king's meat: and as thou seest, deal
with thy servants.
Sohe consented to them in this matter, and proved them ten days. (Daniel 1:12-14)

"During the ten days of waiting Ashpenaz had a terrible time. He couldn't
sleep, he had no appetite, and he had trouble working because he was preoccu-
pied worrying about how the evaluation would turn out. He had a lot at stake. Be-
sides, in those days they hadn't quite worked out the proper division of labor so he
had to play the roles of both program director and evaluator. You s e e . . . . "
The young listeners interrupted Halcolm. They sensed that he was about to
launch into a sermon on the origins of the division of labor when they still wanted
to hear the end of the story about the origins of evaluation. "How did it turn out?"
they asked. "Did Daniel end up looking better or worse from the new diet? Did
Ashpenaz lose his head?"
"Patience, patience," Halcolm pleaded. "Ashpenaz had no reason to worry.
The results were quite amazing. The Great Book says that

at the end of ten days their countenances appeared fairer and fatter in flesh than ali the
children which did eat the portion of the king's meat.
Designing Qualitative Studies !fj. 211

Thus, Melzar took away the portion of their meat, and the wine that they should
drink; and gave them pulse.
As for these four children, God gave them knowledge and skill in ali learning and
wisdom; and Daniel had understanding in ali visions and dreams. Now at the end of
the days that the king had said he should bring them in, then the prince of the eunuchs
brought them in before Nebuchadnezzar. And in ali matters of wisdom and under-
standing, that the king inquired of them, he found them ten times better than ali the
magicians and astrologers that were in ali his realm. (Daniel 1:15-18,20)

"And that, my children, is the story of the first evaluation. Those were the good
old days when evaluations really got used. Made quite a difference to Ashpenaz
and Daniel. Now off with youand see if you can do as well."

From Halcolm's Evaluation Histories

A Meta-Evaluation other things, aside from a change in diet,

either group was involved in that might
have explained the outcomes observed.
A meta-evaluation is an evaluation of an
evaluation. A great deal can be learned 5. Possibility of interaction effects between
about evaluation designs by conducting a the diet and the students/ belief system
meta-evaluation of history's first program (i.e., potential Hawthorne and halo ef-
evaluation. Let us imagine a panei of experts fects).
conducting a rigorous critique of this evalu-
ation of Babylon's compensatory education 6. Outcome criteria vague: Just what is
program for Israeli students: "countenance"?

7. Outcome measurement poorly opera-

1. Small sample size (N = 4). tionalized and nonstandardized.

2. Selectivity bias because recruitment into 8. Single observer with deep personal in-
the program was done by "creaming," volvement in the program introduces
that is, only the best prospects among possibility of selective perception and
the children of Israel were brought into bias in the observations.
the program.
9. Validity and reliability data are not re-
3. Sampling bias because students were ported for the instruments used to mea-
self-selected into the treatment group sure the final, summative outcome ("he
(diet of pulse and water). found them ten times better than ali the
magicians and astrologers").
4. Failure to clearly specify and control the
nature of the treatment, thus allowing 10. Possible reactive effects from the stu-
for the possibility of treatment contami- dents' knowledge that they were being
nation because we don't know what evaluated.
212 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

Despite ali of these threats to internai va- The information he needed concerned the
lidity, not to mention externai validity, the consequences of that specific change and
information generated by the evaluations only that specific change. He showed no in-
appears to have been used. The 10-day for- terest in generalizing the results beyond
ma tive evaluation was used to make a major those four students, and he showed no inter-
decision about the program, namely, to est in convmcing others that the measures he
change the diet for Daniel and his friends. made were valid and reliable. Only he and
The end-of-program summative evaluation Daniel had to trust the measures used, and
conducted by the king was used to judge the so data collection (observation of counte-
program a success. (Daniel did place first in nance) was done in such a way as to be
his class.) Indeed, it would be difficult to meaningful and credible to the primary in-
find a more exemplary model for the uses of tended evaluation users, namely, Ashpenaz
evaluation in making educational policy de- and Daniel. If any bias existed in his obser-
cisions than this "first evaluation" con- vations, given what he had at stake, the bias
ducted under the auspices of Nebuchad- would have operated against a demonstra-
nezzar so many years ago. This case study is tion of positive outcomes rather than in fa-
an exemplar of evaluation research having vor of such outcomes.
an immediate, decisive, and lasting impact While there are hints of whimsy in the
on an educational program. Modern evalua- suggestion that this first evaluation was ex-
tion researchers, flailing away in seemmgly emplary, I do not mean to be completely fa-
futile efforts to affect contemporary govern- cetious. I am serious in suggesting that the
mental decisions, can be forgiven a certain Babylonian example is an exemplar of utili-
nostalgia for the "good old days" in Babylon zation-focused evaluation. It contains and il-
when evaluation really made a difference. lustrates ali the factors modem evaluation
But should the results have been used? researchers have verified as criticai from
Given the apparent weakness of the evalua- studies of utilization (Patton 1997a). The de-
tion design, was it appropriate to make a cision makers who were to use findings gen-
major program decision on the basis of data erated by the evaluation were clearly identi-
generated by such a seemingly weak re- fied and deeply involved in every stage of
search design? the evaluation process. The evaluation ques-
I would argue that not only was use im- tion was carefully focused on needed infor-
pressive in this case, it was also appropriate mation that could be used in the making of a
because the research design was exemplary. specific decision. The evaluation methods
Yes, exemplary, because the study was set up and design were appropriately matched to
in such a way as to provide precisely the in- the evaluation question. The results were
formation needed by the program director understandable, credible, and relevant.
to make the decision he needed to make. Feedback was immediate and utilization
Certainly, it is a poor research design to was decisive. Few modern evaluations can
study the relationship between nutrition meet the high standards for evaluation set
and educational achievement. It is even a by Ashpenaz and Daniel more than 3,000
poor design to decide if ali students should years ago.
be placed on a vegetarian diet. But those This chapter discusses some ways in
were not the issues. The question the direc- which research designs can be appropri-
tor faced was whether to place four specific ately matched to evaluation questions in an
students on a special diet at their request. attempt to emulate the exemplary match be-
Designing Qualitative Studies !fj. 213

Observing countenance

tween evaluation problem and research de- alternative purposes along a continuum
sign achieved in the Babylonian evaluation. from theory to action:
As with previous chapters, I shall emphasize
the importance of being both strategic and 1. Basic research: To contribute to funda-
practical in creating evaluation designs. Be- mental knowledge and theory
ing strategic begins with being clear about 2. Applied research: To illuminate a societal
the purpose of the intended research or eval- concern
uation.
3. Summative evaluation: To determine pro-
gram effectiveness
l Clarity About Purpose: 4. Formative evaluation: To improve a pro-
A Typology gram
Purpose is the controlling force in research. 5. Action research: To solve a specific prob-
Decisions about design, measurement, lem
analysis, and reporting ali flow from pur-
pose. Therefore, the first step in a research Basic and applied researchers publish in
process is getting clear about purpose. The scholarly journals, where their audience is
centrality of purpose in making methods other researchers who will judge their con-
decisions becomes evident from examining tributions using disciplinary standards of
214 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

rigor, validity, and theoretical import. In Did it accomplish his objectives? Should it
contrast, evaluators and action researchers be continued? Could the outcomes he ob-
publish reports for specific stakeholders served be attributed to the program? This is
who will use the results to make decisions, the kind of research we have come to call
improve programs, and solve problems. summative evaluationsumming up judg-
Standards for judging quality vary ments about a program to make a major de-
among these five different types of research. cision about its value, whether it should be
Expectations and audiences are different. continued, and whether the demonstrated
Reporting and dissemination approaches model can or should be generalized to and
are different. Because of these differences, replicated for other participants or in other
the researcher mustbe clear at the beginning places.
about which purpose has priority. No single Now imagine that researchers from the
study can serve ali of these different pur- University of Babylon wanted to study the
poses and audiences equally well. With clar- diet as a manifestation of culture in order to
ity about purpose and primary audience, the develop a theory about the role of diet in
researcher can go on to make specific design, transmitting culture. Their sample, their
data-gathering, and analysis decisions to data collection, their questions, the duration
meet the priority purpose and address the of fieldwork, and their presentation of re-
intended audience. sults would ali be quite different from the
In the Babylonian example, the purpose formative evaluation undertaken by Ash-
was simply to find out if a vegetarian diet penaz and Daniel. The university study
would negatively affect the healthy appear- would have taken much longer than 10 days
ances (countenances) of four participants and might have yielded empirical general-
not zvhy their countenances appeared izations and contributions to theory, yet
healthy or not (a causai question), but would not have helped Ashpenaz make his
luhether the dietary change would affect simple decision. On the other hand, we
countenance (a descriptive question). The might surmise that University of Babylon
design, therefore, was appropriately simple scholars would have scoffed at an evalua-
to yield descriptive data for the purpose of tion done in 10 days, even a formative one.
making a minor program adjustment. No Different purposes. Different criteria for
contribution to general knowledge. No test- judging the research contribution. Different
ing or development of theory. No general- methods. Different audiences. Different
izations. No scholarly publication. No elab- kinds of research.
orate report on methods. Just find out what These are examples of how purpose can
would happen to inform a single decision vary. In the next section, I shall present a
about a possible program change. The par- more formal framework for distinguishing
ticipants in the program were involved in these five different research purposes and
the study; indeed, the idea of putting the diet examine in more depth the implications of
to an empirical test originated with Daniel. varying purposes for making design deci-
In short, we have a very nice example of sim- sions. Previous chapters have presented the
ple formative evaluation. nature and strategies of qualitative inquiry,
The king's examination of program par- philosophical and theoretical foundations,
ticipants at the end of three years was quite and practical applications. In effect, the
different. We might infer that the king was reader has been presented with a large ar-
judging the overall value of the program. ray of options, alternatives, and variations.
Designing Qualitative Studies !fj. 215

How does one sort it ali out to decide what work to generate new theories or test exist-
to do in a specific study? The answer is to get ing theories. Doctoral students are typically
clear about purpose. The framework that expected to make theoretical contributions
follows is meant to facilitate achieving this in their dissertations. Theories encapsulate
necessary clarity about purpose while also the knowledge of a discipline.
illustrating how one can organize a mass of Basic researchers are interested in formu-
observations into some coherent typology lating and testing theoretical constructs and
a major analytical tool of qualitative in- propositions that ideally generalize across
quiry. The sections that follow examine each time and space. The most powerful kinds of
type of research: basic research, applied re- findings in basic science are those findings
search, summative evaluation research, for- that are universal, such as Boyle's law in
mative evaluation, and action research. physics that the volume of a gas at constant
temperature varies inversely with the pres-
Basic Research sure exerted on it. Basic researchers, then,
are searching for fundamental patterns of
The purpose of basic research is knowl- the uni verse, the earth, nature, society, and
edge for the sake of knowledge. Research- human beings. For example, biologists have
ers engaged in basic research want to under- discovered that "changes in DNA are inher-
stand how the world operates. They are in- ited, but changes in proteins (specifically, in
terested in investigating a phenomenon to their amino acid sequence) are n o t . . . , per-
get at the nature of reality with regard to that haps the only universal truth biologists
phenomenon. The basic researcher 's pur- have" (Smith 2000:43). Social science, to
pose is to understand and explain. date, is markedly short of "universal truths."
Basic researchers typically work within Nevertheless, generalizations across time and
specific disciplines, such as physics, biology, space remain the Holy Grail of basic re-
psychology, economics, geography, and so- search and theory.
ciology. The questions and probleins they The findings of basic research are pub-
study emerge from traditions within those lished in scholarly books, journals, and dis-
disciplines. Each discipline is organized sertations. Each discipline has its own tradi-
around attention to basic questions, and the tions, norms, and rules for deciding what
research within that discipline derives from constitutes valid research in that discipline.
concem about those basic questions. Exhibit To be published in the major journals of any
5.1 presents examples of fundamental ques- particular discipline, scientists must engage
tions in several disciplines. in the kind of research that is valued by the
The fundamental questions undergirding researchers in that disciplinary tradition.
each discipline flow from the basic concerns Chapter 3 reviewed theoretical traditions
and traditions of that discipline. Researchers closely associated with qualitative inquiry,
working within any specific disciplinary tra- for example, ethnography and phenomenol-
dition strive to make a contribution to ogy. Qualitative inquiry also contributes to
knowledge in that discipline and thereby basic research through inductive theory de-
contribute to answering the fundamental velopment, a prominent example being the
questions of the discipline. The most presti- "grounded theory" approach of Glaser and
gious contribution to knowledge takes the Strauss (1967), essentially an inductive strat-
form of a theory that explains the phenome- egy for generating and confirming theory
non under investigation. Basic researchers that emerges from close involvement and di-
216 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

lUIBH Fundamental Disciplinary Questions

Anthropology What is the nature of culture?

How does culture emerge?
How is it transmitted?
What are the functions of culture?

Psychology Why do individuais behave as they do?

How do human beings behave, think, feel, and know?
What is normal and abnormal in human development and behavior?

Sociology What holds groups and societies together?

How do various forms of social organization emerge and what are their
functions?
What are the structures and processes of human social organizations?

Political science What is the nature of power?

How is power organized, created, distributed, and used?

Economics How do societies and groups generate and distribute scarce resources?
How are goods and services produced and distributed?
What is the nature of wealth?

Geography What is the nature of and variations in the earth's surface and atmosphere?
How do various forms of life emerge in and relate to variations in the earth?
What is the relationship between the physical characteristics of an area
and the activities that take place in that area?

Biology What is the nature of life?

What are variations in the forms of life?
How have life forms emerged and how do they change?

rect contact with the empirical world. Basic such as those in rural Mxico, typically inno-
qualitative research typically requires a rela- vate and develop specialties in an attempt to
tively lengthy and intensive period of field- establish a niche for themselves in a complex
work. The rigor of field techniques will be economic environment. Chibnik's (2000) ba-
subject to peer review. Particular attention sic research on commercial woodcarving in
must be given to the accuracy, validity, and Oaxaca has led to the theory that such mar-
integrity of the results. ket segmentation resembles the later stages
An example of interdisciplinary theory of product life cycles described in the busi-
development comes from work in basic eco- ness literature and is somewhat analogous
nomic anthropology studying craft com- to the proliferation of equilibrium species in
mercialization and the product differentia- mature or climax stages of ecological succes-
tion that ordinarily accompanies increased sions. Chibnik examined both market de-
craft sales. Artisans in emerging markets, mands and the initiative of artisans and
Designing Qualitative Studies !fj. 217

found that local artisans do not have total on environmental studies often involves re-
freedom in their attempts to create market searchers from a number of disciplines. In
niches since they are restricted by their abili- agricultural research, the field of integrated
ties and the labor and capital they can mobi- pest management (IPM) includes research-
lize. This is a classic example of interdisci- ers from entomology, agronomy, agricul-
plinary theory generation and testing tural economics, and horticulture. Fields of
bridging economics, ethnology, and ecology. interdisciplinary research in the social sci-
ences include gerontology, criminal justice
Applied Research studies, women's studies, and family re-
search. Exhibit 5.2 offers examples of ap-
Applied researchers work on human and plied interdisciplinary research questions for
societal problems. In the example just cited, economic anthropology, social psychology,
had Chibnik examined the problem of creat- political geography, and educational and or-
ing new markets for rural artisans and of- ganizational development. Notice the dif-
fered possible solutions for increased mar- ference between these questions and the
keting, the work would have constituted kinds of questions asked by basic research-
applied rather than basic research. The pur- ers in Exhibit 5.1. Applied researchers are
pose of applied research is to contribute trying to understand how to deal with a sig-
knowledge that will help people understand nificant societal problem, while basic re-
the nature of a problem in order to intervene, searchers are trying to understand and ex-
thereby allowing human beings to more ef- plain the basic nature of some phenomenon.
fectively control their environment. While in Applied qualitative researchers are able
basic research the source of questions is the to bring their personal insights and experi-
traditions within a scholarly discipline, in ences into any recommendations that may
applied research the source of questions is in emerge because they get especially close to
the problems and concerns experienced by the problems under study during fieldwork.
people and articulated by policymakers. Audiences for applied research are typically
Applied researchers are often guided by policymakers, directors and managers of in-
the findings, understandings, and explana- tervention-oriented organiza tions, and pro-
tions of basic research. They conduct studies fessionals working on problems. Timelines
that test applications of basic theory and dis- for applied research depend a great deal on
ciplinary knowledge to real-world problems the timeliness and urgency of the problem
and experiences. The results of applied re- being researched. A good example of ap-
search are published in journals that special- plied research is Emerging Drug Problems, a
ize in applied research within the traditions work sponsored by the U.S. General Ac-
of a problem area or a discipline. counting Office (1998) that examined new
Societal concerns have given rise to a vari- street drugs, recent research on addiction,
ety of new fields that are interdisciplinary in and alternatives for public policy.
nature. These emerging fields reflect the In contrast to basic researchers, who ulti-
long-standing crticism by policymakers mately seek to generalize across time and
that universities have departments but soci- space, applied research findings typically
ety has problems. Applied interdisciplinary are limited to a specific time, place, and con-
fields are especially problem oriented rather dition. For example, a researcher studying
than discipline oriented. For example, work the nature of family problems in the 1980s
218 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

E X H I B I T 5.2 Sample Interdisciplinary Applied Research Questions

Applied economic anthropology How can the prosperous economy of an isolated, small minor-
ity group be preserved when that group encounters new com-
petition from the encroaching global economy?
Applied social psychology How can a group within a large organization develop cohe-
sion and identity within the mission and vaues of its parent
structure and culture?
Applied political geography How can people of previously isolated towns, each with its
own history of local governance, come together to share
power and engage in joint decision making at a regional
levei?
Applied educational and How can students from different neighborhoods with varied
organizational development ethnic and racial backgrounds be integrated in a new magnet
school?

would not expect those problems to be the tial solutions, evaluations examine and
same as those experienced by families in the judge the processes and outcomes aimed at
1880s. While the research might include attempted solutions. Evaluators study pro-
making such comparisons, applied re- grams, policies, personnel, organiza tions,
searchers understand that problems emerge and products. Evaluation research can be
within particular time and space bound- conducted on virtually any explicit attempt
aries. to solve problems or bring about planned
change. As illustrated in the Daniel story of
Evaluation Research: history's "first evaluation" that opened this
Summative and Formative chapter, evaluators distinguish two quite
different purposes for evaluation: (1) sum-
Once solutions to problems are identi- mative evaluations that judge overall effec-
fied, policies and programs are designed to tiveness to inform major decisions about
intervene in society and bring about change. whether a program should continue and (2)
It is hoped that the mtervention and changes formative evaluations that aim to improve
will be effective in helpmg to solve prob- programs.
lems. However, the effectiveness of any Summative evaluations serve the pur-
givenhuman intervention is a matter subject pose of rendering an overall judgment about
to study. Thus, the next point on the theory- the effectiveness of a program, policy, or
to-action research continuum is the conduct product for the purpose of saymg that the
of evaluation and policy research to test the evaluand (thing being evaluated) is or is not
effectiveness of specific solutions and hu- effective and, therefore, should or should
man interventions. not be continued, and has or does not have
While applied research seeks to under- the potential of being generalizable to other
stand societal problems and identify poten- situations. A summative decision implies a
Designing Qualitative Studies !fj. 219

summing-up judgment or a summit (from tion) funding to more permanent funding,

the mountaintop) decision, for example, to or it may lead to program or policy termina-
expand a pilot program to new sites or move tion. Summative evaluations seldom rely
it from temporary (pilot test or demonstra- entirely, or even primarily, on qualitative
220 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

data and naturalistic inquiry because of de- evaluation takes place. Formative evalua-
cision makers' interest in measuring stan- tions rely heavily on process studies,
dardized outcomes, having controlled com- implementation evaluations, case studies,
parisons, and making judgments about and evaluability assessments (see Chapter
effectiveness from relatively larger samples 4). Formative evaluations often rely heavily,
with statistical pre-post and follow-up re- even primarily, on qualitative methods.
sults. Qualitative data in summative evalua- Findings are context specific.
tions typically add depth, detail, andnuance Although formative and summative re-
to quantitative findings, rendering insights main the most basic and classic distinctions
through illuminative case studies and exam- in evaluation, other evaluation purposes
ining individualized outcomes and issues of have emerged in recent years in "a world
quality or excellenceapplications dis- larger than formative and summative"
cussed in Chapter 4. Harkreader and Henry (Patton 1996b). New purposes include ongo-
(2000) have provided an excellent discus- ing "developmental evaluation" for pro-
sion of the challenges of rendering sum- gram and organizational development and
mative judgments about merit and worth; learning (Patton 1994; Preskill and Torres
they use as their example comparative 1999); empowering local groups through
quantitative performance data from Gergia evaluation partcipation (Fetterman 2000a;
schools to assess a democratic reform initia- Patton 1997b); and using the processes of
tive. Fetterman (2000b) shows how qualita- evaluation (process use) to build staff capac-
tive data can be the primary basis for a ity for data-based decision making and con-
summative evaluation. His evaluation of tinuous improvement (Patton 1997a: 87-113,
STEP, a 12-month teacher education pro- 1998). For our analysis here, however, these
gram in the Stanford University School of and related approaches to evaluation share
Education, included fieldwork immersion the general purpose of improvement and
in the program, open-ended interviews with can be included within the broad category
ali students, focus groups, observatons of of formative research along our theory-to-
classrooms, interviews with faculty, digital action continuum. In addition, some evalua-
photography of classroom activities, and tion studies are now designed to generate
qualitative analysis of curricular materiais, generalizable knowledge about effective
as well as a variety of surveys and outcome practices across different projects or pro-
measures. The summative evaluations of a grams based on cluster evaluations, lessons
democratic reform initiative m Gergia and learned, "better" practices, and meta-analy-
of Stanford's STEP program both followed ses (Patton 1997a:70-75). This knowledge-
and built on extensive formative evalua- generating approach to evaluation research,
tion work, the purpose of which we now to the extent that it aims to discover general
examine. principies of effective practice rather than
Formative evaluations, in contrast to render judgment about the merit or worth
summative ones, serve the purpose of im- of a specific intervention, falls roughly
proving a specific program, policy, group of within the category "applied research" in
staff (in a personnel evaluation), or product. this theory-to-action continuum. However,
Formative evaluations aim at forming (shap- the emergence and increased importance of
ing) the thing being studied. No attempt is knowledge-generating evaluations illus-
made in a formative evaluation to generalize trate why these five categories (basic, ap-
findings beyond the setting in which the plied, summative, formative, and action re-
Designing Qualitative Studies !fj. 221

search) cannot be thought of as fixed or informal, the people in the situation are of-
exhaustive; rather, this typology provides ten directly involved in gathering the
general guidance to major formations in the information and then studying themselves,
research landscape without charting every and the results are used internally to attack
hill and valley in that varied and complex specific problems within a program, organi-
territory that research has become. zation, or community. While action research
may be used as part of an overall organiza-
tional or community development process,
Action-Oriented, it most typically focuses on specific prob-
Problem-Solving Research lems and issues within the organization or
community rather than on the overall ef-
The final category along the theory-to- fectiveness of an entire program or organi-
action continuum is action research. Action zation. Thus, along this theory-to-action-
research aims at solving specific problems research continuum, action research has the
within a program, orgariization, or commu- narrowest focus.
nity. Action research explicitly and purpose- The findings of formative evaluation and
fully becomes part of the change process by action research are seldom disseminated be-
engaging the people in the program or orga- yond the immediate program or organiza-
nization in studying their own problems in tion within which the study takes place. In
order to solve those problems (Whyte 1989). many instances, there may not even be a full
As a result, the distinction between research written research report. Publication and dis-
and action becomes quite blurred and the re- semination of findings are more likely to be
search methods tend to be less systematic, through briefings, staff discussions, and oral
more informal, and quite specific to the communications. Summaries of findings
problem, people, and organization for and recommendations will be distributed
which the research is undertaken. for discussion, but the formality of the re-
Both formative evaluation and action reporting and the nature of the research publi-
search focus on specific programs at specific cations are quite different from those in ba-
points in time. There is no intention, typi- sic, applied, or even summative evaluation
cally, to generalize beyond those specific set- research.
tings. The difference between formative An example of action research comes
evaluation and action research centers on from a small rural community in the Mid-
the extent to which the research is system- west in which the town board needed to de-
atic, the different kinds of problems studied, cide what to do with a dilapidated building
and the extent to which there is a special role on a public park. They got a high school class
for the researcher as distinct from the people to put together a simple telephone survey to
being researched. solicit ideas about what to do with the build-
In formative evaluation, there is a formal ing. They also conducted a few focus groups
design and the data are collected and/or an- in local churches. The results showed that
alyzed, at least in part, by an evaluator. For- the townspeople preferred to fix up the
mative evaluation focuses on ways of im- building and restore it as a community cen-
proving the effectiveness of a program, a ter rather than tear it down. The action re-
policy, an organization, a product, or a staff search process took about a month. Based on
unit. In action research, by way of contrast, the findings, a local committee was formed
design and data collection tend to be more to seek volunteers and funds for the restora-
222 [fj. QUALITATIVE DESIGNS AND DATA COLLECTION

INACT-A P :?f!AnC u
fd-i. ! ;:i! ' \ .'!' .!":i' !.'i! i' < !'ki i ' kl i!i!!!!! / !<< ! \ l. >;:,! i n!'!! v .Vi r n.n i' ij=.-';:V.'. ! Y n' T I .vryi r* ; "
s:=;r ri//L.':- /5! .V! iiitfSM.ri.1 uitt ti .=. ! J R' =:. i!'N V lli-r I - P V T * V ! I- : . = , V R D - ITI i"R . 1 ' R r v n
F D F F L L L B T ^ N I.!!'di/iVVi ' I . I I / I . I J = ' : I M U P I . ; ! = = I.V: I I V I R hr M R I : Y - : \ ' : T Y S-RS..>.| : I I T !.:! RI '
1

it ! i,'. . .,' !' !1AV i.i !;=i 1! IVfE-g i.:, IVi h t -! > - :i ii r.! .1 i =i i iV: i" rvi vi ! i.' Vi S "s i i ' i V.TI vVi i\'i r.;i - -
!

tVr:fa iJi iVrilif iVi it Ari liJ k; :i::-:.i : ! '.=: i i'i i. \ V v. IVi : i! " mv! f':'i vh i; ;Vi i' i;.Vi , ?[ vij V, i"y;j |rl
:= ! :
!
: ! ILIS!^!!,. : hES ifl! iS. N ' ! ^ II!.' i.==': ,'i - = V"! r.i: i" . iVi ::V ::ir ::vi/: n!,:: ii'|".'IVI." .'.'i r ; cvri !. ri! , M ' -
C R U . ! K P nmh. IVSI' IIIRI :! !:' !J
, ! :::.: !. ;"= . ? TF* > i VI. i\ =: \h i- r W I I.-
: ; ': I! vi , > UV .! - n T I : - I ; T I :RR-iVi;i

fdfiTr P!:^. J iiili/i:! =;!J1:: : i ! iritiiV.'. I.' !>=: i i i i. i 'i I ; : i i Vi iVi i".' s ' i I.;I ytiy i; |i1 \ \ 11 '.-vi T I ' I. i' ri 'i
llllllllllllllllJIJIJjjJJJIl
I'.'I ITV I-:; i.vi i,:. i r :.=.'' W! I : i I !" r '.:-V I i TI

. i:iirtt .IMVi^:/!:! tij: ii:!v . -1 ri ]!;'j::i rllTIitiii:!:! i:!,i' -ii:!!: |! ii i!, i':'-, iV i'
: Y .. s.!;:: i:=i'!.' i '"i =:'i I'" ^iVVVI V-
i:' i ,: :' :! :>!' Ir, ' :!=i : i.vi!'.': 1-i.i.'5 "i i.'.' br!! iVi. .'i i-1,1-
1

... ztt* ":::':!=:T: ! -INLIISIHT!;!: :=::! !: :,-=:.'!:'.';:; I===

.:!!: ! !=:; !=:T!J;!;:! =::; A 1 = :
: !
:
! V :: tu.- i : v i ! ^ u '. I !
ifi: iTiiilVvl^.iVi!^'n":in!e iTi-âii iliî^JiÎkiiiîiTi.i-îvri^,iir r>y}>.i": ii' IVir vi|:V|V:!' IT'.-i;i;it!'.""
!: i'-.!iTf 'ti-! ii!.': jj! = , n r, !; iiMil /j i!.;: in \ ri; =- I>:=_ " L J - P S ' : " ; i ! < m 11.; i I ( ! I |.i.;! i' i'
i;i i y.^, >. j, ' ' i j' j.' j- j j,,, | ii ' I.VftM ..
lllllllllllllllJIJJIIIJ^
I (ws '! .! >:| HÎ !'
! : '::.;!; :::J; !;!
! ! != I: !:: :;=;', : ii- :',- s:a-.s:: i i.'! I <:V hi *iw> i.-; J,* .i U IVI'YI
! !

:!i**i!'i:i i! 1!;=:!; ii!i.i !!:;.;:!: ItIiJ :- amtfrfl .!i i^^iM lIJi rp k& IVi lIi-iiS itn ':> j'i: i vi I 'i! i i'i'!' i !V ii 'Vi i Vi
!! ^Tiiii' iri1!" i-i -i" =i' fVi iririVin P I !=:; i:i mh C A T A M l;^:; =!:" i.. ' i " ii! S: C ;' !'I'
;.''!'!'!: "i' "VVIV.".: :!;; .;=:;! J| I :;':.=. V . ! - : .1'M.

tion, thereby solving the town board's prob- Researchers engaged in inquiry at vari-
lem of what to do with the budingan ex- ous points along the continuum can have
ample of action-oriented, problem-solving very strong opinions and feelings about re-
research. searchers at other points along the contin-
uum, sometimes generating opposing opin-
The Purpose of ions and strong emotions. Basic and applied
Purpose Distinctions researchers, for example, would often dis-
pute even calling formative and action re-
It is important to understand variations search by the name research. The standards
in purpose along this theory-to-ac tion con- that basic researchers apply to what they
tinuum because different purposes typically would consider good research excludes
lead to different ways of conceptualizing even some applied research because it may
problems, different designs, different types not manifest the conceptual clarity and theo-
of data gathering, and different ways of pub- retical rigor in real-world situations that ba-
licizing and disseminating findings. These sic researchers value. Formative and action
are only partly issues of scholarship. Poli- researchers, on the other hand, may attack
tics, paradigms, and values are also part of basic research for being esoteric and irrele-
the landscape. vant.
Designing Qualitative Studies !fj. 223

Debates about the meaningfulness, rigor, those choices for both the kind of research
significance, and relevance of various ap- undertaken and the researcher's status as a
proaches to research are regular features of professional within various social groups.
university life. On the whole, within univer- Exhibit 5.3 summarizes some of the major
sities and among scholars, the status hierar- differences among the different kinds of re-
chy in science attributes the highest status to search.
basic research, secondary status to applied
research, little status to summative evalua- Examples of Types
tion research, and virtually no status to for-
of Research Questions:
mative and action research. The status hier-
A Family Research Example
archy is reversed in real-world settings,
where people with problems attribute the
greatest significance to action and formative To further clarify these distinctions, it
research that can help them solve their prob- may be helpful to take a particular issue and
lems in a timely way and attach the least im- look at how it would be approached for each
por tance to basic research, which they con- type of research. For illustrative purposes,
sider remote and largely irrelevant to what let's examine the different kinds of questions
they are doing on a day-to-day basis. that can be asked about families for different
research purposes. Ali of the research ques-
The distinctions along the continuum are
tions in Exhibit 5.4 focus on families, but the
not only distinctions about purpose and
purpose and focus of each type of research
how one conducts research, but they also in-
are quite different. With clarity about pur-
volve the issue of what one calls what one
pose, it is possible to turn to consideration of
does. In other words, a person conducting
specific design alternatives and strategies.
basic research for the purpose of contribut-
Clarity about purpose helps in making deci-
ing to theory within a discipline may find it
sions about criticai trade-offs in research and
helpful to call that work applied research to
designs, our next topic.
get certain kinds of funding. Summative
evaluation researchers may describe what
they are doing as formative evaluation to
Criticai Trade-Offs
make their work more acceptable to pro-
in Design
gram staff resistant to being studied. On the
other hand, applied researchers may call Purposes, strategies, and trade-offs these
what they are doing basic research to increase themes go together. A discussion of design
its acceptability among scholars. strategies and trade-offs is necessitated by
In short, there are no clear lines dividmg the fact that there are no perfect research
the points along the continuum. Part of what designs. There are always trade-offs.
determines where a particular kind of re- Limited resources, limited time, and limits
search falls along the continuum is how the on the human ability to grasp the complex
researcher describes what is being done and nature of social reality necessitate trade-
its purpose. Different reviewers of the same offs.
piece of research might well use a differ- The very first trade-offs come in framing
ent label to describe it. What is important for the research or evaluation questions to be
our purposes is that researchers understand studied. The problem here is to determine
the implications of these distinctions, the the extent to which it is desirable to study
choices involved, and the implications of one or a few questions in great depth or to
AGI3 S 9 A Typology of Research Purposes

Types of Desired Levei

Research Purpose Focus oFReseorch Desired Results of Generalizador Key Assumptions Publication Mode Standord forJudging

Basic Knowledge as Questions deemed Contribution to Across time and The world is Major refereed Rigor of research,
research an end in itself; important by one's theory space (ideal) patterned; those scholarly journals universality and
discover truth discipline or personal patterns are knowable in one's discipline, verifiability of theory
intellectual interest and explainable. scholarly books
Applied Understand the Questions deemed Contributions to Within as general Human and societal Specialized academic Rigor and theoretical
research nature and sources important by society theories that can be a time and space as probiems can be journals, applied insight into the
of human and used to formulate possible, but clearly understood and research journals problem
societal problems problem-solving limited application soived with within disciplines,
programs and context knowledge. interdisciplinary
interventions problem-focused
journals
Summative Determine Goals of the Judgments and Ali interventions What works one Evaluation reports Generalizability to
evaluation effective ness of intervention generalzation5 with similar goals place under specified for program funders future efforts and to
human interventions about effective types conditions should and policymakers, other programs and
and actions (programs, of interventions and work elsewhere. specialized journals policy issues
policies, personnel, the conditions under
products) which those efforts
are effective
Formative Improve an Strengths and Recommendations Limited to specific People can and will Oral briefings; Usefulness to and
evaluation intervention: weaknesses of the for improvements setting studied use information to conferences; internai actual use by
A program, policy, specific program, improve what they're report; limited intended users in
organization, or policy, product, or doing. circulaton to similar the setting studied
product personnel being programs, other
studied evaluators
Action Solve problems Organization and immediate action; Here and now People in a setting Interpersonal Feelings about the
resea rch in a program, community problems solving problems as can solve problems by interactions among process among
organization, or quickly as possible studying themseives. research participants; research participants,
community informal unpublished feasibility of the
solution generated
Designing Qualitative Studies !fj. 225

Family Research Example: Research Questions Matched to

E X H I B I T 5.4 Research Category

Basic research What are variations in types of famifes and what functions do those
variations serve?
Applied research What is the divorce rate among different kinds of families in the United States
and what explains different rates of divorce among different groups?
Summative What is the overall effectiveness of a publicly funded educational program
evaluation that teaches family members communication skills where the desired
program outcomes are enhanced communications among family members,
a greater sense of satisfaction with family life, effective parenting
practices, and reduced risk of divorce?
Formative How can a program teaching family communications skills be improved? What
evaluation are the program's strengths and weaknesses? What do participants like and
dislike?
Action research A self-study in a particular organization (e.g., church, neighborhood center) to
figure out what activities would be attractive to families with children of dif-
ferent ages to soive the probiem of low participation in family activities.

study many questions but in less depth And always there are fundamental con-
the "boundary probiem" in naturalistic in- straints of time and resources.
quiry (Guba 1978). Once a potential set of in- Converging on focused priorities typi-
quiry questions has been generated, it is nec- cally proves more difficult than the chal-
essary to begin the process of prioritizing lenge of generating potential questions at
those questions to decide which of them the beginning of a study or evaluation. Doc-
ought to be pursued. For example, for an toral students can be especially adept at
evaluation, should ali parts of the program avoiding focus, conceiving instead of
be studied or only certain parts? Should ali sweeping, comprehensive studies that make
clients be interviewed or only some subset the whole world their fieldwork oyster. In
of clients? Should the evaluator aim at de- evaluations, once involved users begin to
scribing ali program processes or only cer- take seriously the notion that they can learn
tain selected processes in depth? Should ali from finding how whether what they think
outcomes be examined or only certain out- is being accomplished by a program is what
comes of particular interest to inform a is really being accomplished, they soon gen-
pending decision? These are questions that erate a long list of things they'd like to find
are discussed and negotiated with intended out. The evaluation facilitator's role is to
users of the evaluation. In basic research, help them move from a rather extensive list
these kinds of questions are resolved by the of potential questions to a much shorter list
nature of the theoretical contribution to be of realistically possible questions and finally
made. In dissertation research, the doctoral to a focused list of essential and necessary
committee provides guidance on focusing. questions.
226 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Review of relevant literature can also attempt to study. In a narrowly focused

bring focus to a study. What is already study, we might select one particular set of
known? Unknown? What are the cutting- interactions and limit our study to those
edge theoretical issues? Yet, reviewing the for example, the social interactions between
literature can present a quandary in qualita- teachers and children. Broadening the scope
tive inquiry because it may bias the re- somewhat, we might decide to look only at
searcher^ thinking and reduce openness to those interactions that occur in the class-
whatever emerges in the field. Thus, some- room, thereby increasing the scope of the
times a literature review may not take place study to include interactions not only be-
until after data collection. Alternatively, the tween teacher and child but also among
literature review may go on simultaneously peers in the classroom and between any vol-
with fieldwork, permitting a creative inter- unteers and visitors to the classroom and the
play among the processes of data collection, children. Broadening the scope of the study
literature review, and researcher introspec- still more, we might decide to look at ali of
tion (Marshall and Rossman 1989:38-40). As the social relationships that children experi-
with other qualitative design issues, ence in schools; in this case we would move
trade-offs appear at every turn, for there are beyond the classroom to look at interactions
decided advantages and disadvantages to with other personnel in the schoolfor ex-
reviewing the literature before, during, or ample, the librarian, school counselors, spe-
after fieldworkor on a continuai basis cial subject teachers, the custodian, and/or
throughout the study. school administrative staff. Broadening the
A specific example of possible variations scope of the study still further, the educators
in focus will illustrate the kinds of trade-offs might decide that it is important to look at
involved in designing a study. Suppose the social relationships children experience
some educators are interested in studying at home as well as at school so as to under-
how a school program affects the social de- stand better how children experience and
velopment of school-age children. They are affected by both settings, so we would in-
want to know how the interactions of chil- clude in our design interactions with par-
dren with others in the school setting con- ents, siblings, and others in the home.
tribute to the development of social skills. Finally, one might look at the social relation-
They believe that those social skills will be ships experienced throughout the full range
different for different children, and they are of societal contacts that children have, in-
not stire of the range of social interactions cluding church, clubs, and even mass media
that may occur, so they are interested in a contacts.
qualitative inquiry that will capture varia- A case could be made for the importance
tions in program experience and relate those and value of any of these approaches, from
experiences to individualized outcomes. the narrowest focus, looking only at student-
What, then, are trade-offs in determining the teacher interactions, to the broadest focus,
final focus? looking at students' full, complex social
We begin with the fact that any given world. Now lefs add the real-world con-
child has social interactions with a great straint of limited resourcessay, $50,000
many people. The first problem in focusing, and three monthsto conduct the study. At
then, is to determine how much of the social some levei, any of these research endeavors
reality experienced by children we should could be undertaken for $50,000. But it be-
Designing Qualitative Studies !fj. 227

comes dear, immediately, that there are cally produce a wealth of detailed data
trade-offs between breadth and depth. A about a much smaller number of people and
highly focused study of the student-teacher cases.
relationship could consume our entire bud- However, the breadth versus depth
get but allow us to investigate the issue in trade-off also applies within qualitative de-
great depth. On the other hand, we might at- sign options. Human relations specialists
tempt to look at ali social relationships that tellus that we can never fully understand the
children experience but to look at each of experience of another person. The design
them in a relatively cursory way in order, issue is how much time and effort we are
perhaps, to explore which of those relation- willing to rnvest in trying to increase our un-
ships is primary. (If school relationships derstanding about any single person's expe-
have very little impact on social develop- riences. So, for example, we could look at a
ment in comparison with relationships out- narrow range of experiences for a larger
side the school, policymakers could use that number of people or a broader range of ex-
Information to decide whether the school periences for a smaller number of people.
program ought to be redesigned to have Take the case of interviews. Interviewing
greater impact on social development or, al- with an instrument that provides respon-
ternatively, if the school should forget about dents with largely open-ended stimuli typi-
trying to directly affect social development cally takes a great deal of time. In an educa-
at ali.) The trade-offs involved are the clas- tion study, I developed an open-ended
sic trade-offs between breadth and depth, interview for elementary students consist-
which we now turn to in more depth. ing of 20 questions that included items such
as "What do you like most about school?"
and "What don't you like about school?"
Breadth Versus Depth These interviews took between half an hour
and two hours depending on students' ages
In some ways, the differences between and how articulate they were. It would cer-
quantitative and qualitative methods in- tainly have been possible to have longer in-
volve trade-offs between breadth and depth. terviews. Indeed, Ihave conducted in-depth
Qualitative methods permit inquiry into se- interviews with people that ran 6 to 16 hours
lected issues in great depth with careful at- over a period of a couple of days. On the
tention to detail, context, and nuance; that other hand, it would have been possible to
data collection need not be constrained by ask fewer questions, make the interviews
predetermined analytical categories con- shorter, and probe in less depth.
tributes to the potential breadth of qualita- Or consider another example with a fuller
tive inquiry. Quantitative instruments, on range of possibilities. It is possible to study a
the other hand, ask standardized questions single individual over an extended period of
that limit responses to predetermined cate- timefor example, the study, in depth, of
gories (less breadth and depth). This has the one week in the life of one child. This in-
advantage of making it possible to measure volves gathering detailed information about
the reactions of many respondents to a lim- every occurrence in that child's life and ev-
ited set of questions, thus facilitating com- ery interaction involving that child during
parison and statistical aggregation of the the week of the study. With more focus, we
data. By contrast, qualitative methods typi- might study several children during that
228 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

week but capture fewer events. With a still selected as the unit of analysis when there is
more limited approach, say, a daily half- some important characteristic that separates
hour interview, we could mterview yet a people into groups and when that character-
larger number of children on a smaller num- istic has important implications for the pro-
ber of issues. The extreme case would be to gram.
spend ali of our resources and time asking a A different unit of analysis involves fo-
single question of as many children as we cusing on different parts of a program. Dif-
could interview given time and resource ferent classrooms within a school might be
constraints. studied, making the classrooin the unit of
No rule of thumb exxsts to tell a re- analysis. Outpatient and inpatient programs
searcher precisely how to focus a study. in a medicai facility could be studied. The in-
The extent to which a research or evalua- take part of a program might be studied sep-
tion study is broad or narrow depends on arately from the service delivery part of a
purpose, the resources available, the time program as separate units of analysis. Entire
available, and the interests of those in- programs can become the unit of analysis. In
volved. In brief, these are not choices be- state and na tional programs where there are
tween good and bad but choices among al- a number of local sites, the appropriate unit
ternatives, ali of which have merit. of analysis may be local projects. The analyt-
ical focus in such multisite studies is on vari-
ations among project sites more than on
Units of Analysis variations among individuais within pro-
jects.
A design specifies the unit or units of Different units of analysis are not mutu-
analysis to be studied. Decisions about sam- ally exclusive. However, each unit of analy-
ples, both sample size and sampling strate- sis implies a different kind of data collection,
gies, depend on prior decisions about the a different focus for the analysis of data, and
appropriate unit of analysis to study. Often a different levei at which statements about
individual people, clients, or students are findings and conclusions would be made.
the unit of analysis. This means that the pri- Neighborhoods can be units of analysis or
mary focus of data collection will be on what communities, cities, states, cultures, and
is happening to individuais in a setting and even nations in the case of International pro-
how individuais are affected by the setting. grams.
Individual case studies and variation across One of the strengths of qualitative analy-
individuais would focus the analysis. sis is looking at program units holistically.
Comparing groups of people in a pro- This means doing more than aggregating
gram or across programs involves a differ- data from individuais to get overall program
ent unit of analysis. One may be interested m results. When a program, group, organiza-
comparing demographic groups (males tion, or community is the unit of analysis,
compared with females, Whites compared qualitative methods involve observations
with African Americans) or programmatic and description focused directly on that
groups (dropouts vs. people who complete unit: The program, organization, or commu-
the program, people who do well vs. people nity, not just the individual people, becomes
who do poorly, people who experience the case study focus in those settings.
group therapy vs. people who experience in- Particular events, occurrences, or inci-
dividual therapy). One or more groups are dents may also be the focus of study (unit of
Designing Qualitative Studies !fj. 229

analysis). For example, a quality assurance the kids want to be outside, so that's not an
effort in a health or mental health program effective time to gather data. In African vil-
might focus only on those criticai incidents lages, I was given similar scenarios about
in which a patient fails to receive expected or the difficulties of data collection for every
desirable treatment. A criminal justice eval- month in the annual cycle of the agricultural
uation could focus on violent events or in- season. A particular period of time, then, is
stances in which juveniles run away from both an important context for a study and a
treatment. A cultural study may focus on sampling issue.
celebrations. There are limits to how much one can ap-
Sampling can also involve time period ply logic in trying to calculate ali of the pos-
strategies, for example, continuous and on- sible consequences of sampling options,
going observation versus fixed-interval whether the decision is about which time pe-
sampling in which one treats units of time riods to sample or which activities to ob-
(e.g., 15-minute segments) as the unit of observe. The trick is to keep coming back to the
servation. "The advantage of fixed-interval criterion of usefulness. What data collected
sampling over continuous monitoring are during what time period describing what
that fieldworkers experience less fatigue activities will most likely illuminate the in-
and can collect more information at each quiry? For evaluation, what focus of inquiry
sampling interval than they could on a con- will be most useful? There are no perfect
tinuous observation routine" (Johnson and evaluation designs, only more and less use-
Sackett 1998:315). Time sampling (sampling ful ones.
periods or units of time) can be an especially The key issue in selecting and making
important approach because programs, or- decisions about the appropriate unit of
ganiza tions, and communities may function analysis is to decide what it is you want to
in different ways at different times during be able to say something about at the end of
the year. Of course, in some programs there the study. Do you want to have findings
never seems to be a good time to collect data. about individuais, families, groups, or some
In doing school evaluations in the United other unit of analysis? For scholarly inqui-
States, I've been told by educators to avoid ries, disciplinary traditions provide guid-
collecting data before Halloween because ance about relevant units of analysis. For
the school year is just getting started and evaluations, one has to determine what deci-
the kids and teachers need time to get set- sion makers and primary intended users re-
tled in. But the period between Halloween ally need information about. Do they want
and Thanksgiving is really too short to do findings about the different experiences of
very much, and, then, of course, after individuais in programs, or do they want to
Thanksgiving everybody's getting ready for know about variations in program processes
the holidays, so that's not a typical or conve- at different sites? Or both? Such differences
nient period. It then takes students a few in focus will be criticai to the design but may
weeks after the winter break to get their at- not be easy to determine. A decision maker
tention focused back on school and then the is unlikely to say to the evaluator, "The unit
winter malaise sets in and both teachers and of analysis we want to study is ." The
students become deeply depressed with the evaluator mustbe able to hear the real issues
endlessness of winter (at least in northern involved in the decision maker's questions
climes). Then, of course, once spring hits, at- and translate those issues into the appropri-
tention is focused on the close of school and ate unit of analysis, then check out that
230 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

translation with the intended evaluation us- pling as one of the core distinguishing
ers. strategic themes of qualitative inquiry. The
Exhibit 5.5 presents some alternative next section presents variations in, ratio-
units of analysis. Clarity about the unit of nales for, and the details of how to design a
analysis is needed to select a study sample. study based on a purposeful sample.
In Chapter 2, I identified purposeful sam-

!3. Purposeful S a m p l i n g

the motive exalts the action.

Margaret Preston, 1875

learn a great deal about issues of central

Perhaps nothing better captures the dif-
impor tance to the purpose of the inquiry,
ference between quantitative and qualita-
thus the term purposeful sampling.
tive methods than the different logics that
Studying information-rich cases yields in-
undergird sampling approaches.
sights and in-depth understanding rather
Qualitative inquiry typically focuses in
than empirical generalizations. For exam-
depth on relatively small samples, even sin-
ple, if the purpose of an evaluation is to in-
gle cases (N = 1), selected purposefully. Quan-
crease the effectiveness of a program in
titative methods typically depend on larger
reaching lower-socioeconomic groups,
samples selected randomly. Not only are the
one may learn a great deal more by study-
techniques for sampling different, but the
ing in depth a small number of carefully
very logic of each approach is unique be-
selected poor families than by gathering
cause the purpose of each strategy is differ-
standardized information from a large,
ent.
statistically representative sample of the
The logic and power of random sampling
whole program. Purposeful sampling fo-
derive from statistical probability theory A
cuses on selecting information-rich cases
random and statistically representative
whose study will illuminate the questions
sample permits confident generalization
under study. Purposeful sampling is
from a sample to a larger population. Ran-
sometimes called purposive or judgment
dom sampling also controls for selection
sampling: "In judgment sampling, you de-
bias. The purpose of probability-based ran-
cide the purpose you want inforinants (or
dom sampling is generalization from the
communities) to serve, and you go out to
sample to a population and control of selec-
find some" (Bernard 2000:176). There are
tivity errors.
several different strategies for purpose-
What would be "bias" in statistical sam-
fully selecting information-rich cases. The
pling, and therefore a weakness,becomes in-
logic of each strategy serves a particular
tended focus in qualitative sampling, and
purpose.
therefore a strength. The logic and power of
purposeful sampling lie in selecting informa-
tion-rich cases for study in depth. Informa- 1. Extreme or deviant case sampling. This
tion-rich cases are those from which one can strategy involves selecting cases that are
Designing Qualitative Studies !fj. 231

1
|l(Mg|gM Examples of Units of Analysis for Case Studies and
SBilIslIEHsB Comparisons

People Focused Structure Focused

individuais Projects
Small, informal groups (friends, gangs) Programs
Families Organizations
Units in organizations
Perspective/Worldview Based

People who share a culture

People who share a common experience or perspective, for example, dropouts,
graduates, leaders, parents, Internet listserv participants, survivors
Geoaraphv Focused

Neighborhoods Villages
Cities Farms
States Regions
Countries Markets
Ac ti vi tv Focused

Criticai incidents Time periods

Celebrations Crises
Quality assurance violations Events
Time Based

Particular days, weeks, or months Vacations

Winter break Rainy season
Ramadan Dry season
Full moons School term
A political term of office An election period

NOTE: These are not mutually exclusive categories.

information rich because they are unusual fectly representative of U.S. industry as a
or special in some way, such as outstanding whole . . . [but] a list of companies consid-
successes or notable failures. The influential ered to be innovative and excellent by an in-
study of America's best-run companies, formed group of observers of the business
published as In Search ofExcellence, exempli- scene" (Peters and Waterman 1982:19).
fies the logic of purposeful, extreme group Lisbeth Schorr (1988) used a similar strategy
sampling. This study was based on a sample in studying especially effective programs
of 62 companies "never intended to be perfor families in poverty, published as the
232 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

influential book Within Our Reach. Stephen An excellent applied research example
Covey's (1990) best-selling book The 7 Hab- is Angela Browne's (1987) study When Bat-
its of Highly Effective People is based on a pur- tered Women Kill. She conducted in-depth
poseful, extreme group sampling strategy. studies of the most extreme cases of do-
Studies of leadership have long focused on mestic violence to elucidate the phenome-
identifying the characteristics of highly suc- non of battering and abuse. The extreme
cessful leaders, as in Collins's (2001) case nature of the cases is what renders them so
studies of 11 corporate executives in whom powerful. Browne'sbookis an exemplar of
"extreme personal humility blends para- qualitative inquiry using purposeful sam-
doxically with intense professional will" pling for applied research.
(p. 67), what he calls "Levei 5 leaders," the In evaluation, the logic of extreme case
highest levei in his model. In the early days sampling is that lessons may be learned
of AIDS research when HTV infections al- about unusual conditions or extreme out-
most always resulted in death, a small num- comes that are relevant to improving more
ber of cases of people infected with HIV typical programs. Let's suppose that we
who did not develop AIDS became crucial are interested in studying a national pro-
outlier cases that provided important in- gram with hundreds of local sites. We
sights into directions researchers should know that many programs are operating
take in combating AIDS. reasonably well, even quite well, and that
Sometimes cases of dramatic failure offer other programs verge on being disasters.
powerful lessons. The legendary UCLAbas- We also know that most programs are do-
ketball coach John Wooden won 10 national ing "OK." This information comes from
championships from 1964 through 1975, an knowledgeable sources who have made
unparalleled sports achievement. But the site visits to enough programs to have a
game he remembered the most and said he basic idea about what the variation is. The
learned the most from was UCLA's 1974 question is this: How should programs be
overtime loss to North Carolina State in the sampled for the study? If one wanted to
semifinais (quoted in the Los Angeles Times, precisely document the natural variation
December 21,2000). Wooden's focus on that among programs, a random sample
gamethat extreme caseillustrates the would be appropriate, preferably a ran-
learning psychology of extreme group pur- dom sample of sufficient size to be truly
poseful sampling. representative and permit generalizations
This is also the sampling psychology be- to the total population of programs. How-
hind Jim Paul's (1994) book What I Learned ever, some information is already avail-
Losing a Million Dollars. A former governor able on what program variation is like. The
of the Chicago Mercantile Exchange, he question of more immediate interest may
made thousands of trades in many com- concern illuminative cases. With limited re-
modities over a long and distinguished ca- sources and limited time, an evaluator
reer, but what he reports learning the most might learn more by intensively studying
from was a highly unusual combination of one or more examples of really poor pro-
mistakes in which he lost more than $1 mil- grams and one or more examples of really
lion in a few weeks of trading soy beans an excellent programs. The evaluation focus,
extreme but illuminative case. He reports then, becomes a question of understand-
that he ultimately learned to be a winner by ing under what conditions programs get
carefully studying and learning from losing. into trouble and under what conditions
Designing Qualitative Studies !fj. 233

SM/ HMO THE K5-T

=:JH!- :i > L M M I I ! .Ifr.ii I J L'ii3! K I I ! C i:!!!1!:!.::.:'.:!' ir-wi-wpiv!i- :::;:! ^'iiv:'
uvri P R A TF-R I jki VS:;' ; : J : I : ! . V ' T ! : \ k IV IÎMRIIIN! ::=! i'.!!:::;.!i."im' ':i>! == :=.:
: ;
1 ! i-1"O
F I A I S F R V I ! iVi'L"i ': .'I' J:
! , , niI:vi3. :ii J,i '::':;' PAl*
, ! , s , : iVi - sii .=31 -j=
!:: iV r .!:!.'I|<-.!tii:; ..=ij! ii! -d !." iVC l>.Y ii:!:5 V!.; .igi" -jis-::s !.=- ' = j - îi? "O
:"!; 1 rri!:t'i:!.' rh tiq"\ .!:,'ifcliifi-î!iiihiM Cii:ii-' esrtihjpoaw :!:;:;::;( n.- rILTCI ".:
': N ::. I - =7 V- . IVi r =.! . I I I.I : I.= !
P 1 :. I : ! i- :; -i:; -:ij S :: !!=:'.:!; a i : ' : <<=> hi
LMSVNJJTTTVFT SI!''!.!!.!?! ! T ! !\..:i :T.1H P I L M W -
! l.Vv.iVry .ivi
i: !:i i5 .
':! n>= i
: i! ij-: n o ====!
jea-f h lAVa;!. IV h:!::'*:j>iihi V : : I j'fi-5!fa!!! edv.'!'.': h: rui ; tflE^:
! :: p:;.!:. - ::!: :r >;!!
VKS ! ;:' RI I RN.'1 ;:; Y I I J . Y R U M !,!.:!! !"!ii;:'i !iJi.s 0 .v^scrM iV-: ir! !>.I:-::;;Y =:=.':!f::.: = :=s i-n;
:;i'ij::".!i i!;i: ::::tri i WM, ,'f t '.. >S j: i:'il! :Tr:j:l.-:i:"i:i! ' M.ii-:i"ii:, ==!:':;Ori!iri!'i I ! ! i ! i ' " ' !:':-'"ii':i
! '!:;
! J:S.' I:;I I..T-IV=.I'I ;R . S I I D .Ir!'!:!.!'! H I * ! JiVlj:1 :F'. SFT: ! J
!
J^- i: : :':. ': w: .V: :!.;; . =! i-:: === i- K f
!|?iViiliV:::':!"-.^",'; nYl iV-i' i'.!.'i i:!:T:|j-. iJ!!:!:!i.i: !:: i;yiri!ri'i:i;i :!'^! !::! i.!! : !:'!' inixuirs
: :
IVii-
:;r::i;4 if if pvi \ rn h:;ri M:=i!. ,li h:|,ri:':! hihii5 n:i :=!:i: !:-: ". " ii'
! !i.iVi.i'i:- -:'!- : ":; : ) ' : , N :r
:

"JGVT:! I I T 11-5 == . . " I I v'J.'}.R ir>, \!;I'!," II 1';.=!^

1
*:':; IYV^I-i-iVli'::- Y I I V I I J / I . ! ! : X =!'!I-Iil
! I I ; I I " ' ! -LI!!

j i ;: h -:! :'!:.!";;! :;: ;M:|!|T!;;:W!

! iYird i-lMrij:!!:!; ClsilsillM.shl^.'! i:'i;! SiViV ' !='.'' i! !',! ! i!'..i i
:;:!:!' YXV" In livy !Y"P;:Iri R N FRI''"Irn. .'I!::- :T!ii) i i hi i:i: i!'i !' iV :i f i'i iVi l! H iX! , :v,.i" !V!i!
!!"i!::Cis"!! !":' isi""'" x* "15fr".s:.?"!!:':ii i.:
! ! !
i.r'iI'." T!:!":li' Lra i.i\'iW i!'i' I i!' :;!:
:\'>kr!!i:!sv ;:s!'!i:s !wi\iTI ;iTh"! ,1..Ei! ,!:..- 1 j!'i i j ' . . i,Mi!!i:;
!".;::;: J N R;.;I !| !;{ : ;; N N J I V I V sVi.R. I " ; ; " R W - K - M|:|I I !
; D V R , ' ! ! I'!:" / I T I P I N I V . ! O ? ! ! ' : ; !'.': ! ' I i/I .i ,'-!'. ; i'.;

!:';:;:: O" 1'.U'! w-rVC!. .iV:!i: F^IIWIIHII tlDIE hivii'. '; I!^ AWll" i'i.'J i . i
1

f-5-:5j:==:-=: i : : \ > ^ - v i : : i r ^ V n i;i.;.i! !

:: : HJ' ! I:i'I:::!ii-I:.'I-I : I^ri ' r
: !: ! ! : : TS J''I j ,-i'i :' : .

iVii! OV. ! :A1\\\!i/f i=i=.;!i !V ;:!!:V!;= j"jj^nnifii '!j!!;i!;!-ri i.^!S'!'!!ri!,'i!.:!!:!'.!,! iVis.= i!' !'>i":ir" jll.'Hi
:;=: -5=-B:. = :: ' : I S; S: I; R . ! '
5
:
! : . : , : = : :' !=;:;! MN LV! ;:=I.! !:! h"-i I'- r ri T " ! I'- N.L ! .!!;= ! .'! . U

iiiii s: h :i;!.n : ,!!!:.: r^J ..hivvi.:;^:^! .i;^^!^::':! !::il ;: i" i'.': r m-ii vi y . ' =:! i ,'n!:. i i r I '"'
:!! i i1;;' : \ oo?x< !" :' n!; lyix j;; :;v! v cg !-!; :H; :.T N B N - S :! ;R' i V. i.=N -J!:T! -I
! ! !
; I I:' i iVr

k- i.iiv,! vii^^irK!'!.';!;! :ip-.t:.hi. ''.i j-fe! -

! ! ;
| i i: !i:!.!l
: :!' R " : " ' ! ! ! !ri'i"." |i ViSITS' r! !' i i,".! " . -""'r
. v. .. .. ,...1 , ! ..: .. ... . .. j j . I . . ,
; ) , 'i , 1-1 | ! Vi"
;;Ji.-.I;!:'.'::'. J-w xR-!.: T!: -:!S:! - Y!
!! -

iViivi iVrjs' = r i! :':.'T; \'I i'tvi ::S-I!- . : IV!' prt !:,: .-.Ml-, J i;.
!
,1 i s i. vj i !.". ! 11 ,.,,,j

i.!.:. i ViV i V ! i i:!1 ii i! 1 =:-' C!:'V:!'!:i' ! ? i h \Xi w::L.VIII.:i -II !-: HRI
! \ .j:I;YY!.*:T! P brS |: : I I.!." I-S^VS'.

programs exemplify excellence. It is not standing selected cases of special interest,

even necessary to randomly sample poor for example, unexpected dropouts or out-
programs or excellent programs. The re- standing successes. In an evaluation of the
searchers and intended users involved in the Caribbean Agricultural Extension Project,
study think through what cases they could we did case studies of the "outstanding ex-
learn the most from and those are the cases tension agent" selected by peers in each of
that are selected for study. eight Caribbean countries to help the pro-
In a single program the same strategy gram develop curriculum and standards
may apply. Instead of studying some repre- for improving extension practice. The
senta tive sample of people in the setting, the sample was purposefully "biased," not to
evaluator may focus on studying and under- make the program look good, but rather to
234 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

learn from those who were exemplars of nomenon of interest intensely (but not ex-
good practice. In many instances, more can tremely). Extreme or deviant cases may be
be learned from intensively studying exem- so unusual as to distort the manifesta tion of
plary (information-rich) cases than can be the phenomenon of interest. Using the logic
learned from statistical depictions of what of intensity sampling, one seeks excellent or
the average case is like. In other evaluations, rich examples of the phenomenon of inter-
detailed information about special cases can est, but not highly unusual cases.
be used to supplement statistical data about Heuristic research (Chapter 3) uses inten-
the normal distribution of participants. In sity sampling. Heuristic research draws
statistical terms, extreme case sampling fo- explicitly on the intense personal expe-
cuses on outliers (the endpoints of the bell- riences of the researcher, for example, ex-
shaped curve normal distribution) that are periences with loneliness or jealousy. Co-
often ignored in aggregate data reporting. researchers who have experienced these
Ethnomethodologists use a form of ex- phenomena intensely also participate in the
treme case sampling when they do their study. The heuristic researcher is not typi-
field experiments. Ethnomethodologists are cally seeking pathological or extreme mani-
interested in everyday experiences of rou- festations of loneliness, jealousy, or what-
tine living that depend on deeply under- ever phenomenon is of interest. Such
stood, shared understandings among peo- extreme cases might not lend themselves to
ple in a setting (see Chapter 3). One way of the reflective process of heuristic inquiry.
exposing these implicit assumptions and On the other hand, if the experience of
norms on which everyday life is based is to the heuristic researcher and his or her
create disturbances that deviate greatly from coresearchers is quite mild, there won't be
the norm. Observing the reactions to some- much to study. Thus, the researcher seeks a
one eating like a pig in a restaurant and then sample of sufficient intensity to elucidate the
interviewing people about what they saw phenomenon of interest.
and how they felt would be an example of The same strategy can be applied in a pro-
studying a deviant sample to illuminate the gram evaluation. Extreme successes or un-
ordinary. usual failures may be discredited as being
In essence, the logic of extreme group too extreme or unusual to yield useful infor-
sampling is that extreme cases may be informa tion. Therefore, the evaluator may select
mation-rich cases precisely because, by be- cases that manifest sufficient intensity to il-
ing unusual, they can illuminate both the luminate the nature of success or failure, but
unusual and the typical. In proposing an ex- not at the extreme.
treme group sample, as in ali purposeful Intensity sampling involves some prior
sampling designs, the researcher has an obli- information and considerable judgment.
gation to present the rationale and expected The researcher or evaluator must do some
benefits of this strategy as well as to note its exploratory work to determine the nature of
weakness (lack of generalizability). the variation in the situation under study,
then sample intense examples of the phe-
2. Intensity sampling. Intensity sampling nomenon of interest.
involves the same logic as extreme case
sampling but with less emphasis on the ex- 3. Maximum variation (heterogeneity) sam-
tremes. An intensity sample consists of in- pling. This strategy for purposeful sam-
formation-rich cases that manifest the phe- pling aims at capturing and describing the
Designing Qualitative Studies !fj. 235

central themes that cut across a great deal of case studies of individual fellowship recipi-
variation. For small samples, a great deal of ents. Over 20 years of awards, more than 600
heterogeneity can be a problem because in- people had received fellowships. We had
dividual cases are so different from each sufficient resources to do only 40 case stud-
other. The maximum variation sampling ies. We naximized sample variation by cre-
strategy turns that apparent weakness into ating a matrix in which each person in the
a strength by applying the following logic: sample was as different as possible from oth-
Any common patterns that emerge from ers using dimensions such as nature of
great variation are of particular interest and work, stage in career, public visibility, insti-
value in capturing the core experiences and tutional affiliation, age, gender, ethnicity,
central, shared dimensions of a setting or geographic location, mobility, health status,
phenomenon. nationality, and field of endeavor. The the-
How does one maximize variation in a matic patterns of achievement that emerged
small sample? One begins by identifying di- from this diversity allowed us to construct a
verse characteristics or criteria for construct- model to illuminate the primary dimensions
ing the sample. Suppose a statewide proof and factors in the award's impact. A
gram has project sites spread around the theme song emerged from ali the scattered
state, some in rural areas, some in urban ar- noise. That's the power of maximum varia-
eas, and some in suburban areas. The evalu- tion (heterogeneity) sampling.
ation lacks sufficient resources to randomly Thus, when selecting a small sample of
select enough project sites to generalize great diversity, the data collection and anal-
across the state. The evaluator can study a ysis will yield two kinds of findings: (1)
few sites from each area and at least be sure high-quality, detailed descriptions of each
that the geographical variation among sites case, which are useful for documenting
is represented in the study. While the evalua- uniquenesses, and (2) important shared pat-
tion would describe the uniqueness of each terns that cut across cases and derive their
site, it would also look for common themes significance from having emerged out of
across sites. Any such themes take on added heterogeneity. Both are important findings
importance precisely because they emerge in qualitative inquiry.
out of great variation. For example, in study-
ing community-based energy conservation
efforts statewide using a maximum hetero- 4. Homogeneous samples. In direct contrast
geneity sampling strategy, I constructed a to maximum variation sampling is the strat-
matrix sample of 10 commuriities m which egy of picking a small, homogeneous sam-
each community was as different as possible ple, the purpose of which is to describe
from every other community on such char- some particular subgroup in depth. A pro-
acteristics as size, form of local government gram that has many different kinds of par-
(e.g., strong mayor/weak mayor), ethnic di- ticipants may need in-depth information
versity, strength of the economy, demo- about a particular subgroup. For example, a
graphics, and region. In the analysis, what parent education program that involves
stood out across these diverse cases was the many different kinds of parents may focus a
importance of a local, committed cadre of qualitative evaluation on the experiences of
people who made things happen. single-parent female heads of household
In a study of the MacArthur Foundation because that is a particularly difficult group
Fellowship Program, the design focused on to reach and hold in the program.
236 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Focus group interviews are based typi- illuminates key issues that must be consid-
cally on homogeneous groups. Focus ered in any development project aimed at
groups involve open-ended interviews with that kind of village.
groups of five to eight people on specially In evaluation and policy research, the in-
targeted or focused issues. The use of focus terests of decision makers will shape the
groups in evaluation will be discussed at sampling strategy. I remember an evalua-
greater length in the chapter on interview- tion in which the key decision makers had
ing. The point here is that sampling for focus made their peace with the fact that there will
groups typically involves bringing together always be some poor programs and some
people of similar backgrounds and experi- excellent programs, but the programs they
ences to participate in a group interview really wanted more information about were
about major issues that affect them. what they called "those run-of-the-mill pro-
grams that are so hard to get a handle on pre-
5. Typicai case sampling. In describing a cisely because they are so ordinary and don't
culture or program to people not familiar stand out in any definitive way." Given that
with the setting studied, it can be helpful to framing, we employed typical case sam-
provide a qualitative profile of one or more pling. It is important, when using this strat-
typical cases. These cases are selected with egy, to attempt to getbroad consensus about
the cooperation of key informants, such as which cases are typicaland what criteria
program staff or knowledgeable partici- are being used to define typicality.
pants, who can help identify who and what
are typical. Typical cases can also be selected 6. Criticai case sampling. Criticai cases are
using survey data, a demographic analysis those that can make a point quite dramati-
of averages, or other statistical data that cally or are, for some reason, particularly
provide a normal distribution of character- important in the scheme of things. A clue to
istics from which to identify "average-like" the existence of a criticai case is a statement
cases. Keep in mind that the purpose of a to the effect that "if it happens there, it will
qualitative profile of one or more typical happen anywhere," or, vice versa, "if it
cases is to describe and illustrate what is doesn't happen there, it won't happen any-
typical to those unfamiliar with the set- where." Another clue to the existence of a
tingnot to make generalized statements criticai case is a key informant observation
about the experiences of ali participants. to the effect that "if that group is having
The sample is illustrative not definitive. problems, then we can be sure ali the groups
When entire programs or communities are having problems."
are the unit of analysis, the processes and ef- Looking for the criticai case is particularly
fects described for the typical program may important where resources may limit the
be used to provide a frame of reference for evaluation to the study of only a single site.
case studies of "poor" or "excellent" sites. Under such conditions, it makes strategic
When the typical site sampling strategy is sense to pick the site that would yield the
used, the site is specifically selected because most information and have the greatest im-
it is not in any major way atypical, extreme, pact on the development of knowledge.
deviant, or intensely unusual. This strategy While studying one or a few criticai cases
is often appropriate in sampling villages for does not technically permit broad general-
community development studies in Third izations to ali possible cases, logical general-
World countries. A study of a typical village izations can oftenbe made from the weight of
Designing Qualitative Studies !fj. 237

evidence produced in studying a single, crit- it could work anywhere. That makes the crit-
icai case. icai case an especially information-rich
Physics provides a good example of such exemplar, therefore worthy of study as the
a criticai case. In Galileo's study of gravity, centerpiece in a small or "N of 1" sample.
he wanted to find out if the weight of an ob- World-renowned medicai hypnotist Mil-
ject affected the rate of speed at which it ton H. Erickson became a criticai case in the
would fali. Rather than randomly sampling field of hypnosis. Erickson was so skillful
objects of different weights in order to gener- that he became widely known for "his abil-
alize to ali objects in the world, he selected a ity to succeed with 'impossibles'people
criticai casethe feather. If in a vacuum, as who have exhausted the traditional medicai,
he demonstrated, a feather fell at the same dental, psychotherapeutic, hypnotic and re-
rate as some heavier object (a coin), then he ligious avenues for assisting them in their
could logically generalize from this one criti- need, and have not been able to make the
cai comparison to ali objects. His finding changes they desire" (Grinder, DeLozier,
was both useful and credible because the and Bandler 1977:109). If Milton Erickson
feather was a convincing criticai case. couldn't hypnotize a person, no one could.
Criticai cases can be found in social sci- He was able to demonstrate that, under his
ence and evaluation research if one is cre- defnition of hypnosis, anyone could be
ative in looking for them. For example, sup- hypnotized.
pose national policymakers want to get local
communities involved in making decisions 7. Snowballor chain sampling. This isan ap-
about how their local program will be run, proach for locating information-rich key in-
but they aren't sure that the communities formants or criticai cases. The process be-
will understand the complex regulations gins by asking well-situated people: "Who
governing their involvement. The first criti- knows a lot about ? Whom should I
cai case is to evaluate the regulations in a talk to?" By asking a number of people who
community of well-educated citizens; if they else to talk with, the snowball gets bigger
can't understand the regulations, then less and bigger as you accumulate new informa-
educated folks are sure to find the regula- tion-rich cases. In most programs or sys-
tions incomprehensible. Or conversely, one tems, a few key names or incidents are men-
might consider the criticai case to be a com- tioned repeatedly. Those people or events,
munity consisting of people with quite low recommended as valuable by a number of
leveis of education: "lf they can understand different informants, take on special impor-
the regulations, anyone can." tance. The chain of recommended infor-
Identification of criticai cases depends on mants would typically diverge initially as
recognition of the key dimensions that make many possible sources are recommended,
for a criticai case. For example, a criticai case then converge as a few key names get men-
might come from a particularly difficult pro- tioned over and over.
gram location. If the funders of a new pro- The Peters and Waterman (1982) study In
gram are worried about recruiting clients or Search of Excellence began with snowball
participants into a program, it may make sampling, asking a broad group of knowl-
sense to study the site where resistance to edgeable people to identify well-run compa-
the program is expected to be greatest to pro- nies. Rosabeth Moss Kanter's (1983) study of
vide the most rigorous test of program re- innovation reported in The Change Masters
cruitment. If the program works in that site, focused on 10 core case studies of the "most
238 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

innovative" companies. She began by ask- tem that exhibit certain predetermined crite-
ing corporate experts for candidate compa- rion characteristics are routinely identified
nies to study. Nominations snowballed as for in-depth, qualitative analysis. Criterion
she broadened her inquiry and then con- sampling also can be used to identify cases
verged into a small number of core cases from standardized questionnaires for in-
nominated by a number of different expert depth follow-up, for example, ali respon-
informants. dents who report having experienced ongo-
ing workplace discrimination. (This strategy
can only be used where respondents have
8. Criterion sampling. The logic of criterion willingly supplied contact information.)
sampling is to review and study ali cases
that meet some predetermined criterion of
importance, a strategy common in quality 9. Theory-based sampling, operational con-
assurance efforts. For example, the expected struct sampling, and theoretical sampling. A
range of participation in a mental health more conceptually oriented version of crite-
outpatient program might be 4 to 26 weeks. rion sampling is theory-based sampling.
Ali cases that exceed 28 weeks are reviewed The researcher samples incidents, slices of
to find out why the expected range was ex- life, time periods, or people on the basis of
ceeded and to make sure the case was being their potential manifestation or representa-
appropriately handled. Or a quality assur- tion of important theoretical constructs.
ance standard may be that ali patients enter- Buckholt (2001) studied people who met
ing a hospital emergency room, who are not theory-derived criteria for being "resilient"
in a life-threatening situation, receive care in a study of resilience among adult abuse
within 2 hours. Cases that exceed this stan- survivors. The sample becomes, by defini-
dard are reviewed. tion and selection, representative of the phe-
Criticai incidents can be a source of crite- nomenon of interest.
rion sampling. For example, ali incidents of When one is studying people, programs,
client abuse in a program may be objects of organizations, or communities, the popula-
in-depth evaluation in a quality assurance tion of interest can be fairly readily deter-
effort. Ali former mental health clients who mined. Constructs, however, do not have as
commit suicide within three months of re- clear a frame of reference:
lease may constitute a sample for in-depth,
qualitative study. In a school setting, ali stu- For sampling operational instances of con-
dents who are absent 25% or more of the structs, there is no concrete target popula-
time may merit the in-depth attention of a tion. . . . Mostly, therefore, we are forced to
case study. The point of criterion sampling is select on a purposive basis those particular in-
to be sure to understand cases that are likely stances of a construct that past validity stud-
to be information rich because they may re- ies, conventional practice, individual intuition,
veal major system weaknesses that become or consultation with critically minded persons
targets of opportunity for program or sys- suggest offer the closest correspondence to the
tem improvement. construct of interest. Altematively, we can use
Criterion sampling can add an important the same procedures to select multiple opera-
qualitative component to a management in- tional representa tions of each construct, cho-
formation system or an ongoing program sen because they overlap in representing the
monitoring system. Ali cases in the data sys- criticai theoretical components of the con-
Designing Qualitative Studies !fj. 239

struct and because they differ from each other data gathered during fieldwork. The con-
on irrelevant dimensions. This second form of stant comparative method involves sys-
sampling is called multiple operationalism, tematically examining and refining varia-
and it depends more heavily on individual tions in emergent and grounded concepts.
judgment than does the random sampling of Variations in the concept must be sampled
persons from a well-designated, target popu- to rigorously compare and contrast those
lation. Yet, such judgments, while inevitable, variations. (See Chapters 3 and 8 for more
are less well understood than formal sampling detailed discussions of grounded theory.)
methods and are largely ignored by sampling
experts. (Cook, Leviton, and Shadish 1985: 10. Confirming and disconfirming cases. In
163-64) the early part of qualitative fieldwork, the
evaluator is exploringgathering data and
Operational construct sampling simply watching for patterns to emerge. Over time,
means that one samples for study real- the exploratory process gives way to confir-
world examples (i.e., operational examples) matory fieldwork. This involves testing
of the constructs in which one is interested. ideas, confirming the importance and
Studying a number of such examples is meaning of possible patterns, and checking
called "multiple operationalism" (Webb out the viability of emergent findings with
et al. 1966). For example, classic diffusion of new data and additional cases. This stage of
innovations theory (Rogers 1962) predicts fieldwork requires considerable rigor and
that early adopters of some innovation will integrity on the part of the evaluator in look-
be different in significant ways from later ing for and sampling confirming as ivell as
adopters. Doing cases studies on early and disconfirming cases.
late adopters, then, would be an example of Confirmatory cases are additional exam-
theory-based sampling. Such samples are ples that fit already emergent patterns; these
often necessarily purposefully selected be- cases confirm and elabora te the findings,
cause the population of ali early and late adding richness, depth, and credibility.
adopters may not be known, so random Disconfirming cases are no less important at
sampling is not an option. this point. These are the examples that don't
Theoretical sampling is what grounded fit. They are a source of rival interpretations
theorists define as "sampling on the basis of as well as a way of placing boundaries
the emerging concepts, with the aim being to around confirmed findings. They may be
explore the dimensional range or varied "exceptions that prove the rule" or excep-
conditions along which the properties of tions that disconfirm and alter what ap-
concepts vary" (Strauss and Corbin 1998: peared to be primary patterns.
73). In grounded theory, theoretical sam- The source of questions or ideas to be con-
pling supports the constant comparative firmed or disconfirmed may be from stake-
method of analysis. That is, one does theo- holders or previous scholarly literature
retical sampling in grounded theory in order rather than the evaluator's fieldwork. An
to use the constant comparative method of evaluation may in part serve the purpose of
analysis. The two go hand in glove, connect- confirming or disconfirming stakeholders'
ing design and analysis. Theoretical sam- or scholars' hypotheses, these having been
pling permits elucidation and refinement identified during early, conceptual evalua-
of the variations in, manifestations of, and tor-stakeholder design discussions or litera-
meanings of a concept as it is found in the ture reviews.
240 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Thinking about the challenge of finding ever the data lead is a primary strength of
confirming and disconfirming cases empha- qualitative fieldwork strategies. This per-
sizes the relationship between sampling and mits the sample to emerge during field-
research conclusions. The sample deter- work.
mines what the evaluator will have some- During fieldwork, it is impossible to ob-
thing to say aboutthus the importance of serve everything. Decisions must be made
sampling carefully and thoughtfully. about what activities to observe, which peo-
ple to observe and interview, and when to
11. Stratified purposeful sampling. Strat- collect data. These decisions cannot ali be
ified samples are samples within samples. A made in advance. The purposeful sampling
stratified random sample, for example, strategies discussed above provide direction
might stratify by socioeconomic status for sampling but often depend on some
within a larger population so as to make knowledge of the setting being studied. Op-
generalizations and statistically valid com- portunistic, emergent sampling takes ad-
parisons by social class as well as to gener- vantage of whatever unfolds as it unfolds.
alize to the total population. In Chapter 2,1 identified emergent flexi-
Purposeful samples can also be stratified ble designs as one of the core strategic
and nested by combining types of purpose- themes of qualitative inquiry and cited as an
ful sampling. So, for example, one might exemplar the anthropologist Brackette F.
combine typical case sampling with maxi- Williams and her fieldwork on how Ameri-
mum heterogeneity sampling by taking a cans view violence in America:
stratified purposeful sample of above aver-
age, average, and below average cases. This I do impromptu interviews. I don't have some
represents less than a full maximum varia- target number of interviews in mind or prede-
tion sample, but more than simple typical termined questions. It depends on the person
case sampling. The purpose of a stratified and the situation. Airports, for example, are a
purposeful sample is to capture major varia- good place for impromptu interviews with
tions rather than to identify a common core, people. So sometimes, instead of using airport
although the latter may also emerge in the time to write, I interview people about the
analysis. Each of the strata would constitute death penalty or about killing or about death
a fairly homogeneous sample. This strategy in their life. It's called opportunity sampling
differs from stratified random sampling in I'm following where the data take me, where
that the sample sizes are likely to be to o my questions take me. (personal interview)
small for generalization or statistical repre-
sentativeness. Few qualitative studies are as fully emergent
and open-ended as the fieldwork of Wil-
12. Opportunistic or emergent sampling. liams. Her approach exemplifies emergent
Fieldwork often involves on-the-spot deci- opportunity sampling.
sions about sampling to take advantage of
new opportunities during actual data col- 13. Purposeful random sampling. A purpose-
lection. Unlike experimental designs, emer- ful sampling strategy does not automati-
gent qualitative designs can include the op- cally eliminate any possibility for random
tion of adding to a sample to take advantage selection of cases. For many audiences, ran-
of unforeseen opportunities after fieldwork dom sampling, even of small samples, will
has begun. Being open to following wher- substantially increase the credibility of the
Designing Qualitative Studies !fj. 241

results. I recently worked with a program about why certain cases were selected for
that annually appears before the state legis- study, but such a sample still does not permit
lature and tells "war stories" about client statistical generalizations.
successes and struggles, sometimes even in-
cluding a few stories about failures to pro- 14. Sampling politically important cases. Eval-
vide balance. To enhance the credibility of uation is inherently and inevitably political
their reports, the director and staff decided (see Turpin 1989; Falumbo 1987; Patton
to begin collecting evaluation information 1987b). A variation on the criticai case strat-
more systematically. Because they were egy involves selecting (or sometimes avoid-
striving for individualized outcomes, they ing) a politically sensitive site or unit of
rejected the notion of basing the evaluation analysis. For example, a statewide program
entirely on a standardized pre-post instru- may have a local site in the district of a state
ment. They wanted to collect case histories legislator who is particularly influential. By
and do in-depth case studies of clients, but studying carefully the program in that dis-
they had very limited resources and time to trict, evaluation data may be more likely to
devote to such data collection. In effect, staff attract attention and get used. This does not
at each program site, many of whom serve mean that the evaluator then undertakes to
200 to 300 families a year, felt that they could make that site look either good or bad, de-
only do 10 or 15 detailed, in-depth clinicai pending on the politics of the moment. That
case histories each year. We systematized would clearly be unethical. Rather, sam-
the kind of information that would be going pling politically important cases is simply a
into the case histories at each program site strategy for trying to increase the usefulness
and then set up a random procedure for se- and relevance of information where re-
lecting those clients whose case histories sources permit the study of only a limited
would be recorded in depth, thereby sys- number of cases.
tematizing and randomizing their collec- The same political perspective (broadly
tion of war stories. While they cannot gener- speaking) may inform case sampling in ap-
alize to the entire client population on the plied or even basic research studies. A politi-
basis of 10 cases from each program site, cal scientist or historian might select the
they will be able to tell legislators that the election year 2000 Florida vote-counting
stories they are reporting were randomly se- case, the Clinton impeachment effort,
lected in advance of knowledge of how the Nixon's Watergate crisis, or Reagan's Iran-
outcomes would appear and that the infor- Contra scandal for study not only because of
mation collected was comprehensive. The the insights they provide about the Ameri-
credibility of systematic and randomly se- can system of government but because of the
lected case examples is considerably greater likely attention such a study would attract.
than the personal, ad hoc selection of cases A sociologisfs study of a riot or a psycholo-
selected and reported after the factthat is, gisfs study of a famous suicide would likely
after outcomes are known. involve some attention during sampling to
It is criticai to understand, however, that the public and political importance of the
this is a purposeful random sample, not a repre- case.
sentative random sample. The purpose of a
small random sample is credibility, not 15. Convenience sampling. Finally, there is
representa ti veness. A small, purposeful the strategy of sampling by convenience:
random sample aims to reduce suspicion doing what's fast and convenient. This is
242 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

probably the most common sampling strat- ments that will lend credibility to the study
egyand the least desirable. Too often eval- as well as the kinds of arguments that
uators using qualitative methods think that might be used to attack the findings. Rea-
because the sample size they can study will sons for site selections or individual case
be too small to permit generalizations, it sampling need to be carefully articulated
doesn't matter how cases are picked, so they and made explicit. Moreover, it is important
might as well pick ones that are easy to ac- to be open and clear about a study's lrmita-
cess and inexpensive to study While conve- tions, that is, to anticipate and address criti-
nience and cost are real considerations, cisms that may be made of a particular sam-
they should be the last factors to be taken pling strategy, especially from people who
into account after strategically deliberating think that the only high-quality samples are
on how to get the most information of great- random ones.
est utility from the limited number of cases Having weighed the evidence and con-
to be sampled. Purposeful, strategic sam- sidered the alternatives, evaluators and pri-
pling can yield crucial information about mary stakeholders make their sampling de-
criticai cases. Convenience sampling is cisions, sometimes painfully, but always
neither purposeful nor strategic. with the recognition that there are no perfect
designs. The sampling strategy must be se-
Information-Rich Cases lected to fit the purpose of the study, the re-
sources available, the questions being asked,
Exhibit 5.6 summarizes the 15 purposeful and the constraints being faced. This holds
sampling strategies discussed above, plus a true for sampling strategy as well as sample
16th approachcombination or mixed pur- size.
poseful sampling. For example, an extreme
group or maximum heterogeneity approach
S a m p l e Size
may yield an initial potential sample size
that is still larger than the study can handle.
The final selection, then, may be made ran- Qualitative inquiry is rife with ambiguities.
domlya combination approach. Thus, There are purposeful strategies instead of
these approaches are not mutually exclu- methodological rules. There are inquiry ap-
sive. Each approach serves a somewhat dif- proaches instead of statistical formulas.
ferent purpose. Because research and evalu- Qualitative inquiry seems to work best for
ations often serve multiple purposes, more people with a high tolerance for ambiguity.
than one qualitative sampling strategy may (And we're still only discussing design. It
be necessary. In long-term fieldwork, ali of gets worse when we get to analysis.)
these strategies may be used at some point. Nowhere is this ambiguity clearer than in
The underlying principie that is common the matter of sample size.
to ali these strategies is selecting informa- I get letters. I get calls. I get e-mails.
tion-rich casescases from which one can
learn a great deal about matters of impor- Is 10 a large enough sample to achieve maxi-
tance and therefore worthy of in-depth mum variation?
study
In the process of developing the research I started out to interview 20 people for two
design, the evaluator or researcher is trying hours each, but I've lost 2 people. Is 18 large
to consider and anticipate the kinds of argu- enough, or do I have to find 2 more?
Designing Qualitative Studies 243

n&P&l
HBw B fflsiJSiBB
swasin D/wflras^SBfiBSWBSsasaasa
Sampling Strategies
i ^

Type Purpose

Random probabiiity sampling Representativeness: Sample size a function of population size

and desired confidence levei.
1. Simple random sample Permit generalization from sample to the population it
re presents.
2. Stratified random and Increase confidence in making generalizations to particular
cluster samples subgroups.

Purposeful sampling Select information-rich cases strategically and purposefully;

specific type and number of cases selected depends on study
purpose and resources.
1. Extreme or deviant case Learning from unusual manifestations of the phenomenon of
(outlier) sampling interest, for example, outstanding successes/notable failures;
top of the class/dropouts; exotic events; crises.
2. Intensity sampling Information-rich cases that manifest the phenomenon
intensely, but not extremely, for example, good students/
poor students; above average/below average.
3. Maximum variation Document unique or diverse variations that have emerged in
samplingpurposefully adaptng to different conditions. Identify important common
picking a wide range of patterns that cut across variations (cut through the noise of
cases to get variation on variation).
dimensions of interest
4. Homogeneous sampling Focus; reduce variation; simplify analysis; facilitate group
interviewing.
5. Typical case sampling Illustrate or highlight what is typical, normal, average.

6. Criticai case sampling Permits logical generalization and maximum application of

information to other cases because if it's true of this one case,
i f s likely to be true of ali other cases.
7. Snowball or chain Identify cases of interest from sampling people who know
sampling people who know people who know what cases are information
rich, that is, good examples for study, good interview
participants.
8. Criterion sampling Picking ali cases that meet some criterion, for example, ali
children abused in a treatment facility. Quality assurance.
9. Theory-based sampling, Finding manifestations of a theoretical construct of interest so
operational construct as to elaborate and examine the construct and its variations.
sampling, or theoretical
sampling

(continued)
244 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

I
E X H I B I T 5.6

Type Purpose

10. Confirming and Elaborating and deepening initial analysis; seeking exceptions;
disconfirming cases testing variation.
11. Stratified purposefui Illustrate characteristics of particular subgroups of interest;
sampling faciiitate comparisons.
12. Opportunistic or Following new leads during fieldwork; taking advantage of
emergent sampling the unexpected; flexibility.
13. Purposefui random Add credibility when potential purposefui sample is larger than
sampling {still small one can handle. Reduces bias within a purposefui category.
sample size) (Not for generalizations or representativeness.)
14. Sampling polticaNy Attract attention to the study (or avoid attracting undesired
important cases attention by purposefully eliminating from the sample
politically sensitive cases).
15. Convenience sampling Do what's easy to save time, money, and effort Poorest
rationale; lowest credibility. Yieds information-poor cases.
16. Combination or mixed Trianguation; flexibility; meet multiple interests and needs.
purposefui sampling

I want to study just one organization, but in- a small number of people can be very valu-
terview 20 people in the organization. Is my able, especially if the cases are information
sample size 1 or 20 or both? rich. Less depth from a larger number of
people can be especially help fui in exploring
My universal, certain, and confident reply to a phenomenon and trying to document di-
these questions is this: "It depends." versity or understand variation. I repeat, the
There are no rules for sample size in size of the sample depends on what you
qualitative inquiry. Sample size depends on want to find out, why you want to find it out,
what you want to know, the purpose of the how the findings will be used, and what re-
inquiry, what's at stake, what will be useful, sources (including time) you have for the
what will have credibility, and what can be study.
done with available time and resources. To understand the problem of small sam-
Earlier in this chapter, I discussed the ples in qualitative inquiry, it's necessary to
trade-offs between breadth and depth. With place these small samples in the context of
the same fixed resources and limited time, a probability sampling. A qualitative inquiry
researcher could study a specific set of expe- sample only seems small in comparison with
riences for a larger number of people (seek- the sample size needed for representa tive-
ing breadth) or a more open range of experi- ness when the purpose is generalizing from
ences for a smaller number of people a sample to the population of which it is a
(seeking depth). In-depth information from part. Suppose there are 100 people in a pro-
Designing Qualitative Studies !fj. 245

gram to be evaluated. It would be necessary mulated their widely followed eight princi-
to randomly sample 80 of those people pies for organizational excellence by study-
(80%) to make a generalization at the 95% ing 62 companies, a very small sample of the
confidence levei. If there are 500 people in thousands of companies one might study.
the program, 217 people must be sampled Sands (2000) did a fine dissertation studying
(43%) for the same levei of confidence. If a single school principal, describing the
there are 1,000 people, 278 people must be leadership of a female leader who entered a
sampled (28%), and if there are 5,000 people challenging school situation and brought
in the population of interest, 357 must be about constructive change.
sampled (7%) to achieve a 95% confidence Clair Claiborne Park's (2001) single case
levei in the generalization of findings. At the study of her daughter's autism reports 40
other extreme, if there are only 50 people in years of data on every stage of her develop-
the program, 44 must be randomly sampled ment, language use, emotions, capacities,
(88%) to achieve a 95% levei of confidence. barriers, obsessions, communication pat-
(See Fitz-Gibbon and Morris [1987:163] for a terns, emergent artistry, and challenges
table on determining sample size from a overcome and challenges not overcome.
given population.) Park and her husband made systematic ob-
The logic of purposeful sampling is quite servations throughout the years. Eminent
different. The problem is, however, that the medicai anthropologist Oliver Saks re-
utility and credibility of small purposeful viewed the data and determined in his pref-
samples are often judged on the basis of the ace to the book that more data are available
logic, purpose, and recommended sample on the woman in this extraordinary case
sizes of probability sampling. Instead, pur- study than on any other autistic human be-
poseful samples should be judged according ing who has ever lived. Here, then, is the
to the purpose and rationale of the study: epitome of N = 1, in-depth inquiry.
Does the sampling strategy support the The validity, meaningfulness, and in-
study's purpose? The sample, like ali other sights generated from qualitative inquiry
aspects of qualitative inquiry, must be have more to do with the information rich-
judged in contextthe same principie that ness of the cases selected and the obser-
undergirds analysis and presentation of vational/analytical capabilities of the re-
qualitative data. Random probability sam- searcher than with sample size.
ples cannot accomplish what in-depth, pur- This issue of sample size is a lot like the
poseful samples accomplish, and vice versa. problem students have when they are as-
Piaget contributed a major breakthrough signed an essay to write.
to our understanding of how children think
by observing his own two children at length Student: "How long does the paper have to
and in great depth. Freud established the be?"
field of psychoanalysis based originally on
fewer than 10 client cases. Bandler and Instructor: "Long enough to cover the as-
Grinder (1975a, 1975b) founded neurolin- signment."
guistic programming (NLP) by studying
Student: "But how many pages?"
three renowned and highly effective thera-
pists: Milton Erickson, Fritz Perls, and Vir- Instructor: "Enough pages to do justice to
gnia Satir. Peters and Waterman (1982) for- the subjectno more, no less."
246 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Lincoln and Guba (1985) recommend evaluator is obligated to discuss how the
sample selection "to the point of redun- sample affected the findings, the strengths
dancy.... In purposeful sampling the size of and weaknesses of the sampling proce-
the sample is determined by informational dures, and any other design decisions that
considerations. If the purpose is to maxi- are relevant for interpreting and under-
mize information, the sampling is termi- standing the reported results. Exercising
nated when no new information is forth- care not to overgeneralize from purposeful
commg from new sampled units; thus samples, while maximizing to the full the
rednndancy is the primary criterion" (p. 202). advantages of in-depth, purposeful sam-
This strategy leaves the question of sampling, will do much to alleviate concerns
ple size open, another example of the emer- about small sample size.
gent nature of qualitative inquiry. There re-
mains, however, the practical problem of
how to negotiate an evaluation budget or get s. Emergent Designs
a dissertation committee to approve a de- and Protection of
sign if you don't have some idea of sample Human Subjects
size. Sampling to the point of redundancy is
an ideal, one that works best for basic re- Emergent designs pose special problems for
search, unlimited timelines, and uncon- institutional review boards (IRBs) charged
strained resources. with approving research designs to ensure
The solution is judgment and negotiation. protection of human subjects. Such boards
I recommend that qualitative sampling de- typically want to know, in advance of field-
signs specify minimum samples based on ex- work, who will be interviewed and the pre-
pected reasonable coverage of the phenome- cise questions that will be asked. If the topic
non given the purpose of the study and is fairly innocuous and the general line of
stakeholder mterests. One may add to the questioning relatively unobtrusive, an IRB
sample as fieldwork unfolds. One may may be willing to approve the framework of
change the sample if information emerges an emergent design with sample questions
that indicates the value of a change. The de- included, but without full sample specifica-
sign should be understood to be flexible and tion and a formal interview instrument.
emergent. Ye t, at the beginning, for planning Another approach is to ask for approval
and budgetary purposes, one specifies a in stages. This means initially asking for
minimum expected sample size and builds a approval for the general framework of the
rationale for that minimum, as well as crite- inquiry and specifically for the first ex-
ria that would alert the researcher to inade- ploratory stage of fieldwork, including pro-
quacies in the original sampling approach cedures for assuring confidentiality and in-
and/or size. formed consent, then returning periodically
In the end, sample size adequacy, like ali (e.g., quarterly or annually) to update the
aspects of research, is subject to peer review, design and its approval. This is cumber-
consensual validation, and judgment. What some for both the researcher and the IRB, but
is crucial is that the sampling procedures it is a way of meeting IRB mandates and still
and decisions be fully described, explained, implementing an emergent design. This
and justified so that information users and staged-approval approach can also be used
peer reviewers have the appropriate context when the evaluator is developing the design
for judging the sample. The researcher or jointly with program staff and/or partici-
Designing Qualitative Studies !fj. 247

pants and therefore cannot specify the full types of data. The chapters on interviewing,
design at the beginning of the participatory observation, and analysis will include infor-
process. mation that will help in making design deci-
sions. Before turning to those chapters,
Methodological Mixes however, I want to briefly discuss the value
A study inay employ more than one sam- of using multiple methods in research and
pling strategy. It may also include multiple evaluation.

Triangulation

T he method must follow the question. Campbell, many decades ago,

promoted the concept of triangulationthat every method has its
limitations, and multiple methods are usually needed.
Gene V. Glass eulogizing pioneering methodologist
Donald T. Campbell, quoted in Tashakkori and Teddlie (1998:22)

Triangulation strengthens a study by sic domes la Buckminster Fuller). The logic

combining methods. This can mean using of triangulation is based on the premise that
several kinds of methods or data, including
using both quantitative and qualitative ap- no single method ever adequately solves the
proaches. Denzin (1978b) has identified four problem of rival causai factors. Because each
basic types of triangulation: (1) data triangu- method reveals different aspects of empirical
lation, the use of a variety of data sources in a reality, multiple methods of observations
study; (2) investigator triangulation, the use of must be employed. This is termed triangula-
several different researchers or evaluators; tion. I now offer as a final methodological rule
(3) theory triangulation, the use of multiple the principie that multiple methods should be
perspectives to interpret a single set of data, used in every investigation. (Denzin 1978b:28)
and (4) methodological triangulation, the use of
multiple methods to study a single problem Triangulation is ideal. It can also be ex-
or program. pensive. A study's limited budget and time
The term triangulation is taken from land frame will affect the amount of triangulation
surveying. Knowing a single landmark only that is practical, as will political constraints
locates you somewhere along a line in a di- (stakeholder values) in an evaluation. Cer-
rection from the landmark, whereas with tainly, one important strategy for inquiry is
two landmarks (and your own position be- to employ multiple methods, measures, re-
ing the third point of the triangle) you can searchers, and perspectives but to do so
take bearings in two directions and locate reasonably and practically.
yourself at their intersection (Fielding and
Fielding 1986:23). The term triangulation also Most good researchers prefer addressing their
works metaphorically to call to mind the research questions with any methodological
world's strongest geometric shapethe tri- tool available, using the pragmatist credo of
angle (e.g., the formused to construct geode- "what works." For most researchers commit-
248 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

ted to the thorough study of a research prob- or examining how competing theoretical
lem, method is secondary to the research perspectives inform a particular analysis
question itself, and the underlying worldview (e.g., the transcendental phenomenology of
hardy enters the picture, except in the most ab- Husserl vs. the hermeneutic phenomenol-
stract sense. (Tashakkori and Teddlie 1998:22) ogy of Heidegger). A study can also be de-
signed to cut across inquiry approaches and
A rich variety of methodological combi- achieve triangulation by combining qualita-
nations can be employed to illuminate an tive and quantitative methods, a strategy
inquiry question. Some studies intermix in- discussed and illustrated in the next section.
terviewing, observation, and document
analysis. Others rely more on interviews
Mixing Data, Design,
than observatons, and vice versa. Studies
and Analysis Approaches
that use only one method are more vulnera-
ble to errors linked to that particular method
(e.g., loaded interview questions, biased or Borrowing and combining distinct ele-
untrue responses) than studies that use mul- ments from pure or coherent methodologi-
tiple methods in which different types of cal strategies can generate creative mixed in-
data provide cross-data validity checks. quiry strategies that illustrate variations on
Using multiple methods alio ws inquiry into the theme of triangulation. We begin by
a research question with "an arsenal of distinguishing measurement, design, and
methods that have nonoverlapping weak- analysis components of the hypothetico-
nesses in addition to their complementary deductive (quantitative/experimental) and
strengths" (Brewer and Hunter 1989:17). holistic-inductive (qualitative/naturalistic)
However, a common misunderstanding paradigms. The ideal-typical qualitative
about triangulation is that the point is to methods strategy is made up of three parts:
demonstrate that different data sources or (1) qualitative data, (2) a holistic-inductive
inquiry approaches yield essentially the design of naturalistic inquiry, and (3) con-
same result. But the point is really to test for tent or case analysis. In the traditional
such consistency. Different kinds of data hypothetico-deductive approach to re-
may yield somewhat different results be- search, the ideal study would include (a)
cause different types of inquiry are sensitive quantitative data from (b) experimental (or
to different real-world nuances. Thus, un- quasi-experimental) designs and (c) statisti-
derstanding inconsistencies in findings cal analysis.
across different kinds of data can be Measurement, design, and analysis alter-
illuminative. Finding such inconsistencies natives can be mixed to create eclectic de-
ought not be viewed as weakening the credi- signs, like customizing an architectural plan
bility of results, but rather as offering oppor- to tastefully integra te modern, postmodern,
tunities for deeper insight into the relation- and traditional elements, or preparing an el-
ship between inquiry approach and the egant dinner with a French appetizer, a Chi-
phenomenon under study. nese entre, and an American dessertnot
Triangulation within a qualitative in- to everyone's taste, to be sure, but the possi-
quiry strategy can be attained by combining bilities are endless. At least that's the con-
both interviewing and observations, mixing cept. To make the idea of mixed elements
different types of purposeful samples (e.g., more concrete and to illustrate the creative
both intensity and opportunity sampling), possibilities that can emerge out of a flexible
Designing Qualitative Studies !fj. 249

Triangulation

approach to research, it will help to examine style, high drug use), and who are likely can-
alterna tive design possibilities for a single didates for delinquency (ahenated from
program evaluation. The examples that fol- dominant societal values, running with a
low have been constructed under the artifi- "bad" crowd, and angry). The program con-
cial constraint that only one kind of mea- sists of experiential education internships
surement, design, and analysis could be through which these high-risk students get
used in each case. In practice, of course, the individual tutoring in basic skills, part-time
possible mixes are much more varied, be- job placements that permit them to earn in-
cause any given study could include several come while gaining work exposure, and
measurement approaches, varying design participation in peer group discussions
approaches, and varying different analytical aimed at changing health values, establish-
approaches to achieve triangulation. ing a positive peer culture, and increasing
social integra tion. Several evaluation ap-
proaches are possible.
The Case of Opera tion
Reach-Out: Variations in
Program Evaluation Design PURE HYPOTHET1CA L-DEDUCTIVE
APPROACH TO EVALUATION:
EXPERIMENTAL DESIGN,
Let's consider design alternatives for a QUANTITATIVE DATA, AND
comprehensive program aimed at high STATISTICAL ANALYSIS
school students at high risk educa tionally
(poor grades, poor attendance, poor atti- The program does not have sufficient re-
tudes toward school), with highly vulnera- sources to serve ali targeted youth in the
ble health (poor nutrition, sedentary life- population. A pool of eligible youth is estab-
250 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

lished with admission into the program on a ing. Near the end of the program, in-depth
random basis and the remaining group re- interviews are conducted with the partici-
ceives no immediate treatment intervention. pants to learn whatbehaviors have changed,
Before the program begins and one year how they view things, and what their expec-
later, ali youth, both those in the program tations are for the future. Interviews are also
and those in the control group, are adminis- conducted with program staff and some par-
tered standardized instruments measuring ents. These data are content analyzed to
school achievement, self-esteem, anomie, identify the patterns of experiences partici-
alienation, and locus of control. Rates of pants bring to the program, what patterns
school attendance, illness, drug use, and de- characterize their participation in the pro-
linquency are obtained for each group. gram, and what patterns of change are re-
When ali data have been collected at the end por ted by and observed in the participants.
of the year, comparisons between the control
and experimental groups are made using in- MIXED FORM: EXPERIMENTAL
ferential statistics. DESIGN, QUALITATIVE DATA, AND
CONTENT ANALYSIS
PURE QUALITATIVE STRATEGY: As in the pur experimental form, poten-
NATURALISTIC INQUIRY, tial participants are randomly assigned to
QUALITATIVE DATA, AND treatment and control groups. In-depth in-
CONTENT ANALYSIS terviews are conducted with ali youth, both
those in the treatment group and those in the
Procedures for recruiting and selecting control group, and both before the program
participants for the program are determined begins and at the end of the program. Con-
entirely by the staff. The evaluator finds a tent and thematic analyses are performed so
convenient time to conduct an in-depth in- that the control and experimental group pat-
terview with new participants as soon as terns canbe compared and contrasted. (For a
they are admitted into the program, asking detailed example combining experimental
students to describe what school is like for controls and ethnography, see Maxwell,
them, what they do m school, how they typi- Bashook, and Sandlow 1987.)
cally spend their time, what their family life
is like, how they approach academic tasks,
MIXED FORM: EXPERIMENTAL
their views about health, and their behav-
DESIGN, QUALITATIVE DATA,
iors/attitudes with regard to delinquent
AND STATISTICAL ANALYSIS
and criminal activity. In brief, participants
are asked to describe themselves and their Participants are randomly assigned to
social world. The evaluator observes pro- treatment and control groups, and in-depth
gram activities, collecting detailed descrip- interviews are conducted both before the
tive data about staff-participant interactions program and at its end. These interview data,
and conversa tions, staff intervention efforts, in raw form, are then given to a panei of
and youth reactions. The evaluator finds op- judges, who rate each interview along sev-
portunities for additional in-depth inter- eral outcome dimensions operationalized as
views with participants to find out how they a 10-point scale. For both the "pre" inter-
view the program, what kinds of experi- view and the "post" interview, the judges as-
ences they are having, and what they're do- sign ratings on such dimensions as likeli-
Designing Qualitative Studies !fj. 251

hood of success in school (low - 1, high = MIXED FORM: NATURALISTIC

10), likelihood of committing criminal of- INQUIRY, QUANTITATIVE DATA,
fenses (low = 1, high = 10), commitment to AND STATISTICAL ANALYSIS
education, commitment to engaging in pro-
Students are selected for the program ac-
ductive work, self-esteem, and manifesta-
cording to staff criteria. The evaluator enters
tion of desired nutritional and health habits.
the program setting without any predeter-
Inferential statistics are then used to com-
mined categories of analysis or presupposi-
pare these two groups. Judges make the rat-
tions about important variables or variable
ings without knowledge of which partici-
relationships. The evaluator observes im-
pants were in which group. Outcomes on the
portant activities and events in the program,
rated scales are also statistically related to
looking for the types of behaviors and inter-
background characteristics of participants.
actions that will emerge. For each significant
type of behavior or interaction observed, the
MIXED FORM: NATURALISTIC evaluator creates a category and then uses a
INQUIRY, QUALITATIVE DATA, time and space sampling design to count the
AND STATISTICAL ANALYSIS frequency with which those categories of
behavior and interaction are exhibited. The
As in the pure qualitative form, students frequency of the manifestation of observed
are selected for the program on the basis of behaviors and interactions are then statisti-
whatever criteria staff members choose to cally related to such characteristics as group
apply. In-depth interviews are conducted size, duration of the activity, staff-student
with ali students before and at the end of the ratios, and social/physical density.
program. These data are then submitted to a
panei of judges, who rate them on a series of
dimensions similar to those listed in the pre- Alternative Pure
vious example. Change scores are computed and Mixed Strategies
for each individual, and changes are statisti-
cally related to background characteristics Exhibit 5.7 summarizes the six alternative
of the students to determine in a regression design scenarios we created and just re-
format which characteristics of students are viewed for evaluation of "Operation Reach-
likely to predict success in the program. In Out." As these alternative designs illustrate,
addition, observations of program activities purity of approach is only one option. In-
are rated on a set of scales developed to quiry strategies, measurement approaches,
quantify the climate attributes of activities: and analysis procedures can be mixed and
for example, the extent to which the activity matched in the search for relevant and use-
involved active or passive participation, the ful information. That said, it is worth consid-
extent to which student-teacher interaction ering the case for maintaining the integrity
was high or low, the extent to which interac- and purity of qualitative and quantitative
tions were formal or informal, and the extent paradigms. The 12 themes of qualitative
to which participants had input into pro- inquiry described in the second chapter (Ex-
gram activities. Quantitative ratings of ac- hibit 2.1) do fit together as a coherent strat-
tivities based on qualitative descriptions are egy. The openness and personal involve-
then aggregated to provide an overview of ment of naturalistic inquiry mesh well with
the treatment environment of the program. the openness and depth of qualitative data.
252 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Measurement, Design, and Analysis:

E X H I B I T 5.7 Pure and Mixed Combinations

Naturalistic Experimental <0

cg Inquiry Design
t
\ /
\
N

S ?? collect collect collect collect

S
J5 ^^ qualitative quantitative qualitative quantitative
Cv data data data data
L,

& i

perform
V
11
L I
perform perform perform
s content
analysis
statistical
analysis
content
analysis
statistical
analysis

Mixed Strategies
(middle paths)

Genuine openness flows naturally from an gies. Their cautions are not to be dismissed
inductive approach to analysis, particularly lightly. Mixing parts of different approaches
an analysis grounded in the immediacy of is a matter of philosophical and method-
direct fieldwork and sensitized to the desir- ological controversy. Yet, the practical man-
ability of holistic understanding of unique date in evaluation (Patton 1981) to gather
human settings. the most relevant possible information for
Likewise, there is an internai consistency evaluation users outweighs concerns about
and logic to experimental designs that test methodological purity based on episte-
deductive hypotheses derived from theoret- mological and philosophical arguments.
ical premises. These premises identify the The intellectual mandate to be open to what
key variables to consider in testing theory or the world has to offer surely includes meth-
measurmg, controlling, and analyzing hy- odological openness. In practice, it is alto-
pothesized relationships between program gether possible, as we have seen, to combine
treatments and outcomes. The rules and approaches, and to do so creatively (Patton
procedures of the quantitative-experimental 1987a). Just as machines that were originally
paradigm are aimed at producing internally created for separate functions such as print-
valid, reliable, replicable, and generalizable ing, faxing, scanning, and copying have
findings. now been combined into a single integrated
Guba and Lincoln (1988) have argued technological unit, so too methods that were
that the internai consistency and logic of originally created as distinct, stand-alone
each approach, or paradigm, mitigates approaches can now be combined into
against methodological mixing of different more sophisticated and multifunctional
inquiry modes and data collection strate- designs.
Designing Qualitative Studies !fj. 253

Advocates of methodological purity ar- the ideal; quasi-experimental designs often

gue that a single evaluator cannot be both represent what is possible and practical.
deductive and inductive at the same time, or Likewise, full participant observation over
cannot be testing predetermined hypotheses an extended period of time is the qualitative
and still remain open to whatever emerges ideal. In practice, many acceptable and
from open-ended, phenomenological obser- meaningful variations to qualitative inquiry
vation. Yet, in practice, human reasoning is can be designed.
sufficiently complex and flexible that it is This spirit of adaptability and creativity
possible to research predetermined ques- in designing studies is aimed at being prag-
tions and test hypotheses about certain as- matic, responsive to real-world conditions
pects of a program while being quite open and, when doing evaluations, to meeting
and naturalistic in pursuing other aspects of stakeholder information needs. Mixed
a program. In principie, this is not greatly methods and strategies allow creative re-
different from a questionnaire that includes search adaptations to particular settings and
both fixed-choice and open-ended ques- questions, though certain designs pose con-
tions. The extent to which a qualitative ap- straints that exclude other possibilities. It is
proach is inductive or deductive varies not possible, for example, for a program to
along a continuum. As evaluation fieldwork operate as an experiment by assigning par-
begins, the evaluator may be open to what- ticipants to treatment and control groups
ever emerges from the data, a discovery or while at the same time operating the pro-
inductive approach. Then, as the inquiry re- gram under naturalistic inquiry conditions
veals pattems and major dimensions of in- in which ali eligible participants enter the
terest, the evaluator will begin to focus on program (and thus there is no control group
verifying and elucidating what appears to and no random assigmnent). Another in-
be emerginga more deductively oriented compatibility: Qualitative descriptions can
approach to data collection and analysis. be converte d into quantitative scales for
The extent to which a study is naturalistic purposes of statistical analysis, but it is not
in design is also a matter of degree. This ap- possible to work the other way around and
plies particularly with regard to the extent to convert purely quantitative measures into
which the investiga tor places conceptual detailed, qualitative descriptions.
constraints on or makes presuppositions
about the program or phenomenon under
study. In practice, the naturalistic approach Design and Methods Decisions
may often involve moving back and forth
between inductive, open-ended encounters Which research design is best? Which
and more hypothetical-deductive attempts strategy will provide the most useful infor-
to verify hypotheses or solidify ideas that mation to decision makers? No simple and
emerged from those more open-ended expe- universal answer to that question is possi-
riences, sometimes even manipulating ble. The answer in each case will depend on
something to see what happens. the purpose of the study, the scholarly or
These examples of variations in qualita- evaluation audience for the study (what in-
tive approaches are somewhat like the dif- tended users want to know), the funds avail-
ferences between experimental and quasi- able, the political context, and the interests/
experimental designs. Pure experiments are abilities/biases of the researchers. Exhibit 5.8
254 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

E X H I B I T 5.8 Design Issues and Options

issues Design Options and Concems

1. What is the primary Basic research, applied research, summative evaluation,

purpose of the study? formative evaluation, action research. (See Exhibits 5.1-5.4)
2. What is the focus of study? Breadth versus depth trade-offs.
3. What are the units of Individuais, groups, program components, whole program,
analysis? organizations, communities, criticai incidents, time periods, etc.
(See Exhibit 5.5)
4. What will be the sampling Purposeful sampling, probability sampling. Variations in
strategy or strategies? sample size from a single case study to a generalizable
sample. (See Exhibit 5.6)
5. What types of data will Qualitative, quantitative, or both. (See Exhibit 5.7)
be collected?
6. What type and degree of Naturalistic inquiry (no control), experimental design, quasi-
control will be exercised? experimental.
7. What analyttcal approach Inductive, deductive. Content or thematic analysis, statistical
or approaches will be used? analysis, combinations. (See Exhibit 5.7)
8. How will the validity of and Triangulation options, multiple data sources, multiple
confidence in the findings methods, multiple perspectives, multiple investigators.
be addressed?
9. Time issues: When will the Long-term fieldwork, rapid reconnaissance, exploratory
study occur? How will the phase to confirmatory phase, fixe d times versus open timelines.
study be sequenced or
phased?
10. How will logistics and Gaining entry to the setting, access to people and records,
practicalities be handled? contracts, training, endurance, etc.
11. How will ethical issues and Informed consent, protection of human subjects, reactivity,
matters of confidentiality presentation of self, etc.
be handled?
12. What resources will be Personnel, supplies, data collection, materiais, analysis time
available? What will the and costs, reporting/publishing costs.
study cost?

summarizes the issues discussed in this gests a very specific blueprint, but "design
chapter that must be addressed in designing in the naturalistic sense . . . means planning
a study. for certain broad contingencies without,
In qualitative inquiry, the problem of de- however, indicating exactly what will be
sign poses a paradox. The term design sug- done in relation to each" (Lincoln and Guba
Designing Qualitative Studies !fj. 255

TVJO ni^IRlM PlilRSi-llcriV']:.::!'

/iLi "l j "!'!! iVi.V! ! ; .VYti': :.= ! i", .'-I W
M " iTiVi1!!; J: ni." I ..! yrt i I 3'rJ I , - . , : ! i,;; | .!= r=; i ; V ! !n j.-yp! r!i.l.Vvi i.i!- i ii I Wfr-^ * t.fi.1i"i:!'!" n.
V IV "] ! H" I J: i!!' }: i- i M \ vi ! V BFL !:!. =V=:;I "j/i ;: s 5 r i " i !=;; N; Y n '.'i !;j!!!':.!!'' V.S i, 1 M h I! ' I.:: ro ' i! I:
'V!!!'=!''=!:! ' ' T ! U ' i'i! iVii- :i!ir. JfEipr !;: I;:! ji'
.}/::!!= !!=>:= ,"::=;,!i-i<:".M ST! nViriiii 1 1 hj"j'. hU !-.!Hrr t " u - "'i ''"::<i.i ':>.i! i'iVii'riiri.lA:'I MYi!,':V!i.!!
1 1

:'!;: iXijj!!|!I I I ! . . T I Y ! ' ! ! ' i;|- :!|Y,HJJ! I ' . ' I F R ! ' ! ! H * H - R Y LI? i'I!":'.;I:"
R

' ' : M T ! !T:.|; > ; ! ; ' R.3;" ^ri.VH.' ''fiu,' kiiV ':.'' i'v. A!">r .'i 1r
- !

"fite I.ii.isii.:'.V!:' i-ri !:;!'

A I T ! : . H T F I M ' ! \ ! lI,Y>;!I!:>! .H!..IIS:S:'!Rri ! ! ' . ; ' . ! I : "P!!::r". I"'.'IVII 1 Ivi

H
. ''.i.:n"' ii'ri i'i:! ::'! ': i.1;:'!:! "i::;.;. \ \ >i !' :?Cl 1" ! T =:V 'r. r !! ^.'i I; i ! 'T- !i ,!<:Y :'. f T H ! r i i. i i 'i . i'

!h> 5 .' i v !i-:i! '.'!:'!.': %"!' ii !!i:'-ii! yiwffln-J-mV ch^^l t A i 1 .ii .\.':Vi!':; V!.': i.V.'! WVI rr rn! r *: 1l:r ','
: !

! ir!:' u:- .jrd. ;::; :!!!:: l.- "

p 5 1

rm I !:'.' i:=:i l i \. ^ViVi :i =: ;Y V! ;'. > : : H.in ;I; !=;: !,; i r M v-,fti' ==i., i,^-..! 1 > \ L.M*: "! i n!! il! i'0 |i ? " V' . n
i-ihiv.Vin V.:,'jW.::!! : ! : : j V i h :!.!i'V'!:;i m''^^hn-n =:IVsi: II: ^V n! n i:l-yi 1 i;v.:;s =j" n-i>
lVi.Vii! !:!: !3' i ;V ! H: t:;y :;i jj-! n^.Ti'.: ?}, ; " T; ' f,Wi I'- A i . S! i I!1: "'

1985:226). A qualitative design needs to re- writer, have their favorite research methods
main sufficiently open and flexible to permit with which they are familiar and have some
exploration of whatever the phenomenon skill in using. And I suspect we mostly choose
under study offers for inquiry. Qualitative to investigate problems that seem vulnerable
designs continue to be emergent even after to attack through these methods. But we
data collection begins. The degree of flexibil- should at least try to be less parochial than
ity and openness is, however, also a matter cobblers. Let us be done with the arguments of
of great variation among designs. participant observation versus interviewing
What is certain is that different methods as we have largely dispensed with the argu-
can produce quite different findings. The ments for psychology versus sociologyand
challenge is to figure out which design and get on with the business of attacking our
methods are most appropriate, productive, problems with the widestarray of conceptual
and useful in a given situation. Martin Trow and methodological tools that we possess and
(1970) points out (quite nicely, I think) the they demand. This does not preclude discus-
difference between arguments about which sion and debate regarding the relatve useful-
methods are most appropriate for studying ness of different methods for the study of
a particular problem and arguments about specific problems or types of problems. But
the intrinsic and universal superiority of one that is very different from the assertion of the
method over another: general and inherent superiority of one
method over another on the basis of some
Every cobbler thinks leather is the only thing. intrinsic qualities it presumably possesses.
Most social scientists, including the present (P-149)
256 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Professor, for the last

Sounds like it's
four years I've been time to take it to
struggling to define my the next levei.
research problem, but it Fuzzy methods.
still seems too fuzzy.

Sophisticated emergent design strategy

Designing Qualitative Studies !fj. 257

Choices

very path we take leads to fantasies about the path not taken.

Halcolm

This chapter has suggested that research will be able to make the same rational deci-
and evaluation should be built on the foun- sion that I have made.
dation of a "paradigm of choices" rather "Honey comes conveniently packaged in
than become the handmaiden of any single beautifully shaped prisms of the most deli-
and inevitably narrow disciplinary or meth- cate texture. It's ready to eat, slides down the
odological paradigm. But be careful, the throat ever so easily, is a highly nutritious
Sufis would warn us, for the exercise of real source of energy, digests smoothly, and
choice can be elusive. Trow admonishes us leaves a lingering taste of sweetness on the
to "at least try to be less parochial than cob- palate that provides pleasure for hours.
blers." The evaluation sage, Halcolm, might Honey is readily available and requires no
suggest that ali too often the methods deci- special labor to produce since bees do ali the
sions made become like the bear's "deci- work. Its pleasing aroma, light weight, resis-
sion" to like honey. tance to spoilage, and uniformly high qual-
One day, in a sudden impulse of generos- ity make it a food beyond compare. It comes
ity, a bear decided to enlighten the other ani- ready to consumeno peeling, no killing, no
mais in the forest about the marvelous prop- tearing openand there's no waste. Whafs
erties of honey. The bear assembled ali the more, it has so many uses; it can be eaten
other animais together for his momentous alone or added to enhance any other food.
announcement. "I could go on and on, but suffice it to say
"I have studied the matter at great that I have studied the situation quite objec-
length," began the bear, "and I have decided tively and at great length. A fair and rational
that honey is the best of ali foods. Therefore, analysis leads to only one conclusion.
I have chosen to like honey I am going to de- Honey is the supreme food and any reason-
scribe to you the perfect qualities of honey, able animal will undoubtedly make the
which, due to your past prejudices and lack same conscious decision I have made. I have
of experience, you have ignored. Then you chosen to like honey."
Fieldwork Strategies
and Observation Methods

X o LAnde^s+and +ke W o ^ l d

And the children said unto Halcolm, "We want to understand the world. Tell
us, O Sage, what must we do to know the world?"
"Have you read the works of our great thinkers?"
"Yes, Master, every one of them as we were instructed."
"And have you practiced diligently your meditations so as to become One
with the infinity of the universe?"
"We have, Master, with devotion and discipline."
"Have you studied the experiments, the surveys, and the mathematical mod-
els of the Sciences?"
"Beyond even the examinations, Master, we have studied in the innermost
chambers where the experiments and surveys are analyzed, and where the
mathematical models are developed and tested."
"Still you are not satisfied? You would know more?"
"Yes, Master. We want to understand the world."
"Then, my children, you must go out into the world. Live among the peoples
of the world as they live. Learn their language. Participate in their rituais and
routines. Taste of the world. Smell it. Watch and listen. Touch and be touched.
Write down what you see and hear, how they think and how you feel.
"Enter into the world. Observe and wonder. Experience and reflect. To under-
stand a world you must become part of that world while at the same time re-
maining separa te, a part of and apart from.
"Go then, and return to tell me what you see and hear, what you learn, and
what you come to understand."
From Halcolm's Methodological Chronicle

l 259
260 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Folk Wisdom About Human Observation

W
mna
I n the fields of observation, chance favors the prepared mind.

Louis Pasteur (1822-1895)

P eople only see what they are prepared to see.

Ralph Waldo Emerson (1803-1882)

Every student who takes an introductory range, and lasted only twenty seconds. But the
psychology or sociology course learns that observers could not observe ali that hap-
human perception is highly selective. When pened. Some readers chuckled because the ob-
looking at the same scene or object, different servers were researchers, but similar experi-
people will see different things. What people ments have been reported numerous times.
"see" is highly dependent on their inter- They are alike for ali kinds of people. (Katzer
ests, biases, and backgrounds. Our culture et al. 1978:21-22)
shapes what we see, our early childhood so-
cialization forms how we look at the world, Using this story to cast doubt on ali vari-
and our value systems tell us how to inter- eties of observational research manifests
pret what passes before our eyes. How, then, two fundamental fallacies: (1) These re-
can one trust observational data? searchers were not trained as social science
In their classic guide for users of social observers, and (2) they were not prepared to
science research, Katzer, Cook, and Crouch make observations at that particular mo-
(1978) titled their chapter on observation ment. Scientific inquiry using observa-
"Seeing Is Not Believing." They open with tional methods requires disciplined train-
an oft-repeated story meant to demonstrate ing and rigorous preparation.
the problem with observational data. The fact that a person is equipped with
functioning senses does not make that per-
Once at a scientific meeting, a man suddenly son a skilled observer. The fact that ordinary
rushed into the midst of one of the sessions. persons experiencing any particular inci-
Another man with a revolver was chasing dent will highlight and report different
him. They scuffled in plain view of the assem- things does not mean that trained and pre-
bled researchers, a shot was fired, and they pared observers cannot report with accuracy,
rushed out. About twenty seconds had authenticity, and reliability that same inci-
elapsed. The chairperson of the session imrne- dent.
diately asked ali present to write down an ac- Training to become a skilled observer in-
count of what they had seen. The observers cludes
did not know that the ruckus had been
planned, rehearsed, and photographed. Of the B learning to pay attention, see what there
forty reports turned in, only one was less than is to see, and hear what there is hear;
20 percent mistaken about the principal facts,
and most were more than 40 percent mistaken. H practice in writing descriptively;
The event surely drew the undivided atten- H acquiring discipline in recording field
tion of the observers, was in full view at close notes;
Fieldwork Strategies and Observation Methods LET. 261

0 knowing how to separate detail from and ears, my observational senses. A scien-
trivia to achieve the former without be- tific observer cannot be expected to engage
ing overwhelmed by the latter; in systematic observation on the spur of the
using rigorous methods to validate and moment any more than a world-class boxer
triangulate observations; and can be expected to defend his title spontane-
ously on a street corner or an Olympic run-
reporting the strengths and limitations ner can be asked to dash off at record speed
of one's own perspective, which requires because someone suddenly thinks it would
both self-knowledge and self-disclosure. be nice to test the runner's time. Athletes,
artists, musicians, dancers, engineers, and
Training observers can be particularly scientists require training and mental prepa-
challenging because so many people think ration to do their best. Experiments and sim-
that they are "natural" observers and there- ulations that document the inaccuracy of
fore have little to learn. Training to become a spontaneous observations made by un-
skilled observer is a no less rigorous process trained and unprepared observers are no
than the training necessary to become a more indicative of the potential quality of
skilled survey researcher or statistician. Peo- observational methods than an amateur
ple don't "naturally" know how to write community talent show is indicative of what
good survey items or analyze statistics professional performers can do.
and people don't "naturally" know how to Two points are criticai, then, in this intro-
do systematic research observations. Ali ductory section. First, the folk wisdom about
forms of scientific inquiry require training observation being nothing more than selec-
and practice. tive perception is true in the ordinary course
Careful preparation for entering into of participating in day-to-day events. Sec-
fieldwork is as important as disciplined ond, the skilled observer is able to improve
training. Though I have considerable experi- the accuracy, authenticity, and reliability of
ence doing observa tional fieldwork, had I observations through intensive training and
been present at the scientific meeting where rigorous preparation. The remainder of this
the shooting scene occurred my recorded chapter is devoted to helping evaluators and
observations might not have been signifi- researchers move their observations from
cantly more accurate than those of my less the levei of ordinary looking to the rigor of
trained colleagues because I would not have systematic seeing.
been prepared to observe what occurred and,
lacking that preparation, would have been
The Value of
seeing things through my ordinary eyes
Direct Observations
rather than my scientific observer's eyes.
Preparation has mental, physical, intel- I'm often asked by students: "Isn't inter-
lectual, and psychological dimensions. Pas- viewing just as good as observation? Do you
teur said, "In the fields of observation, really have to go see a program directly to
chance favors the prepared mind." Part of evaluate it? Can't you find out ali you need
preparing the mind is learning how to con- to know by talking to people in the program
centrate during the observation. Observa- without going there and seeing it first-
tion, for me, involves enormous energy and hand?"
concentration. I have to "turn on" that con- I reply by relating my experience evaluat-
centration"turn on" my scientific eyes ing a leadership development program with
262 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

two colleagues. As part of a formative evalu- the reader to enter into and understand the
ation aimed at helping staff and funders situation described. In this way, evaluation
clarify and improve the program's design users, for example, can come to understand
before undertaking a comprehensive fol- program activities and impacts through de-
low-up study for a summative evaluation, tailed descriptive information about what
we went through the program as participant has occurred in a program and how the peo-
observers. After completing the six-day ple in the program have reacted to what has
leadership retreat, we met to compare expe- occurred.
riences. Our very first conclusion was that Naturalistic observations take place in the
we would never have understood the pro- field. For ethnographers, the field is a cul-
gram without personally experiencing it. It tural setting. For qualitative organizational
bore little resemblance to our expectations, development researchers, the field will be an
what people had told us, or the official pro- organization. For evaluators, the field is the
gram description. Had we designed the fol- program being studied. Many terms are
low-up study without having participated used for talking field-based observations in-
in the program, we would have completely cluding participant observation, fieldwork,
missed the mark and asked inappropriate qualitative observation, direct observation, and
questions. To absorb the program/s lan- field research. "Ali these terms refer to the cir-
guage, understand nuances of meaning, ap- cumstance of being in or around an on-going
preciate variations m participants7 experi- social setting for the purpose of making a
ences, capture the importance of what qualitative analysis of that setting" (Lofland
happened outside formal activities (during 1971:93),
breaks, over meals, in late-night gatherings Direct, personal contact with and obser-
and parties), and feel the intensity of the re- vations of a setting have several advantages.
treat environmentnothing could have First, through direct observations the in-
substituted for direct experience with the quirer is better able to understand and cap-
program. Indeed, what we observed and ture the context within which people inter-
experienced was that participants were act. Understanding context is essential to a
changed as much or more by what hap- holistic perspective.
pened outside the formal program structure Second, firsthand experience with a set-
and activities as by anything that happened ting and the people in the setting allows an
through the planned curriculum and exer- inquirer to be open, discovery oriented, and
cises. inductive because, by being on-site, the ob-
The first-order purposes of observational server has less need to rely on prior concep-
data are to describe the setting that was ob- tualizations of the setting, whether those
served, the activities that took place in that prior conceptualizations are from written
setting, the people who participated in those documents or verbal reports.
activities, and the meanings of what was ob- A third strength of observational field-
served from the perspectives of those ob- work is that the inquirer has the opportunity
served. The descriptions should be factual, to see things that may routinely escape
accurate, and thorough without being clut- awareness among the people in the setting.
tered by irrelevant minutiae and trivia. The For someone to provide information in an
quality of observational reports is judged by interview, he or she must be aware enough
the extent to which that observation permits to report the desired information. Because
Fieldwork Strategies and Observation Methods LET. 263

ali social systems involve routines, partici- ural result of the bonding among partici-
pants in those routines may take them so pants. We learned that neither explanation
much for granted that they cease to be aware was true. What actually occurred was that,
of important nuances that are apparent only unbeknownst to program staff, the dining
to an observer who has not become fully im- hostess for the hotel where participants
mersed in those routines. stayed initiated the roast. After the second
The participant observer can also dis- evening's meai, when staff routinely de-
cover things no one else has ever really paid parted for a meeting, the hostess would tell
attention to. One of the highlights of the participants what was expected. She even
leadership training program we experi- brought out a photo lbum of past ban-
enced was the final evening banquet at quets and offered to supply joke books, cos-
which staff was roasted. For three nights, tumes, music, or whatever. This 60-year-old
after training ended, participants worked to woman had begun playing what amounted
put together a program of jokes, songs, and to a major staff role for one of the most im-
skits for the banquet. Staff were never portant processes in the programand the
around for these preparations, which lasted staff didn't know about it. We learned about
late into the night, but they had come to it by being there.
count on this culminating event. Month after A fourth value of direct observation is the
month for two years each completely new chance to learn things that people would be
training group had organized a final ban- unwilling to talk about in an interview. In-
quet event to both honor and make fun of terviewees may be unwilling to provide in-
staff. Staff assumed that either prior partici- formation on sensitive topics, especially to
pants passed on this tradition or it was a nat- strangers. A fifth advantage of fieldwork is
264 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

the opportunity to move beyond the selec- succeeding observations. (Becker and Geer
tive perceptions of others. Interviews pres- 1970:32)
ent the understandings of the people being
interviewed. Those understandings consti- Observation-Based
tute important, indeed criticai, information.
Evaluation and Applied
However, it is necessary for the inquirer to
Research in a Political World
keep in mind that interviewees are always
reporting perceptionsselective percep- The preceding review of the advantages
tions. Field observers will also have selective of fieldwork strikes me as fairly straightfor-
perceptions. By making their own percep- ward but a bit abstract. In a moment, well
tions part of the dataa matter of training, consider the details of how to do fieldwork,
discipline, and self-awarenessobservers but to inform that transition and reinforce
can arrive at a more comprehensive view of the importance of direct observation in the
the setting being studied than if forced to real world, let me offer a perspective from
rely entirely on secondhand reports through the world of children's stories. Some of the
interviews. most delightful, entertaining, and suspense-
Finally, getting close to the people in a set- ful fairy tales and fables concern tales of
ting through firsthand experience permits kings who discard their royal robes to take
the inquirer to draw on personal knowledge on the apparel of peasants so that they can
during the formal interpretation stage of move freely among their people to really un-
analysis. Reflection and introspection are derstand what is happening in their king-
important parts of field research. The im- doms. Our modern-day kings and political
pressions and feelings of the observer be- figures are more likely to take television
come part of the data to be used in attempt- crews with them when they make excur-
ing to understand a setting and the people sions among the people. They are unlikely to
who inhabit it. The observer takes in infor- go out secretly disguised, moving through
mation and forms impressions that go be- the streets anonymously, unless they're up
yond what can be fully recorded in even the to mischief. It is left, then, to applied re-
most detailed field notes. searchers and evaluators to play out the fa-
ble, to take on the appropriate appearance
Because [the observer] sees and hears the peo- and mannerisms that will permit easy
ple he studies in many situations of the kind movement among the people, sometimes se-
that normally occur for them, rather than just cretly, sometimes openly, but always with
in an isolated and formal interview, he builds the purpose of better understanding what
an ever-growing fund of impressions, many the world is really like. They are then able to
of them at the subliminal levei, which give report those understandings to our mod-
him an extensive base for the interpretation ern-day version of kings so that policy wis-
and analytic use of any particular datum. This dom canbe enhanced and programmatic de-
wealth of information and impression sen- cisions enlightened. At least thafs the
sitizes him to subtleties which might pass fantasy. Turning that fantasy into reality in-
unnoticed in an interview and forces him to volves a number of important decisions
raise continually new and different questions, about what kind of fieldwork to do. We turn
which he brings to and tries to answer in now to those decisions.
Fieldwork Strategies and Observation Methods LET. 2 6 5

L Variations in Observational Methods

W e shall not cease from exploration

And the end of ali our exploring
Will be to arrive where we started
And know the place for the first time.1

T. S. Eliot (1888-1965)

Observational research explores the along the continuum between these two end
world in many ways. Deciding which obser- points.
vational approaches are appropriate for Nor is it simply a matter of deciding at the
evaluation or action research involves dif- beginning how much the observer will par-
ferent criteria than those same decisions ticipate. The extent of participation can
made to undertake basic social scientific re- change over time. In some cases, the re-
search. These differences emerge from the searcher may begin the study as an onlooker
nature of applied research, the politics of and gradually become a participant as field-
evaluation, the nature of contract funding in work progresses. The opposite can also oc-
most evaluations, and the accountability of cur. An evaluator might begin as a complete
evaluators to information users. Thus, while participant to experience what it is like to be
field methods have their origins in basic an- initially immersed in the program and then
thropological and sociological field meth- gradually withdraw participation over the
ods, using these methods for evaluation of- period of the study until finally taking the
ten requires adaptation. The sections that role of occasional observer from an onlooker
follow will discuss both the similarities and stance.
differences between evaluation field meth- Full participant observation constitutes
ods and basic research field methods. an omnibus field strategy in that it "simulta-
neously combines document analysis, inter-
viewing of respondents and inforinants, di-
Variations in Observer rect participation and observation, and
Involvement: Participant introspection" (Denzin 1978b: 183). If, on the
or Onlooker or Both? other hand, an evaluator observes a pro-
gram as an onlooker, the processes of obser-
The first and most fundamental distinc- vation can be separated from interviewing.
tion that differentiates observational strate- In participant observation, however, no
gies concerns the extent to which the ob- such separation exists. Typically, anthropo-
server will be a participant in the setting logical fieldworkers combine in their field
being studied. This involves more than a notes data from personal, eyewitness obser-
simple choice between participation and vation with information gained from infor-
nonparticipation. The extent of participa- mal, natural interviews and informants' de-
tion is a continuum that varies from com- scriptions (Peito and Peito 1978:5). Thus, the
plete immersion in the setting as full partici- participant observer employs multiple and
pant to complete separation from the setting overlapping data collection strategies: being
as spectator, with a great deal of variation fully engaged in experiencing the setting
266 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

(participation) while at the same time ob- can become a full participant. For example, a
servmg and talking with other participants researcher who is not chemically dependent
about whatever is happening. will not be able to become a full participant,
In the leadership program I evaluated physically and psychologically, in a chemi-
through participant observation, I was a full cal dependency program, even though it
participant in ali exercises and program ac- may be possible to participate in the pro-
tivities using the field of evaluation as my gram as a client. Such participation in a treat-
leadership arena (since ali participants had ment program can lead to important in-
to have an arena of leadership as their focus). sights and understanding about what it is
As did other participants, I developed close like to be in the program; however, the eval-
relationships with some people as the week uator must avoid the delusion that partici-
progressed, sharing meals and conversing pation has been complete. This point is illus-
late into the night. I sometimes took detailed trated by an exchange between an inmate
notes during activities if the activity permit- and a student who was doing participant ob-
ted (e.g., group discussion), while at other servation in a prison.
times I waited until later to record notes
(e.g., after meals). If a situation suddenly be- Inmate: "What are you in here for, man?"
came emotional, for example during a small
Student: "I'm here for a while to find out
group encounter, I would cease to take notes
what it's like to be in prison."
so as to be fully present as well as to keep my
note taking from becoming a distraction. Inmate: "What do you mean 'find out
Unlike other participants, I sat in on staff what it's like' ?"
meetings and knew how staff viewed what
Evaluator: 'Tm here so that I can experi-
was going on. Much of the time I was fully
ence prison from the inside instead of just
immersed in the program experience as a
studying what it's like from out there."
participant, but I was also always aware of
my additional role as evaluation observer. Inmate: "You got to be jerkin' me off, man.
The extent to which it is possible for an 'Experience from the inside . . . ' ? Shit,
evaluator to become a participant in a pro- man, you can go home when you decide
gram will depend partly on the nature of the you've had enough can't you?"
program. In human service and education
Evaluator: "Yeah."
programs that serve children, the evaluator
cannot participate as a child but may be able Inmate: "Then you ain't never gonna know
to participate as a volunteer, parent, or staff what it's like from the inside."
member in such a way as to develop the per-
spective of an insider in one of those adult Social, cultural, political, and interper-
roles. Gender can create barriers to partici- sonal factors can limit the nature and degree
pant observation. Males can't be partici- of participation in participant observation.
pants in female-only programs (e.g., bat- For example, if the participants in a program
tered women's shelters). Females doing ali know each other intimately they may ob-
fieldwork m nonliterate cultures may not be ject to an outsider trying to become part of
permitted access to male-only councils and their close circle. Where marked social class
ceremonies. Programs that serve special differences exist between a sociologist and
populations may also involve natural limita- people in a neighborhood, access will be
tions on the extent to which the evaluator more difficult; likewise, when, as is often the
Fieldwork Strategies and Observation Methods LET. 267

case, an evaluator is well educated and mid- search, the purpose, scope, length, and set-
dle class while welfare program clients are ting for the study will dictate the range and
economically disadvantaged and poorly ed- types of participant observation that are
ucated, the participants in the program may possible.
object to any ruse of "full" participant obser- One final caution: The researcher 's plans
vation. Program staff will sometimes object and intentions regarding the degree of pro-
to the additional burden of including an gram involvement to be experienced may
evaluator in a program where resources are not be the way things ac tually turn out. Lang
limited and an additional participant would and Lang (1960) report that two scientific
unbalance staff-client ratios. Thus, in evalu- participant observers who were studying
ation, the extent to which full participation is audience behavior at a Billy Graham evan-
possible and desirable will depend on the gelical crusade made their "decision for
precise nature of the program, the political Christ" and left their observer posts to walk
context, and the nature of the evaluation down the aisle and join Reverend Graham's
questions being asked. Adult training pro- campaign. Such are the occupational haz-
grams, for example, may permit fairly easy ards (or benefits, depending on your per-
access for full participation by evaluators. spective) of real-world fieldwork.
Offender treatment programs are much less
likely to be open to participant observation
Insider and Outsider Perspectives:
as an evaluation method. Evaluators must
therefore be flexible, sensitive, and adaptive Emic Versus Etic Approaches
in negotiating the precise degree of partici-
pation that is appropriate in any particular People who are insiders to a setting being
observational study, especially where re- studied often have a view of the setting and
porting timelines are constrained so entry any findings about it quite different from that
into the setting must be accomplished rela- of the outside researchers who are conducting
tively quickly. Social scientists who can take the study. (Bartunek and Louis 1996)
a long time to become integrated into the set-
ting under study have more options for Ethnosemanticist Kenneth Pike (1954)
fuller participant observation. coined the terms emic and etic to distinguish
As these examples illustrate, full and classification systems reported by anthro-
complete participation in a setting, what is pologists based on (1) the language and cate-
sometimes called "going native," is fairly gories used by the people in the culture stud-
rare, especially for a program evaluation. ied, an emic approach, in contrast to (2)
Degree of participation and nature of obser- categories created by anthropologists based
vation vary along a wide continuum of on their analysis of important cultural dis-
possibilities. The ideal in evaluation is to tinctions, an etic approach. Leading anthro-
design and negotiate that degree of partici- pologists such as Franz Boas and Edward
pation that will yield the most meaningful Sapir argued that the only meaningful dis-
data about the program given the charac- tinctions were those made by people within
teristics of the participants, the nature of a culture, that is, from the emic perspective.
staff-participant interactions, the socio- However, as anthropologists turned to more
political context of the program, and the in- comparative studies, engaging in cross-cul-
formation needs of intended evaluation tural analyses, distinctions that cut across
users. Likewise, in applied and basic re- cultures had to be made based on the anthro-
268 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

pologisfs analy tical perspective, that is, an emic perspective. This means that the partic-
etic perspective. The etic approach involved ipant observer not only sees what is
"standing far enough away from or outside happeningbut feels what it is like to be a part
of a particular culture to see its separate of the setting or program. Anthropologist
events, primarily in relation to their similari- Hortense Powdermaker (1966) has de-
ties and their differences, as compared to scribed the basic assumption undergirding
events in other cultures" (Pike 1954:10). For participant observation as follows: "To un-
some years a debate raged in anthropology derstand a society, the anthropologist has
about the relative merits of emic versus etic traditionally immersed himself in it, learn-
perspectives (Peito and Peito 1978:55-60; ing, as far as possible, to think, see, feel and
Headland, Pike, and Harris 1990), but, as of- sometimes act as a member of its culture and
ten happens over time, both approaches at the same time as a trained anthropologist
came to be understood as valuable, though from another culture" (p. 9).
each contributes something different. Nev- Experiencing the setting or program as an
ertheless, tension between these perspec- insider accentuates the participant part of
tives remains: participant observation. At the same time,
the inquirer remains aware of being an out-
Today, despite or perhaps because of the new sider. The challenge is to combine participa-
recognition of cultural diversity, the tension tion and observation so as to become capa-
between universalistic and relativistic values ble of understanding the setting as an
remains an uriresolved conundrum for the insider while describing it to and for outsid-
Western ethnographer. In practice, it becomes ers.
this question: By which values are observa-
tions to be guided? The choices seem to be ei- Obtaining something of the understanding of
ther the values of the ethnographer or the an insider is, for most researchers, only a first
values of the observedthat is, in modem step. They expect, in time, to become capable
parlance, either the etic or the emic.... Herein of thinking and acting within the perspective
lies a deeper and more fundamental problem: of two quite different groups, the one m which
How is it possible to understand the other they were reared andto some degreethe
when the other's values are not one's own? one they are studying. They will also, at times,
This problem arises to plague ethnography at be able to assume a mental position peripheral
a time when Western Christian values are no to both, a position from which they willbe able
longer a surety of truth and, hence, no longer to perceive and, hopefully, describe those rela-
the benchmark from which self-confidently tionships, systems and pattems of which an
valid observations can be made. (Vidich and inextricably involved insider is not likely to be
Lyman 2000:41) consciously aware. For what the social scien-
tist realizes is that while the outsider simply
Methodologically, the challenge is to do jus- does not know the meanings or the pattems,
tice to both perspectives during and after the insider is so immersed that he may be
fieldwork and to be clear with one's self and oblivious to the fact that pattems exist. . . .
one's audience how this tension is managed. What fieldworkers eventually produce out of
A participant observer shares as inti- the tension developed by this ability to shift
mately as possible in the life and activities of their point of view depends upon their sophis-
the setting under study in order to develop tication, ability, and training. Their task, in any
an insider's view of what is happening, the case, is to realize what they have experienced
Fieldwork Strategies and Observation Methods LET. 269

and learned and to communicate this in terms useful, a supplementary agenda is often to
that will illurrne. (Wax 1971:3) increase participants' sense of being in con-
trol of, deliberative about, and reflective on
their own lives and situations. Chapter 4 dis-
Who Conducts the Inquiry? cussed these approaches as examples of how
Solo and Team Versus Participatory qualitative inquiry can be applied in sup-
and Collaborative Approaches port of organizational or program develop-
ment and community change.
The ultimate in insider perspective Degrees of collaboration vary along a
comes from involving the insiders as continuum. At one end is the solo field-
coresearchers through collaborative or par- worker or a team of professionals; what
ticipatory research. Collaborative forms of characterizes this end of the continuum is
fieldwork, participatory action research, that researchers completely control the in-
and empowerment approaches to evalua- quiry. At the other end are collaborations
tion have become sufficiently important and with people in the setting being studied,
widespread to make degree of collaboration a sometimes called "coresearchers"; they help
dimension of design choice in qualitative in- design the inquiry, collect data, and are in-
quiry. Participatory action research has a volved in analysis. Along the middle of the
long and distinguished history (Kemmis continuum are various degrees of partial
and McTaggart 2000; Whyte 1989). Collabo- and periodic (as opposed to continuous)
rative principies of feminist inquiry include collaboration.
connectedness and equality between re-
searchers and researched, participatory pro- Overt Versus Covert Observations
cesses that support consciousness-raising
and researcher reflexivity, and knowledge A traditional concern about the validity
generation that contributes to women's lib- and reliability of observational data has
eration and emancipation (Olesen 2000; been the effects of the observer on what is
Guerrero 1999a:15-22; Thompson 1992). In observed. People may behave quite differ-
evaluation, Cousins and Earl (1995) have ad- ently when they know they are being ob-
vocated participatory and collaborative ap- served versus how they behave naturally
proaches to evaluation primarily to increase when they don't think they're being ob-
use of findings. Empowerment evaluation, served. Thus, the argument goes, covert ob-
often using qualitative methods (Fetterman servations are more likely to capture what is
2000a; Fetterman, Kaftarian, and Wanders- really happening than are overt observa-
man 1996), involves the use of evaluation tions where the people in the setting are
concepts and techniques to foster self-deter- aware they are being studied.
mination and help people help themselves Researchers have expressed a range of
by learning to study and report on their own opinions concerning the ethics and morality
issues and concerns. of conducting covert research, what Mitchell
What these approaches have in common (1993:23-35) calls "the debate over secrecy."
is a style of inquiry in which the researcher One end of the continuum is represented by
or evaluator becomes a facilitator, collabora- Edward Shils (1959), who absolutely op-
tor, and teacher in support of those engaging posed ali forms of covert research including
in their own inquiry. While the findings "any observations of private behavior, how-
from such a participatory process may be ever technically feasible, without the explicit
270 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

and fully informed permission of the person problems are (1) misinformation, (2) evasions,
to be observed." He argued that there (3) lies, and (4) fronts. (Douglas 1976:55, 57)
should be full disclosure of the purpose of
any research project and that even partici- Just as degree of participation in field-
pant observation is "morally obnoxious . . . work turned out to be a continuum of varia-
manipulation" unless the observer makes tions rather than an all-or-none proposition,
explicit his or her research questions at the so too is the question of how explicit to be
very beginning of the observation (Shils about the purpose of fieldwork. The extent
1959, quoted in Webb et ai. 1966:vi). to which participants in a program under
study are informed that they are being ob-
At the other end of the continuum is the
served and are told the purpose of the re-
"investigative social research" of Jack
search has varied historically from full dis-
Douglas (1976). Douglas argued that con-
closure to no disclosure, with a great deal of
ventional anthropological field methods
variation along the middle of this contin-
have been based on a consensus view of so-
uum (Junker 1960). Discipline-based ethics
ciety that views people as basically coopera-
statements (e.g., American Psychological
tive, helpful, and willing to have their points
Association, American Sociological Associ-
of view understood and shared with the rest
ation) now generally condemn deceitful and
of the world. In contrast, Douglas adopted a
covert research. Likewise, institutional re-
conflict paradigm of society that led him to
view board (IRB) procedures for the protec-
believe that any and ali covert methods of re-
tion of human subjects have severely con-
search should be considered acceptable op-
strained such methods. They now refuse to
tions in a search for truth.
approve protocols in which research partici-
pants are deceived about the purpose of a
The investigative paradigm is based on the as- study, as was commonly done in early psy-
sumption that profound conflicts of interest, chological research. One of the more infa-
values, feelings and actions pervade social mous examples was Stanley Milgrain's New
life. It is taken for granted that many of the Haven experiments aimed at studying
people one deals with, perhaps ali people to whether ordinary people would follow the
some extent, have good reason to hide from orders of someone in authority by having
others what they are doing and even to lie to these ordinary citizens administer what
them. Instead of trusting people and expect- they were told were behavior modification
ing trust in return, one suspects others and ex- electric shocks to help students learn, shocks
pects others to suspect him. Conflict is the that appeared to the unsuspecting citizens to
reality of life; suspicion is the guiding princi- go as high as 450 volts despite the screams
p i e It's a war of ali and no one gives anyone and protests heard from supposed students
anything for nothing, especially truth on the other side of a wall. The real purpose
Ali competent adults are assumed to know of the study, participants later learned, was
that there are at least four major problems ly- to replicate Nazi prison guard behavior
ing in the way of getting at social reality by among ordinary American citizens (Mil-
asking people what is going on and that these gram 1974).
problems must be dealt with if one is to avoid IRBs also refuse to approve research in
being taken in, duped, deceived, used, put on, which people are observed and studied
fooled, suckered, made the patsy, left holding without their knowledge or consent, as in
the bag, fronted out and so on. These four the infamous Tuskegee Experiment. For
Fieldwork Strategies and Observation Methods LET. 271

40 years, physicians and medicai research- missioned and used by colonial administra-
ers, under the auspices of the U.S. Public tors to maintain control over indigenous
Health Service, studied untreated syphilis peoples. Protection of human subjects pro-
among Black men in and around the county cedures are now an affirmation of our com-
seat of Tuskegee, Alabama, without the in- mitment to treat ali people with respect. And
formed consent of the men studied, men that is as it should be. But the necessity for
whose syphilis went untreated so that the such procedures comes out of a past littered
progress of the disease could be docu- with scientific horrors for which those of us
mented (Jones 1993). Other stories of abuse engaging in research today may still owe
and neglect by researchers doing covert penance. At any rate, we need to lean over
studies abound. In the late 1940s and early backward to be sure that such history is truly
1950s, schoolboys at the Walter E. Fernald behind usand that means being ever vigi-
State School in Massachusetts were rou- lant in fully informing and protecting the
tinely served breakfast cereal doused with people who honor us by agreeing to partici-
radioactive isotopes, without permission of pate in our research, whether they be home-
the boys or their guardians, for the disserta- less mothers (Connolly 2000) or corporate
tion of a doctoral student in nutritional bio- executives (Collins 2001).
chemistry. In the 1960s, the U.S. Army se- However, not ali research and evaluation
cretly sprayed a potentially hazardous falls under IRB review, so the issue of what
chemical from downtown Minneapolis roof- type and how much disclosure to make re-
tops onto unsuspecting citizens to find out mains a matter of debate, especially where
how toxic materiais might disperse during the inquiry seeks to expose the inner work-
biological warfare. Native American chil- ings of cults and extremist groups, or those
dren on the Standing Rock Sioux Reserva- whose power affects the public welfare, for
tion in the Dakotas were used to test an un- example, corporations, labor union boards,
approved and experimental hepatitis A political parties, and other groups with
vaccine without the knowledge or approval wealth and/or power. For example, Maurice
of their parents. In the 1960s and 1970s, sci- Punch (1985, 1989, 1997), formerly of the
entists teste d skin treatments and drugs on Nijenrode Business School in the Nether-
prisoners in a Philadelphia county jail with- lands, has written about the challenges of
out informing them of potential dangers. doing ethnographic studies of corruption in
Doctoral students frustrated by having both private and public sector organiza-
their fieldwork delayed while they await tions, notably the police.
IRB approval need to remember that they One classic form of deception in field-
are paying for the sins of their research fore- work involves pretending to share values
bears for whom deception and covert obser- and beliefs in order to become part of the
vations were standard ways of doing their group being studied. Sociologist Richard
work. Those most subject to abuse were of- Leo carefully disguised his liberal political
ten the most vulnerable in societychil- and social views, instead feigning conserva-
dren, the poor, people of color, the sick, peo- tive beliefs, to build trust with police and
ple with little education, women and men thereby gain admission to interrogation
incarcerated in prisons and asylums, and rooms (Allen 1997:32). Sociologist Leon
children in orphanages or state correctional Festinger (1956) infiltrated a doomsday cult
schools. Anthropological research was com- by lying about his profession and pretend-
272 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

ing to believe in the cult's prophecies. Sociol- know, and, of course, there are the classic sit-
ogist Laud Humphreys (1970) pretended to uations where everyone involved knows
be gay to gather data for his dissertation on that a study is being done and who the eval-
homosexual encounters in public parks. An- uator isbut the evaluator doesn't know
thropologist Carolyn Ellis (1986) pretended that everyone else knows.
to be just visiting friends when she studied a In undertaking participant observation of
Chesapeake Bay fishing culture. Her nega- the community leadership program men-
tive portrayals made their way back to the tioned earlier, my two evaluation colleagues
local people, many of whom were infuri- and I agreed with the staff to downplay our
ated. She later expressed remorse about her evaluation roles and describe ourselves as
deceptions (Allen 1997). "educational researchers" interested in
In traditional scholarly fieldwork, the de- studying the program. We didn't want par-
cision about the extent to which observa- ticipants to think that they were being evalu-
tions would be covert was made by re- ated and therefore worry about our judg-
searchers balancing the search for truth ments. Our focus was on evaluating the
against their sense of professional ethics. In program, not participants, but to avoid in-
evaluation research, the information users creasing participant stress we simply at-
for whom the evaluation is done have a tempted to finesse our evaluation role by
stake in what kind of methods are used, so calling ourselves educational researchers.
the evaluator alone cannot decide the extent Our careful agreement on and rehearsal
to which observations and evaluation pur- of this point with the staff fell apart during
poses will be fully disclosed. Rather, the introductions (at the start of the six-day re-
complexities of program evaluation mean treat) when the program director proceeded
that there are several leveis at which deci- to tell participantsfor 10 minutesthatwe
sions about the covert-overt nature of evalu- were just participants and they didn't have
ation observations must be made. Some- to worry about our evaluating them. The
times only the funders of the program or of longer he went on reassuring the group that
the evaluation know the full extent and pur- they didn't have to worry about us, the more
pose of observations. On occasion, program worried they got. Sensing that they were
staff may be informed that evaluators will be worried, he increased the intensity of his re-
participating in the program, but clients will assurances. While we continued to refer to
not be so informed. In other cases, a re- ourselves as educational researchers, the
searcher may reveal the purpose and nature participants thereafter referred to us as eval-
of program participation to fellow program uators. It took a day and a half to recover our
participants and ask for their cooperation in full participating roles as the participants
keeping the evaluation secret from program got to know us on a personal levei as indi-
staff. On still other occasions, a variety of viduais.
people intimately associated with the pro- Trying to protect the participants (and the
gram may be informed of the evaluation, but evaluation) had backfired and made our en-
public officials who are less closely associ- try into the group even more difficult than it
ated with the program may be kept "in the otherwise would have been. However, this
dark" about the fact that observations are experience sensitized us to what we subse-
under way. Sometimes the situation be- quently observed to be a pattern in many
comes so complex that the evaluator may program situations and activities through-
lose track of who knows and who doesn't out the week, and became a major finding of
Fieldwork Strategies and Observation Methods LET. 273

the evaluation: staff overprotection of and reports conceal names, locations, and other
condescending attitudes toward partici- identifying information so that the people
pants. who have been observed will be protected
Based on this and other evaluation expe- from harm or punitive action. Because the
riences, I recommend full and complete dis- basic researcher is interested in truth rather
closure. People are seldom really deceived than action, it is easier to protect the identity
or reassured by false or partial explana- of inforinants or study settings when doing
tionsat least not for long. Trying to run a scholarly research. In evaluation research,
ruse or scam is simply too risky and adds to however, while the identity of who said
evaluator stress while holding the possibil- what may be possible to keep secret, it is sel-
ity of undermining the evaluation if (and dom possible to conceal the identity of a pro-
usually when) the ruse becomes known. gram, and doing so may undermine the util-
Program participants, over time, will tend to ity of the findings.
judge evaluators first and foremost as peo- Evaluators and decision makers will have
ple not as evaluators. to resolve these issues in each case in accor-
The nature of the questions being studied dance with their own consciences, evalua-
in any particular evaluation will have a pri- tion purposes, political realities, and ethical
mary effect on the decision about who will sensitivities.
be told that an evaluation is under way. In
formative evaluations where staff members
Variations in Duration
and/or program participants are anxious to
of Observations
have information that will help them im-
prove their program, the quality of the data Another important dimension along
gathered may be enhanced by overtly solic- which observational studies vary is the
iting the cooperation of everyone associated length of time devoted to data gathering. In
with the program. Indeed, the ultimate ac- the anthropological tradition of field re-
ceptance and usefulness of formative infor- search, a participant observer would expect
mation may depend on such prior disclo- to spend six months at a minimum, and of-
sure and agreement that a formative ten years, living m the culture being ob-
evaluation is appropriate. On the one hand, served. The fieldwork of Napoleon Chag-
where program funders have reason to be- non (1992) among the Yanomami Indians in
lieve that a program is corrupt, abusive, in- the rain forest at the borders of Venezuela
competently administered, and/or highly and Brazil spanned a quarter century. To de-
negative in impact on clients, it may be develop a holistic view of an entire culture or
cided that an externai, covert evaluation is subculture takes a great deal of time, espe-
necessary to find out what is really happen- cially when, as in the case of Chagnon, he
ing in the program. Under such conditions, was documenting changes in tribal life and
my preference for full disclosure may be nei- threats to the continued existence of these
ther prudent nor practical. On the other once-isolated people. The effects of his
hand, Whyte (1984) has argued that "in a long-term involveinent on the people he
community setting, maintaining a covert studied became controversial (Geertz 2001;
role is generally out of the ques tion" (p. 31). Tiemey 2000a, 2000b), a matter we shall take
Finally, there is the related issue of confi- up later. The point here is that fieldwork in
dentiality. Those who advocate covert re- basic and applied social science aims to un-
search usually do so with the condition that veil the interwoven complexities and funda-
274 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

mental pattems of social lifeactual, per- and organizational development. Such has
ceived, constructed, and analyzed. Such been the case with the extraordinary work of
studies take a long time. Patricia Carini (1975, 1979) at the Prospect
Educational researcher Alan Peshkin School in North Bennington, Vermont.
offers a stellar example of a committed Working with the staff of the school to collect
fieldworker who lived for periods of time in detailed case records on students of the
varied settings in order to study the intersec- school, she established an archive with as
tions between schools and communities. He much as 12 years of detailed documentation
did fieldwork in a Native American commu- about the learning histories of individual
nity; in a high school m a stable, multiethnic students and the nature of the school pro-
midsized city in Califrnia; in rural, east- grams they experienced. Her data included
central Illinois; in a fundamentalist Chris- copies of the students' work (completed as-
tian school; and in a private, residential signments, drawings, papers, projects),
school for elites (Peshkin 1986,1997,2000b). classroom observations, teacher and parent
To collect data, he and his wife Maryann observations, and photographs. Any organi-
lived for at least a year m and with the com- zation with an internai evaluation informa-
munity that he was studying. They shopped tion system can look beyond quarterly and
locally, attended religious services, and de- annual reporting to building a knowledge
veloped close relationships with civic lead- archive of data to document development
ers as well as teachers and students. and change over years instead of just
In contrast, evaluation and action re- months. Participant observations by those
search typically in volve much shorter dura- who manage such systems can and should
tions in keeping with their more modest be an integral part of this kind of knowl-
aims: generating useful information for ac- edge-building organizational data system
tion. To be useful, evaluation findings must that spans years, even decades.
be timely. Decision makers cannot wait for On the other end of the time continuum
years while fieldworkers sift through moun- are short-term studies that involve observa-
tains of field notes. Many evaluations are tions of a single segment of a program,
conducted under enormous pressures of sometimes for only an hour or two. Evalua-
time and limited resources. Thus, the dura- tions that include brief site visits to a number
tion of observations will depend to a consid- of program locations may serve the purpose
erable extent on the time and resources of simply establishing the existence of cer-
available in relation to the information tain leveis of program operations at different
needs and decision deadlines of primary sites. Chapter 1 presented just such an obser-
evaluation users. Later in this chapter we'11 vation of a single two-hour session of an
include reflections from an evaluator about early childhood parent education program
what it was like being a part-time, in which mothers discussed their child-rear-
in-and-out observer of a program for eight ing practices and fears. The site visit obser-
months, but only present 6 hours a week out vations of some 20 such program sessions
of the program's 40-hour week. throughout Minnesota were part of an im-
On the other hand, sustained and ongo- plementation evaluation that reported to the
ing evaluation research may provide annual state legislature how these innovative (at the
findings while, over years of study, accumu- time) programs were operating in practice.
lating an archive of data that serves as a Each site visit lasted no more than a day, of-
source of more basic research into human ten only a half day.
Fieldwork Strategies and Observation Methods LET. 275

Sometimes an entire segment of a pro- which the observer participates in the set-
gram may be of sufficiently short duration ting being studied, the tension between
that the evaluator can participate in the com- insider versus outsider perspectives, the ex-
plete program. The leadership retreat we ob- tent to which the purpose of the study is
served lasted 6 days, plus three 1-day fol- made explicit, and the duration of the obser-
low-up sessions during the subsequent year. vations. A major factor affecting each of
The criticai point is that the length of time these other dimensions is the scope or focus
during which observations take place de- of the study or evaluation. The scope can be
pends on the purpose of the study and the broad, encompassing virtually ali aspects of
questions being asked, not some ideal about the setting, or it can be narrow, involving a
what a typical participant observation must look at only some small part of what is hap-
necessarily involve. Field studies may be pening.
massive efforts with a team of people partic- Parameswaran (2001) wanted to inter-
ipating in multiple settings in order to do view young women in ndia who read West-
comparisons over several years. At times, ern romance novis. Thus, her fieldwork
then, and for certain studies, long-term had a very narrow focus. But to contextu-
fieldwork is essential. At other times and for alize what she learned from interviews, she
other purposes, as in the case of short-term sought "active involvement in my infor-
formative evaluations, it can be helpful for mants' livesbeyond their romance reading."
program staff to have an evaluator provide How did she do this?
feedback based on just one hour of onlooker
observation at a staff meeting, as I have also I ate snacks and lunch at cafs with groups of
done. women, went to the movies, dined with them
My response to students who ask me how at their homes, and accompanied them on
long they have to observe a program to do a shopping trips. I joined women's routine con-
good evaluation follows the line of thought versations during break times and inter-
developed by Abraham Lincoln during one viewed informants at a range of everyday
of the Douglas-Lincoln debates. In an obvi- sites, such as college grounds, homes, and res-
ous reference to the difference in stature be- taurants. I visited used-book vendors, book-
tween Douglas and Lincoln, a heckler asked, stores, and lending libraries with several
"Tell us, Mr. Lincoln, how long do you think readers and observed social interactions be-
a man's legs ought to be?" tween library owners and young women. To
Lincoln replied, "Long enough to reach gain insight into the multidimensional rela-
the ground." tionship between women's romance reading
Fieldwork should last long enough to get and their experiences with everyday social
the job doneto answer the research ques- discourse about romance readers, I inter-
tions being asked and fulfill the purpose of viewed young women's parents, siblings,
the study. teachers, bookstore managers, and owners of
the lending libraries they frequented. (p. 75)

Variations in
The tradition of ethnographic fieldwork
Observational Focus has emphasized the importance of under-
standing whole cultural systems. The vari-
The preceding sections have discussed ous subsystems of a society are seen as inter-
how observations vary in the extent to dependent parts so that the economic
276 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

system, the cultural system, the political sys- program and how many are studied will
tem, the kinship system, and other special- clearly affect such issues as the extent to
ized subsystems could only be understood which the observer is a participant, who will
in relation to each other. In reality, fieldwork know about the evaluation's purpose, and
and observations have tended to focus on a the duration of observations.
particular part of the society or culture be- Chapter 5 discussed how decisions about
cause of specific investigator interests and the focus and scope of a study involve
the need to allocate the most time to those trade-offs between breadth and depth. The
things that the researcher considered most very first trade-off comes in framing the re-
important. Thus, a particular study might search questions to be studied. The problem
present an overview of a particular culture is to determine the extent to which it is desir-
but then go on to report in greatest detail able and useful to study one or a few ques-
about the religious system of that culture. tions in great depth or to study more ques-
In evaluating programs, a broad range of tions but each in less depth. Moreover, in
possible foci makes choosing a specific focus emergent designs, the focus can change over
challenging. One way of thinking about fo- time.
cus options involves distinguishing various
program processes sequentially: (1) pro- Dimensions Along Which
cesses by which participants enter a pro- Fieldwork Varies: An Overview
gram (the outreach, recruitment, and intake
components); (2) processes of orientation to WeVe examine d five dimensions that can
and socialization into the program (the initi- be used to describe some of the primary
ation period); (3) the basic activities that variations in fieldwork. Those dimensions,
comprise program implementation over the discussed in the previous sections, are
course of the program (the service delivery graphically summarized in Exhibit 6.1.
system); and (4) the activities that go on These dimensions canbe used to help design
around program termination, including fol- observational studies and make decisions
low-up activities and client impacts over about the parameters of fieldwork. They can
time. It would be possible to observe only also be used to organize the methods section
one of these program components, some of a report or dissertation in order to docu-
combination of components, or ali of the ment how research or evaluation fieldwork
components together. Which parts of the actually unfolded.

13. What to Observe: A Sensitizing Framework

I keep six honest serving men.

K ^ s They taught me ali I knew:
Their names are What and Why and When
And How and Where and Who.2

Rudyard Kipling
Fieldwork Strategies and Observation Methods LET. 277

fffWfflfffffflfflM

iaUEn Dimensions Showing Fieldwork Variations

1. Role of the observer

Full participant I I Onooker observer

in the setting Part participant/part observer (spectator)

2. Insider versus outsiderperspective

Insider (emic) I I Outsider (etic)

perspective dominant Balance perspective dominant

3. Who conducts the inquiry

Solo researchers, I I People in the

teams of professionals Variations in colaboration setting being studied
and participatory research

4, Disciosure ofthe observer's role to others

Overt: I I Covert:
Fuil disciosure Selective disciosure No disciosure

5. Duration of observations and fieldwork

Short, I I Long-term,
single observation Ongoing over time multiple observations
(e.g., 1 site, 1 hour) (e.g., months, years)

6. Focus of observations

Narrow focus: I I Broad focus:

Single eement Evolving, emergent Holistic view

NASA space scientist David Morrison fields never require the authors to state a
(1999) has noted that in astronomy, geology, 'hypothesis' in order to publish their re-
and planetary science, observation precedes sults" (p. 8).
theory generation "and the journals in these
278 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

A Tecent example is the famous Hubble Space cept as a guide to fieldwork with special at-
Telescope Deep Field in which the telescope tention to the words and meanings that are
obtained a single exposure of many days du- prevalent among the people being studied.
ration of one small field in an unremarkable More generally, however, "a sensitizing con-
part of the sky. The objective was to see fainter cept is a starting point in thinking about the
and farther than ever before, and thus to find class of data of which the social researcher
out what the universe was like early in its his- has no definite idea and provides an initial
tory. No hypothesis was requiredjust the guide to her research" (van den Hoonaard
unique opportunity to look where no one had 1997:2). Sensitizing concepts in the social sci-
ever looked before and see what nature herself ences include loosely operationalized no-
had to tell us. tions such as victim, stress, stigina, and
In many other sciences the culture de- learning organization that can provide some
mands that funding proposals and published initial direction to a study as a fieldworker
papers be written in terms of formulating and inquires into how the concept is given mean-
testing a hypothesis. But I wonder if this is re- ing in a particular place or set of circum-
ally the way the scientific process works, or is stances being studied (Schwandt 2001).
this just an artificial structure imposed for the Rudyard Kipling's poem about his "six
sake of tradition. (Morrison 1999:8) honest serving men," quoted above, consti-
tutes a fundamental and msightful sensitiz-
Part of the value of open-ended naturalis- ing framework identifying the central ele-
tic observations is the opportunity to see ments of good description. In social science,
what there is to see without the blinders of "group process" is a general sensitizing con-
hypotheses and other preconceptions. Pure cept as is the focus on outcomes in evalua-
observation. As Morrison put it so elegantly, tion. Kinship, leadership, socialization,
just the unique opportunity to look where power, and similar notions are sensitizing in
no one has ever looked before and see what that they alert us to ways of organizing ob-
the world has to show us. servations and making decisions about
That's the ideal. However, it's not possi- what to record. Qualitative methodologist
ble to observe everything. The human ob- Norman Denzin (1978a) has captured the es-
server is not a movie camera, and even a sence of how sensitizing concepts guide
movie camera has to be pointed in the right fieldwork:
direction to capture what is happening. For
both the human observer and the camera The observer moves from sensitizing concepts
there must be focus. In fieldwork, this focus to the immediate world of social experience
is provided by the study design and the na- and permits that world to shape and modify
ture of the questions being asked. Once in his conceptual framework. In this way he
the field, however, the observer must some- moves contmually between the realm of more
how organize the complex stimuli experi- general social theory and the worlds of native
enced so that observing thatbecomes and re- people. Such an approach recognizes that so-
mains manageable. cial phenomena, while displaying regulari-
Experienced observers often use "sensi- ties, vary by time, space, and circumstance.
tizing concepts" to orient fieldwork. Quali- The observer, then, looks for repeatable regu-
tative sociologist and symbolic interaction- larities. He uses ritual patterns of dress and
ist Herbert Blumer (1954) is credited with body-spacing as indicators of self-image. He
originating the idea of the sensitizing con- takes special languages, codes, and dialects as
Fieldwork Strategies and Observation Methods LET. 279

indicators of group boundaries. He studies his A note of caution about sensitizing con-
subjecfs prized social objects as indicators of cepts: When they become part of popular
prestige, dignity, and esteem hierarchies. He culture, they can lose much of their original
studies moments of interrogation and deroga- meaning. Philip Tuwaletstiwa, a Hopi geog-
tion as indicators of socialization strategies. rapher, relates the story of a tourist cruising
He attempts to enter his subjecfs closed world through Native American areas of the
of interaction so as to examine the character of Southwest. He overheard the tourist, "ali
private versus public acts and attitudes. (p. 9) agog at half-heard tales about Hopi land,"
ask his wife, "Where are the-powerplaces?"
The notion of "sensitizing concepts" re- "Tell her that's where we plug-in TV," he
minds us that observers do not enter the said (quoted m Milius 1998:92).
field with a completely blank slate. While Overused sensitizing concepts can be-
the inductive nature of qualitative inquiry come desensitizing.
emphasizes the importance of being open to
whatever one can learn, some way of orga-
nizing the complexity of experience is virtu-
Sources of Data
ally a prerequisite for perception itself. Ex-
hibit 6.2 presents examples of common Poet David Wagoner (1999) tells those ob-
sensitizing concepts for program evaluation serving the modern world and afraid of be-
and organizational studies. These common ing lost to follow the advice Native Ameri-
program concepts and organizational di- can elders gave the young when they were
mensions constitute ways of breaking the afraid of being lost in the forest:
complexities of planned human interven-
tions into distinguishable, manageable, and Lost
observable elements. The examples in Ex-
hibit 6.2 are by no means exhaustive of eval- Stand still. The trees ahead and bushes
uation and organizational sensitizing con- beside you
cepts, but they illustrate oft-used ways of Are not lost. Where you are is called
organizing an agenda for inquiry. These con- Here,
cepts serve to guide initial observations as And you must trust it as a powerful
the evaluator or organizational analyst stranger,
watches for incidents, interactions, and con- Must ask permission to know it and
versations that illuminate these sensitizing be known.
concepts in a particular program setting or The forest breathes. Listen. It answers,
organization. Highly experienced evalua- I have made this place around you.
tors and organizational consultants have in- If you leave it, you may come back
ternalized some kind of sensitizing frame- again, saying Here.
work like this to the point where they would No two trees are the same to Raven.
not need to list these concepts in a formal No two branches are the same to Wren.
written design. Less experienced research- If what a tree or a bush does is lost on
ers and dissertation students will usually you,
benefit from preparing a formal list of major You are surely lost. Stand still. The
sensitizing concepts in the formal design forest knows
and then using those concepts to help orga- Where you are. You must let it find
nize and guide fieldwork, at least initially. you.3
280 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

is^aaaMl | Examples of Sensitizing Concepts

p^nMaffiqipfMi ||||i|yi[i j n |im
il MMgBB
Program Evaluation Orgonizational Dimensions

Context Mission/vision language

Goals Centralization/decentralization of participation and decision making
Inputs Externai boundary relationships: Open/closed
Recruitment Routinness/nonroutinness of work, products, decisions
Intake Leadership
Implementation Communication patterns
Processes Organizational culture
Output Hierarchy: Layered/flat
Outcomes Authority patterns
Products Formal/informal networks
Impacts Rewards/punishments, incentives/disincentives
Program theory Success and failure messages/stories
Logic model Degree of integration
Perceived Competition/cooperation
Assets
Barriers
Strengths
Weakn esses
Incentives
Disincentives

The what and how of qualitative inquiry coherent and in-depth illustrations, the ex-
are closely linked. Sources of data are de- amples that follow focus on program
rived from inquiry questions. Knowing evaluation.
what we want to illuminate helps us deter-
mine sources of data for that illumination.
The examples and illustrations that follow The Setting
derive from and build on the sensitizing
framework for program evaluation. Inter- Describing a setting, like a program set-
spersed with this presentation of sources of ting, begins with the physical environment
evaluation data are examples of how to col- within which the program takes place. The
lect observational data. These strategies ap- description of the program setting should be
ply to other inquiry settings, but to provide sufficiently detailed to permit the reader to
Fieldwork Strategies and Observation Methods LET. 2 8 1

visualize that setting. In writing a program interpretive phrases. But such writing can
description, the observer, unlike the novel- also be dull. Metaphors and analogies can
ist, should avoid interpretive adjectives ex- enliven and enrich descriptions, helping
cept as they appear in quotes from partici- readers connect through shared under-
pants about their reactions to and standings and giving them a better feel for
perceptions of that environment. Such ad- the environment being described. I once
jectives as comfortable, beautiful, drab, and evaluated a wilderness education program
stimulating interpret rather than describe that included time at the Grand Canyon. Ex-
and interpret vaguely at that. More purely hibit 6.3 presents my feeble attempt to cap-
descriptive adjectives include ture in words our first view of the Grand
Canyon. Notice the metaphors that run
colors ("a room painted blue with a through the description. Of course, this is
one of those instances where a picture
blackboard at one end"),
would be worth a mountain of words, which
space ("a 40-foot-by-20-foot classroom is why qualitative fieldwork increasingly in-
with windows on one side"), and cludes photography and videography. This
purpose ("a library, the walls lined with excerpt aims at offering a sense ofthe physical
books and tables in the center"). environment more than it offers a literal de-
scription because unless one has been there
Beginners can practice learning to write or seen pictures, the landscape is outside or-
descriptively by sharing a description of a dinary experience.
setting observed with a couple of people and The physical environment of a setting can
asking them if they can visualize the setting be important to what happens in that envi-
described. Another helpful exercise in- ronment. The way the walls look in rooms,
volves two people observing the same envi- the amount of space available, how the
ronment and exchanging their descriptions, space is used, the nature of the lighting, how
watching in particular for the use of inter- people are organized in the space, and the
pretive adjectives instead of descriptive interpretive reactions of program partici-
ones. Vivid description provides sufficient pants to the physical setting can be impor-
information that the reader does not have to tant information about both program imple-
speculate at what is meant. For example, menta tion and the effects of the program on
simply reporting "a crowded room" re- participants.
quires interpretation. Contrast with this:
A common mistake among observers is to
take the physical environment for granted.
The meeting room had a three-person couch Thus, an evaluator may report that the pro-
across one side, six chairs along the adjoining gram took place in "a school." The evaluator
walls next to the couch, and three chairs along may have a mental image of "school" that
the wall facing the couch, which included the matches what was observed, but schools
door. With 20 people in the room, ali standing, vary considerably in size, appearance, and
there was very little space between people. neighborhood setting. Even more so, the in-
Several participants were overheard to say, teriors of schools vary considerably. The
"This room is really crowded." same can be said for criminal justice settings,
health settings, community mental health
Such descriptive writing requires atten- programs, and any other human service ac-
tion to detail and discipline to avoid vague, tivity.
282 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Example of Combining Description and Metaphor to

E X H I B I T 6.3 Provide a Sense of Place

Context for a Wilderness Program: First View From Briqht Anqel Point at the Grand Canvon

We foilowed an asphalt path from the tion supporting a series of sloping sedimentary
lodge a quarter mileto Bright Angel Point, per- rock terraces, the Supai. These sweeping ter-
haps the most popular tourist site at the Grand races, spotted green with sparse desert vege-
Canyon because of its relatively easy accessi- tation, point upward like arrow feathers to a
bility. With cameras amed in ali directions at white sandstone pedestal, the Coconino. A
the spectacular panorama, in a sea of domes- dark red pinnacle of Hermit shae uniquely
tic accents and foreign tongues, we waited our crowns each temple. Eons of erosion have
turn at the edge to behold the magnificent sculpted dramatic variations in every aspect
rock temples of Ottoman Amphitheater: Deva, save one: their common geologic history. I
Brahma, Zoroaster and, in the distance, Thor. studied each separately, wanting to fix in my
Each rises a half mile above the undulating mind the differences between them, but the
grayness of the stark Tonto Platform defining shared symmetry of strata melded them into a
the eight-mile descent of Bright Angel Can- single, massive formation, a half mile high and
yon, a narrow slit hiding the inner gorge that many miles around. Behind me I heard a par-
looks like it had been drawn in black ink to ticipant say softly to no one in particular, al-
outline the base of the temples. Each beginsas most under her breath, "lt's too awesome. I
sheer Redwall that forms a massive founda- feel overwhelmed."

SOURCE: Adapted from Patton (1999a).

During site visits to early childhood edu- cal environments clearly affect people and
cation programs, we found a close associa- programs.
tion between the attractiveness of the facility Variations in the settings for a wilderness
(child-made decorations and colorful post- training program for which I served as par-
ers on the walls, well-organized learning ticipant observer provide an interesting ex-
materiais, orderly teacher area) and other ample of how physical environments affect a
program attributes (parent involvement, program. The explicit purpose of holding
staff morale, clarity of the program's goals the "field conferences" in the wilderness
and theory of action). An attractive, well-or- was to remove people from their everyday
dered environment corresponded to an en- settings in largely urban environments sur-
gaging, well-ordered program. In observing rounded by human-made buildings and the
as well as conducting workshops, I have paraphernalia of modern industrial society.
noted how the arrangement of chairs affects Yet, wilderness environments are no more
participa tion. It is typically much easier to uniform than the environments of human
generate discussion when chairs are in a cir- service programs. During the yearlong pro-
cle rather than in lecture style. The dim light- gram, participants were exposed to four dif-
ing of many hotel conference rooms seems to ferent wilderness environments: the au-
literally drain energy from people sitting in tumn forest in the Gila wilderness of New
those rooms for long periods of time. Physi- Mxico; the rough terrain of Arizona's Kofa
Fieldwork Strategies and Observation Methods LET. 283

Mountains in winter; the muddy, flooding pants behave toward each other in those en-
San Juan River in the canyon lands of Utah vironments. Rudolf Moos (1975) described
during the spring; and among the magnifi- the social-ecological view of programs as
cent rock formations of the Grand Canyon in follows:
suinmer, a desertenvironment. One focus of
the evaluation, then, was to observe how The social climate perspective assumes that
participants responded to the opportunities environments have unique "personalities,"
and constraints presented by these different just like people. Personality tests assess per-
environments: forest, mountains, canyon- sonality traits or needs and provide informa-
lined river, and Grand Canyon desert. tion about the characteristic ways in which
In addition, weather and seasonal differ- people behave. Social environments can be
ences accentuated variations among these similarly portrayed with a great deal of accu-
environments. Program activities were racy and detail. Some people are more sup-
clearly affected by the extent to which there portive than others. Likewise, some social
was rain, cold, wmd, and shelter. In the pro- environments are more supportive than oth-
gram^ theory, weather uncertainties were ers. Some people feel a strong need to control
expected to be a natural part of the program, others. Similarly, some social environments
offering natural challenges for the group to are extremely rigid, autocratic, and control-
deal with. But the program theory also ling. Order, clarity, and structure are impor-
called for participants to engage deeply with tant to many people. Correspondingly, many
each other during evening group discus- social environments strongly emphasize or-
sions. During one 10-day winter field con- der, clarity, and control. (p. 4)
ference that was unusually cold and wet,
participants were miserable, and it became In describing the social environment, the
increasingly difficult to carry on group dis- observer looks for the ways in which people
cussions, thus reducing considerably the organize theinselves into groups and sub-
amount of group process time available and groups. Pattems and frequency of interac-
rushing the interactions that did occur be- tions, the direction of communication pat-
cause of participants' discomfort. Program tems (from staff to participants and
staff learned that they needed to anticipate participants to staff), and changes in these
more clearly the possible variations m phys- pattems tell us things about the social envi-
ical environments, plan for those variations, ronment. How people group together can be
and include the participants in thatplanning illuminative and important. All-male versus
so as to increase their commitment to contin- all-female groupings, male-female interac-
uing the process under difficult physical tions, and interactions among people with
conditions. different background characteristics, racial
identities, and/or ages alert the observer to
pattems in the social ecology of the pro-
The Human/ gram.
Social Environment Decision-making patterns can be a partic-
ularly important part of a program's social
Just as physical environments vary, so too environment. Who makes decisions about
do social environments. The ways in which the activities that take place? To what extent
human beings interact create social-ecologi- are decisions made openly, so that partici-
cal constellations that affect how participants are aware of the decision-making pro-
284 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

cess? How are decisions by staff communi-

cated to participants? Answers to these aBSEFttlNG NATURij-
questions are an important part of the de- 0'i.p!i':'is.Vi ij! i.i V I ! :-II!: i.Vi! -r.' Lrr ;; s R; i; ; SJ-S |! r SO
scription of a prograin's decision environ- !: ' : ^ : ; ; ' ! . : .*!! M-IH :.i.!.'ri !.'
! , !hh ! rn.vrn "y !',;!
ment. lMiTifo.Mi:' r^i/iri h.Vivr.-* ::V" !!:"rn
An observer's descriptions of a social en- S V I w 'I-V!\'X I^I1'!'T:''HI'I\A!.T!:;f .
vironment will not necessarily be the same =!>=
: !! ii; iiVi- :'i > i !'!"i'r! ri Vi ;< ' ! " I " O ; : - N
'.' ! :'.;
as the perceptions of that environment ex- i.>!
! Vi r.';;: "Vri n !v'v.' vir sk-i" ' v .!!=: !!
pressed by participants. Nor is it likely that ;:.ri.;-i:'IV:-i3 .J:-I:V dYiYVi \C >. r.h- s -iivi ;;-i;vj
ali participants will perceve the setting's irii,!'V-:;i:!:i:i!i.i.'!'.i+iS! ;Vi \.'r, ! rr! hy !. ; n iirVi >y
human climate in the same way. At ali times
it is criticai that the observer record partici-
pants' comments in quotation narks, indi-
cating the sourcewho said what?so as to
keep perceptions of participants separate ley, we traced the beginnings of the differenti-
from the observer's or evaluator's own de- ation of Huayopampa from Pa caros back
scriptions and interpreta tions. more than a century. (Whyte 1984:153)

Historical Perspectives Documenting and understanding the

context of a program will require delving
Historical information can shed impor- into its history. How was the program cre-
tant light on the social environment. The his- ated and initially funded? Who were the
tory of a program, community, or organiza- original people targeted for program ser-
tion is an important part of the context for vices, and how have target populations
research. Distinguished qualitative sociolo- change d over time? To what extent and in
gist William Foote Whyte, sometimes called what ways have goals and intended out-
the father of sociological field research, has comes changed over time? What have staff-
reflected on how he came to value historical ing patterns been over time? How has the
research as a criticai part of his fieldwork. program's governance (board) been in-
volved at various stages in the program's
When we began our Peruvian research pro- history? What crises has the program en-
gram, I viewed history as having little value dured? If the program is embedded within a
for understanding the current scene. I thought larger organizational context, what is the
I was only being sympathetic to the interests of history of that organization in relation to the
our Peruvian researchers in suggesting that program? How has the larger political and
they gather historical data on each village for economic environment changed over time,
the last 50 years. and how have those changes affected pro-
Fortunately, the Peruvians refused to ac- gram development? What are the stories
cept the 50-year limit and in some cases people tell about the program's history?
probed up to 500 years in the history of vil- These kinds of questions frame inquiry into
lages or areas. Much of these data on rural the program's history to illuminate context.
commuriities would be of interest only to his- In the 1990s, I evaluated a "free high
torians, However, understanding the paradox school" that had been created during the
of the Mantaro Valley required us to go back to struggles and turmoil of the 1960s. Little
the conquest of Peru, and, in the Chancay Val- about the program's current programming
Fieldzuork Strategies and Observation Methods lj. 285

could be understood outside the context of own feelings as part of the observation.)
its historical emergence. The schooTs image How do behaviors and feelings change over
of itself, its curriculum, and its policies had the course of the activity?
been handed down and adapted from that Finally, the observer looks for closure
intense period of early development. Doing points. What are the signals that a particular
fieldwork in the 1990s could only be done by activity is being ended? Who is present at
traversing the memories and legends of the that time? What is said? How do partici-
school's historical emergence in the 1960s. pants react to the ending of the activity?
How is the completion of this unit of activity
Planned Program related to other program activities and fu-
Implementation Activities ture plans?
and Formal Interactions Each unit of activity is observed and
treated as a self-contained event for the pur-
Most evaluations focus at least some ob- pose of managing field notes. The observa-
servations on planned program activities. tion of a single session of the early childhood
What goes on in the program? What do par- parent education program presented in
ticipants and staff do? What is it like to be a Chapter 1 is an example. Each observed
participant? These are the kinds of questions event or activity can be thought of as a
evaluators bring to the program setting to mini-case write-up of a discrete incident, ac-
document program implementation. tivity, interaction, or event. During analysis,
Build observations around activities that one looks across these discrete units-of-
have a kind of unity about them: a begin- activity cases for patterns and themes, but
ning, some middle point, and a closure during the initial stages of fieldwork the ob-
pointsuch things as a class session, a coun- server will be kept busy just trying to cap-
seling session, meai time in the residential ture self-contained units of activity without
facility, a meeting of some kind, a home visit worrying yet about looking for patterns
in an outreach program, a consultation, or a across activities.
registration procedure. Attending to se- Observing and documenting formal pro-
quence illustrates how the inquiry program activities will constitute a central ele-
gresses over the course of an observation. ment in evaluating planned program imple-
Initially, the observer will focus on how the mentation, but to fully understand a
activity is introduced or begun. Who is pres- program and its effects on participants, ob-
ent at the beginning? What exactly was said? servations should notbe restricted to formal,
How did participants respond or react to planned activities. The next section dis-
what was said? cusses observation of the things that go on
These kinds of basic descriptive ques- between and around formal, planned pro-
tions guide the evaluator throughout the full gram activities.
sequence of observation. Who is involved?
WJwt is being done and saidby staff and par-
Informal Interactions
ticipants? How do they go about what they
do? Where do activities occur? When do and Unplanned Activities
things happen? What are the variations in
how participants engage in planned activi- If observers put a way their seeing and ob-
ties? How does it feel to be engaged in this serving selves as soon as a planned, formal
activity? (The observer records his or her activity ends, they will miss a great deal of
286 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

data. Some programs build in "free" or un- Was it clear to you what they were trying to
structured time between activities, with the get at?
clear recognition that such periods provide
What did you think of the session today?
opportunities for participants to assimilate
what has occurred during formal program- How do you think what went on today fits
matic activities as well as to provide partici- into this whole thing that we're involved in?
pants with necessary breathing space.
Rarely, if ever, can a program or institution Such questioning should be done in an easy,
plan every moment of participants' time. conversational manner so as not to be intru-
During periods of informal interaction sive or so predictable that every time some-
and unplanned activity, it can be particu- one sees you coming they know what ques-
larly difficult to organize observations be- tions you're going to ask. "Get ready, here
cause people are likely to be milling around, comes the evaluator with another endless
coming and going, moving in and out of set of questions." Also, when doing infor-
small groups, with some sitting alone, some mal, conversational interviewing, be sure
writing, some seeking refreshments, and that you are acting in accordance with ethi-
otherwise engaging in a full range of what cal guidelines regarding informed consent
may appear to be random behaviors. How, and confidentiality. (See the earlier discus-
then, can the evaluator observer collect data sion in this chapter about overt versus co-
during such a time? vert fieldwork.)
This scenario illustrates beautifully the How something is said should be re-
importance of staying open to the data and corded along with what is said. At a mom-
doing opportunity sampling. One can't an- ing break in the second day of a two-day
ticipate ali the things that might emerge dur- workshop, I joined the other men in the
ing unplanned program time, so the ob- restroom. As the men lined up to use the fa-
server watches, listens, and looks for cilities, the first man to urinate said loudly,
opportunities to deepen observations, re- "Here's what I think of this program." As
cording what people do, the nature of infor- each man finished he turned to the man be-
mal interactions (e.g., what subgroups are in hind him and said, "Your turn to piss on the
evidence), and, in particular, what people program." This spontaneous group reaction
are saying to each other. This last point is spoke volumes more than answers to formal
particularly important. During periods of interview questions and provided much
unplanned activity, participants have the greater depth of expression than checking
greatest opportunity to exchange views and "very dissatisfied" on an evaluation ques-
to talk with each other about what they are tionnaire.
experiencing in the program. In some cases, Everything that goes on in or around the
the evaluator will simply listen in on conver- program is data. The fact that none of the
sations or there may be opportunities to con- participants talk about a session when it is
duct informal interviews, either with a sin- over is data. The fact that people immedi-
gle participant in natural conversation or ately split in different directions when a ses-
with some small group of people, asking sion is over is data. The fact that people talk
normal, conversational questions: about personal interests and share gossip
that has nothing to do with the program is
So what did you think of what went on this data. In many programs, the most sig-
moming? nificant participant learnings occur during
Fieldzuork Strategies and Observation Methods lj. 287

unstructured time as a result of interactions feeling among some staff members that they
with other participants. To capture a holistic had a responsibility to plan and account for
view of the program, the evaluator observer every moment during the program.
must stay alert to what happens during Participant observation necessarily com-
these informal periods. While others are on bines observing and informal interviewing.
break, the observer is still working. No Observers need to be disciplined about not
breaks for the dedicated field-worker! Well, assuming they know the meaning to partici-
not really. You've got to pace yourself and pants of what they observe without check-
take care of yourself or your observations ing with those participants. During one pe-
will deteriorate into mush. But you get the riod of unstructured time in the wilderness
idea. You may be better off taking a break program, following a fairly intensive group
during part of a formal session time so you activity in which a great deal of interper-
can work (collect data) while others are on sonal sharing had taken place, I decided to
break. pay particular attention to one of the older
As happens in many programs, the par- men in the group who had resisted involve-
ticipants in the wilderness education pro- ment. Throughout the week he had taken
gram I was observing/evaluating began every available opportunity to make it
asking for more free, unstructured time. known that he was unimpressed with the
When we weren't hiking or doing camp program and its potential for impact on him.
chores, a lot of time was spent in formal dis- When the session ended, he immediately
cussions and group activities. Participants walked over to his backpack, pulled out his
wanted more free time to journal. Some sim- writing materiais, and went off to a quiet
ply wanted more time to reflect. Most of ali, spot where he could write. He continued
they wanted more time for informal interac- writing, completely absorbed, until dinner-
tions with other participants. I respected the time an hour later. No one interrupted him.
privacy of one-to-one interactions when I With his legs folded, his notebook in his lap,
observed them and would never attempt to and his head and shoulders bent over the
eavesdrop. I would, however, watch for such notebook, he gave off clear signals that he
interactions and, judging body language was involved, concentrating and working
and facial expressions, I would speculate on something to which he was giving a
when serious interpersonal exchanges were great deal of effort.
taking place. I would then look for natural I suspected as I watched that he was vent-
opportunities to engage each of those partic- ing his rage and dissatisfaction with the pro-
ipants in conversational interviews, telling gram. I tried to figure out how I might read
them I had noticed the intensity of their in- what he had written. I was so intrigued that I
teraction and inquiring whether they were momentarily even considered covert means
willing to share what had happened and of getting my hands on his notebook, but
what significance they attached to the inter- quickly dismissed such unethical mvasion
action. Most appreciated my role in docu- of his privacy. Instead, I looked for a natural
menting the program's unfolding and its ef- opportunity to mitiate a conversation about
fects on participants and were open to his writing. During the evening meai
sharing. It was on the basis of those informal around the campfire, I moved over next to
interviews and observations that I provided him, made some small talk about the
formative feedback to staff about the impor- weather, and then began the following con-
tance of free time and helped alleviate the versation:
288 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

"You know, in documenting experiences letter may have been more for me than for
people are having, I'm trying to track some him. But the most important thing that's
of the different things folks are doing. The been happening for me during this week is
staff have encouraged people to keep jour- the time to think about my family and how
nals and do writing, and I noticed that you important it is to me and I haven't been hav-
were writing fairly intensely before dinner. ing a very good relationship with my son. In
If you're willing to share, it would be helpful fact, it's been pretty shitty and so I wrote him
for me to know how you see the writing fit- a letter. That's ali."
ting into your whole experience with the This short conversation revealed a very
program." different side of this man and an important
He hesitated, moved his food about in his impact of the program on his personal and
bowl a little bit, and then said, "I'm not sure family life. We had several more conversa-
about the program or how it fits in or any of tions along these lines, and he agreed to be a
that, but I will tell you what I was writing. I case example of the family impacts of the
was writing . . . , " and he hesitated because program. Until that time, impacts on family
his voice cracked, "a letter to my teenage son had not even been among the expected or in-
trying to tell him how I feel about him and tended outcomes of the program. It turned
make contact with him about some things. I out to be a major area of impact for a number
don't know if I'll give the letter to him. The of participants.

The Native Language of the Program

T he lunatic, the lover, and the poet

Are of imagination ali compact.
One sees more devils than vast hell can hold;
That is, the madman. The lover, ali as frantic,
Sees Helen's beauty in a brow of Egypt.
The poet's eye, in a fine frenzy rolling,
Doth glance from heaven to earth, from earth to heaven;
And as imagination bodies forth the forms of things unknown,
The poet's pen turns them to shapes
And gives to airy nothing
A local habita tion and a name.

William Shakespeare,
A Midsummer Nighfs Dreatn, Act V, scene 1

As noted in Chapter 2, the Whorf hypoth- that truck drivers were entering "empty"
esis (Schultz 1991) alerts us to the power of warehouses smoking cigarettes and cigars.
language to shape our perceptions and ex- The warehouses often contained invisible,
periences. As an insurance investigator, but highly flammable gases. He interviewed
Benjamin Whorf was assigned to look into truckers and found that they associated the
explosions in warehouses. He discovered word empty with harmless and acted accord-
Fieldzuork Strategies and Observation Methods lj. 289

ingly. Whorfs job, in Shakespeare's terms,i, plex system of language to distinguish dif-
was to turn the truckers' perception of "airyy ferent degrees and types of retarda tion, a
nothing" into the shape of possible danger. language that changes as cultural and politi-
An anthropological axiom insists that one2 cal sensitivities change. People in criminal
cannot understand another culture withoutt justice generate language for distinguishing
understanding the language of the people inn types of offenders or "perps" (perpetrators).
that culture. Language organizes our worldi Fieldwork involves learning the "native lan-
for us by shaping what we see, perceive, and1 guage" of the setting or program being stud-
pay attention to. The things for which peo- ied and attending to variations in connota-
ple have special words tell others what is im- tioris and situational use. The field notes and
portant to that culture. Thus, as studentss reports of the observer should include the
learn in introductory anthropology, Eski- exact language used by participants to com-
mos have many words for snow and Arabss municate the flavor and meaning of "na-
have many words for camel. Likewise, thee tive" program language.
artist has many words for red and differentt Language was especially important in the
kinds of brushes. wilderness education program I evaluated.
Roderick Nash (1986), in his classic study/ These were highly verbal people, well edu-
Wilderness and the American Mind, traces how/ cated, reflective and articulate, who spent a
changing European American perceptionss lot of program time in group discussioris.
of "wilderness" has affected at the deepestt Program staff understood how words can
leveis our cultural, economic, and political1 shape experiences. They wanted partici-
perspectives on deserts, forests, canyons, pants to view the time in the wilderness as a
and rivers. He traced the very idea of wilder- professional development learning experi-
ness to the eighth-century heroic epic char- ence not a vacation, so staff called each week
acter Beowulf, whose bravery was definedi in the wilderness a "field conference." They
by his courage in entering the wildeoraa hoped participants would see the program
place of wild and dangerous beasts, dark< as a "conference" held in the "field." Despite
and foreboding forests, and untamed, pri- the determined efforts of staff, however, the
mordial spirits. In the Judeo-Christian tradi- participants never adopted this language.
tion, wilderness came to connote a place off Almost universally they referred to the
uncontrolled evil that needed to be tamedi weeks in the wilderness as "trips." During
and civilized, while Eastern cultures and re- the second "field conference" the staff capit-
ligions fostered love of wilderness ratherr ulated. Interestingly enough, that capitula-
than fear. Nash credits the Enlightenmentt tion coincided with negative reactions by
with offering new ways of thinking aboutt participants to some logistical inadequacies,
wildernessand new language to shapee unsuccessful program activities, and bad
that changed thinking. weather, ali of which undercut the "confer-
Moving from the wilderness to the inte- ence" emphasis. Staff language reflected
rior territory of organizations, agencies, and1 that change.
programs, language still shapes experiencee Other language emerged that illuminated
and is therefore an important focus during5 participants' experiences. One of the partici-
fieldwork. Programs develop their own lan- pants expressed the hope of "detoxifying" in
guage to describe the problems they deal1 the wilderness. He viewed his return to his
with in their work. Educators who workk everyday world as "poisonous retoxifica-
with learning disabled students have a com- tion." The group immediately adopted this
290 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

language of detoxification and retoxification tendency to personify natural forms. Thus

to refer to "wilderness time" versus ordi- began a sustamed personal interest in how
nary "urban civilization time," ultimately names in the wilderness shaped our experi-
shortening the words to detox and retox. This ences there (Patton and Patton 2001).
language carne to permeate the program's When I took my son into the Grand Can-
culture. yon for a "coming of age" initiation experi-
The discussions in the wilderness often ence (Patton 1999a), he reacted to the prob-
reflected the physical environment in which lem of finding words for that awesome
program activities took place. Participants environment by making up words. For ex-
became skilled at creating analogies and ample, upon seeing the Canyon for the very
metaphors to contrast their urban work lives first time he whispered, "Bue dden," which
with their wilderness experiences. After became our way of describing things too
backpacking ali day, participants could be beautiful and awesome for ordinary words.
heard talking about learning how to "pace Capturing the precise language of partici-
myself in my work," or "shifting the bur- pants honors the emic tradition in anthro-
dens of responsibilities that I carry so that pology: recording participants' own under-
the load is more evenly balanced" (a refer- standings of their experiences. Observers
ence to the experience of adjusting the must learn the language of participants in
weight of the backpack). In the mountains, the setting or program they are observing
after rock climbrng, participants referred to in order to faithfully represent participants
"the danger in taking risks at work without in their own terms and be true to their
support" (a reference to the balay system of worldview.
climbing where someone supports the
climber with a safety rope below). One dis- Nonverbal Communication
cussion focused on how to "find toeholds
and handholds" to bring about change back Social and behavioral scientists have re-
home, "to get on top of the steep wall of reported at length the importance of both ver-
sistance in my institution." They even as- bal and nonverbal communication in hu-
signed numbers to degrees of back-home in- man groups. While recording the language
stitutional resistance corresponding to the of participants, the observer should also at-
numbers used to describe the degree of diffi- tend to nonverbal forms of communication.
culty of various rock climbs. On the ri ver, For example, in educational settings nonver-
participant language was filled with phrases bal communication would include how stu-
like "going with the flow," "learning to mon- dents get the attention of or otherwise ap-
itor professional development like you read proach instructors, such as waving their
and monitor the current," and "trying to hands in the air. In group settings a great
find my way out of the eddies of life." deal of fidgeting and noving about may re-
Because of the power of language to veal things about attention and involve-
shape our perceptions and experiences, ment. How participants dress, express affec-
most participants wanted to know the tion, and sit together or apart are examples
names of the rock forinations, winding can- of nonverbal cues about social norms and
yons, and river rapids we encountered, patterns.
while others, following Desert Solitaire au- Again, the wilderness program provides
thor Edward Abbey (1968), set for them- informative examples. Hugging emerged as
s eives the goal of suppressing the human a nonverbal way of providing support at
Fieldzuork Strategies and Observation Methods lj. 291

times of emotional distress or celebration, how potential interviewees reacted to her

for example, a way to recognize when some- subject matter, the study of young middle-
one had overcome some particularly diffi- class women in ndia who read Western ro-
cult challenge, like making it up across a mance novis. She had to depend on reading
ledge along a cliff face. But subgroups dif- body language to pick up hostility, disap-
fered in amount of and comfort with hug- proval, support, or openness because the
ging, and different field conferences mani- verbal formalities of some interactions of-
fested different amounts of hugging. When fered fewer cues than nonverbal reactions.
the group felt disparate, separated, with Among the young women, giggles, winks,
people on their own "trips," isolated from animated interactions, lowered eyes, and
each other, little hugging occurred either in direct gaze became cues about how the field-
pairs or around the group campfire. When work was progressing.
the depth of connection was deeper, shoul- A caution is in order here. Nonverbal be-
der-to-shoulder contact around the camp- haviors are easily misinterpreted, especially
fire was common and group singing was cross-culturally. Therefore, whenever possi-
more likely. Over time, it became possible to ble and appropriate, having observed what
read the tenor of the group by observing the appear to be significant nonverbal behav-
amount and nature of physical contact par- iors, some effort should be made to follow
ticipants were having with each otherand up with those involved to find out directly
participants in groups with a lot of hugging from them what the nonverbal behaviors re-
and connectedness reported noticeably ally meant. I confirmed with other partici-
greater personal change. pants in the wilderness program the impor-
In evaluating an international develop- tance of hugging as a mechanism that they
ment project, I observed that the three host themselves used to sense the tenor of the
country nationals ("locais") had developed group.
a subtle set of hand signals and gestures that
the American staff never noticed. In meet- Unobtrusive Observations
ings, the host country nationals regularly
communicated with each other and oper- Being observed can make people self-
ated as a team using these nonverbal signals. conscious and generate anxiety, especially
Later, having gained their confidence, I when the observations are part of an evalua-
asked the local staff members about the gestion. Regardless of how sensitively observa-
tures. They told me that the Americans had tions are made, the possibility always exists
insisted that each person participate as an that people will behave differently under
individual on an equal footing in staff meet- conditions where an observation or evalua-
ings and, to support an atmosphere of open- tion is taking place than they would if the ob-
ness, the Americans asked them not to use server were not present.
their own language during staff meetings.
But the locais wanted to operate as a unit to Even when well-intentioned and cooperative,
counter the power of the Americans, so they the research subject's knowledge that he is
developed subtle gestures to communicate participating in a scholarly search may con-
with each other since they were denied use found the invs tigator's data.... It is impor-
of their own language. tant to note early that the awareness of testing
Parameswaran (2001) has described hw need not, by itself, contaminate responses. It is
she relied on reading nonverbal cues to tell a question of probabilities, but the probability
292 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

of bias is high in any study in which a respon- ing. Ali participants were provided with
dentis aware ofhis subjectstatus. (Webb etal. learning logs at the beginning of the first
1966:13) field conference and were encouraged to use
them for private reflections and journaling.
Concern about reactions to being ob- These three-ring binders contained almost
served has led some social scientists to rec- no paper when first given to participants.
ommend covert observations as discussed Participants brought the learning logs back
earlier in this chapter. An alternative strat- each time they returned to the wilderness.
egy involves searching for opportunities to (The program involved four different trips
collect "unobtrusive measures" (Webb et al. over the course of a year.) The extent to
1966). Unobtrusive measures are those which paper had been added to the note-
made without the knowledge of the people books was one indicator of the extent to
being observed and without affecting what which the logs were being used.
is observed. The personnel of the National Forest Ser-
Robert L. Wolf and Barbara L. Tymitz vice and the Bureau of Land Management
(1978) included unobtrusive measures in have a kind of unobtrusive measure they use
their naturalistic inquiry evaluation of the in "evaluating" the wilderness habits of
National Museum of Natural History at the groups that go through an area such as the
Smithsonian Institution. They looked for San Juan River in Utah. The canyons along
"wear spots" as indicators of use of particu- the San Juan River are a very fragile environ-
lar exhibit areas. They decided that worn ment. The regulations for use of that land are
rugs would indicate thepopularity of partic- essentially "take only photographs, leave
ular areas in the museum. The creative eval- only footprints." This means that ali gar-
uator can learn a number of things about a bage, including human waste and feces, are
program by looking for physical clues. to be carried out. It takes several days to go
Dusty equipment or files may indicate down the river. By observing the amount
things that are not used. Areas that are used and types of garbage groups carry out, one
a great deal by children in a school will look can learn a great deal about the wilderness
differentthat is, more wornthan areas habits of various groups and their compli-
that are little used. ance with river regulations.
In a week-long staff training program for The creative observer, aware of the vari-
300 people, I asked the kitchen to systemati- ety of things to be learned from studying
cally record how much coffee was con- physical and social settings, will look for op-
sumed in the morning, afternoon, and eve- portunities to incorporate unobtrusive mea-
ning each day. Those sessions that I judged sures into fieldwork, thereby manifesting a
to be particularly boring had a correspond- "sympathy toward multi-method mquiry,
ingly higher levei of coffee consumption. triangulation, playfulness in data collection,
Active and involving sessions showed less outcroppings as measures, and alternatives
coffee consumption, regardless of time of to self report" (Webb and Weick 1983:210).
day. (Participants could get up and getcoffee A particularly powerful example of un-
whenever they wanted.) obtrusive fieldwork is Laura Palmer's (1988)
In the wilderness program, the thickness study of letters and remembrances left at the
of notebooks called "learning logs" became Vietnam Veterans Memorial in Washington,
an unobtrusive indicator of how engaged D.C., a work she called Shrapnel in the Heart.
participants were in self-reflective journal- For the unobtrusive part of her fieldwork,
Fieldzuork Strategies and Observation Methods lj. 293

Palmer sampled items left at the memorial, intriguing form of analysis involves com-
ali of which are saved and warehoused by paring official statements found in public
the U.S. government. She categorized and documents (brochures, board minutes, an-
analyzed types of items and the content of nual reports) with private memos and what
messages. In some cases, because of identi- the evaluation observer actually hears or
fying information contained in letters or in- sees occurring the program. Client files are
cluded with objects (photographs, baby another rich source of case data to supple-
shoes, artwork), she was able, through in- ment field observations and interviews. For
tensive investigative work, to locate the peo- example, Vesneski and Kemp (2000) coded
ple who left the materiais and interview and analyzed intake sheets and copies of
them. Their stories, the intrusive part of her family plans produced during more than
study, combined with vivid descriptions of 100 "family conferences" involving the ex~
the objects that led her to them, offer dra- tended families of abused or neglected chil-
matic and powerful insights into the effects dren in child welfare decision making in the
of the Vietnam War on the lives of survivors. state of Washington.
In one sense, her analysis of letters, journals, At the very beginning of an evaluation or
photos, and messages can be thought of as a organizational fieldwork, access to poten-
nontraditional and creative form of docu- tially important documents and records
ment analysis, another important fieldwork should be negotiated. The ideal situation
strategy. would include access to ali routine records
on clients, ali correspondence from and to
Documents program staff, financial and budget records,
organizational rules, regulations, memo-
Records, documents, artifacts, and ar- randa, charts, and any other official or unof-
chiveswhat has traditionally been called ficial documents generated by or for the pro-
"material culture" in anthropologyconsti- gram. These kinds of documents provide the
tute a particularly rich source of information evaluator with information about many
about many organizations and programs. things that cannot be observed. They may
Thus, archival strategies and techniques reveal things that have taken place before
constitute part of the repertoire of field re- the evaluation began. They may include pri-
search and evaluation (Hill 1993). In con- vate interchanges to which the evaluator
temporary society, ali kinds of entities leave would not otherwise be privy. They can re-
a trail of paper and artifacts, a kind of spoor veal goals or decisions that might be other-
that can be mined as part of fieldwork. Fam- wise unknown to the evaluator.
ilies keep photographs, children's school- In evaluating the mission fulfillment of a
work, letters, old Bibles with detailed gene- major philanthropic foundation, I examined
alogies, bronze d baby shoes, and other 10 years of annual reports. Each report was
sentimental objects that can inform and en- professionally designed, elegantly printed,
rich family case studies. People who commit and widely disseminatedand each report
suicide leave behind suicide notes that can state d a slightly different mission for the
reveal pattems of despair in a society foundation. It turned out that the president
(Wilkinson 1999). Gangs and others inscribe of the foundation wrote an annual introduc-
public places with graffiti. Organizations of tion and simply stated the mission from
ali kinds produce mountains of records, memory. The publication designer routinely
both public and private. Indeed, an oft- lifted this "mission statement" from the
294 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

presidenfs letter and highlighted it in bold utive staff in the project. Disagreements
font at the beginning of the report, often on about program finances constituted but one
the cover page. From year to year the focus arena of communication difficulties during
changed until, over the course of 10 years, the program, including time in the wilder-
the stated mission had changed dramati- ness. Interviews with those involved re-
cally without official board action, approval, vealed quite different perceptions of the na-
or even awareness. Further invstigation ture of the conflicts, their intensity, and their
through years of board minutes revealed potential for resolution. While participants
that, in fact, the board had never adopted a became aware of some arguments among
mission statement at ali, a matter of consid- staff, for the most part they were unaware of
erable surprise to ali involved. the origins of those conflicts and the extent
As this example shows, documents prove to which program implementation was
valuable not only because of what can be hampered by them.
learned directly from them but also as stimu- My review of files also revealed the enor-
lus for paths of inquiry that can be pursued mous complexity of the logistics for the wil-
only through direct observation and inter- derness education program. Participants
viewing. As with ali information to which an (college deans, program directors, adminis-
evaluator has access during observations, trators) were picked up at the airport in vans
the confidentiality of program records, par- and driven to the wilderness location where
ticularly client records, must be respected. the field conference would take place. Par-
The extent to which actual references to and ticipants were supplied with ali the gear nec-
quotations from program records and docu- essary for surviving in the wilderness. Prior
ments are included in a final report depends to each field trip, staff had many telephone
on whether the documents are considered and written exchanges with individual par-
part of the public record and therefore able ticipants about particular needs and fears.
to be publicized without breach of confiden- Letters from participants, especially those
tiality. In some cases, with permission and new to the wilderness, showed how little
proper safeguards to protect confidentiality, they understood about what they were get-
some information from priva te documents ting into. One seasoned administrator and
can be quoted directly and cited. hard-core smoker inquired, with reference
Program records can provide a behind- to the first 10-day hike in the heart of the Gila
the-scenes look at program processes and wilderness, "Will there be a place to buy cig-
how they came into being. In the wilderness arettes along the way?" Talk about being
program evaluation, program staff made clueless! But by the end of the year of field
their files available to me. I discovered a trips, he had given up smoking. His letter of
great deal of information not available to inquiry alerted me to the impor tance of this
other program participants: letters detailing pre-post observation.
both conceptual and financial debates be- Without having looked over this corre-
tween the technical staff (who led the wil- spondence, I would have missed the extent
derness trips) and the project directors (who to which preparation for the one-week expe-
had responsibility for the overall manage- riences in the wilderness consumed the time
ment of the program). Without knowledge and energy of program staff. The intensity of
of those arguments it would have been im- work involved before the field conferences
possible to fully understand the nature of helped explain the behavior of staff once the
the interactions between field staff and exec- field trips got under way. So much had gone
Fieldzuork Strategies and Observation Methods lj. 295

into the preparations, virtually none of tion that I include among the tasks of the ob-
which was appreciated by or known to pro- server that of noting what does not occur.
gram participants, that program staff would If social science theory, program goals,
sometimes experience a psychological let- implementa tion designs, and/or proposals
down effect and have difficulty energizing suggest that certain things ought to happen
themselves for the actual wilderness experi- or are expected to happen, then it is appro-
ence. priate for the observer or evaluator to note
Learning to use, study, and understand that those things did not happen. If a com-
documents and files is part of the repertoire munity where water is scarce shows no evi-
of skills needed for qualitative inquiry. For dence of conflict over water rights, an an-
an extended discussion of the interpreta tion thropologist could be expected to report and
of documents and material culture, see explain this absence of community conflict.
Hodder (2000). If a school program is supposed to, accord-
ing to its funding mandate and goals, pro-
vide children with opportunities to explore
Observing What the community and no such explorations
Does Not Happen occur, it is altogether appropriate for the
evaluator to note said implementation fail-
The preceding sections have described ure. If the evaluator repor ted only what oc-
the things one can observe in a setting or curred, a question might be left in the mind
program. Observing activities, interactions, of the reader about whether the other activi-
what people say, what they do, and the na- ties had occurred but had simply not been
ture of the physical setting is important in a observed. Likewise, if a criminal justice pro-
comprehensive approach to fieldwork. But gram is supposed to provide one-to-one
what about observing what does not hap- counseling to juveniles and no such counsel-
pen? ing takes place, it is entirely appropriate for
The potential absurdity of speculating the evaluator to note the absence of coun-
about what does not occur is illustrated by a seling.
Sufi story. During a plague of locusts, the In observing early childhood programs,
wise-fool Mulla Nasrudin, always looking the absence of children's art on the walls in
on the bright side, went from village to vil- one center stood out. Indeed, the absence of
lage encouraging people by observing how any colorful posters or art of any kind stood
fortunate they were that elephants had no out because ali other centers' walls were
wings. "You people don't realize how lucky covered with colorful displays. When I
you are. Imagine what life would be like pointed this out, embarrassed staff members
with elephants flying overhead. These lo- explained that they had set in motion a plan-
custs are nothing." ning process for decorating the walls that
To observe that elephants have no wings had become bogged down and they had just
is indeed data. Moreover, elephants have no neglected to get back to the issue because,
fins, claws, feathers, or branches. Clearly, they realized, they got gotten used to the
once one ventures into the area of observ- way things were.
ing what does not happen, there are a near- Thus, it can be appropriate to note that
infinite number of things one could point something did not occur when the ob-
out. The "absence of occurrence" list could serveis basic knowledge of and experience
become huge. It is therefore with some cau- with the phenomenon suggests that the ab-
296 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

sence of some particular activity or factor is goals would include statements about the
noteworthy. This clearly calls for judgment, necessity of staff being sensitive to the par-
common sense, and experience. As eminent ticular needs, interests, and cultural pat-
qualitative methodologist Bob Stake (1995) terns of minorities, but there may not be
has asserted: specific mention of the desired racial com-
position of program staff. If, then, the eval-
One of the principal qualifications of qualita- uator observes that the staff of the pro-
tive researchers is experience. Added to the gram consists entirely of Caucasians, it is
experience of ordnary looking and thinking, appropriate to report that the staff is ali
the experience of the qualitative researcher is White, that is, no people of color are
one of knowing what leads to significant un- among the program staff, the importance
derstanding, recognizmg good sources of of which derives from the location and na-
data, and consciously and unconsciously test- ture of the program.
ing out the veracity of their eyes and robust- Observations of staff interaction and
ness of their interpretations. It requires decision-making processes also provide
sensitivity and skepticism. Much of this meth- opportunities for evaluators to note things
odological knowledge and personality come that do not happen. If, over time, the ob-
from hard work under the criticai examina tion server notes that program planning pro-
of colleagues and mentors. (pp. 49-50) cesses never include participants' rnput in
any systematic or direct way, it may well
Making informed judgments about the be appropriate for the evaluator to pornt
significance of nonoccurrences can be out the absence of such rnput based on ex-
among the most important contributions an periences indicating the significance of
evaluator can make because such feedback participant input in the planning pro-
can provide program staff members or other cesses of other programs.
evaluation users with information that they My evaluation of the wilderness educa-
may not have thought to request. Moreover, tion program included observations about
they may lack the requisite experience or a number of things that did not occur. No
awareness to have noticed the absence of serious injuries occurred at any of the six
that which the evaluator observes. For ex- field conferences in the wildernessim-
ample, the absence of staff conflict is typi- portant information for someone thinking
cally noteworthy because staff conflict is about the possible risks involved in such a
common. Similarly, absence of conflict be- program. No participant refused to shoul-
tween administrative leveis (local, state, and der his or her share of the work that had to
federal) would be noteworthy because such be done in order for the group to live and
conflict is, in my experience, virtually uni- work together in the wilderness. This ob-
versal. servation emerged from discussions with
In many such cases, the observation technical field staff who often worked
about what did not occur is simply a restate- with juveniles in wilderness settings
ment, in the opposite, of what did occur. where uneven sharing of cooking, clean-
That restatement, however, will attract ating, and related responsibilities often led
tention in a way that the initial observation to major group conflicts. The fact that the
might not. For example, if one were observ- groups I observed never had to deal with
ing a program being conducted in a multira- one ortwo people not helping out was
cial community, it is possible that program worth noting.
Fieldzuork Strategies and Observation Methods lj. 297

Perhaps the most important observation tion, may also be presented within the
about what did not happen came from ob- larger case. The qualitative analysis pro-
serving staff meetings. Over time, I noticed a cess typically centers on presentation of
pattern in which s taff held meetings to make specific cases and thematic analysis across
decisions about important issues, but no cases. Knowing this, fieldwork can be or-
such decisions were made. Staff sometimes ganized around nested and layered case
thought that a decision had been made, but studies, which means that some form of
closure was not brought to the decision- nested case sampling must occur.
making process and no responsibility for Let me briefly review the centrality of
follow-up was assigned. Many subsequent case studies as a qualitative inquiry strat-
implementation failures and staff conflicts egy. Chapter 1 opened by citing a number
could be traced to ambiguities and differ- of well-known and influential books
ences of opinion that were left uriresolved at based on case studies, for example, In
staff meetings. By hearing me describe both Search of Excellence: Lessons From America's
what was and was not occurring, staff be- Best-Run Companies by Peters and Water-
came more explicit and effective in making man (1982), Angela Browne's important
decisions. Reporting what did happen in book When Battered Women Kill (1987), and
staff meetings was important, but it was also Sara Lawrence-Lightfoofs six detailed
extremely important to observe what did case studies in Respect (2000:13). Chapter 2
not happen. presented the construction of unique case
studies as a major strategic theme of quali-
Nested and Layered Case tative inquiry. Chapter 3 reviewed theoret-
ical perspectives that are inductively case
Studies During Fieldwork
based. Chapter 4 reviewed at some length
the importance in qualitative evaluation of
A case study is expected to catch the complex- capturing and reporting individualized out-
ity of a single case. The single leaf, even a sin- comes based on case studies of how partici-
gle toothpick, has unique complexitiesbut pants in programs change during a pro-
rarely will we care enough to submit it to case gram and whether they maintain those
study. We study a case when it itself is of very changes afterward. To illustrate this point,
special interest. We look for the detail of inter- in the wilderness education program our
action with its context. Case study is the study evaluation team constructed case studies
of the particularity and complexity of a single of participants using multiple sources of
case, coming to understand its activity within data from fieldwork: (1) background data
important circuinstances. (Stake 1995:xi) gathered through interviews about partic-
ipants' situations and perspectives upon
Months of fieldwork may result in a sin- entering the year of field conferences; (2)
gle case study that describes a village, com- observations of their experiences during
munity, neighborhood, organization, or pro- field conferences; (3) informal and conver-
gram. However, that single case study is sational interviews with them during the
likely to be made up of many smaller wilderness trips; (4) quotations from for-
casesthe stories of specific individuais, mal group interviews (focus groups) held
families, organizational units, and other at various times during the trips; (5) ex-
groups. Criticai incidents and case studies of cerpts from their journals and other per-
specific bounded activities, like a celebra- sonal writings when they were willing to
298 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

share those with us, as they often were; and tiative. As Exhibit 6.4 (p. 300) shows, how-
(6) follow-up telephone interviews with par- ever, within that overall evaluation case
ticipants after each field trip and after the en- study were nested individual case studies
tire program was completed to track the im- documenting individual experiences and
pact of the program on individuais over outcomes; case studies of each yearlong
time. group cohort; and case studies of each sepa-
Let me pause at this point and note some ra te field conference, for example, the 10
confusion in the qualitative literature about days in the Gila wilderness or the 10 days in
terminology. For example, sociologists the Kofa Mountains. Slicing through the
Hamel, Dufour, and Fortin (1993) ask: fieldwork and analysis in other ways were
case studies of particular incidents, for ex-
But is the case study a method? Or is it an ap- ample, the emotional catharsis experienced
proach? . . . Case studies employ various meth- by one participant when she finally man-
ods. These can include interviews, participant aged to overcome her terror and rappel
observation, and field studies. Their goals are down a cliff face, the whole group watching
to reconstruct and analyze a case from a socio- and urging her on, a process that took some
logical perspective. It would thus be more ap- 45 tense minutes. Other mini-cases consisted
propriate to define the case study as an of different units of analysis. A full day's
approach, although the term case method sug- hike could be a case, as could running a spe-
gests that it is indeed a method. (p. 1) cific dangerous rapid on the San Juan River.
Each evening discussion constituted a case
Whatever term or phrase is used, case stud- such that that over the three years, we had
ies depend on clearly defining the object of notes on over 80 discussions of various kinds.
study, that is, the case. But this too is com- Staff meetings made for a different unit of
plex. analysis and therefore a different series of
When more than one object of study or case studies. Thus, extended fieldwork can
unit of analysis is included in fieldwork, and typically does involve many mini- or
case studies may be layered and nested micro-case studies of various units of analy-
within the overall, primary case approach. sis (individuais, groups, specific activities,
William Foote Whyte's (1943) classic study specific periods of time, criticai incidents),
Street Comer Society has long been recog- ali of which together make up the overall
nized as an exemplar of the single-commu- case study, in this example, the final evalua-
nity (N = 1) case study (e.g., Yin 1989) even tion of the wilderness education program.
though his study of "Cornerville" includes Chapter 5 discusses at length various units
the stories (case studies) of several individ- of analysis and sampling strategies for case
ual lower-income youth, some of whom studies (see especially Exhibit 5.5 [p. 231] on
were striving to escape the neighborhood. units of analysis and Exhibit 5.6 [pp. 243-
The wilderness program illustrates how 244] on purposeful sampling strategies).
case studies often are layered and nested. Fieldwork, then, can be thought of as en-
The three-year wilderness program consti- gaging in a series of multilayered and nested
tuted the overall, one might say macro, case case studies, often with intersecting and
study. The final evaluation report presented overlapping units of analysis. One final case
conclusions about the processes and out- study deserves considerationthe observ-
comes of the overall program, a case exam- e i s experiences and reactions. We turn to
ple of a three-year wilderness education ini- that now.
Fieldzuork Strategies and Observation Methods lj. 299

Observing Oneself

P hysician, heal thyself.

Observer, observe thyself.

Halcolm

In the second chapter, I identified voice class, I did not have the language to engage in
and perspective, or reflexivity, as one of the a systematic feminist critique of patriarchy or
central strategic themes of contemporary, nationalism. Feminism for me had been unfor-
postmodern qualitative inquiry. The term tunately constructed as an illness that struck
reflexivity has entered the qualitative lexicon highly Westernized mtellectual lndian wom-
as a way of emphasizing the importance of en who were out of touch with reality.... [I]t
self-awareness, political/ cultural conscious- was my dislocation from ndia to the relatively
ness, and ownership of one's perspective. radicalized context of the United States that
Reflexivity reminds the qualitative inquirer prompted my political development as a femi-
to observe herself or himself so as to be at- nist and a woman of color. (p. 76)
tentive to and conscious of the cultural, po-
litical, social, linguistic, and ideological ori- Given this background and the contro-
gins of her or his own perspective and voice versial focus of her fieldwork (reading of
as well asand often in contrast tothe Western romance novis by young lndian
perspectives and voices of those she or he women), she identified reflective questions
observes and talks to during fieldwork. Re- to guide her reflexive inquiry during and af-
flexivity calls for self-reflection, indeed, crit- ter fieldwork:
icai self-reflection and self-knowledge, and
a willingness to consider how who one is af-
fects what one is able to observe, hear, and How do kinship roles assigned to native schol-

understand in the field and as an observer ars shape social interactions in the field? How

and analyst. The observer, therefore, during can commitments to sisterhood make it diffi-

fieldwork, must observe self as well as oth- cult for femmist ethnographers to achieve crit-

ers, and interactions of self with others. icai distance and discuss female informants'
prejudiced views? (p. 76)
Once again, for continuity, I cite Para-
meswaran (2001), who has written a won-
derfully self-reflective account of her ex- Her personal inquiry into these questions,
perience returning to her native ndia to do reflecting on her own fieldwork experiences
fieldwork as a feminist scholar after being (Parameswaran 2001), is a model of reflex-
educated in United States. ivity.
Many year ago, lndian philosopher J.
Because my parents were fairly liberal com- Krishnamurti (1964) commented on the
pareci to many of my friends' parents, I grew challenges of self-knowledge. Although his
up with a little more awareness than many reflections were directed to the importance
middle- and upper-class Indians of the differ- of lifelong learning rather than to being re-
ences between my life and that of the vast ma- flexive in fieldwork, his ruminations offer a
jority of Indians. Although I questioned some larger context for thinking about how to ob-
restrictions that were specific to women of my serve oneself, a context beyond concern
300 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Nested, Layered, and Overlapping Mini-Case

Studies During Fieldwork: Example From the
E X H I B I T 6.4 Wilderness Education Program Evaluation

The wilderness education program evaluation illustrates how case studies often are layered and
nested. Evaluation of the three-year wilderness program constituted the overall macro case
study. Nested and layered within that overall evaluation were various mini-cases of overlap-
ping and intersecting units of analysis that helped organize and frame fieldwork.
Macro Case Study: Final Evaluation Study of the Three-Year Program
Possible nested, layered, and overlapping mini-case studies
Fieldzuork Strategies and Observation Methods lj. 301

about methodological authenticity, though mately deal with issues of authenticity, reac-
his advice applies to that as well. tivity, and how the observational process
may have affected what was observed as
Self-knowledge comes when you observe well as how the background and predispo-
yourself in your relationship with your fel- sitions of the observer may have con-
low-students and your teachers, with ali the strained what was observed and under-
people around you; it comes when you ob- stood. Each of these areas of methodological
serve the manner of another, his gestures, the inquiry depends on some degree of criticai
way he wears his clothes, the way he talks, his reflexivity.
contempt or flattery and your response; it comes
when you watch everything in you and about
you and see yourself as you see your face in Sources of Data Reviewed
the mirror. . . . Now, if you can look into the
mirror of relationship exactly as you look into This lengthy review of options in what to
the ordinary mirror, then there is no end to observe and sources of data for evaluation
self-knowledge. It is like entering a fathomless fieldwork began with the suggestion that a
ocean which has no s h o r e . . . . ; if you can just sensitizingframework canbe useful as a tool to
observe what you are and move with it, then guide fieldwork. The list of data sources
you will find that it is possible to go infinitely we've reviewed can be used to stimulate
far. Then there is no end to the journey, and thinking about evaluation fieldwork pos-
that is the mystery, the beauty of it. (Krishna- sibilities. Other phenomena and other ob-
murti 1964:50-51, emphasis added) servational arenas would have different
sensitizing frameworks or concepts. The fol-
I realize that Krishnamurti's phrase lowing summarizes the observation and in-
"There is no end to the journey" may strike quiry topics we've reviewed for evaluation:
terror in the hearts of graduate students
reading this in preparation for dissertation n Description of the program setting/
fieldwork or evaluators facing a report physical environment
deadline. But, remember, he's taking about
H Description of the social environment
lifelong learning, of which the dissertation
or a specific evaluation report is but one n Capturing historical perspectives
phase. Just as most dissertations and evalua- o Describing planned program implemen-
tions are reasonably expected to contribute tation activities and structured interac-
incrementai knowledge rather than make tions
major breakthroughs, so too the self-knowl-
Observing informal interactions and un-
edge of reflexive fieldwork is but one phase
planned activities
in a lifelong journey toward self-knowl-
edgebut it's an important phase and a a Recording participants' special program
commitment of growing significance as re- language
flexivity has emerged as a central theme in Observing nonverbal communication
qualitative inquiry.
Watching for unobtrusive indicators
The point here, which we shall take up in
greater depth in the chapters on analysis and H Analyzing documents, files, records,
credibility, is that the observer must ulti- and artifacts
302 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Commenting on notable nonoccurrences L=J. Doing Fieldwork: The

(what does not happen) Data-Gathering Process
Constructing nested and layered case
studies during fieldwork for intersecting The purpose of the research has been clari-
and overlapping units of analysis fied. The primary research questions have
been focused. Qualitative methods using
Observing oneself: Reflexivity
observations have been selected as one of
o Document mg individualized and com- the appropriate methods of data gathering.
mon outcomes (Chapter 4) It is time to enter the field. Now begins the
arduous task of taking field notes.

Creativity in Fieldwork
Field Notes

No checklist can be relied on to guide ali Many options exist for taking field notes.
aspects of fieldwork. A participant observer Variations include the writing materiais
must constantly make judgments about used, the time and place for recording field
what is worth noting. Because it is impossi- notes, the symbols developed by observers
ble to observe everything, some process of as their own method of shorthand, and how
selection is necessary. Plans made during field notes are stored. No universal prescrip-
design should be revised as appropriate tions about the mechanics of and procedures
when important new opportunities and for taking field notes are possible because
sources of data become available. That's different settings lend themselves to differ-
where flexibility and creativity help. Cre- ent ways of proceeding and the precise orga-
ativity can be learned and practiced (Patton nization of fieldwork is very much a matter
1987a). Creative fieldwork means using ev- of personal style and individual work hab-
ery part of oneself to experience and under- its. What is not optional is the taking of
stand what is happening. Creative insights field notes.
come from being directly involved in the set-
Aside from getting along in the setting,
ting being studied.
the fundamental work of the observer is the
I shall return to the issue of creativity in
taking of field notes. Field notes are "the
considering the interpretation of field notes
most important determinant of later bring-
later in this chapter and again in the analysis
ing off a qualitative analysis. Field notes pro-
chapter. For the moment, it is sufficient to ac-
vide the observer's raison d'tre. I f . . . not
knowledge the centrality of creativity in nat-
doing them, [the observer] might as well not
uralistic inquiry and to concur with Virginia
be in the setting" (Lofland 1971:102).
Woolf:
Field notes contain the description of
what has been observed. They should con-
Odd how the creative power at oncebrings the tain everything that the observer believes to
whole universe to order. . . . I mark Henry be worth noting. Don't trust anything to fu-
James' sentence: observe perpetually. Observe ture recall. At the moment one is writing it is
the oncome of age. Observe greed. Observe very tempting, because the situation is still
my own despondency. By that means it be- fresh, to believe that the details or particular
comes serviceable. (quoted in Partnow 1978: elements of the situation can be recalled
185) later. If it's important as part of your con-
Fieldzuork Strategies and Observation Methods lj. 303

tions and conditions. Words such as poor, an-

[v- ger, and uneasy are insufficiently descriptive.
P / IT!!:!):!! nVWl^ M ! => ! ->": i; ri"i:!! ; ,:=; :! -- iit:'!:!.';-- Such interpretive words conceal what actu-
,'.i;i:=i1 | |; :'| h::j"j; i:=.hif i;!:" jl?i !:;=!;. " :!i Si!1- ally went on rather than reveal the details of
I|:i:5:!R;j;;=ji ] !;i,!ii'j::!;i :!;lij:!:!N !r;| i:i"ii' | ,:'I:!:.' i-!i I R - the situation. Such terms have little meaning
; :!i!:i!!i! M^TM '."Jm iSY-fe ' for a person present for the observation.
.ITKRIIRLJ;!: I-M;!^!;!;:: ! !';' " - P E ^ :' I ^ I J M ! ' I !;' U J
:
Moreover, the use of such terms in field
lri l!-' i!r! IliiiiiyK-jjtiiMITyill:':!; l!.iX ;'ii!:"j- !'! hi-r. notes, without the accompanying detailed
s-itra i-lil! fi;i'!!/l!5h;V'Of:i!n
: ! : ;Vi i'- description, means that the fieldworker has
:= !;=! IV! vi;;;':, lVii:n: 'm* i-i-fiV'i:irhnj.Vi:h.
! : ; fallen into the bad habit of primarily record-
'IrT! jlhbii-T- it^Hi^iTi-ili^f ing interpretations rather than description.
=!.=:I-I:=i: U !:!^J';I;I1!IXS!.'! I^^P:1! |;V;.!M!!:!; HI^TEAV:: Particularly revealing are terms that can
1'!:!' ifiiULiljr! j;!i:!i;!::iiii!:!i:i i^i j;=.
! ; ; ; j 1j
; make sense only in comparison to some-
i'!."'!! ip. i!'!'!T l i ) rflt J!iJ!l 'Ji>!!
1 ! ... thing else. The phrase "poorly dressed" re-
:L.!;-iV ith!!:!; hiUiTlvi:!!:' !!:';i!:j;j!':'!: jrllii-^lr-^n quires some frame of reference about what
S'LI::;=V!=::' & I - ^ I ' ; . ; ; . ! J ; J : ; =';;
constitutes "good dress." No skill is more
criticai in fieldwork than learning to be de-
ihTS"! j;!!i!j-"::ii \\ \ . :;::;;!:;! !::L::!V: -: !
scriptive, concrete, and detailed.
Field notes also contain what people say.
Direct quotations, or as near as possible re-
sciousness as an observer, if it's information call of direct quotations, should be captured
that has helped you understand the context, during fieldwork, recording what was said
the setting, and what went on, then as soon during observed activities as well as re-
as possible that information should be cap- sponses garnered during interviews, both
tured in the field notes. formal and conversational. Quotations pro-
First and foremost, field notes are de- vide the "emic perspective" discussed ear-
scriptive. They should be dated and should lierthe insider 's perspectivewhich "is at
record such basic information as where the the heart of most ethnographic research"
observation took place, who was present, (Fetterman 1989:30).
what the physical setting was like, what so- Field notes also contam the observer's
cial interactions occurred, and what activi- own feelings, reactions to the experience,
ties took place. Field notes contain the de- and reflections about the personal meaning
scriptive information that will permit you to and significance of what has been observed.
return to an observation later during analy- Don't deceive yourself into thinking that
sis and, eventually, permit the reader of the such feelings can be conjured up again sim-
study's findings to experience the activity ply by reading the descriptions of what took
observed through your report. place. Feelings and reactions should be re-
The passages in Exhibit 6.5 on the next corded at the time they are experienced,
page illustrate different kinds of descriptive while you are in the field. Both the nature
field notes. On the left side are vague and and intensity of feelings should be recorded.
overgeneralized field notes. On the right In qualitative inquiry, the observer's own
side are more detailed and concrete field experiences are part of the data. Part of the
notes from the same observation. purpose of being in a setting and getting
These examples illustrate the probiem of close to the people in the setting is to permit
using general terms to describe specific ac- you to experience what it is like to be in that
304 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

E X H B I T 6.5 Fieldnotes Comparisons

Vague and
Overgenerolized Notes Detailed and Concrete Nates

1. The new client was 1. At first the new client sat very stiffiy on the chair next to the
uneasy waiting for receptionisfs desk. She picked up a magazine and let the pages
her intake interview. flutter through her fingers very quickly without really looking at
any of the pages. She set the magazine down, looked at her watch,
pulled her skirt down, picked up the magazine again, set it back
down, took out a cigarette and lit it. She watched the receptionist
out of the corner of her eye and glanced at the two or three other
people waiting in the room. Her eyes moved from people to the
magazine to the cigarette to the people to the magazine in rapid
succession, but avoided eye contact. When her name was finaIly
called, she jumped like she was startled.

2. The client was quite 2. When Judy, the snior staff member, told the client that she could
hostile toward the not just do whatever she wanted to do, the client began to yell,
staff person. screaming that Judy couldn't couldn't control her life, accused
Judy of being on a "power trip," and said that she'd "like to beat
the shit out of her," then told her to "go to hell." The client shook
her fist in Judy's face and stomped out of the room, leaving Judy
standing there with her mouth open, looking amazed.

3. The next student 3. The next student who carne into the room wore clothes quite
who carne in to take different from the three previous students. The other students had
the test was very hair carefully combed, clothes clean, pressed, and in good condition
poorly dressed. with colors coordinated. This new student wore soiled pants with
a tear in one knee and a threadbare seat. His flannel shirt was
wrinkled with one tail tucked into the pants and the other tail
hanging out. His hair was disheveled and his hands looked liked
he'd been playing in the engine of a car.

setting. If what it is like for you, the observer come a mechanical recording inachine on
or participant observer, is not recorded in entering the field. Insights, ideas, inspira-
your field notes, then much of the purpose tionsand yes, judgments, toowill occur
for being there is lost. while making observations and recording
Finally, field notes include your insights, field notes. It's not that you sit down early on
interpretations, beginning analyses, and and begin the analysis and, if you're an eval-
working hypotheses about what is happen- uator, make judgments. Rather, it's in the na-
ing in the setting and what it means. While ture of our intellects that ideas about the
you should approach fieldwork with a disci- meaning, causes, and significance of what
plined intention not to impose preconcep- we experience find their way into our
tions and early judgments on the phenome- minds. These insights and inspirations be-
non being experienced and observed, come part of the data of fieldwork and
nevertheless, as an observer you don't be- should be recorded in context in field notes.
Fieldzuork Strategies and Observation Methods lj. 305

I like to set off field interpretations with their journals, but some of the expansion
brackets. Others use parentheses, asterisks, had to be completed after the weeklong field
or some other symbol to distinguish inter- conference. In evaluating a leadership train-
pretations from description. The point is ing program as a participant observer, the
that interpretations should be understood to staff facilitator privately asked me not to
be just that, interpretations, and labeled as take notes during group discussions be-
such. Field-based insights are suffciently cause it made him nervous, even though
precious that you need not ignore them in most other participants were taking notes.
the hopes that, if really important, they will The extent to which notes are openly re-
return la ter. corded during the activities being observed
Field notes, then, contain the ongoing is a function of the observer's role and pur-
data that are being collected. They consist of pose, as well as the stage of participant ob-
descriptions of what is being experienced servation. If the observer or evaluator is
and observed, quotations from the people openly identified as a short-term, externai,
observed, the observer's feelings and reac- nonparticipant observer, participants may
tions to what is observed, and field-gener- expect him or her to write down what is go-
ated insights and interpretations. Field notes ing on. If, on the other hand, one is engaged
are the fundamental database for construct- in longer-term participant observation, the
ing case studies and carrying out thematic early part of the process may be devoted to
cross-case analysis in qualitative research. establishing the participant observer role
with emphasis on participation so that open
Procedurally Speaking taking of notes is deferred until the field-
worker's role has been firmly established
When field notes are written will depend within the group. At that point, it is often
on the kind of observations being done and possible to openly take field notes since, it is
the nature of your participa tion in the set- hoped, the observer is better known to the
ting being studied. In an evaluation of a par- group and has established some degree of
ent education program, I was introduced to trust and rapport.
the parents by the staff facilitator and ex- The wilderness program evaluation in-
plained the purpose of the evaluation and volved three 10-day trips ("field confer-
assured the parents that no one would be ences") with participants at different times
identified. I then openly took extensive notes during the year. During the first field confer-
without participating in the discussions. Im- ence, I never took notes openly. The only
mediately following those sessions, I would time I wrote was when others were also writ-
go back over my notes to fill in details and be ing. During the second field conference, Ibe-
sure what I had recorded made sense. By gan to openly record observations when dis-
way of contrast, in the wilderness education cussions were going on if taking notes did
program I was a full participant engaged not interfere with my participation. By the
in full days of hiking, rock climbing, and third week, I felt I could take notes whenever
rafting/kayaking. I was suffciently ex- I wanted to and I had no indication from
hausted by the end of each day that I seldom anyone that they even paid attention to the
stayed awake making field notes by flash- fact that I was taking notes. By that time I
light while others slept. Rather, each night I had established myself as a participant, and
jotted down basic notes that I could expand my participant role was more primary than
during the time that others were writing in my evaluator role.
306 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

The point here is that evaluator observers the program. By using a combination of ob-
must be strategic about taking field notes, servations, mterviewing, and document
timing their writing and recording in such a analysis, the fieldworker is able to use dif-
way that they are able to get their work done ferent data sources to validate and cross-
without unduly affecting either their partici- check findings. Each type and source of data
pation or their observations. Given those has strengths and weaknesses. Using a com-
constraints, the basic rule of thumb is to bination of data typestriangulation, a re-
write promptly, to complete field notes as currmg theme in this book increases va-
soon and as often as physically and pro- lidity as the strengths of one approach can
grammatically possible. compensate for the weaknesses of another
Writing field notes is rigorous and de- approach (Marshall and Rossman 1989:
manding work. Lofland (1971) has de- 79-111).
scribed this rigor quite forcefully: Limitations of observations include the
possibility that the observer may affect the
Let me not deceive the reader. The writing of situation being observed m unknown ways,
field notes takes personal discipline and time. program staff and participants may behave
It is ali too easy to put off actually writing in some atypical fashion when they know
notes for a given day and to skip one or more they are being observed, and the selective
days. For the actual writing of the notes may perception of the observer may distort the
take as long or longer than did the observa- data. Observations are also limited in focus-
tion! Indeed, a reasonable rule of thumb here ing only on externai behaviorsthe ob-
is to expect and plan to spend as much time server cannot see what is happening inside
writing notes as one spent in observing. This people. Moreover, observational data are of-
is, of course, not i n v a r i a n t . . . but one point is ten constrained by the limited sample of ac-
inescapable. Ali the fun of actually being out tivities actually observed. Researchers and
and about monkeying around in some setting evaluators need other data sources to find
must also be met by cloistered rigor in com- out the extent to which observed activities
mitting to paperand therefore to future use- are typical or atypical.
fulnesswhat has taken place. (p. 104) Interview data limitations include possi-
bly distorted responses due to personal bias,
anger, anxiety, politics, and simple lack of
Observations, Interviews, awareness since interviews can be greatly af-
and Documentation: fected by the emotional state of the inter-
Bringing Together viewee at the time of the interview. Inter-
view data are also subject to recall error,
Multiple Perspectives
reactivity of the interviewee to the inter-
Fieldwork is more than a single method or viewer, and self-serving responses.
technique. For example, evaluation field- Observations provide a check on what is
work means that the evaluator is on-site reported in interviews; interviews, on the
(where the program is happening) observ- other hand, permit the observer to go be-
ing, talking with people, and going through yond externai behavior to explore feelings
program records. Multiple sources of infor- and thoughts.
mation are sought and used because no sin- Documents and records also have limita-
gle source of information can be trusted to tions. They may be incomplete or maccurate.
provide a comprehensive perspective on Client files maintained by programs are no-
Fieldzuork Strategies and Observation Methods lj. 307

toriously variable in quality and complete- wilderness as an environment for experien-

ness, with great detail in some cases and vir- tial education. Each of the staff members
tually nothing in others. Document analysis, described in interviews his or her percep-
however, provides a behind-the-scenes look tions of how these former relationships af-
at the program that may not be directly ob- fected the field operations of the program,
servable and about which the interviewer including difficulties in communication that
might not ask appropriate questions with- had emerged during planning sessions prior
out the leads provided through documents. to the actual field conferences. Some of those
By using a variety of sources and re- conflicts were documented in letters and
sources, the evaluator observer can build on memos. Reading their files and correspon-
the strengths of each type of data collection dence gave me a deeper understanding of
while minimizing the weaknesses of any the different assumptions and values of var-
single approach. This mixed methods, trian- io us staff members. But the documentation
gulated approach to fieldwork is based on would not have made sense without the in-
pragmatism (Tashakkori and Teddlie 1998) terviews, and the focus of the interviews
and is illustrated in my attempt to under- carne from the field observations. Taken to-
stand some of the problems in volve d in staff gether, these diverse sources of informa-
communication during the wilderness edu- tion and data gave me a complete picture of
cation evaluation. I mentioned this example staff relationships. Working back and forth
earlier, but I'd like to expand it here. among individual staff members and group
As noted, two kinds of staff worked in the staff meetings, I was able to use this informa-
program: (1) those who had overall manage- tion to assist staff members in their efforts to
ment and administrative responsibility and improve their communication during the fi-
(2) the technical staff, who had responsibil- nal field conference. Ali three sources of in-
ity for wilderness skills training, field logis- formation proved criticai to my understand-
tics, and safety. The technical staff had exten- ing of the situation, and that understanding
sive experience leading wilderness trips, but enhanced my effectiveness in providing
they also were skilled at facilitating group feedback as a forma tive evaluator.
processes. During the trips, the lines of re-
sponsibility between technical staff and ad-
ministrative staff were often blurred and, on The Technology of
occasion, these ambiguities gave rise to con- Fieldwork and Observation
flicts. I observed the emergence of conflict
early on the first trip but lacked context for The classic image of the anthropological
knowing what wasbehind these differences. fieldworker is of someone huddled in an Af-
Through interviews and casual conversa- rican hut writing voluminously by lantern.
tions during fieldwork, I learned that ali of Contemporary researchers, however, have
the staff, both administrative and technical, available to them a number of technological
had known each other prior to the program. innovations that, when used judiciously, can
Indeed, the program administrative direc- make fieldwork more efficient and compre-
tors had been the college professors of the hensive. First and foremost is the battery-
technical staff while the latter were still un- operated tape recorder or dictaphone. For
dergraduate students. However, the techni- some people, myself included, dictating
cal staff had introduced the directors to the field notes saves a great deal of time while
308 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

increasing the comprehensiveness of the re- sual feedback to staff. Videotaping class-
port. Learning to dictate takes practice, ef- rooms, training sessions, therapeutic inter-
fort, and criticai review of early attempts. actions, and a host of other observational
Tape recorders must be used judiciously so targets can sometimes be less intrusive than
as not to become obtrusive and inhibit pro- a note-taking evaluator. We had great suc-
gram processes or participant responses. A cess taking videos of mothers and children
tape recorder is much more useful for re- playing together in early childhood educa-
cording field notes in priva te than it is as an tion centers. Of course, use of such equip-
instrument to be carried about at ali times, ment must be nego tiated with program staff
available to put a quick end to any conversa- and participants, but the creative and judi-
tion into which the observer enters. cious use of technology can greatly increase
Portable computers have emerged as a the quality of field observations and the util-
fieldwork tool that can facilitate writing ity of the observational record to others.
field notes. Cameras have become standard Moreover, comfort with tape recorders and
accessories in fieldwork. Photographs can video cameras has made it increasingly pos-
help m recalling things that have happened sible to use such technology without undue
as well as vividly capturing the setting for intrusion when observing programs where
others. Digital photography and advances professionals are the participants. In addi-
in printing and photo copying now make it tion, sometimes videotapes originally done
possible to economically reproduce photo- for research or evaluation can subsequently
graphs in research and evaluation reports. be used for future training, program devel-
In the wilderness education evaluation, I opment, and public relatioris, making the
officially became the group photographer, costs more manageable because of added
making photographs available to ali of the uses and benefits. Evaluators learn to bal-
participants. This helped legitimize taking ance costs against benefits and look for mul-
photographs and reduced the extent to tiple uses of more expensive techniques
which other people felt it necessary to carry where there is a need to make judicious deci-
their own cameras at ali times, particularly sions about reducing expenses.
at times when it was possible that the equip- Visual technology can add an important
ment might be damaged. Looking at photo- dimension to fieldwork if the observer
graphs during analysis helped me recall the knows how to use such technology and uses
details of certain activities that I had not it wellfor there is much to learn beyond
fully recorde d in my written notes. I relied how to click the camera or turn on the video
heavily on photographs to add details to de- recorder, especially about integrating and
scriptions of places where criticai events oc- analyzing visual data within a larger field-
curred in the Grand Canyon initiation story I work context (Bali and Smith 1992). More-
wrote about coming of age in modern soci- over, a downside to visual technology has
ety (Patton 1999a). emerged, since it is now possible to not only
Video photography is another technolog- capture images on film and video but also
ical innovation that has become readily ac- change and edit those images in ways that
cessible and common enough that it can distort. In his extensive review of "visual
sometimes be used unobtrusively. For exam- methods" in qualitative inquiry, Douglas
ple, in a formative evaluation of a staff train- Harper (2000) concludes that "now that im-
ing program I used videotapes to provide vi- ages can be created and/or changed digi-
Fieldzuork Strategies and Observation Methods lj. 309

tally, the connection between image and haps somewhat like a rolling chair. (Scott and
'truth' has been forever severed" (p. 721). Eklund 1979:9-11).
This means that issues of credibility apply to
using and reporting visual data as they do to The imagery of a fieldworker following a
other kinds of data. subject around through a day wearing a
Perhaps the ultimate in observer technol- Stenomask offers a stark contrast to that of
ogy for fieldwork is the Stenomask, a the traditional anthropologist doing partici-
sound-shielded microphone attached to a pant observation and trying covertly to
portable tape recorder that is worn on a write notes during informal field inter-
shoulder strap. The handle of the Stenomask views. Taking field notes canbe nearly as in-
contains the microphone switch. The trusive as wearing a Stenomask, as illus-
Stenomask allows the observer to talk into trated in the fieldwork of anthropologist
the recorder while an activity is occurring Carlos Castaneda. In the passage below,
without people in the area being able to hear Castaneda (1973) reports on his negotiations
the dictation. Its use is limited to externai, with Don Juan to become his Native Indian
onlooker observations, as the following pas- key informant on sorcery and indigenous
sage makes clear. drugs. The young anthropologist records
that Don Juan "looked at me piercingly."

Two procedures precede any data taking. The "What are you doing in your pocket?" he
first is orientation of the subject and as many asked, frowning. "Are you playing with your
other persons in the environment as are likely whanger?"
to be present during observations.... During He was referring to my taking notes on a
this phase, the observer goes into the habitat minute pad inside the enormous pockets of
and behaves exactly as he or she will during my wirvdbreaker.
the actual recording. They wear the Steno- When I told him what I was doing he
mask, follow the subject about and run the laughed heartily.
machine, taking mock records. The purpose of I said that I did not want to disturb him by
these activities is exactly what is implied in the writing in front of him.
title, to adapt the subject and others in the en- "If you want to write, write," he said. "You
vironment to the presence of the observer and don't disturb me." (pp. 21-22)
to reduce the effects of that presence to as near
zero as possible. The cardinal rule of the ob- Whether one uses modern technology to
server during this time is to be completely support fieldwork or simply writes down
nonresponding. It has been demonstrated what is occurring, some method of keeping
over and over again that if the observer contin- track of what is observed must be estab-
ues to resist ali social stimuli from the subject lished. In addition, the nature of the record-
and others (and some will occur despite the ing system must be worked out in accor-
most careful orientation) by simply keepmg dance with the participant observer's role,
the mask in place, looking busily at work and the purpose of the study, and consideration
remaining nonresponding, both subjects and of how the data-gathering process will affect
others soon cease emitting stimuli to the ob- the activities and persons being observed.
server and come to truly accept him or her as a Many of these issues and procedures must
present and sometimes mobile but completely be worked out during the initial phase (en-
nonresponding part of the environment, per- try period) of fieldwork.
310 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

13. Stages of Fieldwork cepted. If only he could figure out what to

do, if only he could understand the rules,
Thus far, fieldwork has been described as if then he would happily do what he was sup-
it were a single, integrated experience. Cer- posed to do. Such are the trials of entry into
tainly, when fieldwork goes well it flows the field.
with a certain continuity, but it is useful to Entry into the field for evaluation re-
look at the evolution of fieldwork through search involves two separate parts: (1) nego-'
identifiable stages. Three stages are most of- tia tion with gatekeepers, whoever they may
ten discussed in the participant observation be, about the nature of the fieldwork to be
literature: the entry stage, the routinization done and (2) actual physical entry into the
of data-gathering period, and the closing field setting to begin collecting data. These
stage. The following sections explore each two parts are closely related, for the negotia-
of these stages, again using evaluative re- tions with gatekeepers will establish the
search as the primary example. rules and conditions for how one goes about
playing the role of observer and how that
role is defined for the people being ob-
Entry Into the Field served. In traditional scholarly fieldwork for
the purpose of basic or applied research, the
The writings of anthropologists some- invs tigator unilaterally decides how best to
times present a picture of the early period of conduct the fieldwork. In evaluation stud-
fieldwork that reminds me of the character ies, the evaluator will need to take into ac-
in Franz Kafka's haunting novel The Castle. count the perspectives and interests of the
Kafka's character is a wandering stranger, primary intended users of the evaluation. In
K., with no more identity than that initial. He either case, interactions with those who con-
doesn't belong anywhere, but when he ar- trol entry into the field are primarily strate-
rives at the castle he wants to become part of gic, figuring out how to gain entry while pre-
that world. His efforts to make contact with serving the integrity of the study and the
the faceless authorities who run the castle invs tigator's interests. The degree of diffi-
lead to frustration and anxiety. He can't culty involved varies depending on the pur-
quite figure out what is going on, can't break pose of the fieldwork and the expected or
through their vagueness and impersonal na- real degree of resistance to the study. Where
ture. He doubts himself; then he gets angry the field researcher expects coopera tion,
at the way he is treated; then he feels guilty, gaining entry may be largely a matter of es-
blaming himself for his inability to break tablishing trust and rapport. At the other
through the ambiguous procedures for en- end of the continuum are those research set-
try. Yet, he remains determined to make tings where considerable resistance, even
sense out of the incomprehensible regula- hostility, is expected, in which case gaining
tions of the castle. He is convinced that, after entry becomes a matter of "infiltrating the
ali, where there are rulesand he does find setting" (Douglas 1976:167). And sometimes
that there are rulesthey must fit together entry is simply denied. A doctoral student
somehow, have some meaning, and mani- had negotiations for entry end abruptly in a
fest some underlying logic. There must be school district where she had developed
some way to make contact, to satisfy the good relationships with school personnel
needs of the authorities, to find some pattern and negotiations appeared to be going well.
of behavior that will permit him to be ac- She later learned that she was denied entry
Fieldzuork Strategies and Observation Methods lj. 311

far into the negotiation process because of propriate to consider some other term to de-
community opposition. The local commu- scribe the fieldwork. In our onlooker,
nity had had a very bad experience with a nonparticipatory observations for an imple-
university researcher more than 20 years mentation study of early childhood pro-
earlier and still viewed ali research with grams in Minnesota, we described our role
great suspicion. to local program participants and staff as fol-
A major difference between the entry pro- lows:
cess in anthropological or sociological re-
search and the entry process for evaluation
We're here to be the eyes and ears for state leg-
research is the extent to which fieldworkers
islators. They can't get around and visit ali the
are free to make up whatever story they
programs throughout the state, so they've
want to about the purpose of the study. In
asked us to come out and describe for them
scholarly research, the investigators repre-
what you're doing. That way they can better
sent only themselves and so they are rela-
understand the programs they have funded.
tively free to say whatever they want to say
We're not here to make any judgments about
about why they are doing the research
whether your particular programs is good or
guided by the ethics of their discipline with
bad, We are just here tobe the eyes and ears for
regard to informed consent. The usual
the legislature so that they can see how the leg-
cross-cultural explanation is some variation
islation they've passed has tumed into real
of 'Tm here because I would like to under-
programs. This is your chance to inform them
stand you better and learn about your way
and give them your point of view.
of life because the people from my culture
would like to know more about you." While
anthropologists admit that such an explana- Other settings lend themselves to other
tion almostnever makes sense to indigenous terms that are less threatening than evaluator.
peoples in other cultures, it remains a main- Sometimes a fieldwork project can be de-
stay initial explanation until mutual reci- scribed as documentation. Another term I've
procities can be established with enough lo- heard used by community-based evaluators
cal people for the observation process to is process historian. In the wilderness educa-
become established and accepted in its own tion program I was a full participant ob-
right. server, and staff described my role to partici-
Evaluators and action researchers, how- pants as "keeper of the community record/'
ever, are not just doing fieldwork out of per- making it clear that I was not there to evalu-
sonal or professional interest. They are do- ate individual participants. The staff of the
ing the fieldwork for some decision makers project explained that they had asked me to
and information users who may be either join the project because they wanted some-
known or unknown to the people being one who did not have direct ego involve-
studied. It becomes criticai, then, that evalu- ment in the success or outcomes of the pro-
ators, their funders, and evaluation users gram to observe and describe what went on,
give careful thought to how the fieldwork is both because they were too busy running the
going to be presented. program to keep detailed notes about what
Because the word evaluation has suchneg- occurred and because they were too in-
ative connotations for many people, having volved with what happened to be able to
had negative experiences being evaluated, look at things dispassionately. We had
for example, at school or work, it may be ap- agreed from the beginning that the commu-
312 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

nity record I produced would be accessible While the observer must learn how to be-
to participants as well as staff. have in the new setting, the people in that
In none of these cases did changing the setting are deciding how to behave toward
language automatically make the entry pro- the observer. Mutual trust, respect, and co-
cess smooth and easy. Earlier in this chapter, operation are dependent on the emergence
1 described our atteinpt to be viewed as "ed- of an exchange relationship, or reciprocity
ucational researchers" in evaluating a com- (Jorgensen 1989:71; Gallucci and Perugini
munity leadership program. Everyone fig- 2000), in which the observer obtains data
ured out almost immediately that we were and the people being observed find some-
really evaluators and thafs what partici- thing that makes their cooperation worth-
pants called us. Regardless of the story told while, whether that something is a feeling of
or the terms used, the entry period of field- importance from being observed, useful
work is likely to remain "the first and most feedback, pleasure from interactions with
uncomfortable stage of field work" (Wax the observer, or assistance in some task. This
1971:15). It is a time when the observer is get- reciprocity model of gaining entry assumes
ting used to the new setting, and the people that some reason can be found for partici-
in that setting are getting used to the ob- pants to cooperate in the research and that
server. Johnson (1975) suggests that there some kind of mutual exchange can occur.
are two reasons why the entry stage is both Infiltration lies at the opposite end of the
so important and so difficult: continuum from a negotiated, reciprocity
model of entry. Many field settings are not
First, the achievement of successful entree is a open to observation based on cooperation.
precondition for doing the research. Put sim- Douglas (1976:167-71) has described a num-
ply, no entree, no research [P]ublished re- ber of infiltration strategies, including
ports of researchers' entree experiences "worming one's way in," "using the crow-
describe seemmgly unlimited contingencies bar to pry them open for our observations,"
which may be encountered, ranging from be- showing enough "saintly submissiveness"
ing gleefully accepted to being thrown out on to make members guilty enough to provide
one's ear. But there is a more subtle reason help, or playing the role of a "spineless
why the matter of one's entrance to a research boob" who could never possibly hurt the
setting is seen as so important. This concems people being observed. He has also sug-
the relationship between the initial entree to gested using various ploys of misdirection
the setting and the validity of the data that is where the researcher diverts people's atten-
subsequently collected. The conditions under tion away from the real purpose of the study.
which an initial entree is negotiated may have There is also the "phased-entre tactic" by
important consequences for how the research which the researcher who is refused entree
is socially defined by the members of the set- to one group begins by studying another
ting. These social definitions will have a bear- group until it becomes possible to get into
ing on the extent to which the members trust a the group that is the real focus of the re-
social researcher, and the existence of relations searcher^ attention, for example, begin by
of trust between an observer and the members observing children in a school when what
of a setting is essential to the production of an you really want to observe are teachers or
objective report, one which retains the integ- administrators.
rity of the actor's perspective and its social Often the best approach for gaining
context. (pp 50-51) entre is the "known sponsor approach/'
Fieldzuork Strategies and Observation Methods lj. 313

When employing this tactic, observers use perience the same socialization process that
the legitimacy and credibility of another regular participants experience by becom-
person to establish their own legitimacy and ing part of the initiation process and timing
credibility, for example, the director of an or- their observations to coincide with the be-
ganization for an organizational study, a lo- ginning of a program. Such timing makes
cal leader, elected official, or village chief tain the evaluator one among a number of nov-
for a community study. Of course, it's im- ices and substantially reduces the disparity
portant to make sure that the known spon- between the evaluator's knowledge and the
sor is indeed a source of legitimacy and cred- knowledge of other participants.
ibility. Some prior assessment must be made Beginning the program with other partic-
of the extent to which that person can pro- ipants, however, does not assure the evalua-
vide halo feelings that will be positive and tor of equal status. Some participants may be
helpful. For example, in an evaluation, using suspicious that real difficulties experienced
a program administrator or funders as a by the evaluator as a novice participant are
known sponsor may increase suspicion and phonythat the evaluator is play-acting,
distrust among program participants and only pretending to have difficulty. On the
staff. first day of my participation in the wilder-
The initial period of fieldwork can be ness education program, we had our first
frustrating and give rise to self-doubt. The backpacking experience. The staff leader be-
fieldworker may lie awake at night worry- gan by explaining that "your backpack is
ing about some mistake, some faux pas, your friend." I managed to both pack and
made during the day. There may be times of adjust my "friend" incorrectly. As a result, as
embarrassment, feeling foolish, of question- soon as we hit the trail, I found that the belt
ing the whole purpose of the project, and around my waist holding the backpack on
even feelings of parania. The fact that one is my hips was so tight that my friend was
trained in social science does not mean that making my legs fali asleep. I had to stop sev-
one is immune to ali the normal pains of eral times to adjust the pack. Because of
learning in new situations. On the other these delays and other difficulties I was hav-
hand, the initial period of fieldwork can also ing with the weight and carriage of the pack,
be an exhilarating time, a period of rapid I ended up as the last participant along the
new learning, when the senses are height- trail. The next morning when the group was
ened by exposure to new stimuli, and a time deciding who should carry the map and
of testing one's social, intellectual, emo- walk at the front of the group to learn map
tional, and physical capabilities. The entry reading, one of the participants immediately
stage of fieldwork magnifies both the joys volunteered my name. "Let Patton do it.
and the pains of doing fieldwork. That way he can't hang back at the end of the
Evaluators can reduce the "stick-out- group to observe the rest of us." No amount
like-a-sore-thumb syndrome" by beginning of protest from me seemed to convince the
their observations and participation in a participants that I had ended up behind
program at the same time that participants them ali because I was having trouble hiking
are beginning the program. In traditional (working out my "friendship" with my
fieldwork, anthropologists cannot become backpack). They were convinced I had taken
children again and experience the same so- that position as a strategic place from which
cialization into the culture that children ex- to evaluate what was happening. It is well to
perience. Evaluators, however, can often ex- remember, then, that regardless of the na-
314 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

ture of the fieldwork, during the entry stage about his research, but by the style in which he
more than at any other time, the observer is lives and acts, by the way in which he treats
also the observed. them. In a somewhat shorter run, they will ac-
cept or tolerate him because some relative,
friend, or person they respect has recom-
What You Say and What You Do
mended him to them. (Wax 1971:365)
Fieldworkers' actions speak louder than
their words. Researchers necessarily plan William Foote Whyte (1984:37-63) has ex-
strategies to present themselves and their tracted and summarized entry strategies
function, but participant reactions to state- used in a number of groundbreaking socio-
ments about the researcher's role are quickly logical studies, including the Lynds' study
superseded by judgments based on how the of Middletown, W. Lloyd Warner's study of
person actually behaves. Yankee City, Burleigh Gardner's fieldwork
The relative impor tance of words versus in the deep South, Elliot Liebow's hanging
deeds in establishing credibility is partly a around Tally's Corner, Elijah Anderson's
function of the length of time the observer fieldwork in a Black neighborhood, Ruth
expects to be in a setting. For some direct on- Horowitz's study of a Chicano neighbor-
looker observations, the fieldworker may be hood, Robert Cole's work in Japan, and
present in a particular program for only a Whyte's own experiences in Cornerville.
few hours or a day. The entry problem in They each had to adapt their entry strategy
such cases is quite different from the situa- to the local setting and they ali ended up
tion where the observer expects to be partici- changing what they had planned to do as
pating in the program over some longer pe- they learned from the initial responses to
riod of time, as anthropologist Rosalie Wax their efforts to gain acceptance. These exam-
has noted: ples from those who paved for way for mod-
em fieldworkers demonstrate the impor-
Ali field workers are concerned about explain- tance of careful attention to entry and the
ing their presence and their work to a host of variety of approaches that are possible. The
people. "How shall I introduce myself?" they next section presents a concrete example
wonder, or, "what shall I say I am doing?" If from an evaluation by Joyce Keller.
the field worker plans to do a very rapid and
efficient survey, questions like these are ex- AN ENTRY CASE EXAMPLE:
tremely important. The manner in which an THE PART-TIME OBSERVER
interviewer introduces himself, the precise
words he uses, may mean the difference be- Introduction. The prevous section contrasted the
tween a first-rate job and a failure But if the entry challenges for the one-shot onlooker observer
field worker expects to engage in some variety with those of the long-term participant observer;
of participant observation, to develop and but a great deal of middle ground exists between
maintain long-term rela tionships, to do a these extremes. In this section, loyce Keller, a snior
study that involves the enlargement of his staff member of the Minnesota Center for Social
own understanding, the best thing he can do is Research at the time, describes her entry intofield-
relax and remember that most sensible people ivork as a part-time observer.4 Because limitations
do not believe what a stranger tells them. In of time and resources are common in evaluation,
the long run, his host will judge and trust him, many situations call for a part-time observer.
not because of what he says about himself or Ioyce's reflections capture some ofthe special entry
Fieldzuork Strategies and Observation Methods lj. 315

problems associated zuith this "now yoii're here, be notified in the event of schedule changes.
now you're gone" role. I would have firmly in mind that a subgroup
was to meet on Tuesday at 10:00 a.m. in a cer-
One word can describe my role, at least tain place. I would arrive to find no one
initially, in a recent evaluation assignment: there. Later, I would discover that on Mon-
ambiguous. I was to be neither a participant day the meeting had been changed to
observer nor an outsider coming in for a Wednesday afternoon and no one had been
brief but intensive stint. I was to allocate ap- delegated to tell me. At no time did I seri-
proximately six hours a week for seven ously feel that the changes were planned to
months to observing the team development exclude me; on the contrary, the members'
of a group of 23 professionals in an educa- contrition about their oversight seemed
tional setting. At first, the ambiguity was quite genuine. They had simply forgotten
solely on my side: What, really, was I to do? me.
The team, too busy in the beginning with de- Another area of sudden change that
fining their own roles, had little time to con- caused me difficulty was in policy and pro-
sider mine. Later on, as I became accus- cedure. What had seemed to be firm com-
tomed to my task, the team's curiosity about mitments on ways to proceed or tasks to be
my function began to grow. tackled were being ignored. I came to realize
In their eyes, I served no useful purpose that while a certain amount of this instability
that they could see. I was in the way a great was inherent in the program itself, other
deal of the time inhibiting their private con- shifts in direction were outgrowths of plan-
versations. On the other hand, they ap- ning sessions I had not attended or had not
peared to be concerned about what I was heard the results from after they had oc-
thinking. Some of themmost of thembe- curred. Therefore, keepmg current became
gan to be friendly, to greet me as I came in, to for me a high-priority activity. Not to do so
comment when I missed a team meeting. would have added to my feeling of ambigu-
They came to see me as I saw myself: neither ity. Also, if I had not operated with a certain
really part of the group nor a separate, re- degree of self-confidence, I would have felt
moved force. somehow at fault for coming to a meeting at
Observing their interaction perhaps six the wrong time or place or assuming that a
hours a week out of their 40-hour work week certain decision, which the team had previ-
obviously meant that I missed a great deal. I ously made, was still valid.
needed to develop a sense of when to be I began my observation of this team in its
present, to choose among group meetings, forma tive stage. Had I begun after the team
subgroup meetings, and activities when ali was well established, my difficulties would
the members were to come together. At the have been greater. Nevertheless, many of
same time, I was working on other contracts the team members were already well ac-
which Iimited the amount of adjustable time quainted with each other; ali had been em-
available. "Flexible" was the way I came to ployees of the same school district over a pe-
define my weekly schedule; others, not as riod of time. They were much better versed
charitable, would probably have defined it in what they had come together to accom-
as "shifty." plish than I, whose only orientation was
A hazard that I encountered as I filled my reading the proposal which, upon accep-
ambiguous, flexible role was that I soon distance, had brought them together. I found
covered I was not high on the priority list to also that the proposal and the way they
316 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

plaimed to proceed were, in actuality, far they were interacting about. I noted (and ig-
from identical. nored) a few passing suggestions that
With my observer role to continue over since I was obviously taking notes maybe I
many months, I realized that I must main- could
tain the difficult position of being impartial. I took copious notes before I began to de-
I could not be thought of by the team mem- velop a sense of what was or was not impor-
bers as being closely aligned with their lead- tant to record. When I relaxed more and
ers, nor could I expect the leaders to talk can- aimed for the tone of the meeting my under-
didly and openly with me if they believed standing of the group increased. I had to re-
that I would repeat their confidences to the alize that, as a part-time observer, it was im-
group members. Reluctantly, for I discov- possible for me to understand ali of what
ered several team members with whom was said. My decision frequently was to let
friendships could easily have developed, I this portion of the meeting pass or to jot
declined invitations to social activities out- down a reminder to myself to ask clarifying
side of working hours. questions later.
When I inet with the group for the first Side-stepping sensitive questions from
time, I directed most of my energies to both leaders and team members had to be
matching names and faces. I would be tak- developed into a fine art. As I became more
ing notes at most of the sessions and it was finely tuned to the interactions, and most be-
essential that I could record not only what came aware that I was, I was frequently que-
was said but who said it. At the first session ried as to my perceptions of a particular indi-
everyone, including me, wore a name tag. vidual or situation. On one occasion, I found
But within a few days, they were ali well ac- a team member jumping into an elevator to
quainted and had discarded their name tags; ride two floors with me in a direction he
I was the only one still fumbling for names. didn't want to go so that he could ask me pri-
While being able to greet each member by vately what I thought of another team mem-
name was important, so was knowing some- ber. My response was, "I think she's a very
thing about each one's background. Coffee interesting person," or something equally
breaks allowed me to circulate among the innocuous, and received from him a highly
group and carry on short conversations with raised eyebrow, since the woman in ques-
as many as possible to try to fix in my mind tion had just behaved in a very peculiar
who they were and where they came from, manner at the meeting we had both just at-
which provided insights into why they be- tended.
haved in the group as they did. In-depth interviews with each team
Team members at first expressed a certain member began in the fourth month of my
amount of enthusiasm for minutes to be observation and was the mechanism which
taken of their meetings. This enthusiasm filled in many of the gaps in my understand-
was short-lived, for willing volunteers to ing. The timing was perfect: I had gained
serve as secretary did not emerge. I was dis- enough familiarity with both personnel and
appointed, for, had minutes been kept of the project by that time so that I was knowledge-
meetings and had I been able to rely on re- able, they had come to trust me, and they
ceiving copies, I would have concentrated still cared deeply about the project. (This
solely on observing the interactions and caring diminished for some as the project
would not have had to keep track of what year drew to a close without any real hopes
Fieldzuork Strategies and Observation Methods lj. 317

of refunding for a second year.) My inter- tions and conditions reinforced for me what
view design was intentionally simple and were sometimes at best only vague percep-
open-ended. What I wanted most was for tions. Team members who appeared to be
them to talk about their experiences in terms passive and quiet when I saw them at group
of strengths and weaknesses. meetings were often referred to by their
The amount of new information dimin- team members as hard-working and cre-
ished throughout the six weeks or so that ative when they were out in the field. The in-
was required to interview ali team members. terviews also helped me become aware of
My own performance unquestionably di- misconceptions on my part caused by seeing
minished too as the weeks went on. It was only part of the picture, due to time con-
difficult to be animated and interesting as I straints.
asked the same questions over and over, de- The experience was a new one for me, that
vised strategies with which to probe, and re- of part-time observer. Quite frankly, this
corded perceptions and incidents which I mode of evaluation probably will never be a
had heard many times before. favorite one. On the other hand, it provided
Nevertheless, the interviews appear in a picture that no "snap-shot" evaluation
retrospect to have been a necessary tool of method could have accomplished as interac-
the part-time observer. Bit by bit team mem- tions changed over time and in a situation
bers filled in holes in my information and where the full participant observer role was
their repeated references to particular situa- clearly not appropriate.

Routinization of Fieldwork:
The Dynamics of the Second Stage

hat did you learn in your readings today?" asked Master Halcolm.
"We learned that a journey of a thousand miles begins with the first
step," replied the learners.
"Ah, yes, the importance of beginnings," smiled Halcolm.
"Yet, I am puzzled," said a learner. "Yesterday I read that there are a
thousand beginnings for every ending."
"Ah, yes, the importance of seeing a thing through to the end," affirmed
Halcolm.
"But which is more important, to begin or end?"
"Two great self-deceptions are asserted by the world's self-congratulators:
that the hardest and most important step is the first and that the greatest and
most resplendent step is the last.
"While every journey must have a first and last step, my experience is that
what ultimately determines the nature and enduring value of the journey are
the steps in between. Each step has its own value and importance. Be present
for the whole journey, learners that you are. Be present for the whole journey."
Halcolm
318 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

During the second stage, after the awakening identification involves some re-
fieldworker has established a role and pur- alization of how much I have in common
pose, the focus moves to high-quality data with these people whose world I have been
gathering and opportunistic investigation permitted to enter. At times during field-
following emergent possibilities and build- work I feel a great separation from the peo-
ing on what is observed and learned each ple I'm observing, then at other times I feel a
step along the way. The observer, no longer strong sense of our common humanity. For a
caught up in adjustments to the newness of fieldworker to identify, however briefly,
the field setting, begins to really see what is with the people in a setting or for an evalua-
going on instead of just looking around. As tor to identify with the clients in a program
Florence Nightingale said, "Merely looking can be a startling experience because social
at the sick is not observing." science observers are often quite separated
Describing the second stage as "rou- from those they study by education, experi-
tinization of fieldwork" probably overstates ence, confidence, and income. Such differ-
the case. In emergent designs and ever- ences sometimes make the world of pro-
deepening inquiry, the human tendency to- grams as exotic to evaluators as nonliterate
ward routines yields to the ups and downs cultures are exotic to anthropologists.
of new discoveries, fresh insights, sudden There come times, then, when a field-
doubts, and ever-present questoning of worker must deal with his or her own feel-
othersand often of self. Discipline is ings about and perspectives on the people
needed to maintain high-quality, up-to-date being observed. Part of the sorting-out pro-
field notes. Openness and perseverance are cess of fieldwork is establishing an under-
needed to keep exploring, looking deeper, standing of the relationship between the
diverging broader, and focusing narrower, observed and the observer. When that hap-
always going where the inquiry and data pens, and as it happens, the person involved
take you. Fieldwork is intellectually chal- in fieldwork may be no less startled than Jo-
lenging at times, mind-numbingly dull at seph Conrad's infamous character Marlowe
times, and for many, an emotional roller in Heart o/Darkness. Marlowe had followed
coaster. Appendix 9.1 at the end of Chap- Kurtz, the European ivory trader, up the
ter 9, "A Documenter's Perspective," offers deep river into the Congo where Kurtz had
the reflections of a participant observer con- established himself as a mangod to the tribal
ducting a school evaluation and grappling people there. He used his position to acquire
with changes in fieldwork over time. ivory, but to maintain his position he had to
One of the things that can happen in the perform the indigenous rituais of human
course of fieldwork is the emergence of a sacrifice and cannibalism. Marlowe, deeply
strong feeling of connection with the people enmeshed in the racism of his culture and
being studied. As you come to understand time, was initially horrified by the darkness
the behaviors, ideais, anxieties, and feelings of the jungle and its peoples, but as he
of other people, you may find yourself iden- watched the rituais of those seeming sav-
tifying with their lives, their hopes, and their ages, he found an emergent identification
pain. This sense of identification and con- with them and even entertained the suspi-
nection can be a natural and logical conse- cion that they were not inhuman. He became
quence of having established relationships aware of a linkage between himself and
of rapport, trust, and mutuality. For me, that them:
Fieldzuork Strategies and Observation Methods lj. 319

They howled and leaped and spun, and madee self in the middle of deep generational divi-
horrid faces; but what thrilled you was thee sions between mothers and their daughters,
thought of their humanitylike oursthee teachers and students, bookstore owners
thought of your remote kinship with this wild1 and their clients. She could not risk deeply
and passionate uproar. Ugly. Yes, it was uglyy alienating or completely acquiescing to any
enough; but if you were man enough youx of these important and competing groups,
would admit to yourself that there was in youx for they ali affected her access and the ulti-
just the faintest trace of a response to the terri- mate success of her fieldwork.
ble frankness of that nose, a dim suspicion off In evaluations, the evaluator can be
there being a meaning in it which youyou so3 caught in the middle of tensions between
remote from the night of the first agescom- competing groups and conflicting perspec-
prehend. And why not? (Conrad 1960:70) tives. For example, where divisions exist
among the staff and/or the participants in a
In this passage, Conrad chronicles the2 program, and such divisions are common,
possibility of awakening to unexpected real- the evaluator will be invited, often subtly, to
izations and intense emotions in the course align with one subgroup or the other. In-
of encounters with the unknown and those2 deed, the evaluator may want to become
who are different from us. In many ways, itt part of a particular subgroup to gain further
is our common humanity, whether we are^ insight into and understanding of that sub-
fully aware of it at any given moment or not, group. How such an alliance occurs, and
that makes fieldwork possible. As humani how it is interpreted by others, can greatly
beings, we have the amazing capability to) affect the course of the evaluation.
become part of other people's experiences,,, My experience suggests that it is imprac-
and through watching and reflecting, weB tical to expect to have the same kind of re-
can come to understand soinething aboutt lationshipclose or distantwith every
those experiences. group or faction. Fieldworkers, human be-
As fieldwork progresses, the intrica te^ ings with their own personalities and inter-
web of human relationships can entangle5 ests, will be naturally attracted to some peo-
the participant observer in ways that will1 pie more than others. Indeed, to resist those
create tension between the desire to become2 attrac tions may hinder the observer from
more enmeshed in the setting so as to learni acting naturally and becoming more thor-
more and the need to preserve some dis- oughly integrated into the setting or pro-
tance and perspective. Participant observers3 gram. Recognizing this, the observer will be
carry no immunity to the political dynamicss faced with ongoing decisions about per-
of the settings being observed. Virtually anyj sonal relationships, group involvement, and
setting is likely to include subgroups of peo- how to manage differential associations
ple who may be in conflict with other sub- without losing perspective on what the ex-
groups. These factions or cliques may eitherr perience is like for those with whom the
woo or reject the participant observer, butt fieldworker is less directly involved.
they are seldom neutral. During her field- Perhaps the most basic division that will
work interviewing young women in ndia, always be experienced in program evalua-
Parameswaran (2001) reports efforts by par- tion is the separation of staff and partici-
ents and teachers to get her to inform on the pants. While the rhetoric of many programs
women she interviewed or to influence? attempts to reduce the distinction between
them in a desired direction. She found her- staff and participants, there is almost always
320 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

a distinction between those who are paid for servers among people on both sides of those
their responsibilities in the program (staff) fences. (pp. 96-97)
and those who are primarily recipients of
what the program has to offer (participants). In contrast to Lofland's advice, in evalu-
Sociologically, it makes sense that staff and ating the wilderness education program I
participants would be differentiated, creat- found myself moving back and forth be-
ing a distance that can evolve into conflict or tween a full participant role, where I was
distrust. Participants will often view the identified primarily as a participant, and a
evaluator as no different from the staff or ad- full staff role, where I was identified primar-
ministration, or even the funding sources ily with those who carried responsibility for
virtually any group except the partici- directing the program. During the first field
pants. If the evaluator observer is attempt- conference, I took on the role of full partici-
ing to experience the program as a participant and made as visible as possible my alle-
pant, special effort will be required make giance to fellow participants while main-
participation real and meaningful and to taining distance from the staff. Over time,
become accepted, even trusted, by other however, as my personal relationships with
participants. On the other hand, staff and the staff increased, I became more and more
administrators may be suspicious of the aligned with the staff. This coincided with a
evaluator's relationships with funders or change of emphasis in the evaluation itself,
board members. with the earlier part of the fieldwork being
The point is not to be naive about the tan- directed at describing the participant experi-
gled web of relationships the participant ob- ence and the latter part of the fieldwork be-
server will experience and to be thoughtful ing aimed at describing the workings of the
about how fieldwork, data quality, and the staff and providing formative feedback.
overall inquiry are affected by these connec- However, I was always aware of a ten-
tions and interrelationships, ali of which sion, both within myself and within the
have to be negotiated. group at large, about the extent to which I
Lofland (1971) has suggested that partici- was a participant or a staff member. I found
pant observers can reduce suspicion and that as my observational skills became in-
fear about a study by becoming openly creasingly valued by the program staff I had
aligned with a single broad grouping within to more consciously and actively resist their
a setting while remairiing aloof from that desire to have me take on a more active and
grouping's own internai disputes. explicit staff role. They also made occasional
attempts to use me as an informer, trying to
seduce me into conversations about particu-
Thus, known observers of medicai schools lar participants. The ambiguities of my role
have aligned themselves only with the medi- were never fully resolved. I suspect that
cai students, rather than attempting to parti ci- such ambiguities were inherent in the situa-
pate extensively with both faculty and tion and are to be expected in many evalua-
students. In mental hospitais, known observ- tion fieldwork experiences.
ers have confined themselves largely to men- Managing field relationships involves a
tal patients and restricted their participation different set of dynamics when the inquiry is
with staff. To attempt to participate with both, collaborative or participatory. Under such
extensively and simultaneously, would prob- designs, where the researcher involves oth-
ably have generated suspicion about the ob- ers in the setting in fieldwork, a great deal of
Fieldzuork Strategies and Observation Methods lj. 321

the work consists of facilitating the interac- tagonisms among those who may resent or
tions with co-inquirers, supporting their distrust the special relationships between
data collection efforts, ongoing training in the fieldworker and the key informant. In-
observation and interviewing, managing indeed, howand how muchto make visi-
tegration of field notes ainong different par- ble this relationship involves strategic think-
ticipant researchers, and inonitoring data ing about how others will react and how
quality and consistency. These coliaborative their reactions will affect the inquiry. There's
management responsibilities will reduce the no formal announcement that the "position"
primary researcher's own time for field- of key informant is open, or that it's been
work and will affect how others in the set- filled; the key informant is simply that per-
tings, those who aren't participatory or col- son or those persons with whom the re-
iaborative researchers, view the inquiry and searcher or evaluator is likely to spend con-
the fieldwork director, if that is the role taken siderable time.
on. In some cases, management of the coliab- Key informants mustbe trained or devel-
orative inquiry effort is done by one of the oped in their role, not in a formal sense, but
participants and the trained fieldworker because they will be more valuable if they
serves primarily as a skills and process understand the purpose and focus of the in-
trainer and consultant to the group. Clarity quiry, the issues and questions under inves-
about these roles and divisions of labor can tigation, and the kinds of information that
make or break coliaborative, participatory are needed and most valuable. Anthropolo-
forms of inquiry. Having shared values gists Peito and Peito (1978) made this point
about collaboration does not guarantee ac- in reflecting on their own fieldwork:
tually pulling it off. Coliaborative inquiry is
challenging work, often frustrating, but We noticed that humans differ in their will-
when it works, the findings will carry the ad- ingness as well as their capabilities for ver-
ditional credibility of coliaborative triangu- bally expressing cultural information. Conse-
lation, and the results tend to be rewarding quently, the anthropologist usually finds that
for ali involved, with enduring insights and only a small number of individuais in any
new inquiry skills for those involved. community are good key informants. Some of
the capabilities of key informants are system-
Key Informants atically developed by the field workers, as
they train the informants to conceptualize cul-
One of the mainstays of much fieldwork tural data in the frame of reference employed
is the use of key informants as sources of in- by anthropologists. . . . The key informant
formation about what the observer has not gradually learns the rules of behavior in a
or cannot experience, as well as sources of role vis--vis the interviewer-anthropologist.
explanation for events the observer has actu- (p- 72)
ally witnessed. Key informants are people
who are particularly knowledgeable about The danger in cultivating and using key
the inquiry setting and articulate about their informants is that the researcher comes to
knowledgepeople whose insights can rely on them too much and loses sight of the
prove particularly useful in helping an ob- fact that their perspectives are necessarily
server understand what is happening and limited, selective, and biased. Data from in-
why. Selecting key informants mustbe done formants represent perceptions, not truths.
carefully to avoid arousing hostility or an- Information obtained from key informants
322 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

should be clearly specified as such in the the turtles, who willingly kept me informed
field notes so that the researcher's observa- about the details of what went on in that
tions and those of the informants do not be- group. Without that key informant relation-
come confounded. This may seem like an ship, I would have missed some very impor-
obvious point, and it is, but over weeks and tant information about the kinds of experi-
months of fieldwork it can become difficult ences the turtle participants were having
to decipher what information came from and the significance of the project to them.
what sources unless the fieldworker has a While being part of any setting necessar-
routine system for documenting sources ily involves personal choices about social re-
and uses that system with great discipline, lationships and political choices about group
thoroughness, and care. alliances, the emphasis on making strategic
Key informants can be particularly help- decisions in the field should not be inter-
ful in learning about subgroups to which the preted as suggesting that the conduct of
observer does not or cannot have direct ac- qualitative research in naturalistic settings is
cess. During the second year of the wilder- an ever-exciting game of chess in which
ness education program, one informal players and pieces are manipula ted to ac-
group, mostly women, dubbed themselves complish some ultimate goal. Fieldwork cer-
the "turtles" to set themselves apart from tainly involves times of both exhilaration
participants, mostly men, who had more ex- and frustration, but the dominant motifs in
perience in the wilderness and wanted to fieldwork are hard work, long hours to both
hike at a fast pace, climb the highest peaks, do observations and keep up-to-date with
or otherwise demons trate their prowessa field notes, enormous discipline, attention
group they called somewhat disparagingly to details, and concentration on the mun-
the "truckers" (trucks being unwelcome in dane and day-to-day. The routinization of
the wilderness). Having had a full year of fieldwork is a time of concentrated effort
wilderness experiences the first year of the and immersion in gathering data. Alas, let
program, I didn't qualify to become an inti- the truth be told: The gathering of field
mate part of the turtles. I therefore estab- data involves very little glory and an abun-
lished an informant relationship with one of dance of nose-to-the-grindstone drudgery.

Bringing Fieldwork to a Close

ell, I've gotten to the end of the subjectof the pageof your
patience and my time.

Alice B. Toklas in a letter to Elizabeth Hansen, 1949

In traditional scholarly fieldwork within cific repor ting deadlines, stated in a con-
anthropology and sociology, it can be diffi- tract, that affect the length of and resources
cult to predict how long fieldwork will last. available for fieldwork, and the intended
The major determinant of the length of the uses of evaluative findings.
fieldwork is the investigator's own re- In the previous section, we looked at the
sources, interests, and needs. Evaluation many complex relationships that get formed
and action research typically have quite spe- during fieldwork, relationships with key in-
Fieldzuork Strategies and Observation Methods lj. 323

formarits, hosts, and sponsors in the setting was observed show up more in the field
who helped with entre and may have sup- notes. Some of these explanations have been
ported ongoing fieldwork, helping solve offered by others; some occur directly to the
problems and smooth over diffculties. In observer. In short, analysis and interpreta-
collaborative research, relationships with tion will have begun even before the ob-
coresearchers will have deepened. In any ex- server has left the field.
tended involvement within a setting, friend- Chapter 9 discusses analysis strategies at
ships and alliances are formed. As fieldwork length. At this point, I simply want to recog-
comes to an end, an exit or disengagement nize the fact that data gathering and analysis
strategy is needed. While a great deal of at- flow together in fieldwork, for there is usu-
tention has traditionally been paid to enter- ally no definite, fully anticipated point at
ing the field, much less attention has been which data collection stops and analysis be-
given to the disengagement process, what gins. One process flows into the other. As the
Snow (1980) has called the "neglected prob- observer gains confidence in the quality and
lem in participant observation research." meaningfulness of the data, sophisticated
One side of the coin is disengagement. about the setting under study, and aware
The other side is reentry back to one's life af- that the end draws near, additional data col-
ter extended fieldwork or an all-consuming lection becomes increasingly selective and
project. When I went to do graduate research strategic.
in Tarizania, our team received a lot of sup- As fieldwork draws to a close, the re-
port and preparation for entry, much of it searcher is increasingly concerned with veri-
aimed at avoiding culture shock. But when fication of already-collected data and less
we returned home, we were given no prepa- concerned with generating new inquiry
ration for what it would be like to return leads. While in naturalistic inquiry one
to America's highly commercial, materialis- avoids imposing preconceived analytical
tic, and fast-moving culture after months in categories on the data, as fieldwork comes to
an agrarian, community-oriented, slower- an end, experience with the setting will usu-
moving environment. The culture shock hit ally have led to thinking about prominent
coming home, not going to frica. themes and dimensions that organize what
Interpersonal, cross-cultural, disengage- has been experienced and observed. These
ment, and reentry issues ali deserve atten- emergent ideas, themes, concepts, and di-
tion as fieldwork comes to a close. Relation- mensionsgenerated inductively through
ships with people change and evolve from fieldworkcan also now be deepened, fur-
entry, through the middle days, and into the ther examined, and verified during the clo-
end of fieldwork. So does the fieldworker's sure period in the field.
relationship with the data and engagement Guba (1978) has described fieldwork as
in the inquiry process. That changed en- moving back and forth between the discov-
gagement in the inquiry process is what I ery mode and the verification mode like a
want to focus on here. wave. The ebb and flow of research involves
As you near completion of data gather- moving in and out of periods when the in-
ing, having become fairly knowledgeable vs tigator is open to new inputs, generative
about the setting being observed, more and data, and opportunistic sampling to periods
more attention can be shifted to fine-tuning when the investigator is testing out hunches,
and confirming observed patterns. Possible fine-tuning conceptualization, sifting ideas,
interpretations of and explanations for what and verifying explanations.
324 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

When fieldwork has gone well the ob- ator may have to bring the fieldwork to a
server grows increasingly confident that close before that state of real confidence has
things make sense and begins to believe in fully emerged. Nevertheless, I find that
the data. Glaser and Strauss (1967), com- there is a kind of Parkinson's law in field-
menting on grounded theory as an outcome work: As time runs out, the investigator
of fieldwork, have described the feelings feels more and more the pressure of making
that the traditional field observer has as sense out of things, and some form of order
fieldwork moves to a close, data-based pat- does indeed begin to emerge from the obser-
tems have emerged, and the whole takes vations. This is a time to celebrate emergent
shape: understandings even while retaining the
criticai eye of the skeptic, especially useful
The continuai intermeshing of data collection in questioning one's own confident conclu-
and analysis has directbearing on how the re- sions.
search is brought to a close. When the re-
searcher is convrnced that his conceptual
Evaluation Feedback
framework forms a systematic theory, that it is
a reasonably accurate statement of the matter In doing fieldwork for program evalua-
studied, that it is couched in a form possible tion, in contrast to theory-oriented scholarly
for others to use in studying a similar area, and field research, the evaluator observer must
that he can publish his results with confi- be concerned about providing feedback,
dence, then he has neared the end of his re- making judgments, and generating recom-
search mendations. Thus, as the fieldwork draws to
Why does the researcher trust what he a close, the evaluator observer must begin to
knows? . . . They are his perceptions, his per- consider what feedback is to be given to
sonal experiences, and his own hard-won whom and how.
analyses. A field worker knows that he knows, Giving feedback can be part of the verifi-
not orily because he has been in the field and cation process in fieldwork. My own prefer-
because he has carefully discovered and gen- ence is to provide the participants and staff
erated hypotheses, but also because "in his with descriptions and analysis, verbally and
bons" he feels the worth of his final analysis. informally, and to include their reactions as
He has been living with partial analyses for part of the data. Part of the reciprocity of
many months, testing them each step of the fieldwork can be an agreement to provide
way, until he has built this theory. What is participants with descriptive information
more, if he has participated in the social life of about what has been observed. I find that
is subject, then he has been living by his anal - participants and staff are hungry for such in-
yses, testing them not only by observation and formation and fascinated by it. I also find
interview but also by daily living, (pp. 224-25) that I learn a great deal from their reactions
to my descriptions and analyses. Of course,
This representation of bringing a ifs neither possible nor wise to report every-
grounded theory inquiry to a close repre- thing one has observed. Moreover, the infor-
sents the scholarly inquiry ideal. In the mal feedback that occurs at or near the end
"contracted deliverables" world of program of fieldwork will be different from the find-
evaluation, with limited time and resources, ings that are reported formally based on the
and reporting schedules that may not permit more systematic and rigorous analysis that
as much fieldwork as is desirable, the evalu- must go on once the evaluator leaves the
Fieldzuork Strategies and Observation Methods lj. 325

field. But that formal, systematic analysis lier during each field conference. During the
will take more time, so while one is still in second field conference in the second year,
the field it is possible to share at least some when a number of factors had combined to
findings and to learn from the reactions of make the program quite different from what
those who hear those findings. the staff had hoped for, the end-of-the-con-
Timing feedback in formative evalua- ference evaluation feedback session gener-
tions can be challenging. When the purpose ated an unusual amount of frustration from
is to offer recommendations to improve the the staff because my analyses of what had
program, the program staff will usually be happened had not been shared earlier.
anxious to get that information "ASAP" (as Again, I found some distrust of my insis-
soon as possible). The evaluator observer tence that those interpretations had
may even feel pressured to report findings emerged later rather than sooner as the pat-
prematurely, before having confidence in terns became clear to me.
the patterns that seem to be einerging. I ex- Evaluators who provide formative feed-
perienced this problem throughout the eval- back on an ongoing basis need to be consci-
uation of the wilderness education program. entious in resisting pressures to share find-
During the first year, we met with the staff at ings and interpretations before they have
the end of each field conference program confidence about what they have observed
(the three 10-day field conferences were and sorted out important patternsnot cer-
spread out over a year) to discuss what we tainty, but at least some degree of confi-
had observed and to share interpretations dence. The evaluator is caught in a dilemma:
about those observations. At the very first Reporting patterns before they are clearly
feedback session, the staff reaction was, "I established may lead program staff to inter-
wish you'd told us that in the middle of the vene inappropriately; withholding feedback
week, when we could have done something too long may mean that dysfunctional pat-
about it. Why'd you hold back? We could terns become so entrenched that they are dif-
have used what you've learned to change ficult, if not impossible, to change.
the program right then and there." No ideal balance has ever emerged for me
I tried to expiam that the implications of between continuing observations and pro-
what I observed had only become clear to viding feedback. Timing feedback is a mat-
me an hour or two before our meeting when ter of judgment and strategy, and it depends
my coevaluator and I had sat down with our on the nature of the evaluator's relationship
field notes, looked them over, and discussed with program staff and the nature of the
their significance together. Despite this ex- feedback, especially the balance between
planation, which struck me as altogether what staff will perceive as negative and pos-
reasonable and persuasive and struck the itive feedback. When in doubt, and where
staff as altogether disingenuous, from that the relationship between the evaluator and
moment forth a lingering distrust hung over program staff has not stabilized into one of
the evaluation as staff periodically joked long-term trust, I counsel evaluator observ-
about when we'd get around to telling them ers to err on the side of less feedback rather
what we'd learned next time. Throughout than more. As often happens in social rela-
the three years of the project, the issue of tim- tionships, negative feedback that was wrong
ing feedback surfaced several times a year. is long remembered and often recounted.
As they came increasingly to value our feed- On the other hand, it may be a measure of
back, they wanted it to come earlier and ear- the success of the feedback that program
326 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

staff so fully adopt it that they make it their surement. The physical world canbe altered
own and cease to credit the insights of the by the intrusion of the observer. How much
evaluator. more, then, are social worlds changed by the
Once feedback is given, the role of the intrusion of fieldworkers?
evaluator changes. Those to whom the feed- The effects of observation vary depend-
back was presented are likely to become ing on the nature of the observation, the type
much more conscious of how their behavior of setting being studied, the personality and
and language are being observed. Thus, procedures of the observer, and a host of un-
added to the usual effect of the fieldworker anticipated conditions. Nor is it simply in
on the setting being observed, this feedback fieldwork involving naturalistic inquiry that
dimension of fieldwork increases the impact scientific observers affect what is observed.
of the evaluator observer on the setting in Experimentalists, survey researchers, cost-
which he or she is involved. benefit analysts, and psychologists who ad-
Though this problem of reactivity is ac- minister standardized tests ali affect the sit-
centuated in evaluation, it exists in any ob- uations into which they introduce data col-
servational inquiry. As the researcher pre- lection procedures. The issue is not whether
pares to leave the field, and people react to or not such effects occur; rather, the issue is
that imminent departure, the impact of the how to monitor those effects and take them
researcher's presence on the setting may be- into consideration when interpreting data.
come visible in new ways. Because those ef- A strength of naturalistic inquiry is that
fects have been of such major concern to the observer is suffciently a part of the situa-
people who engage in naturalistic inquiry, tion to be able to understand personally
the final section in this chapter considers this what is happening. Fieldworkers are called
question of how the observer affects what is on to inquire into and be reflective about
observed. how their inquiry intrudes and how those
mtrusions affect findings. But that's not al-
ways easy. Consider the case of anthropolo-
gist Napoleon Chagnon, who did fieldwork
!=f. The Observer and for a quarter century among the isolated and
What Is Observed: primitive Yanomami Indians who lived
Unity and Separation deep in the rain forest at the borders of
Venezuela and Brazil. He studied mortality
The question of how the observer affects rates by dispensing steel goods, including
what is observed has natural as well as so- axes, as a way of persuading people to give
cial science dimensions. The Heisenberg him the names of their dead relatives in vio-
uncertainty principie states that the instru- lation of tribal taboos. Brian Ferguson, an-
ments used to measure velocity and posi- other anthropologist knowledgeable about
tion of an electron alter the accuracy of mea- the Yanomami, believes that Chagnon's
surement, When the scientist measures the fieldwork destabilized relationships among
position of an electron, its velocity is villages, promoted warfare, and introduced
changed, and when velocity is measured, it disease. Chagon denies these charges but
becomes difficult to capture precisely the acknowledges extracting tribal secrets by
electron's position. The process of observ- giving informants gifts like beads and
ing affects what is observed. These are real fishhooks, capitalizing on animosities be-
effects, not just errors of perception or mea- tween individuais, and bribing children for
Fieldzuork Strategies and Observation Methods lj. 327

information when their elders were not server's degree of participation in the
around. He gave away machetes in ex- setting, the visibility and openness of field-
change for blood samples for his genealogi- work, and the duration of fieldwork (see Ex-
cal studies. The long-term effects of his field- hibit 6.1 earlier in this chapter) to anticipate
work have become a matter of spirited certain of the situations that may arise and to
debate and controversy within anthropol- establish strategies for how those situations
ogy (Geertz 2001; Tierney 2000a, 2000b). will be handled. For example, I have been in-
At the other end of the intrusion contin- volved as a participant observer- evaluator
uum we find those qualitative designs in a number of professional development
where "intrusions" are intentionally de- programs where participants were expected
signed because the qualitative inquiry is to exercise increasing control over the curric-
framed as an intended form of desired inter- ulum as the program evolved. Had I fully
vention. This is the case, for example, with participated in such participatory decision
collaborative and participatory forms of in- making, I could have influenced the direc-
quiry in which those people in the setting tion of the program. Anticipating that prob-
who become coresearchers are expected to lem and reviewing the implications with
be affected by participation in the inquiry. program staff, in each case I decided not to
The processes of participation and collabo- participate actively in participant-led deci-
ration can be designed and facilitated to sion making to the full extent I might have
have an impact on participants and collabo- had I not been involved in the role of evalua-
rators quite beyond whatever findings they tor observer. The participatory and empow-
may generate by working together. In the ering philosophy of these programs called
process of participating in an evaluation, for each participant to articulate interests
participants are exposed to and have the op- and help make happen those things that he
portunity to learn the logic of research and or she wanted to have happen. In my role as
the discipline of data-based reasoning, Skills evaluator observer, I had to reduce the ex-
are acquired in problem identification, crite- tent to which I acted out that philosophy so
ria specification, and data collection, analy- as to limit my impact on the direction of the
sis, and interpretation. Acquisition of re- group. 1 aimed my involvement at a levei
search skills and ways of thinking can have a where I would not appear withdrawn from
longer-term impact than the use of findings the process, yet at the same time attempted
from a particular evaluation study. This to minimize my influence, especially where
"learning from the process" as an outcome the group was divided on priorities.
of participatory and collaborative inquiry Another example comes from evaluation
experiences is called process use in contrast to of a community leadership program men-
findings use (Patton 1997a: Chapter 5,1998, tioned previously in this chapter. As a
1999c). three-person team of participant observers,
While it is not possible to know precisely we participated fully in small-group leader-
how collaboration will affect coresearchers ship exercises. When the groups in which we
or to fully anticipate how an observer will af- participated were using concepts inappro-
fect the setting observer, both cases illustrate priately or doing the exercise wrong, we
the need to be thoughtful about the intercon- went along with what participants said and
nections between observers and observed. It did without making corrections. Had we re-
is possible, however, when designing the ally been only participantsand not partici-
study and making decisions about the ob- pant evaluatorswe would have offered
328 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

corrections and solutions. Thus, our roles only a few. Ali imply that the way in which a
made us more passive than we tended natu- person construes his relationship to the phe-
rally to be in order not to domina te the small nomenal world is a function of his point of view
groups. We had anticipated this possibility about it. That is, relationship is not a given nor
in the design stage prior to fieldwork and an absolute, but depends upon a personal per-
had agreed on this strategy at that time. spective. It is also true that perspective can
The role and impact of the evaluator ob- shift, the only necessity of a person's human-
server can change over the course of field- ity being that he takes some stance in relation-
work. Early in the wilderness program, I ship to the events about him. (pp. 8-9)
kept a low profile during participant-led
planning discussions. Later in the program, Carini is here articulating the interde-
particularly during the final field conference pendence between the observer and what
of the second year, I became more engaged is observed. Prior to data collection, the
in discussions about the future direction of fieldworker plans and strategizes about the
the project. hoped-for and expected nature of that inter-
Reporting on the relationship between dependence. But things don't always unfold
the observer and the observed, then, and the as planned, so observers must make some
ways in which the observer may have af- effort to observe themselves observing
fected the phenomenon observed becomes and record the effects of their observations
part of the methodological discussion in on the people observed and, no less impor-
published fieldwork reports and evaluation tant, reflect on changes they've experienced
studies. In that methodological discussion from having been in the setting. This means
(or the methods chapter of a dissertation), being able to balance observation with re-
the observer presents data about the effects flection and manage the tension between en-
of fieldwork on the setting and people gagement and detachment.
therein and also the observer's perspective Bruyn (1966), in his classic work on par-
on what has occurred. As Patricia Carini ticipant observation, articulated a basic
(1975) has explained, such a discussion ac- premise of participant observation: the "role
knowledges that findings inevitably are in- of the participant observer requires both
fluenced by the observer's point of view detachment and personal involvement"
during naturalistic inquiry: (p. 14). To be sure, there is both tension and
ambiguity in this premise. How it plays out
The observer has a point of view that is central in any given situation will depend on both
to the datum and it is in the articula tionin the observer and the phenomenon being ob-
the revela tion of his point of viewthat the served.
datum of inquiry is assumed to emerge. In ef-
fect the observer is here construed as one mo- Thus, we may observe at the outset that while
ment of the datum and as such the fabric of his the traditional role of the scientist is that of a
thought is inextricably woven into the datum neutral observer who remains unmoved, un-
as he is assumed to be constituent of its mean- changed, and untouched in his examination of
ing. From this assumption it is possible to con- phenomena, the role of the participant ob-
sider the relationship of the observer to the server requires sharing the sentiments of peo-
phenomenon under inquiry. Relatedness can ple in social situations; as a consequence he
be stated in many ways: opposition, identity, himself is changed as well as changing to some
proximity, interpenetration, isolation, to name degree the situation in which he is a partici-
Fieldzuork Strategies and Observation Methods lj. 329

pant The effects are reciprocai for observer Fieldwork is not for everyone. Some, like
and observed. The participant observer seeks, Henry James, will find that "innocent and
on the one hand, to take advantage of the infinite are the pleasures of observation."
changes due to his presence in the group by re- Others will find observational research any-
cording these changes as part of his study, and thing but pleasurable. Some students have
on the other hand, to reduce the changes to a described their experiences to me as tedious,
minimum by the manner in which he enters frightening, boring, and "a waste of time,"
into the life of the group. (Bruyn 1966:14) while others have experienced challenge,
exhilaration, personal learning, and intellec-
Whether one is engaged in participant ob- tual insight. More than once the same stu-
servation or onlooker observation, what dent has experienced both the tedium and
happens in the setting being observed will, the exhilaration, the fright and the growth,
to some extent, be dependent on the role as- the boredom and the insight. Whatever the
sumed by the observer. Likewise, the nature adjectives used to describe any particular in-
of the data collected will, to some extent, be dividual^ fieldwork, of this much we are as-
dependent on the role and perspective of the sured: The experience of observing provides
observer. And just as the presence of the ob- the observer with both experience and ob-
server can affect people observed, so too the servations, the interconnection being ce-
observer can be affected. mented by reflection. No less an authority
than William Shakespeare gives us this as-
The Personal surance.
Experience of Fieldwork
Armado: "How hast thou purchased this
The intersection of social science proce- experience?"
dures with individual capabilities and situa- Moth: "By my penny of observation."
tional variation is what makes fieldwork a
highly personal experience. At the end of Love's Labour's Lost
her book Doing Fieldwork, Rosalie Wax (1971)
reflected on how fieldwork changed her:
A Part of and Apart
A colleague has suggested that I reflect on the From the World Observed
extent to which I was changed as a person by
doing field work. I reflected and the result as- The personal, perspective-dependent na-
tonished me. For what I realized was that I had ture of observations can be understood as
not been greatly changed by the things I suf- both a strength and a weakness, a strength
fered, enjoyed or endured; nor was I greatly in that personal involvement permits first-
changed by the things I did (though they hand experience and understanding, and a
strengthened my confidence in myself). What weakness in that personal involvement in-
changed me irrevocably and beyond repair troduces selective perception. In the deep
were the things learned. More specifically, engagement of naturalistic inquiry lies both
these irrevocable changes involved replacing its risks and its benefits. Reflection on that
mythical or ideological assumptions with the engagement, from inside and outside the
correct (though often painful) facts of the situ- phenomenon of interest, crowns fieldwork
ation. (p. 363) with reflexivity and makes the observer the
330 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

OilL^ cwim.iL, oisdm.n-JEP riyi^Ri^vrinN?-

i''i l:ii>ii.il;. i1i ::i!r-liil.i.Vi.Vi ri:,;i".V!:"V.^ii\ .=="!V! / CTII, nD-Ifflrf-'fl!'(,Hl"CiTiFt !:V!i .<iiiVi!j.ii.l
uftirs =!VJ: .!.i'|ri!iiv i'i i.'i !" ': ;J L-i'i !::p, nhii' H i*PFTtatfiVfaj :'bf \TU ^h-ft !! P; ': W :^ h? "i.in
KVlH H=:'i .V..' i ViA
-.;: |;ai-|.-rrs .;>!;:>:>/Vi',.;;.';u.ijn Yj',1;kjiT-.;.in\s: tf!'MXi.:iV:ii!j-'i.!"i)",'!vn!!i^M-n-:'e*0'':
i'i.vh. I J t ;ii i! s: .M.^V Li.'t J A I H 1 I I ! =. ^ - . ! . ^ " V N I - I.V.V- ;;V- ^ rs :;:I : .:' ' : ; . T : ! . ' ' ! : ; ' ! ! ' . H K
!

toai* I.a!vr-:j ' !.':lYi' :.i.!.v ,!i n!! J1 iiVil{{lATifr n.:|;'.r PGTFL h 31 - tf E LV.VL ii" itfV i P/C, T !:: !! in'!i: '.i-riViWf
fhv n.'iT=i-h. !;iVi:r.'!: .';|:M |!i;i.;.i?:;f rijii {TI i\!j i-.YiVi i^i !'*:. iVhin.!,i.! i'i y '.1piin!h:;"' ,'i!n;i!
Mil 5 fl ;!Mii^hY in>% ^Min iViS!p,:!.'-i:=!:r!:! !i,:ifi!\ ii!,'! .!:'' = hi".jvAi.VTM h:n;'!r pajip.;.:;. fl.'- W-H
!

! : ; ; ! ' J " M. :;::.-;!!- I:M.';;! rivn.s:-'J-JU

, , 1

': Ti 'ii"n H p i r ; ! " rVi TiT!.:: 1 v\;; ;! ;:;| W i,:; i \ ^ (I-i'.^1' j' n' ! , 1 , ! hi h ii h : v!;'"".= n ' ' VFTL -

!.;,!>:.:;;; r| nr-viV!r-.rnpni1:.:;=.,h:;i i ! !:!.'I:'.H1 ! ' 'vJir^.Vy {yi^Pi o' O!:=!'"!=::;iVv LI

i.si:li:i-iii/iVvi. .'.'mlii! J 4 H i

observedeven if only by oneself. So we re- the characteristics of the setting, and the
peat Halcolm's refrain that opened this skills, interests, needs, and point of view
chapter: that y o u , as observer, bring to y o u r engage-
ment. Yet, the conduct of observational re-
Go out into the world. Live among the peoples search is not without direction. Exhibit 6.6
of the world as they live. Learn their language. offers a modest list of 10 guidelines for field-
Participate in their rituais and routines. Taste work (not, please notice, commandments,
of the world. Smell it. Watch and listen. Touch just guidelines) by way of reviewing some
and be touched. Write down what you see and of the major issues discussed in this chapter.
hear, how they think and how you feel. Beyond these seemingly simple but decep-
Enter into the world. Observe and wonder. tively complex prescriptions, the point re-
Experience and reflect. To understand a world mains that what you do depends on a great
you must become part of that world while at number of situational variables, your own
the same time remaining separate, a part of capabilities, and careful judgment informed
and apart from. by the strategic themes for qualitative in-
Go then, and return to tell what you see quiry presented in the first chapter (Exhibit
and hear, what you learn, and what you come 2.1).
to understand.
Having considered the guidelines and
strategic themes for naturalistic field-based
Summary Guidelines research, and after the situational con-
for Fieldwork straints on and variations in the conduct of
fieldwork have been properly recognized
A reader who came to this chapter looking and taken into account in the design, there
for specific fieldwork rules and clear proce- remains only the core commitment of quali-
dures would surely be disappointed. tative inquiry to reaffirm. That core commit-
Looking back over this chapter, the major ment was articulated by Nobel laureate
theme seems to be, What you do depends Nicholas Tinbergen in his 1975 acceptance
on the situation, the nature of the i n q u i ^ speech for the Nobel Prize in physiology
Fieldzuork Strategies and Observation Methods lj. 331

E X H I B I T 6.6 Summary Guidelines for Fieldwork

1. Design the fieldwork to be clear about the role of the observer (degree of participation);
the tension between insider (emic) and outsider (etic) perspectives; degree and nature
of coiaboration with coresearchers; disclosure and explanation of the observer's role to
others; duration of observations (short vs. long); and focus of observation {narrow vs.
broad). (See Exhibit 6.1.)
2. Be descriptive in taking field notes. Strive for thick, deep, and rich description.
3. Stay open. Gather a variety of information from different perspectives. Be opportunistic
in following leads and sampling purposefully to deepen understanding. Allow the design
to emerge flexibly as new understandngs open up new paths of inquiry.
4. Cross-validate and trianguiate by gathering different kinds of data: observations,
interviews, documents, artifacts, recordings, and photographs. Use muitiple and mixed
methods.
5. Use quotations; represent people n their own terms. Capture participants' views of their
experiences in their own words.
6. Select key informants wisely and use them carefully. Draw on the wisdom of their
informed perspectives, but keep in mind that their perspectives are selective.
7. Be aware of and strategic about the different stages of fieldwork.
a. Build trust and rapport at the entry stage. Remember that the observer is also being
observed and evaluated.
b. Attend to relationships throughout fieldwork and the ways in which relationships
change over the course of fieldwork, including relationships with hosts, sponsors within
the setting, and coresearchers in collaborative and participatory research.
c. Stay alert and dscipned during the more routine, middle phase of fieldwork.
d. Focus on pulling together a useful synthesis as fieldwork draws to a close. Move
from generating possibilities to verifying emergent patterns and confirming themes.
e. Be disciplined and conscientious in taking detailed field notes at ali stages of
fieldwork.
f. In evaluations and action research, provide formative feedback as part of the
verifcation process of fieldwork. Time that feedback carefully. Observe its impact.
8. Be as involved as possible in experiencing the setting as fully as is appropriate and
manageable while maintaining an analytical perspective grounded in the purpose of
the fieldwork.
9. Separate description from interpretation and judgment.
10. Be reflective and reflexive. Include in your field notes and reports your own experiences,
thoughts, and feelings. Consider and report how your observations may have affected
the observed as well as how you may have been affected by what and how you've
participated and observed. Ponder and report the origins and implications of your own
perspective.
332 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

Fm a researcher.
This is great. A serendipitous
purposeful sample o f a criticai
incident involving conflict for
my study on posimodem en-
tanglements among strangers
in a globai viliage context.

Constructivist Rashonton heaven: Multiple and diverse perspectives

and medicine: "watching and wondering." behaviors, he was able to make a major med-
Tinbergen explained that it was by watching icai and scientific contribution. His research
and wondering that he had, despite being methodology: "watching and wondering."
neither a physiologist nor a medicai doctor,
discovered what tumed out to be a major S Notes
breakthrough in our understanding of au-
tism. His observations revealed that the ma- 1. Excerpt from "Little Gidding" in the Four
jor clinicai research on autism did not hold Quartis by T. S. Eliot. Copyright 1942 by T. S.
up outside clinicai settings. His "watching Eliot; renewed 1970 by Esme Valerie Eliot. Re-
printed by permission of Harcourt, Inc.
and wondering" allowed him to see that
2. Excerpt from "The Elephant's Child," from
normal individuais, those not clinically la- Just So Stories, by Rudyard Kipling. Used by per-
beled as autistic, exhibited under a variety of mission of A. P. Watt Ltd. on behalf of The Na-
circumstances ali of the behaviors described tional Trust for Places of Historical Interest or
as autistic in clinicai research. He also noted Natural Beauty. Original publication 1902.
that children diagnosed as autistic re- 3. From Traveling Light: Collected and Nezu
Poems. Copyright 1999 by David Wagoner.
sponded in nonautistic ways outside the
Used with permission of the University of Illinois
clinicai setting. By observing people in a va- Press.
riety of settings and watching a full range of 4. Used with permission of Joyce Keller.
Between-Chapters Interlude
Outside to Inside, Inside to Outside
Shifting Perspectives

Preface gram evaluator). She recounts those transi-

tions from outside to inside, and inside to
The preceding chapter on fieldwork in- outside, and how they have shaped her per-
cluded discussion of insider (emic) versus spective on research and evaluation. Her
outsider (etic) perspectives. Understand- reflections provide a poignant and insight-
ing different perspectives from inside and ful transition from our discussion of par-
outside a phenomenon goes to the core of ticipant observation in the last chapter to
qualitative inquiry. Experience affects per- interviewing strategies in the next chapter,
spective. Perspective shapes experience. In both methods aimed at bridging insider-
the next chapter, on interviewing, we shall outsider perspectives. Barbara Lee gener-
continue to explore ways of capturing the ously wrote these reflections especially for
experiences and getting deeply into the this book. I thank Barbara for her openness,
perspectives of those who have encoun- courage, commitment, and insights.
tered whatever phenomenon interests us.
The reflections that follow look at the ex-
perience of mental illness from the inside
S "Nothing About Us,
and outside, and how being on the inside
Without Us"
can dramatically change the view from the
outside. Barbara Lee moved from the out-
side (Ph.D. researcher and mental health I was a high school and jnior high school
professional) to the inside as an involun- science teacher, a school psychologist and
tary participant observer (a patient in a guidance counselor, and finally an educa-
locked mental health facility) and back tional researcher, before I completed my
again to the outside (as a professional pro- Ph.D. (on the second try!). I had specifically
a 335
336 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

sought that degree in my mid-40s to get rnaze. Probably the most serious is that what
evaluation and research skills, which I ac- I had assumed to be "treatment" from the
complished nicely at St. Louis University, in viewpoint of the researcher now looked like
one of the few programs in the 1980s de- mostly futile efforts that were most often ex-
signed to train graduate students in evalua- perienced as punishment and threat from
tion theory and methodology. the viewpoint of "patient." I was not asked if
Since I had been a scientist of some sort I would go into the hospital. I was told I had
for ali of my career life, I was a "natural" for to. If I got angry at something, someone gave
the field of evaluation. I thought like a scien- me powerful medications that made me
tist. I was familiar with the practice of re- feel like a zombie.
search inbiology, bacteriology, and field bot- While some of what happened in my hos-
any and had a special interest in medicine. pitaliza tion helped, much of what I experi-
But through some quirks of fate and person- enced made me feel much worse. I was in-
ality, I found myself working as a clircian, carcerated and my jailers looked at me
providing therapy and case management to kindly, certain that I was being locked up,
people with severe mental disorders. Then, strapped to a bed, and injected with medica-
after having been gainfully employed my tions for my own good. Even though I actu-
entire adult life, and successfully raising two ally entered the hospital "voluntarily," the
children who had now produced one grand- threat of involuntary treatment and perma-
child each, I was forced into the locked, psy- nent damage to my ability to earn a living
chiatric ward of a hospital for the third time was the driving force that got me there, and
in my life. kept me there, and taught me to "make nice"
Back at work, after nearly two months in for the staff, lest they refuse to certify me
the hospital, I found myself, for the first sane and let me be free again.
time, looking at my professional work and Asa human services program evaluator, I
reading professional literature with the eyes learned to do needs assessments, to use
of one from the other side of the locked proven treatment methods in a package deal
doors and medicai charts. called a program, to gather data of various
The irony of my situation was obvio us: I kindssometimes even from the people
treated people like me! Thus began a shift of who were getting the program. I learned
viewpoint that has radically altered my how to interpret the data gathered in the en-
practice of evaluation in the field of mental vironmental context in which the program
health. operated and how to get and report reliable
First, I had to throw out some grand as- and credible information to those who make
sumptions. As a scientist, I trusted scientific decisions about programs. Sometimes, I ad-
method and worshipped at the same shrine mit, I even offered my own "expert" judg-
of true experimental design and random as- ment about the value of the program. Now a
signment as everyone else. But now I was whole new set of questions confronted me as
much more conscious that the "lab rats," the a professional.
subjects of our research, literally have minds What can distort the perceptions of those
of their own. Some of the most treasured as- I ask about "needs" and how much is the dis-
sumptions of mental health research were tortion? The providers believe they have the
looking awfully different from inside the well-being of the clients at heart, but the cli-
Shifting Perspectives . 337

ents may experience the treatments as more threatening voices might just as likely be real
disabling than the symptoms of the disease. as hallucinated. The provider who is con-
Clients are taught to mistrust their own fined to the office and distance of a profes-
symptom-distorted thoughts and are flatly sional relationship will not know when
ignored when psychotic, yet virtually ali of there is abuse in the home the client never
them can make reasoned decisions if they speaks of, because the abuser is someone
have adequate information and are asked. It they love or who controls their money as
seems to be assumed that a psychiatric label payee. Mental health workers are put in the
defines the "needs" of people with severe role of defenders of the public purse, and
mental illness. But other needs may be a con- then we wonder why clients feel their safety
sequence of a stingy health care system that net of services threatened with every dollar
won't provide necessary and expensive they are given or earn and fail to trust their
medications unless you are completely dis- "providers."
abled, and social stignia makes it nearly im- In short, I may apply many of the same
possible to get a good job with benefits after ideas, theories, methods, and interpreta-
psychiatric hospitalizations. A clinician who tions as I always did as a program evaluator.
admits to having a psychiatric label of severe But now I always question, not just the va-
mental illness will never be hired, so it can be lidity, reliability, and generalizability of the
survival as well as denial to deny even to evaluation work itself but also the hidden
oneself that one is one of "those crazies." assumptions that surround it. I will always
Providers are taught, in ali sincerity and be seekng to empower those disenfran-
good intentions, to act in a kind of parental chised by custom, poverty, and stigma. Fur-
role toward clients, a benign dictatorship. thermore, I will always be conscious of the
But their "subjects" are people who have al- fact that my work is always limitedand
ready had their dignity as adults medically empoweredby the selection of data and
removed, their privacy invaded, and the job methods tobe used. But if I want to be part of
and relationship underpinnings of Ameri- the solution rather than part of the problem,
can self-esteem destroyed; been told they I better be sure I know what the experience
will be sick for life; and been medicated so of different stakeholders really is, not con-
they cannot perform sexually or sometimes strained by the limited questions I may think
even read a good book. Is it any wonder that to ask, or guided too narrowly by work done
many of them (us) will accept survival as an in the past.
adequate "quality of life"? As an evaluator, I now try to approach my
What are the limits of (a) theory about task with equal measures of chutzpah and
what the "problem" is; (b) the kinds, rele- humility so that I will not fail to challenge ali
vance, and quality of data collected in the the assumptions, especially my own, nor
past; and (c) the stakeholders' (mentally ill ever assume that I have ali of the questions,
people) freedom to express themselves? much less the answers, right. I have adopted
Evaluation designs that test the effects of the motto of the people who do not claim the
treatment programs on the individual don't title of "consumer," because they were not
address the problem of living in a neighbor- given true choice about treatment when they
hood where life is stressful and taxis won't found themselves pinned with psychiatric
take you home after dark, and where the labels: "Nothing about us, without us."
338 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

When was the last time you saw the cli- answer truthfully, "Just last week, or last
ents who are the intended target of the pro- month": Thank you!
gram, sitting at the table with the evaluators
and program providers, freely exchanging Barbara Lee, Ph.D., prosumer and
perspectives and ideas? For those who can mental health program evaluator
Qualitative Interviewing

S e y o n d .Silekrf O b s e ^ v a + i o ^

After much cloistered study, three youths carne before Halcolm to ask how
they might further increase their knowledge and wisdom. Halcolm sensed that
they lacked experience in the real world, but he wanted to have them make the
transition from seclusion in stages.
During the first stage he sent them forth under a six-month vow of silence.
They wore the identifying garments of the muted truth-seekers so that people
would know they were forbidden to speak. Each day, according to their instruc-
tions, they sat at the market in whatever village they entered, watching but
never speaking. After six months in this fashion they returned to Halcolm.
"So," Halcolmbegan, "you have returned. Your period of silence is over. Your
transition to the world beyond our walls of study has begun. What have you
learned so far?"
The first youth answered, "In every village the patterns are the same. People
come to the market. They buy the goods they need, talk with friends, and leave. I
have learned that ali markets are alike and the people in markets always the
same."
Then the second youth reported, "I too watched the people come and go in
the markets. I have learned that ali life is coming and going, people forever mov-
ing to and fro in search of food and basic material things. I understand now the
simplicity of human life."

fi. 339
340 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

|
Halcolm looked at the third youth: "And what have you learned?"
"I saw the same markets and the same people as my fellow travelers, yet I know
not what they know. My mind is filled with questions. Where did the people come
from? What were they thinking and feeling as they came and went? How did they
happen to be at this market on this day? Who did they leave behind? How was to-
day the same or different for them? I have failed, Master, for I am filled with ques-
tions rather than answers, questions for the people I saw. I do not know what I
have learned."
Halcolm smiled. "You have learned most of ali. You have learned the impor-
tance of finding out what people have to say about their experiences. You are
ready now to return to the world, this time without the vow of silence.
"Go forth now and question. Ask and listen. The world is just beginning to
open up to you. Each person you question can take you into a new part of the
world. The skilled questioner and attentive listener know how to enter into an-
other's experience. If you ask and listen, the world will always be new."
From Halcolm's Episte?nological Parables

Rigorous and social scientists have become suspect, as

Skillful Interviewing have our methods. The popular business
magazine Forbes (self-proclaimed "The Cap-
The very popularity of interviewing may italistTool") has opined, "People become so-
be its undoing as an inquiry method. In the ciologists because they hate society, and they
contemporary "interview society" (Fontana become psychologists because they hate
and Frey 2000:646), so much interviewing is themselves" (quoted in Geertz 2001:19).
being done so badly that its credibility may Such glib sarcasm, anti-intellectual at the
be undermined. Television, radio, maga- core, can serve to remind us that we bear the
zines, newsletters, and Web sites feature in- burden of demonstrating that our methods
terviews. In their ubiquity, interviews done involve rigor and skill. Interviewing, seem-
by social scientists become indistinguish- ingly straightforward, easy, and universal,
able in the popular mind from interviews can be done well or poorly. This chapter is
done by talk show hosts. The motivations of about doing it well.

Inner Perspectives

nterviewing is rather like a marriage: everybody knows what it is, an

awful lot of people do it, and yet behind each closed door there is a
world of secrets.

A. Oakley (1981:41)

We interview people to find out from serve. The issue is not whether observa-
them those things we cannot directly ob- tional data are more desirable, valid, or
Qualitative Intervieiving |J, 341

meaningful than self-report data. The fact is said in Es Lebe das Leben I, "I know how to lis-
that we cannot observe everything. We can- ten when clever men are talking. That is the
not observe feelings, thoughts, and inten- secret of what you call my influence." Evalu-
tions. We cannot observebehaviors that took ators must learn how to listen when knowl-
place at some previous point in time. We edgeable people are talking. That may be the
cannot observe situations that preclude the secret of their influence.
presence of an observer. We cannot observe An evaluator, or any interviewer, faces
how people have organized the world and the challenge of making it possible for the
the meanings they attach to what goes on in person being interviewed to bring the inter-
the world. We have to ask people questions viewer into his or her world. The quality of
about those things. the information obtained during an inter-
The purpose of interviewing, then, is to view is largely dependent on the inter-
allow us to enter into the other person's per- viewer. This chapter discusses ways of ob-
spective. Qualitative interviewing begins taining high-quality information by talking
with the assumption that the perspective of with people who have that information.
others is meaningful, knowable, and able to We'll be delving into "the art of hearing"
be made explicit. We interview to find out (Rubin and Rubin 1995).
what is in and on someone else's mind, to This chapter begins by discussing three
gather their stories. different types of interviews. Later sections
Program evaluation interviews, for ex- consider the content of interviews: what
ample, aim to capture the perspectives of questions to ask and how to phrase ques-
program participants, staff, and others asso- tions. The chapter ends with a discussion of
ciated with the program. What does the pro- how to record the responses obtained dur-
gram look and feel like to the people ining interviews. This chapter emphasizes
volved? What are their experiences in the skill and technique as ways of enhancing the
program? What thoughts do people knowl- quality of interview data, but no less impor-
edgeable about the program have concem- tant is a genuine interest in and caring about
ing program operations, processes, and out- the perspectives of other people. If what
comes? What are their expectations? What people have to say about their world is gen-
changes do participants perceive in them- erally boring to you, then you will never be a
selves as a result of their involvement in the great interviewer. Unless you are fascinated
program? It is the responsibility of the evalu- by the rich variation in human experience,
ator to provide a framework within which qualitative interviewing will become drudg-
people can respond comfortably, accurately, ery. On the other hand, a deep and genuine
and honestly to these kinds of questions. interest in learning about people is insuffi-
Evaluators can enhance the use of quali- cient without disciplined and rigorous in-
tative data by generating relevant and high- quiry based on skill and technique.
quality findings. As Hermann Sudermann

!?. Variations in Qualitative Interviewing

O n her deathbed Gertrude Stein asked her beloved companion, Alice B.

Toklas, "What is the answer?" When Alice, unable to speak, reinained
silent, Gertrude asked: "In that case, what is the question?"

fcdhf
342 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

The question in this section is how to for- pose and poses quite varying interviewer
mai questions. There are three basic ap- challenges.
proaches to collecting qualitative data
through open-ended interviews. They in-
volve different types of preparation, concep-
The Informal
tualization, and instrumentation. Each ap-
Conversational Interview
proach has strengths and weaknesses, and
each serves a somewhat different purpose.
The informal conversational interview is
The three alternatives are
the most open-ended approach to inter-
the informal conversational interview, viewing. It is also called "unstructured inter-
viewing" (Fontana and Frey 2000:652). The
the general interview guide approach, and
conversational interview offers maximum
the standardized open-ended interview. flexibility to pursue information in what-
ever direction appears to be appropriate, de-
These three approaches to the design of the pending on what emerges from observing a
interview differ in the extent to which inter- particular setting or from talking with one or
view questions are determined and stan- more individuais in that setting. Most of the
dardized before the interview occurs. questions will flow from the immediate con-
The informal conversational interview relies text. Thus, the conversational interview con-
entirely on the spontaneous generation of stitutes a major tool of fieldwork and is
questions in the natural flow of an interac- sometimes referred to as "ethnographic in-
tion, often as part of ongoing participant ob- terviewing." No predetermined set of ques-
servation fieldwork. The persons being tions would be appropriate under many
talked with may not even realize they are be- emergent field circumstances where the
mg interviewed. The general interview guide fieldworker doesn't know beforehand what
approach involves outlining a set of issues is going to happen, who will be present, or
that are to be explored with each respondent what will be important to ask during an
before interviewing begins. The guide event, incident, or experience.
serves as a basic checklist during the inter- Data gathered from informal conversa-
view to make sure that ali relevant topics tional interviews will be different for each
are covered. In contrast, the standardized person interviewed. The same person may
open-ended interview consists of a set of ques- be interviewed on different occasions with
tions carefully worded and arranged with questions specific to the interaction or event
the intention of taking each respondent at hand. Previous responses canbe revisited
through the same sequence and asking each and deepened. This approach works partic-
respondent the same questions with essen- ularly well where the researcher can stay in
tially the same words. Flexibility in probing the setting for some period of time so as not
is more or less limited, depending on the na- to be dependent on a single interview op-
ture of the interview and the skills of inter- portunity. Interview questions will change
viewers. The standardized open-ended in- over time, and each new interview builds on
terview is used when it is important to those already done, expanding information
minimize varia tion in the questions posed to that was picked up previously, moving in
interviewees. Lefs look at each approach in new directions, and seeking elucidations
greater depth for each serves a different pur- and elaborations frm various participants.
Qualitative Intervieiving |J, 343

Being unstructured doesn't mean that rapid insights, formulate questions quickly
conversational interviews are unfocused. and smoothly, and guard against asking
Sensitizing concepts and the overall pur- questions that impose interpretations on the
pose of the inquiry inform the interviewing. situation by the structure of the questions.
But within that overall guiding purpose, the Data obtained from informal conversa-
interviewer is free to go where the data and tional interviews can be difficult to pull
respondents lead. together and analyze. Because different
The conversational interviewer must "go questions will generate different responses,
with the flow." Depending on how the inter- the researcher has to spend a great deal of
viewer^ role has been defined, the people time sifting through responses to find pat-
being interviewed may not know during terns that have emerged at different points
any particular conversation that data are be- in different interviews with different people.
ing collected. In many cases, participant ob- By contrast, interviews that are more sys-
servers do not take notes during such con- tematized and standardized facilitate analy-
versational interviews, instead writing sis but provide less flexibility and are less
down what they learned later. In other cases, sensitive to individual and situational dif-
it can be both appropriate and comfortable ferences.
to take notes or even use a tape recorder.
The strength of the informal conversa- The Interview Guide
tional method resides in the opportunities it
offers for flexibility, spontaneity, and re- An interview guide lists the questions or
sponsiveness to individual differences and issues that are to be explored in the course of
situational changes. Questions can be per- an interview. An interview guide is pre-
sonalized to deepen communication with pared to ensure that the same basic lmes of
the person being interviewed and to make inquiry are pursued with each person inter-
use of the immediate surroundings and situ- viewed. The interview guide provides top-
ation to increase the concreteness and imme- ics or subject areas within which the inter-
diacy of the interview questions. viewer is free to explore, probe, and ask
A weakness of the informal conversa- questions that will elucidate and illuminate
tional interview is that it may require a that particular subject. Thus, the interviewer
greater amount of time to collect systematic remains free to build a conversation within a
Information because it may take several con- particular subject area, to word questions
versations with different people before a spontaneously, and to establish a conversa-
similar set of questions has been posed to tional style but with the focus on a particular
each participant in the setting. Because this subject that has been predetermined.
approach depends on the conversational The advantage of an interview guide is
skills of the interviewer to a greater extent that it makes sure that the interviewer / eval-
than do more formal, standardized formats, uator has carefully decided how best to use
this go-with-the-How style of interviewing the limited time available in an interview sit-
may be susceptible to interviewer effects, uation. The guide helps make interviewing a
leading questions, and biases, especially number of different people more systematic
with novices. The conversational inter- and comprehensive by delimiting m ad-
viewer must be able to mteract easily with vance the issues to be explored. A guide is
people in a variety of settings, generate essential in conducting focus group inter-
344 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

views for it keeps the interactions focused ject on Reading, illustrates how it is possible
while allowing individual perspectives and to use a detailed outline to conduct a series
experiences to emerge. of interviews with the same respondents
Interview guides can be developed in over the course of a year. The flexibility per-
more or less detail, depending on the extent mitted by the interview guide approach will
to which the interviewer is able to specify become clearer after reviewing the third
important issues in advance and the extent strategy of qualitative interviewing: the stan-
to which it is important to ask questions in dardized open-ended interview.
the same order to ali respondents. Exhibit 7.1
provides an example of an interview guide The Standardized
used with participants in an employment Open-Ended Interview
training program. This guide provides a
framework within which the interviewer This approach requires carefully and
would develop questions, sequence those fully wording each question before the inter-
questions, and make decisions about which view. For example, the interview guide for
information to pursue in greater depth. the employment training program in Exhibit
Usually, the interviewer would not be ex- 7.1 simply lists "work experiences" as a
pected to go into totally new subjects that are topic for inquiry. In a fully structured inter-
not covered within the framework of the view instrument, the question would be
guide. The interviewer does not ask ques- completely specified:
tions, for example, about previous employ-
ment or education, how the person got into You've told me about the courses you've taken
the program, how this program compares in the program. Now I'd like to ask you about
with other programs the trainee has experi- any work experiences you've had. Let's go
enced, or the trainee's health. Other topics back to when you first entered the program
might still emerge during the interview, top- and go through each work experience up to
ics of importance to the respondent that are the present. Okay? So, what was your first
not listed explicitly on the guide and there- work experience?
fore would not normally be explored with
each person interviewed. For example,
Probes: Who did you work for?
trainees might comment on farnily support
What did you do?
(or lack thereof) or personal crises. Com-
ments on such concerns might emerge What do you feel you learned
when, in accordance with the interview doing that?
guide, the trainee is asked for reactions to What did you especially like about
program strengths, weaknesses, and so on, the experience, if anything?
but if farnily is not mentioned by the respon- What did you dislike, if anything?
dent, the interviewer would not raise the is- Transition: Okay, tell me about your next work
sue. experience.
An additional, more detailed example of
an interview guide is included as Appen- Why so much detail? To be sure that each
dix 7.1 at the end of this chapter. The exam- interviewee gets asked the same questions
ple in the chapter appendix, a "descriptive the same stimuliin the same way and
interview" developed by the Educational the same order, including standard probes.
Testing Service Collaborative Research Pro- A doctoral committee may want to see the
Qualitative Intervieiving |J, 345

Evaluation Interview Guide for Participants in

an Employment Training Program

What has the trainee done in the program?

is* activities
is* courses
^ groups
work experiences

Achievements?
f skills attained
is* products produced
is* outcomes achieved
knowledge gained
is* things completed
what can the trainee do that is marketable?

How has the trainee been affected in areas other than job skills?
is* feelings about self
attitudes toward work
is* aspirations
is* interpersonal skills

What aspects of the program have had the greatest impacts?

is* formal courses
v 0 relationships with staff
is* peer relationships
W the way treated in the program
is* contacts
i > work experiences

What problems has the trainee experienced?

t ^ work related
v 0 program related
personal
v 0 family, friends, world outside program

What are the trainee's plans for the future?

work pans
income expectations
v 0 lifestyle expectations/plans

What does the trainee think of the program?

v 0 strengths, weaknesses
things liked, things disliked
^ best components, poorest components
^ things that should be changed
346 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

full interview protocol before approving a ipants before they enter the program, when
dissertation proposal. The institutional re- they leave the program, and again some pe-
view board for protection of human subjects riod of time (e.g., six months) after they have
may insist on approving a structured inter- left the program. For example, a chemical
view, especially if the topic is controversial dependency program would ask partici-
or intrusive. In evaluations, key stake- pants about sobriety issues before, during,
holders may want to be sure that they know at the end of, and after the program. To com-
what program participants will be asked. In pare answers across these time periods, the
team research, standardized interviews en- same questions need to be asked in the same
sure consistency across interviewers. In way each time. Such interview questions are
multisite studies, structured interviews pro- written out in advance exactly the way they
vide comparability across sites. are to be asked during the interview. Careful
In participatory or coliaborative studies, consideration is given to the wording of each
inexperienced and nonresearcher interview- question before the interview. Any clarifica-
ers may be involved in the process, so stan- tions or elabora tions that are to be used are
dardized questions can compensate for vari- written into the interview itself. Probes are
ability in skills. Some evaluations rely on placed in the interview at appropriate places
volunteers to do interviewing; at other times to minimize interviewer effects by asking
program staff may be involved in doing the same question of each respondent,
some interviewing; and in still other in- thereby reducing the need for interviewer
stances interviewers may be novices, stu- judgment during the interview. The stan-
dents, or others who are not social scientists dardized open-ended interview also makes
or professional evaluators. When a number data analysis easier because it is possible to
of different interviewers are used, variations locate each respondent's answer to the same
in data created by differences among inter- question rather quickly and to organize
viewers will become particularly apparent if questions and answers that are similar.
an informal conversational approach to data In summary, there are four major reasons
gathering is used or even if each interviewer for using standardized open-ended inter-
uses a basic guide. The best way to guard views:
against variations among interviewers is to
carefully word questions in advance and 1. The exac t instrument used in the evalua-
train the interviewers not to deviate from the tion is available for inspection by those
prescribed forms. The data collected are still who will use the findings of the study.
open-ended, in the sense that the respon- 2. Variation among interviewers can be
dent supplies his or her own words, minimized where a number of different
thoughts, and insights in answering the interviewers must be used.
questions, but the precise wording of the
questions is determined ahead of time. 3. The interview is highly focused so that
When doing action research or conduct- interviewee time is used efficiently.
ing a program evaluation, it may only be 4. Analysis is facilitated by making re-
possible to interview participants once for a sponses easy to find and compare.
short, fixed time, such as a half hour, so
highly focused questions serve to establish In program evaluations, potential prob-
priorities for the interview. At other times, it lems of legitimacy and credibility for quali-
is possible and desirable to interview partic- tative data can make it politically wise to
Qualitative Intervieiving |J, 347

produce an exact interview form that the To illustrate the standardized open-
evaluator can show to primary decision ended interview, three interviews have been
makers and evaluation users. Moreover, reproduced in Appendix 7.2 at the end of
when generating a standardized form, eval- this chapter. These interviews were used to
uation users can participate more com- gather information from participants in an
pletely in writing the interview instrument. Outward Bound wilderness program for
They not only will know precisely what is disabled persons. The first interview was
going to be asked but, no less important, will conducted at the beginning of the program,
understand what is not going to be asked. the second interview was used at the end of
This reduces the likelihood of the data being the 10-day experience, and the third inter-
attacked later because certain questions view took place six months after the pro-
were missed or asked in the wrong way. By gram.
making it clear, in advance of data collection,
exactly what questions will be asked, the
limitations of the data canbe known and dis- Combining Approaches
cussed before evaluation data are gathered.
While the conversational and interview These contrasting interview strategies are
guide approaches permit greater flexibility by no means mutually exclusive.
and individualiza tion, these approaches A conversational strategy can be used
also open up the possibility, indeed, the like- within an interview guide approach, or you
lihood, that more information will be col- can combine a guide approach with a stan-
lected from some program participants than dardized format by specifying certain key
from others. Those using the findings may questions exactly as they must be asked
worry about how conclusions have been in- while leaving other items as topics to be ex-
fluenced by qualitative differences in the plored at the interviewer's discretion. This
depth and breadth of information received combined strategy offers the interviewer
from different people. flexibility in probing and in determining
In contrast, in fieldwork done for basic when it is appropriate to explore certain sub-
and applied research, the researcher will be jects in greater depth, or even to pose ques-
attempting to understand the holistic tions about new areas of inquiry that were
worldview of a group of people. Collecting not originally anticipated in the interview
the same information from each person instrumenfs development. A common com-
poses no credibility probiem when each per- bination strategy involves using a standard-
son is understood as a unique informant ized interview format in the early part of an
with a unique perspective. The political interview and then leaving the interviewer
credibility of consistent interview findings free to pursue any subjects of interest dur-
across respondents is less of an issue under ing the latter parts of the interview. An-
basic research conditions. other combination would include using the
The weakness of the standardized ap- informal conversational interview early in
proach is that it does not permit the inter- an evaluation project, foliowed midway
viewer to pursue topics or issues that were through by an interview guide, and then
not anticipated when the interview was closing the program evaluation with a stan-
written. Moreover, a structured interview dardized open-ended interview to get sys-
reduces the extent to which individual dif- tematic information from a sample of partic-
ferences and circumstances can be queried. ipants at the end of the program or when
348 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

conducting follow-up studies of partici- While the three strategies vary in the extent
pants. to which the wording and sequencing of
A sensitizing concept can provide the questions are predetermined, no variation
bridge across types of interviews. In doing exists in the principie that the response for-
follow-up interviews with recipients of Mac- mat should be open-ended. The interviewer
Arthur Foundation Fellowships, the sensi- never supplies and predetermines the
tizing concept "enabling," a concept cen- phrases or categories that must be used by
tral to the fellowship's purpose, allowed us respondents to express themselves as is the
to focus interviews on any ways in which case in fixed-response questionnaires. The
receiving the fellowship had enabled re- purpose of qualitative interviewing is to
cipients. "Enabling," or "being enabled," capture how those being interviewed view
broadly defined and open-ended, gave in- their world, to learn their terminology and
terviewees room to share a variety of experi- judgments, and to capture the complexities
ences and outcomes while also letting me of their individual perceptions and experi-
identify some carefully worded, standard- ences. This openness distinguishes qualita-
ized questions for ali interviewees, some in- tive interviewing from the closed question-
terview guide topics that might or might not naire or test used in quantitative studies.
be pursued, and a theme for staying cen- Such closed instruments force respondents
tered during completely open-ended con- to fit their knowledge, experiences, and feel-
versations at the end of the interviews. ings into the researcher's categories. The
fundamental principie of qualitative inter-
Summary of viewing is to provide a framework within
Interviewing Strategies which respondents can express their own un-
derstandings in their own terms.
Ali three qualitative approaches to inter- Exhibit 7.2 summarizes variations in in-
viewing share the commitment to ask genu- terview instrumentation. In reviewing this
inely open-ended questions that offer the summary table, keep in mind that these are
persons being interviewed the opportun- presented as pure types. In practice, any par-
ity to respond in their own words and to ticular study may employ ali or several of
express their own personal perspectives. these strategies together.

13. Question Options

f you ask me I'm gonna tell you.

Roseanne (2001:164)

Six kinds of questions can be asked of Experience and

people. On any given topic, it is possible to Behavior Questions
ask any of these questions. Distinguishing
types of questions forces the interviewer to Questions about what a person does or
be clear about what is being asked and helps has done aim to elicit behaviors, experi-
the interviewee respond appropriately. ences, actions, and activities that would
Variations in Interview Instrumentation

Type of Interview Characterstics Strengths Weaknesses

Informal conversational interview Questions emerge from the immediate context Increases the salience and relevance of Different information colected from different
and are asked in the natural course of things; questions; interviews are built on and emerge peopie with different questions. Less systematic
there is no predetermination of question topics from observations; the interview can be matched and comprehensive if certain questions do not
or wordng. to individuais and circumstances. arise naturally. Data organization and analysis
can be quite difficult.
Interview guide approach Topics and issues to be covered are specified The outline increases the comprehensiveness of Important and salient topics may be
in advance, in outline form; interviewer the data and makes data collection somewhat inadvertently omitted. Interviewer flexibility
decides sequence and wording of questions systematic for each respondent. Logical gaps in in sequencing and wording questions can
in the course of the interview. data can be anticipated and ciosed. Interviews result in substantially different responses
remain fairly conversational and situational. from different perspectives, thus reducing
the comparability of responses.
Standardized open-ended interview The exact wording and sequence of questions Respondents answer the same questions, thus Little flexibility in relating the interview to
are determined in advance. Ali interviewees increasing comparability of responses; data particular individuais and circumstances;
are asked the same basic questions in the are complete for each person on the topics standardized wording of questions may
same order. Questions are worded in a addressed in the interview. Reduces interviewer constrain and limit naturalness and relevance
completely open-ended format. effects and bias when several interviewers are of questions and answers.
used. Permits evaluation users to see and review
the instrumentation used in the evaluation.
Facilitates organization and analysis of the data.

Ciosed, fixed-response interview Questions and response categories are Data analysis is simple; responses can be directly Respondents must fit their experiences and
determined in advance. Responses are fixed; compared and easily aggregated; many questions feelings into the researcher's categories; may
respondent chooses from among these fixed can be asked in a short time. be perceived as impersonal, irrelevant, and
responses. mechanistic. Can distort what respondents
really mean or experienced by so completely
limiting their response choices.
350 LEI. QUALITATIVE DESIGNS AND DATA COLLECTION

have been observable had the observer been has not really been answered. Analytical, in-
present. "If I followed you through a typical terpretive, and opinion statements are not
day, what would I see you doing? What ex- answers to questions about feelings.
periences would I observe you having?" "If I This confusion sometimes occurs because
had been in the program with you, what interviewers give the wrong cues when ask-
would I have seen you doing?" ing questionsfor example, by asking opin-
ion questions using the format "How do you
feel about that?" instead of "What is your
Opinion and opinion about that?" or "What do you think
Values Questions about it?" When you want to understand the
respondents' emotional reactions, you have
Questions aimed at understanding the
to ask about and listen for feeling-level re-
cognitive and interpretive processes of peo-