1
Program Evaluation
Dissertation Outline
Chapter 1: Introduction
Clear statement of the research problem
Evidence that supports the existence of the problem
Probable causes related to the problem
Brief description of the need for an evaluation approach
Descriptive information related to the organization where the program is
located
Logic model- program theory or theory of action
Program description: goals, objectives and scope of the activities
Description of the stakeholders and relevant audience associated with the
evaluation
Purpose of the evaluation, including its significance- identify and define
the concepts and ideas that are central to the study
Research questions- specific, feasible and measurable
Definition of Terms
Chapter 2: Literature Review
Discussion of the elements of the program, including the logic model
Synthesis and summary of the findings relevant to the elements of the
program
Exploration of the gaps or deficiencies in prior research and discussion of
how future studies should replicate, extend and/or differ from past
research
2
Identify critical variables in the proposed evaluation
Chapter 3: Methodology
Choice of evaluation model, including a discussion of how the measures
are grounded in program evaluation literature and a rationale for the
selection of the measures
Development and design of evaluation instruments
Data collection: detailed discussion of study participants, selection
process, procedures for data collection, validity and reliability, and other
pertinent information (e.g., subscales, number of items in each scale)
Costs/benefit analysis of the design with a focus on accuracy, validity and
reliability of study instruments
Data analysis: include descriptive and inferential statistics as appropriate
for quantitative studies and analysis procedures for qualitative studies
Professional Evaluation Standards: identification and discussion of
adherence to the standards. Include a discussion of safeguards put in place
to insure confidentiality during data collection, analysis and reporting of
results. Provide a description of the IRB process.
Provide a study timeline to include the development of the proposal,
research instruments, proposal defense, IRB process, pilot study, data
collection, data clean up, data analysis and interpretation of results
Discuss limitations, delimitations and assumptions
Chapter 4: Results
Summary findings
Results presented in sequence relative to each research question
3
Include relevant tables and figures
Findings are reported without rationale or discussion
Chapter 5: Executive Summary
Introductory paragraph providing an overview of the study
Expansion of results relative to each research question
Link findings to relevant research
Discussion of implications of the findings for practice
Recommendations for the specific program as related to the evaluation
Recommendations future evaluation and research
Appendices: May include
Instruments (questionnaires, interview protocols, observation instruments,
consent forms)
Site approval letter and IRB approval
Redacted information from study site
4
CHAPTER 1
In this section, you will introduce your readers to the issue you are exploring.
Be sure to make your first sentence a compelling “hook.” State the major thesis that
guides your study.
Problem Statement
In this section you will provide a concise statement of the problem (one or
two paragraphs).
Statement of Context for EdD
Here you will make a case for the proposed relationship between the
variables under study.
Example of Level Three Heading
It may be helpful to provide your readers with subheading to guide them
through your argument. This will be even more important in Chapter 2 than in
Chapter 1.
Section Headings
If you divide your section using subheading, be sure to include at least two at
any given level.
Research Questions
After a brief introduction, list your questions in a numbered list. These
research questions, worded in exactly the same way, will be repeated at the
beginning of Chapter 3.
Significance of the Study
5
Here, you will make a case for why the problem you have proposed to study
is both interesting (contestable) and important (negative consequence will be
avoided or positive consequences will ensue with the information that may be
gained from your study).
Definitions of Terms
Provide your readers with constitutive definitions of any specialized terms
that you use, especially those that may have multiple definitions in the literature.
You may quote specific authors whose work you may be following. This is not the
place to discuss operational definitions (i.e., measures of your constructs). These
will be provided in Chapter 3.
6
CHAPTER 3
You’ll probably start with a brief introduction, stating once again the purpose
of your study.
Research Questions
Provide a brief introduction linking what you presented in Chapter 2 to your
research questions. Then list your research questions exactly as they were listed in
Chapter 1.
Method
You should avoid “stacking” headings one of top of the other with no
intervening text. Describe the nature of your study (e.g., mixed methods, quasi-
experimental, program evaluation, etc.)
Participants
Tell who you plan to invite to participate in your study. Be as specific as you
can at this point.
Data Sources
Describe the nature of the data you will use to answer your research
questions. Again, avoid stacking headings.
Measure one. Describe your survey in detail, including the number of items
in each section, the response scale, any available validity and reliability information,
as well one or two sample items.
Measure two. Provide the same information for each measure you will use
in your study, including extant student achievement data from SOLs.
7
Interview protocol. If your study is qualitative in nature, you will describe
your interview protocol for individual or focus group interviews, document or other
artifact analysis, and any other data sources that will be part of your study.
Data Collection
Describe how you will recruit participants to your study, including any
incentives you will offer. Be specific about your data collection plan.
Data Analysis
Describe specifically how you plan to analyze your data. It is helpful to
provide a table that lists each research question along with the data sources and the
data analysis that you plan to use to answer each question. See Table 1.
Table 1
Research Question Data Sources Data Analysis
Question 1
Question 2
Question 3
Question 4
Ethical Considerations
Describe the procedures you will follow to
secure appropriate permissions to conduct your study through the W&M Education
Institutional Review Board (IRB) and to protect your participants from any potential
harm.
8
Assumptions, Delimitations, and Limitations
Describe the underlying assumptions of the research method or approach
that you are selected for your study. The delimitations and limitations sections of
your research proposal describe situations and circumstances that may affect or
restrict your methods and analysis of research data.
Delimitations define the parameters of the investigation. In educational
research the delimitations will frequently deal with such items as
population/sample, treatment(s), setting, and instrumentation. Delimitations are
choices made by the researcher which should be mentioned. They describe the
boundaries that you have set for the study. Limit your delimitations to the things
that a reader might reasonably expect you to do but that you, for clearly explained
reasons, have decided not to do.
Limitations are influences that the researcher cannot control. They are the
shortcomings, conditions or influences that cannot be controlled by the researcher
that place restrictions on your methods and conclusions. Any limitations that might
influence the results should be mentioned. You might consider limitations affecting
the instruments you utilized, the sample, your analysis, the nature of self-reporting,
time constraints, and limitations to the generalizability of the findings.
