Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
☐ Yes Continue appraisal
Does this evidence answer the EBP question?
☐ No STOP, do not continue evidence appraisal
Article Summary Information
Article Title: Enter text
Author(s): Enter text Number: Enter text
Population, size, and setting: Enter text Publication date: Enter text
Complete after appraisal
Evidence level and quality rating: Enter text
Study findings that help answer the EBP question: Enter text
Article Appraisal Workflow
Is this study:
☐ QuaNtitative (collection, analysis, and reporting of numerical data)
Numerical data (how many, how much, or how often) are used to formulate facts, uncover patterns, and generalize to
a larger population; provides observed effects of a program, problem, or condition. Common methods are polls,
surveys, observations, and reviews of records or documents. Data are analyzed using statistical tests.
Go to Section I for QuaNtitative leveling
☐ QuaLitative (collection, analysis, and reporting of narrative data)
Rich narrative data to gain a deep understanding of phenomena, meanings, perceptions, concepts, and experiences
from those experiencing it. Sample sizes are relatively small and determined by the point of redundancy when no new
information is gleaned, and key themes are reiterated (data saturation). Data are analyzed using thematic analysis.
Often a starting point for studies when little research exists; may use results to design empirical studies. Common
methods are focus groups, individual interviews (unstructured or semi-structured), and participation/observations.
Go to Section II for QuaLitative leveling
☐ Mixed methods (results reported both numerically and narratively)
A study design (a single study or series of studies) that uses rigorous procedures in collecting and analyzing both
quaNtitative and quaLitative data. Note: QuaNtitative survey designs with open-ended questions do not meet criteria
for mixed methods research because those questions are not approached using strict quaLitative methods. Mixed
methods studies provide a better understanding of research problems than using either a quaNtitative or quaLitative
approach alone.
Go to Section III for Mixed Methods leveling
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |1
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section I: QuaNtitative Appraisal
A Is this a report of a single research study? ☐ Yes Continue to decision tree
☐ No Go to Section I: B
Level I studies include randomized control
trials (RCTs) or experimental studies
Level II studies have some degree of
investigator control and some manipulation
of an independent variable but lack random
Level
assignment to groups and may not have a
control group
Level III studies lack manipulation of an
independent variable; can be descriptive,
comparative, or correlational; and often use
secondary data
After determining the level of evidence, determine the quality of evidence using the considerations below:
Does the researcher identify what is known and not known about the problem? ☐ Yes ☐ No
Does the researcher identify how the study will address any gaps in knowledge? ☐ Yes ☐ No
Was the purpose of the study clearly presented? ☐ Yes ☐ No
Was the literature review current (most sources within the past five years or a ☐ Yes ☐ No
seminal study)?
Was sample size sufficient based on study design and rationale? ☐ Yes ☐ No
If there is a control group:
Were the characteristics and/or demographics similar in both the control and ☐ Yes ☐ No ☐ N/A
intervention groups? ☐ Yes ☐ No ☐ N/A
Quality
If multiple settings were used, were the settings similar? ☐ Yes ☐ No ☐ N/A
Were all groups equally treated except for the intervention group(s)?
Are data collection methods described clearly? ☐ Yes ☐ No
Were the instruments reliable (Cronbach’s [alpha] > 0.70)? ☐ Yes ☐ No ☐ N/A
Was instrument validity discussed? ☐ Yes ☐ No ☐ N/A
If surveys or questionnaires were used, was the response rate > 25%? ☐ Yes ☐ No ☐ N/A
Were the results presented clearly? ☐ Yes ☐ No
If tables were presented, was the narrative consistent with the table content? ☐ Yes ☐ No ☐ N/A
Were study limitations identified and addressed? ☐ Yes ☐ No
Were conclusions based on results? ☐ Yes ☐ No
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |2
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section I: QuaNtitative Appraisal (continued)
Circle the appropriate quality rating below:
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control;
definitive conclusions; consistent recommendations based on comprehensive literature review that includes thorough
reference to scientific evidence.
Quality
B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control; fairly
definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review that
includes some reference to scientific evidence.
C Low quality: Little evidence with inconsistent results; insufficient sample size for the study design; conclusions
cannot be drawn.
Record findings that help answer the EBP question on page 1
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |3
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section I: QuaNtitative Appraisal (continued)
B Is this a summary of multiple
sources of research evidence?
☐ Yes Continue to decision tree
☐ No Use the Nonresearch Evidence Appraisal tool (Appendix F)
Level
After determining level of evidence, determine the quality of evidence using the considerations below:
Were the variables of interest clearly identified? ☐ Yes ☐ No
Was the search comprehensive and reproducible?
Key terms stated ☐ Yes ☐ No
Multiple databases searched and identified ☐ Yes ☐ No
Inclusion and exclusion criteria stated ☐ Yes ☐ No
Was there a flow diagram that included the number of studies eliminated at each level of
Quality
review? ☐ Yes ☐ No
Were details of included studies presented (design, sample, methods, results, outcomes,
strengths, and limitations)? ☐ Yes ☐ No
Were methods for appraising the strength of evidence (level and quality) described? ☐ Yes ☐ No
Were conclusions based on results?
Results were interpreted ☐ Yes ☐ No
Conclusions flowed logically from the research question, results, and interpretation ☐ Yes ☐ No
Did the systematic review include a section addressing limitations and how they were
addressed? ☐ Yes ☐ No
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |4
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section I: QuaNtitative Appraisal (continued)
Circle the appropriate quality rating below:
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control;
definitive conclusions; recommendations consistent with the study’s findings and include thorough reference to
scientific evidence
Quality
B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control; fairly
definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review
that includes some reference to scientific evidence
C Low quality: Little evidence with inconsistent results; insufficient sample size for the study design;
conclusions cannot be drawn.
Record findings that help answer the EBP question on page 1
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |5
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section II: QuaLitative Appraisal
A Is this a report of a single research study? ☐Yes This is Level III evidence
☐No Go to Section II: B
After determining level of evidence, determine the quality of evidence using the considerations below:
Was there a clearly identifiable and articulated:
Purpose? ☐ Yes ☐ No
Research question? ☐ Yes ☐ No
Justification for design and/or theoretical framework used? ☐ Yes ☐ No
Do participants have knowledge of the subject the researchers are trying to explore? ☐ Yes ☐ No
Were characteristics of study participants described? ☐ Yes ☐ No
Was a verification process used in every step of data analysis (e.g., triangulation, response
☐ Yes ☐ No
validation, independent double check, member checking)? (Credibility)
Does the researcher provide sufficient documentation of their thinking, decisions, and
methods related to the study allowing the reader to follow their decision-making (e.g., how ☐ Yes ☐ No
themes and categories were formulated)? (Confirmability)
Does the researcher provide an accurate and rich description of findings by providing the
information necessary to evaluate the analysis of data? (Fittingness) ☐ Yes ☐ No
Quality
Does the researcher acknowledge and/or address their own role and potential influence during
data collection? ☐ Yes ☐ No
Was sampling adequate, as evidenced by achieving data saturation? ☐ Yes ☐ No
Does the researcher provide illustrations from the data? ☐ Yes ☐ No
If yes, do the provided illustrations support conclusions? ☐ Yes ☐ No
Is there congruency between the findings and the data? ☐ Yes ☐ No
Is there congruency between the research methodology and:
The research question(s) ☐ Yes ☐ No
The methods to collect data ☐ Yes ☐ No
The interpretation of results ☐ Yes ☐ No
Are discussion and conclusions congruent with the purpose and objectives, and supported by
literature? ☐ Yes ☐ No
Are conclusions drawn based on the data collected (e.g., the product of the observations or
☐ Yes ☐ No
interviews)?
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |6
Johns Hopkins Evidence-Based Practice Model for Nursing and Healthcare Professionals
Research Evidence Appraisal Tool
Appendix E
Section II: QuaLitative Appraisal (continued)
Circle the appropriate quality rating below:
A/B High/Good Quality: The report discusses efforts to enhance or evaluate the quality of the data and the
overall inquiry in sufficient detail; it describes the specific techniques used to enhance the quality of the inquiry.
Evidence of at least half or all the following is found in the report:
Transparency: Describes how information was documented to justify decisions, how data were reviewed by
others, and how themes and categories were formulated.
Diligence: Reads and rereads data to check interpretations; seeks opportunity to find multiple sources to
Quality
corroborate evidence.
Verification: The process of checking, confirming, and ensuring methodologic coherence.
Self-reflection and self-scrutiny: Being continuously aware of how a researcher’s experiences, background, or
prejudices might shape and bias analysis and interpretations.
Participant-driven inquiry: Participants shape the scope and breadth of questions; analysis and interpretation
give voice to those who participated.
Insightful interpretation: Data and knowledge are linked in meaningful ways to relevant literature.
C Low quality: Lack of clarity and coherence of reporting, lack of transparency in reporting methods; poor
interpretation of data and offers little insight into the phenomena of interest; few, if any, of the features listed for
high/good quality.
Record findings that help answer the EBP question on page 1
©2022 Johns Hopkins Health System/Johns Hopkins School of Nursing Page |7
