Diane F.

Halpern
Critical Thinking Handout

Critical Thinking Workshop

for Helping our Students Become Better Thinkers

Background
Critical Thinking Can Be Improved With Instruction

One of the goals of education is to facilitate a student’s ability to think clearly about a
wide range of topics encountered in academic settings as well as everyday events. Daily issues
arising from our increasing dependence on science and technology as well as the pressures of
competing social and cultural influences require the use of data analysis, synthesis, and
evaluation. There seems to be unanimous agreement that we need to promote good thinking in
our students. Yet, there are still many schools that do not have explicit programs in thinking
skills instruction. In a review of research on instruction in high school programs, Marzano (1998)
found that programs that taught heuristics (i.e., thinking skills) had an overall effect size of d =
1.17!. This is an effect size that is so large that Cohen, the statistician who popularized the use of
effect size statistics claimed we do not need statistical tests. With effect sizes as large as this, we
need to move from do thinking skills programs “work” to how can we make them “work best?”

Moseley et al. (2005) worked with multiple teams of educators from all sectors in
education in England to classify thinking skills programs. They were charged with answering an
important question on behalf of the Learning and Skills Development Agency (LSDA) in
England. Based on their extensive review, would they recommend that all their post-16 schools
(which are similar to the community colleges in the United States) require critical thinking
coursework? They conducted an extensive classification and evaluation of all critical thinking
programs they could locate and produced several hundred pages of categorized reviews,
including empirical evidence of effectiveness. Following their careful review of the critical
thinking literature, they concluded that approaches that were skill-based are especially useful
because skill-based approaches had specific educational objectives, and thus are easier to assess
and communicate to students and other stakeholders in education. Critical thinking approaches
that are skill-based are also easier for the practice of teaching because the teachers can keep the
framework in mind as a guide during lessons and unexpected questions.

Thinking Critically About Critical Thinking:

Lessons Learned from Cognitive Psychology

A working definition of critical thinking:

Critical thinking is the use of those cognitive skills or strategies that increase
the probability of a desirable outcome. It is purposeful, reasoned, and goal
directed. It is the kind of thinking involved in solving problems, formulating
inferences, calculating likelihoods, and making decisions. Critical thinkers use
these skills appropriately, without prompting, and usually with conscious
Halpern Critical Thinking Handout Page 2

intent, in a variety of settings. That is, they are predisposed to think critically.
When we think critically, we are evaluating the outcomes of our thought
processes--how good a decision is or how well a problem is solved. Critical
thinking also involves evaluating the thinking process--the reasoning that went
into the conclusion we've arrived at or the kinds of factors considered in
making a decision.

Critical thinking is effortful, careful, consciously controlled processing that

maximizes the use of all available evidence and cognitive strategies, and
purposefully strives to overcome individual biases” (Riggio & Halpern, 2006)
(for reviews see Ennis, 1993; Halpern, 2013; Moseley et al. 2005; Sternberg,
Roediger, & Halpern, 2007)

Researchers generally agree that critical thinking is attempting to achieve a

desired outcome by thinking rationally in a goal-oriented fashion.

Instruction in critical thinking is predicated on two assumptions:

(1) that there are clearly identifiable and definable thinking skills which
students can be taught to recognize and apply appropriately, and (2) if
recognized and applied, the students will be more effective thinkers. Let me
give you some examples of what I mean by the term "thinking skills." A general
list of skills that would be applicable in almost any class would include:
understanding how cause is determined, recognizing and criticizing
assumptions, analyzing means-goals relationships, giving reasons to support a
conclusion, assessing degrees of likelihood and uncertainty, incorporating
isolated data into a wider framework, and using analogies to solve problems.

A Four-Part Model for Improving Critical Thinking

1. Explicitly teach the skills of critical thinking

2. Develop the disposition for effortful thinking and learning
3. Direct learning activities in ways that increase the probability of transcontextual
transfer (structure training)
4. Make metacognitive monitoring explicit and overt

On the following pages is a taxonomy of critical thinking skills. It is not meant

to be definitive or exhaustive--it is just a useful list that may help to clarify
some of underlying concepts.
A Taxonomy of Critical Thinking Skills
Halpern Critical Thinking Handout Page 3

1. Verbal Reasoning Skills

The skills listed under this rubric include those that are needed to comprehend and defend
against the persuasive techniques that are embedded in everyday language (also known
as natural language). Thinking and language are closely tied constructs, and the skills
included in this section recognize the reciprocal relationship between language and
thought in which an individual's thoughts determine the language used to express them,
and the language that is used shapes the thoughts.

2. Argument Analysis Skills

An argument is a set of statements with at least one conclusion and one reason that
supports the conclusion. In real life settings, arguments are complex with reasons that run
counter to the conclusion, stated and unstated assumptions, irrelevant information, and
intermediate steps. Arguments are found in commercials, political speeches, textbooks,
and anywhere else where reasons are presented in an attempt to get the reader or listener
to believe that the conclusion is true. The skills of identifying conclusions, rating the
quality of reasons, and determining the overall strength of an argument should be
sharpened in college course work.

3. Skills in Thinking as Hypothesis Testing

The rationale for this category is that much of our day-to-day thinking is like the scientific
method of hypothesis testing. In many of our everyday interactions, people function like
intuitive scientists in order to explain, predict, and control the events in their life. The skills
used in thinking as hypothesis testing are the same ones that are used in scientific
reasoning--the accumulation of observations, formulation of beliefs or hypotheses, and
then using the information collected to decide if it confirms or disconfirms the hypotheses.

4. Using Likelihood and Uncertainty

Because very few events in life can be known with certainty, the correct use of probability
and likelihood plays a critical role in almost every decision. Huff's (1954) tiny, popular
book How To Lie With Statistics is still widely quoted because it explains how easy it is to
mislead someone who does not understand basic concepts in probability. The critical
thinking skills that are subsumed under this heading are an important dimension of a
college-level critical thinking taxonomy.

5. Decision Making and Problem Solving Skills

In some sense, all of the critical thinking skills are used to make decisions and solve
problems, but the ones that are included here involve the generation and selection of
alternatives and judging among them. Many of these skills are especially useful in
quantitative reasoning problems.

Taken together these five categories define an organizational rubric for a skills approach
to critical thinking. They have face validity and can be easily communicated to the general
public and students and offer one possible answer to the question of what college students
need to know and be able to do to compete and cooperate in the world's marketplace and
function as effective citizens in a democratic society.
Halpern Critical Thinking Handout Page 4

An Example:

Sometimes, we unwittingly encourage the exact opposite of what we want students to

learn. Consider for example, many writing assignments where we ask students to take a
position on a controversial issue in psychology and then to write a position paper. This is exactly
the sort of learning activity that would strengthen the bias to seek and consider only confirming
evidence. Instead, we need to help student understand and be able to explain evidence on two
or more sides of an issue, such the evidence for and against evolutionary hypotheses of mate
selection or for and against the idea that parents are important influences on their children. We
want to teach students to gather and assess evidence to determine the best conclusion or
conclusions and not start with what they believe is true and consider only supporting evidence.
Here is an applied example that forces students to consider evidence that supports and fails to
support a hypothesis:
Making Arguments Worksheet

Example 1 Does violence on television really have a negative influence on children's

behavior?

1. State your conclusion. (although you may begin your formal writing here, but sure that
the conclusion follows from your reasons). As you work, this is the last part this is filled
in, not the first.

2. Give three reasons (or some other number) that support your conclusion.

a.

b.

c.

3. Rate each reason as weak, moderate, strong, or very strong.

Rating for a:

Rating for b:

Rating for c:

4. Give three counterarguments (or some other number) that weaken your conclusion.
Rate how much each counterargument weakens the conclusion: little, moderate, much,
or very much.

a.

Rating for a:

b.
Halpern Critical Thinking Handout Page 5

Rating for b:

c.

Rating for c:

5. List any qualifiers (limitations on the reasons for or against—for example some
evidence may be restricted to early childhood)

6. List any assumptions.

7. Are your reasons and counterarguments directly related to your conclusion?

8. What is the overall strength of your argument: weak, moderate, strong, or very strong?

Now that you have completed this worksheet, rate the overall strength of your argument.
Halpern Critical Thinking Handout Page 6

A generic fill-in-the-blank concept map that can be used to understand any theory in any domain
of knowledge. When you fill in the various “boxes”—for example, giving evidence for the
theory—you will be developing a “deep” knowledge of the theory. (Dansereau, 2001).
 Halpern Critical Thinking Handout Page 7

Guiding Thought-Provoking Questioning

(King, 1994)
Generic Questions SpecificThinking SkilIs Induced
What is a new example of. ..? Application

How could. ..be used to. ..? Application

What would happen if. ..? Prediction/hypothesizing

What are the implications of. ..? Analysis/inference

What are the strengths of . . .? Analysis/inference

What are the weaknesses of. ..? Analysis/inference

What is. ..analogous to? Identification and creation of analogies

and metaphors

What do we already know about.? Activation of prior knowledge

How does. ..affect. ..? Activation of relationship (cause-effect)

How does. ..tie in with what we Activation of relationship

learned before?

How does. ..apply to everyday life? Application

What is the counterargument for. ..? Different perspectives

Halpern Critical Thinking Handout Page 8

Another Example:
A complementary approach to critical thinking instruction is based on eliminating or reducing
common thinking fallacies. Consider, for example, this exercise in counterfactual reasoning that
has been used successfully to help college students overcome two cognitive biases--
"overconfidence" and "confirmation bias." These terms refer to the over-reliance on evidence
that confirms the beliefs that we hold. The original research on counterfactual reasoning was
conducted by Koriat, Lichtenstein, and Fischhoff (1980); It has been replicated and extended
many times. In a typical demonstration of overconfidence, people are asked a variety of
common knowledge questions. For example, "The Sabines were part of (a) ancient India or (b)
ancient Rome." Try it--answer the question by selecting either option a or option b. Okay, now,
judge how confident you are about the answer that you selected. Use 1.00 if you are absolutely
certain that you are correct, .5 if you just guessed, so your chance of being correct is 50%. Use
the numbers between .50 and 1.00 to assess how confident you are in your response.
Standard and often replicated results show that, in general, people are over confident in their
knowledge. Answers endorsed with absolute certainty, that is, people who believe they are
100% confident that they are correct, are, in fact, wrong 20% of the time. There is a strong
tendency for people tend to be confident that they are correct when they are often wrong.
Koriat, Lichtenstein, and Fischhoff (1980) have defined overconfidence as an unwarranted belief
in the correctness of one's answers. In a study designed to determine if people can improve in
their ability to assess how confident they should be when answering common knowledge
questions, the researchers added another condition. After selecting answers and judging their
confidence in their answers, subjects were told:

Spell out all the possible reasons that you can find favoring and opposing each of the
answers. Such reasons may include facts that you know, things that you vaguely
remember, assumptions that make you believe that one answer is likely to be correct or
incorrect, gut feelings, associations and the like.

Subjects were then instructed to write each reason in the appropriate cell of a 2 X 2 table
like the one shown below:

Reasons For Reasons Against

Answer A:

Answer B:
Halpern Critical Thinking Handout Page 9

Subjects were encouraged to put responses in all four cells and then to rate each reason on a
7-point scale according to how strongly it was for or against the corresponding answer (1.
weakest possible and 7. strongest possible). Readers should try this for themselves.
Results showed that after listing reasons and judging the strength of each reason,
college students became much more accurate in assessing the true probability that they were
correct. This is an interesting study for many reasons. First, the students did not receive any
feedback about the correctness of their responses--they became more accurate in knowing
what they know and don't know by focusing on information that they had stored in memory and
considering the strength of the evidence that they had. In other words, the improvement
resulted from their own actions.
There is a long history in cognitive psychology showing that “the head remembers what
is it does.” As teachers, what we ask students to do with information to be learned will affect
what is learned and how and when the information will be retrieved at some time in the future.

None of this is easy, but helping our students improve in their ability to think
clearly is the most important goal that we will ever have. Whenever you feel
discouraged, recall Hitler's now infamous quote, "What luck for rulers that men do not
think."

About the Halpern Critical Thinking Assessment (HCTA)

The HCTA uses believable examples with an open-ended response, followed by specific
questions that probe for the reasoning behind an answer. If, for example, the thinking skill of
understanding and recognizing the distinction between correlation and cause were being
assessed, it would be tested with examples taken from medical research (e.g., coffee drinkers
reported more headaches), social policy analysis (e.g., welfare mothers who received job training
were more likely to be employed after one year than welfare mothers who did not receive job
training), and numerous other believable scenarios. Such materials are ecologically valid in that
they are representative of the many examples that could be found in newspapers and everyday
discussions. The open-ended portion of the question allows test-takers to demonstrate whether or
not they spontaneously use the skill. For example, do the test-takers recognize that these are
correlational designs that do not permit causal statements when there are no obvious hints to
consider the issue of design?
Specific probes in the form of alternatives for the forced-choice questions follow the
open-ended responses. These probes allow test takers to demonstrate their understanding of the
concepts, such as whether they are able to recognize the problem of determining cause when they
are provided with hints. (e.g., In understanding these results, is it important to know if the
participants were assigned randomly to the different conditions?) These additional probes show
if test-takers are able to use the skill when they are told that it is needed in a specific situation,
even if the test-taker did not spontaneously recognize that it was needed.
A good critical thinking question with several sequential parts allows for different types
of information about the test taker with a minimal number of questions. The open-ended or
constructed response portion is a test of the type of remembering that cognitive psychologists
call “free recall” because there are few restraints on the type of response that the test-taker can
generate. The multiple-choice or multiple-rating portion shows whether the respondents were
Halpern Critical Thinking Handout Page 10

able to recognize the appropriate skill when it was presented in a list of alternatives. This sort of
response is a measure of “recognition memory.” Cognitive psychologists distinguish between
recall and recognition paradigms in the assessment of memory. It is believed that these two
types of recollection utilize different cognitive processes and that students prepare differently for
tests that require free recall (e.g., essay tests) than those that require recognition (e.g., multiple
choice tests). In general, lower scores are expected on free recall tests than recognition tests
because free recall requires a search through memory plus some sort of verification that the
answer retrieved is correct; recognition requires only the verification stage and provides a less
stringent measure of memory. For more information contact [email protected]

In one research project, Marin and Halpern (2011) provided critical thinking instruction for high
school seniors in very low performing schools. Here are some samples of our materials:

Why think about thinking?

• TMI - daily information overload requires

organization and evaluation.
• Consumer society – tv, radio, movies,
advertisements persuade consumers to buy and
believe certain things.
• Decision-making – students & parents have
many important decisions ahead.
• Problem-solving – Employers need people with
problem-solving skills, not only knowledge.
Halpern Critical Thinking Handout 11

During this and the following sessions, we’ll

learn & think about thinking…
More goals…
Some of our goals are to help you: • Identify how reasons can be good or bad.
• Be more aware of your thinking. • Look for hidden assumptions.
• Develop a critical thinking approach to the • Recognize that if two things are related
things you see & hear in the media, doesn’t mean one causes the other.
including music and tv, radio,magazines. • Teach you to use a decision making tool
• Think carefully about arguments people so you can make decisions you can live
use to persuade you. with!

We’ll begin with this question… In critical thinking terms,

• An argument is not
• How do you define
a disagreement!
an
An argument is a set
ARGUMENT? of statements that
contains at least one
conclusion and one
reason for believing
that the conclusion is
true.
Halpern Critical Thinking Handout 12

Good thinking requires Persuasion – how your

Information & Evidence thinking can be influenced
• Facts • A healthy skepticism • What are fallacies? • Four Fallacies
is good---ask – Poor reasoning
questions to help you – Errors in argument – Guilt/virtue by
• Opinions
evaluate the association
evidence
• Arguments based on – Popularity
reason, not emotion
– Weak Analogies

– Tradition

Before we begin…
• Use the paper and
pencil provided to
Welcome to Session Three Draw A Scientist.
Thinking About Thinking
• We will collect these
What are Mental Models? later.
Halpern Critical Thinking Handout 13

Why think about what we’re

What are Mental Models?
thinking?
Mental models are ways of thinking that • Assumptions may be wrong.
become automatic --- we don’t stop to • Beliefs may be inaccurate.
think about our thinking. • Prejudices may be unfair or unjust.
• Faulty thinking can occur.
• Assumptions • Faulty thinking = Problems!
• Beliefs
• Prejudices
Let’s check out a common mental model…

How about this: If so, you’re not alone!

• Before we started, • When asked to “Draw-

you were asked to A-Scientist”, most
people (children,
draw a picture of a
teenagers & adults)
scientist. draw:
• White men
• What did you draw? • White lab coats
• Bushy eyebrows!
• Glasses
• Something like this?
• Wild Hair


Drawing From Breaking Science Stereotypes, by Bodzin & Gehringer, Science & Children, January 2001
Halpern Critical Thinking Handout 14

Decision Making…
• Takes effort.

Session # 4: • It’s not simple! There’s

usually no simple
Making Decisions either/or.

• It involves several
The Worksheet Method alternatives.

• Each alternative also has

considerations.

Mapping a Decision 1. Defining the Problem

• For example, your
PROBLEM parents want you to be
more productive during
ALTERNATIVE ALTERNATIVE ALTERNATIVE the hours after school.
• Here’s the problem:
consideration consideration consideration consideration
“What to do in the
hours after school?’”
consideration consideration
Halpern Critical Thinking Handout 15

But wait! Are all of these equally good

2. What are some alternatives? alternatives?
Here are some possible • Probably not.
alternatives: Alternatives, the
1. Get a part-time possible actions, are
job. not created equal.
2. Participate in
academic club. Some alternatives
3. Sports. have more going for
4. Volunteer at them than
the local others…let’s take a
library. closer look…

References: For additional examples and references see

Halpern, D. F. (2013). Thought and Knowledge: An Introduction to Critical Thinking (5th

ed.). New York, NY: Psychology Press.

Marin, L., & Halpern, D. F. (2011). Pedagogy for developing critical thinking in adolescents:
Explicit instruction produces greatest gains. Thinking Skills and Creativity, 6, 1-13. Doi:
10.1016/j.tsc.2010.08.002

Butler, H. A., & Halpern, D. F. (2011). Critical Thinking. Oxford Bibliographies Online. New
York, N.Y.: Oxford University Press. Available from OBO website:
http://aboutobo.com/

Halpern, D. F. (2006). The nature and nurture of critical thinking. In R. Sternberg, R. Roediger,
& D. F. Halpern (Eds.). Critical Thinking in Psychology (pp. 1-14). Cambridge, MA:
Cambridge University Press.

Riggio, H. R. & Halpern, D. F. (2006). Understanding Human Thought: Educating Students as

Critical Thinkers. In W. Buskist & S. F. Davis (Eds.), Handbook of the Teaching of
Psychology (pp.78-84). Malden, MA: Blackwell Publishing.

Butler, H. A., (2012). Halpern Critical Thinking Assessment predicts real-world outcomes of
critical thinking. Applied Cognitive Psychology, doi: 10.1002/acp.2851
Halpern Critical Thinking Handout 16