Book 13
Book 13
Randolph A. Smith
Lamar University
Assessing Student Lear ning
ESSENTIALS OF EFFECTIVE TEACHING SERIES
Although many faculty truly enjoy teaching, one of the necessary evils of teaching
is assessing student learning and assigning grades. I have often heard faculty remark
that teaching is great except for the grading aspect. Based on the number of student
complaints about grades that faculty receive, it is doubtful that students are any
more positive about the grading aspect of school than are faculty. Nevertheless, it
is an imperative for faculty to assess student learning and give grades on the basis
of those assessments. In this booklet, I focus on the process of assessing student
learning and not on grading-I believe that the better job we as faculty do at assessing
student learning, the less onerous the grading process becomes.
There are several important decisions faculty members must make as they plan
assessments of their students’ learning. Although there is no prescribed order in
which faculty must make these decisions, I will cover them in the order that seems
like a logical starting point to end point.
5
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
at the end of a learning process-a class unit, a project, an entire course. Because
ESSENTIALS OF EFFECTIVE TEACHING SERIES
of this temporal arrangement, students may not benefit much in terms of their
learning from summative assessments. The ultimate example of not benefitting
from a summative assessment is a final exam in a course. At most, students
may find out their grade on the exam but rarely get any feedback other than that
grade. Depending on when a project such as a paper is due and returned (if
at all), students may also not get any feedback that will benefit their learning.
Although students can probably expect to get some feedback from exams that
they take early in a semester, the amount of feedback can vary tremendously, as
does the degree of attention that students pay to that feedback. Thus, one of the
major problems with summative assessment is that the feedback process may not
benefit student learning very much, if at all.
On the other hand, formative assessment is designed solely for the purpose of
providing students with feedback. For example, an instructor might give students
daily quizzes over the material but not use performance on the quizzes as part of the
grade for the course. In this manner, students are getting formative feedback about
how well they know the material as they go through the course, but the assessment
is not summative because it does not contribute toward their grade. Students can
use information from formative assessments to get feedback about how well their
approach toward the course (e.g., reading, studying, processing the material) is
going. If the formative feedback is not good, students can alter their approach to
the course before they take a summative assessment that counts toward the grade.
By the same token, faculty can use the results from formative assessments to alter
the way they are approaching the course (Cangelosi, 2000). For example, if I am
teaching a course and use a formative assessment process, I might find out that
all the students perform very poorly on a specific topic. Based on this feedback, I
might decide that I want to alter my teaching approach in hopes of helping students
learn more or better. In contrast, I might decide that my assessment instrument was
too difficult or did not match well with the manner in which I was teaching. Finally, I
might determine that students did not take the formative assessment measurement
very seriously and that I do not need to change anything about how I am teaching.
This last conclusion might be appropriate if some of the students performed well on
the formative measure and others did not. That information would tell me that the
assessment measure I used was not unreasonably difficult given that some students
performed well on it. I might want to conduct further diagnosis with the students who
performed poorly to determine if I can learn why they performed so poorly.
6
Assessing Student Lear ning
7
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
Faculty members must make determinations about the relative weight of the
assessments they use in classes. The fewer summative assessments used, the more
high stakes they become. The larger the number of summative assessments used,
the more they move toward lower stakes. However, it is difficult to conceive of an
exam or a term paper as ever being considered low stakes by the students who have
to complete the assignment. Related to this issue, research studies show both higher
student preference for and better performance with more frequent testing (Abbott
& Falstrom, 1977; Bangert-Drowns, Kulik, & Kulik, 1991; Peckham & Roe, 1977).
These results seem to show that students prefer low-stakes rather than high-stakes
assessment so that each assessment has less weight in determining the final grade.
Three key points from Smith’s (2011) booklet are worth a reminder at this point.
First, your learning outcomes must follow from your learning goals. Remember that
learning goals can be broad and general, but learning outcomes must be specific.
Second, your student learning outcomes must be stated in measurable terms. You
cannot develop an assessment of student learning on something that you cannot
8
Assessing Student Lear ning
measure. Third, learning outcomes are typically linked to a level or type of student
learning. Bloom’s (1956) taxonomy is the most common example of different levels of
student learning, but there is also an updated version (Anderson & Krathwohl, 2001)
as well as a digital version of Bloom’s taxonomy (Churches, 2008). Remember that
the level at which you have students learn information determines the level at which
you can assess that information. If you have asked students only to memorize a list
of famous people in your discipline, then you should not expect the students to be
able to analyze the work of one of the people on the list or to compare and contrast
the work of two people on the list. It is important to remember that the different
levels of learning are represented by different verbs-verbs that should appear in your
learning objectives. Thus, you must assess in a manner that is consistent with the
verbs you used in your learning objectives.
There are many different types of assessments; one type of classification scheme
that divides them into two groups is objective versus nonobjective assessments (Suskie,
2009). Although Suskie (2009) used the term “subjective” rather than nonobjective, I
prefer to avoid that term because it implies possible favoritism or bias in scoring.
An objective assessment is essentially any test that can be computer scored, such
as multiple-choice, true-false, or matching questions, plus fill-in-the-blank questions.
Objective assessments have right or wrong answers such that each question is
scored on an all-or-none basis; partial credit for objective items is rare or nonexistent.
In contrast, nonobjective assessments involve tasks such as writing, research, or
some other task completion. However, as Suskie noted, it is not correct to associate
objective assessments as being quantitative and nonobjective assessments as being
qualitative. Many nonobjective assessments yield quantitative data.
It is important that you match the type of assessment to your learning objectives.
Objective assessments tend to be most useful for measuring student performance
at the lower levels of Bloom’s (1956) or other taxonomies. Therefore, if you are
introducing your students to large amounts of material they have never before
encountered, objective assessments such as multiple-choice or true-false tests allow
you to quickly and easily assess how much material they have learned.
9
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
book or research reports, or projects that require them to use material they have
ESSENTIALS OF EFFECTIVE TEACHING SERIES
previously learned in new and unique combinations, just to name a few possibilities.
Table 1
Examples of Learner-Centered Assessments from Angelo and Cross (1993)
Minute Paper
A few minutes before class ends, stop and ask students to write answers to
two questions: “What was the most important thing you learned during this class”
“What important question remains unanswered?” (Angelo & Cross, 1993, p. 148)
Not only does the first question assess recall, but it also forces students to
evaluate the information they received in class. The recall aspect, of course, allows
the instructor to determine whether students are understanding the information
correctly or if they have misconceptions. The second question provides the
instructor a good place to begin teaching at the next class meeting.
10
Assessing Student Lear ning
Muddiest Point
Ask students “What was the muddiest point in -?” (Angelo & Cross, 1993, p.
154). You can fill in the blank with a variety of stimuli: class, lecture, chapter in
text, film, assignment, and so on. This learner-centered assessment provides the
instructor with feedback about what students have found or are finding difficult
to learn. Armed with this information, instructors have a much better idea of what
information to emphasize in class rather than by guessing.
Directed Paraphrasing
The instructor asks students to put some important concept into their own
words, usually directed at a specific audience or for a specific purpose and
typically avoiding the professional jargon of the academic discipline. This
learner-centered assessment makes students go beyond simple memorization
and regurgitating information on an exam. If students do not truly understand the
material, they will have a difficult time rephrasing it.
In plain language and in less than five minutes, paraphrase what you have
read about computer viruses-such as the Michelangelo virus-for a vice president
of a large insurance firm who is ultimately responsible for database security. Your
aim is to convince her to spend time and money “revaccinating” thousands of
workstations. (Angelo & Cross, 1993, p. 233)
Suskie (2009, p. 167) recommended beginning to write test items based on a “test
11
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
blueprint”. A test blueprint is simply an outline for your test that includes all the learning
ESSENTIALS OF EFFECTIVE TEACHING SERIES
objectives that you want students to know for the test. Using a test blueprint ensures
that you will not forget to include an important objective that you meant to cover on
the test. Likewise, using a test blueprint would let you know if you were writing test
items over material that you did not consider as important as your actual learning
objectives. Anyone who has compiled many tests during a teaching career without
using a test blueprint knows how easy it is to make one of these mistakes. Using a test
blueprint will help you allocate items on the test in terms of their importance to your
learning objectives rather than being tied to the textbook. For example, perhaps you
consider the opening chapter of your textbook to contain less important information
than Chapters 2 and 3, which you are also including on your first exam. Based on
your learning objectives and a test blueprint, you will include more exam questions
from Chapters 2 and 3 than from Chapter 1. Without such guidance, however, you
might develop a test that has 10 items from each chapter-in other words, a test that
does not match your learning objectives very well.
Finally, using a test blueprint can help you avoid a student criticism that some
exams evoke: “This test doesn’t seem to cover what we learned in class”. As long
as you have communicated your learning objectives to your students, have taught
information related to your learning objectives, and have followed your learning
objectives in constructing the exam, there should not be a mismatch.
Suskie (2009, pp. 170-171) and Hales and Marshall (2004, pp. 65-88) presented a
list of tips for writing good multiple-choice items compiled from testing experts and many
studies. Bear in mind, however, Suskie pointed out that it is difficult to follow all guidelines
without ever violating some of them. Thus, you should look at these ideas to see how
your multiple-choice questions stack up. If you find yourself to be a frequent violator of
any of these recommendations, it would be wise to try to reduce your violations.
• Use vocabulary that is as simple as possible (unless you are testing for vocabulary)-
all students should have an equal chance on items regardless of language skill.
• Avoid using questions that are interrelated-a student should not be able to
use information from one question to answer another question, nor should
missing one item cause a student to automatically miss another item.
12
Assessing Student Lear ning
• Make the item stem a complete question-students should not have to read
the answer options to understand the question being asked.
• Try to avoid negatives in the stem; if you must use such items, emphasize
the negative word (e.g., NOT, not, not)-anxious students may read over the
negative word and not see it.
• Make sure all options are grammatically correct-avoid giving clues to correct
answers through grammar.
• It is not necessary to have the same number of options for every question-if
there are only three plausible options (e.g., “goes up,” “goes down,” “stays
the same”), then use only those.
• Arrange the answer options logically-if there is a logical order to the options
(e.g., numbers that increase or decrease, arrange words alphabetically), use
it to make it easier for students to locate the correct answer if they know it.
• Avoid the “none of the above” option-a student may know incorrect answers
but not the correct answer; if you do use it, use it more than once as both
incorrect and correct alternatives.
• Avoid the “all of the above” option-it can penalize slow readers and students who
select a correct option without reading further; it can reward a student for incomplete
understanding (if 2 options are correct, “all of the above” must be the answer).
• Good distractors let you know where students went wrong-create an incorrect
answer to match each type of error a student could make.
This is a long list of recommendations-one reason for its length is that multiple-
choice questions are one of the most frequently used test item format. Educational
13
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
researchers have devoted a great deal of time and effort to studying this format.
ESSENTIALS OF EFFECTIVE TEACHING SERIES
Although the list is long, you should remember that it is difficult to follow all of these
guidelines all the time. Maximizing good testing practice and minimizing difficult or
confusing items are the goals you should strive to attain.
One reason that faculty may turn to using essay questions for assessment is that
they can write essay questions in a minimum amount of time, particularly compared to
good multiple-choice questions. However, as many faculty have later discovered, an
essay question written in haste may turn out to be quite difficult for students to answer
and/or hard to grade easily or well. One way to avoid these problems is by following
guidelines developed to facilitate writing good essay questions. Hales and Marshall
(2004, pp. 159-165) listed nine guidelines for developing quality essay items.
• Define the problem clearly-you want all your students responding to the
same essay question; if they have to interpret it, you will likely get multiple
interpretations.
• Keep the problem limited-although essay questions are usually broader than
multiple-choice questions, a question that is too broad tends to overwhelm
students because of a lack of direction.
• Give explicit directions-avoid making the students guess “what does the
teacher want?” If you want examples, complete sentences, a graph, or have
some other specific requirement, say so.
14
Assessing Student Lear ning
questions; the instructor might react more favorably to some questions than
to others. Also, if students know they will have a choice of essays, they may
choose not to learn some of the material.
• Make a scoring guide for each question-even if you do not write it, you should
have a “best answer” in mind. Preparing a scoring guide will minimize scoring
errors and increase the reliability of grading.
A rubric is a standardized scoring guide for an essay answer. Rubrics are often
used when there are different people scoring the same question over many students-
for example, scoring the free response section of Advanced Placement exams in the
US, which can have hundreds of graders scoring thousands of student responses.
A team of experienced question readers develops a rubric and then trains other
question readers in using the rubric so that all students’ essays are scored using the
same criteria, even though there are many different readers.
To create a rubric, you must first decide all the different dimensions or elements on
which you want to score an essay answer (or any student performance such as a term
paper, classroom presentation, etc.). For example, Hales and Marshall (2004, p. 203)
gave an example rubric used for grading an essay for an English composition class.
The instructor decided to score students on their Ideas and Content, Organization,
Voice, Word Choice, Sentence Fluency, and Convention (e.g., spelling, punctuation).
At that point, the instructor had to decide whether to count those six criteria equally or
weight them differentially-because it was a composition course, the instructor counted
the criteria equally. Finally-and this point is key in building a rubric-the instructor must
develop a point system for rating each of the criteria and give a verbal example. In
this example, the instructor decided to use a 3-point rating scale for each criterion.
For the Ideas and Content criterion, the instructor assigned 3 points for essays
with “Clear main theme; strong ideas; high-level detail” (Hales & Marshall, 2004, p.
203). An essay judged to have “A discernable main theme not clearly articulated;
insufficient detail” (Hales & Marshall, 2004, p. 203) received 2 points for Ideas and
Content. Finally, a student received 1 point for the Ideas and Content criterion if the
15
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
instructor found “No main theme; little detail” (Hales & Marshall, 2004, p. 203). In this
ESSENTIALS OF EFFECTIVE TEACHING SERIES
manner, the instructor could grade each student’s essay on these six criteria, with
each criterion being scored on a 1-3 basis. Thus, an excellent essay would receive a
score of 18 (6 criteria x 3 points each), whereas a very weak essay would be scored
with a 6 (6 criteria x 1 point each). Using a rubric makes the scoring a simple matter
for the instructor. By giving each student a copy of the rubric with marked scores,
the instructor is able to provide much more detailed feedback than simply giving an
overall grade. Both of these outcomes are ideal as far as assessment is concerned.
Summary
Clearly, assessing student learning well is much more involved than some faculty
and (probably) most students believe. However, given the crucial nature of faculty
members’ jobs of assessing learning, it is important to take the time necessary to do
a good job. Note how much of this booklet has been devoted to important aspects
of assessment before actually writing the assessment instrument. I hope that this
pattern sufficiently emphasizes the importance of being well prepared to assess
student learning before you write an exam.
In closing, the important points to take away from this booklet include the following:
16
Assessing Student Lear ning
• An objective assessment is essentially any test that can be computer scored, such as
multiple-choice, true-false, or matching questions, plus fill-in-the-blank questions.
• Research has documented best practices for writing both objective items and
nonobjective items for assessment.
References
- Abbott, R. D., & Falstrom, P. (1977). Frequent testing and personalized
systems of instruction. Contemporary Educational Psychology, 2, 251-257.
- Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C.-L. C. (1991). Effects of frequent
classroom testing. Journal of Educational Research, 85, 89-99.
- Students who were frequently tested scored about 0.1 standard deviations
17
Deanship of Skills Development ...
Distinguished Achievement and Commitment to Development
Assessing Student Lear ning
- Peckham, P. D., & Roe, M. D. (1977). The effects of frequent testing. Journal
of Research & Development in Education, 10, 40-50.
- This booklet in this series examines the practice and advantages of writing
student learning objectives.
18
King Saud University, 2013
King Fahd National Library Cataloging-in- Publlcation Data