0% found this document useful (0 votes)
304 views26 pagesBrown & Abeywickrama (2010) - Assessing Grammar
Uploaded by
Joaquín EgurenCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
304 views26 pagesBrown & Abeywickrama (2010) - Assessing Grammar
Uploaded by
Joaquín EgurenCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
_cHAPTER | J
ASSESSING GRAMMAR.
AND VOCABULARY —___—
292
OBJECTIVES: After reading this chapter, you will be able to
+ state a rationale for treating + analyze the ae of lexical
form-focused assessment (of ability and ais ly d mt ‘0 the
grammar and vocabulary) as a —a of vecabulary
criterion that differs in purpose knowledge kh
and context from assessing one or _ + develop assessments that focus
several of the four skills ‘on specifically identified forms
* discern, through the backdrop of of language
the nature of grammar, the + design assessments that target one
purposes and contexts for or several of the modes of
assessing grammatical knowledge performance, ranging from
perceptive recognition of forms to
extensive reading
You have now had ample opportunities to examine many different assessment tech.
niques in cach of the four skills of listening, speaking, reading, and writing, Before
that, we surveyed some of the so-called alternatives in assessment that are com
monly used in language programs. Your head may be spinning with all the options
that are possible to bring to bear to your language classroom. From intensive focus
on the“bits and pieces” of language to extensive skills like listening to speeches and
reading books, and from portfolios to selfassessment, the possibilities are limitless.
There is one more dominant “hot topic” in this field that we now turn to
ssing grammar and vocabulary, more technically known as form-focused
assessment. One could argue that we've already discussed syntactic and lexical
forms in the process of covering the four skills, Afterall Imost all of the microskills
that you've been reading about in the Previous four chapters are abilities that
require a learner to focus on form. In assessing speaking, for example, you might
Es pie = ne snntents production of phonemes, stress patterns, or verb bcs
such a focus compres ana rocabul ty. All of this is formfocused assessment i
in the same way that Tormiocuse ee mecumate Soo eine nun a
F main ed instruction is also an integral aspect of comm
nleative languagetcaching methodology,
Scanned with CamScanner cuaniee 1
1 Assessing Grammar and vi
and Vocabutar
lary 293
ow we know tha dhe langwageteaching Held hi
aps Nowearies—with language forms, especially ‘gram been consumed—over per
Pepe, stancadized (ets ecient ener eS cro
M se in perfect harmony with classes and vcencnin he ane ‘on form, all of
oe Had aspects of language. We Tae eon tiar inienae iearees aero
ir infamous—for spending months or pe eee
mora anguages in the form ofits grammar roles, but gain bea an ec
are ability 10 se the language. However, this Fe eae eet el
Ae pradigms is not to be confused with an informed aires
mes arom & reasonable intermingling of cs 9» See ee
(eat i, Seotmunication for real-world pragmatic uses) cus on micanies
ao Siehin a paradigm of communicative methodology, what does i
vqgow" the grammar, vocabulary: phonology, and discourse rules ae
pow does that knowledge—whether explicit or implici influence the Paap
Teamner to use Kanguage in the real world? These are esc dar
ailie our attention in this chapeer £9 the place of form-focused assessment in all
of the other aspects of language ‘assessment that have already been considered in
this book. How does one assess grammar? Is it appropriate to propose to fest one's
wpilty to comprehend or produce vocabulary? How can a teacher assess a student's
implicit knowledge of forms? Must such knowledge be assessed through explicit
focus on form? 1 look at this often-misunderstood domain of
Janguage teaching and testing.
abilities of 4
We now take a careful
ASSESSING GRAMMAR
In the four skills of listening,
mar is at the center of language Use so it
we believed that knowing a language meant knowing
that language. Although the importance of grammar in language
changed, there has been 4 Jot of debate about the place of teaching grammar in the
language class. Over the years, the focus on grammar has changed, depending on
the teaching method. For example, the Grammar “Translation Method (see TBP.
Chapter 2) was more about learning the structures of the IANguse™ than using it for
comimuniéaciee purposes. In contacts the Direct cus on grammar
Method didn’t fo'
teaching at all because it was thought that ‘grammar would be earned through XP”
ame manner that native speal
ers acquire their
a and interaction, in much the si sai
rst language.
* F sand 1
The Communicative Language Teaching er of the 1980s ah ye if
an approach to language teaching, that emphasized oe ‘ eee”
one cases focus on form got lost i c. A few me
lotion that students should pay ™
allowed to “absorb” language (Krashen, 1997). Todays the g0
the knowledge of
peaking, reading, and writing,
r many years
isn't surprising that for
the grammatical structures of
learning hasn't
990s ushered in
Scanned with CamScanner 294 cuir 11 Assessing Grammar and Vocabulary
age learner
language teaching is to help the language leat
tively inthe real word; however, most COMMWN TT ncrefore some attentio,
grammatical competence is integral to lang
to teaching grammar is advocated. Al
sessed as grammar has also change
‘Translation paradigm, tests consisted a
rules, provide an accurate transtation of a &
word. Today, the knowledge of grammar
nication through listening, speaking, readin: a
In terms of assessment, “grammar is centra
g, and writing in the second language
be able to communicate efjeg,
ative approaches recognize
se |
long with teaching grammar, What has heer
dt over time. For example, in a Gramma
the learners’ ability to recite grammaticy
a text, of supply a grammatically accuray
evaluated by its correct use in commy,
10 language description and te,
fore it’s important to kn,
sess . 498), and there! OW
taker performance” (Rimmer, 2006, P. in this discussi
how best to assess a learner's knowledge of grammar. To begin Scussion, we
‘ ledge.
need to first understand what we mean by grammatical knowledg
Defining Grammatical Knowledge
If you recall from Chapter 1, we discussed communicative competence (Canale
Swain, 1980) as consisting of four component:
&
: grammatical, sociolinguistic, dis.
course, and strategic competencies. In this model, grammatical competence was
defined as knowledge of rules of phonology, lexis, syntax, and semantics, but the
model did not clearly show us how these were associated. Larsen-Freeman (199,
1997), influenced by the communicative competence model of language pedagogy,
characterized grammatical knowledge to show the interrelatedness of phonology,
lexis, syntax, and semantics. Her conceptualization of grammatical knowledge con-
sisted of three interconnected elements:
1. grammatical forms or the structures of a language
2. the grammatical meanings of those forms
3. their pragmatic meaning or use in a given context
The elements above refer to the followin;
how words are formed, and
phology and syntax
Grammatic
veyed by the form, It is concerned with the meani
@) The pragmatic or implied meanin
choices a learner makes in a given coms
¢ municative event.
Grammatical knowledge of form, m
and beyond the sentence or what is cor
ig: (1) Form is both morphology, or
syntax, how words are strung together; both mor-
are concerned with the linguistic accuracy of language. 2)
‘al meaning consists of both the literal and intended message that is con-
ingfulness of the language used.
8 tesults from the appropriate language
caning, and use occurs at the sentence level
yond t mmonly called the discourse level. Discourse
Constraints include rules for cohesion
(,
management (e.g., given/new inform,
hedging devices to ind
at the discourse level,
ate disagreement),
Pragmatic meanin;
» pronominal reference), information
‘ttion), and interactional knowledge (>
which identify relationships in langu'8°
18 depends on the contextual, social!
i
Scanned with CamScanner MMT Nesessing ¢
essing
MH" a Voetulyy
uistic, sociocultural, psychological
and can occur at both the sentence
‘To assess
and thetorie:
i and di
this grammatical knowled
mea
IScourse level
Ine, we cay
7 Pea 4 0 target one
\ runs of form and meaning. Purpur’s (2np4 9 a ) frame OF MOE OF the ex
‘ 11.1 on the next page, offers an excellent taxonomy of compan’ SHO in Figure
: knowledge along with a list of possible grammar pyre en onetS Of grammatia,
: sure cach of the points. Purpura suggested this on Bat Could be use
help define the ability or construct of grammy Be 88 a guide to
that you can use such a chart
examples of components.
DESIGNING ASSESSMENT TASKS: SELECTED RESPONSE
is brief summary of what gra is
With this bi y of what grammar knowledge is, we can now more clearly
examine different types of tests that can be used to measure this knowledge.
The input for selected response tasks can be language (or nonlanguage as in a
gesture or picture) of any length from one word to several sentences of discourse,
‘The test-taker is expected to select the correct response, which is meant to measure
the knowledge of grammatical form and/or meaning. These responses are often
scored dichotomously (0 or 1) although sometimes, depending on how the abil
or construct is defined, partial credit scoring (e.g., 0, 1, or 2) may be used. Scoring
is discussed in more detail in Chapter 12.
Multiple-Choice Tasks
The most common selected response task presents a blank or underlined
word/words in a sentence and the test-taker must choose the correct response from
options that are given. The advantages of the multiple-choice tasks are iain 5
easy to administer and score, and the disadvantages are that the; peal i
create, can promote guessing from test-takers, and are nee ne ae
being authentic language use. However, theses tasks are very be eee
standardized testing environments. Let's look at some exampres
tasks for grammatical form and grammatical meaning:
Grammatical form
Carson: Did you see the movie Titanic last week?
Ethan: Yes, Mary loved It, and _____—"
A. loved too
B. Ido
C. do did |
D. s0 do!
Scanned with CamScanner ‘ =
296 CHAPHER 11 Assessing Grammar and Vocabulary
Grammatical knowledge «Pragmatic knowledge
xy
Pragmatic meanings
Grammatical form Grammatical meaning conn aaaopitenese
(aca | aningtulness naturale
(accuracy) {imeaningfulness) tonal atane
SI \L LEVEL ‘SENTENTIAL OR:
ENTENTIAL LEVEL len! ‘SENTENTIAL DISCOURSE
Pranologesy Phonoasieal contextual mening
graphological form raphological meaning
$ Segmental fms Sabina pas * interpersonal
'* prosodic forms (stress, rhythm, + interrogatives, tags . a 7
Ftonaton, volume) * emprasislonrest Sociolinguistic meaning
+ sound-spelng correspondences ‘ homophony (they re, there) * social identity markers (gender,
2 wing ystems ‘ omegrapty (he win, to Wind) an, statis, group members
+ cata identity markers
Lexical form Lexical meaning Gilet. natweness)
© orthographic forms: ‘© denotation and connotation ‘© social meanings (power,
* sytate features and estctons meanings of erm expressions poitoness)
(ouns) meanings of false cognates «+ Tegistr variation and modality
+ mopholgial regulary 1 Sonal atrites of woeds (esters in speaking wring)
+ word tomation (compounding, denoting physical atractvenes) + Soil norms petreces,
ervatinal affaton + prtayplaty (words ganoting and expectations
+ countabilty and gender restrictions physical attractiveness) * register variation and genres
* co-oocurence restrictions (*depend '* polysemy (head of person/bed/table) (academic, ESP)
on, sot of) 1 Ealocation able and ena) ° i
« formula forms Sociocultural meaning
; * cultural meanings (uta
Morphosyntactic form Morphosyntactic meaning references, urate meanings,
+ inflectional anes (£0) = time/duraion meaphon
derivational affixes (un-) + reversive (pack/unpack) ‘© cultural norms, preferences, and
‘syntactic structures (lense, aspect) + interrogation, passivization expectations, (naturalness,
+ Simpl, compound a * cause-etetfactuaeountrtactal frequency and use of ais,
complex sentences formulaic expressions,
«Voce, mood, word order toocatons
‘© modality differences (speaking,
i
DISCOURSE OR Lol DISCOURSE OR ae
SSUPRASENTENTIAL LEVEL SUPRASENTENTIAL LEVEL Psychological meaning
Cohesive form Cohesive meaning Selene, npc, ne
«referential forms (personal, * possession, recircity peor tony, roe
demonstrative, comparative) + Spatal temporal or re clam
+ pian spas syenoeais ¥
‘excl forms (epettion « Infrmatonal ink to avod ; ;
'* logical connectors (therefore) redundancy Rhetorical meaning
2 tdccency pave Se cet, causal + heres
Information +<>| Information
| ag Tasks or sness-Raising "
| otcl 8 ising Tasks
| ese M considered particularly hel
| Mare consdere ly helpful for learne
| ‘andor meaning, learners become aware ofthe By attendin
| Tesi English (Elis 1997). In these types of tasks teste
vesiine or circle) that they have identified a speeitc ea 08nd
eCific :
In the following example, test-take feature in th
, testtakers must dist lang
dal would: st distinguish between the rey
0
asks contain a wide range of input j
1B¢ Of input in the form of lan
IBUARE oF no
onlangu
ge
18 consciou
" usly to
Ce Of specific language
cate cu
sample:
gypes oF the MO
meaning (Purpura, 2004, p. 134)
Grammatical
Directions: Circle would when it refers to the habitual i
irefes to the present or future ast. Underline it when
You know? You tink youve gotit bad, When I was ki a Kid we woud have o
Mato and from school everyday. We would even do t wien it memes °
mie, nd we would never even think of complaining, We woul smile and go about
sapeinss. | wouldn't change those days for anything, Would you now please hut
up and take out the garbage?
DESIGNING ASSESSMENT TASKS: LIMITED PRODUCTION
inlimited production tasks, the input in the item i langue (or nonlanguage) infor
maton. ike selected response tasks, the input could be 2 single sentence of a longer
reich of discourse. However, unlike selected response tasks, the testtaker's response
represents only limited amount of language production, This response can vary from
asingle word to a sentence depending on the grammatical ability or construct that is
sated, Sometimes the range of possible correct answers forthe espn can be large.
Scoring ofthese responses can be either dichotomous oF partial credit, Dichotomous
scoring means that there is only one criterion for correctness (form or meaning), and
testtakers get it either right or wrong. Partial credit scoring, on the other hand, can
beused with multiple criteria for correctness (form and ‘meaning) and allows for dding
up the scores for the item in terms of full (2), partial (1),.or 0 credit (0). Limited
responses can also be scored holistically oF 4 alytically. Among the most common lin
ited production tasks are gap-fll, shortanswer, ‘and dialogue-comp!
fetion tasks.
Hilling Tasks
a sentence, dialogue, oF passize in which
vine made to test one oF more Ae of
The language is presented in the form of
edge and grammatical ii:
1
lumber of words are deleted, The deletions
imatical knowledge, such as vocabulary know!
Scanned with CamScanner 300 couvneg 11 Assssing Grammar and Vocabulary
sponse for the deletion oF gap basey
c
¢ appropriate r the r
nsider the following ¢
iB XAMples,
‘Testetakers must choose th :
i suage is presented CO"
the context in which the lang
Grammatical form
‘ Jeep. Recently, | ha
book right before | go t0 s ly, have been
sing oqaphien 1 the biographies of Charlie Chaplin ang
Mahatma Gandhi, and for the past few nights, | Ba
Oprah Winfrey,
re the grammatical forms (simp,
le, the criteria for correctness a!
resent perfect, “have read"; ang
“read”; past tense, “read”;
shave been reading”).
In this exampl
habitual present tense,
past perfect continuous tense,
Grammatical form and meaning
also called the “Great River,” is the longest river in the
United States. The river at Lake Itasca in the state of Minnesota
The river ends at the Gulf of Mexico where it______ over halfa
million cubic feet of water into the gulf.
‘The Mississippi River,
In this example, the criteria for correctness are both grammatical form and
meaning in terms of simple habitual present tense verb forms and the lexical
meaning of the words. So the first deleted word from the item is “originates and
thus the appropriate responses for the first deletion in addition to “originates” could
be either “begins” or “starts” but not words such as “initiates” or “creates” because,
although they have the same meaning as “originates” they are not meaningfully cor
rect in the above context. Similarly, the next deleted word is “discharges,” ‘and other
appropriate responses are “releases” or “expels” but not “dismisses” or “detonates’
because, although they have the same lexical meaning as the original word, in the
context of the sentences they are grammatically inappropriate,
Short-Answer Tasks
In some assessment tasks, the input is presented in the form of a question or ues
tions following a reading passage or oral/visual stimulus. The expected testtaker
ponse can vary from a single word to a sentence or two. These short answers Ci
be scored dichotomously (right or wrong) for a single criterion for correctness of
with partial credit for multiple criteria for correctness. Let's take a look at som?
example tasks on the next page.
at
Scanned with CamScanner Fr
Grammatical form and meaning
Chant 11 Assessing Grammar and Vocabulary 301
irections: Read the following paragraph, then answer each question below in
sentence or two.
The Modern Olympics
The modern revival of the Olympic Games is due in a large measure to the efforts
of Pierre, baron de Coubertin, of France. They were held, appropriately enough,
in Athens in 1896, but that meeting and the ones that followed in Paris (1900)
and St. Louis (1904) were hampered by poor organization and the absence of
worldwide representation. The first successful meet was held at London in 1908;
since then the games have been held in cities throughout the world. World War |
prevented the Olympic meeting of 1916, and World War Il the 1940 and 1944
meetings. The number of entrants, competing nations, and events has increased
steadily since then. (Columbia Electronic Encyclopedia, 2008)
Question 1: What did Pierre, baron de Coubertin do?
Question 2: Why were the first few modern Olympic Games not successful?
|
Testing grammatical form and meaning
Directions: Look at the picture, then answer the question below.
Question: What is the woman doing?
In both of these examples, the test-taker must provide a response that is both
grammatically correct and meaningful in terms of the question that is asked. In the
first example, the response must be in the past tense and must show understanding
/
Scanned with CamScanner B02 cramnee 11 Asessng Grammar and Vorabaliny
siation andl the idea that He fmes lacked attendaneg
the present continuous tense ang
must Use ic
ng, a computer, “surfing the Internet»
poor oF
ple, the response
abulary such as "U
of terms such
In the second ex
must use appropri
or*sitting ata desk.”
jate voca
Dialogue-Completion Tasks
Here the input is presented in the form of a short Sa a dialogue in which
a part of the exchange or the entire exchange is left blank and the expectey
tically correct. Like the other short-answer tasks
response is meant to be gramma the other
terion for correctness can be the form or form and meaning.
Testing grammatical form and meaning
Directions: Fill in the blanks in each conversation with one or more words that
are grammatically correct in the context.
Conversation 1
Mayumi; How was your trip to Los Angeles?
Karina: It
Mayumi: | knew you would enjoy your visit! So, what did you do?
Karina: | Disneyland and Universal Studios.
nversation 2
Server: What can | get you?
Customer:
Server: That's a good choice—the pastrami sandwich is o1
Ine of Our most popu
Customer: | would also like a Coke. pepusrens
Server:
Customer: Okay, then I'l have lemonade.
In both exa cs e ct
the inp ath examples, the correct response requires thatthe testaker understand
in the frst ofthe teen ctsition that is provided by the other participant, Thus
© two exchanges, the past tens a “ "
to Los Ammen in ex » the past tense marker “was” in “how was your ti
een et ‘tes to the testtaker that the event is past and theicbes the
that the response he H he past tense. In addition, the test-taker must understand
st be positive because
ect are se the next part of the dialogue indi
. Likewise, in the a ialogue indicates
nen the second example, when the customer requests CEP
tain beverage, the
testtaker must ps
: st provide
ext utterance oF the customen ne {ME FesPonse of the server based on the
Scanned with CamScanner MMT Assessing ¢
Hamann aed Vocatulany yy
ING ASSESSMENT TASKS: EXTENDED PRODUCTION
ut for extended Asks is ustally presented
Mg Fength and can be either kanguaye
‘jor. The purpose of extended producti
from the U sttaker
nei
in the form ofa prompt The input
OF Nonlanguage (yesture of ietase)
Tee On ts W obtain larger amounts of jar
fecal nO of ne &feative construction: therefore
tasks are likely 10 elicit instances of authentic tang
te sponses of test-takers are usually
maple correct options, these extended production taske are
pos scales. When constructing the rating scale, the test designer needs to fi
ae the grammatical ability th at will be assessed and the levels of ability, ae
siyieh must be able to be explicitly described in the scoring rubric. the following
Wy scoring rubric on 2 five-point scale for assessing the knowledge of syntax
Ei apted from Bachman and Palmer (1996), _
ah
use. On the other
Open-ended with a number of
often scored using
Fiver ‘ale for assessing synt
ble 11.1. Five-point scal 1g sy
tied pp. 275-276)
f ‘evel of ability Description ey =
t
factic knowledge (Bachman & Palmer,
None There is no evidence of knowledge of syntactic form or there is not
enough information to judge the performance.
1 Limited There is limited evidence of knowledge of syntactic form but range
of use is small and the accuracy is poor
qa There is moderate evidence of knowledge of syntactic form: the
range of use and accuracy is moderate.
There is extensive evidence of knowledge of syntactic form the
range of use is large with few exceptions and the accuracy is good
3 Extensive
with few errors.
4 Complete There is complete evidence of knowledge of syntactic form; the
range of use is unlimited and the accuracy shows control.
Information Gap Tasks
This task, more commonly known as the info-gap task, presents the input in terms
Of incomplete information. ‘That is, one testtaker is given half—or some—of the
information and another test taker is given complementary information. Both te:
takers then have to question each other to get all the information. The need for nego-
tiation makes this type of task suitable for measuring a testtaker's grammatical
Knowledge to communicate functional meanings. ‘The task can also be used to mea
Sure pragmatic knowledge because the reciprocal nature of the performance
Requires the language of the test-takers to display politeness, formality, appropriate:
Scanned with CamScanner Se
304 Care 11 Assessing Grammar and Vocabulary
he following is an example of an info,
onal conventions.
ness, and other conversational convention apt Doel iekk
Rap task adapted from Purpurt (2004), The task aims ¢ know!
al form, and meaning and use such ,
RC i il er interactional form, @ 1 as
cage of question formation, other intent ieee See
request for clarification and repair. Each ker receives his 0 own inform,
ation solicited from a partne,
tion along with a card with blanks for filling in informat @ partner.
Grammatical form, meaning, and pragmatic use
Working with a partner, ask questions to find out about the other Pee
Then, using all your information, prepare a short report comparing the two
famous painters.
ker B
Test-Taker A Tes!
Claude Monet Vincent van Gogh
Nationality: Dutch
Year of birth: 1853
Style of painting: Post-Impressionist
Well-known paintings: The Woman in the Well-known paintings: The Starry Night;
Green Dress; Water Lilies Self Portrait
Year of death: 1926 Year of death: 1890
Nationality: French
Year of birth: 1840
Style of painting: Impressionist
In this example, Students A and B must pose questions to each other to gather
information about the other painter. This requires the correct syntax of question
formation such as in “What is Monet’s nationality?” or “Is van Gogh Dutch?” Test.
takers may also need to clarify information in the process and must use the correct
form and meaning of the language for that purpose. For example, they might use
modals for requests such as “Could you repeat that again please?”
Role-Play or Simulation Tasks
‘The input in roleplay tasks present testtnkers with a language or nonlanguage
Prompt that asks them to take on a role or simulate a situation to solve a problem,
make a decision, or perform some transaction collaboratively, The expected
trot poe oF role-play or simulation task can contain a large amount of language,
and therefore it can be used to measure a test-taker’s knowledge of grammatical
form, meaning, and pragmatic use.
An example of a problemsolving role play is given on the next page. The
Prompt describes an issue that needs to be resolved and Suggests ways to solve the
Problem. The prompt also specifies three Possible roles that test-takers could take
On 4s part of the problem solving, |
Scanned with CamScanner VY
CIMT Assessing Gramma anil Vorabulary 90
focabudlnry 305
eanmatica form, meaning, and pragmatic use
yourstate Is facing a budget cut and the governors finance
pow to allocate the limited amount of money. The mayor has eae a een
find out what the ctzens think, You would tke to see money given rs a ime
‘ con
tach person must present his or her issue to the
roup, and tt
must decide which issue will be allocated money, ovhs ancien asa group you
eeeeeeee eee
Member A: The streets in the old part of town need repair. It is important to restore
the streets because many people who visit the city often go to the old part of town to
gee the historic homes. Tourism provides income.
Member he children's community center is going to close, which will leave many
children with no place to go after school. If the Place closes, parents will have to ie
changes in their work schedule to look after their children. In order to Provide a safe
place for children to meet, play, and interact, it is important we keep the children’s
community center open.
Member C: The local soccer team has been trying to make renovations to their
soccer stadium for months. They need new seats and a new pitch if the soccer team
is going to win games. In the past the soccer team was very successful and made the |
city very well known.
Inthis example, the test-takers must be able to provide responses that are gram-
matically correct in both form and meaning and that are appropriate for arguments
and counterarguments—for example, “I think we should . . . because” or “Yes, but if
we did that, we would . ..” or “I see your point, however . . .” In addition to the
grammatical form and meaning, this type of task lends itself well to assessing prag-
matic knowledge in terms of the role that the testtaker plays such as mayor, con-
cerned citizen, mother, etc. Because the responses of the testtakers are extended
and do not have one correct answer, they could be scored using a ruting scale.
ASSESSING VOCABULARY
Words are the basic building blocks of a language; we use them to create sentences,
larger paragraphs, and whole texts. As native speakers, we rapidly acquire vocabu-
lary in childhood, and this development continues as we encounter new experi
ences and concepts, but for the second language learner’ the process is demanding,
Sometimes requiring a more conscious effort, Some second language learners make
studied attempt to enlarge their vocabulary, jotting down and memorizing lists of
Words or relying on their bilingual dictionaries to overcome the vocabulary of lex:
ical gaps in their knowledge. For others, vocabulary acquisition seems t occur
‘more naturally, the by-product of a knack for automatic language Processing
Scanned with CamScanner aN
306 cuyrnee 1 Assessing Granmmar and Vocabulary
researchers and teachers recognize that vocabulary knowleg
renal! second language ability and are now focusing on ways to
the knowledge of vocabulary. To begin this discu
s
ge is
teach,
sion,
let’s look at what vocabulary
The Nature of Vocabulary
When we describe the nature of vocabulary we immediately nes Of “words? 55
what are words, and how do we define them for testing purposes? Consider the fo,
ata s,a i pos
lowing paragraph adapted from a recent Newsweek maga:
There are big lies. And little lies. And somewhere in between there are the lies
we tell our doctors. Even the most famous doctor of all, Hippocrates, knew that
those pesky Greek patients might tell a fib or two. To find out if they were
stretching the truth, Hippocrates measured their pulse rates,
First of all, we can identify words as tokens and types. Tokens are all the words
in the paragraph, which in this case totals 53. Types, on the other hand, do not
count words that are repeated, only words that are of different forms. So in the
above paragraph, the word “lies” occurs three times but is counted only once, Both
“doctor” and“doctors” appear, but they get counted as two types, even though they
are in the same word family. Most vocabulary tests would not test two derivatives
of the same family; otherwise one is most likely testing grammatical knowledge
(€g,, “doctor” and its plural counterpart, “doctors").
Another set of categories that we need to consider when we talk about knowl
cdge of words is the difference between function words and content words.
Function words—prepositions, articles, conjunctions, and other “little” words—are
belonging more to the grammar of the language than vocabulary. In isol-
tion, function words mostly show the association among content words in sen.
tances, Content words are nouns, verbs, adjectives, and adverbs. Generally then, its
content words that we focus on in vocabulary tests.
Some vocabulary tests might focus on larger |
verbs (put up with, “run into"), compound nouns (“personal computer’ “fish
tank"), or idioms (“a pretty. Penny,’ “against the clock," “actions speak louder than
words"), which have meaning only asa whole unit,
Research (Moon, 1997; Nattinger & DeCarrico 1992; Pawley & Synder, 1983)
has also identified prefabricated Aguage that language users have at thei dis
Posal for communicat bricated language or lexical phrases, as
Nattinger and DeCarrico called them, are groups of words that seem to have a gr
m: eal structure but operitte as ngle unit and hay articular function in com
munication. ‘The authors identified four {ypes of lexical phrases:
xical items such as phrasal
n. The pret
1. Poly wor
Is are short fixed phras
lifying, marking fluency
, $0 10 speak,
's that perform a variety of functions such
> disagreement, ete, For example: for the most
» and wait 4 minute,
Scanned with CamScanner caine 11
Assessing Grammar and Vocab
veabulary 307
nstitutionalized expres
gach as proverbs and formut
te kettle Dlack, nice 10 meet you, how's ae
_ phrasal constraints are mediumength ae Wee YoU later, etc,
‘with one or two slots that can be filled by vario that have basic structure
example: pours truly/sincerely, as far as 1 enous words oF phrases, For
4, sentence builders are phrases that provide ee feli/am aware, ec
Sentence with one or two slots where whole ide: Framework fora complete
example: that reminds me of X, on the other by as can be expressed. For
also X, etc. (Nattinger & DeCarrico, 1992, pp. ne fe and not only X but
¢ fixed in for
xg HY
Or example: pot calling
In vocabulary testing, these larger lexical iter
than single words, partly because ee seecved es attention
«tests that lend themselves more easily to ingiaee oa fare Deen sere
Peso easier to identify (from word lists and texts) 28 well Soc
peeause larger lexical phrases can vary in grammatical form sod fa Pee
functions in spoken and written discourse, they are more Sone
them more difficult to identify and evaluate. However, larger jexieal it oe
weed in vocabulary testing, especially when they are part caesar
context-dependent vocabulary measures’ (Read, 2000, p. 24). compre
hensive and
Defining Lexical Knowledge
S0 what does it mean to “know” a vocabulary item? One way to answer this ques-
tion is to try to clarify everything a learner has to do to acquire a vocabulary item.
Richards (1976, p. 83) outlined a series of assumptions about vocabulary ability that
developed out of linguistic theory:
1. The native speaker of a language continues to expand his or her
vocabulary in adulthood, whereas there is comparatively little
development of syntax in adult life.
2. Knowing a word means knowing the degree of probability of encountering
that word in speech or print, For many words we also “know” the sort of
words most likely to be found associated with the word.
3. Knowing a word implies knowing the limitations impose
word according to variations of function and situation.
4. Knowing a word means knowing the syntactic behavior associated with
that word.
5. Knowing a word entails
the derivations that can be made from it.
6. Knowing a word entails knowledge of the network of associations between
that word and other words in language.
.d on the use of the
knowledge of the underlying form of 2 word and
Scanned with CamScanner 308
‘curr 17 Assessing Grammar and Vocabulary
ic value of a word,
7. Knowing a word means knowing the sa fiferent meanings
8. Knowing a word means knowing many 0!
associated with the word.
; wach further by specifying the scope
Nation (1990) took Richards's (1976) ee ep a ab : ie re scope
of the learners tas to include the distinction between readhttee snd Prodi :
vocabulary knowledge. We may be able to recognize a word when We Se or hes
it, But are we able to use it in our speech or writing? He P rivetion ofa word
requires a different (and perhaps more complex) set of abilities EG ose needed
for reception of a word, so both modes of performance need to be taken into
ment.
rer ecsdtetand the construct of vocabulary ability, ls go back to our
discussion of communicative language testing in Chapter 1 (p. XX). Following
Canale and Swain’s (1980) model of communicative competence, Bachman (1990)
and later Bachman and Palmer (1996) included not only language knowledge (gram.
matical and sociolinguistic competence) but also strategic competence, a set of
“metacognitive strategies that provide language users with the ability to, or capacity
to create or interpret discourse” (p. 67) as part of their model of communicative
competence, Thus Bachman and Palmer's definition of language ability included
both knowledge of language and the ability to put language to use in context. Other
researchers (Chapelle, 1994) accounted for both the explicit knowledge of vocabu-
lary and the ability (more implicitly) to put vocabulary knowledge to use ina given
context. Three components make up Chapelle’s definition of vocabulary ability:
1, the context of vocabulary use
2. vocabulary knowledge and fundamental processes
3. metacognitive strategies for vocabulary use
1. Vocabulary in context. Traditionally in testing, we view context as the sen-
fence or environment in which the target word occurs. However, from a comm
nicative language use position, context is more than just the linguistic environment
in which @ word occurs; it also includes different types of pragmatic knowledge
That is, the meaning of the target word has to be viewed within the social and cul
tural environment as well,
when teenagers
context of the conver
rather
some’
confi
lk about a “babe” or describe an event as “da bomb? the
r ion should signal that the first case is not “a small baby” but
Mcelooking giel and the second a description roughly equivalent to*A¥e
oe fe Consider Read's (2000) example of a British-American English
ver the word “to table." In British English, the term to table means “0
Scanned with CamScanner CUMPIEE LT Assessing Gn
Grammar and Vocab
ww" (the issue is broug _
ss oer (ne issue is ict on heey ee
