Despite the prevalent use of structural equation modeling (SEM) in second language testing and le... more Despite the prevalent use of structural equation modeling (SEM) in second language testing and learning and the growing awareness of the value of replication studies as a means for knowledge accumulation and critical appraisal of previous studies, the existing structural equation models have been infrequently reproduced, with the exception of . This article summarizes the significance of the reproduction of structural equation models in second language testing and learning, illustrates the procedures for such reproduction, and discusses the value and difficulties of such reproduction by answering frequently asked questions.
This study examines a classroom-based second-language speaking test that uses picture description... more This study examines a classroom-based second-language speaking test that uses picture description tasks and a rating scale of functional adequacy (FA) in terms of measurement quality. We used four tasks and FA scores from 371 Japanese learners of English at two public senior high schools and 44 undergraduate and graduate students. Each examinee was evaluated by two or three raters. We used many-facet Rasch measurement to stack longitudinal data and analyzed the test scores in detail. We found that examinees, tasks, raters, and the FA rating scale worked well, with some concerns. For example, some examinees showed serious underfit, and large differences in rater severity were apparent. The bias analysis suggested that the percentages of biased patterns were small between examinees and tasks, between examinees and raters, and between tasks and raters. Based on our results, implications for assessment are provided, such as the need to conduct rigorous rater training.
Second language (L2) listening comprehension is a function of many variables. We focused on metac... more Second language (L2) listening comprehension is a function of many variables. We focused on metacognitive awareness, which we measured using the Metacognitive Awareness Listening Questionnaire (MALQ; Vandergrift et al., 2006), and meta‐analyzed (a) the factor structure of the MALQ and (b) the relationship between metacognitive awareness and L2 listening comprehension. We used meta‐analytic structural equation modeling to synthesize 29 studies that provided Pearson's product‐moment correlation matrices from 4,574 learners. Results showed (a) that the MALQ measured metacognitive awareness as a single factor with five subcomponents (their interrelationship and relative contribution to metacognitive awareness varied) and (b) that metacognitive awareness explained listening comprehension (b* = .306). The results were moderated by publication type, the response format of listening comprehension tests, and participant type. The findings can help researchers to better conceptualize the ...
Despite the prevalent use of structural equation modeling (SEM) in second language testing and le... more Despite the prevalent use of structural equation modeling (SEM) in second language testing and learning and the growing awareness of the value of replication studies as a means for knowledge accumulation and critical appraisal of previous studies, the existing structural equation models have been infrequently reproduced, with the exception of . This article summarizes the significance of the reproduction of structural equation models in second language testing and learning, illustrates the procedures for such reproduction, and discusses the value and difficulties of such reproduction by answering frequently asked questions.
This study examines a classroom-based second-language speaking test that uses picture description... more This study examines a classroom-based second-language speaking test that uses picture description tasks and a rating scale of functional adequacy (FA) in terms of measurement quality. We used four tasks and FA scores from 371 Japanese learners of English at two public senior high schools and 44 undergraduate and graduate students. Each examinee was evaluated by two or three raters. We used many-facet Rasch measurement to stack longitudinal data and analyzed the test scores in detail. We found that examinees, tasks, raters, and the FA rating scale worked well, with some concerns. For example, some examinees showed serious underfit, and large differences in rater severity were apparent. The bias analysis suggested that the percentages of biased patterns were small between examinees and tasks, between examinees and raters, and between tasks and raters. Based on our results, implications for assessment are provided, such as the need to conduct rigorous rater training.
Second language (L2) listening comprehension is a function of many variables. We focused on metac... more Second language (L2) listening comprehension is a function of many variables. We focused on metacognitive awareness, which we measured using the Metacognitive Awareness Listening Questionnaire (MALQ; Vandergrift et al., 2006), and meta‐analyzed (a) the factor structure of the MALQ and (b) the relationship between metacognitive awareness and L2 listening comprehension. We used meta‐analytic structural equation modeling to synthesize 29 studies that provided Pearson's product‐moment correlation matrices from 4,574 learners. Results showed (a) that the MALQ measured metacognitive awareness as a single factor with five subcomponents (their interrelationship and relative contribution to metacognitive awareness varied) and (b) that metacognitive awareness explained listening comprehension (b* = .306). The results were moderated by publication type, the response format of listening comprehension tests, and participant type. The findings can help researchers to better conceptualize the ...
Uploads
Papers by Rie Koizumi