A framework for analyzing reading test characteristics

1978

In order to provide a structure for describing different approaches to testing, five dimensions along which tests may differ are identified. The dimensions are (1) test uses, (2) item generation, (3) item revision, (4) assessment of precision, and (5) validation. Within each dimension, variations are described reflecting Buros' (1977) distinction between differentiation and measurement. These dimensions are used to profile representative tests from the area of reading comprehension. Only standardized, norm-referenced achievement tests, whose uses (dimension 1) emphasize differentiation, were found to have an inference system (dimension 2 through 5) consistent with those intended uses. No tests were found having inference systems consistent with such intended uses as certifying competence, diagnosing strengths and weaknesses, and tracking progress-uses which emphasize the measurement function of tests. Tests constructed using a domain-referenced approach would yield such an inference system and fill some gaps in the array of currently available measures of reading comprehension.

The aim of this review article is to explore the principles involved in testing reading abilities. It considers the key issues i.e. the selection of text for testing reading comprehension, item format, text length, use of different genres and the role of background knowledge to test individual student levels or processes. Most of the studies followed different taxonomies to test reading comprehension. Bloom's taxonomy had extensively been used by many researchers to measure low and high order skills. The data was collected from different sources i.e. books and journals. Results, based on previous literature, showed that the construction of reading tests was highly dependent on the purpose of measuring intended skills or sub skills at lower and higher cognitive levels of the students. Text selection and its length, text type, item formats, background knowledge of test takers were considered according to the levels of test takers.

British Journal of Educational Psychology, 2006

Background. The Neale Analysis of Reading Ability (NARA; Neale, 1997) is a widely used assessment of reading comprehension and word reading accuracy. Spooner, Baddeley, and Gathercole (2004) questioned the suitability of the NARA for identifying children with specific reading comprehension deficits. Aims and methods. An evaluation of the NARA measurement of word reading and reading comprehension level was undertaken in relation to models of reading ability. Appropriate control measures were considered. The strengths and weaknesses of different forms of reading comprehension were also evaluated. Results. Previous research into reading comprehension difficulties using the NARA has adopted satisfactory control measures in relation to word reading ability. There are limitations associated with all the considered forms of reading comprehension assessment. Conclusions. If administered and interpreted appropriately, the NARA is an effective instrument for researchers and practitioners who need to assess both word reading accuracy and reading comprehension and to identify children with a dissociation between these two aspects of reading.

1980

The procedures used to establish the validity of the Minnesota Reading Assessment (MBA) are described in this paper. The MBA is described as a measure designed for use with students in community colleges, business schools, technical schools, and other secondary and postsecondary training institutions in order to measure student competence in specific reading skills: reading rate and retention, vocabulary, and comprehension. The paper presents data drawn from the validity studies, which were completed in connection with the standardization of the test and which involved students in vocationally oriented programs, in community colleges, in four-year colleges, and in programs for the hearing impaired. (FL)

Psicothema, 2016

ECOMPLEC.Sec is a reading comprehension test for secondary students, conceived from a multidimensional perspective in line with large-scale educational surveys such as PISA or PIRLS. The objective of this study was to validate the theoretical model of ECOMPLEC.Sec. A bifactor model that postulates the existence of a general reading comprehension factor and three specific factors provided a good fit to the data. 1,912 adolescents (13-18 years) participated in this study. Data analysis included construct validity via confirmatory factor analysis, and factors were regressed onto observed covariates for the interpretation of the constructs. Reliability was calculated from a non-linear SEM in order to justify the interpretability of the observed scale and subscale scores. the bifactor model exhibited a significantly better fit to the data than the second-order model. Furthermore, construct validity analysis suggests the existence of specific reading comprehension factors. Finally, the re...

1978

The minimal comprehension principle asserts that the act of comprehension must entail an interaction between an incoming linguistic message and the reader's world knowledge. An analysis of current tests of reading comprehension indicates that test passages are likely to draw broadly from knowledge of the world, so that some of the variability in test performance must to attritutatle to differences in prior knowledge rather than to differences in reading skill. Most tests of reading camprehension currently in use do not distinguish between these two sources of variability. This does not affect their usefulness as predictive instruments but greatly restricts the use of the tests for diagnosis of reading., difficulties or assessment of educational gains. Optimal tests for these purposes would reduce variability due to prior knowledge or reasoning ability. Various suggestions for achieving this control are in need of empirical verification. (AA)

Universal Journal of Educational Research

Reading comprehension has an important place in lifelong learning. It is an interactive process between the reader and the text. Students need reading comprehension skills at all educational levels and for all school subjects. Determining the level of students' reading comprehension skills is the subject of testing and evaluation. Tests used to measure students' success are expected to provide accurate and reliable information. Therefore, the aim of this study is to develop a valid and reliable reading comprehension test appropriate for fourth grade reading comprehension learning outcomes. The participants in the study were 245 fourth graders from three schools at low, middle and high socioeconomic levels in the central district of Kars. Items with discrimination indices below .30 and those that did not differ by 27% in the upper and lower groups on the independent groups t-test were excluded from the test. The difficulty indices of the test items ranged from .37 to .79. The KR-20 reliability coefficient was found to be 0.83, indicating that this multiple choice reading comprehension test is valid and reliable.

2020

This review article discusses the criteria for developing tests for reading skills. Reading skills are input language skills whichare considered as the most basic skills in language learning.The text focuses on the basic approaches used in developing reading tests, selection criteria of texts and authenticity, selection of test items, and the role of Bloom taxonomy in testing reading skills. Asian journal of language testing, Language testing international,Journal of research in reading,Reading research quarterly, Reading for professional purposes, Journal of reading, Journal of reading behaviour, and Language testing are reviewed to answer the research questions.Discussions are analysed and concluded that the level of the learner, objectives, and text itself decide the types of questions that can be constructed and included in reading tests.Different text may tend to different test items such as true-false, multiple-choice, matching, provide information in the table, and yet to add open-ended questions.

National Assessments seeks to include all students in the sampling frame from which students are selected to participate in the assessment. However, some students with disabilities (SWD) are either unable to take tests under standard testing conditions or are unable to perform at their best under standard testing conditions. In many testing situations, accommodations to standard testing conditions are given to SWD to improve measurement of their knowledge, skills and abilities. This practice is in the pursuit of more valid test score interpretation; however, it produces the ultimate psychometric oxymoron -an accommodated standardized test. In this paper, I review validity issues related to test accommodations and summarize some empirical studies in this area. The focus of the paper is on accommodations for reading tests because some types of accommodations on these tests are particularly controversial. The specific accommodations emphasized in this review are extended time and oral (read-aloud) accommodations. A review of professional standards, validity theory, and recent empirical research in this area suggests that extended time accommodations may be appropriate for reading tests, but read-aloud accommodations are likely to alter the construct measured. Suggestions for determining when to provide accommodations and how to report scores from accommodated test administrations are provided.

Language Assessment Quarterly, 2019

With the advancement of Cognitive Diagnostic Assessment (CDA) and the pertinent statistical models, different domains of large-scale testing and assessment have been examined for the sake of reporting more diagnostic information. Applying the generalized deterministic input, noisy, “and” gate (G-DINA) model, the current study analyzed a high-stakes L2 reading comprehension test as an integral section of the PhD degree’s entrance exam in Iran. It aimed at examining the reading comprehension attributes underlying this high stakes test in an attempt to check the capability of CDA models, (in this case G-DINA), in providing diagnostic information for test developers and users. The items’ response data were analyzed in R, “GDINA” package, version 1.4.2. With data collected from multiple sources including the current literature on the sub-skills of reading comprehension, test specifications, test-takers’ think-aloud verbal protocols, and expert panel’s judgments, an initial Q-matrix, including five sub-skills, was developed and then validated. Data analysis, using the validated version of Q-matrix, showed the sub-skill (attribute) prevalence and its difficulty. Code-related sub-skills were the easiest and the most prevalent ones and the connecting/synthesizing sub-skills were the most difficult and the least prevalent ones for the test-takers. Skill mastery profile results verified the relationship among the sub-skills of reading skill and also the prominence of some of these sub-skills over others.