Papers by Susan Embretson
Psychological Assessment, 2018
Working memory capacity is an important construct in psychology because of its relationship with ... more Working memory capacity is an important construct in psychology because of its relationship with many higher-order cognitive abilities and psychopathologies. Working memory capacity is often measured using a type of paradigm known as complex span. Some recent work has focused on shortening the administration time of the complex span tasks, resulting in different versions of these tasks being used . Variations in the complex span tasks, such as the number of set sizes, can lead to varying power to discriminate individuals at different ability levels. Thus, research findings may be inconsistent across populations due to differing appropriateness for the ability levels. The present study uses a combination of item response theory and correlational analyses to better understand the psychometric properties of the operation span, symmetry span, and rotation span. The findings show that the typical administration of these tasks, particularly the operation span, is not suitable for above average ability samples (Study 1; n ϭ 573). When larger set sizes are added to the tasks (Study 2; n ϭ 351), predictive validity and discriminability is improved for all complex span tasks, however the operation span is still inferior to the spatial tasks. The authors make several conclusions about which tasks and set sizes should be used depending on the intended population, and further suggest avoiding the standard-length operation span for average or higher ability populations. The authors take a novel approach at further understanding a set of popular tasks used to measure working memory capacity. By using an advanced modeling technique known as item response theory, we demonstrate that the most widely used of these tasks, the operation span, is not good at measuring individuals of high cognitive ability. The authors increase the number of to-beremembered items in the tasks and demonstrate that they are much better at measuring higher-ability individuals.
Journal of Educational Measurement, Mar 1, 1986
This study examines the influence of processing strategies, and the associated metacomponents tha... more This study examines the influence of processing strategies, and the associated metacomponents that determine when to apply them, on the construct validity of a verbal reasoning test. Three strategies for solving verbal analogy items were examined: a rule‐oriented strategy, an association strategy, and a partial rule strategy. Construct validity was studied in two separate stages: construct representation and nomothetic span. For construct representation, evidence was obtained that all three strategies, and their related metacomponents, are associated with performance on analogy items. For nomothetic span, the current study found that all three strategies contribute to individual differences in verbal reasoning and to the predictive validity of the test. The results of this study also point to the utility of metacomponents as constructs for describing and understanding test performance. Implications of the results for test development and theories of aptitude are elaborated.
Multivariate Behavioral Research, Jan 10, 2022
Academic Press eBooks, 1985
Page 76. CHAPTER FOUR Fundamental Measurement for Psychology Benjamin D. Wright University of Chi... more Page 76. CHAPTER FOUR Fundamental Measurement for Psychology Benjamin D. Wright University of Chicago A new measurement in psychology has emerged from a confluence of scientific and social forces that are producing a revolution in social science methodology. ...
Measurement: Interdisciplinary Research & Perspective, 2004
“The Second Century of Ability Testing: Some Predictions and Speculations” did not include predic... more “The Second Century of Ability Testing: Some Predictions and Speculations” did not include predictions about the ability construct or the role of fundamental measurement principles. All commentators raised issues about the nature of the ability construct (Barrett, Bunderson, Fisher, Golden & Goldman, Mislevy, Sternberg,). The diverse viewpoints represented in these comments highlight well the complexity of the construct issues. Measurement principles were also mentioned by several authors (Barrett, Fisher, Golden & Goldman, Mislevy). Some of these commentaries have directly opposing concerns about the role of measurement models in ability testing. Although the commentaries had several other valuable points, the construct and measurement issues need summarization and integration. Later, I summarize and elaborate the issues in these areas and speculate on their place in the second century of ability testing.
Elsevier eBooks, 2010
Cognitive psychology has an important impact on ability and achievement testing because many of t... more Cognitive psychology has an important impact on ability and achievement testing because many of the item types on educational tests have been studied as cognitive tasks. In this article, a system of designing tests from the cognitive perspective is described. Further, appropriate statistical models for estimating the effects of specific variables on psychometric properties are also described. Examples are presented from two different tests to show how the properties of items are predicted from the cognitive variables. The estimated effect parameters for the cognitive variables may be useful for item banking and test equating if the model has sufficient predictive power.
Measurement: Interdisciplinary Research & Perspective, Nov 30, 2010
's article "Cognitively Based Assessment of, for, and as Learning (CBAL), which appeared in issue... more 's article "Cognitively Based Assessment of, for, and as Learning (CBAL), which appeared in issue 8(2-3), presents an integrated approach to assessment in which a theory of action for the intended consequences (i.e., benefits) drives both assessments and instructional practices. In the Cognitively Based Assessment of, for, and as Learning (CBAL) project assessment is the mainstay of instruction. CBAL has several distinctive features: (1) a goal to facilitate learning as well as to assess it, (2) an explicitly formulated student competency model, (3) a basis in scientific learning principles as well as standards-based content, (4) an integration of formative and summative assessments, and (5) an explicitly developed theory of action. The theory of action, formulated by Bennett (2010), is a "depiction of an intervention that explains the cause-effect relationships among input, activities, and intended outcomes." Thus, intended and intermediate outcomes are specified, along with the inputs and activities that are related to these outcomes. Bennett's broad and inclusive conceptualization of assessment, not only provides a strong impact on the activities and ultimate effectiveness of CBAL, but also provides a model for other assessment programs. Combined with a proscribed rigorous research to support the validity claims, this systematic approach seemingly has potential to improve student competency levels. Bennett does consider possible unintended consequences in his theory of action for CBAL, particularly issues in "teaching to the test" and the possible narrowing of the competency construct by focusing instruction on very specific formats and strategies. There is a larger issue This is a commentary on "Cognitively Based Assessment of, for, and as Learning (CBAL): A Preliminary Theory of Action for Summative and Formative Assessment," by Randy Elliot Bennett.
Journal of Educational Measurement, Mar 1, 1996
An important trend in educational measurement is the use of principles of cognitive psychology to... more An important trend in educational measurement is the use of principles of cognitive psychology to design achievement and ability test items• Many studies show that manipulating the stimulus features of items influences the processes, strategies, and knowledge structures that are involved in solution• However, little is known about how cognitive design influences individual differences• That is, does applying cognitive design principles change the background skills and abilities that are associated with successful performance? This study compared the correlates of two spatial ability tests that used the same item type but different test design principles (cognitive design versus psychometric design)• The results indicated differences in factorial complexity in the two tests; specifically, the impact of verbal abilities was substantially reduced by applying the cognitive design principles•
Educational Measurement: Issues and Practice, Sep 1, 2016
Examinees' thinking processes have become an increasingly important concern in testing. The r... more Examinees' thinking processes have become an increasingly important concern in testing. The responses processes aspect is a major component of validity, and contemporary tests increasingly involve specifications about the cognitive complexity of examinees' response processes. Yet, empirical research findings on examinees' cognitive processes are not often available either to provide evidence for validity or to guide the design or selection of items. In this article, studies and developments from the author's research program are presented to illustrate how empirical studies on examinees' thinking processes can impact item and test design.
Intelligence, Mar 1, 1995
Although both general control processing (i.e., global metacomponents) and working memory capacit... more Although both general control processing (i.e., global metacomponents) and working memory capacity have emerged as primary explanations of abstract intelligence, their relative impact on individual differences rarely has been compared. This study examines the impact of general control processing and working memory capacity on an important measuring task for abstract intelligence. A new multicomponent latent trait model (MLTM) for covert responses was applied to item response data for matrix problems. With this model, working memory capacity could be separated from general control processing because item processing requirements for the former varies systematically across items (Carpenter, Just, & Shell, 1990). whereas the latter remains constant. Structural equation modeling indicated that both processing abilities were significant sources of individual differences. However, general control processing had a stronger impact. The results, and their limitations, are discussed in the context of prior theory and research.
Uploads
Papers by Susan Embretson