Skip to main content

Lorrie Shepard

Followers

86

Following

7

Co-authors

6

Public Views

Lorrie A. Shepard, PhD is University Distinguished Professor at the University of Colorado Boulder. Her research focuses on psychometrics and the use and misuse of tests in educational settings. Her technical work has contributed to validity theory, standard setting, and statistical models for detecting test bias. Her research studies on test use have addressed the identification of learning disabilities, readiness screening for kindergarten, grade retention, teacher testing, effects of high-stakes accountability testing, and most recently the use of classroom assessment to support teaching and learning.

less

Marmara University

Marmara University

Janelle T Scott

University of California, Berkeley

Dr Maurice Coker

University of Calabar, Calabar, Nigeria.

Michael S Merry

University of Amsterdam

Harry Brighouse

University of Wisconsin-Madison

York University

Mark Castrodale

Brock University

Nele Kortendiek

Goethe-Universität Frankfurt am Main

Letitia Graybill

Monmouth University

Interests

Uploads

Papers by Lorrie Shepard

Inflated Test Score Gains: Is It Old Norms or Teaching the Test? Effects of Testing Project. Final Deliverable--March 1989

It is increasingly recognized, following the lead of J. J. Cannell, that actual gains in educatio... more It is increasingly recognized, following the lead of J. J. Cannell, that actual gains in educational achievement may be much more modest than dramatic gains reported by many state assessments and many test publishers. An overview is presented of explanations of spurious test score gains. Focus is on determining how test-curriculum alignment and teaching the test influence the meaning of scores. Findings of a survey of 3tate testing directors are summarized, and the question of teaching the test is examined. Some frequently presented explanations refer to norms used; others refer to aspects of teaching the test. Directors of testing from 46 states (four states conduct no state testing) replied to a survey about testing. Forty states clearly had high-stakes testing. T most pervasive source of high-stakes pressure identified by respondents was media coverage. Responses indicate that test-curriculum alignment and teaching the test are distorting instruction. A possible solution is to develop new tests e.tery year, changing the tests rather than the norms. Two tables present explanations for test score inflation and selected survey responses. (SLD)

Testing and Assessment for the Good of Education

Educational Researcher, Mar 1, 2016

Early presidents of the American Educational Research Association were leaders in the testing mov... more Early presidents of the American Educational Research Association were leaders in the testing movement. Their intentions were to improve education by means of testing, which included both IQ and achievement tests. Early measurement experts acknowledged in scholarly articles that IQ tests could not measure inherited ability of groups with vastly different opportunities to learn, and yet ability testing was promoted as a beneficial means for matching instruction to individual differences until the insights of the civil rights era in the 1960s. Standard achievement measures were developed importantly to allow valid comparisons across school systems and over time, but the representations of learning that were adequate 100 years ago came to have distorting effects on teaching and learning. Today’s young psychometricians have opportunities to create new assessments in partnership with curriculum experts, but they should remain alert to the ways that well-intentioned assessment systems have been corrupted in the past.

Validity for What Purpose?

Teachers College Record, Sep 1, 2013

Background/ContextThe evolution of validity understandings from mid-century to now has emphasized... more Background/ContextThe evolution of validity understandings from mid-century to now has emphasized that test validity depends on test purpose—adding consequence considerations to issues of interpretation and evidentiary warrants.PurposeTo consider the tensions created by multiple purposes for assessment and sketch briefly how we got to where we are; furthermore, to address two critically important purposes: the accountability purpose versus the learning purpose for assessment.Research DesignThis is an analytic, closing commentary to this special section.ConclusionsWhen a test is used as an educational reform, the theory of action behind the reform should be made explicit and that theory or series of claims and assumptions is what should be examined in the validity evaluation. As to the prospect of improving the teaching profession by the use of value-added methods, I believe that this is an overly ambitious use of a potentially useful statistical tool. As these systems are being implemented, we can and should conduct validity studies designed to detect plausible shortcomings and side effects as well as intended outcomes.

Classroom Assessment Principles to Support Learning and Avoid the Harms of Testing

Educational Measurement: Issues and Practice, Mar 1, 2018

NEPC Talks Education: An Interview with Lorrie Shepard About Standardized Testing Policy

Design Principles for New Systems of Assessment: Rather Than Being Led by National Testing Mandates, State and Local Leaders Should Design Balanced Assessment Systems Guided by Coherence, Research on Learning, and Attention to Equity

Phi Delta Kappan, Mar 1, 2017

WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen... more WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen l. daVidSon is a postdoctoral research associate, all at the University of Colorado Boulder, School of Education. by lorrie a. Shepard, William R. penuel, and kristen l. davidson When Congress passed the Every Student Succeeds Act (ESSA) in December 2015, it carried forward many of the same testing requirements that existed under No Child Left Behind (NCLB). But at the same time, it softened the consequences, taking away the federal government's power to determine what will happen to schools that fail to meet specifi c testing goals. From now on, states and districts can decide for themselves what achievement targets to set, and they can choose to focus on needed supports instead of sanctions for their lowest-performing schools. Free from strict Adequate Yearly Progress accounting, many state and district education leaders are exploring new ways to collect information about student learning as well as new ways to use that information. Where NCLB mandated testing strictly for summative purposes (i.e., to judge how well students, teachers, and schools have performed), ESSA permits and even provides some funding to encourage states to develop balanced assessment systems. Such systems encompass not just summative tests but also local, formative assessments, which could include curriculum-embedded assessments designed to provide teachers with insights about instructional supports that are needed. Rather than being led by national testing mandates, state and local leaders should design balanced assessment systems guided by coherence, research on learning, and attention to equity.

Parent Opinions About Standardized Tests, Teacher's Information and Performance Assessments. A Case Study of the Effects of Alternative Assessment in Instruction, Student Learning and Accountability Practices

As part of a larger study of implementation of alternative asse.sment, a survey of parent opinion... more As part of a larger study of implementation of alternative asse.sment, a survey of parent opinions about standardized tests and performance assessments was conducted in three elementary schools. In the three participating schools, 3rd-grade teachers attended workshops on assessment development and implemented these practices in their classrooms. Samples of 69 parents from participating schools and 36 from three control schools were interviewed after completing questionnaires. Findings suggest that parents' favorable ratings of standardized national tests, supported by a Gallup Poll on the issue, do not imply a preference for such measures over other less formal sources of information for monitoring their children's progress or for judging the quality of education at their local schools. Parents tended to rely on the teacher to tell them how their child was doing relative to others, and they seldom mentioned comparison to external and national norms. Even for accountability purposes, parents preferred talking to the teacher and seeing student work. Most parents endorsed the performance assessment problems they saw, although a few expressed concern over the subjectivity of such measures. Twelve tables present survey and interview findings. An appendix presents excerpts from some parent interviews. (Contains 12 references.) (SLD)

MMSS: Methods for Identifying Biased Test Items

This book is highly recommended for all involved in test development or who have any interest in ... more This book is highly recommended for all involved in test development or who have any interest in the use of tests in education or in other fields. The necessary mathematics are presented clearly but never obscure the important messages in the book. It will certainly be referred to constantly in my future work in this area. --Educational Research The fundamental goal of the Measurement Methods for the Social Sciences series is to make complex measurement concepts, topics, and methods available to readers with limited mathematical background but a strong desire to understand, as well as use, methods that are on the forefront of social science assessment. With this book on item bias detection methods, Gregory Camilli and Lorrie Shepard have achieved this goal admirably. --from the Foreword by Richard M. Jaeger What can item bias methods do--and not do--when applied to real test data? Aimed at helping researchers understand how item bias methods work, this book provides practical advice and specific details on the most useful methods for particular testing situations. Beginning with a review of early bias methods and the fairness issues associated with the topic of test bias, the authors explain the logic of each method in terms of how differential item functioning (DIF) is defined by the method--and how well the method can be expected to work in various situations. In addition, chapters include a summary of findings regarding the behavior of the various indexes in empirical studies, especially their reliability, correlation with known bias criteria, and correlations with other bias methods. The book concludes with a set of principles for deciding when DIF should be interpreted as evidence of bias

Classroom Assessment to Support Teaching and Learning

Annals of The American Academy of Political and Social Science, May 1, 2019

Classroom assessment includes both formative assessment, used to adapt instruction and help stude... more Classroom assessment includes both formative assessment, used to adapt instruction and help students to improve, and summative assessment, used to assign grades. These two forms of assessment must be coherently linked through a well-articulated model of learning. Sociocultural theory is an encompassing grand theory that integrates motivation and cognitive development, and it enables the design of equitable learning environments. Learning progressions are examples of fine-grained models of learning, representing goals, intermediate stages, and instructional means for reaching those goals. A model for creating a productive classroom learning culture is proposed. rather than seeking coherence with standardized tests, which undermines the learning orientation of formative assessment, I propose seeking coherence with ambitious teaching practices. The proposed model also offers ways to minimize the negative effects of grading on learning. Support for teachers to learn these new assessment practices is most likely to be successful in the context of professional development for new curriculum and standards.

What Doesn't Work: Explaining Policies of Retention in Early Grades

Phi Delta Kappan, 1987

Will National Tests Improve Student Learning

Phi Delta Kappan, 1991

Claims that national tests will improve student learning are explored, asking whether national ex... more Claims that national tests will improve student learning are explored, asking whether national examinations will ensure high-quality instruction and greater student learning and whether tests developed to meet urgent political deadlines will retain essential features of authentic curriculum-driven assessments. Part I presents research evidence on the negative effects of standardized testing, such La the effects of high stakes testing on scores, the curriculum, and instruction. The National Education Goals Panel's (NEGP's) version of national examinations is presented in Part II, wit.h attention to their proposals intended to forestall negative effects of traditional tests. Part III identifies curricular and technical problems that must be resolved before the NEOP's vision cam be achieved. These include: (1) development of world class rather than lowest comnon denominator standards; (2) development of incorruptible performance tasks; (3) teacher training in curriculum and instruction; (4) high standards for all students without reinstitution of tracking; and (5) cost. If tests are developed before these problems are resolved, new tests are likely to have the same pernicious effects as the old. There is a 32-item list of references. (SLD)

Curricular Coherence in Assessment Design

Teachers College Record, Nov 1, 2004

An Analysis of the Content and Difficulty of the CSAP 10th-Grade Mathematics Test

A Computer Program to Aid the Detection of Biased Test Items

Educational and Psychological Measurement, Sep 1, 1985

This program was written to identify items on ability tests that favor one particular group of ex... more This program was written to identify items on ability tests that favor one particular group of examinees in relation to another. That is, controlling for group differences in ability, do some items have a greater likelihood of being correctly answered in one of two groups? ' SHEPARD, Camilli, and Williams (1985) determined that the signed chi-square (xzs) and residualized delta indices (Ar) were the most nearly accurate approximate statistics (of those widely known) for detecting biased test items. These statistics adequately approximated more elegant indices based on item response theory (IRT). These two indices are practical alternatives to IRT indices: (a) they are computationally simpler and (b) they may be obtained with relatively small sample sizes. (An unsigned approximate chi-square index [x ' "1 was also examined by Shepard et al., but was not recommended for detecting bias). Chi-Sqimre Indices The chi-square bias method is a rough approximation to IRT methods. It follows the same definition of unbiasedness; that is, individuals with equal ability should have the same probability of success on a test question. Rather than representing a continuous ability dimension, however, the total score scale is divided into crude intervals, usually five. Within these intervals, examinees from different groups are expected to have equal probabilities of success

Assessment of Learning Disabilities

The assessment and diagnosis of learning disabilities (LD) in the school is problematic. How do e... more The assessment and diagnosis of learning disabilities (LD) in the school is problematic. How do educators determine who is learning disabled? What practices are recommended? The main focus of the paper is on specific, relatively technical points that influence the validity of assessment. Since technical concerns are only one of the factors influencing the validity of picements, this paper is organized into two sections: the context of LD identification and technical issues in LD assessment. Specific propositions regarding the context of LD identification are advanced with supporting evidence: overidentification in the LD category; ambiguity in the definition of LD and local idiosyncratic criteria; students' needs for special help; parental demand and pressure from regular education; teaching and system failures; and the consequences of overidentification. Technical topics related to steps in the assessment process are referral bias; normal variability and clinicians' vertigo; technical adequacy of tests; specialists' knowledge of test adequacy and measurement concepts; significant ability-achievement discrepancy; interpreting subtest scatter; using age norms to evaluate processing deficits; behavorial indicators, informal assessments, and clinical hypothesis testing; and exclusion and bias. Recommendations made include contextual changes that are likely to help clinicians be willing to make more rigorous diagnoses and improved training and retraining of specialists. (PN)

Design principles for new systems of assessment

Phi Delta Kappan, Mar 1, 2017

WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen... more WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen l. daVidSon is a postdoctoral research associate, all at the University of Colorado Boulder, School of Education. by lorrie a. Shepard, William R. penuel, and kristen l. davidson When Congress passed the Every Student Succeeds Act (ESSA) in December 2015, it carried forward many of the same testing requirements that existed under No Child Left Behind (NCLB). But at the same time, it softened the consequences, taking away the federal government's power to determine what will happen to schools that fail to meet specifi c testing goals. From now on, states and districts can decide for themselves what achievement targets to set, and they can choose to focus on needed supports instead of sanctions for their lowest-performing schools. Free from strict Adequate Yearly Progress accounting, many state and district education leaders are exploring new ways to collect information about student learning as well as new ways to use that information. Where NCLB mandated testing strictly for summative purposes (i.e., to judge how well students, teachers, and schools have performed), ESSA permits and even provides some funding to encourage states to develop balanced assessment systems. Such systems encompass not just summative tests but also local, formative assessments, which could include curriculum-embedded assessments designed to provide teachers with insights about instructional supports that are needed. Rather than being led by national testing mandates, state and local leaders should design balanced assessment systems guided by coherence, research on learning, and attention to equity.

The Place of Testing Reform in Educational Reform

Educational Researcher, May 1, 1993

Formative Assessment: Caveat Emptor

The Future of Assessment, 2017

... In classrooms, formative assessment can readily be done in the context of mathematics problem... more

Using Learning and Motivation Theories to Coherently Link Formative Assessment, Grading Practices, and Large‐Scale Assessment

Educational Measurement: Issues and Practice, 2018

To support equitable and ambitious teaching practices, classroom assessment design must be ground... more To support equitable and ambitious teaching practices, classroom assessment design must be grounded in a research‐based theory of learning. Compared to other theories, sociocultural theory offers a more powerful, integrative account of how motivational aspects of learning—such as self‐regulation, self‐efficacy, sense of belonging, and identity—are completely entwined with cognitive development. Instead of centering assessment within systems that support use of interim and end‐of‐year standardized tests, we argue for a vision of formative assessment based on discipline‐specific tasks and questions that can provide qualitative insights about student experience and thinking, including their identification with disciplinary practices. At the same time, to be consistent with a productive formative assessment culture, grading policies should avoid using points and grades “to motivate” students but should create opportunities for students to use feedback to improve their work. We argue for...

Social Models of Learning and Assessment

The Handbook of Cognition and Assessment, 2016

Inflated Test Score Gains: Is It Old Norms or Teaching the Test? Effects of Testing Project. Final Deliverable--March 1989

It is increasingly recognized, following the lead of J. J. Cannell, that actual gains in educatio... more It is increasingly recognized, following the lead of J. J. Cannell, that actual gains in educational achievement may be much more modest than dramatic gains reported by many state assessments and many test publishers. An overview is presented of explanations of spurious test score gains. Focus is on determining how test-curriculum alignment and teaching the test influence the meaning of scores. Findings of a survey of 3tate testing directors are summarized, and the question of teaching the test is examined. Some frequently presented explanations refer to norms used; others refer to aspects of teaching the test. Directors of testing from 46 states (four states conduct no state testing) replied to a survey about testing. Forty states clearly had high-stakes testing. T most pervasive source of high-stakes pressure identified by respondents was media coverage. Responses indicate that test-curriculum alignment and teaching the test are distorting instruction. A possible solution is to develop new tests e.tery year, changing the tests rather than the norms. Two tables present explanations for test score inflation and selected survey responses. (SLD)

Testing and Assessment for the Good of Education

Educational Researcher, Mar 1, 2016

Early presidents of the American Educational Research Association were leaders in the testing mov... more Early presidents of the American Educational Research Association were leaders in the testing movement. Their intentions were to improve education by means of testing, which included both IQ and achievement tests. Early measurement experts acknowledged in scholarly articles that IQ tests could not measure inherited ability of groups with vastly different opportunities to learn, and yet ability testing was promoted as a beneficial means for matching instruction to individual differences until the insights of the civil rights era in the 1960s. Standard achievement measures were developed importantly to allow valid comparisons across school systems and over time, but the representations of learning that were adequate 100 years ago came to have distorting effects on teaching and learning. Today’s young psychometricians have opportunities to create new assessments in partnership with curriculum experts, but they should remain alert to the ways that well-intentioned assessment systems have been corrupted in the past.

Validity for What Purpose?

Teachers College Record, Sep 1, 2013

Background/ContextThe evolution of validity understandings from mid-century to now has emphasized... more Background/ContextThe evolution of validity understandings from mid-century to now has emphasized that test validity depends on test purpose—adding consequence considerations to issues of interpretation and evidentiary warrants.PurposeTo consider the tensions created by multiple purposes for assessment and sketch briefly how we got to where we are; furthermore, to address two critically important purposes: the accountability purpose versus the learning purpose for assessment.Research DesignThis is an analytic, closing commentary to this special section.ConclusionsWhen a test is used as an educational reform, the theory of action behind the reform should be made explicit and that theory or series of claims and assumptions is what should be examined in the validity evaluation. As to the prospect of improving the teaching profession by the use of value-added methods, I believe that this is an overly ambitious use of a potentially useful statistical tool. As these systems are being implemented, we can and should conduct validity studies designed to detect plausible shortcomings and side effects as well as intended outcomes.

Classroom Assessment Principles to Support Learning and Avoid the Harms of Testing

Educational Measurement: Issues and Practice, Mar 1, 2018

NEPC Talks Education: An Interview with Lorrie Shepard About Standardized Testing Policy

Design Principles for New Systems of Assessment: Rather Than Being Led by National Testing Mandates, State and Local Leaders Should Design Balanced Assessment Systems Guided by Coherence, Research on Learning, and Attention to Equity

Phi Delta Kappan, Mar 1, 2017

WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen... more WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen l. daVidSon is a postdoctoral research associate, all at the University of Colorado Boulder, School of Education. by lorrie a. Shepard, William R. penuel, and kristen l. davidson When Congress passed the Every Student Succeeds Act (ESSA) in December 2015, it carried forward many of the same testing requirements that existed under No Child Left Behind (NCLB). But at the same time, it softened the consequences, taking away the federal government's power to determine what will happen to schools that fail to meet specifi c testing goals. From now on, states and districts can decide for themselves what achievement targets to set, and they can choose to focus on needed supports instead of sanctions for their lowest-performing schools. Free from strict Adequate Yearly Progress accounting, many state and district education leaders are exploring new ways to collect information about student learning as well as new ways to use that information. Where NCLB mandated testing strictly for summative purposes (i.e., to judge how well students, teachers, and schools have performed), ESSA permits and even provides some funding to encourage states to develop balanced assessment systems. Such systems encompass not just summative tests but also local, formative assessments, which could include curriculum-embedded assessments designed to provide teachers with insights about instructional supports that are needed. Rather than being led by national testing mandates, state and local leaders should design balanced assessment systems guided by coherence, research on learning, and attention to equity.

Parent Opinions About Standardized Tests, Teacher's Information and Performance Assessments. A Case Study of the Effects of Alternative Assessment in Instruction, Student Learning and Accountability Practices

As part of a larger study of implementation of alternative asse.sment, a survey of parent opinion... more As part of a larger study of implementation of alternative asse.sment, a survey of parent opinions about standardized tests and performance assessments was conducted in three elementary schools. In the three participating schools, 3rd-grade teachers attended workshops on assessment development and implemented these practices in their classrooms. Samples of 69 parents from participating schools and 36 from three control schools were interviewed after completing questionnaires. Findings suggest that parents' favorable ratings of standardized national tests, supported by a Gallup Poll on the issue, do not imply a preference for such measures over other less formal sources of information for monitoring their children's progress or for judging the quality of education at their local schools. Parents tended to rely on the teacher to tell them how their child was doing relative to others, and they seldom mentioned comparison to external and national norms. Even for accountability purposes, parents preferred talking to the teacher and seeing student work. Most parents endorsed the performance assessment problems they saw, although a few expressed concern over the subjectivity of such measures. Twelve tables present survey and interview findings. An appendix presents excerpts from some parent interviews. (Contains 12 references.) (SLD)

MMSS: Methods for Identifying Biased Test Items

This book is highly recommended for all involved in test development or who have any interest in ... more This book is highly recommended for all involved in test development or who have any interest in the use of tests in education or in other fields. The necessary mathematics are presented clearly but never obscure the important messages in the book. It will certainly be referred to constantly in my future work in this area. --Educational Research The fundamental goal of the Measurement Methods for the Social Sciences series is to make complex measurement concepts, topics, and methods available to readers with limited mathematical background but a strong desire to understand, as well as use, methods that are on the forefront of social science assessment. With this book on item bias detection methods, Gregory Camilli and Lorrie Shepard have achieved this goal admirably. --from the Foreword by Richard M. Jaeger What can item bias methods do--and not do--when applied to real test data? Aimed at helping researchers understand how item bias methods work, this book provides practical advice and specific details on the most useful methods for particular testing situations. Beginning with a review of early bias methods and the fairness issues associated with the topic of test bias, the authors explain the logic of each method in terms of how differential item functioning (DIF) is defined by the method--and how well the method can be expected to work in various situations. In addition, chapters include a summary of findings regarding the behavior of the various indexes in empirical studies, especially their reliability, correlation with known bias criteria, and correlations with other bias methods. The book concludes with a set of principles for deciding when DIF should be interpreted as evidence of bias

Classroom Assessment to Support Teaching and Learning

Annals of The American Academy of Political and Social Science, May 1, 2019

Classroom assessment includes both formative assessment, used to adapt instruction and help stude... more Classroom assessment includes both formative assessment, used to adapt instruction and help students to improve, and summative assessment, used to assign grades. These two forms of assessment must be coherently linked through a well-articulated model of learning. Sociocultural theory is an encompassing grand theory that integrates motivation and cognitive development, and it enables the design of equitable learning environments. Learning progressions are examples of fine-grained models of learning, representing goals, intermediate stages, and instructional means for reaching those goals. A model for creating a productive classroom learning culture is proposed. rather than seeking coherence with standardized tests, which undermines the learning orientation of formative assessment, I propose seeking coherence with ambitious teaching practices. The proposed model also offers ways to minimize the negative effects of grading on learning. Support for teachers to learn these new assessment practices is most likely to be successful in the context of professional development for new curriculum and standards.

What Doesn't Work: Explaining Policies of Retention in Early Grades

Phi Delta Kappan, 1987

Will National Tests Improve Student Learning

Phi Delta Kappan, 1991

Claims that national tests will improve student learning are explored, asking whether national ex... more Claims that national tests will improve student learning are explored, asking whether national examinations will ensure high-quality instruction and greater student learning and whether tests developed to meet urgent political deadlines will retain essential features of authentic curriculum-driven assessments. Part I presents research evidence on the negative effects of standardized testing, such La the effects of high stakes testing on scores, the curriculum, and instruction. The National Education Goals Panel's (NEGP's) version of national examinations is presented in Part II, wit.h attention to their proposals intended to forestall negative effects of traditional tests. Part III identifies curricular and technical problems that must be resolved before the NEOP's vision cam be achieved. These include: (1) development of world class rather than lowest comnon denominator standards; (2) development of incorruptible performance tasks; (3) teacher training in curriculum and instruction; (4) high standards for all students without reinstitution of tracking; and (5) cost. If tests are developed before these problems are resolved, new tests are likely to have the same pernicious effects as the old. There is a 32-item list of references. (SLD)

Curricular Coherence in Assessment Design

Teachers College Record, Nov 1, 2004

An Analysis of the Content and Difficulty of the CSAP 10th-Grade Mathematics Test

A Computer Program to Aid the Detection of Biased Test Items

Educational and Psychological Measurement, Sep 1, 1985

This program was written to identify items on ability tests that favor one particular group of ex... more This program was written to identify items on ability tests that favor one particular group of examinees in relation to another. That is, controlling for group differences in ability, do some items have a greater likelihood of being correctly answered in one of two groups? ' SHEPARD, Camilli, and Williams (1985) determined that the signed chi-square (xzs) and residualized delta indices (Ar) were the most nearly accurate approximate statistics (of those widely known) for detecting biased test items. These statistics adequately approximated more elegant indices based on item response theory (IRT). These two indices are practical alternatives to IRT indices: (a) they are computationally simpler and (b) they may be obtained with relatively small sample sizes. (An unsigned approximate chi-square index [x ' "1 was also examined by Shepard et al., but was not recommended for detecting bias). Chi-Sqimre Indices The chi-square bias method is a rough approximation to IRT methods. It follows the same definition of unbiasedness; that is, individuals with equal ability should have the same probability of success on a test question. Rather than representing a continuous ability dimension, however, the total score scale is divided into crude intervals, usually five. Within these intervals, examinees from different groups are expected to have equal probabilities of success

Assessment of Learning Disabilities

The assessment and diagnosis of learning disabilities (LD) in the school is problematic. How do e... more The assessment and diagnosis of learning disabilities (LD) in the school is problematic. How do educators determine who is learning disabled? What practices are recommended? The main focus of the paper is on specific, relatively technical points that influence the validity of assessment. Since technical concerns are only one of the factors influencing the validity of picements, this paper is organized into two sections: the context of LD identification and technical issues in LD assessment. Specific propositions regarding the context of LD identification are advanced with supporting evidence: overidentification in the LD category; ambiguity in the definition of LD and local idiosyncratic criteria; students' needs for special help; parental demand and pressure from regular education; teaching and system failures; and the consequences of overidentification. Technical topics related to steps in the assessment process are referral bias; normal variability and clinicians' vertigo; technical adequacy of tests; specialists' knowledge of test adequacy and measurement concepts; significant ability-achievement discrepancy; interpreting subtest scatter; using age norms to evaluate processing deficits; behavorial indicators, informal assessments, and clinical hypothesis testing; and exclusion and bias. Recommendations made include contextual changes that are likely to help clinicians be willing to make more rigorous diagnoses and improved training and retraining of specialists. (PN)

Design principles for new systems of assessment

Phi Delta Kappan, Mar 1, 2017

WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen... more WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen l. daVidSon is a postdoctoral research associate, all at the University of Colorado Boulder, School of Education. by lorrie a. Shepard, William R. penuel, and kristen l. davidson When Congress passed the Every Student Succeeds Act (ESSA) in December 2015, it carried forward many of the same testing requirements that existed under No Child Left Behind (NCLB). But at the same time, it softened the consequences, taking away the federal government's power to determine what will happen to schools that fail to meet specifi c testing goals. From now on, states and districts can decide for themselves what achievement targets to set, and they can choose to focus on needed supports instead of sanctions for their lowest-performing schools. Free from strict Adequate Yearly Progress accounting, many state and district education leaders are exploring new ways to collect information about student learning as well as new ways to use that information. Where NCLB mandated testing strictly for summative purposes (i.e., to judge how well students, teachers, and schools have performed), ESSA permits and even provides some funding to encourage states to develop balanced assessment systems. Such systems encompass not just summative tests but also local, formative assessments, which could include curriculum-embedded assessments designed to provide teachers with insights about instructional supports that are needed. Rather than being led by national testing mandates, state and local leaders should design balanced assessment systems guided by coherence, research on learning, and attention to equity.

The Place of Testing Reform in Educational Reform

Educational Researcher, May 1, 1993

Formative Assessment: Caveat Emptor

The Future of Assessment, 2017

... In classrooms, formative assessment can readily be done in the context of mathematics problem... more

Using Learning and Motivation Theories to Coherently Link Formative Assessment, Grading Practices, and Large‐Scale Assessment

Educational Measurement: Issues and Practice, 2018

To support equitable and ambitious teaching practices, classroom assessment design must be ground... more To support equitable and ambitious teaching practices, classroom assessment design must be grounded in a research‐based theory of learning. Compared to other theories, sociocultural theory offers a more powerful, integrative account of how motivational aspects of learning—such as self‐regulation, self‐efficacy, sense of belonging, and identity—are completely entwined with cognitive development. Instead of centering assessment within systems that support use of interim and end‐of‐year standardized tests, we argue for a vision of formative assessment based on discipline‐specific tasks and questions that can provide qualitative insights about student experience and thinking, including their identification with disciplinary practices. At the same time, to be consistent with a productive formative assessment culture, grading policies should avoid using points and grades “to motivate” students but should create opportunities for students to use feedback to improve their work. We argue for...

Social Models of Learning and Assessment

The Handbook of Cognition and Assessment, 2016