Papers by Ellen Osmundson
National Center for Research on Evaluation, Standards, and Student Testing, Mar 1, 2005
This evaluation report summarizes Year 4 of the Math and Science Academy (MSA), an initiative of ... more This evaluation report summarizes Year 4 of the Math and Science Academy (MSA), an initiative of the Northern New Mexico Council on Excellence in Education (NNMCEE). The report begins with an overview of the project and its objectives, and then outlines the research questions and methods used to carry out the evaluation. Findings from the Year 4 evaluation are presented next; the report concludes with recommendations and refinements for future years of MSA.
National Center for Research on Evaluation, Standards, and Student Testing, Jun 1, 2010
This study examines measures of formative assessment practice using data from a study of the impl... more This study examines measures of formative assessment practice using data from a study of the implementation and effects of adding curriculum embedded measures to a handson science program for upper elementary school students. The authors present a unifying conception for measuring critical elements of formative assessment practice, illustrate common measures for doing so, and investigate the relationships among and between scores on these measures. Findings raise important issues with regard to both the challenge of obtaining valid measures of teachers' assessment practice and the uneven quality nature of current teacher practice.
National Center for Research on Evaluation, Standards, and Student Testing, 2011
Overview of the Study This efficacy study was designed to examine the traditional FOSS curriculum... more Overview of the Study This efficacy study was designed to examine the traditional FOSS curriculum (Delta Publishing, Full Option Science System/FOSS, magnetism and electricity, structures of life, and water modules, 2005), and the new ASK/FOSS curriculum (magnetism and electricity, structures of life, and water modules, 2005), a revised version of the original FOSS curriculum, to determine impact on student learning. The enhanced curriculum includes slightly revised instructional materials and a coordinated embedded assessment system, with components developed to support teachers' ongoing assessment of student learning and progress towards specified learning goals. The study involves randomly assigned treatment/control groups of 3 rd and 4 th grade teachers. Year 3 of the project was the study year for ASK/FOSS Cohort 1 (AZ), and the pilot (or practice) year for additional schools and teachers, ASK/FOSS Cohort 2 (AZ, TX and WA). This report presents data from both study cohorts. Table 1 summarizes the data sources, and the type of information provided by each instrument, designed to capture the most critical components of the ASK/FOSS curriculum. In the following sections of the report, analyses and findings are presented based on these data sources and assessment components.
Department of Education. The findings and opinions expressed herein are those of the author(s) an... more Department of Education. The findings and opinions expressed herein are those of the author(s) and do not necessarily reflect the positionsor policies of AACC, WestEd, or the U.S. Department of Education. To cite from this report, please use the following as your APA reference: Herman, J. L., Osmundson, E., & Dietel, R. (2010). Benchmark assessments for improved learning (AACC Report).
Although first popular as an instructional tool in the classroom, knowledge mapping has been used... more Although first popular as an instructional tool in the classroom, knowledge mapping has been used increasingly as an assessment tool. Knowledge mapping is expected to measure deep conceptual understanding and allow students to characterize relationships among concepts in a domain visually. Our research examines the validity of knowledge mapping as an assessment tool in science. Our approach to investigating this validity is three-pronged. First, we outline a model for the creation of knowledge mapping tasks, proposing a standard set of steps and using content area and educational experts to ensure the content validity of the measures. Next, we describe a scoring method used to evaluate student performance, including a discussion of the method's reliability and its relationship to other possible scoring systems. Finally, we present our statistical results including comparative analyses, our multitrait-multimethod validity analyses involving two traits (students' understanding...
This paper focuses on understanding formative assessment in the classroom,one effective strategy ... more This paper focuses on understanding formative assessment in the classroom,one effective strategy for improving student learning.I begin by explaining a basic problem facing our educational system,and suggest a potential solution.I follow with a brief overview of the background and research on formative assessment,and then present our current working model of the formative assessment process.In the next section,I discuss emerging findings from our research work with teachers and formative assessment.The paper concludes by proposing next steps in understanding formative assessment.
The goal of this report was to test the use of sensor-based skill measures in evaluating performa... more The goal of this report was to test the use of sensor-based skill measures in evaluating performance differences in rifle marksmanship. Ten shots were collected from 30 novices and 9 experts. Three measures for breath control and one for trigger control were used to predict skill classification. The data were fitted with a logistic regression model using holdout validation to assess the quality of model classifications. Individually, all four measures were significant; when considered together, only three measures were significant predictors for level of expertise (p <.05). Overall percent correct in shot classification for the testing data was 90.0%, with a sensitivity of 67.5%, and 96.0% specificity.
PsycEXTRA Dataset
Although first popular as an instructional tool in the classroom, knowledge mapping has been used... more Although first popular as an instructional tool in the classroom, knowledge mapping has been used increasingly as an assessment tool. Knowledge mapping is expected to measure deep conceptual understanding and allow students to characterize relationships among concepts in a domain visually. Our research examines the validity of knowledge mapping as an assessment tool in science. Our approach to investigating this validity is three-pronged. First, we outline a model for the creation of knowledge mapping tasks, proposing a standard set of steps and using content area and educational experts to ensure the content validity of the measures. Next, we describe a scoring method used to evaluate student performance, including a discussion of the method's reliability and its relationship to other possible scoring systems. Finally, we present our statistical results including comparative analyses, our multitrait-multimethod validity analyses involving two traits (students' understanding of hearing and of vision) and three different measurement methods (knowledge mapping, essay, and multiple-choice tasks), critical proposition analyses, and analyses of students' propositional elaborations. Results show knowledge maps to be sensitive to students' competency level, with mixed MTMM results. We conclude with a discussion of implications and directions for future work. As evidence mounts regarding the limitations of standardized multiple-choice testing, educators increasingly are looking for ways of assessing students' scientific conceptual understanding that may be missed by more traditional measures. Students' performance on knowledge mapping tasks has emerged as one possible 1 We wish to acknowledge and thank Joanne Michiuye and Ali Abedi of CRESST/UCLA for their invaluable technical support, Robby Klein and Andrew Shpall for their content expertise, and Uyen Bui for her interest, participation, and content expertise. Finally, we are deeply grateful to Sharon Sutton, Jan Cohn, and their students for their assistance and participation in this research.
Principal Leadership, 2008
... (Contains 4 figures.). Abstractor: ERIC. Reference Count: 4. Note: N/A. Identifiers: N/A. Rec... more ... (Contains 4 figures.). Abstractor: ERIC. Reference Count: 4. Note: N/A. Identifiers: N/A. Record Type: Journal. ... Microfiche to Megabytes. ERIC Microfiche Digitization. Help ERIC expand online access to documents currently available only on microfiche. Learn more about our efforts ...
Drawing from a large efficacy study in upper elementary science, this report had three purposes: ... more Drawing from a large efficacy study in upper elementary science, this report had three purposes: First to examine the quality of teachers' content-pedagogical knowledge in upper elementary science; second, to analyze the relationship between teacher knowledge and their assessment practice; and third, to study the relationship between teacher knowledge, assessment practice, and student learning. Based on data from 39 teachers, we found that students whose teachers frequently analyzed and provided feedback on student work had higher achievement than students whose teachers spent less time on such activities. Our findings support other research indicating the power of wellimplemented formative assessment to improve learning.
PsycEXTRA Dataset, 2000
The objective of this study was to investigate how computer-based knowledge mapping could be used... more The objective of this study was to investigate how computer-based knowledge mapping could be used simultaneously as an instructional tool and an assessment tool in a classroom setting. Data are presented that demonstrate how knowledge mapping served as a tool to support, facilitate, promote, and evaluate students' development of understandings in science. In this study, knowledge mapping was (a) integrated into instruction, (b) employed as a repeated measure to capture the ongoing development of ideas, (c) used individually as well as collaboratively, (d) scored according to algorithms that emphasized the recursive and incremental nature of both learning and the development of scientific ideas, and (e) accessed online through the computer. A nonequivalent control group design was utilized. Fifty-two fourth-and fifth-grade students from two intact classrooms participated in this study. Both groups generated pretest and posttest knowledge maps of their understandings of the digestive, respiratory, and circulatory systems. Students in the experimental group created three additional (collaborative) maps during the course of instruction. Students in the control condition worked in small groups on three occasions to research the human body by using the Internet and related instructional materials. Results suggest that for students in the experimental group, repeated use of the mapping software supported and facilitated the development of scientific and principled understandings. Further, collaborative work with the mapper also afforded students the opportunity to establish connections between the systems in the human body to more fully develop their understandings of the domain-both integral components of learning and the development of scientific understandings. Overview of the Study The search continues for new and better ways to understand how children learn. We acknowledge the complexity of human development, yet measures that capture the complex and interactive nature of the growth of ideas and conceptual 1 We wish to acknowledge and thank Joanne Michiuye and Ali Abedi of UCLA/CRESST for their invaluable technical support. We also wish to thank Ekow Sey, Jeff Higa, Andrew Shpall, Caroline Jeffries and P. J. Osmundson for their content expertise. A special thanks to Uyen Bui for her interest, participation, and content expertise. Finally we are deeply grateful to Ms. Sharon Sutton, Ms. Jan Cohn, and their students for their assistance and participation in this research.
National Center for Research on Evaluation, Standards, and Student Testing, 2010
This study examines measures of formative assessment practice using data from a study of the impl... more This study examines measures of formative assessment practice using data from a study of the implementation and effects of adding curriculum embedded measures to a handson science program for upper elementary school students. The authors present a unifying conception for measuring critical elements of formative assessment practice, illustrate common measures for doing so, and investigate the relationships among and between scores on these measures. Findings raise important issues with regard to both the challenge of obtaining valid measures of teachers’ assessment practice and the uneven quality nature of current teacher practice.
National Center For Research on Evaluation Standards and Student Testing, Mar 1, 2005
This evaluation report summarizes Year 4 of the Math and Science Academy (MSA), an initiative of ... more This evaluation report summarizes Year 4 of the Math and Science Academy (MSA), an initiative of the Northern New Mexico Council on Excellence in Education (NNMCEE). The report begins with an overview of the project and its objectives, and then outlines the research questions and methods used to carry out the evaluation. Findings from the Year 4 evaluation are presented next; the report concludes with recommendations and refinements for future years of MSA. MSA Project Goals and Objectives The Math and Science Academy is an ongoing project geared toward the development of quality teaching and learning to support student achievement. It is a collaborative effort, developed jointly by the Northern New Mexico Council on Excellence in Education (NNMCEE) and local school districts (Chama, Española, and Mora), the Northern Network for Rural Education,
National Center For Research on Evaluation Standards and Student Testing, 2006
Theory and research suggest the critical role that formative assessment can play in student learn... more Theory and research suggest the critical role that formative assessment can play in student learning. The use of assessment in guiding instruction has long been advocated: Through the assessment of students' needs and the monitoring of student progress, learning sequences can be appropriately designed, instruction adjusted during the course of learning, and programs refined to be more effective in promoting student learning goals. Moving toward more modern pedagogical conceptions, assessment moves from an information source on which to base action to part and parcel of the teaching and learning process. The following study provides food for thought about the research methods needed to study teachers' assessment practices and the complexity of assessing their effects on student learning. On the one hand, our study suggests that effective formative assessment is a highly interactive endeavor, involving the orchestration of multiple dimensions of practice, and demands sophisticated qualitative methods for study. On the other, detecting and understanding learning effects in small samples, even with the availability of comparison groups, poses difficulties to say the least. Long-standing theory and research suggest the critical role that formative assessment can play in student learning. With roots in Ralph Tyler's curriculum rationale (1949), B.F. Skinner's behaviorism and programmed instruction (1953, 1960),
National Center For Research on Evaluation Standards and Student Testing, Dec 1, 2006
This article reports findings on growth in three science teachers' expertise with interpretation ... more This article reports findings on growth in three science teachers' expertise with interpretation of student work over 1 year of participation in a program. The program was designed to strengthen classroom assessment in the middle grades. Using a framework for classroom assessment expertise, we analyzed patterns of teacher learning, and the roles of the professional program and the quality of the assessments provided with teachers' instructional materials. The premise of this volume is that formative assessment is a critical component of effective instructional practice. To support student learning, teachers need to gather ongoing evidence of student progress, and use the information to give students helpful feedback and make appropriate adjustments in instruction. The importance of classroom assessment is represented in guides issued by professional organizations
Uploads
Papers by Ellen Osmundson