This report presents findings from testing approximately 650 children in two cohorts at the begin... more This report presents findings from testing approximately 650 children in two cohorts at the beginning and end of their kindergarten years and from 9 full days of observations of their classrooms. Data were collected in three school districts, of which two had half-day kindergarten programs and one a full-day program. Descriptive findings and results from multiple regression analyses reveal strong relationships between the children's Wide Range Achievement Test scores upon entering kindergarten and their spring scores on three measures of reading performance, the WRAT, the Chicago, and the Woodcock Reading Comprehension paragraphs. There were also substantial and significant differences among districts on these end-of-year measures. No differences were identified for time of day (AM versus PM class) or for the interaction of teacher and time of day. Among the process variables, "confirming feedback" was most frequently significant at the teacher level, and teachers in District A collectively accounted for 60% of the significant findings with respect to process variables. These results suggest the importance of what happens in kindergarten classrooms, not merely the length of the school day. They also raise the question of whether differences between the districts at the end of kindergarten will be evident in the early and middle elementary grades.
In order to provide a structure for describing different approaches to testing, five dimensions a... more In order to provide a structure for describing different approaches to testing, five dimensions along which tests may differ are identified. The dimensions are (1) test uses, (2) item generation, (3) item revision, (4) assessment of precision, and (5) validation. Within each dimension, variations are described reflecting Buros' (1977) distinction between differentiation and measurement. These dimensions are used to profile representative tests from the area of reading comprehension. Only standardized, norm-referenced achievement tests, whose uses (dimension 1) emphasize differentiation, were found to have an inference system (dimension 2 through 5) consistent with those intended uses. No tests were found having inference systems consistent with such intended uses as certifying competence, diagnosing strengths and weaknesses, and tracking progress-uses which emphasize the measurement function of tests. Tests constructed using a domain-referenced approach would yield such an inference system and fill some gaps in the array of currently available measures of reading comprehension.
... James L. Wardrop, Reed G. Williams, and Terry F. Hatch Department of Educational Psychology a... more ... James L. Wardrop, Reed G. Williams, and Terry F. Hatch Department of Educational Psychology and College of Medicine University of Illinois ... Recently, Shatzer, DaRosa, et al.7 investigated the effects of two station lengths in an end-of-clerkship surgery examination using ...
A study was conducted to determine how children develop reading ability in first and second grade... more A study was conducted to determine how children develop reading ability in first and second grade. Approximately 315 children from three school districts in the midwest participated. The study began with the development of a heuristic model that guided data collection on measures of entering student ability, home background, home support, classroom instructional processes, and student performance at the end of each grade level. Linear structural models were developed at both grade levels using LISREL to explain variance in students' reading development. As expected, the study found a relationship between entering ability and achievement at the end of each grade level. At the first-grade level, interplay was found between entry-level student performance and teachers' classroom activities. In addition, classroom activities were found to affect some behaviors at home. Once students brought schoolwork home, parents worked with them on it. At the second-grade level, it was found that teachers continued to be affected by students' entering abilities. Teachers gave more sustained feedback to lower performers, and they emphasized letter sounds and background knowledge interactions with these same students, although these behaviors did not show a relationship to students' end-of-year performance. Teachers' behaviors, unfortunately, were also found to be related to students' home backgrounds. In addition, home support activities were not found to be influenced by home background as they had been in the earlier grades. The results are discussed in terms of eight issues that became clear in the examination of both the firstand second-grade findings. Three of these issues are (a) the diminished effects of home background, (b) positive effects for seatwork, and (c) students' abilities as influences on teachers' instructional practices.
Trends for kindergarten though second-grade children were identified from data collected in a lon... more Trends for kindergarten though second-grade children were identified from data collected in a longitudinal study of how children develop science concepts. The study involved approximately 325 children from three school districts. A heuristic model of science learning was developed representing children's entering ability, home background, home support, teaching processes, instructional material characteristics, and end-of-year performance. Data were collected in each area, pooled, then submitted. to LISREL analyses to produce a structural model for science learning at each grade level. Results showed that children's entering ability in science was the best predictor of their end-of-year science learning overall. The number of hours fathers worked was related to entering ability for kindergarten children, with fathers of higher ability children working more hours. Mothers' education level had by far the highest loadings at all grade levels; mothers' occupation, the lowest. Children's participation in science-related home activities made the greatest contribution to end-of-year performance for all grades, whereas experiences with adults and the number of books and magazines in the home affected only kindergartners' performances. No teaching variable had a significant relationship to end-of-year performance of kindergartners, but teachers' use of sustained feedback after children's incorrect responses was a significant factor in first graders' performance. Time spent in science activities and teachers' uses of science application questions contributed significantly to the performance of second graders. Teachers' coverage of content contributed negatively to end-of-year performance for both first and second graders.
A study was conducted to explore how children learn to read in kindergarten. The study employed a... more A study was conducted to explore how children learn to read in kindergarten. The study employed a heuristic model that includes entering ability, home background, instructional processes, home support for literacy development, and measures of student ability at the end of kindergarten. Children were tested, whole-day classroom observations were made, and questionnaires were sent to parents in order to collect information that could be used to develop data-driven linear structural relationships between variables to explain how children develop reading ability. Results showed that children's performances in reading at the end of kindergarten are directly affected by their knowledge of letters when they enter kindergarten, by their active participation in reading activities at home, and by the amount of phonics instruction and word meaning instruction that they receive in school. These results are discussed in terms of what parents and teachers can do within these areas to affect reading development directly. An explanation of the formation of two constructs, one for decoding ability and the other for comprehension ability at the end of kindergarten, is also offered.
With the appearance of the Cronbach, Gleser, Nanda, & Rajaratnam (1972) volume, I thought the fin... more With the appearance of the Cronbach, Gleser, Nanda, & Rajaratnam (1972) volume, I thought the final word had been written about generalizability theory (GT). Nevertheless, I was an enthusiastic reader of Brennan's (1983/1992) monograph, which succeeded in simplifying the Cronbach et al. notation and presenting GT in a form more readily accessible to measurement specialists and advanced students. What makes this newer volume unique is not its treatment of the basics of GT (the earlier monograph is more appropriate for this purpose), but its presentation, in a single location, of standard GT (Chapters 1-6), GT involving unbalanced designs (Chapters 7-8), and multivariate GT (Chapters 9-12). In addition to the 12 chapters and 9 appendices contained in the book, Brennan (1983) has also made a significant contribution through the preparation and distribution of computer software for doing GT analyses: GENOVA, for univariate balanced designs; urGENOVA for a limited set of unbalanced designs; and mGENOVA for univariate or multivariate analyses of either balanced or (some) unbalanced designs. These are described in moderate detail in Appendices F-H of the book, and they are freely available online. (Appendix F is actually a "mini-manual" for GENOVA.) As one who teaches a unit on GT in a graduate course on test theories, I would not use Brennan's book as a text. It is not what I would consider "learner-friendly." Both organizationally and notationally, I do not consider it to be structured for use as a textbook. On the other hand, this book belongs in every measurement specialist's reference collection, as a resource to which to turn for insight into virtually any question about GT, whether it deals with univariate or multivariate applications. There is simply nowhere else to get such a comprehensive presentation of the theory.
Much of the confusion about the notion of reliability was cleared with the publication of Technic... more Much of the confusion about the notion of reliability was cleared with the publication of Technical Recommendations for Psychological Tests and Diagnostic Techniques (American Psychological Association, 1954) and Technical Recommendations for Achievement Tests (American Educational Research Association, 1955). The subsequent publication of Standards for Educational and Psychological Tests and Manuals (American Psychological Association, 1966), while it introduced considerable changes in the conception of validity, redefined reliability in essentially the same terms. The term "reliability" was clarified through the recognition of its several meanings and, consequently, that " 'reliability coefficient' is a generic term referring to various types of evidence; each type of evidence suggesting a different meaning" (APA, 1966, p. 25). Although the explicit definition of a series of alternative conceptions of reliability clarified the issue it did not resolve a number of important questions of interpretation. For example, there is no clear guide for the selection of an appropriate reliability coefficient nor is there any resolution of the problem of interpreting differences in the values of the various coefficients for the same test.
The work upon which this publication is based was performed pursuant to Contract No. 400-81-0030 ... more The work upon which this publication is based was performed pursuant to Contract No. 400-81-0030 of the National Institute of Education. It does not, however, necessarily reflect the views of this agency. The authors are indebted to William W. Cooley and Richard G. Lomax for supplying data and other information necessary for the analyses described in this paper.
The present investigation was conducted to predict children's loneliness and social satisfaction ... more The present investigation was conducted to predict children's loneliness and social satisfaction growth curves from changes in their peer victimization status. Toward this aim, 388 children (193 boys, 195 girls) were interviewed at five points: as children entered kindergarten (in the fall) and spring of kindergarten through third grade. At each assessment, data were gathered on the frequency of children's peer victimization and degree of loneliness and social satisfaction. Groups were formed on the basis of timing and duration of children's victimization status. Hierarchical linear modeling was used to test several hypotheses regarding the nature of victimized children's growth curves. For instance, consistent with the Onset Hypothesis, the trajectories that emerged for children who moved from nonvictim to victim classification showed increasing levels of loneliness and decreasing social satisfaction. In contrast, findings for the Cessation Hypothesis were mixed, which suggests that children moving from victim to nonvictim status do not necessarily evidence significant improvements in loneliness or social satisfaction. The somewhat disparate trajectories that emerged for loneliness and social satisfaction are discussed.
Teaching the test" has been defined in terms of teaching those particular content knowledges or s... more Teaching the test" has been defined in terms of teaching those particular content knowledges or skills needed to answer the test items correctly. Evidence of several sorts examined in this paper clearly indicates that New Century was teaching students in Providence, R.I., the Gates-MacGinitie Reading Test, which was used to assess their vocabulary achievement. The coincidence between vocabulary taught in the instructional package and the vocabulary required to respond correctly to test items on the Gates-MacGinitie was determined to be much greater than could be attributed to chance, and the data showed that the teaching program neerled be only moderately effective to improve substantially student gains in grade-equivalent scores in the test. On the basis of the analyses summarized in the paper, if the instructional materials are only 30 percent effective, scores should average nearly twice those which would normally be found. (MEM)
The research and development reported herein was performed pursuant to a contract with the United... more The research and development reported herein was performed pursuant to a contract with the United States Office of Education, Department of Health, Education, and Welfare, under the provisions of the Cooperative Research Program. Center No. C-03 / Contract OE 5-10-154 WOV.IMPIPMMINWPIPPmw PREFACE A major objective of the Wisconsin Research and Development Center for Cognitive Learning is to develop an environment in local school buildings and systems which facilitates both student learning and research, development, and innovative activities. This report is concerned with the description and evaluation of such facilitative organizations and their activities in, several elementary schools in the Manitowoc Public School system. The report further demonstrates how instructional and supervisory personnel in the public schools, working with personnel at the Center who possess specialized knowledge in various disciplines, cooperate to extend knowledge and improve educational practice through research and development activities. Many people, other than the R & D personnel and unit leaders denoted as authors, contributed their skills in planning, executing, or evaluating the activities reported herein. In the Manitowoc Schools Miss Helen Hoyer, Supervisor of Elementary Education, Dr. Norris Sanders, Supervisor of Educational Research and Directbr of Title III, and Vernon Childs, Assistant Superintendent, generously cave of their time to aid in the planning of the experiments. Professor Herbert J. Klausmeier, Principal Investigator of Project MOT) _'LS, initiated the idea of R & I Units and assumed primary responsibility for the conceptualization of the total R & I program and for the broad implementation strategies in the local schools. Professor Klausmeier wrote the introductory and concluding sections of this report. Mrs. Doris Cook assumed primary responsibility for working with the building personnel during the year. She, Dr. James Wardrop, and Mrs. Mary Quilling served as consultants for the experiments reported. Other Center personnel who assisted in data collection and analyses include Mrs. Barbara Kennedy, Mr. James Bavry, and Mr. Louis Pingel. The authors acknowledge with appreciation the contributions of the above.
This report presents findings from testing approximately 650 children in two cohorts at the begin... more This report presents findings from testing approximately 650 children in two cohorts at the beginning and end of their kindergarten years and from 9 full days of observations of their classrooms. Data were collected in three school districts, of which two had half-day kindergarten programs and one a full-day program. Descriptive findings and results from multiple regression analyses reveal strong relationships between the children's Wide Range Achievement Test scores upon entering kindergarten and their spring scores on three measures of reading performance, the WRAT, the Chicago, and the Woodcock Reading Comprehension paragraphs. There were also substantial and significant differences among districts on these end-of-year measures. No differences were identified for time of day (AM versus PM class) or for the interaction of teacher and time of day. Among the process variables, "confirming feedback" was most frequently significant at the teacher level, and teachers in District A collectively accounted for 60% of the significant findings with respect to process variables. These results suggest the importance of what happens in kindergarten classrooms, not merely the length of the school day. They also raise the question of whether differences between the districts at the end of kindergarten will be evident in the early and middle elementary grades.
In order to provide a structure for describing different approaches to testing, five dimensions a... more In order to provide a structure for describing different approaches to testing, five dimensions along which tests may differ are identified. The dimensions are (1) test uses, (2) item generation, (3) item revision, (4) assessment of precision, and (5) validation. Within each dimension, variations are described reflecting Buros' (1977) distinction between differentiation and measurement. These dimensions are used to profile representative tests from the area of reading comprehension. Only standardized, norm-referenced achievement tests, whose uses (dimension 1) emphasize differentiation, were found to have an inference system (dimension 2 through 5) consistent with those intended uses. No tests were found having inference systems consistent with such intended uses as certifying competence, diagnosing strengths and weaknesses, and tracking progress-uses which emphasize the measurement function of tests. Tests constructed using a domain-referenced approach would yield such an inference system and fill some gaps in the array of currently available measures of reading comprehension.
... James L. Wardrop, Reed G. Williams, and Terry F. Hatch Department of Educational Psychology a... more ... James L. Wardrop, Reed G. Williams, and Terry F. Hatch Department of Educational Psychology and College of Medicine University of Illinois ... Recently, Shatzer, DaRosa, et al.7 investigated the effects of two station lengths in an end-of-clerkship surgery examination using ...
A study was conducted to determine how children develop reading ability in first and second grade... more A study was conducted to determine how children develop reading ability in first and second grade. Approximately 315 children from three school districts in the midwest participated. The study began with the development of a heuristic model that guided data collection on measures of entering student ability, home background, home support, classroom instructional processes, and student performance at the end of each grade level. Linear structural models were developed at both grade levels using LISREL to explain variance in students' reading development. As expected, the study found a relationship between entering ability and achievement at the end of each grade level. At the first-grade level, interplay was found between entry-level student performance and teachers' classroom activities. In addition, classroom activities were found to affect some behaviors at home. Once students brought schoolwork home, parents worked with them on it. At the second-grade level, it was found that teachers continued to be affected by students' entering abilities. Teachers gave more sustained feedback to lower performers, and they emphasized letter sounds and background knowledge interactions with these same students, although these behaviors did not show a relationship to students' end-of-year performance. Teachers' behaviors, unfortunately, were also found to be related to students' home backgrounds. In addition, home support activities were not found to be influenced by home background as they had been in the earlier grades. The results are discussed in terms of eight issues that became clear in the examination of both the firstand second-grade findings. Three of these issues are (a) the diminished effects of home background, (b) positive effects for seatwork, and (c) students' abilities as influences on teachers' instructional practices.
Trends for kindergarten though second-grade children were identified from data collected in a lon... more Trends for kindergarten though second-grade children were identified from data collected in a longitudinal study of how children develop science concepts. The study involved approximately 325 children from three school districts. A heuristic model of science learning was developed representing children's entering ability, home background, home support, teaching processes, instructional material characteristics, and end-of-year performance. Data were collected in each area, pooled, then submitted. to LISREL analyses to produce a structural model for science learning at each grade level. Results showed that children's entering ability in science was the best predictor of their end-of-year science learning overall. The number of hours fathers worked was related to entering ability for kindergarten children, with fathers of higher ability children working more hours. Mothers' education level had by far the highest loadings at all grade levels; mothers' occupation, the lowest. Children's participation in science-related home activities made the greatest contribution to end-of-year performance for all grades, whereas experiences with adults and the number of books and magazines in the home affected only kindergartners' performances. No teaching variable had a significant relationship to end-of-year performance of kindergartners, but teachers' use of sustained feedback after children's incorrect responses was a significant factor in first graders' performance. Time spent in science activities and teachers' uses of science application questions contributed significantly to the performance of second graders. Teachers' coverage of content contributed negatively to end-of-year performance for both first and second graders.
A study was conducted to explore how children learn to read in kindergarten. The study employed a... more A study was conducted to explore how children learn to read in kindergarten. The study employed a heuristic model that includes entering ability, home background, instructional processes, home support for literacy development, and measures of student ability at the end of kindergarten. Children were tested, whole-day classroom observations were made, and questionnaires were sent to parents in order to collect information that could be used to develop data-driven linear structural relationships between variables to explain how children develop reading ability. Results showed that children's performances in reading at the end of kindergarten are directly affected by their knowledge of letters when they enter kindergarten, by their active participation in reading activities at home, and by the amount of phonics instruction and word meaning instruction that they receive in school. These results are discussed in terms of what parents and teachers can do within these areas to affect reading development directly. An explanation of the formation of two constructs, one for decoding ability and the other for comprehension ability at the end of kindergarten, is also offered.
With the appearance of the Cronbach, Gleser, Nanda, & Rajaratnam (1972) volume, I thought the fin... more With the appearance of the Cronbach, Gleser, Nanda, & Rajaratnam (1972) volume, I thought the final word had been written about generalizability theory (GT). Nevertheless, I was an enthusiastic reader of Brennan's (1983/1992) monograph, which succeeded in simplifying the Cronbach et al. notation and presenting GT in a form more readily accessible to measurement specialists and advanced students. What makes this newer volume unique is not its treatment of the basics of GT (the earlier monograph is more appropriate for this purpose), but its presentation, in a single location, of standard GT (Chapters 1-6), GT involving unbalanced designs (Chapters 7-8), and multivariate GT (Chapters 9-12). In addition to the 12 chapters and 9 appendices contained in the book, Brennan (1983) has also made a significant contribution through the preparation and distribution of computer software for doing GT analyses: GENOVA, for univariate balanced designs; urGENOVA for a limited set of unbalanced designs; and mGENOVA for univariate or multivariate analyses of either balanced or (some) unbalanced designs. These are described in moderate detail in Appendices F-H of the book, and they are freely available online. (Appendix F is actually a "mini-manual" for GENOVA.) As one who teaches a unit on GT in a graduate course on test theories, I would not use Brennan's book as a text. It is not what I would consider "learner-friendly." Both organizationally and notationally, I do not consider it to be structured for use as a textbook. On the other hand, this book belongs in every measurement specialist's reference collection, as a resource to which to turn for insight into virtually any question about GT, whether it deals with univariate or multivariate applications. There is simply nowhere else to get such a comprehensive presentation of the theory.
Much of the confusion about the notion of reliability was cleared with the publication of Technic... more Much of the confusion about the notion of reliability was cleared with the publication of Technical Recommendations for Psychological Tests and Diagnostic Techniques (American Psychological Association, 1954) and Technical Recommendations for Achievement Tests (American Educational Research Association, 1955). The subsequent publication of Standards for Educational and Psychological Tests and Manuals (American Psychological Association, 1966), while it introduced considerable changes in the conception of validity, redefined reliability in essentially the same terms. The term "reliability" was clarified through the recognition of its several meanings and, consequently, that " 'reliability coefficient' is a generic term referring to various types of evidence; each type of evidence suggesting a different meaning" (APA, 1966, p. 25). Although the explicit definition of a series of alternative conceptions of reliability clarified the issue it did not resolve a number of important questions of interpretation. For example, there is no clear guide for the selection of an appropriate reliability coefficient nor is there any resolution of the problem of interpreting differences in the values of the various coefficients for the same test.
The work upon which this publication is based was performed pursuant to Contract No. 400-81-0030 ... more The work upon which this publication is based was performed pursuant to Contract No. 400-81-0030 of the National Institute of Education. It does not, however, necessarily reflect the views of this agency. The authors are indebted to William W. Cooley and Richard G. Lomax for supplying data and other information necessary for the analyses described in this paper.
The present investigation was conducted to predict children's loneliness and social satisfaction ... more The present investigation was conducted to predict children's loneliness and social satisfaction growth curves from changes in their peer victimization status. Toward this aim, 388 children (193 boys, 195 girls) were interviewed at five points: as children entered kindergarten (in the fall) and spring of kindergarten through third grade. At each assessment, data were gathered on the frequency of children's peer victimization and degree of loneliness and social satisfaction. Groups were formed on the basis of timing and duration of children's victimization status. Hierarchical linear modeling was used to test several hypotheses regarding the nature of victimized children's growth curves. For instance, consistent with the Onset Hypothesis, the trajectories that emerged for children who moved from nonvictim to victim classification showed increasing levels of loneliness and decreasing social satisfaction. In contrast, findings for the Cessation Hypothesis were mixed, which suggests that children moving from victim to nonvictim status do not necessarily evidence significant improvements in loneliness or social satisfaction. The somewhat disparate trajectories that emerged for loneliness and social satisfaction are discussed.
Teaching the test" has been defined in terms of teaching those particular content knowledges or s... more Teaching the test" has been defined in terms of teaching those particular content knowledges or skills needed to answer the test items correctly. Evidence of several sorts examined in this paper clearly indicates that New Century was teaching students in Providence, R.I., the Gates-MacGinitie Reading Test, which was used to assess their vocabulary achievement. The coincidence between vocabulary taught in the instructional package and the vocabulary required to respond correctly to test items on the Gates-MacGinitie was determined to be much greater than could be attributed to chance, and the data showed that the teaching program neerled be only moderately effective to improve substantially student gains in grade-equivalent scores in the test. On the basis of the analyses summarized in the paper, if the instructional materials are only 30 percent effective, scores should average nearly twice those which would normally be found. (MEM)
The research and development reported herein was performed pursuant to a contract with the United... more The research and development reported herein was performed pursuant to a contract with the United States Office of Education, Department of Health, Education, and Welfare, under the provisions of the Cooperative Research Program. Center No. C-03 / Contract OE 5-10-154 WOV.IMPIPMMINWPIPPmw PREFACE A major objective of the Wisconsin Research and Development Center for Cognitive Learning is to develop an environment in local school buildings and systems which facilitates both student learning and research, development, and innovative activities. This report is concerned with the description and evaluation of such facilitative organizations and their activities in, several elementary schools in the Manitowoc Public School system. The report further demonstrates how instructional and supervisory personnel in the public schools, working with personnel at the Center who possess specialized knowledge in various disciplines, cooperate to extend knowledge and improve educational practice through research and development activities. Many people, other than the R & D personnel and unit leaders denoted as authors, contributed their skills in planning, executing, or evaluating the activities reported herein. In the Manitowoc Schools Miss Helen Hoyer, Supervisor of Elementary Education, Dr. Norris Sanders, Supervisor of Educational Research and Directbr of Title III, and Vernon Childs, Assistant Superintendent, generously cave of their time to aid in the planning of the experiments. Professor Herbert J. Klausmeier, Principal Investigator of Project MOT) _'LS, initiated the idea of R & I Units and assumed primary responsibility for the conceptualization of the total R & I program and for the broad implementation strategies in the local schools. Professor Klausmeier wrote the introductory and concluding sections of this report. Mrs. Doris Cook assumed primary responsibility for working with the building personnel during the year. She, Dr. James Wardrop, and Mrs. Mary Quilling served as consultants for the experiments reported. Other Center personnel who assisted in data collection and analyses include Mrs. Barbara Kennedy, Mr. James Bavry, and Mr. Louis Pingel. The authors acknowledge with appreciation the contributions of the above.
Uploads
Papers by James Wardrop