Videos by Jason Ravitz
Talks by Jason Ravitz
This video lecture introduces the basics of program evaluation with useful resources and steps le... more This video lecture introduces the basics of program evaluation with useful resources and steps leaders can take to increase the utility of evaluation for informing decisions, making judgements, improving programs, and increasing understanding. See comments on video for links to resources.
Based on a successful professional development course developed at Google, participants will learn a four step process for planning effective evaluations. This includes: describing the program (its purpose, who it serves and how it works); defining the evaluation (its particular purpose, focus, audience and context); planning the evaluation (with specific questions, procedures, analyses and reporting strategies) and managing the evaluation (by defining responsibilities, managing time frames and budget).
This was originally presented as part of Google's Education on Air series.
https://educationonair.withgoogle.com/events/2016-dec
Presentation given December 3, 2016. Richmond, CA.
Highlighted Papers by Jason Ravitz
Paper presented at Annual Meetings of the Association for Educational Communications and Technology (AECT), 2022
Research Experiences for Undergraduates (REU) programs at universities, and industry mentoring pr... more Research Experiences for Undergraduates (REU) programs at universities, and industry mentoring programs, often have difficulties evaluating and assuring quality student experiences. Results from evaluations for two universities conducted in 2021 indicate that weekly reflection surveys and dashboards can raise the visibility of issues, and have the potential to improve mentoring and outcomes. This strategy has provided insights for instruction and program evaluation, suggesting both can benefit from collecting ongoing feedback on program challenges, accomplishments and interactions.
aper presented at Annual Meetings of the Association of Educational Communications and Technology (AECT), 2022
This study describes a partnership to teach the Startup Generation entrepreneurship curriculum wh... more This study describes a partnership to teach the Startup Generation entrepreneurship curriculum while measuring the growth of 21st century workforce skills. We use a framework that has proven useful in numerous studies (Hixson, Ravitz & Whisman, 2014) and provide a factor analysis with evidence supporting more recently developed student measures. Analysis of outcomes and processes suggests the curriculum provides meaningful learning opportunities, the measurement tools are helpful, and interactive dashboards can support more effective coaching.
This paper is for educators who wish to measure the impacts of technology in schools. It should b... more This paper is for educators who wish to measure the impacts of technology in schools. It should be of interest to: principals, administrators, and technology-coordinators, technology-using educators, researchers, policy makers, funders, and those who are held accountable for technology integration. The purpose of this document is to help educators communicate effectively about the impacts of their technology use. It presents the “error model” of research as a way to approach evidence about technology impacts and provides examples from several studies.
Buck Institute for Education, Jan 1, 2008
This report highlights characteristics of reform models in general, and New Tech teachers in part... more This report highlights characteristics of reform models in general, and New Tech teachers in particular using questions from BIE's national survey of PBL and high school reform. This report was provided for the New Technology Foundation (now New Technology Network). It starts by comparing teachers in all the reform networks to others in the survey. The second set of findings focus on how New Tech teachers specifically compared to others in the study.
Peabody Journal of Education, Jul 13, 2010
This study describes the status of small school reforms in U.S. high schools and contemplates the... more This study describes the status of small school reforms in U.S. high schools and contemplates theirfuture. It asks how cultural and instructional reforms differ across school reform types. Analysesfocus on indicators of teacher and student culture as well as instructional reforms including project-based learning (PBL) and other inquiry-related practices. Findings are based on data from a nationalsurvey completed by 395 high school teachers who were responsible for and had used PBL in coreacademic subjects. Study participants taught in large, comprehensive high schools; in schools thathad converted to small learning communities; and in newly created small school start-ups. Some of these small schools and conversions were based on a reform model, and others were not. Teachersin reform model schools reported the greatest number of cultural and instructional reforms, followedby teachers in other small schools. Reform models were particularly strong on instructional reformsand student culture. In general, start-up teachers reported more success implementing reforms thanteachers in conversion schools, and teacher culture was reformed much more often than studentculture and instruction. These findings help shed light on how widely practices and conditions havespread throughout the broader small schools movement, and which of these (including extensive PBLuse) only seem to flourish in schools that subscribe to a holistic reform model.
Interdisciplinary Journal of Problem-based Learning, Jan 1, 2009
The articles in this issue effectively summarize three decades of Problem Based Learning (PBL) re... more The articles in this issue effectively summarize three decades of Problem Based Learning (PBL) research. The meta-analysis (Walker & Leary) and meta-synthesis (Strobel & van Barneveld) articles review outcomes of studies conducted from 1976 to 2007 that compared a PBL curriculum to a traditional curriculum. The third article offers a critique of assessments used in these studies. This commentary highlights the conclusions of the articles and their unique contributions to our understanding of PBL and the breadth of its impact. Issues to be addressed in future research are discussed.
How can we accelerate innovation and ensure effective dissemination of knowledge about online lea... more How can we accelerate innovation and ensure effective dissemination of knowledge about online learning resources? This paper advocates strategies that systematically link online professional development with the research, development and diffusion cycle. The systemic approach we describe can accelerate knowledge advancement and help manage change by improving communication among teachers, trainers, developers and researchers. The examples that are provided are set within two funded projects in the United States that led to the development of two distinct but related strategies—the Online Site Evaluation Form for educators (a web-based review form used in workshops) and a six-week online course on technology-supported assessments. Both strategies make it easier to give feedback to developers and offer incentives to do so in ways that help teachers to learn about online resources individually and with colleagues. The examples are discussed with analysis of their strengths and weaknesses in supporting different modes of interaction. We highlight implications for instructional development, professional development, research and knowledge management in online communities.
In N. Sabelli and R. Pea. (Eds.). Uniting people, technology and powerful ideas for learning. Six years of knowledge networking in learning sciences and technologies, 88-95. http://hal.archives-ouvertes.fr/hal-00190623/, 2004
Technology Supported Assessments (TSA) lasted 6 weeks and ran twice, in the spring and summer of ... more Technology Supported Assessments (TSA) lasted 6 weeks and ran twice, in the spring and summer of 2001, as part of a U.S. Department of Education PT3 Catalyst Grant involving the Concord Consortium in Concord, Massachusetts, and the University of Virginia. It was structured around weekly Readings, Activities, and Discussions (RAD) initiated by the instructor. Each week, the class read about and made use of one of these research-based tools and discussed its applicability to their work.
The SYLLABUS with a list of readings and activities is available via the attached links, along with the FULL REPORT from the project.
This study examines the relationships between teacher-reported use of online resources, and prepa... more This study examines the relationships between teacher-reported use of online resources, and preparedness, implementation challenges, and time spent implementing project- or problem-based learning, or approaches that are similar to what we call “PBL” in general. Variables were measured using self-reports from those who teach in reform network high schools that emphasize PBL approaches (n = 166) and those who do not (n = 164). In both school types, technology use was positively related to the amount of PBL use and teacher preparedness. We used path analysis (two-group SEM) to test a model that predicted online technology use in the context of PBL would have a negative relationship to perceived challenges and a positive relationship to more preparedness, and that these would predict time spent using this approach. Control variables included teacher professional engagement, use of interdisciplinary instruction, and school wide emphasis on PBL or inquiry. Data support many of the predicted relationships, including a direct relationship between online feature use and time on PBL for reform network schools. Outside the reform network schools the path from online feature use to time on PBL was unclear, with only indirect effects. These results suggest areas for further investigation to avoid overstating the role of online technologies.
From 2008 to 2010, project-based learning (PBL) was a major focus of the Teacher Leadership Insti... more From 2008 to 2010, project-based learning (PBL) was a major focus of the Teacher Leadership Institute (TLI), undertaken by the West Virginia Department of Education (WVDE), as a method for teaching 21st century skills. Beginning in January 2011, a summative evaluation was conducted to investigate the effect of PBL implementation on teachers’ perceived ability to teach and assess 21st century skills and on student achievement.
Overall, there were substantial and statistically significant effect size differences between teachers who used PBL with extended professional development and other teachers in the sample. Compared with the matching group, the extensively trained PBL-using teachers taught 21st century skills more often and more extensively. This finding applied across the four content areas, in classrooms serving students with a range of performance levels, and whether or not their schools had block scheduling. The study also found that teachers did not feel as successful at assessing the skills as they did teaching them.
Students of these teachers performed no differently on WESTEST 2 than a matched set of students taught by non-PBL-using teachers or teachers who had not received extensive training. Although students of PBL-using teachers did not show WESTEST 2 gains that exceeded a matched group of students, their performance should serve to mitigate the concern among some educators that engaging in PBL will impede standardized test preparation. This study also provided evidence of the potential of PBL to become part of the larger educational landscape by working in different types of schools.
36th Annual Proceedings of the Association for Educational Communications and Technology, Vol 2, pp. 565-572, Nov 1, 2013
Recent growth in use of project based learning in K-12 raises questions that warrant expanded res... more Recent growth in use of project based learning in K-12 raises questions that warrant expanded research efforts. This paper shares results of a survey conducted to assess how researchers are keeping up with PBL use in schools, what topics are being addressed, and to identify opportunities for collaborative investigation. Results are
based on unique features of PBL identified by participants, as well as common features and concerns. This paper also discusses efforts to create an interactive Google Site designed to serve as a shared resource that enables people to update or contribute to this work in the future.
Annual Meeting of the American Educational …, Jan 1, 2002
This paper explores questions about whether there is a positive or negative relationship between ... more This paper explores questions about whether there is a positive or negative relationship between student computer use and achievement, and whether results vary by the amount of school or home computer use. We find that generally there is an inverse relationship between in-school computer use and student achievement. However, there is a positive overall relationship between student achievement and computer proficiency, i.e., reported capability with a variety of software. Importantly, the student software capability is related to use both at school and at home. Because of the differing interpretations that are possible and the importance of the topic, caution is urged. Researchers and policy makers must think carefully when interpreting correlational results between achievement and student technology use measures, regardless of whether the direction of the proposed relationship is positive or negative.
In making investments in Idaho through the Opportunity I initiative, the J.A. & Kathryn Albertson... more In making investments in Idaho through the Opportunity I initiative, the J.A. & Kathryn Albertson Foundation has acknowledged the hopeful promise of technology and also the need to assess the impacts of technology in classrooms. This report interprets data about student and teacher technology use to portray what is happening in Idaho classrooms and to provide guidance for future funding decisions. It brings together several data sets including a statewide Idaho School Technology Inventory, standardized tests of student achievement (ITBS/TAP), and a teacher survey that has been used in national studies of teacher pedagogy and technology use.
We synthesize findings about data sets and draw attention to patterns of technology use and achievement in Idaho. We describe basic patterns of computer use and achievement that suggest higher-achieving schools and students use computers more than lower achieving schools and students. We then draw comparisons between students, teachers and schools that have similar characteristics to see if technology use is associated with student achievement, with other things being equal.
Ravitz, J. and Lake, D. (1996). An authentic tool for teacher: The OII WWW site evaluation form. Proceedings of the FSU/AECT Distance Education Conference, Talahassee, FL. June 20-23, 1996.
A product of the R&D phases of National School Network (http://nsn.musenet.org), the Web Site Eva... more A product of the R&D phases of National School Network (http://nsn.musenet.org), the Web Site Evaluation Form sets out process for providing a series of criteria that teachers, students and community partners can use to evaluate and categorize Web sites and Web-based curriculum offerings for their usefulness in the classroom and overall quality. The tool can be used to develop a database of valued resources by local communities of interest or by groups via the Internet.
The form has been used as a professional development tool by the Online Internet Institute (http://oii.org), in courses at Syracuse University and LeMoyne College, and within a consortium of schools in Central New York (OCM, BOCES). Discovered on the Internet by others, it was subsequently adopted (without our knowledge until reviews started coming in) by teacher educators at Vermont College and Northern Illinois University. This coupled with unsolicited reviews from individuals illustrates the inherent appeal of the project.
Uploads
Videos by Jason Ravitz
Talks by Jason Ravitz
Based on a successful professional development course developed at Google, participants will learn a four step process for planning effective evaluations. This includes: describing the program (its purpose, who it serves and how it works); defining the evaluation (its particular purpose, focus, audience and context); planning the evaluation (with specific questions, procedures, analyses and reporting strategies) and managing the evaluation (by defining responsibilities, managing time frames and budget).
This was originally presented as part of Google's Education on Air series.
https://educationonair.withgoogle.com/events/2016-dec
Presentation given December 3, 2016. Richmond, CA.
Highlighted Papers by Jason Ravitz
The SYLLABUS with a list of readings and activities is available via the attached links, along with the FULL REPORT from the project.
This teacher survey is available for re-use in studies of 21st century teaching and learning. It has demonstrated excellent reliability, improving on reliable measures from previous studies (std. alpha > .90, inter-item correlations > .58). Support for content validity is based on review of existing frameworks and measures. Support for concurrent validity includes strong relationships to time spent using project-based learning.
Overall, there were substantial and statistically significant effect size differences between teachers who used PBL with extended professional development and other teachers in the sample. Compared with the matching group, the extensively trained PBL-using teachers taught 21st century skills more often and more extensively. This finding applied across the four content areas, in classrooms serving students with a range of performance levels, and whether or not their schools had block scheduling. The study also found that teachers did not feel as successful at assessing the skills as they did teaching them.
Students of these teachers performed no differently on WESTEST 2 than a matched set of students taught by non-PBL-using teachers or teachers who had not received extensive training. Although students of PBL-using teachers did not show WESTEST 2 gains that exceeded a matched group of students, their performance should serve to mitigate the concern among some educators that engaging in PBL will impede standardized test preparation. This study also provided evidence of the potential of PBL to become part of the larger educational landscape by working in different types of schools.
based on unique features of PBL identified by participants, as well as common features and concerns. This paper also discusses efforts to create an interactive Google Site designed to serve as a shared resource that enables people to update or contribute to this work in the future.
We synthesize findings about data sets and draw attention to patterns of technology use and achievement in Idaho. We describe basic patterns of computer use and achievement that suggest higher-achieving schools and students use computers more than lower achieving schools and students. We then draw comparisons between students, teachers and schools that have similar characteristics to see if technology use is associated with student achievement, with other things being equal.
The form has been used as a professional development tool by the Online Internet Institute (http://oii.org), in courses at Syracuse University and LeMoyne College, and within a consortium of schools in Central New York (OCM, BOCES). Discovered on the Internet by others, it was subsequently adopted (without our knowledge until reviews started coming in) by teacher educators at Vermont College and Northern Illinois University. This coupled with unsolicited reviews from individuals illustrates the inherent appeal of the project.
Based on a successful professional development course developed at Google, participants will learn a four step process for planning effective evaluations. This includes: describing the program (its purpose, who it serves and how it works); defining the evaluation (its particular purpose, focus, audience and context); planning the evaluation (with specific questions, procedures, analyses and reporting strategies) and managing the evaluation (by defining responsibilities, managing time frames and budget).
This was originally presented as part of Google's Education on Air series.
https://educationonair.withgoogle.com/events/2016-dec
Presentation given December 3, 2016. Richmond, CA.
The SYLLABUS with a list of readings and activities is available via the attached links, along with the FULL REPORT from the project.
This teacher survey is available for re-use in studies of 21st century teaching and learning. It has demonstrated excellent reliability, improving on reliable measures from previous studies (std. alpha > .90, inter-item correlations > .58). Support for content validity is based on review of existing frameworks and measures. Support for concurrent validity includes strong relationships to time spent using project-based learning.
Overall, there were substantial and statistically significant effect size differences between teachers who used PBL with extended professional development and other teachers in the sample. Compared with the matching group, the extensively trained PBL-using teachers taught 21st century skills more often and more extensively. This finding applied across the four content areas, in classrooms serving students with a range of performance levels, and whether or not their schools had block scheduling. The study also found that teachers did not feel as successful at assessing the skills as they did teaching them.
Students of these teachers performed no differently on WESTEST 2 than a matched set of students taught by non-PBL-using teachers or teachers who had not received extensive training. Although students of PBL-using teachers did not show WESTEST 2 gains that exceeded a matched group of students, their performance should serve to mitigate the concern among some educators that engaging in PBL will impede standardized test preparation. This study also provided evidence of the potential of PBL to become part of the larger educational landscape by working in different types of schools.
based on unique features of PBL identified by participants, as well as common features and concerns. This paper also discusses efforts to create an interactive Google Site designed to serve as a shared resource that enables people to update or contribute to this work in the future.
We synthesize findings about data sets and draw attention to patterns of technology use and achievement in Idaho. We describe basic patterns of computer use and achievement that suggest higher-achieving schools and students use computers more than lower achieving schools and students. We then draw comparisons between students, teachers and schools that have similar characteristics to see if technology use is associated with student achievement, with other things being equal.
The form has been used as a professional development tool by the Online Internet Institute (http://oii.org), in courses at Syracuse University and LeMoyne College, and within a consortium of schools in Central New York (OCM, BOCES). Discovered on the Internet by others, it was subsequently adopted (without our knowledge until reviews started coming in) by teacher educators at Vermont College and Northern Illinois University. This coupled with unsolicited reviews from individuals illustrates the inherent appeal of the project.
Although students of PBL-using teachers did not show WESTEST 2 gains that exceeded a matched group of students, their performance should serve to mitigate the concern among some educators that engaging in PBL will impede standardized test preparation."
Many people are finding the AERA paper "Using project based learning to teach 21st century skills: Findings from a statewide initiative" Please help me share the *more useful* FULL REPORT with Instrument instead: https://www.academia.edu/1999374/ "
Work-based learning includes a variety of approaches designed to increase students' career understanding and job skills. It has variously been called experiential learning, experience-based career education, school-to-work initiatives, cooperative education, tech prep, youth apprenticeships, coordinated workforce development, job shadowing, internships, and mentoring.