Table 2 Selected Out-of-School Time Performance-Based Assessments at MHS
Related Figures (6)
Figure 1. Continuum of engagement from the learner’s perspective A Google Scholar search of student engagement yielded 2.9 million ar- ticles, 1.3 million from the last decade. Fredricks et al. (2004), acknowl- edging the construct’s “theoretically messy” state, considered engagement a multidimensional, “meta” construct involving a dynamic blend of be- havioral, cognitive, and affective processes that interact and determine a learner’s investment, commitment, and participation. Because these di- mensions may vary in intensity, often defying easy observation because of their interiority within learners, engineering activity settings to foster and evaluate engagement is a complex undertaking. Analysis involved deductive and inductive coding. Preliminary analysis occurred in weekly three-hour meetings to debrief observations and inter- views. For example, while debriefing an interview in which the principal em- phasized OST PBAs (“project-based learning is at the heart [of MHS]”), we sharpened analytic focus on these PBAs as learning sites. Understandings from these meetings informed coding scheme development and a pilot using NVivo software. Following multiple rounds of coding interview tran- scripts, our five-member team examined pairwise coding comparison re- ports within NVivo to determine interrater agreement. We reviewed pair agreement levels per code (10 combinations of pairs possible in a five-mem- ber team) and calculated an overall code average. To refine codebook, we examined codes under 80% agreement. Pairwise single-code agreement averages were averaged to calculate overall team interrater agreement aver- age. By the 10th round, we had achieved a kappa coefficient of .68; accord- ing to Landis and Koch (1977), this score indicates “substantial” interrater agreement.' From this point, we coded transcripts independently, and team members became case experts on different classrooms and PBA events. Illustrations of coded data most relevant for the present study (Appendix B) included two broad bins. First, instructional practices/structures/values included schoolwide instruction, assessment, student engagement, student outcomes, challenge, culture/language, collaboration, dialogic interaction, and civic awareness. Second, organizational practices/structures/values in- cluded professional development, formal structures, informal structures, administrator/teacher roles, and human and social capital.