The objective of the present study was to establish reliability and validity evidence of a simple... more The objective of the present study was to establish reliability and validity evidence of a simple procedure to measuring conceptual knowledge, the Concept Retrieval Technique (CRT). The CRT procedure requires test takers to freely retrieve concepts from memory they feel are relevant to a given topic, with raters awarding one mark for each correctly retrieved concept when matched against a target word list. Four studies were conducted. In Study 1 (N = 73), inter-rater agreement for the marking procedure of the CRT was determined by means of Cohen's kappa. The kappa was κ = .85. Study 2 (N = 164) explored how consistent inter-rater agreement was across different subjects, age groups, and raters. Mean kappa was κ = .92 suggesting that the CRT can reliably be marked. Study 3 was concerned with establishing the convergent and discriminant validity of the CRT (N = 55). The correlation between the CRT scores and scores on essay-type items was r = .69 and between assessments that measured other ability domains r = .04 to .21 (n.s.). In Study 4, an experiment was conducted to determine the construct validity of the CRT (N = 45). Participants either acquired, or did not acquire, new knowledge of a particular topic and this manipulation was consistently reflected in their CRT scores. The results of the studies suggest that the CRT is a reliable and valid procedure to measure conceptual knowledge. Implications for education are discussed.
Interdisciplinary Journal of Problem-Based Learning, 2017
for additional information. This is an Open Access journal. This means that it uses a funding mod... more for additional information. This is an Open Access journal. This means that it uses a funding model that does not charge readers or their institutions for access. Readers may freely read, download, copy, distribute, print, search, or link to the full texts of articles. This journal is covered under the CC BY-NC-ND license.
The objective of the present study was to establish reliability and validity evidence of a simple... more The objective of the present study was to establish reliability and validity evidence of a simple procedure to measuring conceptual knowledge, the Concept Retrieval Technique (CRT). The CRT procedure requires test takers to freely retrieve concepts from memory they feel are relevant to a given topic, with raters awarding one mark for each correctly retrieved concept when matched against a target word list. Four studies were conducted. In Study 1 (N = 73), inter-rater agreement for the marking procedure of the CRT was determined by means of Cohen's kappa. The kappa was κ = .85. Study 2 (N = 164) explored how consistent inter-rater agreement was across different subjects, age groups, and raters. Mean kappa was κ = .92 suggesting that the CRT can reliably be marked. Study 3 was concerned with establishing the convergent and discriminant validity of the CRT (N = 55). The correlation between the CRT scores and scores on essay-type items was r = .69 and between assessments that measur...
textabstractThe objective of this thesis is to propose a new measure of conceptual knowledge, tha... more textabstractThe objective of this thesis is to propose a new measure of conceptual knowledge, that can be applied in diverse educational environments. The Concept Retrieval Technique will be proposed as a viable alternative to overcome the shortcomings of conventional assessment measures. The Concept Retrieval Technique requires students to freely recall concepts or ideas for a given topic with raters awarding one mark for each correctly recalled concept, when matched against a target word list. The retrieval mechanism in the Concept Retrieval Technique is based on spreading activation theory. Basically, this theory states that from one input concept, activation will spread along the links to other concepts that are connected to it. Then, from each of these concepts to connected others, and so on. The scores for the Concept Retrieval Technique are hypothesized to represent the knowledge network including relevant connections for an individual student. It is developed based on welles...
Context: Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney,... more Context: Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, has utilised Project Based Learning (PBL) in all Key Learning Areas in Years 7-10 for almost a decade. Following the development of an integrative Science, Technology, Engineering and Mathematics (iSTEM) syllabus and its subsequent endorsement by the New South Wales Board of Studies, Teaching, and Educational Standards (BOSTES), it was deemed an opportune course to introduce and deliver utilising a fully PBL approach. The primary aim of the course's introduction in 2015 was to increase engagement and to help improve the communication skills of students as they progress into senior Science and Mathematics courses. Moreover, given constructivist based approaches to learning (like PBL) are growing in popularity within engineering faculties, this also became an opportunity to educate potential young engineers in a way they are likely to encounter at university. Purpose: To illustrate ho...
Research Conference 2021: Excellent progress for every student: Proceedings and Program
Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW h... more Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW has avoided the longer periods of remote learning that our colleagues in Victoria and other countries have experienced, we have nonetheless been provoked to reflect on the nature of schooling and the systemic support we provide to transform the learning of each student and enrich the professional lives of staff within our Catholic learning community. At Catholic Education Diocese of Parramatta (CEDP), a key pillar of our approach is to create conditions that enable everyone to be a leader. Following the initial lockdown period in 2020 when students learned remotely, we undertook an informal teacher voice piece with the purpose of engaging teachers and leaders from across our 80 schools in Greater Western Sydney to reflect on and capture key learnings. This project revealed teachers and leaders reported very high feelings of self-efficacy, motivation and confidence in their capacity to lea...
Interdisciplinary Journal of Problem-Based Learning
for additional information. This is an Open Access journal. This means that it uses a funding mod... more for additional information. This is an Open Access journal. This means that it uses a funding model that does not charge readers or their institutions for access. Readers may freely read, download, copy, distribute, print, search, or link to the full texts of articles. This journal is covered under the CC BY-NC-ND license.
The aim of this paper is to provide an overview of the introduction, implementation, evolution, h... more The aim of this paper is to provide an overview of the introduction, implementation, evolution, hybridization, and initial research into the constructivist instructional models deployed within a secondary (high) school in Australia. A concomitant aim is to relate some of the consequences of whole school pedagogical change, which have included the implementation of project- and problem-based learning, the flipped classroom, and a derivative hybridized form, referred to here as “ flipped PBL.” Moreover, a er a decade of using constructivist approaches, we initiated educational research to better understand some of the effects of these changes and to explore the reasons behind the successful implementation of the models. While still in its infancy, the nature of this research and some of the preliminary findings are detailed here.
CONTEXT Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, ... more CONTEXT Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, has utilised Project Based Learning (PBL) in all Key Learning Areas in Years 7-10 for almost a decade. Following the development of an integrative Science, Technology, Engineering and Mathematics (iSTEM) syllabus and its subsequent endorsement by the New South Wales Board of Studies, Teaching, and Educational Standards (BOSTES), it was deemed an opportune course to introduce and deliver utilising a fully PBL approach. The primary aim of the course's introduction in 2015 was to increase engagement and to help improve the communication skills of students as they progress into senior Science and Mathematics courses. Moreover, given constructivist based approaches to learning (like PBL) are growing in popularity within engineering faculties, this also became an opportunity to educate potential young engineers in a way they are likely to encounter at university. PURPOSE To illustrate how utilising a PBL approach in the delivery of the iSTEM course can enhance student learning. APPROACH To engage students, the NSW BOSTES endorsed course, iSTEM, was packaged into engaging authentic projects utilising the PBL pedagogy. The course was delivered as an elective course in Year 9. Students worked in groups to solve a variety of engineering-based problems in units centred on aerodynamics, coding, mechatronics, hydraulics and pneumatics. In groups, students were involved in the design, testing, fabrication and reporting processes. Emphasis was also placed on kinaesthetic learning with the school investing in a fabrication laboratory (FabLab) containing equipment such as a CNC mill, a laser cutter, vacuum former and 3D printers. Further emphasis was placed on engaging experts and the use of real-world scenarios. To gain some insights into the impact of iSTEM's implementation, students were surveyed and interviewed regarding their levels of engagement and their modes of learning at the midway point of the 2016 course. Teacher observations were also recorded regarding the suitability of iSTEM and PBL and student participation. RESULTS An analysis of student surveys, recorded interviews and teacher observations indicate students are engaged with the mode of learning and that the iSTEM course is well suited to a PBL approach. Some reported benefits of the PBL approach include peer-coaching within groups and development of the ability to grapple with and articulate more complex material to peers in more readily understandable terms. One area requiring attention is the development of Maths and Science resources to complement this approach to learning. Additionally, if the course were to be accommodated into year 10 in the future, there would need to be greater emphasis on material science and other content and skills that may complement the preliminary HSC courses. CONCLUSIONS The iSTEM course is very well suited to PBL but its effect can be maximised by utilising a fully PBL approach in its delivery. Student engagement and satisfaction with iSTEM is very high and there are perceived benefits to students in other related disciplines.
The purpose of this study was to collect and analyse the perceptions of current PBL STEM students... more The purpose of this study was to collect and analyse the perceptions of current PBL STEM students and PBL-educated HSC Graduates (2011-2014) currently studying STEM-related (largely engineering) degrees at university. Conclusions: whilst possibly holding rather naïve ideas of the nature of tertiary STEM education, current students identified a correlation between the engineering profession and their current mode(s) of learning; specifically, how the skills central to the PBL pedagogies align well with the engineering profession and its accreditation processes (with these perceptions being reconfirmed by Alumni). Current students also held higher expectations than alumni (who are more aware of the reality) of the use of PBL in a tertiary setting here and abroad. PBL-educated alumni (graduating from 2011 onwards) are developing greater positive self-efficacy towards Engineers Australia’s (EA) accreditation process which recognises the importance of work-related competencies that are very similar to the soft skills targeted in a PBL environment. Furthermore, and in a wider sense, the findings of this study are supportive of the notion that constructivist approaches to learning might be better able to meet the growing demands from government, industry and peak professional bodies for the teaching and learning of fundamental 21st century ‘employability’ skills.
The objective of the present study was to establish reliability and validity evidence of a simple... more The objective of the present study was to establish reliability and validity evidence of a simple procedure to measuring conceptual knowledge, the Concept Retrieval Technique (CRT). The CRT procedure requires test takers to freely retrieve concepts from memory they feel are relevant to a given topic, with raters awarding one mark for each correctly retrieved concept when matched against a target word list. Four studies were conducted. In Study 1 (N = 73), inter-rater agreement for the marking procedure of the CRT was determined by means of Cohen's kappa. The kappa was κ = .85. Study 2 (N = 164) explored how consistent inter-rater agreement was across different subjects, age groups, and raters. Mean kappa was κ = .92 suggesting that the CRT can reliably be marked. Study 3 was concerned with establishing the convergent and discriminant validity of the CRT (N = 55). The correlation between the CRT scores and scores on essay-type items was r = .69 and between assessments that measured other ability domains r = .04 to .21 (n.s.). In Study 4, an experiment was conducted to determine the construct validity of the CRT (N = 45). Participants either acquired, or did not acquire, new knowledge of a particular topic and this manipulation was consistently reflected in their CRT scores. The results of the studies suggest that the CRT is a reliable and valid procedure to measure conceptual knowledge. Implications for education are discussed.
Interdisciplinary Journal of Problem-Based Learning, 2017
for additional information. This is an Open Access journal. This means that it uses a funding mod... more for additional information. This is an Open Access journal. This means that it uses a funding model that does not charge readers or their institutions for access. Readers may freely read, download, copy, distribute, print, search, or link to the full texts of articles. This journal is covered under the CC BY-NC-ND license.
The objective of the present study was to establish reliability and validity evidence of a simple... more The objective of the present study was to establish reliability and validity evidence of a simple procedure to measuring conceptual knowledge, the Concept Retrieval Technique (CRT). The CRT procedure requires test takers to freely retrieve concepts from memory they feel are relevant to a given topic, with raters awarding one mark for each correctly retrieved concept when matched against a target word list. Four studies were conducted. In Study 1 (N = 73), inter-rater agreement for the marking procedure of the CRT was determined by means of Cohen's kappa. The kappa was κ = .85. Study 2 (N = 164) explored how consistent inter-rater agreement was across different subjects, age groups, and raters. Mean kappa was κ = .92 suggesting that the CRT can reliably be marked. Study 3 was concerned with establishing the convergent and discriminant validity of the CRT (N = 55). The correlation between the CRT scores and scores on essay-type items was r = .69 and between assessments that measur...
textabstractThe objective of this thesis is to propose a new measure of conceptual knowledge, tha... more textabstractThe objective of this thesis is to propose a new measure of conceptual knowledge, that can be applied in diverse educational environments. The Concept Retrieval Technique will be proposed as a viable alternative to overcome the shortcomings of conventional assessment measures. The Concept Retrieval Technique requires students to freely recall concepts or ideas for a given topic with raters awarding one mark for each correctly recalled concept, when matched against a target word list. The retrieval mechanism in the Concept Retrieval Technique is based on spreading activation theory. Basically, this theory states that from one input concept, activation will spread along the links to other concepts that are connected to it. Then, from each of these concepts to connected others, and so on. The scores for the Concept Retrieval Technique are hypothesized to represent the knowledge network including relevant connections for an individual student. It is developed based on welles...
Context: Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney,... more Context: Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, has utilised Project Based Learning (PBL) in all Key Learning Areas in Years 7-10 for almost a decade. Following the development of an integrative Science, Technology, Engineering and Mathematics (iSTEM) syllabus and its subsequent endorsement by the New South Wales Board of Studies, Teaching, and Educational Standards (BOSTES), it was deemed an opportune course to introduce and deliver utilising a fully PBL approach. The primary aim of the course's introduction in 2015 was to increase engagement and to help improve the communication skills of students as they progress into senior Science and Mathematics courses. Moreover, given constructivist based approaches to learning (like PBL) are growing in popularity within engineering faculties, this also became an opportunity to educate potential young engineers in a way they are likely to encounter at university. Purpose: To illustrate ho...
Research Conference 2021: Excellent progress for every student: Proceedings and Program
Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW h... more Internationally, the COVID-19 pandemic has profoundly disrupted the education sector. While NSW has avoided the longer periods of remote learning that our colleagues in Victoria and other countries have experienced, we have nonetheless been provoked to reflect on the nature of schooling and the systemic support we provide to transform the learning of each student and enrich the professional lives of staff within our Catholic learning community. At Catholic Education Diocese of Parramatta (CEDP), a key pillar of our approach is to create conditions that enable everyone to be a leader. Following the initial lockdown period in 2020 when students learned remotely, we undertook an informal teacher voice piece with the purpose of engaging teachers and leaders from across our 80 schools in Greater Western Sydney to reflect on and capture key learnings. This project revealed teachers and leaders reported very high feelings of self-efficacy, motivation and confidence in their capacity to lea...
Interdisciplinary Journal of Problem-Based Learning
for additional information. This is an Open Access journal. This means that it uses a funding mod... more for additional information. This is an Open Access journal. This means that it uses a funding model that does not charge readers or their institutions for access. Readers may freely read, download, copy, distribute, print, search, or link to the full texts of articles. This journal is covered under the CC BY-NC-ND license.
The aim of this paper is to provide an overview of the introduction, implementation, evolution, h... more The aim of this paper is to provide an overview of the introduction, implementation, evolution, hybridization, and initial research into the constructivist instructional models deployed within a secondary (high) school in Australia. A concomitant aim is to relate some of the consequences of whole school pedagogical change, which have included the implementation of project- and problem-based learning, the flipped classroom, and a derivative hybridized form, referred to here as “ flipped PBL.” Moreover, a er a decade of using constructivist approaches, we initiated educational research to better understand some of the effects of these changes and to explore the reasons behind the successful implementation of the models. While still in its infancy, the nature of this research and some of the preliminary findings are detailed here.
CONTEXT Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, ... more CONTEXT Parramatta Marist High, a Catholic systemic comprehensive boys school in Western Sydney, has utilised Project Based Learning (PBL) in all Key Learning Areas in Years 7-10 for almost a decade. Following the development of an integrative Science, Technology, Engineering and Mathematics (iSTEM) syllabus and its subsequent endorsement by the New South Wales Board of Studies, Teaching, and Educational Standards (BOSTES), it was deemed an opportune course to introduce and deliver utilising a fully PBL approach. The primary aim of the course's introduction in 2015 was to increase engagement and to help improve the communication skills of students as they progress into senior Science and Mathematics courses. Moreover, given constructivist based approaches to learning (like PBL) are growing in popularity within engineering faculties, this also became an opportunity to educate potential young engineers in a way they are likely to encounter at university. PURPOSE To illustrate how utilising a PBL approach in the delivery of the iSTEM course can enhance student learning. APPROACH To engage students, the NSW BOSTES endorsed course, iSTEM, was packaged into engaging authentic projects utilising the PBL pedagogy. The course was delivered as an elective course in Year 9. Students worked in groups to solve a variety of engineering-based problems in units centred on aerodynamics, coding, mechatronics, hydraulics and pneumatics. In groups, students were involved in the design, testing, fabrication and reporting processes. Emphasis was also placed on kinaesthetic learning with the school investing in a fabrication laboratory (FabLab) containing equipment such as a CNC mill, a laser cutter, vacuum former and 3D printers. Further emphasis was placed on engaging experts and the use of real-world scenarios. To gain some insights into the impact of iSTEM's implementation, students were surveyed and interviewed regarding their levels of engagement and their modes of learning at the midway point of the 2016 course. Teacher observations were also recorded regarding the suitability of iSTEM and PBL and student participation. RESULTS An analysis of student surveys, recorded interviews and teacher observations indicate students are engaged with the mode of learning and that the iSTEM course is well suited to a PBL approach. Some reported benefits of the PBL approach include peer-coaching within groups and development of the ability to grapple with and articulate more complex material to peers in more readily understandable terms. One area requiring attention is the development of Maths and Science resources to complement this approach to learning. Additionally, if the course were to be accommodated into year 10 in the future, there would need to be greater emphasis on material science and other content and skills that may complement the preliminary HSC courses. CONCLUSIONS The iSTEM course is very well suited to PBL but its effect can be maximised by utilising a fully PBL approach in its delivery. Student engagement and satisfaction with iSTEM is very high and there are perceived benefits to students in other related disciplines.
The purpose of this study was to collect and analyse the perceptions of current PBL STEM students... more The purpose of this study was to collect and analyse the perceptions of current PBL STEM students and PBL-educated HSC Graduates (2011-2014) currently studying STEM-related (largely engineering) degrees at university. Conclusions: whilst possibly holding rather naïve ideas of the nature of tertiary STEM education, current students identified a correlation between the engineering profession and their current mode(s) of learning; specifically, how the skills central to the PBL pedagogies align well with the engineering profession and its accreditation processes (with these perceptions being reconfirmed by Alumni). Current students also held higher expectations than alumni (who are more aware of the reality) of the use of PBL in a tertiary setting here and abroad. PBL-educated alumni (graduating from 2011 onwards) are developing greater positive self-efficacy towards Engineers Australia’s (EA) accreditation process which recognises the importance of work-related competencies that are very similar to the soft skills targeted in a PBL environment. Furthermore, and in a wider sense, the findings of this study are supportive of the notion that constructivist approaches to learning might be better able to meet the growing demands from government, industry and peak professional bodies for the teaching and learning of fundamental 21st century ‘employability’ skills.
Uploads
Papers by Gavin Hays