Papers by Paula Galvao de Barba
One of the great promises of learning analytics is the ability of digital systems to generate mea... more One of the great promises of learning analytics is the ability of digital systems to generate meaningful data about students' learning interactions that can be returned to teachers. If provided in appropriate and timely ways, such data could be used by teachers to inform their current and future teaching practice. In this paper we showcase the learning analytics tool, Loop, which has been developed as part of an Australian Government Office of Learning and Teaching project. The project aimed to develop ways to deliver learning analytics data to academics in a meaningful way to support the enhancement of teaching and learning practice. In this paper elements of the tool will be described. The paper concludes with an outline of the next steps for the project including the evaluation of the effectiveness of the tool.
Springer eBooks, 2020
This is a pre-print of a chapter on the Handbook of Educational Communications and Technology 202... more This is a pre-print of a chapter on the Handbook of Educational Communications and Technology 2020. Personal use is permitted, but it cannot be uploaded in an Open Source repository. The permission from the published must be obtained for any other commercial purpose. This chapter may not exactly replicate the published version due to editorial changes and/or formatting and corrections during the final stage of publication. Interested readers are advised to consult the published version and buy the book.
Using learning analytics to explore help-seeking learner profiles in MOOCs
In online learning environments, learners are often required to be more autonomous in their appro... more In online learning environments, learners are often required to be more autonomous in their approach to learning. In scaled online learning environments, like Massive Open Online Courses (MOOCs), there are differences in the ability of learners to access teachers and peers to get help with their study than in more traditional educational environments. This exploratory study examines the help-seeking behaviour of learners across several MOOCs with different audiences and designs. Learning analytics techniques (e.g., dimension reduction with t-sne and clustering with affinity propagation) were applied to identify clusters and determine profiles of learners on the basis of their help-seeking behaviours. Five help-seeking learner profiles were identified which provide an insight into how learners' help-seeking behaviour relates to performance. The development of a more in-depth understanding of how learners seek help in large online learning environments is important to inform the way support for learners can be incorporated into the design and facilitation of online courses delivered at scale.
Journal of Computer Assisted Learning, Mar 6, 2016
Over the last five years, massive open online courses (MOOCs) have increasingly provided learning... more Over the last five years, massive open online courses (MOOCs) have increasingly provided learning opportunities across the world in a variety of domains. As with many emerging educational technologies, why and how people come to MOOCs needs to be better understood, and importantly what factors contribute to learners' MOOC performance. It is known that online learning environments require greater levels of self-regulation, and that high levels of motivation are crucial to activate these skills. However, motivation is a complex construct and research on how it functions in MOOCs is still in its early stages. Research presented in this article investigated how motivation and participation influence students' performance in a MOOC, more specifically those students who persist to the end of the MOOC. Findings indicated that the strongest predictor of performance was participation, followed by motivation. Motivation influenced and was influenced by students' participation during the course. Moreover, situational interest played a crucial role in mediating the impact of general intrinsic motivation and participation on performance. The results are discussed in relation to how educators and designers of MOOCs can use knowledge emerging from motivational assessments and participation measures gleaned from learning analytics to tailor the design and delivery of courses.
Analytics of What? Negotiating the seduction of big data and learning analytics
In this chapter we focus specifically on the development and impact of the emerging field of lear... more In this chapter we focus specifically on the development and impact of the emerging field of learning analytics – the analysis of student-based data to improve learning and learning environments – and discuss how it, and big data, can be used to address educational problems. We consider the current state of play in learning analytics research and development and examine why big data and learning analytics are seductive. This is followed by a consideration of some of the main challenges facing institutions in harnessing big data and implementing learning analytics and the recommendation that the key focus needs to be on learning. We conclude by suggesting that while big data and analytics have come a fair distance in a short period of time, they still have a way to go. Refocussing research, development and policy on how big data and learning analytics can be used to genuinely support productive student learning processes is one way of negotiating the path ahead.
ASCILITE Publications, Nov 18, 2022
Automatic authorship verification is known to be a challenging machine learning task. In this pap... more Automatic authorship verification is known to be a challenging machine learning task. In this paper, we examine the efficacy of an enhanced common n-gram profile-based approach to assist educational institutions to validate students' essays and assignments through their writing styles. We investigated the impact that essays with different cognitive load requirements have in students' writing styles, which may or may not impact authorship verification methods. A total of 46 undergraduate students completed six essays in a laboratory study. Although results showed small and mixed effects of the tasks differing in cognitive load on the different writing product metrics, students' essays and assignments texts contained features that remained stable across essays requiring different levels of cognitive load. These results suggest that our approach could be successfully used in authorship verification, potentially helping to address issues related to academic integrity in higher education settings.
ASCILITE Publications, Sep 23, 2022
Teaching students to think and act as scientists through inquiry is at the core of recent science... more Teaching students to think and act as scientists through inquiry is at the core of recent science education. Although self-regulated learning (SRL) is acknowledged as crucial to performing scientific inquiry, much is yet to be understood about the specifics of students' interactions with the scientific process. In the current study, we conducted an exploratory investigation of the role of students' SRL and related attitudes when completing an online scientific inquirybased task. A task with a Predict-Observe-Explain learning design was used to examine the role of students' SRL and attitudes within specific phases of the scientific inquiry process. Participants were 233 students from an online undergraduate course. Four groups were identified with differing levels of SRL skills, challenge and confidence. We found that students with low SRL skills who also perceived a learning situation as challenging and had low confidence in their ability to learn, had difficulties designing effective experiments and correctly interpreting data. Implications and future studies are discussed.
Frontiers in Psychology, Aug 1, 2022
ASCILITE Publications, Nov 18, 2022
Student-facing learning analytics dashboards have the potential to reconnect students with their ... more Student-facing learning analytics dashboards have the potential to reconnect students with their purpose for learning, reminding them of their goals and promoting reflection about their learning journey. However, far less is known about the specifics of the relationship between different types of visualisations and data presented in dashboards and their impact on students' motivation. In this study, we used a Human-Centred Design method across three iterations to (1) understand how students prioritise similar visualisations when presenting different data (2) examine how they interact with these, and (3) propose a dashboard design that would accommodate students' different motivational needs. In the first iteration, 26 participants ranked their preferred visualisations using paper prototypes; in the second iteration, a digital wireframe was created based on the results from the first iteration to conduct user tests with two participants; and in the third iteration, a high-fidelity prototype was created to reflect findings from the previous iterations. Overall, findings showed that students mostly valued setting goals and monitoring their progress from a multiple goals approach, and were reluctant about comparing their performance with peers due to concerns related to promoting unproductive competition amongst peers and data privacy. Implications for educators and learning designers are discussed.
Journal of learning Analytics, Dec 13, 2019
Keystroke logging and clickstream data, both emergent areas of study in the field of learning ana... more Keystroke logging and clickstream data, both emergent areas of study in the field of learning analytics, present promising alternative methods of detecting and preventing contract cheating. The current study examines whether analysis of keystroke and clickstream data can detect when a student is creating their own authentic writing or transcribing from another source. Participants were 62 university students (47 women, 15 men) who completed three writing tasks under experimental conditions: free writing, general transcription, and self-transcription. Analyses revealed that while completing the free-writing task, participants typed in shorter bursts with longer pauses and typed more slowly with more revisions compared to the two transcription tasks. Model-based clustering was able to accurately distinguish the free-writing task from the two transcription tasks based on patterns of bursts and writing speed. Overall, these results suggest that keystroke and clickstream analysis may be able to distinguish between a student writing an authentic piece of work and one transcribing a completed work. These findings signal significant implications for the detection of contract cheating. Notes for Practice • Contract cheating is a significant issue for the higher education sector. There is a strong need for reliable methods to detect or prevent contract cheating. Using keystroke logging and clickstream data shows potential for this purpose. • The current study examined whether it is possible to detect whether a student is creating their own work or transcribing from another source. • The results indicate that it is possible to use patterns of writing activity and speed to accurately distinguish between students creating authentic writing and transcribing from another source. • These findings point to a promising approach for detecting contract cheating and plagiarism when students transcribe work from another source.
Australasian Journal of Educational Technology, Mar 28, 2018
Learning Analytics in the Classroom
The field of learning analytics has progressed significantly since the first Learning Analytics a... more The field of learning analytics has progressed significantly since the first Learning Analytics and Knowledge (LAK) conference in 2011. In recent years, the emphasis on technical and statistical aspects of data and analytics has given way to a greater emphasis on what these data mean in the classroom context. This panel session is aimed at examining the emerging role that data and analytics play in understanding and supporting student learning in higher education. Specifically, the panel will focus on the importance of transdisciplinarity and how translation from data to action can occur in the classroom context. The aim of this session is to broaden the conversation about learning analytics within the ASCILITE community. From there, the panel will discuss ways in which learning analytics can have a greater impact on learning design in physical and digital learning environments.
Journal of learning Analytics, 2019
Keystroke logging and clickstream data, both emergent areas of study in the field of learning ana... more Keystroke logging and clickstream data, both emergent areas of study in the field of learning analytics, present promising alternative methods of detecting and preventing contract cheating. The current study examines whether analysis of keystroke and clickstream data can detect when a student is creating their own authentic writing or transcribing from another source. Participants were 62 university students (47 women, 15 men) who completed three writing tasks under experimental conditions: free writing, general transcription, and self-transcription. Analyses revealed that while completing the free-writing task, participants typed in shorter bursts with longer pauses and typed more slowly with more revisions compared to the two transcription tasks. Model-based clustering was able to accurately distinguish the free-writing task from the two transcription tasks based on patterns of bursts and writing speed. Overall, these results suggest that keystroke and clickstream analysis may be able to distinguish between a student writing an authentic piece of work and one transcribing a completed work. These findings signal significant implications for the detection of contract cheating. Notes for Practice • Contract cheating is a significant issue for the higher education sector. There is a strong need for reliable methods to detect or prevent contract cheating. Using keystroke logging and clickstream data shows potential for this purpose. • The current study examined whether it is possible to detect whether a student is creating their own work or transcribing from another source. • The results indicate that it is possible to use patterns of writing activity and speed to accurately distinguish between students creating authentic writing and transcribing from another source. • These findings point to a promising approach for detecting contract cheating and plagiarism when students transcribe work from another source.
Australasian Journal of Educational Technology
Smart learning environments (SLE) provide students with opportunities to interact with learning r... more Smart learning environments (SLE) provide students with opportunities to interact with learning resources and activities in ways that are customised to their particular learning goals and approaches. A challenge in developing SLEs is providing resources and tasks within a single system that can seamlessly tailor learning experience in terms of time, place, platform, and form. In this paper we introduce the iCollab platform, an adaptive environment where learning activities are moderated through conversation with an intelligent agent who can operate across multiple web-based platforms, integrating formal and informal learning opportunities. Fifty-eight undergraduate computer science students were randomly assigned to either an intervention or control group for the 12 weeks of the pilot study. Learning analytics were used to examine their interactions with iCollab, while their course performance investigated the impact of using iCollab on learning outcomes. Results from the study show...
Supporting self-regulated learning with learning analytics
Learning Analytics in the Classroom, 2018
This work aims to characterize students’ writing processes using keystroke logs and understand ho... more This work aims to characterize students’ writing processes using keystroke logs and understand how the extracted characteristics influence the text quality at specific moments of writing. Earlier works have proposed predictive models characterizing students’ writing processes and mainly rely on distribution-based measures of pauses obtained from the overall keystroke logs. However, the effect of isolated phases of writing has not been evaluated in these models. Moreover, current theories on writing suggest that the quality of writing depends on when specific writing behaviours are performed. This view is not examined in the keystroke logging analysis literature. Addressing the mentioned challenges, the two contributions of this work are: a) characterizing students’ writing processes connected to isolated writing phases and examining their influence on writing quality; and b) temporal analysis of keystrokes and investigating whether the significance of writing characteristics varies ...
One of the great promises of learning analytics is the ability of digital systems to generate mea... more One of the great promises of learning analytics is the ability of digital systems to generate meaningful data about students’ learning interactions that can be returned to teachers. If provided in appropriate and timely ways, such data could be used by teachers to inform their current and future teaching practice. In this paper we showcase the learning analytics tool, Loop, which has been developed as part of an Australian Government Office of Learning and Teaching project. The project aimed to develop ways to deliver learning analytics data to academics in a meaningful way to support the enhancement of teaching and learning practice. In this paper elements of the tool will be described. The paper concludes with an outline of the next steps for the project including the evaluation of the effectiveness of the tool.
Uploads
Papers by Paula Galvao de Barba