Book chapters by Vitomir Kovanovic
Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015
This report forms one part in a series of articles offering an overview of the state of distance,... more This report forms one part in a series of articles offering an overview of the state of distance, online, and blended learning, and positioning them in relation to the emerging domain of digital learning. This particular report focuses on blended learning (BL), referring to the practices that combine (or blend) traditional face-to-face (f2f) learning with online learning (OL). as the concept of BL continues to gain traction in educational settings, researchers are attempting to establish and verify the learning gains it brings. This report seeks to outline the debate regarding BL definitions, pedagogical benefits, and deficiencies that arise in academic studies, and reflect on the future direction for BL. Our critical overview of the state and development of BL is structured to reflect the dominant themes of twenty systematically selected second-order academic studies of BL. This report reviews main findings around such dominant themes as the effectiveness of BL, recommended instructional practices in BL delivery and design, as well as the state of research into BL. The findings suggest that advances in technology have fueled the development of BL from a grassroots practice to an emerging research field. The implementation of BL practices by including both online and f2f modes of delivery positively influence student performance, making BL an attractive educational provision. at present, the field of BL is still dependent on the modes of delivery it is derived from, drawing heavily on OL in both theory and in practice. The field of BL is a dynamically changing area, and much of the critique of the existing research noted here is likely to be rapidly addressed in future work. That being said, a critical overview of the field suggests that it can further mature by adopting a digital learning perspective in its own activities.
Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015
This report is one of a series of reports describing the historical developments and current stat... more This report is one of a series of reports describing the historical developments and current state of distance education, online learning, and blended learning. with the intent of informing future research and practice in the emerging discipline of digital learning, this tertiary study focuses on the history and state of distance education, and the understanding of the large body of empirical research as captured by secondary studies (i.e., meta-analyses and systematic literature reviews). we conducted an automated search for secondary studies in several online digital libraries, and a manual search through Google Scholar and the ten most relevant academic journals. Our search identified 339 secondary studies in the domains of distance education, online learning, and blended learning. of those, 37 secondary studies on distance education research and practice met the selection criteria for final inclusion in our study. Based on the analysis of these secondary sources, three main themes emerged: i) comparison of distance education and traditional classroom instruction, ii) identification of important factors of distance education delivery, and iii) factors of institutional adoption of distance education. our results indicate that distance education, when properly planned, designed, and supported by the appropriate mix of technology and pedagogy, is equivalent to, or in certain scenarios more effective than, traditional face-to-face classroom instruction. This highlights the importance of instructional design and the active role of institutions play in providing support structures for instructors and learners. The implications for future research and practice are discussed.
Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015
This report analyzes findings from research into online learning in order to provide guidelines f... more This report analyzes findings from research into online learning in order to provide guidelines for further research and practice. within this tertiary study, we performed a systematic review of thirty-two second-order studies that address issues of teaching and learning in online settings. From the examination of the studies included in the review, four prominent topics emerged: i) comparison of online learning with the traditional classroom, ii) comparison of various instructional practices within two or more online courses, iii) perspectives of students and instructors regarding learning and teaching in online settings, and iv) adoption of online learning in institutions of higher and adult education. except for showing no significant difference in effectiveness of online learning compared to traditional face-to-face settings, the studies within the first theme also provided directions for further research, necessary to better understand what practices work best in online settings. Our findings further indicate that contemporary research into online learning almost univocally agrees that structured online discussions with clear guidelines and expectations, well-designed courses with interactive content and flexible deadlines, and continuous instructor involvement that includes the provision of individualized, timely, and formative feedback are the most promising approaches to fostering learning in online environments. however, this also implies a more complex role for the instructor in online settings, and a need for research on instructional strategies that would allow for the development of student self-regulatory skills. implications for future research and practice, as well as the position of online learning within the broader aspect of digital learning are further discussed.
Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds) Handbook of Learning Analytics and Educational Data Mining, 2017
The field of learning analytics recently attracted attention from educational practitioners and r... more The field of learning analytics recently attracted attention from educational practitioners and researchers interested in the use of large amounts of learning data for understanding learning process and improving learning and teaching practices. In this chapter, we introduce content analytics – a particular form of learning analytics focused on the analysis of different forms of content related to learning. While several publications provided brief overviews of content analytics, the goal of this chapter is to define content analytics and provide a comprehensive overview of the most important studies in the published literature to date. Given the early stage of the learning analytics field, the focus of this chapter is on the important problems and challenges for which existing content analytics approaches are suitable and have been successfully used in the past. We also reflect on the current trends in content analytics and their position within a broader domain of educational research.
Encyclopedia of Information Science and Technology, Second Edition, 2009
Agent-based systems are one of the most important and exciting areas of research and development ... more Agent-based systems are one of the most important and exciting areas of research and development that emerged in information technology (IT) in the past two decades. In a nutshell, an agent is a computer program that is capable of performing a flexible, autonomous action in typically dynamic and unpredictable domains (Luck, McBurney, Shehory, & Willmott, 2005).
Encyclopedia of Information Communication Technology, 2009
Since the AAAI (http://www.aaai.org) Spring Symposium in 1994, intelligent software agents and ag... more Since the AAAI (http://www.aaai.org) Spring Symposium in 1994, intelligent software agents and agent-based systems became one of the most significant and exciting areas of research and development (R&D) that inspired many scientific and commercial projects. In a nutshell, an agent is a computer program that is capable of performing a flexible, autonomous action in typically dynamic and unpredictable domains (Luck, McBurney, Shehory, & Willmott, 2005). Agents emerged as a response of the IT research community to the new data-processing requirements that traditional computing models and paradigms were increasingly incapable to deal with (e.g., the huge and ever-increasing quantities of available data). Agent-oriented R&D has its roots in different disciplines. Undoubtedly, the main contribution to the field of autonomous agents came from artificial intelligence (AI) which is focused on building intelligent artifacts; and if these artifacts sense and act in some environment, then they can be considered agents (Russell & Norvig, 1995). Also, object-oriented programming (Booch, 2004), concurrent object-based systems (Agha, Wegner, & Yonezawa, 1993), and human-computer interaction (Maes, 1994) are fields that have constantly driven forward the agent R&D in the last few decades.
Journal papers by Vitomir Kovanovic
HERDSA Review of Higher Education, 2019
It has been almost a decade since the emergence of learning analytics, a bricolage field of resea... more It has been almost a decade since the emergence of learning analytics, a bricolage field of research and practice that focuses on understanding and optimising learning and learning environments. Since the initial efforts to make sense of large learning-related datasets, learning analytics has come a long way in developing sophisticated methods for capturing various proxies of learning. Researchers in the field also quickly recognised the necessity to tackle complex and often controversial issues of privacy and ethics when dealing with learner-generated data. Finally, despite huge interests in analytics across various stakeholders – governments, educational institutions, teachers, and learners – learning analytics is still facing many challenges when it comes to broader adoption. This article provides an overview of this journey, critically reflecting on the existing research, providing insights into the recent advances, and discussing the future of the field, positioning learning analytics within the broader agenda of systems thinking as means of advancing its institutional adoption.
IEEE Transactions on Learning Technologies, 2019
Abstract-Learning in computer-mediated setting represents a complex, multidimensional process. ... more Abstract-Learning in computer-mediated setting represents a complex, multidimensional process. This complexity calls for a comprehensive analytical approach that would allow for understanding of various dimensions of learner generated discourse and the structure of the underlying social interactions. Current research, however, primarily focuses on manual or, more recently, supervised methods for discourse analysis. Moreover, discourse and social structures are typically analyzed separately without the use of computational methods that can offer a holistic perspective. This paper proposes an approach that addresses these two challenges i) by using an unsupervised machine learning approach to extract speech acts as representations of knowledge construction processes and finds transition probabilities between speech acts across different messages; and ii) by integrating the use of discovered speech acts to explain the formation of social ties and predicting course outcomes. We extracted six categories of speech acts from messages exchanged in discussion forums of two MOOCs and each category corresponded to knowledge construction processes from well-established theoretical models. We further showed how measures derived from discourse analysis explained the ways how social ties were created that framed emerging social networks. Multiple regression models showed that the combined use of measures derived from discourse analysis and social ties predicted learning outcomes.
Learning and Instruction, 2019
Recent developments in educational technologies have provided a viable solution to the challenges... more Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.
The Internet and Higher Education, 2019
This paper examines the discrete learning strategies employed within a massive open online course... more This paper examines the discrete learning strategies employed within a massive open online course and their relationship to the student learning experience. The theoretical framework centered on the Community of Inquiry model of online education, which outlines the three critical dimensions (presences) of student learning experience: teaching, social, and cognitive presence. The Community of Inquiry survey instrument, administered as the part of the post-course survey, was used to measure student perceived levels of the three presences. Cluster analysis revealed three different groups of students with unique study strategies: limited users, selective users, and broad users. The strategies adopted significantly differed in student use of available tools and resources as well as the perceived levels of cognitive presence. The results also indicate there were significant differences regarding student commitment to learning, motivations and goals for enrolling in a MOOC, as well as goal orientation, approaches to learning, and the use of different study strategies. Implications for research and practice of online learning are further discussed.
Distance Education in China, 2017
In 2011, the phenomenon of MOOCs had swept the world of education and put online education in the... more In 2011, the phenomenon of MOOCs had swept the world of education and put online education in the focus of the public discourse around the world. Although researchers were excited with the vast amounts of MOOC data being collected, the benefits of this data did not stand to the expectations due to several challenges. The analyses of MOOC data are very time-consuming and labor-intensive, and require a highly advanced set of technical skills, often not available to the education researchers. Because of this MOOC data analyses are rarely done before the courses end, limiting the potential of data to impact the student learning outcomes and experience.
In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-in-progress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system’s perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications.
Computers & Education, 2018
This study presents an evaluation of the Community of Inquiry (CoI) survey instrument developed b... more This study presents an evaluation of the Community of Inquiry (CoI) survey instrument developed by Arbaugh et al. (2008) within the context of Massive Open Online Courses (MOOCs). The study reports the results of a reliability analysis and exploratory factor analysis of the CoI survey instrument using the data of 1487 students from five MOOC courses. The findings confirmed the reliability and validity of the CoI survey instrument for the assessment of the key dimensions of the CoI model: teaching presence, social presence, and cognitive presence. Although the CoI survey instrument captured the same latent constructs within the MOOC context as in the Garrison's three-factor model (Garrison et al., 1999), analyses suggested a six-factor model with additional three factors as a better fit to the data. These additional factors were 1) course organization and design (a sub-component of teaching presence), 2) group affectivity (a sub-component of social presence), and 3) resolution phase of inquiry learning (a sub-component of cognitive presence). The emergence of these additional factors revealed that the discrepancies between the dynamics of the traditional online courses and MOOCs affect the student perceptions of the three CoI presences. Based on the results of our analysis, we provide an update to the famous CoI model which captures the distinctive characteristics of the CoI model within the MOOC setting. The results of the study and their implications are further discussed.
Online Learning, 2018
Dual-layer MOOCs are an educational framework designed to create customizable modality pathways t... more Dual-layer MOOCs are an educational framework designed to create customizable modality pathways through a learning experience. The basic premise is to design two framework choices through a course - one that is instructor guided and the other that is student-determined and open. Learners have the option to create their own customized pathway by choosing or combining both modalities as they see fit at any given time in the course. This mixed-methods study sought to understand the patterns that learners engaged in during a course designed with this pathway framework. The results of the quantitative examination of the course activity are presented, as well as the categories and themes that arose from the qualitative research. The results of the analysis indicates that learners value the ability to choose the pathway that they engage the course in. Additional research is needed to improve the technical and design aspects of the framework.
Review of Educational Research, 2018
Despite a surge of empirical work on student participation in online learning environments, the c... more Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis of learning-related constructs used in the prediction and measurement of student engagement and learning outcome. Based on our literature review, we identify current gaps in the research, including a lack of solid frameworks to explain learning in open online setting. Finally, we put forward a novel framework suitable for open online contexts based on a well-established model of student engagement. Our model is intended to guide future work studying the association between contextual factors (i.e., demographic, classroom, and individual needs), student engagement (i.e., academic, behavioral, cognitive, and affective engagement metrics), and learning outcomes (i.e., academic, social, and affective). The proposed model affords further interstudy comparisons as well as comparative studies with more traditional education models.
The International Review of Research in Open and Distributed Learning, 2018
The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has freq... more The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has frequently been contested, particularly when learner interactions are limited to MOOC forums. The establishment of social presence—a perceived sense of somebody being present and " real " —is among the strategies to tackle the challenges of online learning and could be applied in MOOCs. Thus far, social presence in MOOCs has been under-researched. Studies that previously examined social presence in MOOCs did not account for the peculiar nature of open online learning. In contrast to the existing work, this study seeks to understand how learners perceive social presence, and the different nuances of social presence in diverse MOOC populations. In particular, we compare perceptions of social presence across the groups of learners with different patterns of forum participation in three edX MOOCs. The findings reveal substantial differences in how learners with varying forum activity perceive social presence. Perceptions of social presence also differed in courses with the varying volume of forum interaction and duration. Finally, learners with sustained forum activity generally reported higher social presence scores that included low affectivity and strong group cohesion perceptions. With this in mind, this study is significant because of the insights into brings to the current body of knowledge around social presence in MOOCs. The study's findings also raise questions about the effectiveness of transferring existing socio-constructivist constructs into the MOOC contexts.
The Internet and Higher Education, 2018
Interactions between learners are fundamental when implementing connectivist pedagogy. Essentiall... more Interactions between learners are fundamental when implementing connectivist pedagogy. Essentially, the establishment of “connections” among students underpins the learning process. These connections can be interpreted as learners’ available pool of social capital, access to which is leveraged through a distributed learning environment such as a MOOC. This study applies linear mixed models to explore the factors associated with social capital for learners in two connectivist MOOCs. Using Coh-Metrix, a computational linguistics tool, we analyzed interactions distributed via Twitter, blogs and Facebook, and examined how learners’ linguistic characteristics are associated with their social capital distributed within a network. Our analyses show that the language used by learners is related to the creation of ties between them. We also observed the role of media, time and learner activity on the development of social capital. The findings suggest that pedagogical considerations are essential to help learners leverage access to potential social capital in a networked learning context.
Learning: Research and Practice, 2017
The field of learning analytics was founded with the goal to harness vast amounts of data about l... more The field of learning analytics was founded with the goal to harness vast amounts of data about learning collected by the extensive use of technology. After the early formation, the field has now entered the next phase of maturation with a growing community who has an evident impact on research, practice, policy, and decision-making. Although learning analytics is a bricolage field borrowing from many related other disciplines, there is still no systematized model that shows how these different disciplines are pieced together. Existing models and frameworks of learning analytics are valuable in identifying elements and processes of learning analytics, but they insufficiently elaborate on the links with foundational disciplines. With this in mind, this paper proposes a consolidated model of the field of research and practice that is composed of three mutually connected dimensions – theory, design, and data science. The paper defines why and how each of the three dimensions along with their mutual relations is critical for research and practice of learning analytics. Finally, the paper stresses the importance of multi-perspective approaches to learning analytics based on its three core dimensions for a healthy development of the field and a sustainable impact on research and practice.
Journal of Educational and Behavioral Statistics, 2016
In recent years, a wide array of tools have emerged for the purposes of conducting educational da... more In recent years, a wide array of tools have emerged for the purposes of conducting educational data mining (EDM) and/or learning analytics (LA) research. In this article, we hope to highlight some of the most widely used, most accessible, and most powerful tools available for the researcher interested in conducting EDM/LA research. We will highlight the utility that these tools have with respect to common data preprocessing and analysis steps in a typical research project as well as more descriptive information such as price point and user-friendliness. We will also highlight niche tools in the field, such as those used for Bayesian knowledge tracing (BKT), data visualization, text analysis, and social network analysis. Finally, we will discuss the importance of familiarizing oneself with multiple tools—a data analysis toolbox—for the practice of EDM/LA research.
Journal of Learning Analytics, 2015
With the widespread adoption of Learning Management Systems (LMS) and other learning technology, ... more With the widespread adoption of Learning Management Systems (LMS) and other learning technology, large amounts of data – commonly known as trace data – are being recorded and are readily accessible to educational researchers. Among different uses of trace data, it has been extensively used to calculate time that students spent on different learning activities – commonly referred to as student time-on-task. Extracted time-on-task measures are then used to build predictive models of student learning in order to understand and improve learning processes. While time-on-task measures have been extensively used in Learning Analytics research, the details of their estimation are rarely described and the consequences that this process entails are not fully examined. This paper presents findings from two experiments that looked at the different time-on-task estimation methods and how they influence the final research findings. Based on modeling different student performance measures with popular statistical methods in two datasets (one online and one blended), our findings indicate that time-on-task estimation methods play an important role in shaping the final study results. This is particularly true for online setting where the amount of interaction with LMS is typically higher. The primary goal of this paper is to raise awareness and initiate a debate on the important issue of time-on-task estimation within a broader learning analytics community. Finally, the paper provides an overview of commonly adopted time-on-task estimation methods in educational and related research fields.
This paper describes a study that looked at the effects of different teaching presence approaches... more This paper describes a study that looked at the effects of different teaching presence approaches in communities of inquiry, and ways in which student–student online discussions with high levels of cognitive presence can be designed. Specifically, this paper proposes that high-levels of cognitive presence can be facilitated in online courses, based on the community of inquiry model, by building upon existing research in i) self-regulated learning through externally-facilitated regulation scaffolding and ii) computer-supported collaborative learning through role assignment. We conducted a quasi-experimental study in a fully-online course (N = 82) using six offerings of the course. After performing a quantitative content analysis of online discussion transcripts, a multilevel linear modeling analysis showed the significant positive effects of both externally-facilitated regulation scaffolding and role assignment on the level of cognitive presence. Specifically, the results showed that externally-facilitated regulation scaffolding had a higher effect on cognitive presence than extrinsically induced motivation through grades. The results showed the effectiveness of role assignment to facilitate a high-level of cognitive presence. More importantly, the results showed a significant effect of the interaction between externally-facilitated regulation scaffolding and role assignment on cognitive presence. The paper concludes with a discussion of practical and theoretical implications.
Uploads
Book chapters by Vitomir Kovanovic
Journal papers by Vitomir Kovanovic
In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-in-progress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system’s perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications.
In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-in-progress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system’s perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications.
In this paper, we present the results of a study that looked at the content of the public discourse related to MOOCs. We identified the most important themes and topics in MOOC-related mainstream news reports. Our results indicate that coverage of MOOCs in public media is rapidly decreasing: by the middle of 2014, it decreased by almost 50% from the highest activity during 2013. In addition, the focus of those discussions is also changing. While the majority of discussions during 2012 and 2013 were focused on MOOC providers, the announcements of their partnerships, and million dollar investments, the current focus of MOOC discourse seems to be moving toward more productive topics focused on the overall position of MOOCs in the global educational landscape. Among different topics that this study discovered, government-related issues and the use of data and analytics are some of the topics that seem to be growing in popularity during the first half of 2014.
In this paper we focus on abstractions and language constructs that define Highway’s approach to integration. We also cover implementation of enterprise integration patterns using Highway since they represent various common situations in enterprise application integration development.
This thesis presents novel methods for understanding and assessing the levels of cognitive presence based on learning analytics techniques and the data collected by learning environments. We first outline a comprehensive model for cognitive presence assessment which builds on the well-established evidence-cantered design (ECD) assessment framework. The proposed assessment model provides a foundation of the thesis, showing how the developed analytical models and their components fit together and how they can be adjusted for new learning contexts. The thesis shows two distinct and complementary analytical methods for assessing students’ cognitive presence and its development. The first method is based on the automated classification of student discussion messages and captures learning as it is observed in the student dialogue. The second analytics method relies on the analysis of log data of students’ use of the learning platform and captures the individual dimension of the learning process. The developed analytics also extend current theoretical understanding of the cognitive presence construct through data-informed operationalization of cognitive presence with different quantitative measures extracted from the student use of online discussions.
We also examine methodological challenges of assessing cognitive presence and other forms of cognitive engagement through the analysis of trace data. Finally, with the intent of enabling for the wider adoption of the CoI model for new online learning modalities, the last two chapters examine the use of developed analytics within the context of Massive Open Online Courses (MOOCs). Given the substantial differences between traditional online and MOOC contexts, we first evaluate the suitability of the CoI model for MOOC settings and then assess students’ cognitive presence using the data collected by the MOOC platform. We conclude the thesis with the discussion of practical application and impact of the present work and the directions for the future research.