AI Automarking

2021

This paper investigates how educational technologies that use different combinations of artificial and human intelligence are incorporated into classroom instruction, and how they ultimately affect learning. We conducted a field experiment to study two technologies that allow teachers to outsource grading and feedback tasks on writing practices of high school seniors. The first technology is a fully automated evaluation system that provides instantaneous scores and feedback. The second one uses human graders as an additional resource to enhance grading and feedback quality in aspects in which the automated system arguably falls short. Both technologies significantly improved students' essay scores in a large college admission exam, and the addition of human graders did not improve effectiveness in spite of increasing perceived feedback quality. Both technologies also similarly helped teachers engage more frequently on personal discussions on essay quality with their students. Ta...

Journal of Measurement and Evaluation in Education and Psychology, 2024

In the past few years, as artificial intelligence (AI) and large language models (LLM) have rapidly entered our lives, we have witnessed groundbreaking innovations across numerous fields. The rapid pace of these changes has been met with excitement by some and apprehension by others. However, we all agree that they have made tremendous contributions so far and their contributions in the future will reshape our existence. The field of educational assessment is no exception. With this in mind, we issued a call for a special issue themed “Opportunities and Challenges of AI in Educational Assessment.” which finally included seven distinguished articles on subthemes of fair and responsible use of AI in educational assessment, learning analytics, automated scoring, and real-life examples of AI and LLM.

The growing use of generative AI tools built on large language models (LLMs) calls the sustainability of traditional assessment practices into question. Tools like OpenAI's ChatGPT can generate eloquent essays on any topic and in any language, write code in various programming languages, and ace most standardized tests, all within seconds. We conducted an international survey of educators and students in higher education to understand and compare their perspectives on the impact of generative AI across various assessment scenarios, building on an established framework for examining the quality of online assessments along six dimensions. Across three universities, 680 students and 87 educators, who moderately use generative AI, consider essay and coding assessments to be most impacted. Educators strongly prefer assessments that are adapted to assume the use of AI and encourage critical thinking, while students' reactions are mixed, in part due to concerns about a loss of creativity. The findings show the importance of engaging educators and students in assessment reform efforts to focus on the process of learning over its outputs, alongside higher-order thinking and authentic applications.

International Journal of Electronics Automation, 2024

A proposed AI system is used to grade exams automatically. It addresses inefficiencies in human assessment. A GPT model trained on graded replies is used for evaluation, and TrOCR is used for precise handwritten text recognition. Efficiency and less bias are provided by this method, although there are still issues. More work is needed to assess open-ended questions and make sure they are understandable. To automate many aspects of exam evaluation, including grading, feedback, and plagiarism detection, it first examines the evolution of AI technologies, including machine learning, deep learning, and natural language processing. It also examines the potential for AI-driven assessment tools to enhance learning outcomes, reduce teacher workloads, and provide students with personalized feedback. Additionally, the study highlights several challenges, such as addressing. Our algorithm makes use of developments in two important fields of AI. To reduce bias, careful curation of training data is required. In its conclusion, the study emphasizes how important it is that the system be able to handle different question formats, deal with ambiguities, and incorporate human assessment. A promising first step toward an efficient, equitable, and AI-powered exam grading system is this research.

A Journal for New Zealand Herpetology, 2023

This review paper explores the role of technology in education, with a focus on automation, machine learning, and artificial intelligence. The paper provides an overview of the advantages and disadvantages of automation in education, including its ability to reduce workload for educators and ensure consistency in grading, as well as its potential to limit personalization and human interaction in education.The paper also examines the use of artificial intelligence in education, including its ability to provide personalized learning experiences and automate routine tasks. However, the paper highlights the ethical considerations associated with using AI in education, including concerns around data privacy, transparency, and the potential for AI to perpetuate existing social and economic inequalities.The paper concludes with a comparative analysis of automation, machine learning, and artificial intelligence in education, highlighting the unique benefits and challenges of each approach. The paper also discusses the challenges and ethical considerations associated with implementing these technologies in education, including concerns around bias, data privacy, and equity. A comprehensive overview of the use of technology in education and highlights the need for thoughtful consideration of the potential advantages and disadvantages of each approach, as well as the ethical implications of using these technologies in education.

Education and Information Technologies

There have been giant leaps in the field of education in the past 1-2 years.. Schools and colleges are transitioning online to provide more resources to their students. The COVID-19 pandemic has provided students more opportunities to learn and improve themselves at their own pace. Online proctoring services (part of assessment) are also on the rise, and AI-based proctoring systems (henceforth called as AIPS) have taken the market by storm. Online proctoring systems (henceforth called as OPS), in general, makes use of online tools to maintain the sanctity of the examination. While most of this software uses various modules, the sensitive information they collect raises concerns among the student community. There are various psychological, cultural and technological parameters need to be considered while developing AIPS. This paper systematically reviews existing AI and non-AI-based proctoring systems. Through the systematic search on Scopus, Web of Science and ERIC repositories, 43 paper were listed out from the year 2015 to 2021. We addressed 4 primary research questions which were focusing on existing architecture of AIPS, Parameters to be considered for AIPS, trends and Issues in AIPS and Future of AIPS. Our 360-degree analysis on OPS and AIPS reveals that security issues associated with AIPS are multiplying and are a cause of legitimate concern. Major issues include Security and Privacy concerns, ethical concerns, Trust in AI-based technology, lack of training among usage of technology, cost and many more. It is difficult to know whether the benefits of these Online Proctoring technologies outweigh their risks. The most reasonable conclusion we can reach in the present is that the ethical justification of these technologies and their various capabilities requires us to rigorously ensure that a balance is struck between the concerns with the possible benefits to the best of our abilities. To the best of our knowledge, there is no such analysis on AIPS and OPS. Our work further addresses the issues in AIPS in human and technological aspect. It also lists out key points and new technologies that have only recently been introduced but could significantly impact online education and OPS in the years to come.

Proceedings of the 2002 InSITE Conference

Automated marking of assignments consisting of written text would doubtless be of advantage to teachers and education administrators alike. When large numbers of assignments are submitted at once, teachers find themselves bogged down in their attempt to provide consistent evaluations and high quality feedback to students within as short a timeframe as is reasonable, usually a matter of days rather than weeks. Educational administrators are also concerned with quality and timely feedback, but in addition must manage the cost of doing this work. Clearly an automated system would be a highly desirable addition to the educational tool-kit, particularly if it can provide less costly and more effective outcome. In this paper we present a description and evaluation of four automated essay grading systems. We then report on our trial of one of these systems which was undertaken at Curtin University of Technology in the first half of 2001. The purpose of the trial was to assess whether autom...

Proceedings of the Ninth ACM Conference on Learning @ Scale

This demo will present the implementation of an AI powered assessment and feedback tool-Graide. Attendees will have access to demo grading data which will show the capabilities of the platform. The intent of this project is to reduce assessment and feedback workload at scale. CCS CONCEPTS • Software and its engineering → Software design engineering; • Computing methodologies → Artificial intelligence; Machine learning.

2024

This systematic review explores the transformative role of artificial intelligence (AI) in shaping assessment practices within 21st-century education. It critically examines the integration of AI technologies such as Automated Essay Scoring (AES), adaptive learning systems, and learning analytics, emphasizing their contributions to personalized learning experiences and real-time feedback mechanisms. The review identifies key opportunities for AI to enhance educational assessment, including the automation of scoring and the provision of adaptive feedback. However, it also addresses significant ethical challenges such as algorithmic bias, data privacy, and the need for transparency. We urge policymakers and educators to establish robust ethical guidelines and invest in comprehensive educator training to ensure the responsible use of AI in educational settings. The future directions suggest an increase in the integration of AI technologies, emphasizing the need for ongoing research to enhance validity, reliability, and address ethical considerations in AI-driven assessment practices.

Journal of Excellence , 2022

Artificial Intelligence (AI) technology is to make human life easy and trouble-free and contribute to the advancement of human development. AI is a driving technological force of the twenty-first century and it has been a centre of discussion in technological innovations for its unlimited potential to alter the scenario of social interaction through resolving social challenges and virtually transform every industry. Education is the top priority of present society because it is a fundamental human right that builds peace and drives sustainable development across the world. The integration and application of AI in the classrooms will make teaching and learning effective by supporting teachers and learners in the process through the usage of robotic technology and sensors. AI-based technology facilitates inclusive and equitable quality education along with ensuring universal access to lifelong learning for all across the world. The technology of AI has been advanced and sophisticated that can recognize the gesture of the students and understand their mood and ease during the lecture even it can read facial expressions and posture of the students to understand difficulties and problems they are facing in the lecture and recommends altering the lesson. AI technology-based assessment system can be used to assess students' knowledge, understanding, skills such as collaboration and persistence and characteristics such as confidence and motivation etc. AI technology has developed speech-to-text transcription, predictive text and facial recognition promising an inclusive future for all learners.