Final Report: Assessment in Team Games

The Interservice/Industry …, 2004

DARWARS is envisioned to be a virtual training environment in which multiple distributed individuals will interact with each other and with synthetic entities to in order to acquire operational skills. Training in teamwork skills will be a significant component of the DARWARS experience. In particular, DARWARS will provide the capability for training multiple teams of players at the same time, where the teams will interact in a variety of ways, thereby providing training opportunities for a wide variety of skills. Accordingly, DARWARS faces a significant challenge in assessing teamwork skills as players interact with the simulation and with each other, and in providing that assessment to players in the form of coaching during a training session or feedback in an After Action Review (AAR). Assessing the performance of distributed teams in a simulation-based environment faces three major challenges: (1) creating situations in which the relevant teamwork skills are appropriate; (2) measuring behavior in these situations; and (3) providing assessment to learners at the appropriate moment and the appropriate level of detail. This paper will review our methods, discuss the measures most relevant to DARWARS, and present an example of teamwork measurement in a multi-player commercial game scenario designed to teach teamwork skills relevant to military teams.

Computer games and team and individual learning, 2007

Modern multi-player computer games (or multi-player games) show great potential for enabling distributed training of a variety of skills, including effective teamwork, in a variety of simulated environments. We explored the utility of a fantasy-based multi-player game for training teamwork skills. Forty members of the United States Army Infantry participated in the study. The lessons learned indicate that multi-player game-based training systems can elicit teamwork behaviors and provide a viable environment in which those behaviors may be ...

Vocations and Learning, 2014

As teams have become fundamental parts of today's organisations, the need for these teams to function and learn efficiently and effectively is widely emphasised. Also in military contexts team learning is vital. The current article examines team learning behaviour in military teams as it aims to cross-validate a team learning model that was originally developed and tested in an educational context. This team learning model includes several socio-cognitive factors that precede, constitute, and result from team learning behaviour. Findings based on path analyses indicate that psychological safety, social cohesion and group potency are positively related to team learning behaviour in military teams. In addition, team learning behaviour does not only foster the construction of mutually shared cognition and transactive memory systems, but also relates positively to the effectiveness of military teams.

Lecture Notes in Computer Science, 2014

There is not much research on team collaboration in digital entertainment games, nor is there much evidence for the efficacy of game-based team training or the validity of game-based team assessment. This is a shortcoming because of an increasing pervasiveness of serious games in organizational life, e.g. for operational training, management and leadership. Is it possible to establish marked relationships between psychometric constructs that measure 'team composition and performance' and 'analytics' that unobtrusively measure gameplay performance? If so, what are the implications for gamebased team research and assessment? The authors conducted explorative, quasi-experimental (field) experiments with the multiplayer serious game TeamUp. One field experiment was conducted with 150 police officers as part of task-specific two-day team training. Research data were gathered through pre-game and post-game questionnaires on team constructs such as 'psychological safety' and 'team cohesion'. A large quantity of in-game data was logged to construct indicators like 'time needed to complete the task', 'speak time' and 'avoidable mistakes' to measure team performance. The conclusion of the analysis is that 'team cohesion' and 'psychological safety' correlate moderately and significantly with in-game performance indicators. Teams with an unequal individual game performance speak the most, while teams with an equally low or equally high individual performance spend significantly less time speaking. The indicative findings support the need to further develop validated analytics and game-based environments for team research and assessment.

Applied Ergonomics

Highlights  A teamwork assessment protocol was constructed.  Teams undergoing combat training were assessed.  Improvements in teamwork dimensions as a result of training were evident.  The protocol developed may have applicability in further naturalistic settings.

2006

Abstract: We analyzed an extensive data trace of the on-line multi-player first-person-shooter game America's Army to understand the traits of the social and dynamic networks present in the game. Analyses were performed at the player level, team level, and clan level. Statistical analysis methods are used to examine the data at those three levels. In addition, the dynamic social networks of the teams are examined using a variety of social network analysis methods.

Playing video games is a pervasive activity among middle school students. Researchers are now investigating whether such play carries any educational benefits. One aspect of many video games is teamwork. The goal of the present research was to investigate whether playing video games cooperatively with or competitively against others affects later teamwork behavior and team performance. 60 middle school students, all without previous video game playing experience, participated in the present study. They were randomly assigned to one of three experimental conditions: no video game playing, cooperative video game playing, and competitive video game playing. Those in the video game conditions then were taught how to play Modern Warfare 3 and played for two hours. They were randomly paired with another subject from the same condition and either played with or against that person. Two weeks later, all participants were randomly paired with new partners from the same experimental condition and played a Prisoner's Dilemma game where points are awarded to players based on whether they play cooperatively or competitively with each other. Total team scores were tallied after 20 trials. Results showed that those who played Modern Warfare 3 cooperatively scored the highest, while those who played it competitively scored the lowest. Results suggest that video game playing can bolster teamwork and team performance when played cooperatively and impair teamwork and team performance when played competitively.

Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2014

Teamwork is paramount in many modern day career fields. It is important for HF students and professionals to understand the knowledge, skills, and attitudes requisite for excellent teamwork and to grasp the many hurdles that exist in appropriately measuring its major constructs. Although theories can be imparted didactically, and ideas for measurement discussed, conducting team based research continues to be extremely difficult in particular for measuring team behaviors. This article discusses a potential remedy to this issue through the team-based game "Pandemic © ". This game has players work together on teams of 2-4 individuals whose goal is to cure the world from a set of 4 diseases that are eradicating humanity. It forces individuals to be resourceful and work together-and exemplifies many of the behavioral, attitudinal and cognitive components of teamwork. This article will discuss a) the major theories of teamwork that should be taught alongside this game as an in-class activity, b) the KSAs, team processes, and measurable outcomes that Pandemic entails, and c) an example of ways to integrate the game into semester long activities to give students insight into teamwork and teamwork measurement.

Educational Technology Research and Development

Our study examined using a computer-based learning game as a tool to facilitate teaching and learning teamwork skills. The game was applied to an undergraduate level human resource management course at a business school in Finland. We focused on students’ experiences and key learning outcomes of collaborative learning of teamwork skills through the game, and our analysis highlighted two key features of learning outcomes. First, the computer-based learning game promoted students’ self-reflection and evaluation of their individual team roles. Second, although the game taught students the importance of continuous information sharing in teamwork, their evaluation of team outcomes was performance driven; students predominantly took team efficiency as a sign of good teamwork. This case revealed the benefits and challenges of using computer-based learning games as a pedagogical tool. We identified what is required from lecturers and students for the game to be successful, and how team valu...

2023 IEEE Frontiers in Education Conference (FIE), 2023

This paper presents a systematic literature review that examines the enhancement of teamwork through games. The importance of teamwork in various domains is highlighted, emphasizing its role in achieving common goals and fostering effective group efforts. However, teaching teamwork poses challenges, as educators need to address dynamic interactions and develop students' collaborative skills and abilities. Furthermore, the history of games in education is explored, tracing their roots to John Locke's educational theories and their application in facilitating engaging and effective learning experiences. The methodology section outlines the inclusion and exclusion criteria for selecting relevant studies. Peer-reviewed articles published after 2010 were included, focusing on primary research that specifically addresses teamwork. Two electronic databases, Taylor & Francis, PubMed, and IEEE Xplore, were utilized to gather resources, ensuring reliable and abundant data. The data analysis involved a summary and evaluation of published research on enhanced teamwork approaches. A systematic literature review table was constructed to synthesize and summarize the selected studies. The PRISMA model was followed for identifying, assessing eligibility, and including relevant articles in the review. The results section presents key findings from the selected studies, showcasing their study designs, research questions, target populations, domains, and learning outcomes. The studies highlight the potential of teamwork to enhance team performance in games and the effectiveness of game-based training for fostering teamwork. Additionally, computer-based learning games were found to encourage collaborative learning and help students understand the significance of teamwork in achieving desired outcomes. This systematic literature review contributes to the understanding of how games can be used as effective tools for enhancing teamwork. It provides insights into the existing research landscape and identifies gaps and opportunities for further investigation. The findings have implications for educators, trainers, and organizations seeking to foster effective teamwork skills through game-based approaches. This paper can serve as a valuable resource for researchers and practitioners interested in exploring the intersection of teamwork and games in various contexts.