Papers by Charlotte Sennersten
arXiv (Cornell University), Nov 22, 2021
Surface cracks on buildings, natural walls and underground mine tunnels can indicate serious stru... more Surface cracks on buildings, natural walls and underground mine tunnels can indicate serious structural integrity issues that threaten the safety of the structure and people in the environment. Timely detection and monitoring of cracks is crucial to managing these risks, especially if the systems can be made highly automated through robots. Visionbased crack detection algorithms using deep neural networks have exhibited promise for structured surfaces such as walls or civil engineering tunnels, but little work has addressed highly unstructured environments such as rock cliffs and bare mining tunnels. To address this challenge, this paper presents PointCrack3D, a new 3D-point-cloud-based crack detection algorithm for unstructured surfaces. The method comprises three key components: an adaptive down-sampling method that maintains sufficient crack point density, a DNN that classifies each point as crack or non-crack, and a post-processing clustering method that groups crack points into crack instances. The method was validated experimentally on a new large natural rock dataset, comprising coloured LIDAR point clouds spanning more than 900 m 2 and 412 individual cracks. Results demonstrate a crack detection rate of 97% overall and 100% for cracks with a maximum width of more than 3 cm, significantly outperforming the state of the art. Furthermore, for crossvalidation, PointCrack3D was applied to an entirely new dataset acquired in different location and not used at all in training and shown to detect 100% of its crack instances. We also characterise the relationship between detection performance, crack width and number of points per crack, providing a foundation upon which to make decisions about both practical deployments and future research directions.
Understanding gameplay requires a consideration of basic epistemological questions about the natu... more Understanding gameplay requires a consideration of basic epistemological questions about the nature of understanding. Grounded in a tradition of philosophical hermeneutics, it is possible to approach the understanding of gameplay as a matter of generating mappings to explanatory frameworks in alternative interpretation paradigms. It is especially useful to consider gameplay from perspectives of cognitive science, semiotics, consciousness studies and aesthetics. Each of these approaches provides a different but compatible perspective on understanding play. Integrating these perspectives without losing their differences provides a comprehensive theoretical framework for play analysis
Proceedings of the First International Conference on Underground Mining Technology, 2017
Unmanned aerial vehicles (UAVs) are being employed in a rapidly increasing number of applications... more Unmanned aerial vehicles (UAVs) are being employed in a rapidly increasing number of applications in mining, including the emerging area of mapping underground void spaces such as stopes, which are otherwise inaccessible to humans, automated ground vehicles and survey technologies. Void mapping can provide both visual rock surface and 3D structural information about stopes, supporting more effective planning of ongoing blast designs. Underground stope mapping by UAVs, however, involves overcoming a number of engineering challenges to allow flights beyond operator line-of-sight where there is no global positioning system (GPS), natural or artificial light, or existing communications infrastructure. This paper describes the construction of a UAV sensor suite that uses sound navigation and ranging (SONAR) data to create a rough 3D model of the underground UAV operational environment in real time to provide operators with high situational awareness for beyond line-of-sight operations. The system also provides a backup when dust obscures visual sensors to provide situation awareness and a coarser, but still informative, 3D model of the underground space. Typically, light detection and ranging (LIDAR) systems have superseded SONAR sensors for similar applications. LIDAR is much more accurate than SONAR, but has several disadvantages. SONAR sensor data is sparse, and therefore much easier to process in real time on-board the UAV than LIDAR. The SONAR sensor hardware is also lighter than current LIDAR systems, which is of importance regarding the constrained payload capacity of UAVs. However, the most important factor that makes SONAR stand out in this application is its ability to operate in dusty or smoke-filled environments. The UAV system was tested both above and below-ground using a predefined path with check point locations for the UAV to follow. Due to the lack of GPS, survey points in combination with photogrammetry allowed the UAV's location to be estimated. This allowed the system to be tested to determine how accurate the SONAR data is in comparison with 3D modelling via photogrammetry of images from a separate digital single-lens reflex camera. Comparing the shape of void surfaces determined by photogrammetry with that determined by SONAR provides quantifiable accuracy when the photogrammetry models are used as ground truth data. Above-ground and underground pilot studies have determined that SONAR sensors provide acceptable accuracy compared with modelling via photogrammetry, sufficient to provide effective situational awareness for human operation of the UAV beyond line-of-sight.
Advanced Cognitive Technologies can use cognitive architectures as a basis for higher level reaso... more Advanced Cognitive Technologies can use cognitive architectures as a basis for higher level reasoning in Artificial Intelligence (AI). Adaptive Control of Thought – Rational (ACT-R) is one such cognitive architecture that attempts to replicate aspects of human thought and reasoning. The research reported in this paper has developed an enhancement to ACT-R that will allow greater understanding of the environment the AI is situated in. Former research has shown that humans perform simple mental simulations to predict the outcomes of events when faced with complex physical problems. Inspired by this, the research reported here has developed Predictive ACT-R (PACT-R), based upon integrating a three dimensional (3D) simulation of the AI’s environment to allow it to predict, reason about, and then act on, what is happening, or about to happen, in its environment. Here, it is demonstrated by application in an autonomous squash player that the predictive version of ACT-R achieves significan...
Safety in underground mines is an important aspect for mining companies. Geological failures such... more Safety in underground mines is an important aspect for mining companies. Geological failures such as roof collapse and rockfalls are one of the most fatal safety hazards in an underground environment. A way to prevent such hazards is to detect early signs of geological failures and hence implement safety measures. The presence of surface cracks is one of the early signs of a geological failure and the current method of detecting them is by sending a geotechnical engineer to survey underground tunnels. This is a risky operation due to the unpredictable hazards and harsh underground environment. Deploying a remote vehicle attached with suitable sensors with the ability to autonomously detect early signs, such will mitigate such risk and also assist geologists to interpret massive amount of data quickly. Several vision-based methods to automatically detect cracks in images can be found in the literature, however, no indication of the performance of such methods in the context of underg...
Workshop on the Cognitive Science of Games and Gameplay Craig A. Lindley ([email protected]), ... more Workshop on the Cognitive Science of Games and Gameplay Craig A. Lindley ([email protected]), Charlotte Sennersten ([email protected]) Game Design, Cognition and Artificial Intelligence Research Group Gotland University College and Blekinge Technical College, Cramergatan 3, SE-621 86, Sweden Jussi Holopainen ([email protected]) Nokia Research Centre, Finland Wijnand A. IJsselsteijn ([email protected]) Human-Technology Interaction Group, Department of Technology Management, Technical University of Eindhoven, Nederlands Simon Niedenthal ([email protected]) School of Arts and Communication, Malmo University, Beijerskajen 8, 205 06 Malmo, Sweden Niklas Ravaja ([email protected]) Center for Knowledge and Innovation Research, Helsinki School of Economics, Tammasaarenkatu 3, FIN-00180 Helsinki, Finland Keywords: games; gameplay; ludology. than the rarefied experiments of traditional cognitive psychology. Abstract Cognitive science has always had a strong...
Multimedia Tools and Applications, 2018
This study investigates individuals' cognitive load processing abilities while engaged on a decis... more This study investigates individuals' cognitive load processing abilities while engaged on a decision-making task in serious games, to explore how a substantial cognitive load dominates over the physiological arousal effect on pupil diameter. A serious game was presented to the participants, which displayed the on-line biofeedback based on physiological measurements of arousal. In such dynamic decision-making environment, the pupil diameter was analyzed in relation to the heart rate, to evaluate if the former could be a useful measure of cognitive abilities of individuals. As pupil might reflect both cognitive activity and physiological arousal, the pupillary response will show an arousal effect only when the cognitive demands of the situation are minimal. Evidence shows that in a situation where a substantial level of cognitive activity is required, only that activity will be observable on the pupil diameter, dominating over the physiological arousal effect indicated by the pupillary response. It is suggested that it might be possible to design serious games tailored to the cognitive abilities of an individual player, using the proposed physiological measurements to observe the moment when such dominance occurs.
IEEE Sensors Journal, 2016
For remote characterization of inaccessible underground mine voids, we are developing an Unmanned... more For remote characterization of inaccessible underground mine voids, we are developing an Unmanned Aerial Vehicles (equipped with multiple sensors including cameras) to fly into the mine voids to map their shape, condition, and most importantly mineralization of the surface. X-ray Fluorescence Spectroscopy (XRF) analysis is normally conducted on rock samples in order to detect present elements (that constitutes minerals). Mining company staffs, however, are able to judge rock types based upon visual features alone. This implies there are some associations between XRF signatures and visual features of rocks. Inspired by this, we have developed a machine learning approach to predict the presence of elements in rocks, for inferring probable rock and mineral types, from imaging features. Note that there exists a number of works in the literature for classifying rocks from digital images. However, to the best of our knowledge, limited attempt has been made to find association between digital imaging features and XRF signatures for mineralogy discovery that we have addressed in this paper. The machine learning algorithm is trained offline based on visual imaging and X-ray Fluorescence Spectroscopy (XRF) analysis data of collected rock samples in a laboratory. The imaging features provide the visual cues and XRF data provides information on element presence/concentration. The machine learning algorithm (regression) discovered the non-linear relationship between these feature spaces and was able to predict element presence with high accuracy as evidenced from experimental results.
2015 IEEE SENSORS, 2015
This study investigated the applicability of machine learning algorithms to detect the presence o... more This study investigated the applicability of machine learning algorithms to detect the presence of elements in underground mines from rock surface images, which is proposed as a heuristic classification method inspired by the ability of human geologists to make judgments about the location of ore veins by eye. A regression algorithm was investigated to find associations between image features and X-Ray Fluorescence (XRF) signatures indicating elemental content of the surface and near-surface region of the rocks. A set of image processing algorithms was used to extract color distribution, edge orientation statistics, and texture of the rock surfaces. XRF signatures were obtained from the same samples, providing a semi-quantitative measure of element concentration. The process was performed on a set of 20 rock samples. The regression algorithm was then trained to find a mapping between image features and the semi-quantitative element concentrations (corresponding with XRF peaks). Experimental results demonstrate the potential effectiveness of the proposed approach in the context of a specific ore body.
Abstract Afully developed nomology for the study of games requires the development of explanatory... more Abstract Afully developed nomology for the study of games requires the development of explanatory theoretical constructs associated with ,validating observational techniques. Drawing from cognition sciences, a framework isproposed based upon the integration of schematheory with attention theory. Cognitive task analysis provides a foundation ,for preliminary schema descriptions, which can then be elaborated according to more detailed models of cognitive and attentional processes. The resulting theoryprovides a rich explanatory framework for the cognitive processes underlying game play, as well as detailed hypotheses for the hierarchical structure of pleasures and rewards motivating players. Gameengagem,ent is accounted for as a process of schemaselection or development, while immersion is explained in terms of schema execution. This framework is being developed not only to explain,the substructures of game play, but also to provide schema models that may inform game design processes ...
Key Engineering Materials, 2010
Evaluating the effectiveness of virtual environments as training and analysis systems one must ta... more Evaluating the effectiveness of virtual environments as training and analysis systems one must take into account both strongly and weakly defined measures of visual behaviour and associated experience. The investigation of cross correlations between strongly defined measures of logged gaze behaviours, and weakly defined measures of subjective perceptions of visual behaviour, reveals significant discrepancies. The existence of these discrepancies casts doubt upon the effectiveness of using self-reporting questionnaires to assess training effectiveness. However, making participants aware of these discrepancies can be a potentially powerful method for increasing the effectiveness of training using virtual worlds.
This paper discusses the development of the Aiming Game, a serious game intended to be used as a ... more This paper discusses the development of the Aiming Game, a serious game intended to be used as a tool for training emotion regulation. The game is part of an intervention package designed to support training of financial investors in becoming aware of their emotional states as well as providing them with a toolbox which can be used for training to counteract cognitive biases which may interfere with their trading activities. The paper discusses how such a game can be implemented as well as how it can be effectively evaluated. The evaluation is mostly focused on the effectiveness of the induction of emotional arousal by the game, which is supported by standardized game design methods and patterns.
Cognitive robotics can take advantage of distributed, web-based information as a foundation for c... more Cognitive robotics can take advantage of distributed, web-based information as a foundation for comprehending 3D objects in a 3D scanned world. The proposed CogOnto model makes possible grounding a cognitive computing system with sensor data gathered from diverse and heterogeneous sources, associated with humanly crafted symbolic descriptors. The system supports cognitive embodiment within the totality of an information ecology, and not just within the physical world where an individual robot, essentially a mobile peripheral device, is located. The informed system uses 3D objects as common denominators for shared world comprehension.
not more than 200 words) Playing a commercial PC or consol game is a highly visual activity, rega... more not more than 200 words) Playing a commercial PC or consol game is a highly visual activity, regardless of whether the purpose is entertainment or situated learning as discussed in the Serious Games field. If more information about the visual attention of the player can be recorded and easily analysed, important design information can be extracted. A range of different eyetracking equipment exists on the market and has been used in many studies over the years. However, very few studies describe dynamic stimuli involving the visual interaction of the user/player with a moving 3D object displayed on a computer screen. The reasons for this are that methods and software developed for eyetracking studies of static 2D stimuli are inappropriate for dynamic 3D stimuli, and manual analysis of dynamic 3D visual interaction is extremely time consuming. In order to address this, the authors have developed a software interface between the TobiiTM eyetracking system and the HiFi Game Engine for use in automated logging of dynamic 3D objects of gaze attention. This report describes the verification study performed to assess the performance of this integration between the eyetracker, logging tools and game engine. Detailed analysis shows effective results within the derived accuracy range, which is certainly sufficient for studies from a small scale to large scales necessary for extensive statistical analysis. The work presented in the report has been conducted in collaboration between FOI, Blekinge Institute of Technology and Gotland College.
2009 Conference in Games and Virtual Worlds for Serious Applications, 2009
Gameplay research about experiential phenomena is a challenging undertaking, given the variety of... more Gameplay research about experiential phenomena is a challenging undertaking, given the variety of experiences that gamers encounter when playing and which currently do not have a formal taxonomy, such as flow, immersion, boredom, and fun. These informal terms require a scientific explanation. Ludologists also acknowledge the need to understand cognition, emotion, and goaloriented behavior of players from a psychological perspective by establishing rigorous methodologies. This paper builds upon and extends prior work in an area for which we would like to coin the term "affective ludology." The area is concerned with the affective measurement of player-game interaction. The experimental study reported here investigated different traits of gameplay experience using subjective (i.e., questionnaires) and objective (i.e., psychophysiological) measures. Participants played three Half-Life 2 game level design modifications while measures such as electromyography (EMG), electrodermal activity (EDA) were taken and questionnaire responses were collected. A level designed for combat-oriented flow experience demonstrated significant high-arousal positive affect emotions. This method shows that emotional patterns emerge from different level designs, which has great potential for providing real-time emotional profiles of gameplay that may be generated together with selfreported subjective player experience descriptions.
International Journal of Computer Games Technology, 2008
Schema theory provides a foundation for the analysis of game play patterns created by players dur... more Schema theory provides a foundation for the analysis of game play patterns created by players during their interaction with a game. Schema models derived from the analysis of play provide a rich explanatory framework for the cognitive processes underlying game play, as well as detailed hypotheses for the hierarchical structure of pleasures and rewards motivating players. Game engagement is accounted for as a process of schema selection or development, while immersion is explained in terms of levels of attentional demand in schema execution. However, schemas may not only be used to describe play, but might be used actively as cognitive models within a game engine. Predesigned schema models are knowledge representations constituting anticipated or desired learned cognitive outcomes of play. Automated analysis of player schemas and comparison with predesigned target schemas can provide a foundation for a game engine adapting or tuning game mechanics to achieve specific effects of engag...
Research on financial decision-making shows that traders and investors with high emotion regulati... more Research on financial decision-making shows that traders and investors with high emotion regulation capabilities perform better in trading. But how can the others learn to regulate their emotions? 'Learning by doing' sounds like a straightforward approach. But how can one perform 'learning by doing' when there is no feedback? This problem particularly applies to learning emotion regulation, because learners can get practically no feedback on their level of emotion regulation. Our research aims at providing a learning environment that can help decision-makers to improve their emotion regulation. The approach is based on a serious game with real-time biofeedback. The game is settled in a financial context and the decision scenario is directly linked to the individual biofeedback of the learner's heart rate data. More specifically, depending on the learner's ability to regulate emotions, the decision scenario of the game continuously adjusts and thereby becomes more (or less) difficult. The learner wears an electrocardiogram sensor that transfers the data via Bluetooth to the game. The game itself is evaluated at several levels.
Extending …, 2007
The distinction between implicit and unselfconscious design cultures on one hand and explicit, se... more The distinction between implicit and unselfconscious design cultures on one hand and explicit, self-conscious design cultures on the other provides a principle for interrelating a variety of game design approaches within a coherent game design meta-model. The design ...
Uploads
Papers by Charlotte Sennersten