Papers by Christian Klaes
The Journal of neuroscience : the official journal of the Society for Neuroscience, Jan 4, 2012
Studies in neuroscience, psychology and behavioral economics, 2023
bioRxiv (Cold Spring Harbor Laboratory), May 18, 2023
Decision making has been intensively studied in the posterior parietal cortex in non-human primat... more Decision making has been intensively studied in the posterior parietal cortex in non-human primates on a single neuron level. In humans decision making has mainly been studied with psychophysical tools or with fMRI. Here, we investigated how single neurons from human posterior parietal cortex represent numeric values informing future decisions during a complex two-player game. The tetraplegic study participant was implanted with a Utah electrode array in the anterior intraparietal area (AIP). We played a simplified variant of Black Jack with the participant while neuronal data was recorded. During the game two players are presented with numbers which are added up. Each time a number is presented the player has to decide to proceed or to stop. Once the first player stops or the score reaches a limit the turn passes on to the second player who tries to beat the score of the first player. Whoever is closer to the limit (without overshooting) wins the game. We found that many AIP neurons selectively responded to the face value of the presented number. Other neurons tracked the cumulative score or were selectively active for the upcoming decision of the study participant. Interestingly, some cells also kept track of the opponent's score. Our findings show that parietal regions engaged in hand action control also represent numbers and their complex transformations. This is also the first demonstration of complex economic decisions being possible to track in single neuron activity in human AIP. Our findings show how tight are the links between parietal neural circuits underlying hand control, numerical cognition and complex decision-making.
arXiv (Cornell University), Mar 30, 2023
Objective. Invasive brain-computer interface (BCI) research is progressing towards the realizatio... more Objective. Invasive brain-computer interface (BCI) research is progressing towards the realization of the motor skills rehabilitation of severely disabled patients in the real world. The size of invasively implanted microelectrode arrays and the selection of an efficient online spike sorting algorithm are two key factors that play pivotal roles in the successful decoding of the user intentions. Recently, a very small but dense microelectrode array with 3072 channels was developed and implanted to decode the intention of the user. The process of spike sorting includes the selection of channels that record the spike activity (SA) and determines the SA of different sources (neurons), on selected channels individually. The neural data recorded with dense microelectrode arrays is time-varying and often contaminated with non-stationary noise. Unfortunately, currently available state-of-the-art spike sorting algorithms are incapable of handling the massively increasing amount of time-varying data resulting from the dense microelectrode arrays, which makes the spike sorting one of the fragile components of the online BCI decoding framework. Approach. This study proposed an adaptive and self-organized algorithm for online spike sorting, named as Adaptive SpikeDeep-Classifier (Ada-SpikeDeepClassifier). Our algorithm uses SpikeDeeptector for the channel selection, an adaptive background activity rejector (Ada-BAR) for discarding the background events, and an adaptive spike classifier (Ada-Spike classifier) for classifying the SA of different neural units. By concatenating SpikeDeeptector, Ada-BAR and Ada-Spike classifier, the process of spike sorting is accomplished. Results. The proposed algorithm is evaluated on two different categories of data: a human dataset recorded in our lab, and a publicly available simulated dataset to avoid subjective biases and labeling errors. The proposed Ada-SpikeDeepClassifier outperformed our previously published SpikeDeep-Classifier and eight other spike sorting algorithms. Significance. To the best of our knowledge, the proposed algorithm is the first spike sorting algorithm that automatically learns the abrupt changes in the distribution of noise and SA. The proposed algorithm is artificial neural network-based, which makes it an ideal candidate for its hardware implementation on neuromorphic chips that is also suitable for wearable invasive BCI.
Current Directions in Biomedical Engineering
This paper presents a software-based Python framework for developing future AI-enhanced end-to-en... more This paper presents a software-based Python framework for developing future AI-enhanced end-to-end Brain-Computer-Interfaces (BCI). This framework contains modules from the emulated analogue front-end and from neural signal pre-processing for invasive neural applications. These modules can be assembled into several pipeline versions for evaluation and benchmarking. The aim of this framework is to accelerate the development of BCIs due to system-wide optimizations in order to set the requirements for hardware development without prior knowledge on the basis of accuracy (recall and precision) and latency. In the next step, the pipeline can be optimised for on-chip and embedded execution.
Frontiers in Human Neuroscience
The human brain has been an object of extensive investigation in different fields. While several ... more The human brain has been an object of extensive investigation in different fields. While several studies have focused on understanding the neural correlates of error processing, advances in brain-machine interface systems using non-invasive techniques further enabled the use of the measured signals in different applications. The possibility of detecting these error-related potentials (ErrPs) under different experimental setups on a single-trial basis has further increased interest in their integration in closed-loop settings to improve system performance, for example, by performing error correction. Fewer works have, however, aimed at reducing future mistakes or learning. We present a review focused on the current literature using non-invasive systems that have combined the ErrPs information specifically in a reinforcement learning framework to go beyond error correction and have used these signals for learning.
arXiv (Cornell University), Jun 21, 2022
Objective: Electroencephalography (EEG) and electromyography (EMG) are two noninvasive bio-signal... more Objective: Electroencephalography (EEG) and electromyography (EMG) are two noninvasive bio-signals, which are widely used in human machine interface (HMI) technologies (EEG-HMI and EMG-HMI paradigm) for the rehabilitation of physically disabled people. Successful decoding of EEG and EMG signals into respective control command is a pivotal step in the rehabilitation process. Recently, several Convolutional neural networks (CNNs) based architectures are proposed that directly map the raw time-series (EEG and EMG signal) into decision space (intended action of the user). Since CNNs are end-to-end learning algorithms, the process of meaningful features extraction and classification are performed simultaneously. However, these networks are tailored to the learn the expected characteristics of the given bio-signal. Henceforth, the implication of these algorithms is usually limited to single HMI paradigm. In this work, we addressed the question that can we build a single architecture which is capable of learning distinct features from different HMI paradigms and still successfully classify them. Approach: In this work, we introduce a single hybrid model called ConTraNet, which is based on CNN and Transformer architectures that is equally useful for EEG-HMI and EMG-HMI paradigms. ConTraNet uses CNN block to introduce inductive bias in the model and learn local dependencies, whereas the Transformer block uses the selfattention mechanism to learn the long-range or global dependencies in the signal, which are crucial for the classification of EEG and EMG signals. Main results: We evaluated and compared the ConTraNet with state-of-the-art methods on three publicly available datasets (BCI Competition IV dataset 2b, Physionet MI-EEG dataset, Mendeley sEMG dataset) which belong to EEG-HMI and EMG-HMI paradigms. ConTraNet outperformed its counterparts in all the different category tasks (2-class, 3-class, 4-class, and 10-class decoding tasks). Significance: Most HMI studies introduce the algorithms that are tailored to the characteristics of its expected bio-signal and validate their results on the dataset/s, which belong to only single paradigm. Contrarily, we introduced ConTraNet and validated the results on two different HMI paradigms, which contain the data of 2, 3, 4 and 10-classes. Furthermore, the generalization quality of ConTraNet remains equally good for both paradigms, which suggest that ConTraNet is robust to learn distinct features from different HMI paradigms and generalizes well as compared to the current state of the art algorithms.
We report the presence of a tingling sensation perceived during self touch without physical stimu... more We report the presence of a tingling sensation perceived during self touch without physical stimulation. We used immersive virtual reality scenarios in which subjects touched their body using a virtual object. This touch resulted in a tingling sensation corresponding to the location touched on the virtual body. We called it “phantom touch illusion” (PTI). Interestingly the illusion was also present when subjects touched invisible (inferred) parts of their limb. We reason that this PTI results from tactile gating process during self-touch. The reported PTI when touching invisible body parts indicates that tactile gating is not exclusively based on vision, but rather on multi-sensory, top-down input involving body schema. This finding shows that representations of own body are defined top-down, beyond the available sensory information.
Planning goal-directed movements requires the combination of visuospatial with abstract contextua... more Planning goal-directed movements requires the combination of visuospatial with abstract contextual information. Our sensory environment constrains possible movements to a certain extent. However, contextual information guides proper choice of action in a given situation and allows flexible mapping of sensory instruction cues onto different motor actions. We used anti-reach tasks to test the hypothesis that spatial motor-goal representations in cortical sensorimotor areas are gain modulated by the behavioral context to achieve flexible remapping of spatial cue information onto arbitrary motor goals. We found that gain modulation of neuronal reach goal representations is commonly induced by the behavioral context in individual neurons of both, the parietal reach region (PRR) and the dorsal premotor cortex (PMd). In addition, PRR showed stronger directional selectivity during the planning of a reach toward a directly cued goal (pro-reach) compared with an inferred target (anti-reach). PMd, however, showed stronger overall activity during reaches toward inferred targets compared with directly cued targets. Based on our experimental evidence, we suggest that gain modulation is the computational mechanism underlying the integration of spatial and contextual information for flexible, rule-driven stimulus-response mapping, and thereby forms an important basis of goal-directed behavior. Complementary contextual effects in PRR versus PMd are consistent with the idea that posterior parietal cortex preferentially represents sensory-driven, "automatic" motor goals, whereas frontal sensorimotor areas are stronger engaged in the representation of rule-based, "inferred" motor goals.
Scientific Reports, 2022
Brain-computer interfaces (BCIs) enable communication between humans and machines by translating ... more Brain-computer interfaces (BCIs) enable communication between humans and machines by translating brain activity into control commands. Electroencephalography (EEG) signals are one of the most used brain signals in non-invasive BCI applications but are often contaminated with noise. Therefore, it is possible that meaningful patterns for classifying EEG signals are deeply hidden. State-of-the-art deep-learning algorithms are successful in learning hidden, meaningful patterns. However, the quality and the quantity of the presented inputs are pivotal. Here, we propose a feature extraction method called anchored Short Time Fourier Transform (anchored-STFT), which is an advanced version of STFT, as it minimizes the trade-off between temporal and spectral resolution presented by STFT. In addition, we propose a data augmentation method derived from l2-norm fast gradient sign method (FGSM), called gradient norm adversarial augmentation (GNAA). GNAA is not only an augmentation method but is a...
2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021
Brain-Computer Interface systems can contribute to a vast set of applications such as overcoming ... more Brain-Computer Interface systems can contribute to a vast set of applications such as overcoming physical disabilities in people with neural injuries or hands-free control of devices in healthy individuals. However, having systems that can accurately interpret intention online remains a challenge in this field. Robust and data-efficient decoding-despite the dynamical nature of cortical activity and causality requirements for physical function-is among the most important challenges that limit the widespread use of these devices for real-world applications. Here, we present a causal, data-efficient neural decoding pipeline that predicts intention by first classifying recordings in short sliding windows. Next, it performs weighted voting over initial predictions up to the current point in time to report a refined final prediction. We demonstrate its utility by classifying spiking neural activity collected from the human posterior parietal cortex for a cue, delay, imaginary motor task. This pipeline provides higher classification accuracy than state-of-the-art time windowed spiking activity based causal methods, and is robust to the choice of hyper-parameters. Clinical relevance-We have tested our decoder during delayed imaginary grasp tasks on data from the human posterior parietal cortex-a relatively understudied region of the brain thought to contribute to motor intention. Our results provide new insight into the underlying neural dynamics of this region. In fact, the most discriminating information-and the greatest utility of voting-appear to occur during the early phases of the task. This makes our approach most useful to short-latency control of brain-computer systems such as neuroprosthetics.
Journal of Neural Engineering, 2020
Objective. Recent advancements in electrode designs and micro-fabrication technology has allowed ... more Objective. Recent advancements in electrode designs and micro-fabrication technology has allowed existence of micro-electrode arrays with hundreds of channels for single cell recordings. In such electrophysiological recordings, each implanted micro-electrode can record the activities of more than one neuron in its vicinity. Recording the activities of multiple neurons may also be referred as multiple unit activity (MUA). However, for any further analysis, main goal is to isolate the activity of each recorded neuron and thus called single unit activity (SUA). This process may also be referred as spike sorting or spike classification. Recent approaches to extract SUA are time consuming, mainly due to the requirement of human intervention at various stages of spike sorting pipeline. Lack of standardization is another drawback of the current available approaches. Therefore, in this study we proposed a standard spike sorter "SpikeDeep-Classifier", a fully automatic spike sorting algorithm. Approach. We proposed a novel spike sorting pipeline, based on a set of supervised and unsupervised learning algorithms. We used supervised, deep learning-based algorithms for extracting meaningful channels and removing background activities (noise/artifacts) from the extracted channels. We also showed that the process of clustering becomes straightforward , once the noise/artifact is completely removed from the data. Therefore, in the next stage, we applied a simple clustering algorithm (Kmean) with predefined maximum number of clusters. Lastly, we used a similarity-based criterion to keep the distinct clusters and merge the similar looking clusters. Main results. We evaluated our algorithm on a dataset collected from two different species (humans and non-human primates (NHPs)) without any retraining. Data is recorded from five different brain areas with two different recording hardware and three different electrodes types. We showed that the "SpikeDeep-Classifier" has the potential of being automated and reproducible spike sorting algorithm, universally. We compared the result of algorithm with ground-truth labels. We also validated our algorithm on publicly available labeled dataset. Significance. The results demonstrated that the SpikeDeep-Classifer provides a universal solution to fully automatic offline spike sorting problem. Clinical trial registration number The clinical trial registration number for patients implanted with the Utah array is NCT 01849822.
Brain-computer interfaces (BCIs) enable communication between humans and machines by translating ... more Brain-computer interfaces (BCIs) enable communication between humans and machines by translating brain activity into control commands. Electroencephalography (EEG) signals are one of the most used brain signals in non-invasive BCI applications but are often contaminated with noise. Therefore, it is possible that meaningful patterns for classifying EEG signals are deeply hidden. State-of-the-art deep-learning algorithms are successful in learning hidden, meaningful patterns. However, the quality and the quantity of the presented inputs is pivotal. Here, we propose a novel feature extraction method called anchored Short Time Fourier Transform (anchoredSTFT), which is an advanced version of STFT, as it minimizes the trade-off between temporal and spectral resolution presented by STFT. In addition, we propose a novel augmentation method, called gradient norm adversarial augmentation (GNAA). GNAA is not only an augmentation method but is also used to harness adversarial inputs in EEG dat...
arXiv: Neurons and Cognition, 2018
In electrophysiology, microelectrodes are the primary source for recording neural data of single ... more In electrophysiology, microelectrodes are the primary source for recording neural data of single neurons (single unit activity). These microelectrodes can be implanted individually, or in the form of microelectrodes arrays, consisting of hundreds of electrodes. During recordings, some channels capture the activity of neurons, which is usually contaminated with external artifacts and noise. Another considerable fraction of channels does not record any neural data, but external artifacts and noise. Therefore, an automatic identification and tracking of channels containing neural data is of great significance and can accelerate the process of analysis, e.g. automatic selection of meaningful channels during offline and online spike sorting. Another important aspect is the selection of meaningful channels during online decoding in brain-computer interface applications, where threshold crossing events are usually for feature extraction, even though they do not necessarily correspond to ne...
Proceedings of the 2020 5th International Conference on Big Data and Computing, 2020
Anomaly detection is one of the key issues in the domain of unsupervised machine learning gaining... more Anomaly detection is one of the key issues in the domain of unsupervised machine learning gaining importance in diverse research and application areas. In the presented work, we aim to compare a specifically developed statistical method with a generic sparse autoencoder approach for the anomaly detection problem. To compare performance, we apply both methods to real energy consumption data taken from supermarket stores. Firstly, the raw data was analyzed and the feature vectors were constructed. Secondly, Tukey's test was implemented on the constructed feature vectors to determine the outliers. Finally as a second method, we designed a generic unsupervised undercomplete autoencoder for the detection of the outliers in order to compare in several experiments the performance of both approaches. We also provided a discussion on the computational complexity of the techniques since, it is an important issue in real application domains.
Brain-computer interfaces (BCIs) enable direct communication between humans and machines by trans... more Brain-computer interfaces (BCIs) enable direct communication between humans and machines by translating brain activity into control commands. EEG is one of the most common sources of neural signals because of its inexpensive and non-invasive nature. However, interpretation of EEG signals is non-trivial because EEG signals have a low spatial resolution and are often distorted with noise and artifacts. Therefore, it is possible that meaningful patterns for classifying EEG signals are deeply hidden. Nowadays, state-of-the-art deep-learning algorithms have proven to be quite efficient in learning hidden, meaningful patterns. However, the performance of the deep learning algorithms depends upon the quality and the amount of the provided training data. Hence, a better input formation (feature extraction) technique and a generative model to produce high-quality data can enable the deep learning algorithms to adapt high generalization quality. In this study, we proposed a novel input format...
Mushiake, Hajime, Yasuyuki Tanatsugu, and Jun Tanji. Neuiments, we chose to study the neuronal ac... more Mushiake, Hajime, Yasuyuki Tanatsugu, and Jun Tanji. Neuiments, we chose to study the neuronal activity in the PMv ronal activity in the ventral part of premotor cortex during targetrather than PMd, because a previous study of temporal inactireach movement is modulated by direction of gaze. J. Neurophysvation of PMv and PMd revealed that the PMv is more iol. 78: 567-571, 1997. We recorded 200 neurons from the ventral critically involved in visual guidance of arm-reaching movepart of the premotor cortex (PMv) and 110 neurons from the ments than the PMd (Kurata and Hoffman 1994). primary motor cortex (MI) of a monkey performing a visually cued Soechting and colleagues (Flanders et al. 1992; Soechting arm-reaching task with a delay. We compared neuronal activity in and Flanders 1992) put forth a hypothesis that, in visually the premovement period while the monkey reached the target with guided arm-reaching tasks, the neural representation of target the eyes fixating on either a left or right fixation target. Our data parameters is first transformed from a retinocentric to a demonstrate that about half of the movement-related activity in the PMv was modulated by the direction of gaze. In contrast, a vast shoulder-centered representation. In later stages the transformajority of the activity of MI neurons and about half of PMv mation is made from kinematic to dynamic parameters of neurons were not influenced by the direction of gaze. We further arm movements. It is of interest to know the extent to which analyzed the movement-related activity during the reaching movethe neuronal activity in the PMv and MI is involved in the ment to targets at the top, bottom, left, and right of each fixation early or late stage of the transformation (Jeannerod 1994; point. The magnitude of activity of neurons showing the gaze-Kalaska and Crammond 1992). Relevant to this issue are direction selectivity was primarily determined by the position of recent reports that responses to visual signals of the PMv the reaching target relative to the eye-fixation target, and not by and motor-set-related activity of the PMd neurons were modthe position of the target relative to the animal's body. These data ulated by the direction of gaze (Boussaoud 1995; Boussaoud suggest that a part of the coordinate transformation of the motor et al. 1993). The visual receptive field was also modulated command signals concerning the direction of reaching from the retinotopic to body-centered frame of reference may occur at the by the position of arm (Graziano et al. 1994). In these level of premotor cortex but not in MI.
Uploads
Papers by Christian Klaes