Papers by Tetiana Aksenova
Frontiers in Human Neuroscience
IntroductionIn brain-computer interfaces (BCI) research, recording data is time-consuming and exp... more IntroductionIn brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation.MethodsWe evaluated the multilinear model and two deep learning (DL) models on a long-term BCI & Tetraplegia (ClinicalTrials.gov identifier: NCT02550522) clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. I...
Frontiers in Human Neuroscience
IntroductionMotor Brain–Computer Interfaces (BCIs) create new communication pathways between the ... more IntroductionMotor Brain–Computer Interfaces (BCIs) create new communication pathways between the brain and external effectors for patients with severe motor impairments. Control of complex effectors such as robotic arms or exoskeletons is generally based on the real-time decoding of high-resolution neural signals. However, high-dimensional and noisy brain signals pose challenges, such as limitations in the generalization ability of the decoding model and increased computational demands.MethodsThe use of sparse decoders may offer a way to address these challenges. A sparsity-promoting penalization is a common approach to obtaining a sparse solution. BCI features are naturally structured and grouped according to spatial (electrodes), frequency, and temporal dimensions. Applying group-wise sparsity, where the coefficients of a group are set to zero simultaneously, has the potential to decrease computational time and memory usage, as well as simplify data transfer. Additionally, online ...
La réalisation d'une interface cerveau machine EEG nécessite généralement l'utilisation d... more La réalisation d'une interface cerveau machine EEG nécessite généralement l'utilisation d'un grand nombre d'électrodes, causant la gêne de l'utilisateur et augmentant considérablement le coût calculatoire des traitements. Cependant, un choix judicieux de l'emplacement des ces électrodes peut permettre une réduction importante de leur nombre sans perte significative en performance. Cet article présente une méthode de sélection automatique d'un sous-ensemble quasi optimal d'électrodes et de filtres spatiaux calculés par Common Spatial Pattern (CSP) . Cette méthode, basée sur un calcul de coefficient de détermination multiple et l'utilisation du critère d'Akaike, est traitée de manière à résister aux artefacts par l'utilisation d'estimateurs robustes de variance et de matrice de covariance . Il est ainsi montré qu'une réduction très importante du nombre d'électrode est possible sans perte d'information sur les caractéristiques...
arXiv (Cornell University), Oct 5, 2022
In brain signal processing, deep learning (DL) models have become commonly used. However, the per... more In brain signal processing, deep learning (DL) models have become commonly used. However, the performance gain from using endto-end DL models compared to conventional ML approaches is usually significant but moderate, typically at the cost of increased computational load and deteriorated explainability. The core idea behind deep learning approaches is scaling the performance with bigger datasets. However, brain signals are temporal data with a low signal-to-noise ratio, uncertain labels, and nonstationary data in time. Those factors may influence the training process and slow down the models' performance improvement. These factors' influence may differ for end-to-end DL model and one using hand-crafted features. As not studied before, this paper compares models that use raw ECoG signal and time-frequency features for BCI motor imagery decoding. We investigate whether the current dataset size is a stronger limitation for any models. Finally, obtained filters were compared to identify differences between hand-crafted features and optimized with backpropagation. To compare the effectiveness of both strategies, we used a multilayer perceptron and a mix of convolutional and LSTM layers that were already proved effective in this task. The analysis was performed on the long-term clinical trial database (almost 600 minutes of recordings) of a tetraplegic patient executing motor imagery tasks for 3D hand translation. For a given dataset, the results showed that end-to-end training might not be significantly better than the hand-crafted features-based model. The performance gap is reduced with bigger datasets, but considering the increased computational load, end-to-end training may not be profitable for this application.
arXiv (Cornell University), Sep 8, 2022
Objective. In brain-computer interfaces (BCI) research, recording data is time-consuming and expe... more Objective. In brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation. Approach. We evaluated the multilinear model and two deep learning (DL) models on a long-term BCI & Tetraplegia NCT02550522 clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. In the experiment, a participant executed 3D virtual hand translation using motor imagery patterns. We designed multiple computational experiments in which training datasets were increased or translated to investigate the relationship between models' performance and different factors influencing recordings. Main results. For all tested decoders, our analysis showed that adding more data to the training dataset may not instantly increase performance for datasets already containing 40 minutes of the signal. DL decoders showed similar requirements regarding the dataset size compared to the multilinear model while demonstrating higher decoding performance. Moreover, high decoding performance was obtained with relatively small datasets recorded later in the experiment, suggesting motor imagery patterns improvement and patient adaptation during the long-term experiment. Finally, we proposed UMAP embeddings and local intrinsic dimensionality as a way to visualize the data and potentially evaluate data quality. Significance. DL-based decoding is a prospective approach in BCI which may be efficiently applied with real-life dataset size. Patient-decoder co-adaptation is an important factor to consider in long-term clinical BCI.
In the article tensor-input/tensor-output blockwise Recursive N-way Partial Least Squares (RNPLS)... more In the article tensor-input/tensor-output blockwise Recursive N-way Partial Least Squares (RNPLS) regression is considered. It combines the multi-way tensors decomposition with a consecutive calculation scheme and allows blockwise treatment of tensor data arrays with huge dimensions, as well as the adaptive modeling of time-dependent processes with tensor variables. In the article the numerical study of the algorithm is undertaken. The RNPLS algorithm demonstrates fast and stable convergence of regression coefficients. Applied to Brain Computer Interface system calibration, the algorithm provides an efficient adjustment of the decoding model. Combining the online adaptation with easy interpretation of results, the method can be effectively applied in a variety of multi-modal neural activity flow modeling tasks.
The Brain Computer Interface (BCI) is a system which translates recordings of the brain's neu... more The Brain Computer Interface (BCI) is a system which translates recordings of the brain's neural activity into commands for external devices. For neuronal signal decoding, a model is adjusted to an individual brain during the BCI system learning (calibration). BCI control system identification represents an example of analysis of complex system having no definite theory.The specificity of the BCI tasks can be summarized as multi-way structure of data,high dimension, and variability.Methods of inductive modeling are particularly efficient for fast and efficient structural modeling. Brain computer interface represents a new challenging task for application of inductive modeling methods and algorithms
In this paper, nonlinearity is introduced to linear neural activity decoders to improve continuou... more In this paper, nonlinearity is introduced to linear neural activity decoders to improve continuous hand trajectory prediction for Brain-Computer Interface systems. For decoding the high-dimensional data-tensor, a kernel regression was coupled with multilinear PLS (NPLS). Two ways to introduce nonlinearity were studied: a generalized linear model with kernel link function and kernel regression in the NPLS latent variables space (inside or outside the NPLS iterations). The efficiency of these approaches was tested on the publically available database of the simultaneous recordings of three-dimensional hand trajectories and epidural electrocorticogram (ECoG) signals of a Japanese macaque. Compared to linear methods, nonlinearity did not significantly improve the prediction accuracy but did significantly improve the smoothness of the prediction.
Scientific Reports
Brain–computer interfaces (BCIs) translate brain signals into commands to external effectors, and... more Brain–computer interfaces (BCIs) translate brain signals into commands to external effectors, and mainly target severely disabled users. The usability of BCIs may be improved by reducing their major constraints, such as the necessity for special training sessions to initially calibrate and later keep up to date the neural signal decoders. In this study, we show that it is possible to train and update BCI decoders during free use of motor BCIs. In addition to the neural signal decoder controlling effectors (control decoder), one more classifier is proposed to detect neural correlates of BCI motor task performances (MTP). MTP decoders reveal whether the actions performed by BCI effectors matched the user’s intentions. The combined outputs of MTP and control decoders allow forming training datasets to update the control decoder online and in real time during free use of BCIs. The usability of the proposed auto-adaptive BCI (aaBCI) is demonstrated for two principle BCIs paradigms: with ...
Journal of Neural Engineering, Sep 9, 2021
Objective. The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an i... more Objective. The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an important scientific question as new implantable recording devices can be used for medical purposes such as Brain-Computer Interface (BCI) or brain monitoring. Approach. The long-term clinical validation of wireless implantable multi-channel acquisition system for generic interface with neurons (WIMAGINE), a wireless 64-channel epidural ECoG recorder was investigated. The WIMAGINE device was implanted in two quadriplegic patients within the context of a BCI protocol. This study focused on the ECoG signal stability in two patients bilaterally implanted in June 2017 (P1) and in November 2019 (P2). Methods. The ECoG signal was recorded at rest prior to each BCI session resulting in a 32 month and in a 14 month follow-up for P1 and P2 respectively. State-of-the-art signal evaluation metrics such as root mean square (RMS), the band power (BP), the signal to noise ratio (SNR), the effective bandwidth (EBW) and the spectral edge frequency (SEF) were used to evaluate stability of signal over the implantation time course. The time-frequency maps obtained from task-related motor activations were also studied to investigate the long-term selectivity of the electrodes. Main results. Based on temporal linear regressions, we report a limited decrease of the signal average level (RMS), spectral distribution (BP) and SNR, and a remarkable steadiness of the EBW and SEF. Time-frequency maps obtained during motor imagery, showed a high level of discrimination 1 month after surgery and also after 2 years. Conclusions. The WIMAGINE epidural device showed high stability over time. The signal evaluation metrics of two quadriplegic patients during 32 months and 14 months respectively provide strong evidence that this wireless implant is well-suited for long-term ECoG recording. Significance. These findings are relevant for the future of implantable BCIs, and could benefit other patients with spinal cord injury, amyotrophic lateral sclerosis, neuromuscular diseases or drug-resistant epilepsy.
Journal of Neural Engineering, 2022
Objective. The article aims at addressing 2 challenges to step motor brain-computer interface (BC... more Objective. The article aims at addressing 2 challenges to step motor brain-computer interface (BCI) out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration. Approach. Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based recursive exponentially weighted Markov-switching multi-linear model (REW-MSLM) decoder is proposed. REW-MSLM uses a mixture of expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a ‘gating’ model. A Hidden Markov model appr...
2020 International Joint Conference on Neural Networks (IJCNN), 2020
Quantifying and evaluating properly the performances is a critical issue in BCI experiments. The ... more Quantifying and evaluating properly the performances is a critical issue in BCI experiments. The choice of the most adapted metrics can be difficult because they are specific to the experimental paradigm, task control, and data. In the current study evaluation criteria for closed-loop adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces are examined. The challenges such as imbalanced multi-class bias for discrete decoding (classification), online computing cost and dynamic analysis of results in time are considered. There are two main objectives of the study: identifying the best suited performances metrics according to the requirements and combining several levels of evaluation in the whole BCI system (decoder and patient oriented). The main novelty of the study is the combination of several common metrics with new temporal metrics and a time dynamic approach which allows adequately reflect the performance of adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces BCI systems. Additional response time and blocking error patterns reveal complementary information for BCI system performance evaluation. Developed criteria are applied to performance evaluation of 8-dimensional exoskeleton control by tetraplegic patient for both decoder and user-centered metrics.
Journal of Neural Engineering, 2022
Objective. Motor brain-computer interfaces (BCIs) are a promising technology that may enable moto... more Objective. Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship. Approach. In this study,...
2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 2019
The Brain Computer Interface (BCI) is a technology that provides non-muscular communication chann... more The Brain Computer Interface (BCI) is a technology that provides non-muscular communication channels to utilize the external devices. The signal from the brain is recorded, then classified. To classify, time-frequency features are frequently extracted using Continuous Wavelet Transform (CWT). However, the CWT is computationally heavy. This becomes the bottleneck of the real-time processing of BCI systems. We introduce a new method to solve the CWT using piecewise polynomials approximation over mother wavelets. The implementation shows that our method, piecewise polynomials based CWT (PCWT), is clearly better than the state of the art computation using the Fast Fourier Transform in the smaller buffer, which is preferable in the real-time systems. The proposed method can apply to other applications and should provide an effective approach to the real-time signal processing.
La presente invention porte sur un procede pour filtrer le signal de l'activite neuronale pen... more La presente invention porte sur un procede pour filtrer le signal de l'activite neuronale pendant une stimulation du cerveau profond haute frequence (DBS) pour eliminer l'artefact de stimulus dans le signal observe. Ce procede consiste a approximer des trajectoires de signal observe dans un espace de phase. Le signal observe est considere comme une somme des artefacts de stimulation induits par le signal de stimulation ; le signal de stimulation est suppose etre une solution d'une equation differentielle ordinaire comprenant un systeme auto-oscillant avec un cycle de limite stable. Le procede consiste egalement a decouper le signal observe et le son derive en segments, chaque segment correspondant a une periode de stimulation ; a collecter N periodes selectionnees de stimulation pour un ensemble d'entrainement ; a estimer le cycle limite du systeme auto-oscillant ; a synchroniser chaque artefact du signal observe au cycle limite estime ; a soustraire le cycle limite ...
Error correlates are thought to be promising for BCIs as a way to perform error correction or pre... more Error correlates are thought to be promising for BCIs as a way to perform error correction or prevention, or to label data in order to perform online adaptation of BCIs’ control models. Current state-of-the-art BCIs are motor-imagery-based invasive BCIs and thus have no access to neural data apart from sensory-motor cortices. We investigated at the single trial level the presence and detectability of error correlates in the primary motor cortex during observation or motor imagery (MI) control of a BCI with two discrete classes by a tetraplegic user. We show that error correlates can be detected using a broad range of classifiers, namely Support Vector Machine (SVM), logistic regression, N-way Partial Least Squares (NPLS), Multilayer Perceptron (MLP) and Convolutional Neural Network (CNN) with respective mean AUC of the ROC curve of 0.645, 0.662, 0.642, 0.680 and 0.630 in the observation condition, and 0.623, 0.605, 0.603, 0.626 and 0.580 in the MI-control condition. We also suggest ...
arXiv: Signal Processing, 2020
Objective. Brain-computer interfaces (BCIs) create a new communication pathway between the brain ... more Objective. Brain-computer interfaces (BCIs) create a new communication pathway between the brain and an effector without neuromuscular activation. BCI experiments highlighted high intra and inter-subjects variability in the BCI decoders. Although BCI model is generally relying on neurological markers generalizable on the majority of subjects, it requires to generate a wide range of neural features to include possible neurophysiological patterns. However, the processing of noisy and high dimensional features, such as brain signals, brings several challenges to overcome such as model calibration issues, model generalization and interpretation problems and hardware related obstacles. Approach. An online adaptive group-wise sparse decoder named Lp-Penalized REW-NPLS algorithm (PREW-NPLS) is presented to reduce the feature space dimension employed for BCI decoding. The proposed decoder was designed to create BCI systems with low computational cost suited for portable applications and tes...
PloS one, 2016
In the current paper the decoding algorithms for motor-related BCI systems for continuous upper l... more In the current paper the decoding algorithms for motor-related BCI systems for continuous upper limb trajectory prediction are considered. Two methods for the smooth prediction, namely Sobolev and Polynomial Penalized Multi-Way Partial Least Squares (PLS) regressions, are proposed. The methods are compared to the Multi-Way Partial Least Squares and Kalman Filter approaches. The comparison demonstrated that the proposed methods combined the prediction accuracy of the algorithms of the PLS family and trajectory smoothness of the Kalman Filter. In addition, the prediction delay is significantly lower for the proposed algorithms than for the Kalman Filter approach. The proposed methods could be applied in a wide range of applications beyond neuroscience.
Uploads
Papers by Tetiana Aksenova