Systematic benchmarking of T1-MRI preprocessing pipelines and classical machine learning models against 3D CNNs for Alzheimer's disease classification — the Swiss Army KnifE for brain MRI research.
This repository accompanies the IEEE Healthcom 2026 paper:
Classical MRI Pipelines Rival Deep Learning for Dementia Classification: A Benchmarking Study Anastasia Gailly de Taurines, Payam Barnaghi, Ben Glocker, Fiona Kekwick, Alexander Capstick, Paresh Malhotra IEEE International Conference on E-health Networking, Application & Services (Healthcom), 2026
We systematically benchmark six widely used T1-weighted MRI preprocessing pipelines — FSL, SPM, CAT12, FreeSurfer, FastSurfer, and SynthSeg — combined with seven classical ML classifiers (SVM, Random Forest, Logistic Regression, Naïve Bayes, kNN, XGBoost, MLP), against three standard 3D CNN architectures (ResNet18, ResNet50, DenseNet121) for binary CN vs. AD classification.
Models are trained and evaluated on 1,701 participants from ADNI, with external validation of the best-performing approaches on 575 participants from AIBL. The framework evaluates classification performance, interpretability (SHAP / Grad-CAM), computational efficiency, and generalisability.
Key finding: Classical pipelines - particularly CAT12 + XGBoost (AUCROC: 96.1 ± 1.0%) - match or exceed 3D CNN baselines (ResNet50 AUCROC: 94.1 ± 1.4%) while requiring a fraction of the computational resources and offering substantially greater interpretability via SHAP analysis.
sake-public/
├── LICENCE
├── README.md
├── CITATION.cff
├── classical_ml/
│ ├── SAKE_run-analysis.ipynb
│ └── SAKE_functions.py
├── deep_learning/
│ ├── CNN-run.ipynb
│ └── SAKE_train.py
├── envs/
│ ├── classical_ml_environment.yml
│ └── deep_learning_environment.yml
└── preprocessing/
├── preproc.sh # Biobank-protocol preprocessing for CNNs
├── sake.sh # SAKE: runs FSL/SPM/CAT12/FreeSurfer/FastSurfer/SynthSeg
└── ext/ # Atlases, standard templates, auxiliary scripts
# Install FSL, SPM, CAT12, FreeSurfer, FastSurfer, SynthSeg here, as instructed in the requirements.txt file
This repository does not include MRI data. Access must be requested independently:
- ADNI (Alzheimer's Disease Neuroimaging Initiative): adni.loni.usc.edu
- AIBL (Australian Imaging, Biomarker & Lifestyle study): aibl.csiro.au
After downloading, participants with an MCI diagnosis at the time of their selected scan were excluded. Only the most recent T1 scan per participant was used.
conda env create -f envs/classical_ml_environment.yml
conda activate classical_ml_environmentKey dependencies: Python 3.12.4, scikit-learn 1.5.1, XGBoost, SHAP, pandas, numpy, scipy, matplotlib, seaborn, bayesian-optimization.
External neuroimaging toolboxes must be installed separately (see links below):
conda env create -f envs/deep_learning_environment.yml
conda activate deep_learning_environmentKey dependencies: Python 3.12.12, PyTorch 2.2.2+cu121, Lightning 2.2.0, MONAI 1.5.2, TorchIO 0.19.6, torchmetrics, wandb, tqdm.
Hardware note: CNN training was conducted on an NVIDIA A100 80 GB GPU. A CUDA-capable GPU is strongly recommended.
bash preprocessing/sake.shRuns all six preprocessing pipelines (FSL, SPM, CAT12, FreeSurfer, FastSurfer, SynthSeg) on your T1 MRI data. Each pipeline follows its standard workflow to extract cortical and subcortical regional morphometric features, using the DKT atlas (FSL, SPM, FastSurfer, CAT12) or DK atlas (FreeSurfer, SynthSeg). Atlases, standard templates, and auxiliary scripts are in preprocessing/ext/; external toolboxes must be installed there separately (see Installation).
Open and run classical_ml/SAKE_run-analysis.ipynb. This notebook covers:
- Feature loading and z-score normalisation (training-set parameters only)
- Three feature selection strategies: no selection, L1 (Lasso), MRMR
- Bayesian hyperparameter optimisation (nested 5-fold CV)
- Classification threshold moving via Youden's Index
- Repeated Monte-Carlo cross-validation (5 seeds, 85/15 train/test split, stratified by diagnosis and sex)
- Performance evaluation: accuracy, AUCROC, precision, sensitivity, specificity, F1
- SHAP interpretability analysis
bash preprocessing/preproc.shApplies the UK Biobank protocol: skull stripping and bias-field correction (FSL), non-linear registration to MNI152 1 mm standard space, resampling to 2 mm isotropic resolution (109×109×109 voxels), and per-subject z-scoring.
python deep_learning/SAKE_train.py \
--csv_train /path/to/train.csv \
--csv_test /path/to/test.csv \
--model resnet50 \
--seed 42Or use the interactive deep_learning/CNN-run.ipynb notebook. Training uses the Adam optimiser (lr=0.001, weight_decay=1e-4), binary cross-entropy loss, up to 100 epochs with early stopping (patience=20) based on validation AUCROC. Data augmentation is applied via TorchIO (random gamma, Gaussian noise, simulated motion, bias field, random affine, elastic deformation; p=0.5 per transform).
| Approach | Best combination | AUCROC | Accuracy |
|---|---|---|---|
| Classical ML | CAT12 + XGBoost (no feat. selection) | 96.1 ± 1.0% | 90.7 ± 1.5% |
| CNN | ResNet50 | 94.1 ± 1.4% | 85.9 ± 4.5% |
| Approach | Best combination | AUCROC |
|---|---|---|
| Classical ML | CAT12 + XGBoost (MRMR) | 96.5 ± 1.9% |
| CNN | ResNet50 | 84.5 ± 7.2% |
Values are mean ± SD across 5 Monte-Carlo cross-validation seeds.
If you use this code, please cite:
@inproceedings{gaillydetaurines2026sake,
title = {Classical MRI Pipelines Rival Deep Learning for Dementia Classification: A Benchmarking Study},
author = {Gailly de Taurines, Anastasia and Barnaghi, Payam and Glocker, Ben and Kekwick, Fiona and Capstick, Alexander and Malhotra, Paresh},
booktitle = {IEEE International Conference on E-health Networking, Application \& Services (Healthcom)},
year = {2026}
}Data used in preparation of this work were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012).
Data were also obtained from the Australian Imaging, Biomarker & Lifestyle (AIBL) study of ageing (aibl.csiro.au).
This project is licensed under the terms of the LICENSE file included in this repository.