Purpose: Developing robust artificial intelligence (AI) models for medical image analysis require... more Purpose: Developing robust artificial intelligence (AI) models for medical image analysis requires large quantities of diverse, well-curated data, which can prove challenging to collect due to privacy concerns, disease rarity, or diagnostic label quality. Collecting datasets for training diagnostic models for retinopathy of prematurity (ROP), a potentially-blinding disease, suffers from all of these challenges. Progressively-growing generative adversarial networks (PGANs) may help, as they can synthesize highly-realistic images that may increase both the size and diversity of medical datasets. Design: Diagnostic validation study of convolutional neural networks (CNNs) for plus disease detection, a component of severe ROP, using synthetic data. Participants: 5842 retinal fundus images (RFIs) collected from 963 preterm infants. Methods: Retinal vessel maps (RVMs) were segmented from RFIs. PGANs were trained to synthesize RVMs with normal, pre-plus, or plus disease vasculature. CNNs were trained, using real or synthetic RVMs, to detect plus disease from two real RVM test datasets. Main Outcome Measures: Features of real and synthetic RVMs were evaluated using Uniform Manifold Approximation and Projection (UMAP). Similarities were evaluated at the dataset and feature level using Frechet Inception Distance (FID) and Euclidean distance, respectively. CNN performance was assessed via area under the receiver operating characteristics (AUROC) curve; AUROCs were compared via bootstrapping and Delong's test for correlated ROC curves. Confusion matrices were compared using McNemar's 𝜒2 and Cohen's κ. Results: The CNN trained on synthetic RVMs had a significantly higher AUROC (0.971, p = 0.006, p = 0.004) and classified plus disease more similarly to a set of eight international experts (κ = 0.922) than the CNN trained on real RVMs (AUC = 0.934, κ = 0.701). Real and synthetic RVMs overlapped, by plus disease diagnosis, on the UMAP manifold, showing that synthetic images spanned the disease severity spectrum. FID and Eucl [...]
Advances in generative adversarial networks have allowed for engineering of highly-realistic imag... more Advances in generative adversarial networks have allowed for engineering of highly-realistic images. Many studies have applied these techniques to medical images. However, evaluation of generated medical images often relies upon image quality and reconstruction metrics, and subjective evaluation by laypersons. This is acceptable for generation of images depicting everyday objects, but not for medical images, where there may be subtle features experts rely upon for diagnosis. We implemented the pix2pix generative adversarial network for retinal fundus image generation, and evaluated the ability of experts to identify generated images as such and to form accurate diagnoses of plus disease in retinopathy of prematurity. We found that, while experts could discern between real and generated images, the diagnoses between image sets were similar. By directly evaluating and confirming physicians' abilities to diagnose generated retinal fundus images, this work supports conclusions that ...
Model brittleness is a key concern when deploying deep learning models in real-world medical sett... more Model brittleness is a key concern when deploying deep learning models in real-world medical settings. A model that has high performance at one institution may suffer a significant decline in performance when tested at other institutions. While pooling datasets from multiple institutions and retraining may provide a straightforward solution, it is often infeasible and may compromise patient privacy. An alternative approach is to fine-tune the model on subsequent institutions after training on the original institution. Notably, this approach degrades model performance at the original institution, a phenomenon known as catastrophic forgetting. In this paper, we develop an approach to address catastrophic forget-ting based on elastic weight consolidation combined with modulation of batch normalization statistics under two scenarios: first, for expanding the domain from one imaging system's data to another imaging system's, and second, for expanding the domain from a large multi...
Building robust deep learning-based models requires large quantities of diverse training data. In... more Building robust deep learning-based models requires large quantities of diverse training data. In this study, we investigate the use of federated learning (FL) to build medical imaging classification models in a real-world collaborative setting. Seven clinical institutions from across the world joined this FL effort to train a model for breast density classification based on Breast Imaging, Reporting & Data System (BI-RADS). We show that despite substantial differences among the datasets from all sites (mammography system, class distribution, and data set size) and without centralizing data, we can successfully train AI models in federation. The results show that models trained using FL perform 6.3% on average better than their counterparts trained on an institute's local data alone. Furthermore, we show a 45.8% relative improvement in the models' generalizability when evaluated on the other participating sites' testing data.
Federated learning is an emerging research paradigm for enabling collaboratively training deep le... more Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of mo...
Saliency maps have become a widely used method to make deep learning models more interpretable by... more Saliency maps have become a widely used method to make deep learning models more interpretable by providing post-hoc explanations of classifiers through identification of the most pertinent areas of the input medical image. They are increasingly being used in medical imaging to provide clinically plausible explanations for the decisions the neural network makes. However, the utility and robustness of these visualization maps has not yet been rigorously examined in the context of medical imaging. We posit that trustworthiness in this context requires 1) localization utility, 2) sensitivity to model weight randomization, 3) repeatability, and 4) reproducibility. Using the localization information available in two large public radiology datasets, we quantify the performance of eight commonly used saliency map approaches for the above criteria using area under the precision-recall curves (AUPRC) and structural similarity index (SSIM), comparing their performance to various baseline meas...
Segmentation tasks in medical imaging are inherently ambiguous: the boundary of a target structur... more Segmentation tasks in medical imaging are inherently ambiguous: the boundary of a target structure is oftentimes unclear due to image quality and biological factors. As such, predicted segmentations from deep learning algorithms are inherently ambiguous. Additionally, "ground truth" segmentations performed by human annotators are in fact weak labels that further increase the uncertainty of outputs of supervised models developed on these manual labels. To date, most deep learning segmentation studies utilize predicted segmentations without uncertainty quantification. In contrast, we explore the use of Monte Carlo dropout U-Nets for the segmentation with additional quantification of segmentation uncertainty. We assess the utility of three measures of uncertainty (Coefficient of Variation, Mean Pairwise Dice, and Mean Voxelwise Uncertainty) for the segmentation of a less ambiguous target structure (liver) and a more ambiguous one (liver tumors). Furthermore, we assess how the...
OBJECTIVE To validate a vascular severity score as an appropriate output for artificial intellige... more OBJECTIVE To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of ROP, 3rd edition (ICROP3) committee. DESIGN Validation study of an AI-based ROP vascular severity score Subjects, Participants, and/or Controls: 34 ROP experts from the ICROP3 committee. METHODS Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, pre-plus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labelled with a vascular severity score from the Imaging and Informatics in ROP deep learning (i-ROP DL) system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. MAIN OUTCOME MEASURES Weighted kappa and Pearson correlation coefficients (CC) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. RESULTS The mean weighted kappa and CC for all inter-observer pairs for plus disease image comparison was 0.67 and 0.88 respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, p < 0.001) and the ophthalmoscopic diagnosis of stage (p < 0.001 by ANOVA) among all experts. CONCLUSIONS The ROP vascular severity score correlates well with the ICROP committee member's labels for plus disease and stage, which had significant inter-grader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.
Despite the intense attention and investment into clinical machine learning (CML) research, relat... more Despite the intense attention and investment into clinical machine learning (CML) research, relatively few applications convert to clinical practice. While research is important in advancing the state-of-the-art, translation is equally important in bringing these technologies into a position to ultimately impact patient care and live up to extensive expectations surrounding AI in healthcare. To better characterize a holistic perspective among researchers and practitioners, we survey several participants with experience in developing CML for clinical deployment about their learned experiences. We collate these insights and identify several main categories of barriers and pitfalls in order to better design and develop clinical machine learning applications.
International Journal of Intelligent Robotics and Applications, 2017
This paper presents a robotic platform, capable of autonomous tunnel inspection, developed under ... more This paper presents a robotic platform, capable of autonomous tunnel inspection, developed under ROBO-SPECT European union funded research project. The robotic vehicle consists of a robotized production boom lift, a high precision robotic arm, advanced computer vision systems, a 3D laser scanner and an ultrasonic sensor. The autonomous inspection of tunnels requires advanced capabilities of the robotic vehicle and the computer vision sub-system. The robot localization in underground spaces and on long linear paths is a challenging task, as well as the mm accurate positioning of a robotic tip installed on a five-ton crane vehicle. Moreover, the 2D and 3D vision tasks, which support the inspection process, should tackle with poor and variable lighting conditions, low textured lining surfaces and the need for high accuracy. This contribution describes the final robotic vehicle and the developments as designed for concrete lining tunnel inspection. Results from the validation and benchmarking of the system are also included following the final tests at the operating Egnatia Motorway tunnels in northern Greece.
2014 22nd International Conference on Pattern Recognition, 2014
In this paper, we propose a Multilayer Markovian model for change detection in registered aerial ... more In this paper, we propose a Multilayer Markovian model for change detection in registered aerial image pairs with large time differences. A Three Layer Markov Random Field takes into account information from two different sets of features namely the Modified HOG (Histogram of Oriented Gradients) difference and the Gray-Level (GL) Difference. The third layer is the resultant combination of the two layers. Thus we integrate both the texture level as well as the pixel level information to generate the final result. The proposed model uses pairwise interaction retaining the sub-modularity condition for energy. Hence a global energy optimization can be achieved using a standard min-cut/ max flow algorithm ensuring homogeneity in the connected regions.
Although deep learning models have great promise for clinical applications, there are numerous ob... more Although deep learning models have great promise for clinical applications, there are numerous obstacles to training effective deep learning models. There is a need for collecting large quantities of diverse data, which often can only be achieved through multiinstitutional collaborations. One approach to multiinstitutional studies is to build large central repositories, but this is hindered by concerns about data sharing, including patient privacy, data deidentification, regulation, intellectual property, and data storage. These challenges have made centrally hosting data less impractical. An alternative approach is to have the data hosted locally and have the model trained in a collaborative fashion. Depending on the collaborative learning approach, model weights, model gradients, or smashed data are shared instead of raw patient data. These approaches can also reduce the communication overhead while reducing the need to share private patient data. In this chapter, we will review a...
IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium
Semantic segmentation is pivotal for remote sensing image analysis. Although existing segmentatio... more Semantic segmentation is pivotal for remote sensing image analysis. Although existing segmentation techniques perform well on similar landscape images, their generalization capability on an entirely different landscape is extremely poor. One of the primary reasons is that they partially or wholly, neglect the underlying relationship that exist in the joint space of input and output variables. Thus, effectively they lack to impose structure in their output predictions which is necessary for successful segmentation. In this paper, we address this problem and propose a novel solution by modeling the joint distribution of input-output variable which in turn enforces some structure in the initial segmentation mask. To this end, we first detect erroneous labels, in the form of Error maps, in the initial building masks. These Error maps are then used to correct the corresponding erroneous labels through a replacement technique. We evaluate our methodology on the benchmark Inria Aerial Image Labeling dataset, which is a large scale high resolution dataset for building footprint segmentation. In contrast to previous methods, our predicted segmentation masks are much closer to ground truth, owning to the fact that they are able to effectively correct both the large errors as well as the blobby effects. We lastly perform on par with other state-of-the-arts, validating the efficacy of our technique.
Advances in generative adversarial networks have allowed for engineering of highly-realistic imag... more Advances in generative adversarial networks have allowed for engineering of highly-realistic images. Many studies have applied these techniques to medical images. However, evaluation of generated medical images often relies upon image quality and reconstruction metrics, and subjective evaluation by laypersons. This is acceptable for generation of images depicting everyday objects, but not for medical images, where there may be subtle features experts rely upon for diagnosis. We implemented the pix2pix generative adversarial network for retinal fundus image generation, and evaluated the ability of experts to identify generated images as such and to form accurate diagnoses of plus disease in retinopathy of prematurity. We found that, while experts could discern between real and generated images, the diagnoses between image sets were similar. By directly evaluating and confirming physicians' abilities to diagnose generated retinal fundus images, this work supports conclusions that ...
While success of Deep Learning (DL) in automated diagnosis can be transformative to the medicinal... more While success of Deep Learning (DL) in automated diagnosis can be transformative to the medicinal practice especially for people with little or no access to doctors, its widespread acceptability is severely limited by inherent black-box decision making and unsafe failure modes. While saliency methods attempt to tackle this problem in non-medical contexts, their apriori explanations do not transfer well to medical usecases. With this study we validate a model design element agnostic to both architecture complexity and model task, and show how introducing this element gives an inherently self-explanatory model. We compare our results with state of the art non-trainable saliency maps on RSNA Pneumonia Dataset and demonstrate a much higher localization efficacy using our adopted technique. We also compare, with a fully supervised baseline and provide a reasonable alternative to it's high data labelling overhead. We further investigate the validity of our claims through qualitative e...
Model brittleness is a primary concern when deploying deep learning models in medical settings ow... more Model brittleness is a primary concern when deploying deep learning models in medical settings owing to inter-institution variations, like patient demographics and intra-institution variation, such as multiple scanner types. While simply training on the combined datasets is fraught with data privacy limitations, fine-tuning the model on subsequent institutions after training it on the original institution results in a decrease in performance on the original dataset, a phenomenon called catastrophic forgetting. In this paper, we investigate trade-off between model refinement and retention of previously learned knowledge and subsequently address catastrophic forgetting for the assessment of mammographic breast density. More specifically, we propose a simple yet effective approach, adapting Elastic weight consolidation (EWC) using the global batch normalization (BN) statistics of the original dataset. The results of this study provide guidance for the deployment of clinical deep learni...
Federated learning is an emerging research paradigm for enabling collaboratively training deep le... more Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of mo...
Purpose: Developing robust artificial intelligence (AI) models for medical image analysis require... more Purpose: Developing robust artificial intelligence (AI) models for medical image analysis requires large quantities of diverse, well-curated data, which can prove challenging to collect due to privacy concerns, disease rarity, or diagnostic label quality. Collecting datasets for training diagnostic models for retinopathy of prematurity (ROP), a potentially-blinding disease, suffers from all of these challenges. Progressively-growing generative adversarial networks (PGANs) may help, as they can synthesize highly-realistic images that may increase both the size and diversity of medical datasets. Design: Diagnostic validation study of convolutional neural networks (CNNs) for plus disease detection, a component of severe ROP, using synthetic data. Participants: 5842 retinal fundus images (RFIs) collected from 963 preterm infants. Methods: Retinal vessel maps (RVMs) were segmented from RFIs. PGANs were trained to synthesize RVMs with normal, pre-plus, or plus disease vasculature. CNNs were trained, using real or synthetic RVMs, to detect plus disease from two real RVM test datasets. Main Outcome Measures: Features of real and synthetic RVMs were evaluated using Uniform Manifold Approximation and Projection (UMAP). Similarities were evaluated at the dataset and feature level using Frechet Inception Distance (FID) and Euclidean distance, respectively. CNN performance was assessed via area under the receiver operating characteristics (AUROC) curve; AUROCs were compared via bootstrapping and Delong's test for correlated ROC curves. Confusion matrices were compared using McNemar's 𝜒2 and Cohen's κ. Results: The CNN trained on synthetic RVMs had a significantly higher AUROC (0.971, p = 0.006, p = 0.004) and classified plus disease more similarly to a set of eight international experts (κ = 0.922) than the CNN trained on real RVMs (AUC = 0.934, κ = 0.701). Real and synthetic RVMs overlapped, by plus disease diagnosis, on the UMAP manifold, showing that synthetic images spanned the disease severity spectrum. FID and Eucl [...]
Advances in generative adversarial networks have allowed for engineering of highly-realistic imag... more Advances in generative adversarial networks have allowed for engineering of highly-realistic images. Many studies have applied these techniques to medical images. However, evaluation of generated medical images often relies upon image quality and reconstruction metrics, and subjective evaluation by laypersons. This is acceptable for generation of images depicting everyday objects, but not for medical images, where there may be subtle features experts rely upon for diagnosis. We implemented the pix2pix generative adversarial network for retinal fundus image generation, and evaluated the ability of experts to identify generated images as such and to form accurate diagnoses of plus disease in retinopathy of prematurity. We found that, while experts could discern between real and generated images, the diagnoses between image sets were similar. By directly evaluating and confirming physicians' abilities to diagnose generated retinal fundus images, this work supports conclusions that ...
Model brittleness is a key concern when deploying deep learning models in real-world medical sett... more Model brittleness is a key concern when deploying deep learning models in real-world medical settings. A model that has high performance at one institution may suffer a significant decline in performance when tested at other institutions. While pooling datasets from multiple institutions and retraining may provide a straightforward solution, it is often infeasible and may compromise patient privacy. An alternative approach is to fine-tune the model on subsequent institutions after training on the original institution. Notably, this approach degrades model performance at the original institution, a phenomenon known as catastrophic forgetting. In this paper, we develop an approach to address catastrophic forget-ting based on elastic weight consolidation combined with modulation of batch normalization statistics under two scenarios: first, for expanding the domain from one imaging system's data to another imaging system's, and second, for expanding the domain from a large multi...
Building robust deep learning-based models requires large quantities of diverse training data. In... more Building robust deep learning-based models requires large quantities of diverse training data. In this study, we investigate the use of federated learning (FL) to build medical imaging classification models in a real-world collaborative setting. Seven clinical institutions from across the world joined this FL effort to train a model for breast density classification based on Breast Imaging, Reporting & Data System (BI-RADS). We show that despite substantial differences among the datasets from all sites (mammography system, class distribution, and data set size) and without centralizing data, we can successfully train AI models in federation. The results show that models trained using FL perform 6.3% on average better than their counterparts trained on an institute's local data alone. Furthermore, we show a 45.8% relative improvement in the models' generalizability when evaluated on the other participating sites' testing data.
Federated learning is an emerging research paradigm for enabling collaboratively training deep le... more Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of mo...
Saliency maps have become a widely used method to make deep learning models more interpretable by... more Saliency maps have become a widely used method to make deep learning models more interpretable by providing post-hoc explanations of classifiers through identification of the most pertinent areas of the input medical image. They are increasingly being used in medical imaging to provide clinically plausible explanations for the decisions the neural network makes. However, the utility and robustness of these visualization maps has not yet been rigorously examined in the context of medical imaging. We posit that trustworthiness in this context requires 1) localization utility, 2) sensitivity to model weight randomization, 3) repeatability, and 4) reproducibility. Using the localization information available in two large public radiology datasets, we quantify the performance of eight commonly used saliency map approaches for the above criteria using area under the precision-recall curves (AUPRC) and structural similarity index (SSIM), comparing their performance to various baseline meas...
Segmentation tasks in medical imaging are inherently ambiguous: the boundary of a target structur... more Segmentation tasks in medical imaging are inherently ambiguous: the boundary of a target structure is oftentimes unclear due to image quality and biological factors. As such, predicted segmentations from deep learning algorithms are inherently ambiguous. Additionally, "ground truth" segmentations performed by human annotators are in fact weak labels that further increase the uncertainty of outputs of supervised models developed on these manual labels. To date, most deep learning segmentation studies utilize predicted segmentations without uncertainty quantification. In contrast, we explore the use of Monte Carlo dropout U-Nets for the segmentation with additional quantification of segmentation uncertainty. We assess the utility of three measures of uncertainty (Coefficient of Variation, Mean Pairwise Dice, and Mean Voxelwise Uncertainty) for the segmentation of a less ambiguous target structure (liver) and a more ambiguous one (liver tumors). Furthermore, we assess how the...
OBJECTIVE To validate a vascular severity score as an appropriate output for artificial intellige... more OBJECTIVE To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of ROP, 3rd edition (ICROP3) committee. DESIGN Validation study of an AI-based ROP vascular severity score Subjects, Participants, and/or Controls: 34 ROP experts from the ICROP3 committee. METHODS Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, pre-plus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labelled with a vascular severity score from the Imaging and Informatics in ROP deep learning (i-ROP DL) system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. MAIN OUTCOME MEASURES Weighted kappa and Pearson correlation coefficients (CC) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. RESULTS The mean weighted kappa and CC for all inter-observer pairs for plus disease image comparison was 0.67 and 0.88 respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, p < 0.001) and the ophthalmoscopic diagnosis of stage (p < 0.001 by ANOVA) among all experts. CONCLUSIONS The ROP vascular severity score correlates well with the ICROP committee member's labels for plus disease and stage, which had significant inter-grader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.
Despite the intense attention and investment into clinical machine learning (CML) research, relat... more Despite the intense attention and investment into clinical machine learning (CML) research, relatively few applications convert to clinical practice. While research is important in advancing the state-of-the-art, translation is equally important in bringing these technologies into a position to ultimately impact patient care and live up to extensive expectations surrounding AI in healthcare. To better characterize a holistic perspective among researchers and practitioners, we survey several participants with experience in developing CML for clinical deployment about their learned experiences. We collate these insights and identify several main categories of barriers and pitfalls in order to better design and develop clinical machine learning applications.
International Journal of Intelligent Robotics and Applications, 2017
This paper presents a robotic platform, capable of autonomous tunnel inspection, developed under ... more This paper presents a robotic platform, capable of autonomous tunnel inspection, developed under ROBO-SPECT European union funded research project. The robotic vehicle consists of a robotized production boom lift, a high precision robotic arm, advanced computer vision systems, a 3D laser scanner and an ultrasonic sensor. The autonomous inspection of tunnels requires advanced capabilities of the robotic vehicle and the computer vision sub-system. The robot localization in underground spaces and on long linear paths is a challenging task, as well as the mm accurate positioning of a robotic tip installed on a five-ton crane vehicle. Moreover, the 2D and 3D vision tasks, which support the inspection process, should tackle with poor and variable lighting conditions, low textured lining surfaces and the need for high accuracy. This contribution describes the final robotic vehicle and the developments as designed for concrete lining tunnel inspection. Results from the validation and benchmarking of the system are also included following the final tests at the operating Egnatia Motorway tunnels in northern Greece.
2014 22nd International Conference on Pattern Recognition, 2014
In this paper, we propose a Multilayer Markovian model for change detection in registered aerial ... more In this paper, we propose a Multilayer Markovian model for change detection in registered aerial image pairs with large time differences. A Three Layer Markov Random Field takes into account information from two different sets of features namely the Modified HOG (Histogram of Oriented Gradients) difference and the Gray-Level (GL) Difference. The third layer is the resultant combination of the two layers. Thus we integrate both the texture level as well as the pixel level information to generate the final result. The proposed model uses pairwise interaction retaining the sub-modularity condition for energy. Hence a global energy optimization can be achieved using a standard min-cut/ max flow algorithm ensuring homogeneity in the connected regions.
Although deep learning models have great promise for clinical applications, there are numerous ob... more Although deep learning models have great promise for clinical applications, there are numerous obstacles to training effective deep learning models. There is a need for collecting large quantities of diverse data, which often can only be achieved through multiinstitutional collaborations. One approach to multiinstitutional studies is to build large central repositories, but this is hindered by concerns about data sharing, including patient privacy, data deidentification, regulation, intellectual property, and data storage. These challenges have made centrally hosting data less impractical. An alternative approach is to have the data hosted locally and have the model trained in a collaborative fashion. Depending on the collaborative learning approach, model weights, model gradients, or smashed data are shared instead of raw patient data. These approaches can also reduce the communication overhead while reducing the need to share private patient data. In this chapter, we will review a...
IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium
Semantic segmentation is pivotal for remote sensing image analysis. Although existing segmentatio... more Semantic segmentation is pivotal for remote sensing image analysis. Although existing segmentation techniques perform well on similar landscape images, their generalization capability on an entirely different landscape is extremely poor. One of the primary reasons is that they partially or wholly, neglect the underlying relationship that exist in the joint space of input and output variables. Thus, effectively they lack to impose structure in their output predictions which is necessary for successful segmentation. In this paper, we address this problem and propose a novel solution by modeling the joint distribution of input-output variable which in turn enforces some structure in the initial segmentation mask. To this end, we first detect erroneous labels, in the form of Error maps, in the initial building masks. These Error maps are then used to correct the corresponding erroneous labels through a replacement technique. We evaluate our methodology on the benchmark Inria Aerial Image Labeling dataset, which is a large scale high resolution dataset for building footprint segmentation. In contrast to previous methods, our predicted segmentation masks are much closer to ground truth, owning to the fact that they are able to effectively correct both the large errors as well as the blobby effects. We lastly perform on par with other state-of-the-arts, validating the efficacy of our technique.
Advances in generative adversarial networks have allowed for engineering of highly-realistic imag... more Advances in generative adversarial networks have allowed for engineering of highly-realistic images. Many studies have applied these techniques to medical images. However, evaluation of generated medical images often relies upon image quality and reconstruction metrics, and subjective evaluation by laypersons. This is acceptable for generation of images depicting everyday objects, but not for medical images, where there may be subtle features experts rely upon for diagnosis. We implemented the pix2pix generative adversarial network for retinal fundus image generation, and evaluated the ability of experts to identify generated images as such and to form accurate diagnoses of plus disease in retinopathy of prematurity. We found that, while experts could discern between real and generated images, the diagnoses between image sets were similar. By directly evaluating and confirming physicians' abilities to diagnose generated retinal fundus images, this work supports conclusions that ...
While success of Deep Learning (DL) in automated diagnosis can be transformative to the medicinal... more While success of Deep Learning (DL) in automated diagnosis can be transformative to the medicinal practice especially for people with little or no access to doctors, its widespread acceptability is severely limited by inherent black-box decision making and unsafe failure modes. While saliency methods attempt to tackle this problem in non-medical contexts, their apriori explanations do not transfer well to medical usecases. With this study we validate a model design element agnostic to both architecture complexity and model task, and show how introducing this element gives an inherently self-explanatory model. We compare our results with state of the art non-trainable saliency maps on RSNA Pneumonia Dataset and demonstrate a much higher localization efficacy using our adopted technique. We also compare, with a fully supervised baseline and provide a reasonable alternative to it's high data labelling overhead. We further investigate the validity of our claims through qualitative e...
Model brittleness is a primary concern when deploying deep learning models in medical settings ow... more Model brittleness is a primary concern when deploying deep learning models in medical settings owing to inter-institution variations, like patient demographics and intra-institution variation, such as multiple scanner types. While simply training on the combined datasets is fraught with data privacy limitations, fine-tuning the model on subsequent institutions after training it on the original institution results in a decrease in performance on the original dataset, a phenomenon called catastrophic forgetting. In this paper, we investigate trade-off between model refinement and retention of previously learned knowledge and subsequently address catastrophic forgetting for the assessment of mammographic breast density. More specifically, we propose a simple yet effective approach, adapting Elastic weight consolidation (EWC) using the global batch normalization (BN) statistics of the original dataset. The results of this study provide guidance for the deployment of clinical deep learni...
Federated learning is an emerging research paradigm for enabling collaboratively training deep le... more Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of mo...
Uploads
Papers by Praveer SINGH