Recent work has seen a surge of sparse representation based classification (SRC) methods applied ... more Recent work has seen a surge of sparse representation based classification (SRC) methods applied to automatic target recognition problems. While traditional SRC approaches used l0 or l1 norm to quantify sparsity, spike and slab priors have established themselves as the gold standard for providing general tunable sparse structures on vectors. In this work, we employ collaborative spike and slab priors that can be applied to matrices to encourage sparsity for the problem of multi-view ATR. That is, target images captured from multiple views are expanded in terms of a training dictionary multiplied with a coefficient matrix. Ideally, for a test image set comprising of multiple views of a target, coefficients corresponding to its identifying class are expected to be active, while others should be zero, i.e. the coefficient matrix is naturally sparse. We develop a new approach to solve the optimization problem that estimates the sparse coefficient matrix jointly with the sparsity inducing parameters in the collaborative prior. ATR problems are investigated on the mid-wave infrared (MWIR) database made available by the US Army Night Vision and Electronic Sensors Directorate, which has a rich collection of views. Experimental results show that the proposed joint prior and coefficient estimation method (JPCEM) can: 1.) enable improved accuracy when multiple views vs. a single one are invoked, and 2.) outperform state of the art alternatives particularly when training imagery is limited.
Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentatio... more Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentation. Open challenges remain in scenarios where labeled 3D segmentation maps for training are severely limited, as is often the case in practice, and in ensuring robustness to noise. Inspired by the observation that 3D vessel structures project onto 2D image slices with informative and unique edge profiles, we propose a novel deep 3D vessel segmentation network guided by edge profiles. Our network architecture comprises a shared encoder and two decoders that learn segmentation maps and edge profiles jointly. 3D context is mined in both the segmentation and edge prediction branches by employing bidirectional convolutional long-short term memory (BCLSTM) modules. 3D features from the two branches are concatenated to facilitate learning of the segmentation map. As a key contribution, we introduce new regularization terms that: a) capture the local homogeneity of 3D blood vessel volumes in the presence of biomarkers; and b) ensure performance robustness to domain-specific noise by suppressing false positive responses. Experiments on benchmark datasets with ground truth labels reveal that the proposed approach outperforms state-of-the-art techniques on standard measures such as DICE overlap and mean Intersection-over-Union. The performance gains of our method are even more pronounced when training is limited. Furthermore, the computational cost of our network inference is among the lowest compared with state-of-the-art.
Fine-grained image classification has emerged as a significant challenge because objects in such ... more Fine-grained image classification has emerged as a significant challenge because objects in such images have small inter-class visual differences but with large variations in pose, lighting, and viewpoints, etc. Most existing work focuses on highly customized feature extraction via deep network architectures which have been shown to deliver state of the art performance. Given that images from distinct classes in fine-grained classification share significant features of interest, we present a new deep network architecture that explicitly models shared features and removes their effect to achieve enhanced classification results. Our modeling of shared features is based on a new group based learning wherein existing classes are divided into groups and multiple shared feature patterns are discovered (learned). We call this framework Group based deep Shared Feature Learning (GSFL) and the resulting learned network as GSFL-Net. Specifically, the proposed GSFL-Net develops a specially desi...
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019
Image dehazing continues to be one of the most challenging inverse problems. Deep learning method... more Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model based methods and have helped define a new state of the art in achievable dehazed image quality. Yet, practical challenges remain in dehazing of real-world images where the scene is heavily covered with dense haze, even to the extent that no scene information can be observed visually. Many recent dehazing methods have addressed this challenge by designing deep networks that estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse of the haze model may then be used to estimate the dehazed image. In this work, we develop two novel network architectures to further this line of investigation. Our first model, denoted as At-DH, designs a shared DenseNet based encoder and two distinct DensetNet based decoders to jointly estimate the scene information viz. A and t respectively. This in contrast to most recent efforts (include those published in CVPR'18) that estimate these physical parameters separately. As a natural extension of At-DH, we develop the AtJ-DH network, which adds one more DenseNet based decoder to jointly recreate the haze-free image along with A and t. The knowledge of (ground truth) training dehazed/clean images can be exploited by a custom regularization term that further enhances the estimates of model parameters A and t in AtJ-DH. Experiments performed on challenging benchmark image datasets of NTIRE'19 and NTIRE'18 demonstrate that At-DH and AtJ-DH can outperform state-of-the-art alternatives, especially when recovering images corrupted by dense haze.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020
The emergence of deep learning methods that complement traditional model-based methods has helped... more The emergence of deep learning methods that complement traditional model-based methods has helped achieve a new state-of-the-art for image dehazing. Many recent methods design deep networks that either estimate the haze-free image (J) directly or estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t) followed by using the inverse of the haze model to estimate the dehazed image. However, both kinds of methods fail in dealing with non-homogeneous haze images where some parts of the image are covered with denser haze and the other parts with shallower haze. In this work, we develop a novel neural network architecture that can take benefits of the aforementioned two kinds of dehazed images simultaneously by estimating a new quantity-a spatially varying weight map (w). w can then be used to combine the directly estimated J and the results obtained by the inverse model. In our work, we utilize a shared DenseNet-based encoder, and four distinct DenseNet-based decoders that estimate J, A, t, and w jointly. A channel attention structure is added to facilitate the generation of distinct feature maps of different decoders. Furthermore, we propose a novel dilation inception module in the architecture to utilize the nonlocal features to make up the missing information during the learning process. Experiments performed on challenging benchmark datasets of NTIRE'20 and NTIRE'18 demonstrate that the proposed method-namely, AtJwD-can outperform many state-of-the-art alternatives in the sense of quality metrics such as SSIM, especially in recovering images under non-homogeneous haze.
2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Apr 1, 2018
Automated histopathological image analysis offers exciting opportunities for the early diagnosis ... more Automated histopathological image analysis offers exciting opportunities for the early diagnosis of several medical conditions including cancer. There are however stiff practical challenges: 1.) discriminative features from such images for separating diseased vs. healthy classes are not readily apparent, and 2.) distinct classes, e.g. healthy vs. stages of disease continue to share several geometric features. We propose a novel Analysis-synthesis model Learning with Shared Features algorithm (ALSF) for classifying such images more effectively. In ALSF, a joint analysis and synthesis learning model is introduced to learn the classifier and the feature extractor at the same time. In this way, the computation load in patch-level based image classification can be much reduced. Crucially, we integrate into this framework the learning of a low rank shared dictionary and a shared analysis operator, which more accurately represents both similarities and differences in histopathological images from distinct classes. ALSF is evaluated on two challenging databases: (1) kidney tissue images provided by the Animal Diagnosis Lab (ADL) at the Pennsylvania State University and (2) brain tumor images from The Cancer Genome Atlas (TCGA) database. Experimental results confirm that ALSF can offer benefits over state of the art alternatives.
IEEE Transactions on Geoscience and Remote Sensing, 2020
The low resolution of infrared (IR) images makes feature extraction for classification of a chall... more The low resolution of infrared (IR) images makes feature extraction for classification of a challenging work. Learning-based methods, therefore, are preferred to be used on such raw imagery. In this article, in order to avoid difficulties in feature extraction, a novel multitask extension of the widely used sparse-representation-classification (SRC) method is proposed in both single and multiview set-ups. That is, the test sample could be a single IR image or images from different views. In both single-view and multiview scenarios, we try to employ collaborative spike and slab priors. This is because the traditional sparsity-inducing measures such as the $l_{0}$ -row pseudonorm makes it hard to capture the sparse structure of the coefficient matrix when expanded in terms of a training dictionary, and the priors are proved to be able to capture fairly general sparse structures. Furthermore, a joint prior and sparse coefficient estimation method (JPCEM) is proposed for the first time in this article in order to alleviate the need to handpick prior parameters required before classification. Multiple experiments are conducted on a synthetic Comanche Forward Looking IR (FLIR) Automatic Target Recognition (ATR) database collected by Army Research Lab and a challenging mid-wave IR (MWIR) image ATR database made available by the U.S. Army Night Vision and Electronic Sensors Directorate. The final results substantiate the merits of the proposed JPCEM through comparisons with other state-of-the-art methods, including both the ones based on SRC and the ones constructed using deep learning frameworks.
IEEE Transactions on Geoscience and Remote Sensing, 2018
Dimensional accuracy is important in fabricating miniaturized product in order to reduce the mate... more Dimensional accuracy is important in fabricating miniaturized product in order to reduce the material waste and machining cost as well as to achieve a better quality product. This paper presents the analysis and modelling of dimensional accuracy in dry micro wire electrical discharge machining with control parameters of gap voltage and wire tension. The investigation was performed on stainless steel using integrated multi-process micro machine tools DT 110 with compressed air as the dielectric fluid and tungsten as the wire electrode. The dimensional accuracy was determined through kerf width differences of the machined slots. The kerf width was measured using scanning electron microscope. Full factorial was used to design the experiment while analysis of variance (ANOVA) was used to analyse the results as well as to evaluate the adequacy of the developed model. Based on ANOVA, both parameters; gap voltage and wire tension have high influence on kerf width differences. The optimum machining parameters for minimum kerf width differences were found to be 85 V gap voltage and 10 % wire tension. The developed model is adequate since the percentage error (2.13 %) is relatively small. It is recommended that different type of gases should be used for further investigation in order to determine the accuracy of the dry micro wire EDM.
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019
This paper reviews the second NTIRE challenge on image dehazing (restoration of rich details in h... more This paper reviews the second NTIRE challenge on image dehazing (restoration of rich details in hazy image) with focus on proposed solutions and results. The training data consists from 55 hazy images (with dense haze generated in an indoor or outdoor environment) and their corresponding ground truth (haze-free) images of the same scene. The dense haze has been produced using a professional haze/fog generator that imitates the real conditions of haze scenes. The evaluation consists from the comparison of the dehazed images with the ground truth images. The dehazing process was learnable through provided pairs of haze-free and hazy train images. There were ∼ 270 registered participants and 23 teams competed in the final testing phase. They gauge the state-of-the-art in image dehazing.
Recent work has seen a surge of sparse representation based classification (SRC) methods applied ... more Recent work has seen a surge of sparse representation based classification (SRC) methods applied to automatic target recognition problems. While traditional SRC approaches used l0 or l1 norm to quantify sparsity, spike and slab priors have established themselves as the gold standard for providing general tunable sparse structures on vectors. In this work, we employ collaborative spike and slab priors that can be applied to matrices to encourage sparsity for the problem of multi-view ATR. That is, target images captured from multiple views are expanded in terms of a training dictionary multiplied with a coefficient matrix. Ideally, for a test image set comprising of multiple views of a target, coefficients corresponding to its identifying class are expected to be active, while others should be zero, i.e. the coefficient matrix is naturally sparse. We develop a new approach to solve the optimization problem that estimates the sparse coefficient matrix jointly with the sparsity inducing parameters in the collaborative prior. ATR problems are investigated on the mid-wave infrared (MWIR) database made available by the US Army Night Vision and Electronic Sensors Directorate, which has a rich collection of views. Experimental results show that the proposed joint prior and coefficient estimation method (JPCEM) can: 1.) enable improved accuracy when multiple views vs. a single one are invoked, and 2.) outperform state of the art alternatives particularly when training imagery is limited.
Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentatio... more Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentation. Open challenges remain in scenarios where labeled 3D segmentation maps for training are severely limited, as is often the case in practice, and in ensuring robustness to noise. Inspired by the observation that 3D vessel structures project onto 2D image slices with informative and unique edge profiles, we propose a novel deep 3D vessel segmentation network guided by edge profiles. Our network architecture comprises a shared encoder and two decoders that learn segmentation maps and edge profiles jointly. 3D context is mined in both the segmentation and edge prediction branches by employing bidirectional convolutional long-short term memory (BCLSTM) modules. 3D features from the two branches are concatenated to facilitate learning of the segmentation map. As a key contribution, we introduce new regularization terms that: a) capture the local homogeneity of 3D blood vessel volumes in the presence of biomarkers; and b) ensure performance robustness to domain-specific noise by suppressing false positive responses. Experiments on benchmark datasets with ground truth labels reveal that the proposed approach outperforms state-of-the-art techniques on standard measures such as DICE overlap and mean Intersection-over-Union. The performance gains of our method are even more pronounced when training is limited. Furthermore, the computational cost of our network inference is among the lowest compared with state-of-the-art.
Fine-grained image classification has emerged as a significant challenge because objects in such ... more Fine-grained image classification has emerged as a significant challenge because objects in such images have small inter-class visual differences but with large variations in pose, lighting, and viewpoints, etc. Most existing work focuses on highly customized feature extraction via deep network architectures which have been shown to deliver state of the art performance. Given that images from distinct classes in fine-grained classification share significant features of interest, we present a new deep network architecture that explicitly models shared features and removes their effect to achieve enhanced classification results. Our modeling of shared features is based on a new group based learning wherein existing classes are divided into groups and multiple shared feature patterns are discovered (learned). We call this framework Group based deep Shared Feature Learning (GSFL) and the resulting learned network as GSFL-Net. Specifically, the proposed GSFL-Net develops a specially desi...
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019
Image dehazing continues to be one of the most challenging inverse problems. Deep learning method... more Image dehazing continues to be one of the most challenging inverse problems. Deep learning methods have emerged to complement traditional model based methods and have helped define a new state of the art in achievable dehazed image quality. Yet, practical challenges remain in dehazing of real-world images where the scene is heavily covered with dense haze, even to the extent that no scene information can be observed visually. Many recent dehazing methods have addressed this challenge by designing deep networks that estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t). The inverse of the haze model may then be used to estimate the dehazed image. In this work, we develop two novel network architectures to further this line of investigation. Our first model, denoted as At-DH, designs a shared DenseNet based encoder and two distinct DensetNet based decoders to jointly estimate the scene information viz. A and t respectively. This in contrast to most recent efforts (include those published in CVPR'18) that estimate these physical parameters separately. As a natural extension of At-DH, we develop the AtJ-DH network, which adds one more DenseNet based decoder to jointly recreate the haze-free image along with A and t. The knowledge of (ground truth) training dehazed/clean images can be exploited by a custom regularization term that further enhances the estimates of model parameters A and t in AtJ-DH. Experiments performed on challenging benchmark image datasets of NTIRE'19 and NTIRE'18 demonstrate that At-DH and AtJ-DH can outperform state-of-the-art alternatives, especially when recovering images corrupted by dense haze.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020
The emergence of deep learning methods that complement traditional model-based methods has helped... more The emergence of deep learning methods that complement traditional model-based methods has helped achieve a new state-of-the-art for image dehazing. Many recent methods design deep networks that either estimate the haze-free image (J) directly or estimate physical parameters in the haze model, i.e. ambient light (A) and transmission map (t) followed by using the inverse of the haze model to estimate the dehazed image. However, both kinds of methods fail in dealing with non-homogeneous haze images where some parts of the image are covered with denser haze and the other parts with shallower haze. In this work, we develop a novel neural network architecture that can take benefits of the aforementioned two kinds of dehazed images simultaneously by estimating a new quantity-a spatially varying weight map (w). w can then be used to combine the directly estimated J and the results obtained by the inverse model. In our work, we utilize a shared DenseNet-based encoder, and four distinct DenseNet-based decoders that estimate J, A, t, and w jointly. A channel attention structure is added to facilitate the generation of distinct feature maps of different decoders. Furthermore, we propose a novel dilation inception module in the architecture to utilize the nonlocal features to make up the missing information during the learning process. Experiments performed on challenging benchmark datasets of NTIRE'20 and NTIRE'18 demonstrate that the proposed method-namely, AtJwD-can outperform many state-of-the-art alternatives in the sense of quality metrics such as SSIM, especially in recovering images under non-homogeneous haze.
2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Apr 1, 2018
Automated histopathological image analysis offers exciting opportunities for the early diagnosis ... more Automated histopathological image analysis offers exciting opportunities for the early diagnosis of several medical conditions including cancer. There are however stiff practical challenges: 1.) discriminative features from such images for separating diseased vs. healthy classes are not readily apparent, and 2.) distinct classes, e.g. healthy vs. stages of disease continue to share several geometric features. We propose a novel Analysis-synthesis model Learning with Shared Features algorithm (ALSF) for classifying such images more effectively. In ALSF, a joint analysis and synthesis learning model is introduced to learn the classifier and the feature extractor at the same time. In this way, the computation load in patch-level based image classification can be much reduced. Crucially, we integrate into this framework the learning of a low rank shared dictionary and a shared analysis operator, which more accurately represents both similarities and differences in histopathological images from distinct classes. ALSF is evaluated on two challenging databases: (1) kidney tissue images provided by the Animal Diagnosis Lab (ADL) at the Pennsylvania State University and (2) brain tumor images from The Cancer Genome Atlas (TCGA) database. Experimental results confirm that ALSF can offer benefits over state of the art alternatives.
IEEE Transactions on Geoscience and Remote Sensing, 2020
The low resolution of infrared (IR) images makes feature extraction for classification of a chall... more The low resolution of infrared (IR) images makes feature extraction for classification of a challenging work. Learning-based methods, therefore, are preferred to be used on such raw imagery. In this article, in order to avoid difficulties in feature extraction, a novel multitask extension of the widely used sparse-representation-classification (SRC) method is proposed in both single and multiview set-ups. That is, the test sample could be a single IR image or images from different views. In both single-view and multiview scenarios, we try to employ collaborative spike and slab priors. This is because the traditional sparsity-inducing measures such as the $l_{0}$ -row pseudonorm makes it hard to capture the sparse structure of the coefficient matrix when expanded in terms of a training dictionary, and the priors are proved to be able to capture fairly general sparse structures. Furthermore, a joint prior and sparse coefficient estimation method (JPCEM) is proposed for the first time in this article in order to alleviate the need to handpick prior parameters required before classification. Multiple experiments are conducted on a synthetic Comanche Forward Looking IR (FLIR) Automatic Target Recognition (ATR) database collected by Army Research Lab and a challenging mid-wave IR (MWIR) image ATR database made available by the U.S. Army Night Vision and Electronic Sensors Directorate. The final results substantiate the merits of the proposed JPCEM through comparisons with other state-of-the-art methods, including both the ones based on SRC and the ones constructed using deep learning frameworks.
IEEE Transactions on Geoscience and Remote Sensing, 2018
Dimensional accuracy is important in fabricating miniaturized product in order to reduce the mate... more Dimensional accuracy is important in fabricating miniaturized product in order to reduce the material waste and machining cost as well as to achieve a better quality product. This paper presents the analysis and modelling of dimensional accuracy in dry micro wire electrical discharge machining with control parameters of gap voltage and wire tension. The investigation was performed on stainless steel using integrated multi-process micro machine tools DT 110 with compressed air as the dielectric fluid and tungsten as the wire electrode. The dimensional accuracy was determined through kerf width differences of the machined slots. The kerf width was measured using scanning electron microscope. Full factorial was used to design the experiment while analysis of variance (ANOVA) was used to analyse the results as well as to evaluate the adequacy of the developed model. Based on ANOVA, both parameters; gap voltage and wire tension have high influence on kerf width differences. The optimum machining parameters for minimum kerf width differences were found to be 85 V gap voltage and 10 % wire tension. The developed model is adequate since the percentage error (2.13 %) is relatively small. It is recommended that different type of gases should be used for further investigation in order to determine the accuracy of the dry micro wire EDM.
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019
This paper reviews the second NTIRE challenge on image dehazing (restoration of rich details in h... more This paper reviews the second NTIRE challenge on image dehazing (restoration of rich details in hazy image) with focus on proposed solutions and results. The training data consists from 55 hazy images (with dense haze generated in an indoor or outdoor environment) and their corresponding ground truth (haze-free) images of the same scene. The dense haze has been produced using a professional haze/fog generator that imitates the real conditions of haze scenes. The evaluation consists from the comparison of the dehazed images with the ground truth images. The dehazing process was learnable through provided pairs of haze-free and hazy train images. There were ∼ 270 registered participants and 23 teams competed in the final testing phase. They gauge the state-of-the-art in image dehazing.
Uploads
Papers by Xuelu Li