Progressive Network based on Detail Scaling and Texture Extraction: A More General Framework for Image Deraining

Abstract: Many feature extraction components have been proposed for image deraining tasks, aiming to improve feature learning. However, few models have addressed the integration of multi-scale features from derain images. The fusion of multiple features at different scales in one model has the potential to significantly enhance the authenticity and detail of raingy images restoration. This study introduces a migratable multi-scale feature blending model, which is a progressive learning model based on detail dilation and texture extraction. First, the degraded image is sent to the detail dilation module, which is designed to increase the detailed outline and obtain the coarse image features. Second, the extracted feature maps are sent to the multi-scale feature extraction (MFE) module and the multiscale hybrid strategy (MHS) module for improved texture restoration. Third, the simple convolution modules are replaced by an optimized transformer model to more efficiently extract contextual features and multi-scale information in images. Finally, a progressive learning strategy is employed to incrementally restore the degraded images. Empirical results show that our proposed module for progressive restoration achieves near state-of-the-art performance in several rain removal tasks. In particular, our model exhibits better rain removal realism compared to state-of-the-art models.

---

Network Architecture

Trackers

Here is a list of the improved trackers.

• DIDMDN: Density-aware single image de-raining using a multi-stream dense network, CVPR, 2018. [Paper]
• PReNet: Progressive image deraining networks: A better and simpler baseline, CVPR, 2019. [Paper]
• MPRNet: Multi-Stage Progressive Image Restoration, CVPR, 2021. [Paper]
• SPAIR: Spatially-Adaptive Image Restoration Using Distortion-Guided Networks, ICCV, 2021. [Paper]
• Restormer: Efficient transformer for high-resolution image restoration, 2022. [Paper]

Findings

• Empirical results show that our proposed module for progressive inpainting achieves near state-of-the-art performance in several rain removal tasks.

Description

This is the official repository of the paper Progressive network based on detail scaling and texture extraction: A more general framework for image deraining

🔧 Dependencies and Installation

• Python >= 3.8 (Recommend to use Anaconda or Miniconda)
• PyTorch >= 2.0.0

pip install -r requirements.txt

Training and Evaluation

We consider synthetic rain datasets (Rain200L, 422 Rain200H, Rain100L, and Rain100H) and three real-world 424 datasets (SPAData, RainDrop, and RID). In addition, we additionally consider more complex scene rain 426 streak removal datasets, RainKITTI2015 and JRSRD.

Updates

19/11/2023 Paper accepect at Neurocomputing! 🐣🐣🐣

BibTeX

If you find DTPNet useful for your research and applications, please cite using this BibTeX:

@article{huang2024progressive,
  title={Progressive network based on detail scaling and texture extraction: A more general framework for image deraining},
  author={Huang, Jiehui and Tang, Zhenchao and He, Xuedong and Zhou, Jun and Zhou, Defeng and Chen, Calvin Yu-Chian},
  journal={Neurocomputing},
  volume={568},
  pages={127066},
  year={2024},
  publisher={Elsevier}
}