(NeurIPS 2024) MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space
Jiangwei Weng, Zhiqiang Yan, Ying Tai, Jianjun Qian, Jian Yang, Jun Li*
Nanjing University of Science and Technology & Nanjing University
This repository represents the official implementation of the paper titled "MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space".
- Oct 31, 2024: Training & Testing codes and pre-trained models are released!
- Sep 26, 2024: Our paper has been accepted by NeurIPS 2024. Code and Models will be released.
Abstract: * Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data.In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependencies. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with layer normalization (LN) as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. *
Quantitative comparisons on LOL-V2-real, LOL-V2-syn, SMID, SDSD-indoor and SDSD- outdoor datasets. The best result is in red color while the second best result is in blue color. (click to expand)
LOL-V2-real and LOL-V2-syn (click to expand)
SMID, SDSD-indoor and SDSD-ourdoor (click to expand)
RealWorld Experimental Evaluation (click to expand)
Please see [Conda.sh] for the installation of dependencies required to run MambaLLIE.
Download our models from Google Drive. Put them in folder pretrained_weights.
# activate the environment
conda activate MambaLLIE
# Train LOL-v2-real
python3 basicsr/train.py --opt options/train/LOLv2_real.yml
# Test LOL-v2-real
python3 basicsr/test.py --opt options/test/LOLv2_real.yml
This code is based on BasicSR, MambaIR, Retinexformer, SNR-Net, and LSKNet. Thanks for their awesome work.
Should you have any question, please contact [email protected].
If you use MambaLLIE, please consider citing:
@article{MambaLLIE,
title={MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space},
author={Jiangwei Weng and Zhiqiang Yan and Ying Tai and Jianjun Qian and Jian Yang and Jun Li},
journal={NeurIPS 2024},
year={2024}
}