Enhancing Visibility in Nigh!

ime Haze Images Using Guided

APSF and Gradient Adaptive Convolution
Yeying Jin∗ Beibei Lin∗ Wending Yan
National University of Singapore National University of Singapore Huawei International Pte Ltd
Singapore, Singapore Singapore, Singapore Singapore, Singapore
e0178303@[Link] [Link]@[Link] [Link]@[Link]

Yuan Yuan Wei Ye Robby T. Tan

Huawei International Pte Ltd Huawei International Pte Ltd National University of Singapore
Singapore, Singapore Singapore, Singapore Singapore, Singapore
arXiv:2308.01738v4 [[Link]] 21 Jan 2024

yuanyuan10@[Link] yewei10@[Link] [Link]@[Link]

Night Haze Ours Liu-22 [52] Wang-22 [77]

Figure 1: Our nighttime dehazing results compared to existing methods, we can handle night glow and low-light conditions.
ABSTRACT correction. This attention has high values on low-light regions
Visibility in hazy nighttime scenes is frequently reduced by multiple and low values on haze and glow regions. Extensive evaluation on
factors, including low light, intense glow, light scattering, and the real nighttime haze images, demonstrates the e!ectiveness of our
presence of multicolored light sources. Existing nighttime dehazing method. Our experiments demonstrate that our method achieves a
methods often struggle with handling glow or low-light conditions, PSNR of 30.38dB, outperforming state-of-the-art methods by 13%
resulting in either excessively dark visuals or unsuppressed glow on GTA5 nighttime haze dataset. Our data and code is available at:
outputs. In this paper, we enhance the visibility from a single night- [Link]
time haze image by suppressing glow and enhancing low-light
regions. To handle glow e!ects, our framework learns from the CCS CONCEPTS
rendered glow pairs. Speci"cally, a light source aware network is • Computing methodologies → Arti!cial intelligence; Com-
proposed to detect light sources of night images, followed by the puter vision; Computer vision tasks.
APSF (Atmospheric Point Spread Function)-guided glow rendering.
Our framework is then trained on the rendered images, resulting KEYWORDS
in glow suppression. Moreover, we utilize gradient-adaptive convo- nighttime, haze, glow, low-light, gradient, edge, texture, APSF
lution, to capture edges and textures in hazy scenes. By leveraging
extracted edges and textures, we enhance the contrast of the scene ACM Reference Format:
without losing important structural details. To boost low-light inten- Yeying Jin, Beibei Lin, Wending Yan, Yuan Yuan, Wei Ye, and Robby T.
Tan. 2023. Enhancing Visibility in Nighttime Haze Images Using Guided
sity, our network learns an attention map, then adjusted by gamma
APSF and Gradient Adaptive Convolution. In Proceedings of the 31st ACM
∗ Both authors contributed equally to this research. International Conference on Multimedia (MM ’23), October 29-November 3,
2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 13 pages. https:
Permission to make digital or hard copies of all or part of this work for personal or //[Link]/10.1145/3581783.3611884
classroom use is granted without fee provided that copies are not made or distributed
for pro"t or commercial advantage and that copies bear this notice and the full citation
on the "rst page. Copyrights for components of this work owned by others than the 1 INTRODUCTION
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior speci"c permission Nighttime hazy or foggy images often su!er from reduced visibility.
and/or a fee. Request permissions from permissions@[Link]. In addition to the common issues faced by night images, such
MM ’23, October 29-November 3, 2023, Ottawa, ON, Canada as low light, noise, uneven light distribution, and multiple light
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-0108-5/23/10. . . $15.00 colors, nighttime hazy or foggy images also exhibit a strong glow
[Link] and particle veiling e!ect. Despite these challenges, addressing
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

them is crucial for many applications. These include self-driving Extensive experiments on nighttime images demonstrate the e!ec-
cars, autonomous drones, and surveillance [3], as haze during the tiveness of our approach in quantitative and qualitative evaluations.
nighttime are natural phenomena that are frequent and inevitable. Our method achieves 30.38dB of PSNR, which outperforms existing
Daytime haze removal methods cannot handle the unique chal- nighttime dehazing methods by 13%.
lenges posed by nighttime haze. Traditional non-learning daytime
dehazing methods (e.g., [4, 16, 23, 72]) rely on the haze imaging
model [32]. However, this model is not valid at night due to the pres- 2 RELATED WORK
ence of arti"cial light sources and the complexity of illumination Early dehazing methods utilized multiple images [46, 56] or priors
colors. As a result, unlike in daytime, we cannot assume a uniform for atmospheric light and transmission estimation [5, 16, 23, 72].
atmospheric light color. Moreover, this daytime haze model does With the advent of deep learning, numerous networks were pro-
not account for the visual appearance of glow. posed to estimate the transmission map [6, 63, 95] or output clean
Existing nighttime dehazing methods produce unsatisfactory images end-to-end [18, 29, 37, 39, 62, 64, 81, 90, 92, 93]. Recent fully
dark visuals or unmitigated glow e!ects. Non-deep learning meth- supervised [15, 19, 49, 61, 69, 76, 80, 83, 101], semi-supervised [10,
ods (e.g., [45, 51, 77, 88]) introduce certain constraints on glow. 38, 43, 47, 65], zero-shot [35, 36], and unsupervised [17, 24, 48, 89,
However, they struggle with dark results due to the imprecise de- 100] methods have been developed. However, these methods strug-
composition of glow and background layers or the use of dark chan- gle with nighttime haze due to non-uniform, multi-colored arti"cial
nel prior [22] for dehazing. The main challenge faced by learning- light and the absence of clean ground truth data for training.
based methods is the absence of real-world paired training data, as Optimization-based nighttime dehazing methods have followed
obtaining clear ground truth images of hazy nighttime scenes that the atmospheric scattering model (e.g., [1, 2, 59]), new imaging
include glow and multiple light sources is intractable. A learning- model (e.g., [52, 73, 96, 97]), etc. Pei and Lee [59] transfer the airlight
based method [98] has attempted to address this issue by utilizing colors of hazy nighttime images to daytime and use DCP to dehaze.
synthetic data. However, this method is unable to e!ectively sup- Ancuti et al. [1, 2] introduce a fusion-based method and Laplacian
press glow since the synthetic dataset does not account for the glow pyramid decomposition to estimate local airlight. Zhang et al. [97]
e!ect. A semi-supervised deep learning-based network [87] su!ers use illumination compensation, color correction and DCP to dehaze.
from artifacts and loss of low-frequency scene details. Zhang et al. [96] propose maximum re#ectance prior (MRP). Tang et
In this paper, our goal is to enhance visibility in a single nighttime al. [73] use Retinex theory and Taylor series expansion. Liu et
haze image by suppressing glow and enhancing low-light regions. al. [51, 53] use regularization constraints. Wang et al. [77] proposed
Our glow suppression includes two main parts: APSF-guided glow the gray haze-line prior and variational model. Existing nighttime
rendering and gradient adaptive convolution. Our glow rendering dehazing methods depend on local patch-based atmospheric light
method uses an APSF-guided approach to create glow e!ects for estimation, assuming uniformity within a small patch. Therefore,
various light sources. We employ a light source aware network to their performance is sensitive to the patch size. These methods are
detect the locations of light sources in images and then apply APSF- not adaptive and time-consuming in optimization. Unlike them, our
guided glow rendering to these sources. Our framework learns from method is learning-based, more e$cient, practical and fast.
the rendered images and thus can suppress glow e!ects in di!erent Recently, learning-based nighttime dehazing methods [50] have
light sources. Our gradient adaptive convolution captures edges been proposed. Zhang et al. [98] train the network using synthetic
and textures from hazy images. To be speci"c, edges are obtained nighttime hazy images through fully supervised learning. However,
by computing the pixel di!erences [71] between neighboring pixels, this approach does not account for glow, leaving it in the results.
while the bilateral kernel [74] is used to extract textures of images. Yan et al. [87] propose a semi-supervised method employing high-
Both edges and textures are then fed into our framework to enhance low frequency decomposition and a grayscale network. However,
the image details. To enhance the visibility of non-light regions, their results tend to be dark, with lost details. This is because coarse
we introduce a novel attention-guided enhancement module. The frequency-based decomposition methods struggle to e!ectively sep-
hazy regions have low weights, while the dark regions have high arate glow, leading to reduced brightness and visibility of the scene.
weights in the attention map. As shown in Fig. 1, our method not The DeGlow-DeHaze network [33] estimates transmission followed
only handles glow e!ects but also enhances the low-light regions. by DehazeNet [6]. However, the atmospheric light estimated by the
Overall, our contributions can be summarized as follows: DeHaze network is obtained from the brightest region and assumed
to be globally uniform, which is invalid at nighttime [11–14, 26, 66–
• To our knowledge, our method is the "rst learning-based 68]. In contrast, our results can suppress glow and, at the same time,
network that handles night glow and low-light conditions enhance low-light regions.
in one go. The glow of a point source, referred to as the Atmospheric
• We present a light source aware network and APSF-guided Point Spread Function (APSF), has been studied in various works.
glow rendering to simulate glow e!ects from di!erent light Narasimhan and Nayar [57] "rst introduced APSF and developed
sources. By learning from the APSF-guided glow rendering a physics-based model for the multiple scattered light. Metari et
data, our framework e!ectively suppresses glow e!ects in al. [54] model the APSF kernel for multiple light scattering. Li et
real-world hazy images. al. [45] decompose glow from the input image using a layer sep-
• Since night images contain less contrast, we employ gradient- aration method [42], constrain glow by its smooth attribute, and
adaptive convolution for edge enhancement and the bilateral dehaze using DCP. Park et al. [58] and Yang et al. [88] follow the
kernel for texture enhancement. nighttime haze model and use weighted entropy and super-pixel to
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

: pixel difference
(3) Gradient Adaptive Conv

: pixel location : bilateral kernel (4) Attention Guided Enhance Real/Fake

Edge Texture Enhance

(1) Dehaze and Deglow

Input Haze Light Source Aware Output Clean Enhanced Output

(2) Guided APSF

Input Clean Light Source Map APSF2D Output Haze

Figure 2: (1) Our deglowing framework 𝐿𝐿 have two inputs: one to learn from real haze images 𝑀𝑀 and the other to learn from
real clean reference images 𝑀𝐿 . For input haze images 𝑀𝑀 , 𝐿𝐿 output clean images 𝑁𝐿 . For input clean images 𝑀𝐿 , 𝐿𝐿 output clean
images 𝐿𝐿 (𝑀𝐿 ). (2) APSF guide glow generator 𝐿𝑀 to generate glow 𝑁𝑀 on reference images 𝑀𝐿 . (3) the upper left is the gradient
adaptive convolution, from the gradient convolution (the blue window), we obtain edges; from the adaptive bilateral kernel
(the red), we enhance texture details. (4) the upper right is attention-guided enhancement module.

estimate atmospheric light and transmission map. However, these Algorithm 1 Light Source Map Detection
methods, after glow removal, simply apply daytime dehazing to
nighttime dehazing, which results in low visibility and color dis-
1:
! " #
Generate an initial light source mask by thresholding
1 if max𝐿 ↗ {𝑄,𝑅,𝑆 } 𝑀𝑂,𝑃
𝐿 > 0.8
tortion in their outcomes. Previous works have primarily focused the night input 𝑀 , 𝑇ˆ 𝑂,𝑃 ↔
on optimization-based approaches, while our work is the "rst to 0 otherwise
incorporate the APSF prior into a nighttime learning network. 2: Re!ne 𝑇ˆ 𝑂,𝑃 to 𝑇𝑂,𝑃 using alpha matting [34],
3: Calculate the$percentage of pixels in the mask,
3 PROPOSED METHOD (𝑇 )
𝑈𝑉𝑊𝑋𝑄_𝑌𝑍 ↔ numel(𝑇 ) ↘ 100
Fig. 2 shows our pipeline, including glow suppression and low- 4: Obtain the light source image, 𝑆𝑁 ↔ 𝑀 ≃ 𝑇
light enhancement. Our glow suppression has two parts: deglowing
network 𝐿𝐿 and glow generator 𝐿𝑀 . Our deglowing network 𝐿𝐿
transforms real haze images 𝑀𝑀 to clean images 𝑁𝐿 . We employ a light, and 𝑄 is the transmission, modeled as 𝑄 (𝑃) = 𝑎 ↑𝑈𝑉 (𝑊 ) , where
discriminator 𝑂𝐿 to determine whether the generated clean images 𝑏 is the extinction coe$cient. Light sources play an important role
𝑁𝐿 and the reference image 𝑀𝐿 are real or not. Our novelty in the and can be utilized in three ways: (1) to inform 𝐿𝐿 about the location
pipeline lies in these 3 ideas: APSF-guided glow rendering, gradient- of light sources (as shown in Fig. 4), (2) to guide 𝐿𝑀 to generate
adaptive convolution, and attention-guided enhancement. glow output 𝑁𝑀 , and (3) to render the glow data 𝑀𝑅 using APSF (as
shown in Fig. 5). For (1), we de"ne light source consistency loss to
3.1 Light Source Aware Network keep the light source regions consistent in input and output images,
Nighttime scenes often contain active light sources such as street- therefore to maintain the same color and shape of these regions:
lights, car headlights, and building lights. These sources can cause
Lls = |𝑁𝐿 ≃ 𝑇 ↑ 𝑆𝑁 | 1, (2)
strong glow in hazy nighttime scenes. The appearance of haze and
glow in nighttime scenes can be modeled as [45]: where 𝑆𝑁 is the light source, 𝑁𝐿 is the output clean image, 𝑇 is the
soft matting map, ≃ is element-wise multiplication. The process
𝑀𝑀 (𝑃) = 𝑀𝐿 (𝑃)𝑄 (𝑃) + 𝑅(𝑃) (1 ↑ 𝑄 (𝑃)) + 𝑆𝑁 (𝑃) ↓ APSF, (1)
of obtaining them is depicted in Fig. 3 and Algorithm 1, which
where 𝐿 (𝑃) = 𝑆𝑁 (𝑃) ↓ APSF, is the glow map, 𝑆𝑁 (𝑃) represents involves the following steps: First, we identify the regions that con-
the light sources, and APSF stands for Atmospheric Point Spread tain light sources. Next, an initial light source mask 𝑇ˆ is generated
Function, ↓ denotes the 2D convolution. 𝑀𝑀 is an observed hazy by thresholding the night image. To obtain a more accurate separa-
image. 𝑀𝐿 is the scene radiance (without haze). 𝑅 is the atmospheric tion of the light sources from the surrounding areas and to ensure
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

Pinhole

Im
ag
e
Pl
an
e
Glow
Incident
Light GET2D
Radius
Scatterer

Point Source （2）

Phase Function APSF Function APSF2D

（1） （3） （1）

（2）
Night Clean Rendered Glow

: 2D convolution
Initial Light Source Mask Refined Light Source Matting Light Source Map Boosted Glow : element-wise multiplication

Figure 3: We show Algorithm 1, light source map detection: (1) We !rst generate an initial light source mask 𝑇ˆ based on
intensity, (2) then re!ne the mask using alpha matting [34] to obtain light source soft matting 𝑇. (3) By multiplying the
light source map 𝑇 with the night clean image 𝑀𝐿 , we obtain the light source map 𝑆𝑁 . After obtaining the light source, we
show Algorithm 2, APSF-guided nighttime glow rendering: (1) Next, we perform APSF 2D convolution on the light source map
to render glow 𝐿. (2) Finally, by combining the night clean and glow image, we obtain the rendered glow image 𝑀𝑅 . More results
are shown in Fig. 5.

Figure 4: We show the light source maps 𝑆𝑁 of night haze.

Algorithm 2 APSF-based Nighttime Glow Rendering

1: Compute the APSF function in Algorithm 3,
𝑅𝑐𝑑𝑒 ↔ !"#$%&’()(𝑓,𝑔 , 𝑕)
2: Convert the APSF function to an APSF2D, Figure 5: We show with APSF, we can render glow 𝑀𝑅 (bottom)
𝑅𝑐𝑑𝑒 2𝑂 ↔ ’%)2D(𝑅𝑐𝑑𝑒 ) on night clean 𝑀𝐿 (top), with the help of light source maps.
3: Perform APSF2D convolution on the light source image,
𝐿 ↔ 𝑆𝑁 ↓ 𝑅𝑐𝑑𝑒 2𝑂,
4: Calculate the parameter for combining the clean and images. Then, we combine the clean and glow images to render a
glow image, 𝑖 ⇐ N (0, 1), realistic glow e!ect in the "nal image. In Fig. 5, we provide exam-
𝑗 ↔ 0.4196 · 𝑈𝑉𝑊𝑋𝑄_𝑌𝑍 2 ↑ 4.258 · 𝑈𝑉𝑊𝑋𝑄_𝑌𝑍 + 11.35 + 0.05 · 𝑖, ples of paired nighttime clean 𝑀𝐿 and glow 𝑀𝑅 images, demonstrating
5: Combine the clean and glow image to render night glow, the e!ectiveness of our approach.
𝑀𝑅 ↔ 0.99 · 𝑀𝐿 + 𝑗 · 𝐿 In Fig. 3 (top left), we use the phase function 𝑐 (cos 𝑓 ) [25] to ap-
6: Add Gaussian noise, 𝑀𝑅 ↔ A**N+&"%(𝑀𝑅 ) proximate light scattering in nighttime glow. The scattering angle
𝑓 is de"ned as the angle between the incident (𝑓 ⇒, 𝑘 ⇒ ) and scattered
light (𝑓, 𝑘) directions, where 𝑘 ⇒ = cos 𝑓 ⇒ and 𝑘 = cos 𝑓 represent the
smoother transitions, we re"ne the initial mask 𝑇ˆ to a matting map angular distribution of light. The scattered light intensity 𝑀 (𝑔 , 𝑘),
𝑇 using alpha matting [34]. Finally, we isolate the light sources 𝑆𝑁 considering optical thickness (𝑔 ), measures the attenuation of light
by applying the element-wise multiplication of the night image 𝑀 due to fog. Here, 𝑔 = 𝑏𝑙, with 𝑏 representing the attenuation coe$-
and the re"ned soft matting map 𝑇. cient and 𝑙 representing the distance between the isotropic source
and the pinhole camera. The shape of the phase function 𝑐 depends
3.2 APSF-Guided Nighttime Glow Rendering on the size of the scatterer, and hence the weather conditions. For
After obtaining the light source maps, we present an APSF-based large particles, such as fog, 𝑐 exhibits a strong peak in the direction
method for rendering nighttime glow e!ects, as shown in Fig. 3 of light incidence.
and Algorithm 2. First, we compute the APSF function in Algo- Algorithm 3 shows the APSF weight calculation process [57].
rithm 3 and convert it into a 2D format, which allows us to perform There are three input parameters: angle 𝑓 (-180⇑ to 180⇑ ), optical
2D convolution on the light source maps, resulting in the glow thickness 𝑔 , and forward scattering parameter 𝑕 that indicates the
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

Algorithm 3 APSF Weights Calculation (a) Vanilla Conv : pixel location

Require: 𝑓 , optical thickness 𝑔 , forward scattering parameter 𝑕 (b) Gradient Conv : pixel difference between and

Ensure: weights (c) Adaptive Conv : bilateral kernel

1: 𝑘 ↔ cos(𝑓 ) Input Output Weights
2: if look-up table (LUT) for Legendre polynomials exists then
3: Load LUT
4: else
5: Generate LUT and save to "le
(2𝑋+1) (1↑𝑌𝐿↑1 )
6: De"ne 𝑗 m (𝑚) ↔ 𝑚 + 1, 𝑏 m (𝑚, 𝑕) ↔ 𝑋
7: De"ne 𝑊m (𝑚, 𝑗 m , 𝑏 m ,𝑔 ) ↔ 𝑎 (↑𝑈 m𝑍 ↑𝑎 m log(𝑍 ) ) (a) Vanilla Conv (d) Gradient Adaptive Conv
8: Initialize 𝑛𝑎𝑉𝑊𝑋𝑄𝑌 to all zeros
Figure 7: Our gradient adaptive convolution combine (b) gra-
9: for 𝑚 ↔ 1 to 200 (number of polynomials) do
dient convolution (blue) to capture edge, and (c) adaptive
10: if 𝑚 == 1 then bilateral kernel (red) to obtain texture.
11: 𝑆m-1 (𝑘) ↔ 1
12: else
13: 𝑆m-1 (𝑘) ↔ 𝑆m (𝑘)
14: 𝑆m (𝑘) ↔ interpolate LUT for 𝑚 and 𝑘 3.3.1 Edge Capture using Gradient Convolution. We utilize nearby
15: 𝑜 ↔ 𝑊m (𝑚, 𝑗 m (𝑚), 𝑏 m (𝑚, 𝑕),𝑔 )(𝑆m-1 (𝑘) + 𝑆m (𝑘)), Eq. (3). information to preserve the gradient information and improve con-
16: 𝑛𝑎𝑉𝑊𝑋𝑄𝑌 ↔ 𝑛𝑎𝑉𝑊𝑋𝑄𝑌 + 𝑜 trast by extracting edges using gradient-adaptive convolution. The
17: 𝑛𝑎𝑉𝑊𝑋𝑄𝑌 ↔ 𝑛𝑎𝑉𝑊𝑋𝑄𝑌 · 𝑔 2 (normalize) main idea of gradient convolution is to use local binary patterns
(LBP) [60, 71] in a 𝑝 ↘ 𝑝 neighborhood, where the central pixel
serves as a threshold and the values of the surrounding pixels are
compared to the central pixel. If a neighboring pixel’s value exceeds
the central pixel’s value, a position marker of 1 is assigned; other-
wise, a value of 0 is assigned. The vanilla convolution and gradient
convolutions are expressed as follows:
% ( )
v𝑂⇒ = w p𝑂 ↑ p 𝑃 v 𝑃 , (Vanilla Conv) (4)
%
𝑃 ↗ω (𝑂 )
( )
v𝑂⇒ = w p𝑂 ↑ p 𝑃 (v 𝑃 ↑ v⇒𝑃 ), (Gradient Conv) (5)
𝑃 ↗ω (𝑂 )

where v = (v1, . . . , v𝑏 ), v𝑂 ↗ R𝐿 over 𝑞 pixels and 𝑟 channels, is

input image features. w is the weight in the ω(·), 𝑝 ↘ 𝑝 convolu-
tion window. We use [p𝑂 ↑ p 𝑃 ] to denote indexing of the spatial
dimensions of an array with 2D spatial o!sets. This convolution
Figure 6: We show using gradient adaptive convolution, can ⇒
operation results in a 𝑟 ⇒ -channel output, v𝑂⇒ ↗ R𝐿 , at each pixel 𝑉.
obtain edges (middle) and textures (below) in haze images. In Fig. 7 (a) and (b), instead of using input pixel values in Eq. (4), we
use the di!erences between neighboring pixels in Eq. (5) to capture
the gradient information.
atmospheric conditions [55]. The phase function 𝑐 can be expanded We "ne-tune the pre-trained pixel di!erence network [71] on
using Legendre polynomials series 𝑆𝑋 (𝑚 stand for order) [7]: nighttime haze images. Then, we use this "ne-tuned network 𝑂
to extract edges from input haze images and enforce the consis-
%
⇓ tency between the input 𝑀𝑀 and the output 𝑁𝐿 . Our self-supervised
𝑀 (𝑔 , 𝑘) = 𝑊𝑋 (𝑔 ) (𝑆𝑋↑1 (𝑘) + 𝑆𝑋 (𝑘)) , (3) gradient loss is de"ned as follows:
𝑋=1
& ' L𝑅 = |𝑂 (𝑁𝐿 ) ↑ 𝑂 (𝑀𝑀 )| 1 . (6)
where: 𝑊𝑋 (𝑔 ) = 𝑀 0𝑎 ↑𝑈𝐿𝑍 ↑𝑎𝐿 log𝑍 , 𝑗𝑋 = 𝑚+1, 𝑏𝑋 = 2𝑋+1
𝑋 1 ↑ 𝑕𝑋↑1 .
3.3.2 Texture Capture using Bilateral Kernel. A single CNN may
3.3 Gradient Adaptive Convolution not be su$cient for e!ectively handling uneven light distribution
Nighttime hazy images su!er from low contrast, missing textures, within a nighttime haze image. Therefore, we propose using a
and uneven illumination. Vanilla 2D convolution, which computes bilateral kernel in adaptive convolution to extract texture details,
the output based solely on input pixel values and the weights in the as shown in Fig. 7 (c):
convolution kernel, may not be su$cient for enhancing nighttime % & ' ( )
haze images. Hence, to improve the edge and texture details shown v𝑂⇒ = 𝑠 f𝑂 , f 𝑃 w p𝑂 ↑ p 𝑃 v 𝑃 , (Adaptive Conv) (7)
in Fig. 6, we utilize gradient-adaptive convolution. 𝑃 ↗ω (𝑂 )
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

where 𝑠 is bilateral kernal, depends on pixel features

" f [70]. We
& ' * *2 #
use color features f = (𝑡, 𝑊, 𝑢), 𝑠 f𝑂 , f 𝑃 = exp ↑ 2𝑎1 1 *f𝑂 ↑ f 𝑃 * ,
( ) " * *2 #
w p𝑂 ↑ p 𝑃 = exp ↑ 2𝑎1 2 *p𝑂 ↑ p 𝑃 * .
Since the features obtained using the bilateral kernel are noise-
reduced, edge-preserved, and detail-enhanced, they help extract
high-frequency texture details that are less a!ected by haze and
glow, as shown in Fig. 6 (bottom). To enforce the consistency be-
tween the input 𝑀𝑀 and the output 𝑁𝐿 and improve the quality of
the output, we de"ne the self-supervised bilateral kernel loss as: Figure 8: We show attention maps to enhance low-light.

L𝑐 = |𝑠 (𝑁𝐿 ) ↑ 𝑠 (𝑀𝑀 )| 1 . (8) Table 1: Results on GTA5 nighttime haze dataset.

PSNR ⇔ SSIM ⇔
In summary, our gradient-adaptive convolution captures edge in- Input Image 18.987 0.6764
formation by considering the di!erences between neighboring pix- Li [45] 21.024 0.6394
els instead of their pixel values, as re#ected in the gradient term Zhang [96] 20.921 0.6461
(v 𝑃 ↑v⇒𝑃 ). Additionally, it accounts for spatially varying illumination Ancuti [1] 20.585 0.6233
Yan [87] 26.997 0.8499
and non-uniform haze distribution by utilizing the adaptive term
𝑠 (f𝑂 , f 𝑃 ), which helps to preserve texture information and adapt CycleGAN [102] 21.753 0.6960
more e!ectively to nighttime scenes. w/o Gradient Adaptive Conv 27.913 0.8673
w/o APSF Guided Render 28.913 0.8776
Our Result 30.383 0.9042
3.4 Network and Other Losses
In Fig. 2, our glow suppression is CycleGAN [102]-based network. Table 2: Summary of comparisons between ours and existing
We use the deglowing network 𝐿𝐿 , which is coupled with a discrim- learning-based nighttime dehazing methods. SL is short for
inator 𝑂𝐿 . To ensure reconstruction consistency, we have another supervised learning, SSL is semi-supervised learning.
haze generator 𝐿𝑀 coupled with its discriminator 𝑂𝑀 . 𝐿𝐿 input Learning Methods Glow Edge, Texture Low Light Light Source Uneven light
hazy images 𝑀𝑀 and output clean images 𝑁𝐿 . Our haze generator SSL Ours ↭ ↭ ↭ ↭ ↭
𝐿𝑀 transforms the output images 𝑁𝐿 to reconstructed clean images SL Kuanar [33] ↭ ↘ ↘ ↘ ↘

𝑀ˆ𝑀 . By imposing the cycle-consistency constraints, our deglowing SSL Yan [87] ↭ ↭ ↘ ↘ ↘
SL Zhang [96] ↘ ↘ ↘ ↘ ↘
network 𝐿𝐿 learns to remove the real-world glow. Thanks to the
cycle-consistency constraints, we are allowed to use unpaired data
to optimize our deglowing network, thus reducing the domain gap where 𝑣 is the enhanced parameter in [21], we set 0.3 in our exper-
between the synthetic datasets and real-world glow images. iments, 𝑁𝐿 is the dehazed output, 𝑁𝑔 is the enhanced result, 𝑅 is
Besides the self-supervised light source consistency loss Lls , the attention map.
gradient loss L𝑅 and bilateral kernel loss L𝑐 , with weights {1, 0.5,
5}, we followed [102], use other losses to train our network. They 4 EXPERIMENTAL RESULTS
are adversarial loss L𝑑𝑉𝑒 , cycle consistency loss L𝐿𝑓𝐿 , identity loss
L𝑂𝑉𝑔𝑏 , with weights {1, 10, 10}. 4.1 Datasets
GTA5 [87] is a synthetic nighttime dehazing dataset, which is
3.5 Low-light Region Enhancement generated by the GTA5 game engine. It includes 864 paired images,
where 787 paired images are used as the training set and the rest
Nighttime dehazing often leads to dark results due to low-light con- images are taken as the test set.
ditions [20, 27, 30, 31, 78, 79, 82]. To address this, we incorporate a RealNightHaze is a real-world night dehazing dataset. It includes
low-light enhancement module to improve the visibility of object 440 night hazy images, where 150 images are from [98], 200 images
regions. Our approach involves generating an attention map that are from [87] and the rest images are collected from the Internet.
highlights these regions, allowing our method to focus on enhanc-
ing their intensity. We then apply a low-light image enhancement 4.2 Comparison on Synthetic Datasets
technique [21] to enhance the region with the assistance of the
attention map, even in scenes with low light. In this section, we compare our method with existing state-of-the
art methods, including Yan [87], Zhang [97], Li [45] Ancuti [1],
Attention Map To obtain the soft attention maps 𝑅 shown in Fig. 8, Zhang [96], Yu [94], Zhang [98], Liu [51] and Wang [77]. The sum-
we input the night haze images and re"ne the coarse map using [44]. mary of the main di!erences is show in Table 2. The experimental
The re"ned attention map 𝑅 exhibits high values in object regions results are shown in Table 1. It can be observed that our method
and low values in uniform regions, such as the sky. Therefore, we achieves a signi"cant performance improvement. We adopt two
can distinguish between the object and the haze regions. widely used metrics PSNR, SSIM in generation [75] and restora-
tion [8, 9, 28, 40, 41, 84–86, 91] tasks. Our method achieves a PSNR
𝑕
𝑁𝑔 = (1 ↑ 𝑅) · 𝑁𝐿 + 𝑅 · 𝑁𝐿 , (9) of 30.383 and a SSIM of 0.904, outperforming Yan’s method [87] by
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

(a) Input (b) Ours (c) Liu-22 [52] (d) Wang-22 [77]

(e) Zhang [98] (f) Yan-20 [87] (g) Yu-19 [94] (h) Zhang [96]

(a) Input (b) Ours (c) Liu-22 [52] (d) Wang-22 [77]

(e) Zhang [98] (f) Yan-20 [87] (g) Yu-19 [94] (h) Zhang [96]

(a) Input (b) Ours (c) Liu-22 [52] (d) Wang-22 [77]

(e) Zhang [98] (f) Yan-20 [87] (g) Yu-19 [94] (h) Ancuti-20 [2]
Figure 9: Visual comparisons of di"erent nighttime dehazing methods on real nighttime hazy scenes. Our results are more
realistic and e"ective in nighttime dehazing. Zoom-in for better visualization.
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

Table 3: User study evaluation on the real night images, our method obtained the highest mean (the max score is 10), showing
our method is e"ective in nighttime dehazing, deglowing and low-light enhancement. Our method is also visual realistic. The
best result is in red whereas the second and third best results are in blue and purple, respectively.
Aspects Ours Yan [87] Zhang [97] Li [45] Ancuti [1] Zhang [96] Yu [94] Zhang [98] Liu-22 [52] Wang-22 [77]
[Link]⇔ 9.1 ± 0.99 8.9 ± 1.75 4.2 ± 2.42 6.1 ± 2.00 5.2 ± 2.35 4.4 ± 2.01 4.8 ± 2.26 4.6 ± 2.07 3.9 ± 2.12 5.5 ± 1.91
[Link]⇔ 9.1 ± 0.91 7.9 ± 1.12 3.2 ± 1.96 6.2 ± 1.83 5.2 ± 2.14 4.0 ± 2.09 3.7 ± 2.08 4.4 ± 2.05 5.3 ± 1.99 5.7 ± 1.76
[Link]-light⇔ 8.5 ± 1.30 7.9 ± 1.93 7.1 ± 2.31 5.5 ± 2.33 5.4 ± 2.01 5.6 ± 1.82 6.5 ± 2.12 5.6 ± 1.75 5.5 ± 1.92 5.4 ± 1.89
[Link]⇔ 8.9 ± 0.94 8.0 ± 1.28 4.6 ± 1.93 4.7 ± 2.07 6.7 ± 1.90 4.9 ± 1.93 5.7 ± 1.90 4.9 ± 1.97 3.8 ± 1.81 5.9 ± 1.67

14% and 5%, respectively. This is because our method learns from the
APSF-guided glow rendering and thus e!ectively removes the glow
e!ects. Another advantage is that we introduce a gradient-adaptive
convolution to capture the edges and textures. The obtained edges
and textures are then used to enhance the structural details of the
enhanced images, leading to superior performance.
Input 𝑖𝑀 w/o Enhance 𝑗𝑁 Attention 𝑘 w/ Enhance 𝑗𝑂
4.3 Comparison on Real-World Datasets Figure 10: Ablation study of low-light enhancement.
Fig. 9 show the qualitative results, including Liu [51], Wang [77],
Zhang [98], Yan [87], Yu [94], Zhang [96] and our method. It can
be found that our method signi"cantly enhances the visibility of
nighttime hazy images. Speci"cally, most state-of-the-art methods
cannot su$ciently remove haze since their methods su!er from the
domain gap between the synthetic datasets and real-world images.
Yan et al [87] proposes a semi-supervised framework for night-
time foggy removal and can remove most hazy e!ects. However,
(a) Input 𝑖𝑀 (b) w/o L𝑃 (c) w/o L𝑄 (d) w/ GAC (e) Enhance
their method over-suppresses hazy images, and thus their outputs
Figure 11: Ablation study of gradient adaptive conv. (GAC).
become too dark.
In contrast, our method handles glow and low-light conditions.
As shown in Fig. 9 (b), our method not only removes the haze
of the input images but also enhances the light. For instance, the
details of trees and buildings are clear. This is because our method
gradient capture convolution can e!ectively preserve the structural
simulates the glow rendering by utilizing Atmospheric Point Spread
details of hazy images. By taking full advantage of the gradient
Function (APSF) and thus can e!ectively remove haze or glow in
maps and textures, our framework generates sharper results.
a real-world night hazy image. Moreover, we propose a gradient-
Low Light Enhancement Fig. 10 shows the results of low-light
adaptive convolution to capture the edges and textures from hazy
enhancement from (b) to (d).
images. The captured edges and textures are then used to boost the
details of images, leading to superior performance. Furthermore, we
introduce an attention map to enhance the low-light regions. As a 5 CONCLUSION
result, our method achieves a signi"cant performance improvement. In this paper, we have proposed a novel nighttime visibility enhance-
We also conduct user studies on real-world night hazy images. ment framework, addressing both glow and low-light conditions.
The experiment results are shown in Table 3. It can be found that Our framework includes three core ideas: APSF-guided glow render-
our methods get the highest scores in all aspects. ing, gradient-adaptive convolution and attention-guided low-light
enhancement. Our framework suppresses glow e!ects via learning
4.4 Ablation Study from the APSF-guided glow rendering data. Thanks to our APSF-
Our framework includes three core parts: APSF-guided glow render- guided glow rendering, we allow to use a semi-supervised method
ing, gradient-adaptive convolution and attention-guided enhance- to optimize our network, thus handling glow e!ects in di!erent
ment. To prove the e!ectiveness of each part, we conduct ablation light sources. Our gradient-adaptive convolution is proposed to
studies on real-world night hazy images. capture edges or textures from a nighttime hazy image. Bene"ting
APSF-guided glow rendering Fig. 5 (bottom) shows the results of from the captured edges or textures, our framework e!ectively
our glow rendering. We can obverse that our method can accurately preserves the structural details. Our low-light region enhancement
detect the location of light sources (middle). Also, the rendered boosts the intensity of dark or over-suppressed regions via atten-
results e!ectively simulate the glow e!ects. tion map. Both quantitative and qualitative experiments show that
Gradient-adaptive convolution Fig. 6 show the results of the our method achieves a signi"cant performance improvement. More-
gradient edge maps (middle) and textures (bottom). Fig. 11 (b) and (c) over, the ablation study proves the e!ectiveness of each core idea.
show results without gradient loss L𝑅 and without kernel loss L𝑐 , Handling scenarios with diverse domain shifts [99] will be a focus
and (d) is with gradient capture convolution. It can be found that our of our future research.
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

ACKNOWLEDGMENTS 33, 12 (2011), 2341–2353.

[24] Lu-Yao Huang, Jia-Li Yin, Bo-Hao Chen, and Shao-Zhen Ye. 2019. Towards Unsu-
This research/project is supported by the National Research Foun- pervised Single Image Dehazing With Deep Learning. In 2019 IEEE International
dation, Singapore under its AI Singapore Programme (AISG Award Conference on Image Processing (ICIP). IEEE, 2741–2745.
[25] Akira Ishimaru et al. 1978. Wave propagation and scattering in random media.
No: AISG2-PhD/2022-01-037[T]). Vol. 2. Academic press New York.
[26] Xin Jin, Jia-Wen Xiao, Ling-Hao Han, Chunle Guo, Ruixun Zhang, Xialei Liu,
REFERENCES and Chongyi Li. 2023. Lighting Every Darkness in Two Pairs: A Calibration-
Free Pipeline for RAW Denoising. Proceedings of the IEEE/CVF International
[1] Cosmin Ancuti, Codruta O Ancuti, Christophe De Vleeschouwer, and Alan C
Conference on Computer Vision.
Bovik. 2016. Night-time dehazing by fusion. In 2016 IEEE International Conference
[27] Yeying Jin, Ruoteng Li, Wenhan Yang, and Robby T Tan. 2023. Estimating
on Image Processing (ICIP). IEEE, 2256–2260.
re#ectance layer from a single image: Integrating re#ectance guidance and
[2] Cosmin Ancuti, Codruta O Ancuti, Christophe De Vleeschouwer, and Alan C
shadow/specular aware learning. In Proceedings of the AAAI Conference on
Bovik. 2020. Day and night-time dehazing by local airlight estimation. IEEE
Arti!cial Intelligence, Vol. 37. 1069–1077.
Transactions on Image Processing 29 (2020), 6264–6275.
[28] Yeying Jin, Aashish Sharma, and Robby T Tan. 2021. DC-ShadowNet: Single-
[3] Sriparna Banerjee and Sheli Sinha Chaudhuri. 2021. Nighttime image-dehazing:
Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classi"er
a review and quantitative benchmarking. Archives of Computational Methods in
Guided Network. In Proceedings of the IEEE/CVF International Conference on
Engineering 28, 4 (2021), 2943–2975.
Computer Vision. 5027–5036.
[4] Dana Berman, Shai Avidan, et al. 2016. Non-local image dehazing. In Proceedings
[29] Yeying Jin, Wending Yan, Wenhan Yang, and Robby T Tan. 2022. Structure
of the IEEE conference on computer vision and pattern recognition. 1674–1682.
Representation Network and Uncertainty Feedback Learning for Dense Non-
[5] Dana Berman, Tali Treibitz, and Shai Avidan. 2018. Single image dehazing using
Uniform Fog Removal. In Proceedings of the Asian Conference on Computer Vision.
haze-lines. IEEE transactions on pattern analysis and machine intelligence 42, 3
2041–2058.
(2018), 720–734.
[30] Yeying Jin, Wenhan Yang, and Robby T Tan. 2022. Unsupervised night image
[6] Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. 2016. De-
enhancement: When layer decomposition meets light-e!ects suppression. In
hazenet: An end-to-end system for single image haze removal. IEEE Transactions
European Conference on Computer Vision. Springer, 404–421.
on Image Processing 25, 11 (2016), 5187–5198.
[31] Yeying Jin, Wenhan Yang, Wei Ye, Yuan Yuan, and Robby T Tan. 2022. ShadowD-
[7] Subrahmanyan Chandrasekhar. 2013. Radiative transfer. Courier Corporation.
i!usion: Di!usion-based Shadow Removal using Classi"er-driven Attention
[8] Sixiang Chen, Tian Ye, Yun Liu, Erkang Chen, Jun Shi, and Jingchun Zhou. 2022.
and Structure Preservation. arXiv preprint arXiv:2211.08089 (2022).
Snowformer: Scale-aware transformer via context interaction for single image
[32] Harald Koschmieder. 1924. Theorie der horizontalen Sichtweite. Beitrage zur
desnowing. arXiv preprint arXiv:2208.09703 (2022).
Physik der freien Atmosphare (1924), 33–53.
[9] Sixiang Chen, Tian Ye, Yun Liu, Taodong Liao, Jingxia Jiang, Erkang Chen,
[33] Shiba Kuanar, Dwarikanath Mahapatra, Monalisa Bilas, and KR Rao. 2022. Multi-
and Peng Chen. 2023. MSP-former: Multi-scale projection transformer for
path dilated convolution network for haze and glow removal in nighttime images.
single image desnowing. In ICASSP 2023-2023 IEEE International Conference on
The Visual Computer 38, 3 (2022), 1121–1134.
Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5.
[34] Anat Levin, Dani Lischinski, and Yair Weiss. 2007. A closed-form solution
[10] Zeyuan Chen, Yangchao Wang, Yang Yang, and Dong Liu. 2021. PSD: Princi-
to natural image matting. IEEE transactions on pattern analysis and machine
pled synthetic-to-real dehazing guided by physical priors. In Proceedings of the
intelligence 30, 2 (2007), 228–242.
IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7180–7189.
[35] Boyun Li, Yuanbiao Gou, Shuhang Gu, Jerry Zitao Liu, Joey Tianyi Zhou, and Xi
[11] Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, and Chen Change Loy.
Peng. 2021. You only look yourself: Unsupervised and untrained single image
2022. Flare7k: A phenomenological nighttime #are removal dataset. Advances
dehazing neural network. International Journal of Computer Vision 129, 5 (2021),
in Neural Information Processing Systems 35 (2022), 3926–3937.
1754–1767.
[12] Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yihang Luo, and
[36] Boyun Li, Yuanbiao Gou, Jerry Zitao Liu, Hongyuan Zhu, Joey Tianyi Zhou, and
Chen Change Loy. 2023. Flare7K++: Mixing Synthetic and Real Datasets for
Xi Peng. 2020. Zero-shot image dehazing. IEEE Transactions on Image Processing
Nighttime Flare Removal and Beyond. arXiv preprint arXiv:2306.04236 (2023).
29 (2020), 8457–8466.
[13] Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Qingpeng Zhu, Qian-
[37] Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. 2017.
hui Sun, Wenxiu Sun, Chen Change Loy, Jinwei Gu, Shuai Liu, et al. 2023. MIPI
Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international
2023 Challenge on Nighttime Flare Removal: Methods and Results. In Proceed-
conference on computer vision. 4770–4778.
ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[38] Lerenhan Li, Yunlong Dong, Wenqi Ren, Jinshan Pan, Changxin Gao, Nong
2852–2862.
Sang, and Ming-Hsuan Yang. 2019. Semi-supervised image dehazing. IEEE
[14] Yuekun Dai, Yihang Luo, Shangchen Zhou, Chongyi Li, and Chen Change Loy.
Transactions on Image Processing 29 (2019), 2766–2779.
2023. Nighttime Smartphone Re#ective Flare Removal Using Optical Center
[39] Runde Li, Jinshan Pan, Zechao Li, and Jinhui Tang. 2018. Single image dehaz-
Symmetry Prior. In Proceedings of the IEEE/CVF Conference on Computer Vision
ing via conditional generative adversarial network. In Proceedings of the IEEE
and Pattern Recognition (CVPR). 20783–20791.
Conference on Computer Vision and Pattern Recognition. 8202–8211.
[15] Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming-
[40] Xin Li, Xin Jin, Jianxin Lin, Sen Liu, Yaojun Wu, Tao Yu, Wei Zhou, and Zhibo
Hsuan Yang. 2020. Multi-scale boosted dehazing network with dense feature
Chen. 2020. Learning disentangled feature representation for hybrid-distorted
fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern
image restoration. In Computer Vision–ECCV 2020: 16th European Conference,
recognition. 2157–2167.
Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer, 313–329.
[16] Raanan Fattal. 2008. Single image dehazing. ACM transactions on graphics (TOG)
[41] Xin Li, Bingchen Li, Xin Jin, Cuiling Lan, and Zhibo Chen. 2023. Learning
27, 3 (2008), 1–9.
Distortion Invariant Representation for Image Restoration from A Causality
[17] Alona Golts, Daniel Freedman, and Michael Elad. 2019. Unsupervised single
Perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and
image dehazing using dark channel prior loss. IEEE Transactions on Image
Pattern Recognition. 1714–1724.
Processing 29 (2019), 2692–2701.
[42] Yu Li and Michael S Brown. 2014. Single image layer separation using relative
[18] Chunle Guo, Ruiqi Wu, Xin Jin, Linghao Han, Zhi Chai, Weidong Zhang, and
smoothness. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Chongyi Li. 2023. Underwater Ranker: Learn Which Is Better and How to Be
Recognition. 2752–2759.
Better. In Proceedings of the AAAI conference on arti!cial intelligence.
[43] Yi Li, Yi Chang, Yan Gao, Changfeng Yu, and Luxin Yan. 2022. Physically Disen-
[19] Chun-Le Guo, Qixin Yan, Saeed Anwar, Runmin Cong, Wenqi Ren, and Chongyi
tangled Intra-and Inter-Domain Adaptation for Varicolored Haze Removal. In
Li. 2022. Image Dehazing Transformer with Transmission-Aware 3D Position
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni-
Embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and
tion. 5841–5850.
Pattern Recognition. 5812–5820.
[44] Yu Li, Fangfang Guo, Robby T Tan, and Michael S Brown. 2014. A contrast en-
[20] Lanqing Guo, Renjie Wan, Wenhan Yang, Alex Kot, and Bihan Wen. 2022. En-
hancement framework with JPEG artifacts suppression. In European conference
hancing low-light images in real world via cross-image disentanglement. arXiv
on computer vision. Springer, 174–188.
preprint arXiv:2201.03145 (2022).
[45] Yu Li, Robby T Tan, and Michael S Brown. 2015. Nighttime haze removal with
[21] Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement
glow and multiple light colors. In Proceedings of the IEEE international conference
via illumination map estimation. IEEE Transactions on image processing 26, 2
on computer vision. 226–234.
(2016), 982–993.
[46] Zhuwen Li, Ping Tan, Robby T Tan, Danping Zou, Steven Zhiying Zhou, and
[22] Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Single image haze removal using
Loong-Fah Cheong. 2015. Simultaneous video defogging and stereo recon-
dark channel prior. IEEE transactions on pattern analysis and machine intelligence
struction. In Proceedings of the IEEE conference on computer vision and pattern
33, 12 (2010), 2341–2353.
[23] Kaiming He, Jian Sun, and Xiaoou Tang. 2011. Single image haze removal using recognition. 4988–4997.
dark channel prior. IEEE transactions on pattern analysis and machine intelligence
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

[47] Beibei Lin, Yeying Jin, Wending Yan, Wei Ye, Yuan Yuan, Shunli Zhang, and In Proceedings of the IEEE/CVF International Conference on Computer Vision.
Robby Tan. 2024. NightRain: Nighttime Video Deraining via Adaptive-Rain- 5117–5127.
Removal and Adaptive-Correction. arXiv:2401.00729 [[Link]] [72] Robby T Tan. 2008. Visibility in bad weather from a single image. In 2008 IEEE
[48] Wei Liu, Xianxu Hou, Jiang Duan, and Guoping Qiu. 2020. End-to-end single conference on computer vision and pattern recognition. IEEE, 1–8.
image fog removal using enhanced cycle consistent adversarial networks. IEEE [73] Qunfang Tang, Jie Yang, Xiangjian He, Wenjing Jia, Qingnian Zhang, and Haibo
Transactions on Image Processing 29 (2020), 7819–7833. Liu. 2021. Nighttime image dehazing based on Retinex and dark channel prior
[49] Xiaohong Liu, Yongrui Ma, Zhihao Shi, and Jun Chen. 2019. Griddehazenet: using Taylor series expansion. Computer Vision and Image Understanding 202
Attention-based multi-scale network for image dehazing. In Proceedings of the (2021), 103086.
IEEE/CVF International Conference on Computer Vision. 7314–7323. [74] Carlo Tomasi and Roberto Manduchi. 1998. Bilateral "ltering for gray and
[50] Yun Liu, Zhongsheng Yan, Sixiang Chen, Tian Ye, Wenqi Ren, and Erkang Chen. color images. In Sixth international conference on computer vision (IEEE Cat. No.
2023. NightHazeFormer: Single Nighttime Haze Removal Using Prior Query 98CH36271). IEEE, 839–846.
Transformer. arXiv preprint arXiv:2305.09533 (2023). [75] Jiadong Wang, Xinyuan Qian, Malu Zhang, Robby T Tan, and Haizhou Li.
[51] Yun Liu, Zhongsheng Yan, Jinge Tan, and Yuche Li. 2023. Multi-Purpose Oriented 2023. Seeing What You Said: Talking Face Generation Guided by a Lip Reading
Single Nighttime Image Haze Removal Based on Uni"ed Variational Retinex Expert. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Model. IEEE Transactions on Circuits and Systems for Video Technology 33, 4 Recognition. 14653–14662.
(2023), 1643–1657. [Link] [76] Ming Wang, Xianda Guo, Beibei Lin, Tian Yang, Zheng Zhu, Lincheng Li, Shunli
[52] Yun Liu, Zhongsheng Yan, Aimin Wu, Tian Ye, and Yuche Li. 2022. Nighttime Zhang, and Xin Yu. 2023. DyGait: Exploiting Dynamic Representations for
image dehazing based on variational decomposition model. In Proceedings of High-performance Gait Recognition. In Proceedings of the IEEE/CVF International
the IEEE/CVF conference on computer vision and pattern recognition Workshop. Conference on Computer Vision (ICCV). 13424–13433.
640–649. [77] Wenhui Wang, Anna Wang, and Chen Liu. 2022. Variational Single Nighttime
[53] Yun Liu, Zhongsheng Yan, Tian Ye, Aimin Wu, and Yuche Li. 2022. Single Image Haze Removal With a Gray Haze-Line Prior. IEEE Transactions on Image
nighttime image dehazing based on uni"ed variational decomposition model Processing 31 (2022), 1349–1363.
and multi-scale contrast enhancement. Engineering Applications of Arti!cial [78] Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, and Alex
Intelligence 116 (2022), 105373. Kot. 2022. Low-light image enhancement with normalizing #ow. In Proceedings
[54] Samy Metari and François Deschenes. 2007. A new convolution kernel for of the AAAI Conference on Arti!cial Intelligence, Vol. 36. 2604–2612.
atmospheric point spread function applied to computer vision. In 2007 IEEE 11th [79] Yufei Wang, Yi Yu, Wenhan Yang, Lanqing Guo, Lap-Pui Chau, Alex C Kot, and
international conference on computer vision. IEEE, 1–8. Bihan Wen. 2023. ExposureDi!usion: Learning to Expose for Low-light Image
[55] William Edgar Knowles Middleton. 1957. Vision through the atmosphere. Enhancement. arXiv preprint arXiv:2307.07710 (2023).
Springer. [80] Haiyan Wu, Yanyun Qu, Shaohui Lin, Jian Zhou, Ruizhi Qiao, Zhizhong Zhang,
[56] Srinivasa G Narasimhan and Shree K Nayar. 2000. Chromatic framework for Yuan Xie, and Lizhuang Ma. 2021. Contrastive learning for compact single
vision in bad weather. In Proceedings IEEE Conference on Computer Vision and image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision
Pattern Recognition. CVPR 2000 (Cat. No. PR00662), Vol. 1. IEEE, 598–605. and Pattern Recognition. 10551–10560.
[57] Srinivasa G Narasimhan and Shree K Nayar. 2003. Shedding light on the weather. [81] Ruiqi Wu, Zhengpeng Duan, Chunle Guo, Zhi Chai, and Chongyi Li. 2023.
In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recog- RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors. In
nition, 2003. Proceedings., Vol. 1. IEEE, I–I. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni-
[58] Dubok Park, David K Han, and Hanseok Ko. 2016. Nighttime image dehazing tion.
with local atmospheric light and weighted entropy. In 2016 IEEE International [82] Wanyu Wu, Wei Wang, Zheng Wang, Kui Jiang, and Xin Xu. 2023. From Gener-
Conference on Image Processing (ICIP). IEEE, 2261–2265. ation to Suppression: Towards E!ective Irregular Glow Removal for Nighttime
[59] Soo-Chang Pei and Tzu-Yen Lee. 2012. Nighttime haze removal using color Visibility Enhancement. arXiv:2307.16783 [[Link]]
transfer pre-processing and dark channel prior. In 2012 19th IEEE International [83] Boxue Xiao, Zhuoran Zheng, Yunliang Zhuang, Chen Lyu, and Xiuyi Jia. 2022.
conference on image processing. IEEE, 957–960. Single UHD image dehazing via interpretable pyramid network. Available at
[60] Matti Pietikäinen. 2010. Local binary patterns. Scholarpedia 5, 3 (2010), 9775. SSRN 4134196 (2022).
[61] Xu Qin, Zhilin Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia. 2020. FFA- [84] Zeyu Xiao, Jiawang Bai, Zhihe Lu, and Zhiwei Xiong. 2023. A dive into sam
Net: Feature fusion attention network for single image dehazing. In Proceedings prior in image restoration. arXiv preprint arXiv:2305.13620 (2023).
of the AAAI Conference on Arti!cial Intelligence, Vol. 34. 11908–11915. [85] Zeyu Xiao, Xueyang Fu, Jie Huang, Zhen Cheng, and Zhiwei Xiong. 2021.
[62] Yanyun Qu, Yizi Chen, Jingying Huang, and Yuan Xie. 2019. Enhanced Pix2pix Space-time distillation for video super-resolution. In Proceedings of the IEEE/CVF
Dehazing Network. In Proceedings of the IEEE Conference on Computer Vision conference on computer vision and pattern recognition. 2113–2122.
and Pattern Recognition. 8160–8168. [86] Zeyu Xiao, Zhiwei Xiong, Xueyang Fu, Dong Liu, and Zheng-Jun Zha. 2020.
[63] Wenqi Ren, Si Liu, Hua Zhang, Jinshan Pan, Xiaochun Cao, and Ming-Hsuan Space-time video super-resolution using temporal pro"les. In Proceedings of the
Yang. 2016. Single image dehazing via multi-scale convolutional neural networks. 28th ACM International Conference on Multimedia. 664–672.
In European conference on computer vision. Springer, 154–169. [87] Wending Yan, Robby T Tan, and Dengxin Dai. 2020. Nighttime defogging using
[64] Wenqi Ren, Lin Ma, Jiawei Zhang, Jinshan Pan, Xiaochun Cao, Wei Liu, and high-low frequency decomposition and grayscale-color networks. In European
Ming-Hsuan Yang. 2018. Gated fusion network for single image dehazing. In Conference on Computer Vision. Springer, 473–488.
Proceedings of the IEEE conference on computer vision and pattern recognition. [88] Minmin Yang, Jianchang Liu, and Zhengguo Li. 2018. Superpixel-based single
3253–3261. nighttime image haze removal. IEEE transactions on multimedia 20, 11 (2018),
[65] Yuanjie Shao, Lerenhan Li, Wenqi Ren, Changxin Gao, and Nong Sang. 2020. Do- 3008–3018.
main Adaptation for Image Dehazing. In Proceedings of the IEEE/CVF Conference [89] Yang Yang, Chaoyue Wang, Risheng Liu, Lin Zhang, Xiaojie Guo, and Dacheng
on Computer Vision and Pattern Recognition. 2808–2817. Tao. 2022. Self-Augmented Unpaired Image Dehazing via Density and Depth
[66] Aashish Sharma, Loong-Fah Cheong, Lionel Heng, and Robby T Tan. 2020. Decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision
Nighttime Stereo Depth Estimation using Joint Translation-Stereo Learning: and Pattern Recognition. 2037–2046.
Light E!ects and Uninformative Regions. In 2020 International Conference on [90] Tian Ye, Sixiang Chen, Yun Liu, Yi Ye, Jinbin Bai, and Erkang Chen. 2022. To-
3D Vision (3DV). IEEE, 23–31. wards real-time high-de"nition image snow removal: E$cient pyramid network
[67] Aashish Sharma and Robby T Tan. 2021. Nighttime Visibility Enhancement by with asymmetrical encoder-decoder architecture. In Proceedings of the Asian
Increasing the Dynamic Range and Suppression of Light E!ects. In Proceedings Conference on Computer Vision. 366–381.
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11977– [91] Tian Ye, Sixiang Chen, Yun Liu, Yi Ye, Erkang Chen, and Yuche Li. 2022. Un-
11986. derwater light "eld retention: Neural rendering for underwater imaging. In
[68] Aashish Sharma, Robby T Tan, and Loong-Fah Cheong. 2020. Single-Image Cam- Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni-
era Response Function Using Prediction Consistency and Gradual Re"nement. tion. 488–497.
In Proceedings of the Asian Conference on Computer Vision. [92] Tian Ye, Yunchen Zhang, Mingchao Jiang, Liang Chen, Yun Liu, Sixiang Chen,
[69] Yuda Song, Zhuqing He, Hui Qian, and Xin Du. 2023. Vision transformers and Erkang Chen. 2022. Perceiving and modeling density for image dehazing.
for single image dehazing. IEEE Transactions on Image Processing 32 (2023), In European Conference on Computer Vision. Springer, 130–145.
1927–1941. [93] Hu Yu, Naishan Zheng, Man Zhou, Jie Huang, Zeyu Xiao, and Feng Zhao.
[70] Hang Su, Varun Jampani, Deqing Sun, Orazio Gallo, Erik Learned-Miller, and Jan 2022. Frequency and spatial dual guidance for image dehazing. In Computer
Kautz. 2019. Pixel-adaptive convolutional neural networks. In Proceedings of the Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022,
IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11166–11175. Proceedings, Part XIX. Springer, 181–198.
[71] Zhuo Su, Wenzhe Liu, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pietikäi- [94] Teng Yu, Kang Song, Pu Miao, Guowei Yang, Huan Yang, and Chenglizhao Chen.
nen, and Li Liu. 2021. Pixel di!erence networks for e$cient edge detection. 2019. Nighttime single image dehazing via pixel-wise alpha blending. IEEE
Access 7 (2019), 114619–114630.
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

[95] He Zhang and Vishal M Patel. 2018. Densely connected pyramid dehazing Conference on Multimedia. 2355–2363.
network. In Proceedings of the IEEE conference on computer vision and pattern [99] Xin Zhang and Ying-Cong Chen. 2023. Adaptive domain generalization via
recognition. 3194–3203. online disagreement minimization. IEEE Transactions on Image Processing (2023).
[96] Jing Zhang, Yang Cao, Shuai Fang, Yu Kang, and Chang Wen Chen. 2017. Fast [100] Shiyu Zhao, Lin Zhang, Ying Shen, and Yicong Zhou. 2021. Re"neDNet: A
haze removal for nighttime image using maximum re#ectance prior. In Proceed- weakly supervised re"nement framework for single image dehazing. IEEE
ings of the IEEE conference on computer vision and pattern recognition. 7418–7426. Transactions on Image Processing 30 (2021), 3391–3404.
[97] Jing Zhang, Yang Cao, and Zengfu Wang. 2014. Nighttime haze removal based [101] Zhuoran Zheng, Wenqi Ren, Xiaochun Cao, Xiaobin Hu, Tao Wang, Fenglong
on a new imaging model. IEEE, 4557–4561. Song, and Xiuyi Jia. 2021. Ultra-high-de"nition image dehazing via multi-guided
[98] Jing Zhang, Yang Cao, Zheng-Jun Zha, and Dacheng Tao. 2020. Nighttime de- bilateral learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern
hazing with a synthetic benchmark. In Proceedings of the 28th ACM International Recognition (CVPR). IEEE, 16180–16189.
[102] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired
Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In
Computer Vision (ICCV), 2017 IEEE International Conference on.
MM ’23, October 29-November 3, 2023, O!awa, ON, Canada Yeying Jin et al.

Table 4: The comparison with dataset [98], for PSNR and SSIM, higher values indicate better performance. We tested our method
and Yan [87] on a GPU GTX 3090 using an image resolution of 512x512 to measure the runtime. Other numbers in the table are
borrowed from [98].
NHR NHM NHC
Type Method Venue Parameters Time (s)
PSNR ⇔ SSIM ⇔ PSNR ⇔ SSIM ⇔ PSNR ⇔ SSIM ⇔
Zhang NDIM [97] ICIP’14 14.31 0.53 14.58 0.56 11.12 0.29 - 5.63
Li GS [45] ICCV’15 17.32 0.63 16.84 0.69 18.84 0.55 - 22.52
Opti. Zhang FAST-MRP [96] CVPR’17 16.95 0.67 13.85 0.61 19.17 0.58 - 0.236
Zhang MRP [96] CVPR’17 19.93 0.78 17.74 0.71 23.02 0.69 - 1.769
Zhang OSFD [98] MM’20 21.32 0.80 19.75 0.76 23.10 0.74 - 0.576
Zhang NDNET [98] MM’20 28.74 0.95 21.55 0.91 26.12 0.85 - 0.0074
Learning Yan [87] ECCV’20 21.05 0.62 17.54 0.45 15.06 0.46 50M 0.97
Learning Ours MM’23 26.56 0.89 33.76 0.92 38.86 0.97 21M 1.20

Figure 12: We show with APSF, we can render glow 𝑀𝑅 (bottom) on night clean 𝑀𝐿 (top).

Figure 13: We show with APSF, we can render glow 𝑀𝑅 (bottom) on night clean 𝑀𝐿 (top).
Enhancing Visibility in Nigh!ime Haze Images Using Guided APSF and Gradient Adaptive Convolution MM ’23, October 29-November 3, 2023, O!awa, ON, Canada

Figure 14: We show with APSF, we can render glow 𝑀𝑅 (bottom) on night clean 𝑀𝐿 (top).

Figure 15: We show with APSF, we can render glow 𝑀𝑅 (bottom) on night clean 𝑀𝐿 (top).