1
University of Wisconsin Madison
Electrical Computer Engineering
ECE533 Digital Image Processing
Image Denoising Using Wavelet
Thresholding Techniques
Yang Yang
2
Abstract
Wavelet transforms enable us to represent signals with a high degree of
scarcity. Wavelet thresholding is a signal estimation technique that
exploits the capabilities of wavelet transform for signal denoising. The aim
of this project was to study various thresholding techniques such as
SureShrink, VisuShrink and BayeShrink and determine the best one for
image denoising.
3
TABLE OF CONTENTS
1 INTRODUCTION.......................................................... 4
2 THRESHOLDING......................................................... 4
2.1 INTRODUCTION............................................................ 4
2.2 HARD AND SOFT THRESHOLDING................................. 5
2.3 THRESHOLD SELECTION .............................................. 6
2.4 COMPARISON WITH UNIVERSAL THRESHOLD .............. 7
3 IMAGE DENOISING USING THRESHOLDING.... 8
3.1 VISUSHRINK................................................................ 9
3.2 SURESHRINK ............................................................... 9
3.3 BAYESSHRINK........................................................... 10
4 EXPERIMENTAL RESULTS .................................... 10
5 CONCLUSIONS.......................................................... 24
REFERENCES................................................................ 25
4
1 Introduction
In many applications, image denoising is used to produce good estimates of the original
image from noisy observations. The restored image should contain less noise than the
observations while still keep sharp transitions (i.e. edges).
Wavelet transform, due to its excellent localization property, has rapidly become an
indispensable signal and image processing tool for a variety of applications, including
compression and denoising [1, 2, 3]. Wavelet denoising attempts to remove the noise
present in the signal while preserving the signal characteristics, regardless of its
frequency content. It involves three steps: a linear forward wavelet transform, nonlinear
thresholding step and a linear inverse wavelet transform.
Wavelet thresholding (first proposed by Donoho [1, 2, 3]) is a signal estimation technique
that exploits the capabilities of wavelet transform for signal denoising. It removes noise
by killing coefficients that are insignificant relative to some threshold, and turns out to be
simple and effective, depends heavily on the choice of a thresholding parameter and the
choice of this threshold determines, to a great extent the efficacy of denoising.
Researchers have developed various techniques for choosing denoising parameters and so
far there is no best universal threshold determination technique.
The aim of this project was to study various thresholding techniques such as
SureShrink[1], VisuShrink[3] and BayesShrink[5] and determine the best one for image
denoising.
2 Thresholding
2.1 Introduction
The plot of wavelet coefficients in Fig 1 suggests that small coefficients are dominated by
noise, while coefficients with a large absolute value carry more signal information than
noise. Replacing noisy coefficients (small coefficients below a certain threshold value)
5
by zero and an inverse wavelet transform may lead to a reconstruction that has lesser
noise.
Fig 1: A noisy signal in time domain and wavelet domain. Note the scarcity of coefficients.
2.2 Hard and soft thresholding
Hard and soft thresholding with threshold are defined as follows:
The hard thresholding operator is defined as:
D(U, ) = U for all |U|>
= 0 otherwise
The soft thresholding operator on the other hand is defined as:
D(U, ) = sgn(U)max(0, |U| - )
Fig2: Hard Thresholding Fig3: Soft Thresholding
Hard threshold is a keep or kill procedure and is more intuitively appealing. The
transfer function of the same is shown in Fig 2. The alternative, soft thresholding
(whose transfer function is shown in Fig3), shrinks coefficients above the threshold in
absolute value. While at first sight hard thresholding may seem to be natural, the
0 500 1000 1500 2000 2500
10
5
0
5
10
15
20
25
0 500 1000 1500 2000 2500
6
4
2
0
2
4
6
8
6
continuity of soft thresholding has some advantages. It makes algorithms
mathematically more tractable [3]. Moreover, hard thresholding does not even work
with some algorithms such as the GCV procedure [4]. Sometimes, pure noise
coefficients may pass the hard threshold and appear as annoying blips in the output.
Soft thesholding shrinks these false structures.
2.3 Threshold selection
As one may observe, threshold selection is an important question when denoising. A
small threshold may yield a result close to the input, but the result may still be noisy. A
large threshold on the other hand, produces a signal with a large number of zero
coefficients. This leads to a smooth signal. Paying too much attention to smoothness,
however, destroys details and in image processing may cause blur and artifacts.
The setup is as follows:
1. The original signals have length 2048.
2. We step through the thresholds from 0 to 5 with steps of 0.2 and at each step
denoise the four noisy signals by both hard and soft thresholding with that
threshold.
3. For each threshold, the MSE of the denoised signal is calculated.
4. Repeat the above steps for different orthogonal bases, namely, Haar, Daubechies.
The results are tabulated in the table 1
Table 1: Best thresholds, empirically found with different denoising schemes, in terms of MSE
7
2.4 Comparison with Universal threshold
The threshold N
UNIV
ln 2 = (N being the signal length,
2
being the noise
variance) is well known in wavelet literature as the Universal threshold. It is the
optimal threshold in the asymptotic sense and minimizes the cost function of the
difference between the function and the soft thresholded version of the same in the L2
norm sense (i.e. it minimizes
2
Orig Thresh
Y Y E ). In our case, N=2048, = 1, therefore
theoretically, ( ) 905 . 3 ) 1 ( 2048 ln 2 = =
UNIV
. As seen from the table, the best
empirical thresholds for both hard and soft thresholding are much lower than this value,
independent of the wavelet used. It therefore seems that the universal threshold is not
useful to determine a threshold. However, it is useful for obtain a starting value when
nothing is known of the signal condition. One can surmise that the universal threshold
may give a better estimate for the soft threshold if the number of samples is larger (since
the threshold is optimal in the asymptotic sense).
Figure 4: MSE V/s Threshold values for the four test signals.
0 1 2 3 4 5
0
0.2
0.4
0.6
0.8
1
threshold
M
S
E
Blocks
soft
hard
0 1 2 3 4 5
0
0.2
0.4
0.6
0.8
1
threshold
M
S
E
Bumps
0 1 2 3 4 5
0
0.2
0.4
0.6
0.8
1
threshold
M
S
E
Heavisine
0 1 2 3 4 5
0
0.2
0.4
0.6
0.8
1
threshold
M
S
E
Doppler
8
3 Image Denoising using Thresholding
The problem boils down to finding an optimal threshold such that the mean squared error
between the signal and its estimate is minimized. The wavelet decomposition of an
image is done as follows: In the first level of decomposition, the image is split into 4
subbands, namely the HH, HL, LH and LL subbands. The HH subband gives the
diagonal details of the image; the HL subband gives the horizontal features while the LH
subband represents the vertical structures. The LL subband is the low resolution
residual consisting of low frequency components and it is this subband which is further
split at higher levels of decomposition. The different methods for denoising we
investigate differ only in the selection of the threshold.
Fig5: 2D DWT
The basic procedure remains the same:
1. Calculate the DWT of the image.
2. Threshold the wavelet coefficients.(Threshold may be universal or subband
adaptive)
9
3. Compute the IDWT to get the denoised estimate.
Soft thresholding is used for all the algorithms due to the following reasons: Soft
thresholding has been shown to achieve near minimax rate over a large number of Besov
spaces [3]. Moreover, it is also found to yield visually more pleasing images. Hard
thresholding is found to introduce artifacts in the recovered images. We now study
three thresholding techniques- VisuShrink, SureShrink and BayesShrink and investigate
their performance for denoising on my own picture.
3.1 VisuShrink
VisuShrink is thresholding by applying the Universal threshold proposed by Donoho and
Johnstone [2]. This threshold is given by M log 2 where is the noise variance
and M is the number of pixels in the image. It is proved in [2] that the maximum of any
M values iid as N(0,
2
) will be smaller than the universal threshold with high
probability, with the probability approaching 1 as M increases. Thus, with high
probability, a pure noise signal is estimated as being identically zero. However, for
denoising images, VisuShrink is found to yield an overly smoothed estimate as seen in
Figure 7.1-7.4. This is because the universal threshold (UT) is derived under the
constraint that with high probability, the estimate should be at least as smooth as the
signal. So the UT tends to be high for large values of M, killing many signal coefficients
along with the noise. Thus, the threshold does not adapt well to discontinuities in the
signal.
3.2 SureShrink
SureShrink is a thresholding by applying subband adaptive threshold, a separate threshold
is computed for each detail subband based upon SURE (Steins unbiased estimator for
risk), a method for estimating the loss
2
in an unbiased fashion. In our case let
wavelet coefficients in the jth subband be { Xi : i =1,,d },
is the soft threshold
10
estimator
) (
i t i
X X =
, we apply Steins result [1] to get an unbiased estimate of the risk
( )
( )
2
x E
t
:
{ } ( )
2
1
, min : # 2 ) ; (
=
+ =
d
i
i i
t X t X i d X t SURE
For an observed vector x (in our problem, x is the set of noisy wavelet coefficients in a
subband), we could find the threshold
S
t
that minimizes SURE (t: x),
) ; ( min arg X t SURE t
S
=
The results obtained for the image on my picture using SureShrink are shown in Figure
8.1-8.5. The Db5 wavelet coefficient was used with 4 levels decomposition. Clearly,
the results are much better than VisuShrink. The sharp features of image are retained
and the MSE is considerably lower. This because SureShrink is subband adaptive.
3.3 BayesShrink
BayesShrink [5] is an adaptive data-driven threshold for image denoising via wavelet
soft-thresholding. The threshold is driven in a Bayesian framework, and we assume
generalized Gaussian distribution (GGD) for the wavelet coefficients in each detail
subband and try to find the threshold T which minimizes the Bayesian Risk. The results
obtained by BayesShrink for my own picture is shown in Figure 9.1-9.5. The Db5
wavelet was used with four levels decomposition. We found that BayesShrink performs
better than SureShrink in terms of MSE. The reconstruction using BayesShrink is
smoother and more visually appealing than one obtained using SureShrink.
4 Experimental Results
Following are the results of running the denoising algorithms for the methods discussed
above on my picture. The denoising is done after adding the Gaussian noise with standard
deviation 18 and mean 0 in the original picture. The MSE is calculated and compared
for all the methods (see Figure 10). The results are provided as a bar chart in the end as
11
well they are tabulated. The figures and MSE are compared for the global and adaptive
thresholding techniques. The results are provided for thresholds selected by VisuShrink
hard and soft methods and thresholds selected by default universal threshold for hard and
soft thresholding. Similar thing is done with the BayesShrink and SureShrink methods.
12
GLOBAL THRESHOLDING
Threshold Selection by
Universal Thresholding and VISU Shrink
Universal Threshold:
Original Image Noisy Image
Denoised Image using Haar at level 1 obtained
by wavelet coefficients thresholding using global
positive HARD threshold
Denoised Image using Haar at level 1 obtained
by wavelet coefficients thresholding using global
positive SOFT threshold
13
Original Image Noisy Image
Denoised Image using Daubechies5 at level 1
obtained by wavelet coefficients thresholding
using global positive HARD threshold
Denoised Image using Daubechies5 at level 1
obtained by wavelet coefficients thresholding
using global positive SOFT threshold
14
Original Image Noisy Image
Denoised Image using Daubechies5 at level 2 obtained by
wavelet coefficients thresholding using global positive
HARD threshold
Denoised Image using Daubechies5 at level 2 obtained by
wavelet coefficients thresholding using global positive
SOFT threshold
15
VISU Shrink
Figure 7.1
Original Image Noisy Image
VisuShrink by Hard Thresholding Haar at level 1 VisuShrink by Soft Thresholding Haar at level 1
16
Figure 7.2
Original Image Noisy Image
VisuShrink by Hard Thresholding db5 at level 1 VisuShrink by Soft Thresholding db5 at level 1
17
Figure 7.3
Original Image Noisy Image
VisuShrink by Hard Thresholding db5 at level 2 VisuShrink by Soft Thresholding db5 at level 2
18
Figure 7.4
Original Image Noisy Image
VisuShrink by Hard Thresholding db5 at level 3 VisuShrink by Soft Thresholding db5 at level 3
19
ADAPTIVE THRESHOLDING
Threshold Selection by
Bayes Shrink and SURE Shrink
Figure 8.1 & Figure 9.1
Original Image Noisy Image
Bayes Shrink with Haar at level 1 SURE Shrink with Haar at level 1
20
Figure 8.2 & 9.2
Original Image Noisy Image
Bayes Shrink with Daubechies5 at level 1 SURE Shrink with Daubechies5 at level 1
21
Figure 8.3 & 9.3
Original Image Noisy Image
Bayes Shrink with db5 at level 2 SURE Shrink with db5 at level 2
22
Figure 8.4 & 9.4
Original Image Noisy Image
Bayes Shrink with db5 at level 3 SURE Shrink with db5 at level 3
23
Figure 8.5 & 9.5
Original Image Noisy Image
Bayes Shrink with db5 at level 4 SURE Shrink with db5 at level 4
24
Image Denoising MSE vs Methods
0
20
40
60
80
100
120
140
Bayes
SHRINK
SURE
SHRINK
Defaul Hard Default Soft Visu Shrink
Hard
Visu Shrink
Soft
Methods
M
S
E
Haar level 1
db5 level1
db5 level2
db5 level3
db5 level4
Figure 10 Comparison between all the methods based on MSE
5 Conclusions
We have seen that wavelet thresholding is an effective method of denoising noisy signals.
We first tested hard and soft on noisy versions of the standard 1-D signals and found the
best threshold. We then investigated many soft thresholding schemes such as
VisuShrink, SureShrink and BayesShrink for denoising images. We found that subband
adaptive thresholding performs better than a universal thresholding. Among these,
BayesShrink gave the best results. This validates the assumption that the GGD is a very
good model for the wavelet coefficient distribution in a subband.
An important point to note is that although SureShrink performed worse than
BayesShrink, it adapts well to sharp discontinuities in the signal. This was not evident
in the natural images we used for testing. It would be instructive to compare the
performance of these algorithms on artificial images with discontinuities (such as medical
images). It would also be interesting to try denoising (and compression) using other
25
special cases of the GGD such as the Laplacian (GGD with = 1). Most images can
be described with a GGD with shape parameter ranging from 0.5 to 1. So a
Laplacian prior may give better results than a Gaussian prior ( = 2) although it may
not be as easy to work with.
References
[1] Iain M.Johnstone David L Donoho. Adapting to smoothness via wavelet shrinkage.
Journal of the Statistical Association, 90(432):12001224, Dec 1995.
[2] David L Donoho. Ideal spatial adaptation by wavelet shrinkage. Biometrika,
[3] David L Donoho. De-noising by soft thresholding. IEEE Transactions on Information
Theory, 41(3):613627, May 1995.
[4] Maarten Jansen. Noise Reduction by Wavelet Thresholding, volume 161. Springer
Verlag, United States of America, 1 edition, 2001.
[5] Martin Vetterli S Grace Chang, Bin Yu. Adaptive wavelet thresholding for image
denoising and compression. IEEE Transactions on Image Processing, 9(9):15321546,
Sep 2000.
[6] Carl Taswell. The what, how and why of wavelet shrinkage denoising. Computing in
Science and Engineering, pages 1219, May/June 2000.
