Accepted Manuscript

A Model-based Approach to Camera’s Auto Exposure Control

Yuanhang Su, Joe Yuchieh Lin, C.-C. Jay Kuo

PII: S1047-3203(16)00020-1
DOI: http://dx.doi.org/10.1016/j.jvcir.2016.01.011
Reference: YJVCI 1672

To appear in: J. Vis. Commun. Image R.

Received Date: 14 June 2015

Accepted Date: 19 January 2016

Please cite this article as: Y. Su, J.Y. Lin, C.-C. Jay Kuo, A Model-based Approach to Camera’s Auto Exposure
Control, J. Vis. Commun. Image R. (2016), doi: http://dx.doi.org/10.1016/j.jvcir.2016.01.011

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers
we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting proof before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
 A Model-based Approach to Camera’s Auto Exposure
Control

Yuanhang Sua,1,∗, Joe Yuchieh Linb , C.-C Jay Kuob

a Shanghai Aerospace Electronic Technology Institute, 1777 Zhongchun Road, Minhang
District, Shanghai, China
b University of Southern California, Ming Hsieh Department of Electrical Engineering, 3740

McClintock Avenue, Los Angeles, CA, United States

Abstract
A fast and robust camera’s auto exposure (AE) technique is proposed in this
work. It is achieved by modeling the luminance characteristics of the imaging
sensor as a concave or convex function of a control parameter (e.g., exposure
time or speed) and the optimal control parameter is computed using a modified
secant algorithm with fast convergence. Furthermore, the proposed solution is
able to adjust the control parameter automatically in the presence of erroneous
exposure. Its superior performance is confirmed by experimental results .
Keywords: Auto Exposure Control, Convex Function, Digital Cameras,
Numerical Analysis

1. Introduction

Auto exposure (AE) control is an important function in modern digital cam-

eras. It enables a camera to automatically adjust its exposure settings to adapt
to the scene for the imaging purpose. Despite the recent development of the
5 high dynamic range (HDR) imaging system in the high end digital camera mar-
ket, AE is still widely used in today’s digital cameras. This is especially true for
video cameras and medium to low end cameras in smart phones and laptops.
Furthermore, AE is one of the required modules in existing HDR imaging sys-
tems. Although the AE problem and its implementation have been well studied,
10 there are still a few challenges to address.
One of the challenges is to develop a general AE solution that covers a wide
variety of imaging sensors. Different sensors use different AE control methods,
where each method has its own specific control parameters. Specifically, camera

∗ Corresponding Author: Yuanhang Su, Shanghai Aerospace Electronic Technology Insti-

tute, Shanghai, China; [email protected]; +8613817443693

1 Present Address: 5-A-201 Changxing Hao Rizi, Xinbei District, Changzhou, Jiangsu,

China

Preprint submitted to Journal of Visual Communication and Image RepresentationJanuary 23, 2016
 aperture control, automatic gain control (AGC) and electrical shutter control
15 are three main methods [1]. Camera aperture control adjusts the amount of
light intensity or irradiance to pass onto the sensor. AGC adjusts the analog
or digital gain of input signals. Electrical shutter control adjusts the exposure
time for the complementary-oxide-semiconductor (CMOS) imaging sensor or
the shutter speed for the charge-coupled-device (CCD) imaging sensor.
20 An assumption of a linear relationship between the exposure time and the
image brightness level (BL), which is the averaged brightness value per pixel,
was made in Kuno et al. [2] and Liang et al. [3]. However, the limited dynamic
range of imaging sensors contributes to the non-linear characteristics due to over-
or under-exposure as shown in Figure 1, where a simplified relationship between
25 the image BL and the control parameter of an imaging sensor is illustrated. It
can be seen that the so called linear relationship is actually piece-wise linear.
Details of this non-linearity will be further discussed in Section 3.4.

Camera Response.png

Figure 1: Simplified relationship between image brightness and exposure

time of an imaging sensor.

The AE control in CCD cameras is achieved by selecting a proper shutter

speed, which has a reciprocal relationship with BL. A root-finding technique,
30 called the false position method, was adopted by Cho et al. [1] to determine the
shutter speed for desired AE performance. The AE control in CMOS cameras,
which are most common in today’s consumer electronics, is determined by the
exposure time. An iterative method to search for the proper exposure time
was proposed by Liang et al. [3]. Although these two methods work well in a
35 normal lighting condition that satisfies their respective specifications, they do
encounter a tumbling effect under a dim lighting condition. This problem is still
not well resolved today. Such a constraint limits the AE control performance

2
 for imaging sensors. The cause of the tumbling effect and its solution will be
elaborated in Section 3.
40 To implement AE control in a resource constrained environment such as
phone cameras presents another challenge. Some advanced techniques devel-
oped to tackle high contrast lighting conditions are computationally intensive,
and they are not as competitive as the HDR camera [4, 5] in the high end mar-
ket. The histogram of image brightness or other statistical approaches are used
45 [6, 7, 8, 9, 10] to detect whether the region of interest is over-/under-exposed
and whether the dynamic range of a camera system should be extended. Ob-
ject detection and image segmentation techniques are used [3, 10, 11, 12] to
determine proper exposure settings. Despite these recent developments, none
of them provide robust performance when erroneous exposure occurs.
50 To address the aforementioned challenges, a fast and robust AE control al-
gorithm was recently developed in Su et al. [13] with several attractive features.
First, it covers a wide variety of camera sensors yet allows fast and simple im-
plementation. Second, it adjusts itself automatically when erroneous exposure
happens. The proposed AE control algorithm is based on convex or concave
55 modeling of the relationship between a luminance function and its control pa-
rameter. It determines the control parameter at a predefined BL value using a
modified secant method. The proposed algorithm can also be integrated into
the HDR camera system as a multiple exposure function [4, 5, 11].
As compared with prior work [13], the material in sections 3 and 4 is new.
60 In this work, the convex or concave luminance model is generalized into four
categories. The properties of monotonic convergence and error tolerance of the
AE control method are proved mathematically. The performance of classic AE
control methods can also be well explained. The current work has more thorough
mathematical treatment in Section 3 and more extensive experimental results
65 in Section 4.
The rest of this paper is organized as follows. The mathematical background
is reviewed in Section 2. The AE function of an imaging sensor is treated as a
root finding problem with respect to a convex or concave function in Section 3
and the merits of the proposed AE control algorithm are proved. Experimental
70 results are shown in Section 4.Finally, concluding remarks are given in Section
5.

2. Background Review

Root-finding algorithms have been developed in numerical analysis to solve

f(X)=0 for variable X without the exact form of f (·). In practice, this can be
75 generalized to the solution of f(X)-Ft =0, where Ft =f(Xt ) is the target function
value and Xt is the final solution. Several root-finding algorithms are reviewed
and their limitations are explained below.

3
 2.1. Bisection Method
Given two arbitrary initial points x0 and x1 of opposite signs, the bisection
80 method calculates new point xn+1 recursively as
Xn + Xn−1
Xn+1 = , n = 1, 2, 3... (1)
2
where Xn and Xn−1 are chosen according to the root bracketing rule such that
f (Xn ) and f (Xn−1 ) are of opposite signs throughout the iteration. This is
accomplished by either keeping Xn−1 unchanged or updating Xn−1 with the
value of Xn depending on the sign of f (Xn+1 ). Then, update Xn with the value
85 of Xn+1 . The above iteration stops if the distance between f (Xn+1 ) and Ft
(or the distance between Xn+1 and Xn ) is less than a preset threshold. Then,
Xn+1 is chosen as the desired solution
In the context of AE control, x0 and x1 can be chosen as the minimum and
maximum of the control parameter, respectively, to ensure that the solution lies
90 in interval [x0 , x1 ]. One of the advantages of the bisection method is that it
always converges so that a solution can always be found if f (·) is continuous.

Figure 2: Illustration of the tumbling effect of the bisection method with

f (x) = −ln(x + 20), where ln is the natural logarithm. The target value is
Ft = -9.5 while the initial values are X0 = 50 and X1 = 32000. (a) The
first iteration gives X2 = 16025, f (X2 ) = -10 which is smaller than Ft . (b)
The second iteration gives X3 = 8037.5, f (X3 ) = -9 which is larger than
Ft .

4
 However, the bisection method is not suitable for AE control since its root
bracketing rule does not guarantee monotonic convergence. That is, f (Xn+1 )
may alternate its sign under certain circumstances, which leads to a phenomenon
95 called the tumbling effect. One example is given in Figure 2. The observer will
see alternating image brightness between over-exposure and under-exposure as
a result of the tumbling effect.

2.2. False Position Method

The false position method is adopted for AE control by Kiran et al. [14].
100 By following a similar procedure, it calculates new point xn+1 recursively as

f (Xn )(Xn − Xn−1 )

Xn+1 = Xn − , n = 1, 2, 3... (2)
f (Xn ) − f (Xn−1 )
The graphical illustration of the above iterative process can be found in Cho
et al. [1] and Kiran et al. [14]. Since the false position method also uses the
root bracketing rule, the tumbling effect is inevitable.

Method.png Method Invalid Region.png

Figure 3: Illustration of the iterative root-finding process of the secant

method with f (x) = −ln(x + 20). (a) Ft = -10, Xn−1 = 1000, Xn = 5000
and Xn+1 = 8711.8, and (b) Ft = -9, Xn−1 = 32000, Xn = 30000 and
Xn+1 = -10610.

2.3. Secant Method

105 Unlike the previous two methods, the secant method does not require the
root to be bracketed. It selects two arbitrary points x0 and x1 and calculates
the new point xn+1 recursively using Eq. 2. An exemplary iterative process is
shown in Figure 3.
The relaxation of the root bracketing rule compromises its ability to con-
110 verge. Even if the method does converge, it may not converge monotonically so
that the tumbling effect may still exist. As shown in Figure 3b, the calculated
xn+1 lies beyond the valid region, and the secant method does not converge in
this case. Furthermore, divergence may happen if the method is applied to a
function that has multiple turning points.

5
115 Once a wrong xn+1 or f (Xn+1 ) value is reached, the error will propagate in
future iterations in these root-finding methods. This is especially severe in the
false position and the secant methods. A monotonically convergent root-finding
method will be presented to overcome this issue in the next section.

Convex.png Convex.png

Figure 4: Illustration of the modified secant method with final solution

Xt under two exemplary cases: (a) a decreasing convex model f (x) =
−ln(x + 20), Xmin = 50, Xmax = 32000, Yf ixed = -4 and Xn = 5000, and
Ft = -10, (b) an increasing convex model f (x) = exp(x), where exp is the
natural exponential, Xmin = -5, Xmax = 5, Yf ixed = 149, Xn = 0, and
Ft = 74.

Concave.png Concave.png

Figure 5: Illustration of the modified secant method for (a) a decreasing

concave model in form of f (x) = ln(−x + 32000), Xmin = 50, Xmax =
32000, Yf ixed = -3.44, Xn = 5000 and Ft = 6, and (b) an increasing
concave model in form of f (x) = ln(x), Xmin = 50, Xmax = 32000, Yf ixed
= 2.912, Xn = 10000, and Ft = 7.14.

6
 3. Model-Based AE Control Method

120 3.1. Model-based Modified Secant Method

A modified secant method is proposed to determine X for a convex or concave
function, f (·) , defined between Xmin and Xmax .
As shown in Figure 4 and Figure 5, Xn is updated with respect to a fixed
point denoted by (Xf ixed , Yf ixed ). By imposing the constraint

Xn+1 − Xf ixed Xn − Xf ixed

|tan θn | = = , (3)
Ft − Yf ixed f (Xn ) − Yf ixed
125 or, equivalently,
Xn+1 − Xf ixed Xn − Xf ixed
= , (4)
Ft − Yf ixed f (Xn ) − Yf ixed
an iterative formula is obtained to update xn+1 in form of

αXn + f (Xn )Xf ixed − β

Xn+1 = , n = 0, 1, 2... (5)
f (Xn ) − Yf ixed
where α = Ft -Yf ixed , β=Ft ·Xf ixed , Ft =f (Xt ) is a predefined target value chosen
by the user, and Xt is the final solution.
Typically, Ft is set to the mid-value of the full dynamic range (e.g., 128 for an
130 8-bit image). The above iteration stops if the distance between f (Xn+1 ) and Ft
(or the distance between Xn+1 and Xn ) is less than a preset threshold. Then,
Xn+1 is chosen as the desired solution. The positions of the fixed point have
to be carefully set to ensure monotonic convergence as shown in Table 1. The
recommended fix point positions are shown in Table 2, where ε is a small real
135 positive number, which is often set to 1 for the ease of integer implementation
in an embedded system.

Table 1: Restrictions of fixed point positions

Xf ixed Yf ixed
Decreasing convex model Xf ixed ≤Xmin Yf ixed > f (Xf ixed )
Increasing convex model Xf ixed ≥Xmax Yf ixed > f (Xf ixed )
Decreasing concave model Xf ixed ≥Xmax Yf ixed < f (Xf ixed )
Increasing concave model Xf ixed ≤Xmin Yf ixed < f (Xf ixed )

Table 2: Recommended fixed point positions

Xf ixed Yf ixed
Decreasing convex model Xmin f (Xmin )+ε
Increasing convex model Xmax f (Xmax )+ε
Decreasing concave model Xmax f (Xmax )−ε
Increasing concave model Xmin f (Xmin )−ε

7
 If f (Xmin ) or f (Xmax ) in Table 2 is unknown, the minimum and the maxi-
mum of the dynamic range is used for f (Xmin ) and f (Xmax ), respectively (e.g.,
0 for f (Xmin ) and 255 for f (Xmax ) for an 8 bit image). It is worthwhile to
140 point out that Xn cannot be the same as Xf ixed ; otherwise, Xn+1 =Xn . In
practice, one can set Xn =Xmin +ε if Xn =Xmin or Xn =Xmax -ε if Xn =Xmax ,
where ε is a small positive number.
The formulae of tan θn and Xn+1 for each model using these recommended
fixed point positions are summarized below:

145 3.1.1. Decreasing Convex Model

Xn+1 − Xmin Xn − Xmin

tan θn = = (6)
Yf ixed − Ft Yf ixed − f (Xn )
αXn − f (Xn )Xmin + β
Xn+1 = , n = 0, 1, 2... (7)
Yf ixed − f (Xn )
where α=Yf ixed -Ft and β=Ft ·Xmin .

3.1.2. Increasing Convex Model

Xmax − Xn+1 Xmax − Xn

tan θn = = (8)
Yf ixed − Ft Yf ixed − f (Xn )
αXn − f (Xn )Xmax + β
Xn+1 = , n = 0, 1, 2... (9)
Yf ixed − f (Xn )
where α=Yf ixed -Ft and β=Ft ·Xmax .

3.1.3. Decreasing Concave Model

Xmax − Xn+1 Xmax − Xn

tan θn = = (10)
Ft − Yf ixed f (Xn ) − Yf ixed
f (Xn )Xmax + αXn − β
Xn+1 = , n = 0, 1, 2... (11)
f (Xn ) − Yf ixed
150 where α=Ft -Yf ixed and β=Ft ·Xmax .

3.1.4. Increasing Concave Model

Xn+1 − Xmin Xn − Xmin

tan θn = = (12)
Ft − Yf ixed f (Xn ) − Yf ixed
f (Xn )Xmin + αXn − β
Xn+1 = , n = 0, 1, 2... (13)
f (Xn ) − Yf ixed
where α=Ft -Yf ixed and β=Ft ·Xmin .

8
 3.2. Analysis of Convergence Properties
The model-based AE control method has three properties: monotonic con-
155 vergence, robustness against error and a linear convergence rate. The proof of
all properties is given below. The properties are only proved with respect to a
decreasing convex model since they can be proved in a similarly fashion for other
models. In the following, f (·) is assumed to be continuous and monotonically
decreasing.

160 3.2.1. Monotonic Convergence

It can be derived from Eqs. 6 and 7:
Yf ixed − Ft
tan θn = · tan θn−1 , n = 1, 2, 3... (14)
Yf ixed − f (Xn )
Since f (·) is continuous and monotonically decreasing and Yf ixed > f (Xn ),
it is concluded that, if Xn ≤Xt , then tan θn ≥ tan θn−1 , where Xt is the solution
and Ft =f (Xt ). Together with Eq. 6, one obtains Xn+1 ≥Xn .
165 By the definition of the convex function f (·): X → R, ∀X1 , X2 ∈ X, ∀w ∈
[0, 1], f (wX1 +(1−w)X2 ) ≤ wf (X1 )+(1−w)f (X2 ). For Xn ≤Xt , by substituting
X1 and X2 with Xmin and Xt , respectively, and with and Xn = wX1 +(1−w)X2
and Yf ixed >f (Xmin ), one has

f (Xn ) ≤ wYf ixed + (1 − w)f (Xt ), ∀w ∈ [0, 1]. (15)

This means point (Xn , f (Xn )) lies below the straight line connecting fixed
170 point (Xf ixed , Yf ixed ) and target point (Xt , Ft ). Therefore, if Xn ≤Xt , then
θn ≤ θt < π/2 and tan θn ≤ tan θt . This leads to Xn+1 ≤Xt . As a result,
Xn ≤Xn+1 ≤Xt . It can be similarly proved that if Xn >Xt , then Xt <Xn+1 <Xn .
Thus, the modified secant method converges monotonically.

3.2.2. Robustness
175 The error is introduced by defining an error function:

Ferror (Xn ) = f (Xn ) + εn , n = 0, 1, 2... (16)

where f (·) is the ground truth, εn is the error in the nth iteration. Eq. 14 can
be rewritten as:
Yf ixed − Ft
tan θn = · tan θn−1 , n = 1, 2, 3... (17)
Yf ixed − Ferror (Xn )
For the monotonic convergence property to hold, Ferror (Xn ) has to satisfy
the following condition:

Ft ≤ Ferror (Xn ) ≤ wYf ixed + (1 − w)Ft , ∀w ∈ [0, 1) (18)

9
180 when Xn ≤Xt , and
Ft − wYf ixed
≤ Ferror (Xn ) < Ft , ∀w ∈ (0, 1) (19)
1−w
when Xn >Xt . Eqs. 18 and 19 mean that as long as point (Xn , Ferror (Xn ))
lies within the triangular area defined by three vertices (Xmin , Yf ixed ), (Xmin ,
Ft ) and (Xt , Ft ) as shown in Figure 4a if Xn ≤Xt or the one defined by three
vertices of (Xt , Ft ), (Xmax , Ft ) and (Xmax , fmin ) if Xn >Xt , the monotonic
185 convergence property still holds with

(Xt − Xmax )Yf ixed + (Xmax − Xmin )Ft

fmin = (20)
Xt − Xmin
The value of Yf ixed can be increased to enlarge the two triangular areas for
better error tolerance performance, yet to trade for a slower convergence speed
as discussed below.

3.2.3. Linear Convergence Rate

190 Let Xn =Xt +ξn and ff (Xn )=f (Xn )-Ft . Then, Xt is the number Xn converges
to. The convergence rate of sequence Xn can be analyzed as follows. Eq. 5 can
be rewritten as

α(Xt + ξn ) + ff (Xt + ξn )Xf ixed

Xt + ξn+1 = , n = 0, 1, 2... (21)
ff (Xt + ξn ) + α
0 00
Assume ff (·) is second-order differentiable. If ff (Xt ) and ff (Xt ) are not
zero, Taylor’s formula gives
0
ff (Xt + ξn ) = ff (Xt ) + ff (Xt )ξn + R1 (ξn ), n = 0, 1, 2... (22)
195 where R1 (ξn ) is the Taylor series expansion remainder.
When ff (Xt )=0 and R1 (ξn ) is negligible, the following approximation holds:
0
ff (Xt + ξn ) ≈ ff (Xt )ξn (23)
Eq. 21 can be rewritten as
0
(Xf ixed − Xt )ff (Xt ) + α
ξn+1 ≈ ξn , n = 0, 1, 2... (24)
α
Then, the rate of convergence is

Xf ixed − Xt 0
µ= 1+ ff (Xt ) (25)
Ft − Yf ixed
0
By the decreasing convex function property f (x) ≥ f (y) + f (y)(x − y), so
0
200 |Ft − Yf ixed | > |(Xf ixed − Xt )ff (Xt )|. Besides, since Ft -Yf ixed <0 and (Xf ixed −
0
Xt )ff (Xt ) > 0, it is concluded µ ∈ (0, 1). Thus, sequence Xn converges linearly.
As Yf ixed is larger, µ becomes larger and, hence, a slower convergence rate.

10
 3.3. Previous Methods Revisited
The pros and cons of the work by Cho et al. [1] and Liang et al. [3] for AE
205 control can be easily understood based on the above mathematical framework.
The response function curve in Cho et al. [1] is similar to the one in Figure
5a. It is assumed that the BL reaches zero when shutter speed reaches its
theoretical maximum and the AE method was equivalent to a modified secant
method applied to a decreasing concave model. To verify this claim, the fixed
210 point (Xf ixed , Yf ixed ) is chosen to be (Xmax , 0) so that Eq. 5 can be rewritten
as:

(Xmax − Xn )Ft
Xn+1 = Xmax − , n = 0, 1, 2... (26)
f (Xn )
This is exactly the method proposed in Cho et al. [1]. However, the fixed
point, which is the right most point of the response function curve as suggested in
Cho et al. [1], does not satisfy the condition listed in Table 1. A problem would
215 arise when BL becomes zero before the shutter speed reaches its theoretical
maximum due to dim lighting conditions. Under this circumstance, the response
function is no longer concave but partially concave, leading to the tumbling effect
or even failed AE control.
The work in Liang et al. [3] is essentially a modified secant method with an
220 increasing concave model. To verify this claim, the fixed point (Xf ixed , Yf ixed )
is chosen to be (0, 0) and Eq. 5 can be rewritten as
Xn · Ft
Xn+1 = , n = 0, 1, 2... (27)
f (Xn )
which is the method proposed in Liang et al. [3]. There exists one problem
for this solution. The response of an imaging sensor is only partially concave.
As shown in Figure 1, the underexposure segment and the linear segment con-
225 stitute a convex model while the linear segment and the overexposure segment
constitute a concave model. The work in Liang et al. [3] does not take the un-
derexposure part into consideration. As a result, the tumbling effect will appear
under certain lighting conditions.
One may attempt to solve this problem by adjusting the aperture size of the
230 imaging sensor so that there is no underexposure under any lighting condition.
This idea is however not practical in a real world scenario. The same tumbling
effect will arise when the convex model is used due to the existence of overex-
posure. To tackle this problem in a more robust manner, a decreasing convex
model is introduced in the following part.

235 3.4. Generalized Modeling of AE Functions

The situation that the AE function of imaging sensors does satisfy the con-
vex/concave model strictly is examined in this subsection. Most digital sensors
in the market have a quasi-linear characteristic with a typical dynamic range of
55dB. Such sensors are mostly CMOS based and often use the exposure time as
240 the control parameter. Their quasi-linearity is actually caused by non-linearity

11
 as shown in Figure 1. Although it is difficult to characterize a non-linear func-
tion in general, the quasi-linearity of an imaging sensor can be well captured by
a decreasing convex model as detailed below.
The response function shown in Figure 1 can be characterized by the follow-
245 ing equations:

0
 if Tmin ≤ T ≤ Tu
f (T ) = k(T − Tu ) if Tu < T < To (28)

k(To − Tu ) if To ≤ T ≤ Tmax


where T is the exposure time, k is a constant, Tmin is the minimum exposure

time, Tmax is the maximum exposure time, Tu is the time where the underexpo-
sure segment ends, To is the time where the overexposure segment begins and
f (T ) is the image brightness as a function of T . By defining a new control pa-
250 rameter called the exposure speed in form of S = 1/T , where S is the exposure
speed. Eq. 28 can be rewritten as

k/So − k/Su if Smin ≤ S ≤ So

f (S) = k/S − k/Su if So < S < Su (29)

0 if Su ≤ S ≤ Smax


where Smin = 1/Tmax , Smax = 1/Tmin , Su = 1/Tu and So = 1/To . Given

0 < Tmin < Tu < To < Tmax , it can be easily proved that, if the interval between
Tmin and Tu is the same as that between To and Tmax , So − Smin > Smax − Su .

12
 Figure 6: Illustration of (a) the shape of f (T ) where Tmin = 1, Tu = 2, To
= 3 and Tmax = 4 and (b) the corresponding f (S), where the overexposure
segment, lying in the right-most segment of f (T ) and the left-most of f (S),
is contracted from the T space to the S space.

255 The response of an imaging sensor in the T space and the S space are shown
in Figure 6a and Figure 6b, respectively. It is apparent from these figures that,
the further right a segment lies in the T space, the more it is contracted in the
S space. This means the overexposure segment is more contracted than other
segments. In the real world case, due to the large T value, the overexposure
260 segment is contracted to such an extent that it does not have any significant
impact on the shape of the model. As a result, the model can be treated as a
decreasing convex one.
By substituting X with 1/T in Eq. 7, new exposure time Tn+1 can be
iteratively updated via

[Yf ixed − f (Tn )] · Tn · Tmax

Tn+1 = , n = 0, 1, 2... (30)
[Ft − f (Tn )] · Tn + α
265 where α=(Yf ixed -Ft )·Tmax , and Tmax = 1/Xmin . Typically, Ft is chosen to
be the mid-value and Yf ixed is set to the maximum value plus one and Tn is
changed to Tmax -1 if Tn happens to be Tmax . To give an example, Ft = 128 and
Yf ixed = 256 for an 8-bit image. For video cameras, Tmax should not exceed the
designated frame displaying time, which is the inverse of video frame rate. For
270 example, Tmax <40 ms if the frame rate is 25 frame per second. Depending on

13
 the definition of time unit in a particular imaging sensor, one can choose Tmax
as the theoretical or empirical maximum exposure time. For example, if the
time unit is millisecond and the maximum possible exposure time is 20 ms, the
value of Tmax should be at least 20 ms.

275 4. Experimental Results

Figure 7: The under-exposed, the over-exposed and the AE adjusted win-

dow scenes for Test No. I are shown in (a), (b) and (c), respectively.

14
 Figure 8: The under-exposed, the over-exposed and the AE adjusted indoor
scenes for Test No. II are shown in (a), (b) and (c), respectively.

In this section, experimental results of the proposed AE control method

applied to video cameras are provided. The video frame rate is 25 frames per
second. The target BL is 128. The pixel array of the camera sensor has a
dimension of 2048×20482 . The image brightness response as a function of the
280 exposure time is quasi-linear and its dynamic range is 60 dB.
To demonstrate the robustness of the proposed AE control method, the
occurrence of the tumbling effect of various AE control methods is counted and
compared with the presence of exposure error, which is introduced by delaying
camera’s response to an updated exposure setting. This is conducted with the
285 delay of a few frames while the previous exposure setting is still in use during
the transition, resulting in erroneous exposure. This is used to simulate the
exposure error encountered when the hardware capacity of a camera system is

2 It should be noted that the raw 2048 × 2048 image generated by the camera sensor has

only one color channel (say, a Bayer filtered RGGB or other formatted color mosaic). It has
to be further processed (e.g. color demosaiced, down-sampled, and re-scaled). As a result,
the pictures of Figure 7 and Figure 8 are not of dimension 2048 × 2048.

15
 unable to keep up with data flow or real time computation as a result of flawed
architecture design or worn out hardware.
290 The two test cases are shown in Figure 7 and Figure 8. The performance of
the proposed AE method, the method proposed in Liang et al. [3], the linear
based method [2] and three other root-finding AE control methods are compared
in Tables 3, 4, 5 and 6. Only CMOS imaging sensors are compared since they
are widely used in contemporary consumer electronics, and the CCD-based AE
295 control method proposed in Cho et al. [1] is excluded for this reason.
Test No. I is a window scene while Test No. II is an indoor scene. The
following AE cases are considered.
• Case 1: AE applied to the under-exposed window scene

• Case 2: AE applied to the over-exposed window scene

300 • Case 3: AE applied to the under-exposed indoor scene
• Case 4: AE applied to the over-exposed indoor scene

Case 1, Case 2 and AE adjusted images of the window scene are shown
in Figure 7a, Figure 7b, and Figure 7c, respectively. Case 3, Case 4 and AE
adjusted images of the indoor scene are shown in Figure 8a, Figure 8b, and
305 Figure 8c, respectively.
Table 3 shows the number of frames at which the tumbling effect is detected.
The tumbling effect is detected using the following rule. If BLn−1 > BLt and
BLn < BLt , or BLn−1 < BLt and BLn > BLt , where BLn is the BL of the nth
frame, and BLt denotes the target BL (or Ft in Eq. 5), then frame n encounters
310 the tumbling effect. The proposed model-based AE method is the most robust
and does not have any tumbling effect.

Table 3: Number of tumbling effect of occurrences

Total Case1 Case2 Case3 Case4

False position 6 2 1 2 1
Secant 2 1 0 1 0
Bisection 7 1 3 1 2
Kuno[2] 0 0 0 0 0
Liang[3] 13 2 1 6 4
Proposed 0 0 0 0 0

16
 Table 4: AE control time in seconds

Mean Case1 Case2 Case3 Case4

False position 20.00 20.00 20.00 20.00 20.00
Secant 20.00 20.00 20.00 20.00 20.00
Bisection 0.48 0.48 0.48 0.48 0.48
Kuno[2] 0.08 0.08 0.08 0.08 0.08
Liang[3] 0.35 0.24 0.16 0.56 0.44
Proposed 1.23 0.72 0.48 2.6 1.12

Table 5: Adjusted BL (error %)

Mean
Error Case1 Case2 Case3 Case4
(%)
False position 98.2 4(96.9) 5(96.1) 0(100) 0(100)
Secant 21.5 172(34.4) 170(32.8) 140(9.4) 140(9.4)
Bisection 22.3 174(35.9) 167(30.5) 147(14.8) 138(7.8)
Kuno[2] 25.8 77(39.8) 173(35.2) 107(16.4) 113(11.7)
Liang[3] 0.8 126(1.6) 128(0.0) 130(1.6) 128(0.0)
Proposed 2.9 124(3.1) 132(3.1) 125(2.3) 132(3.1)

Table 6: Summary of convergence property

Case1 Case2 Case3 Case4

False position NO NO NO NO
Secant NO NO NO NO
Bisection YES YES YES YES
Kuno[2] N/A N/A N/A N/A
Liang[3] YES YES YES YES
Proposed YES YES YES YES

Table 4 compares the AE control time. The control process is forced to stop
if the time exceeds 20 seconds. The proposed method is competitive in its speed.
This is especially true when it deals with the overexposed scene. Its convergence
315 speed slows down for the underexposed scene due to the longer underexposure
segment as shown in Figure 6b. Although the method of Liang et al. [3] is
faster, it is not as robust. The method of Kuno et al. [2] fixes the number of
frames for AE control to 2 to give the fastest decision. However, this method
often fails as shown in Table 5. The deviation of the adjusted BL with respect
320 to the target BL in percentages is listed in Table 5. The proposed AE method
and the AE method by Liang et al. [3] offer the best results.
Finally, the convergence property of the six AE control methods is summa-
rized in Table 6. If an algorithm converges, it has YES in the corresponding

17
 box. Since Kuno et al. [2] is not an iterative method, the concept of converge
325 does not apply (N/A). It can be seen that the exposure error has a great im-
pact on both the false position method and the secant method, and they fail
to converge. Such failure is also reflected in Table 4 where the control time of
the two methods is 20 seconds which is the maximum time allowed. Based on
the above discussion, it is easy to conclude that the proposed model-based AE
330 method gives the most robust performance. Its control speed is sufficiently fast,
and its complexity is low.

5. Conclusion

A fast and robust model-based AE control method was proposed in this work
by applying the modified secant algorithm to a convex or concave luminance
335 model. Its monotonically convergent and error-tolerance properties were proved
mathematically. Its superior performance in striking a good balance between
convergence speed, stability and robustness was explained and demonstrated.
It is desired to integrate the proposed AE control method into the HDR camera
system in the near future.

340 References

[1] M. H. Cho, S. G. Lee, B. D. Nam, The fast auto exposure algorithm based
on the numerical analysis, in: Proceedings of the SPIE Conference on Sen-
sors, Cameras, and Applications for Digital Photography, San Jose, 1999,
pp. 93–99.

345 [2] T. Kuno, H. Sugiura, A new automatic exposure system for digital still
cameras, IEEE Trans. Consumer Electronics 44 (1) (1998) 192–199.
[3] J. Y. Liang, Y. J. Qin, Z. L. Hong, An auto-exposure algorithm for detecting
high contrast lighting conditions, in: Proceedings of the IEEE International
Conference on ASIC, Guilin, China, 2007, pp. 725–728.

350 [4] P. Debevec, J. Malik, Recovering high dynamic range radiance maps from
photographs, in: Proceedings of the 24th Annual Conference on Computer
Graphics and Interactive Techniques(ACM SIGGRAPH), New York, NY,
USA, 1997, pp. 369–378.

[5] T. Mitsunaga, S. K. Nayar, Radiometric self calibration, in: Proceedings of

355 the IEEE Computer Society Conference on Computer Vision and Pattern
Recognition, Fort Collins, CO, USA, 1999, pp. 472–479.
[6] S. Schulz, M. Grimm, R. Grigat, Using brightness histogram to perform
optimum auto exposure, WSEAS Trans. System and Control 2 (2) (2007)
93–100.

18
360 [7] S. Shimizu, T. Kondo, T. Kohashi, M. Tsuruta, T. Komuro, A new algo-
rithm for exposure control based on fuzzy logic for video cameras, IEEE
Trans. Consumer Electronics 2 (2) (2007) 93–100.
[8] S. Yousefi, H. R. Rabiee, P. Mianjy, Optimal exposure detection function
for digital and smart-phone camera applications, in: Proceedings of the
365 IEEE International Conference on Consumer Electronics, Las Vegas, NA,
USA, 2012, pp. 149–150.
[9] W.-C. Kao, L.-W. Cheng, C.-Y. Chien, W.-K. Lin, Robust brightness mea-
surement and exposure control in real-time video recording, IEEE Trans.
Instrumentation and Measurement 60 (4) (2011) 1206–1216.

370 [10] M. Murakami, N. Honda, An exposure control system of video cameras

based on fuzzy logic using color information, in: Proceedings of the IEEE
International Conference on Fuzzy Systems, New Orleans, LA, USA, 1996,
pp. 2181–2187.
[11] J. S. Lee, Y. Y. Jung, B. S. Kim, S. J. Ko, An advanced video camera
375 system with robust af, ae and awb, IEEE Trans. Consumer Electronics
47 (3) (2001) 694–699.
[12] T. Haruki, K. Kikuchi, Video camera system using fuzzy logic, in: Pro-
ceedings of the IEEE International Conference on Consumer Electronics,
Rosemont, IL, USA, 1992, pp. 322–323.

380 [13] Y. Su, C.-C. J. Kuo, Fast and robust cameras auto exposure control using
convex or concave model, in: Proceedings of the IEEE International Con-
ference on Consumer Electronics, Las Vegas, NV, USA, 2015, pp. 13–14.
[14] B. R. Kiran, K. M. Krishana, S. A. S. Basha, G. V. N. A. H. Vardhan,
K. V. S. V. N. Raju, V. V. Kumari, False position based auto exposure al-
385 gorithm for properly exposing the leather samples in the leather industries,
International Journal of Computer Applications 83 (14) (2013) 7–9.

19
 This work formulates the auto exposure (AE) control problem as a
root finding problem with a convex/concave model.
 The work solves the AE control problem by apply an iterative root
finding algorithm to the convex/concave model.
 The proposed solution is robust, fast and stable even under
hardware malfunction.
 The superiority of the proposed solution is approved
mathematically and demonstrated experimentally.
 This work sheds lights on previous AE control solutions under one
unified mathematical framework.