562 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO.

4, APRIL 2006

Dynamic Active Contours for Visual Tracking

Marc Niethammer, Allen Tannenbaum, and Sigurd Angenent

Abstract—Visual tracking using active contours is usually set in or our environment (e.g., controlling the movement of a plane,
a static framework. The active contour tracks the object of interest based on visual input). The latter will most likely encompass the
in a given frame of an image sequence. A subsequent prediction first (possibly resulting in nested control loops). Both tasks can
step ensures good initial placement for the next frame. This ap-
proach is unnatural; the curve evolution gets decoupled from the be accomplished by means of feedback mechanisms. In either
actual dynamics of the objects to be tracked. True dynamical ap- case we need a good estimate of object position. Once we have
proaches exist, all being marker particle based and thus prone to this estimation, we either have fulfilled our task (e.g., for surveil-
the shortcomings of such particle-based implementations. In par- lance applications), or use this information to close a control
ticular, topological changes are not handled naturally in this frame- loop. This brings to mind a broad range of applications. Indeed,
work. The now classical level set approach is tailored for evolutions
of manifolds of codimension one. However, dynamic curve evolu- be it for medical or military use, the need for visual tracking is
tion is at least a codimension two problem. We propose an effi- ubiquitous.
cient, level set based approach for dynamic curve evolution, which Visual feedback control differs significantly from classical
addresses the artificial separation of segmentation and prediction control. Sensors are imaging devices (usually cameras) which
while retaining all the desirable properties of the level set formu-
deliver an abundance of information of which only a fraction
lation. It is based on a new energy minimization functional which,
for the first time, puts dynamics into the geodesic active contour may be needed for a specific control task. The sensor output
framework. needs to be preprocessed to extract the relevant information for
Index Terms—Dynamic active contours, geodesic active con- the tracking problem, e.g., the position of an object to be fol-
tours, level set methods, visual tracking. lowed. Preprocessing usually encompasses noise-suppression
(e.g., image smoothing) and segmentation to delineate objects
from their background and from each other. Segmentation has
I. INTRODUCTION been an active field of study in various application areas, most
prominently in the medical sciences. Static segmentation prob-
O BJECT tracking can be accomplished in many ways in-
cluding by mechanical, acoustical, magnetic, inertial, or
optical sensing, and by radio and microwaves, to mention a few.
lems are challenging. Segmentation algorithms usually need to
be adapted to the problem at hand. There is no omnipotent seg-
The ideal tracker should be “tiny, self-contained, complete, ac- mentation algorithm. Visual tracking is a dynamic segmenta-
curate, fast, immune to occlusions, robust, tenacious, wireless, tion problem, where segmentations change over time. Here, ad-
and cheap” [1], [2]. As of now such a tracker does not exist; ditional information (e.g., apparent image motion) is available,
tradeoffs are necessary, and a method should be chosen based on but further degrees of freedom and thus difficulties are intro-
the application in mind. Optical sensing is unobtrusive and can duced, for example related to processing speed in real-time ap-
be simplified by choosing a simple (possibly prespecified) work plications. Visual tracking poses various interesting questions to
environment, or by altering the appearance of the objects to be control theoreticians and engineers, among others:
tracked (e.g., by painting them, or by mounting light sources • How can one properly deal with the unusual sensor
on them). The desired objects to be tracked then become much signal, i.e., image information, projections on the
easier to detect. However, in certain instances (e.g., for an un- image plane, correspondences for stereo vision, etc.?
cooperative object to be followed) this is not possible. Visual • How should uncertainties be modeled? In most cases
tracking is the task of following the positions of possibly mul- only very simple motion and system models are avail-
tiple objects based on the inputs of one or many cameras (the able. Delays may be significant in case of computation-
optical sensors). In the context of visual tracking, we can dis- ally demanding tracking algorithms.
tinguish between the two tasks of locating and following an ob- • How should robustness or tracking quality be quanti-
ject (e.g., for surveillance applications), and influencing objects fied? For example, what is the most suitable metric for
the space of curves?
Manuscript received May 2, 2005; revised October 21, 2005 and November Humans and animals perform visual tracking tasks with ease
9, 2005. Recommended by Associate Editor J. P. Hespanha. This work was sup- every day: Following cars in traffic, watching other people, fol-
ported in part by grants from the AFOSR, MURI, MRI-HEL, the ARO, the NIH,
and the NSF. lowing the lines of text in a document, etc. These mundane tasks
M. Niethammer is with the Brigham and Women’s Hospital, Departments of seem simple, but robust reliable algorithms and their computer
Psychiatry and Radiology, Harvard Medical School, Boston, MA 02215 USA implementation have proven to be quite challenging [3]. We rely
(e-mail: [email protected]).
A. Tannenbaum is with the School of Electrical and Computer Engineering, on a highly developed brain, assumptions about the world ac-
Georgia Institute of Technology, Atlanta, GA 30332-0250 USA (e-mail: tan- quired throughout a lifetime, and our eyes as visual sensors.
[email protected]). The design of algorithms which would make a machine behave
S. Angenent is with the Department of Mathematics, University of Wisconsin,
Madison, WI 53706 USA (e-mail: [email protected]). and perceive similarly to humans in all situations is a daunting
Digital Object Identifier 10.1109/TAC.2006.872837 task which is far from being solved. However, if we are only
0018-9286/$20.00 © 2006 IEEE
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 563

interested in a specific application, the problem becomes more completely unaware of its state. In contrast, Terzopoulos and
tractable. Visual tracking is a relatively well defined problem Szeliski [33] or Peterfreund [34] view curve evolution from a
when dealing with well defined environments.1 dynamical systems perspective; both methods are marker par-
Applications for visual tracking are diverse. Some key areas ticle based and are fast, but they may suffer from numerical
of research include the following. problems (e.g., in the case of sharp corners [35]–[37]). In the
• Vehicle guidance and control: See [4]–[9] for applica- static case, level set methods are known to handle sharp cor-
tions to autonomous driving.2 See Sinopoli et al. [10] ners, topological changes, and to be numerically robust. In their
and Sharp et al. [11] for visual tracking systems for the standard form, they are restricted to codimension one problems,
navigation and the landing of an unmanned aerial ve- and thus not suitable for dynamic curve evolution. Extensions
hicle, respectively. of level set methods to higher codimensions exist and a level set
• Surveillance and identification: See [12]–[14] for ap- formulation for dynamic curve evolution is desirable [38], [39].
plications to target tracking and biometric identifica- We will present a straightforward level set based dynamic curve
tion. evolution framework in this paper.
• Robotics/Manufacturing: See Corke [15] and Hutchin- The results of this paper relate to dynamic snakes [33] as
sion et al. [16] for discussions on visual servo control geodesic or conformal active contours [30], [29] relate to
which requires the visual tracking of objects/object the original snake formulation [27]. Here we are advocating
features as a preprocessing stage. Here visual tracking a different philosophy to dynamic curve evolution. Instead
is used to increase the bandwidth and accuracy of of discretizing evolution equations upfront (early lumping),
robots. We note that visual grasping falls into this we keep the partial differential equations as long as possible
category of tasks. (late lumping [40]), resulting in a more natural and geometric
• User interfaces: See [17] for real-time fingertip formulation.
tracking and gesture recognition, and [3] for virtual We demonstrate that we can attach information to a contour
environments. evolving in a level set framework. This is related to the approach
• Video processing: See [18] for automated addition of in [41] and is a crucial step toward more flexible level set based
virtual objects to a movie. approaches. Most level set based evolution equations in image
• Medical applications: See [19] for applications to vi- processing are static and/or do not possess state information.
sion guided surgery (surgical instrument tracking) and This can be a major drawback, e.g., if we want to follow a con-
[20] for medical image tracking. tour portion over time.
A wide variety of algorithms for visual tracking exists, e.g., Error injection is a standard method from control theory
feature trackers, blob trackers, contour/surface trackers. See to construct observers. All snakes using an observer (e.g.,
[21]–[26], and the references therein. All of these have their Kalman filter-based or particle filter-based) use error injection.
own advantages and disadvantages. The seminal paper of Kass Observers for marker particle based systems are finite dimen-
et al. [27] spawned a huge interest in the area of contour/surface sional. Our proposed approach requires an observer for an
tracking algorithms. This is the class of algorithms on which infinite dimensional, nonlinear system. The existing theory for
we will focus in this paper. such systems is still in its infancy; system theoretic results are
available only in special cases. We restrict ourselves to error
injection resembling a nonlinear, infinite dimensional observer
II. PROBLEM STATEMENT, MOTIVATION, AND SCOPE
if we are close enough to the basin of attraction of the object
Typical geometric active contours [28]–[32] are static. How- of interest. The incorporation of the optical flow constraint
ever, variational formulations many times appear to be dynamic is natural in this framework. Our formulation restricts the
because the resulting Euler–Lagrange equations are solved by propagation to the direction normal to the object direction; this
gradient descent, introducing an artificial time parameter. This is exactly measured by the optical flow, in contrast to previous
time parameter simply describes the evolution of the gradient approaches [33] for dynamic snakes which do not restrict the
descent. It will usually not be related to physical time. A two direction of movement. Thus, even though error injection is
step approach is typically used for visual tracking by static ac- classical, it is novel in this level set framework.
tive contours. First, the curve evolves on a static frame until con- We now briefly summarize the contents of the remaining
vergence (or for a fixed number of evolution steps). Second, the sections of this paper. Section III gives a brief overview over
location of the curve in the next frame is predicted. In the sim- existing methods for contour based visual tracking. Section IV
plest case, this prediction is the current location. Better predic- introduces the concept of static curve evolution and positions
tion results can be achieved by using optical flow information, it in relation to classical image processing. Section V reviews
for example. In this two step approach, the curve is not moving the fundamentals of parameterized dynamic curve evolution.
intrinsically, but instead is placed in the solution’s vicinity by Section VI introduces geometric dynamic curve evolution and
an external observer (the prediction algorithm). The curve is discusses the evolution equations. The level set formulation
for normal geometric dynamic curve evolution is given in Sec-
1This is an overview of application areas for visual tracking. Due to the chal-
tion VII. Sections VIII and X deal with error injection into the
lenging nature of the general visual tracking problem, task-specific algorithms
are usually necessary. There is no “silver bullet” [1]. evolution equations and occlusion detection, respectively. Sim-
2Exemplary for these research efforts are the European Prometheus and the ulation results obtained on real image sequences are presented
American PATH programs. in Section XI. Section XII discusses our results and future
564 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

work. We also include some appendices with the derivations of mensional observers which are (unlike observers for linear sys-
the key formulas. tems) understood only for special system classes.
Infinite-dimensional curve descriptions have been used
III. ALTERNATIVE CONTOUR-BASED TRACKING in combination with the trivial motion model [27] (i.e., no
METHODOLOGIES dynamic motion is assumed, all shape changes are purely
static), in combination with finite dimensional motion models
The literature on tracking is vast. To give a complete overview
[44]–[47], as well as in combination with infinite-dimensional
on tracking methodologies is beyond the scope of this paper.
motion models [48], [49]. Since finite-dimensional motion
We limit ourselves to a brief overview of the (what we think)
models cannot account for arbitrary elastic deformations they
closest approaches, i.e., contour based tracking methodologies,
are frequently combined with an elastic update step: this is
highlighting their differences.3
called deformotion in [45] and employed for tracking purposes
A possible classification for contour based trackers is based
in conjunction with a simple observer structure in [47] and in a
on
particle-filtering framework in [44]. Particle-filtering [42] has
• the motion model: finite dimensional (parametric) or been very popular in the computer vision community, but is
infinite dimensional; usually restricted to low-dimensional state spaces to keep the
• the curve model: finite dimensional, or infinite dimen- computational cost reasonable. In [42], an affine motion model
sional; is used, there is no elastic deformation of the curve. Rathi et
• the solution method: optimization, or integration in al. [44] extend the particle-filtering approach to include elastic
time; curve deformation in the observer update step. However, the
• the type of curve influence terms (boundary, area, sta- state space is not truly infinite-dimensional, in particular there
tistics, etc.). is no infinite-dimensional motion model.
Most visual tracking approaches employ finite dimensional Approaches using infinite-dimensional motion models for vi-
motion models and finite dimensional curve evolution models. sual tracking usually employ some form of passive advection,
If the curve change over time is only described by the motion e.g., the curve gets pushed forward through an external vector
model (the motion group), i.e., if there is no change of the curve field for example established through an optical flow computa-
shape and consequently no curve evolution model, curve based tion [48] or through a motion segmentation step [49].
trackers can easily be cast as finite dimensional observation In this paper, we are interested in adding dynamics to the
problems. Approaches include all flavors of the Kalman filter curve evolution itself, so that it is no longer passively advected,
(“classical” Kalman filter, extended Kalman filter, unscented but possesses an intrinsic velocity associated with every point
Kalman filter or sigma point filter), probability data associa- on the curve. The approach taken is to construct an infinite-di-
tion filters, and particle filters [42]. Finite-dimensional motion mensional dynamically evolving curve based on the ideas by
groups are usually chosen to be translation, translation plus ro- Terzopoulos [33]. It is a dynamical systems viewpoint which
tation, or the affine transformation group. does not require static optimization steps as in many other ap-
Extending these finite dimensional filtering methods to elastic proaches.
deformations is generally not straightforward, since the evolution
equations or observations tend to be nonlinear. One approach
is to parameterize the curve shape. This can for example be IV. STATIC CURVE EVOLUTION
done by Fourier modes, principal component analysis, or in the Image processing in the line of traditional signal processing
simplest possible case by a piecewise linear approximation of the is concerned with low-level vision tasks: e.g., performing image
boundary (a particle-based method). In the latter dynamic case denoising, edge detection, deconvolutions, etc. In this setting
[33], [34], [43], the boundary is represented by a prespecified images are treated as multidimensional signals, and there are
number of points plus their associated velocities. Increasing usually no high-level assumptions regarding the image content
the degrees of freedom has the disadvantage of increasing (e.g., looking specifically to find an object of specific texture,
the computational complexity. This is particularly true for etc.). On the other side of the spectrum is high-level vision
particle filtering approaches, which can only handle a moderate (high-level reasoning) which tries to address the latter problem
number of states without becoming computationally intractable. of what is represented in an image. The image is to be decom-
Also, parameterizing a curve shape (independent of the type posed into meaningful subparts (the process of segmentation;
of parameterization used) introduces a strong shape bias: i.e., e.g., foreground, background, uniform regions) which are sub-
the shape is assumed to lie in a certain (prespecified) class. sequently analyzed (e.g., which types of objects do the seg-
This may be desired in case of an object with clearly defined mented regions correspond to). Creating such a high-level vi-
shape, but may be unwanted if objects are allowed to deform sion system is a tremendously hard task, far from being solved.
completely elastically. Tasks that are straightforward for humans turn out to be strik-
Moving away from finite dimensional to infinite-dimensional ingly difficult in the algorithmic setting of computers, requiring
curve representations results in great flexibility, but comes at assumptions about and simplifications of the vision problem to
the cost of less theoretical insight. Nonlinear infinite-dimen- be approached. There is no segmentation algorithm that works
sional curve evolution equations require nonlinear infinite-di- for all cases. A popular simplification is to assume that ob-
3In what follows we refer to contour based visual tracking based methods, jects in an image are separated from their background, for ex-
even if we only write visual tracking. ample by intensity edges, variations in image statistics, color,
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 565

etc. Template matching is a common approach for image seg- with the original snake formulation is that it is not geometric,
mentations of known objects. Unfortunately, while robust, tem- i.e., the derivatives do not represent clear geometric quantities
plate matching is also inherently inflexible. If the image repre- (e.g., normals and curvature) and the solution depends on the
sents anything not accounted for in the template (e.g., an addi- somewhat arbitrary parameterization . On the other hand, the
tional protrusion in the shape of the object) the template will geodesic active contour [48], [30], another curve-based segmen-
not be able to capture it. Solid objects may be described by tation method, is completely geometric. To understand the mo-
their interior, or (if only the shape outline is sufficient) by their tivation behind the geodesic active contour it is instructive to
boundary curves. Boundary descriptions are the vantage point look at the length minimizing flow first, i.e., the curve evolution
for segmentations by curve evolution. The assumption here is that minimizes
that whatever object we are looking to segment may be de-
scribed by a closed curve representing its boundary. Kass et
al. [27] introduced what is known as the classical snake model
for curve based segmentation. The basic idea is to minimize where denotes arclength and the length of the curve . The
an energy functional depending on image influences (i.e., at- gradient descent scheme that minimizes curve length is
tracting it to edges) and curve shape. Given a parameterized
curve in the plane of the form , where (4)
, is the parame-
terization, , (i.e., the curve is closed), where denotes the unit-inward normal to and
and is an artificial time, the energy is the signed curvature. Equation (4) is known as the geometric
heat equation or the Euclidean curve shortening flow. Gage and
Hamilton [50] proved that a planar embedded convex curve con-
verges to a round point when evolving according to (4). (A round
(1) point is a point that, when the curve is normalized in order to en-
close an area equal to , is equal to the unit disk.) Grayson [51]
proved that a planar embedded nonconvex curve converges to
is minimized, where and are parameterization de- a convex one, and from there to a round point from Gage and
pendent design parameters (usually set constant) and is Hamilton result. Note that in spite of the local character of the
some potential function (with the desired location of forming evolution, global properties are obtained, which is a very inter-
a potential well). A common choice for the potential function is esting feature of this evolution. For other results related to the
Euclidean shortening flow, see [50]–[55]. The Euclidean curve
shortening flow only depends on the curve shape. There is no
image influence term. The idea of geodesic active contours is to
where denotes image position, is the image inten- introduce a conformal factor (in analogy to the potential
sity, is a positive integer, and is a Gaussian. See Fig. 1 for an function introduced above) into the energy functional, to mini-
illustration of curve parameterization. In most applications, the mize the weighted length4
rigidity term is disregarded (i.e., ). The energy (1) is
independent of time. It is a static optimization problem, which (5)
may be solved by means of calculus of variations. The corre-
sponding Euler–Lagrange equation for the candidate minimizer The gradient flow corresponding to (5) is
of is
(6)
(2)
Equation (6) only involves geometric terms, the curvature and
the normal and is completely independent of parameteriza-
tion. The term is the geometric analog to the elasticity term
The right hand side of (2) can be interpreted as an infinite-di- of (3) and the gradient term gets replaced by
mensional gradient. Consequently, moving into the negative its projection onto . There is no correspondence to the rigidity
gradient direction results in the gradient descent solution term of the parametric snake, however this term is frequently
scheme for (1) discarded due to its fourth-order derivative. See [56] and [57]
for more details. Many extensions to and variations of the ac-
(3) tive contour exist (e.g, adding an inflationary term). For more
information, see [57] and the references therein.
The solution of (1) is a tradeoff between the elasticity term Neither the snake (3) nor the geodesic active contour (6) are
(trying to shrink the curve length), the rigidity term (favoring truly dynamic curve evolutions. In both cases only the steady
straight curve segments), and the image influence term (trying to state solution on a static image is sought. In visual tracking, ob-
attract the curve for example to an intensity edge). The tradeoff jects move over time. Consequently tracking with closed curves
implies that sharp corners are usually not represented well (un- 4Recently direction dependent conformal factors have been introduced, i.e.,
less they are coded explicitly into the method). Problematic g(C ; C ).
566 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

Fig. 1. Parameterized curve evolution. The parametrization travels with a particle. In general, the parametrization will not stay uniformly spaced. The black disk
and the asterisk indicate particles attached to the curve; their assigned value for p will stay the same throughout the evolution.

implies estimating closed curves moving in space and time. This Computing the first variation of the action integral (7) and
is not readily described by the snake or the geodesic active con- setting it to zero yields the Euler–Lagrange equations for the
tour. Section V describes a dynamic extension to the parametric candidate minimizer [58] in force balance form
snake. However, the objective of this paper is the dynamic ex-
tension of the geodesic active contour which will be discussed (9)
in Section VI.
Equation (9) depends on the parametrization and is therefore
V. PARAMETRIZED DYNAMIC CURVE EVOLUTION not geometric (see Xu et al. [56] for a discussion on the relation-
In this section, we review parameterized dynamic curve evo- ship between parametric and geometric active contours). Our
lution [33]. We also introduce the mathematical setup required proposed methodology (see Section VI) will be completely in-
to derive the geometric dynamic curve evolution equations of dependent of parametrization. It will be geometric.
Section VI.
We consider the evolution of closed curves of the form VI. GEOMETRIC DYNAMIC CURVE EVOLUTION
in the plane, where and
In this section, we will present the geometric dynamic curve
[51], with being the time, and the curve’s
evolution equations, which evolve according to physically mo-
parametrization (see Fig. 1 for an illustration). The classical for-
tivated time. These evolution equations constitute a geometric
mulation for dynamic curve evolution proposed by Terzopoulos
formulation of the parameterized dynamic approach reviewed
and Szeliski [33] is derived by means of minimization of the ac-
in Section V in analogy with the connection between parame-
tion integral
terized and geometric curve evolution described in Section IV.
Minimizing (7) using the potential energy of the geodesic active
(7) contour (5)

where the subscripts denote partial derivatives (e.g., is the

curve’s velocity). The Lagrangian, , is the differ-
ence between the kinetic and the potential energy. The potential and the kinetic energy
energy of the curve is the energy of the snake (1)

results in the Lagrangian

The kinetic energy is
(10)

Computing the first variation of the action integral (10)

where corresponds to mass per unit length. The Lagrangian is yields
then

(8) (11)
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 567

Fig. 2. Normal direction, C constant and linearly increasing.

which is geometric and a natural extension of the geodesic active accelerates the curve toward the potential well formed by .
contour approach [29], [30] (see Appendix I for a derivation). Note that points toward the potential well. The term
Here is the unit inward normal, the unit tangent
vector to the curve, denotes curvature and is the
arclength parameter [59].
We can consider the term in (11) as a force
exerted by the image potential on the curve . Compare this accelerates the curve based on its smoothness properties and
to the evolution equation for geodesic active contours as given
in [57] and [60] ( ). From a controls (13)
perspective, this can be interpreted as a control law, based on
and its spatial gradient , which is designed to move the curve represents a smoothing term for the tangential velocity. We can
closer to the bottom of the potential well formed by . decompose the velocity change at every point on the curve
Equation (11) describes a curve evolution that is only influ- into its tangential and normal components as
enced by inertia terms and information on the curve itself. To
increase robustness the potential energy can include region-
based terms (see, for example, [61]–[63]). This would change
the evolution (11), but such changes pose no problem to our
We assume that the tangential and the normal components
proposed level set approach.
change approximately linearly close to the point of interest. A
The state–space form of (11) is
Taylor series expansion (at arclength ) yields

(12)

where , , ,
, , and are scalar functions in
and its derivatives. The evolution describes the movement of a
curve in , where the geometrical shape can be recovered by
the simple projection

In order to appreciate the effect of the term (13), it is sufficient

to consider the two fundamental cases depicted in Figs. 2 and 3.
The normal component (depicted in Fig. 2) is irrelevant for the
A. Interpretation of the Evolution Terms for the Geometric evolution, since in this case. The tangential com-
Dynamic Curve Evolution ponent (depicted in Fig. 3) will counteract tangential gradients
To get an understanding of (11) it is fruitful to look at the of . The two cases correspond to a linearly and a paraboli-
effect of its individual terms. The term cally increasing velocity in the tangential direction. In both
cases, the term will counteract this tendency of
tangentially diverging particles on the curve, ideally smoothing
out the tangential velocities over the curve .
568 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

Fig. 3. Tangential direction, C constant and linearly increasing.

Fig. 4. Behavior of the term 0 (C 1 C ) T . (a) Normal. (b) Tangential.

The term cles on the curve. This can be interpreted as a rubberband effect.
Assume that the rubberband gets pulled at one point. This will
elongate the rubberband. Since the point at which it is pulled
governs the transport of particles along the tangential direc- stays fixed (no movement except for the displacement due to
tion. To understand what is occurring locally, we assume we are the pulling) particles next to it flow away from it. The triangular
looking at a locally linear piece of the curve and decompose the velocity shape in the tangential direction also induces tangential
velocity into motion of the particles. However, this motion will counteract the
initial tangential direction and will thus also lead to a smoothing
effect on the change of tangential velocity vector over arclength.
It is instructive to look at a triangular velocity shape in the
B. Normal Geometric Dynamic Curve Evolution
normal direction [as shown in Fig. 4(a)] and in the tangential
direction [as shown in Fig. 4(b)]. The triangular velocity shape To get a quantitative interpretation of the behavior of the
in the normal direction induces a tangential movement of parti- curve evolution (11), it is instructive to derive the corresponding
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 569

evolution equations for the tangential and normal velocity com- The time evolution for can then be decomposed into
ponents of the curve.
We can write

(14)

where the parametrization is independent of time and travels

with its particle (i.e., every particle corresponds to a specific If we choose as
value for all times), and and correspond to the tangential
and the normal speed functions, respectively. By substituting
(14) into (11) and using results from [64] (see Appendix II) we
obtain the two coupled partial differential equations
we obtain

(15)
which is a curve evolution equation without a tangential com-
ponent. For all times, , the curve will move along its normal
Here, and are the transport terms for the tan-
direction. However, the tangential velocity is still present in the
gential and the normal velocity along the contour, and
update equation for . After some algebraic manipulations, we
is the well known geodesic active contour image influence
arrive at
term [30], [29]. In contrast to the static geodesic active contour,
this term influences the curve’s normal velocity rather than di-
rectly the curve’s position. It can be interpreted as a force. Fi- (17)
nally, the terms and incorporate
the dynamic elasticity effects of the curve. If we envision a ro- which depends on the time derivative of the reparameterization
tating circle, we can interpret the term function , which in turn depends on the tangential component
as a rubberband (i.e., if we rotate the circle faster it will try to . The left-hand side of (17) represents a transport term along
expand, but at the same time it will try to contract due to its the curve, the speed of which depends on the time derivative of
then increasing normal velocity; oscillations can occur). If we the reparameterization function .
restrict the movement of the curve to its normal direction (i.e.,
if we set ) we obtain C. Special Solutions
To illustrate the behavior of (15) and (16), we study a simple
(16) circular example. Assume . Then . Further-
more, assume that we evolve a circle with radius and constant
initial velocities
This is a much simpler evolution equation. In our case it is iden-
tical to the full evolution (15) if the initial tangential velocity is
zero. The image term only influences the normal velocity evo-
lution . It does not create any additional tangential velocity. Then the normal evolution reduces to
Thus, if , then ; the flow with is
contained in (11) as an invariant subsystem. The restriction to
curve movement in the normal direction is a design choice to
simplify the approach. See Section VI-C for an illustration of (18)
the influence of a tangential velocity component.
If there is an initial tangential velocity, and/or if the image and the full evolution becomes
influence contributes to the normal velocity and to the tan-
gential velocity , the normal evolution equation will not nec-
essarily be equivalent to the full evolution (15). We can always
parametrize a curve such that the tangential velocity term van-
ishes. Specifically, if we consider a reparameterization
(19)

where we made use of the facts that (given the initial conditions
for the circle)
where then
570 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

Fig. 5. Left column: Evolution of the radius for the normal velocity evolution (dashed line) and the full velocity evolution (solid line). Middle column: Evolution
of normal velocity (dashed line) and tangential velocity (solid line) for the full velocity approach. Right column: Evolution of normal velocity for the normal
velocity evolution. (a) = 0:1, = 0, R = 100, = 0, = 0. (b) = 1, = 0, R = 100, = 0, = 0. (c) = 1, = 0, R = 100,
= 0 :1 , = 0 :1 .

and added an artificial friction term, with and being the cles disappear in finite time. The evolutions of the radius look
friction coefficients for the tangential and the normal velocity, similar in both cases. Due to the large friction coefficients a large
respectively. Since we are dealing with a circle with constant amount of energy gets dissipated; oscillations no longer occur.
initial velocity conditions, evolving on a uniform potential field Equations (18) and (19) do not exhibit the same behavior. De-
, we know that the solution will be rotationally invariant (with pending on the initial value for , they will have fundamentally
respect to the origin of the circle). Thus we can evolve in (19) different solutions. For , and in (19), the
by using only its normal velocity. solution is (geometrically) stationary, and the circle will keep its
Fig. 5 shows the evolution of the radius, , the tangential ve- shape and rotate with velocity for all times. Also if ,
locity (if applicable), the normal velocity for a small initial in this example case, both evolutions will be identical.
value of , a larger initial value of , and with added friction,
respectively. VII. LEVEL SET FORMULATION
Fig. 5(a) shows the results for , , , There are different ways to implement the derived curve
, . We see that while in the normal evolution case evolution equations (see, for example, [38]); many numerical
the circle accelerates rapidly and disappears in finite time, this schemes exist. In this paper, we will restrict ourselves to level
is not the case when we do not neglect the tangential velocity: set based curve representations. In contrast to the classical level
Then the circle oscillates. It rotates faster if it becomes smaller set approach [65], where the curve evolution speed is usually
and slower if it becomes larger. Due to the small initial tangen- based on geometric properties of the curve or induced by some
tial velocity the radius evolution is initially similar in both cases. external process, the level set approach developed in this paper
The oscillation effect is more drastic with increased initial tan- attaches a velocity field to the curve and evolves it dynamically.
gential velocity ( ). This can be seen in Fig. 5(b). Fig. 5(c) We distinguish between full and partial level set implementa-
shows the results with added friction ( ). Both cir- tions. In the full case, curves evolve in a space consistent with
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 571

the dimensionality of the problem. Geometric dynamic curve Substituting (21) into (16) and using the relation
evolution would thus be performed in in the simplest case
(since we are looking at planar curves). The codimensionality
will increase if additional information is to be attached to the
curve. Normal geometric dynamic curve evolution would be at
least a problem in . If is the dimensionality of the problem yields
the curve can for example be implicitly described by the zero
level set of an -dimensional vector distance function or the in- (23)
tersection of hypersurfaces [66]. Full level set approaches
of this form are computationally expensive, since the evolutions The left-hand side of (23) is the material derivative for the
are performed in high dimensional spaces. Furthermore, it is normal velocity. If we use extension velocities, (23) simplifies
not obvious how to devise a methodology comparable to a to
narrow band scheme [67] in the case of a representation based
on intersecting hypersurfaces.
A partial level set approach uses a level set formulation for the
propagation of an implicit description of the curve itself (thus
allowing for topological changes), but explicitly propagates the Since the extensions are normal to the contours, normal propa-
velocity information associated with every point on the contour gation of the level set function will guarantee a constant velocity
by means of possibly multiple transport equations. This method value along the propagation direction (up to numerical errors).
has the advantage of computational efficiency (a narrow band Specifically in this case and thus
implementation is possible in this case, and the evolution is per-
formed in a low dimensional space) but sacrifices object sepa-
ration: Tracked objects that collide will be merged.
In what follows, we will restrict ourselves to a partial level set For an alternative derivation,5 we change our Lagrangian, and
implementation of the normal geometric dynamic curve evolu- extend it over a range of level sets. For each time and
tion (i.e., ). We will investigate the full level set , let
implementation, including tangential velocities, in our future
work.
Using the Lagrangian
A. Partial Level Set Approach for the Normal Geometric
Curve Evolution
The curve is represented as the zero level set of the function

we obtain the action integral

where is a point in the image plane. We

choose to be a signed distance function, i.e., , a.e.,
such that outside the curve and inside the curve
which is
. Since the evolution of the curve’s shape is independent of the
tangential velocity, we can write the level set evolution equation
for an arbitrary velocity as

(20)

where
(24)

where is the one-dimensional Hausdorff measure and we

applied the coarea formula [68]. This casts the minimization
In our case , where problem into minimization over an interval of level sets in a
fixed coordinate frame ( and are time independent coordi-
(21) nates in the image plane). Using (22), we express as

is the spatial normal velocity at the point . This simplifies (20) (25)
to
5This will yield directly the normal evolution equation, without the detour of
(22) deriving (15).
572 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

Substituting (25) into (24) yields Define

which is the new -dependent action integral to be minimized. Our set of estimated contour point candidates is the set of
Then, if and only if potential edge points in

The curve evolution is thus governed by the equation system where is a Gaussian, is the disk around with radius
, and is the current image intensity. Given some problem
specific likelihood function the selected contour point is
the likelihood maximum
(26)

Expanding (26) yields again

at position

The equation system (26) constitutes a conservation law for the It is sufficient to estimate normal velocity, since the curve evolu-
normal velocity . The propagation of the level set function tion equation does not take tangential velocity components into
is described (as usual) by a Hamilton–Jacobi equation. account. The estimation then can be performed (assuming we
have brightness constancy from image frame to image frame for
VIII. ERROR INJECTION a moving image point) by means of the optical flow constraint
without the need for regularization. Note that we compute this
A system governed by a time-independent Lagrangian (i.e., estimate only on a few chosen points in . The optical flow con-
) will preserve energy [58], but this is not necessarily de- straint is given as
sirable. Indeed, envision a curve evolving on a static image with
an initial condition of zero normal velocity everywhere and with
an initial position of nonminimal potential energy. The curve
will oscillate in its potential well indefinitely. One solution to where and are the velocities in the and the
this problem is to dissipate energy [33], which can be accom- direction, respectively. We restrict the velocities to the normal
plished by simply adding a friction term to (26). However, to direction by setting
increase robustness it is desirable to be able to dissipate and to
add energy to the system in a directed way. A principled way
to do this would be to use an observer to drive the system state
of the evolving curve to the object(s) to be tracked. In our case
this is not straightforward, since we are dealing with an infinite
This yields
dimensional nonlinear system. In order for the curve to approx-
imate the dynamic behavior of the tracked objects we use error
injection. This guarantees convergence of the curve to the de-
sired object(s) if the curve is initially in the appropriate basin of
attraction. and thus the desired velocity estimate
To perform error injection, we need an estimated position and
velocity vector for every point on the curve . Define the line
through the point on the current curve as

We define

and the set of points in an interval on the line as

NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 573

Fig. 7. Correspondence point x , inside correspondence point x , and outside

correspondence point x of the curve C^. C represents the contour of the object
to be tracked.

additional benefits. Thus, instead of updating a level set evo-

Fig. 6. Feature search is performed in the normal direction to the curve. The
search region is only allowed to intersect the curve at its origin of search (i.e., lution equation over all of (which incurs an update cost of
s ; s ; s ; . . .). , if is represented on a square domain with dis-
crete gridpoints) the computational domain is chosen to be
We propose using the following observer-like dynamical a band surrounding the zero level set up to a certain distance.
system: This is the idea of the narrowband method [69]. If the narrow-
band consists of points, the computational complexity con-
sequently reduces from to . Frequently, the speed
function is only defined or sensible on or very close to the zero
level set and needs to be extended to the whole computational
domain. This may be accomplished for example by the fast
(27) marching method [70], [65], [71] with a computational com-
plexity of or with a fast sweeping method [72] with
to dynamically blend the current curve into the desired curve a computational complexity of . The latter is extremely
(see Fig. 7). Here, and are the error injection gains efficient for many “simple” flow fields (i.e., flow fields that are
for and , respectively. Any terms related to image features spatially regular and do not change direction frequently) that are
are computed at the current location of the contour. The error encountered in practice (e.g., normal extensions as employed in
injection gains are weighted by the likelihood of the cor- this paper), but may require a relatively large number of itera-
respondence points as a measure of prediction quality. The ad- tions for flow fields that (for an individual particle stream line)
ditional terms and with tunable weighting factors and fluctuate in direction.
are introduced to allow for curve and velocity regularization For a narrowband implementation (with points) of the
if necessary, where tracking algorithm proposed in Section VIII the computational
complexity is thus for every evolution step of

and
which includes the search for the feature points to determine
. Reinitialization of (which has to be performed relatively
In case no correspondence point for a point on the zero level set infrequently if extension velocities for are used) is of
of is found, the evolution equation system (27) is replaced by or (for a fast sweeping scheme and fast marching,
respectively). The evolution of
(28)

for this point.

is again of complexity for every evolution step,
IX. COMPUTATIONAL COMPLEXITY OF THE ALGORITHM or for the velocity extension, and to find
Level set methods increase the computational complexity of the feature points . The computational complexity to find the
curve evolution approaches. In the planar case, a one-dimen- values for a feature point and is constant, scales with the
sional curve is represented as the zero level set of a function de- length of the line segment the search is performed over, but
fined on the two-dimensional image plane, . Level set methods gets multiplied by the number of points in the narrowband .
are of interest numerically (e.g., there is no fixed number of The overall computational complexity of the algorithm is thus
particles to represent a curve and topological changes are han- when using a fast sweeping method or for
dled naturally), however, the evolution of the level set func- the fast marching method. Only the redistancing and the compu-
tion far away from the zero level set is in general irrelevant tation of the extension velocities cannot be easily parallelized.
and increases the computational complexity without providing The proposed algorithm is in principle of the same order of
574 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

computational complexity as the standard geodesic active con- corresponding standard deviations are , , , and ; the
tour (though admittedly with a larger computational cost per means are , , , . To compute the values of and
point, especially if the feature point search is not parallelized) we make use of the currently detected correspondence point ,
for which real time implementations at standard camera frame and its interior and exterior correspondence points.
rates exist. The probability for an occlusion is given by Bayes’ formula
as
X. OCCLUSION DETECTION
An occlusion in the context of this paper is any image change
that renders the object to be tracked partially (partial occlusion)
or completely (full occlusion) unobservable, e.g., when an ob-
ject moves in between the camera and the object to be tracked We initialize and everywhere. The
and thus covers up parts or all of the latter. Tracking algorithms priors at time step are the smoothed posteriors of time step
need to utilize shape information and/or (at least for short-time . In case 0), (i.e., the probability
partial occlusions) make use of the time history of the object is left unchanged), in all other cases
being tracked (i.e., its dynamics) to be able to tolerate occlu-
sions. Static segmentation methods that do not use shape infor-
mation will in general not be able to handle occlusions.
This section introduces a simple occlusion detection algo- where
rithm6 based on ideas in [73] to be used in conjunction with the
dynamic tracking algorithm proposed in Section VIII to handle
short-time partial occlusions. if outside of
The inside and the outside correspondence points are defined otherwise
as (see Fig. 7) if outside of
otherwise
if inside of
otherwise
The occlusion detection strategy is split into the following six
if inside of
subcases for every point on the contour.
otherwise
0) There is no correspondence point.
1) Only the correspondence point is present.
2) The point is moving outward, the correspondence point
is present, but not its outside correspondence point.
3) The point is moving inward, the correspondence point
is present, but not its inside correspondence point.
and
4) The point is moving outward, both the correspondence
point and its outside correspondence point are present.
5) The point is moving inward, both the correspondence
point and its inside correspondence point are present.
We define the following Gaussian conditional probabilities:
To estimate the current rigid body motion, the following system:

is solved, where . We set

and .
The evolution equation is changed to
where is the estimated time to occlusion, is the velocity
of the point ahead, overlined symbols denote negated values
(i.e., means not occluded), , are the
probabilities of and given an occlusion, and
and given there is no occlusion, respectively. The
6More sophisticated, and less parametric, occlusion detection algorithms are
conceivable; however, this is not the main focus of our work, and the one pro-
posed is sufficient to show that the dynamic geodesic active contour can handle This is an interpolation between (27) and (28) based on the oc-
occlusions when combined with a suitable occlusion detection algorithm. clusion probability.
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 575

Fig. 8. Illustration of the shape of the weighting function w (x ) for the fish
sequence.

Fig. 11. Six frames of a fish sequence. Tracking using the geodesic active
contour. (a) Frame 0. (b) Frame 30. (c) Frame 45. (d) Frame 60. (e) Frame 75.
(f) Frame 90.

For the car sequence, we define

Fig. 9. Three frames of a fish sequence. This is a color image. (a)

Frame 0. (b) Frame 80. (c) Frame 90. (Color version available online at
http://ieeexplore.ieee.org.) This is a measure of angle difference between edge orientation at
correspondence points and the normal of the curve. Ideally, both
should be aligned. The likelihood for a contour point candidate
is then computed as

and the occlusion detection of Section X is performed.

In both cases occlusions are handled. For the fish sequence
the occlusion is dealt with implicitly. The occluding fish moves
Fig. 10. Three frames of a car sequence. This is a color image. (a) over the tracked fish quickly, so that the inertia effects keep the
Frame 0. (b) Frame 14. (c) Frame 55. (Color version available online at fish at a reasonable location. For comparison Fig. 11 shows the
http://ieeexplore.ieee.org.) tracking of the fish in six frames of the same fish sequence by
means of a geodesic active contour. Here, the motion model is
XI. SIMULATION RESULTS static (i.e., the converged to location at frame is the initial
The proposed tracking algorithm is tested on two real video condition for the curve evolution at frame ) and the tracking
sequences. Fig. 9 shows three frames of a fish sequence and result at every frame represents the steady state of the geodesic
Fig. 10 shows three frames of a car sequence. In both cases active contour evolution (6). While the fish is tracked initially,
occlusions occur. For the fish sequence no occlusion detection is the tracking contour subsequently adheres to a second fish and
performed, to demonstrate the behavior of the normal geometric finally looses track completely.
curve evolution algorithm alone, on an image sequence with a For the car example the occlusion (the lamp post) is treated
short-time partial occlusion. Define7 explicitly by means of the proposed occlusion detection algo-
rithm. In both cases the likelihood functions do not incorporate
any type of prior movement information. Doing so would in-
crease robustness, but limit flexibility. Finally, since this active
if contour model is edge-based, the dynamic active contour cap-
if tures the sharp edge of the shadow in the car sequence. Presum-
if ably this could be handled by including more global area-based
otherwise. terms or shape information in the model.
The used likelihood function for the fish sequence is
XII. CONCLUSION AND FUTURE WORK
In this paper, we proposed a new approach for visual tracking
The function depends on the image intensity , the potential based on dynamic geodesic active contours. This methodology
function , and the distance to the contour. incorporates state information (here, normal velocity, but any
7This
other kind of state information can be treated in a similar way)
is simply a monotonic function which increases like a sigmoid up to
x = p , linearly increases for x 2(p ; p ], linearly decreases to zero for with every particle on a contour described by means of a level
x 2(p ; p ] and is zero everywhere else. See Fig. 8 for an illustration. set function. It has the potential to deal with partial occlusions.
576 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

Edge-based approaches trade off robustness for versatility. If We compute the Gâteaux variation by taking the derivative with
strong, clear edge information exists they are a very useful class respect to for ; see the first equation shown at the
of algorithms; however, in many cases more robust techniques bottom of the page. Assuming to be constant integration by
arerequired.Methodsthatincorporate area-basedinfluenceterms parts yields (29), as shown at the bottom of the page. The
have proven to be very efficient for certain applications. To add boundary terms occurring from the integrations by parts drop
more robustness to our methodology, we are currently working out for closed curves. Then, (since (29) has to be fulfilled for
on a dynamic area based approach based on elasticity theory. any )
Our proposed algorithm searches for image features (i.e.,
likelihood maxima) along normal lines of an evolving con-
tour. Thus, the algorithm lies between purely edge-based and (30)
purely area-based approaches. This ties in very well with the where
proposed occlusion detection algorithm, but it places much
importance in finding the “correct” correspondence points. The
main disadvantage of the occlusion detection algorithm is the
large number of tunable parameters it requires. Devising a less To simplify (30), we use the following correspondences:
parametric occlusion algorithm would be beneficial.
We also do not claim that our algorithm is optimal for the spe-
cific image sequences presented. Indeed, whenever possible, ad-
ditional information should be introduced. If we know we want
to track a car, we should make use of the shape information we
have. Not all deformations will make sense in this case. Our
main contribution lies in putting dynamic curve evolution into
a geometric framework and in demonstrating that we can trans-
port any kind of information along with a curve (e.g., marker
particles, enabling us to follow the movement of specific curve
parts over time). This gives us increased flexibility and enables
fundamentally different curve behaviors than in the static (non-
informed) case often used in the computer vision community. Specifically, it follows that
Furthermore, applications for dynamic geodesic active contours
need not be restricted to tracking, e.g., applications in computer
graphics are conceivable (where the curve movement would
then be physically motivated), not necessarily involving an un-
derlying real image (e.g., we could design artificial potential
fields enforcing a desired type of movement). Plugging everything in (30) results in
As geodesic active contours extend to geodesic active sur-
faces, dynamic geodesic active contours can be extended to dy-
namic geodesic surfaces. Our main focus for future research will (31)
be an extension toward area based dynamic evolutions.
which is (11).
APPENDIX I
GEOMETRIC DYNAMIC CURVE EVOLUTION APPENDIX II
COUPLED NORMAL AND TANGENTIAL EVOLUTION
Equation (11) is derived as follows: assume the curve gets
perturbed by yielding the curve The general version of the geometric dynamic curve evolution
equation is given in (31) where is the unit inward normal and

The action integral (7) becomes We can always write

We choose the parameterization such that it is independent of

time. The parameterization thus travels with its particle.

(29)
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 577

Let us derive the general (without prespecified special repa- This must be true for all . Equation (32) reduces to the fol-
rameterization ) evolution equations for and (see [64] for lowing two coupled partial differential equations:
details on some of the equations used). Using an arbitrary curve
parameterization (with , , and
) define
(33)

Arclength is then given by

ACKNOWLEDGMENT

The authors would like to thank E. Pichon and A. Wake for

Then some very helpful comments.

REFERENCES

We can also compute [1] G. Welch and E. Foxlin, “Motion tracking: No silver bullet, but a
respectable arsenal,” IEEE Comput. Graph. Appl., vol. 22, no. 6, pp.
24–38, 2002.
[2] S. Julier and G. Bishop, “Tracking: How hard can it be?,” IEEE Comput.
Graph. Appl., vol. 22, no. 6, pp. 22–23, 2002.
From the previous expressions follows: [3] B. D. Allen, G. Bishop, and G. Welch, “Tracking: Beyond 15 minutes
of thought: Siggraph 2001 course 11,” in Course Notes, Annu. Conf.
Computer Graphics and Interactive Techniques (Siggraph 2001), 2001.
and [4] B. Ulmer, “VITA II – Active collision avoidance in real traffic,” in Proc.
Intelligent Vehicles Symp., 1994, pp. 1–6.
This yields [5] J. Malik and S. Russell, “A machine vision based surveillance system
for California Roads,” Univ. California, Berkeley, Tech. Rep. UCB-ITS-
PRR-95-6, 1995.
[6] D. Beymer, P. McLauchlan, B. Coifman, and J. Malik, “A real-time com-
puter vision system for measuring traffic parameters,” in Proc. Conf.
Computer Vision and Pattern Recognition, 1997, pp. 495–501.
[7] U. Franke, D. Gavrila, S. Görzig, F. Lindner, F. Paetzold, and C. Wöhler,
“Autonomous driving goes downtown,” IEEE Intell. Syst., vol. 13, no.
and 6, pp. 40–48, 1999.
[8] E. D. Dickmanns, “The 4D-approach to dynamic machine vision,” in
Proc. 33rd Conf. Decision and Control, 1994, pp. 3770–3775.
[9] P. F. McLauchlan and J. Malik, “Vision for longitudinal vehicle control,”
in Proc. Conf. Intelligent Transportation Systems, 1997, pp. 918–923.
[10] B. Sinopoli, M. Micheli, G. Donato, and T. J. Koo, “Vision based navi-
Some simple algebra results in gation for an unmanned aerial vehicle,” in Proc. Int. Conf. Robotics and
Automation, 2001, pp. 1757–1764.
[11] C. S. Sharp, O. Shakernia, and S. S. Sastry, “A vision system for landing
an unmanned aerial vehicle,” in Proc. Int. Conf. Robotics and Automa-
tion, 2001, pp. 1720–1727.
[12] Video-Based Surveillance Systems, P. Remagnino, G. A. Jones, N.
Paragios, and C. S. Regazzoni, Eds., Kluwer Academic, Norwell,
MA, 2001.
[13] R. Tanawongsuwan and A. Bobick, “Gait recognition from time-
from (31), which can be written as normalized joint-angle trajectories in the walking plane,” in Proc.
Conf. Computer Vision and Pattern Recognition, vol. 2, 2001, pp.
726–731.
[14] B. Bhanu, D. E. Dudgeon, E. G. Zelnio, A. Rosenfeld, D. Casasent, and I.
S. Reed, “Introduction to the special issue on automatic target detection
and recognition,” IEEE Trans. Image Process., vol. 6, no. 1, pp. 1–6, Jan.
1997.
[15] P. Corke, “Visual control of robotic manipulators – A review,” in Visual
With Servoing. Singapore: World Scientific, 1993, vol. 7, Robotics and Au-
tomated Systems, pp. 1–31.
[16] S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo
control,” IEEE Trans. Robot. Automat., vol. 12, no. 5, pp. 651–670, Oct.
1996.
[17] K. Oka, Y. Sato, and H. Koike, “Real-time fingertip tracking and gesture
recognition,” IEEE Comput. Graph. Appl., vol. 22, no. 6, pp. 64–71,
it follows that 2002.
[18] K. Kansy, T. Berlage, G. Schmitgen, and P. Wiskirchen, “Real-time inte-
gration of synthetic computer graphics into live video scenes,” in Proc.
Conf. Interface of Real and Virtual Worlds, 1995, pp. 93–101.
[19] L. F. Hotraphinyo and C. N. Riviere, “Precision measurement for micro-
surgical instrument evaluation,” in Proc. 23rd Annu. EMBS Int. Conf.,
2001, pp. 3454–3457.
[20] N. Ayache, I. Cohen, and I. Herlin, “Medical image tracking,” in Active
Vision. Cambridge, MA: MIT Press, 1992, pp. 3–20.
(32) [21] Active Vision, A. Blake and A. Yuille, Eds., MIT Press, Cambridge, MA,
1992.
578 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 51, NO. 4, APRIL 2006

[22] A. Blake and M. Isard, Active Contours: Springer Verlag, 1998. [52] S. Angenent, “Parabolic equations for curves on surfaces, Part I. Curves
[23] D. J. Kriegman, G. D. Hager, and A. S. Morse, Eds., The Confluence of with p-integrable curvature,” Ann. Math., vol. 132, pp. 451–483, 1990.
Vision and Control. New York: Springer-Verlag, 1998, vol. 237, Lec- [53] , “Parabolic equations for curves on surfaces, Part II. Intersections,
ture Notes in Control and Information Sciences. blow-up, and generalized solutions,” Ann. Math., vol. 133, pp. 171–215,
[24] I. J. Cox, “A review of statistical data association techniques for motion 1991.
correspondence,” Int. J. Comput. Vision, vol. 10, no. 1, pp. 53–66, 1993. [54] M. Grayson, “Shortening embedded curves,” Ann. Math., vol. 129, pp.
[25] A. Mitiche and P. Bouthemy, “Computation and analysis of image mo- 71–111, 1989.
tion: A synopsis of current problems and methods,” Int. J. Comput. Vi- [55] B. White, “Some recent developments in differential geometry,” Math.
sion, vol. 19, no. 1, pp. 29–55, 1996. Intell., vol. 11, pp. 41–47, 1989.
[26] M. J. Swain and M. A. Stricker, “Promising directions in active vision,” [56] C. Xu, A. Yezzi, and J. L. Prince, “On the relationship between para-
Int. J. Comput. Vision, vol. 12, no. 2, pp. 109–126, 1993. metric and geometric active contours,” in Proc. 34th Asilomar Conf. Sig-
[27] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour nals, Systems and Computers, vol. 1, 2000, pp. 483–489.
models,” Int. J. Computer Vision, pp. 321–331, 1988. [57] G. Sapiro, Geometric Partial Differential Equations and Image Anal-
[28] V. Caselles, F. Catte, T. Coll, and F. Dibos, “A geometric model for ysis. Cambridge, U.K.: Cambridge Univ. Press, 2001.
active contours in image processing,” Numerische Mathematik, vol. 66, [58] J. L. Troutman, Variational Calculus and Optimal Control, 2nd
pp. 1–31, 1993. ed. New York: Springer-Verlag, 1996.
[29] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” Int. [59] M. P. do Carmo, Differential Geometry of Curves and Surfaces. En-
J. Comput. Vision, vol. 22, no. 1, pp. 61–79, 1997. glewood Cliffs, NJ: Prentice-Hall, 1976.
[30] S. Kichenassamy, A. Kumar, P. Olver, A. Tannenbaum, and A. Yezzi, [60] A. Tannenbaum, “Three snippets of curve evolution theory in computer
“Conformal curvature flows: From phase transitions to active vision,” vision,” Math. Comput. Model. J., vol. 24, pp. 103–119, 1996.
Arch. Rational Mech. Anal., vol. 134, no. 3, pp. 275–301, 1996. [61] N. Paragios and R. Deriche, “Geodesic active regions: A new frame-
[31] J. Shah, “A common framework for curve evolution, segmentation, work to deal with frame partition problems in computer vision,” J. Vi-
and anisotropic diffusion,” in Proc. Conf. Computer Vision and Pattern sual Commun. Image Represent., vol. 13, pp. 249–268, 2002.
Recognition, 1996, pp. 136–142. [62] A. Yezzi, A. Tsai, and A. Willsky, “A fully global approach to image
[32] R. Malladi, J. A. Sethian, and B. C. Vemuri, “Shape modeling with front segmentation via coupled curve evolution equations,” J. Visual Commun.
propagation: A level set approach,” IEEE Trans. Pattern Anal. Mach. Image Represent., vol. 13, pp. 195–216, 2002.
Intell., vol. 17, no. 2, pp. 158–175, Apr. 1995. [63] , “Medical image segmentation via coupled curve evolution equa-
[33] D. Terzopoulos and R. Szeliski, “Tracking with Kalman snakes,” in Ac- tions with global constraints,” in Proc. IEEE Workshop on Mathematical
tive Vision. Cambridge, MA: MIT Press, 1992, pp. 3–20. Methods in Biomedical Image Analysis, 2000, pp. 12–19.
[34] N. Peterfreund, “Robust tracking of position and velocity with Kalman [64] B. B. Kimia, A. Tannenbaum, and S. W. Zucker, “On the evolution of
snakes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 6, pp. curves via a function of curvature. I. The classical case,” J. Math. Anal.
564–569, Dec. 1999. Appl., vol. 163, no. 2, pp. 438–458, 1992.
[35] S. Osher and J. Sethian, “Fronts propagating with curvature dependent [65] J. A. Sethian, “A fast marching level set method for monotonically ad-
speed: Algorithms based on Hamilton-Jacobi formulations,” J. Comput. vecting fronts,” Proc. Natl. Acad. Sci., vol. 93, no. 4, pp. 1591–1595,
Phys., vol. 79, pp. 12–49, 1988. 1996.
[36] J. A. Sethian, Level Set Methods and Fast Marching Methods, 2nd [66] J. Gomes, O. Faugeras, and M. Kerckhove, “Using the vector distance
ed. Cambridge, MA: Cambridge Univ. Press, 1999. functions to evolve manifolds of arbitrary codimension,” in Scale-Space
[37] S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Sur- and Morphology in Computer Vision. New York: Springer-Verlag,
faces. New York: Springer-Verlag, 2003. 2001, vol. 2106, Lecture Notes in Computer Science, pp. 1–13.
[38] M. Niethammer and A. Tannenbaum, “Dynamic level sets for visual [67] J. Gomes and O. Faugeras, “Shape representation as the intersection of
tracking,” in Proc. Conf. Decision and Control, vol. 5, 2003, pp. 0
n k hypersurfaces,” INRIA, Tech. Rep. 4011, 2000.
4883–4888. [68] F. Bethuel and J.-M Ghidaglia, Geometry in Partial Differential Equa-
[39] , “Dynamic geodesic snakes for visual tracking,” in Proc. Conf. tions. Singapore: World Scientific, 1994, ch. 1, pp. 1–17.
Computer Vision and Pattern Recognition, vol. 1, 2004, pp. 660–667. [69] D. Adalsteinsson and J. A. Sethian, “A fast level set method for propa-
[40] A. V. Wouwer and M. Zeitz, “State estimation in distributed parameter gating interfaces,” J.. Comput. Phys., vol. 118, pp. 269–277, 1995.
systems,” in Control Systems, Robotics and Automation, Theme in En- [70] J. N. Tsitsiklis, “Efficient algorithms for globally optimal trajectories,”
cyclopedia of Life Support Systems. Oxford, U.K.: EOLSS Publishers, IEEE Trans. Autom. Control, vol. 50, no. 9, pp. 1528–1538, Sep. 1995.
2001. [71] D. Adalsteinsson and J. A. Sethian, “The fast construction of extension
[41] D. Adalsteinsson and J. A. Sethian, “Transport and diffusion of material velocities in level set methods,” J. Comput. Phys., vol. 148, no. 1, pp.
quantities on propagating interfaces via level set methods,” J. Comput. 2–22, 1999.
Phys., vol. 185, no. 1, pp. 271–288, 2003. [72] C. Y. Kao, S. Osher, and Y.-H. Tsai, “Fast sweeping methods for static
[42] M. Isard and A. Blake, “Condensation – Conditional density propagation Hamilton-Jacobi Equations,” UCLA, Los Angeles, CA, Tech. Rep.
for visual tracking,” Int. J. Comput. Vision, vol. 29, no. 1, pp. 5–28, 1998. 03-75, 2003.
[43] N. Peterfreund, “The PDAF based active contour,” in Proc. Int. Conf. [73] S. Haker, G. Sapiro, and A. Tannenbaum, “Knowledge-based segmenta-
Computer Vision, 1999, pp. 227–233. tion of SAR data with learned priors,” IEEE Trans. Image Process., vol.
[44] Y. Rathi, N. Vaswani, A. Tannenbaum, and A. Yezzi, “Particle filtering 9, no. 2, pp. 299–301, Feb. 2000.
for geometric active contours with application to tracking moving and
deforming objects,” in Proc. Conf. Computer Vision and Pattern Recog-
nition, vol. 2, Jun. 2005, pp. 2–9.
[45] A. Yezzi and S. Soatto, “Deformotion: Deforming motion, shape average
and the joint registration and approximation of structures in images,” Int.
J. Comput. Vision, vol. 53, no. 2, pp. 153–167, 2003.
[46] N. Paragios and R. Deriche, “Geodesic active regions and level set
methods for motion estimation and tracking,” Comput. Vision Image
Understanding, vol. 97, pp. 259–282, 2005.
[47] J. D. Jackson, A. J. Yezzi, and S. Soatto, “Tracking deformable moving
objects under severe occlusions,” in Proc. Conf. Decision and Control, Marc Niethammer received the Dipl.-Ing. in engi-
vol. 3, 2004, pp. 2990–2995. neering cybernetics from the Universität Stuttgart,
[48] V. Caselles and B. Coll, “Snakes in movement,” SIAM J. Numer. Anal., Stuttgart, Germany, in 2000, and the M.S. degree in
vol. 33, no. 6, pp. 2445–2456, 1996. engineering science and mechanics, the M.S. degree
[49] N. Paragios and R. Deriche, “Geodesic active contours and level sets in applied mathematics, and the Ph.D. degree in
for the detection and tracking of moving objects,” IEEE Trans. Pattern electrical and computer engineering, all from the
Anal. Mach. Intell., vol. 22, no. 3, pp. 266–280, Jun. 2000. Georgia Institute of Technology, Atlanta, in 1999,
[50] M. Gage and R. S. Hamilton, “The heat equation shrinking convex plane 2002, and 2004, respectively.
curves,” J. Diff. Geometry, vol. 23, pp. 69–96, 1986. He is currently a Research Fellow at the Psychiatry
[51] M. Grayson, “The heat equation shrinks embedded plane curves to round Neuroimaging Lab, Brigham and Women’s Hospital,
points,” J. Diff. Geometry, vol. 26, pp. 285–314, 1987. Harvard Medical School, Cambridge, MA.
NIETHAMMER et al.: DYNAMIC ACTIVE CONTOURS FOR VISUAL TRACKING 579

Allen Tannenbaum received the Ph.D. degree in Sigurd B. Angenent received the Ph.D. degree in
mathematics from Harvard University, Cambridge, mathematics from the University of Leiden, Leiden,
MA, in 1976. The Netherlands, in 1986.
He is a faculty member at the Georgia Institute He spent one year at the California Institute of
of Technology, Atlanta. His research interests are in Technology, Pasadena, on a NATO fellowship. He is
control theory, image processing, and computer vi- currently a Professor of mathematics at the Univer-
sion. sity of Wisconsin, Madison. His interests center on
nonlinear partial differential equations in differential
geometry. He is also interested in exploring the vast
interface between pure mathematics, engineering,
and the sciences.