Mechatronics 15 (2005) 371–402

Object tracking based on the modular

active shape model
Won Kim *, Ju-Jang Lee
Department of Electrical Engineering and Computer Science,
Korea Advanced Institute Science and Technology, 373-1 Kusong-Dong,
Yusong-Gu, Taejon, Republic of Korea

Abstract

The active contour model (ACM) was proposed by Kass et al. in 1988 (M. Kass, A. Witkin,
D. Terzopoulos, Snake: Active contour models, Int. J. Comput. Vision (1988) 321–331) for
segmentation and tracking of target objects in image space. In their theory some kinds of ener-
gies were designed to extract the boundaries of targets by giving higher or lower values of
snake energy to them. This concept has been revised and developed by a few engineers in
the field of computer vision to make their active contour models have a prior shape informa-
tion in their energy equations, more robust characteristics, adaptive abilities in the elastic
parameters, and so on. In spite of these efforts on ACMs, some basic problems such as weak-
ness to strong surrounding edges, the drift of snaxels due to the changes of illumination con-
ditions and sensitivity to cluttered environments, are still substantial problems in the
applications of ACM to real environments. This paper proposes the design and an implemen-
tation of a tracking scheme with a combination of the ACM and the active shape model
(ASM), in which the point distribution models are systematically constructed on the projective
point of view. In this paper there are three main contributions. Firstly, the combination of the
ACM and the ASM is tested to implement a model-based visual tracking system. Secondly, a
systematical approach is proposed to construct a few individual point distribution models
(PDM) generated on the basis of the projection relation. Finally, the modular active shape
model (MASM) is designed to integrate the results of the principal component analysis
(PCA) on the individually generated PDMs. By this concept the independent ideas by
designers on the shape variations of the target are naturally integrated into the MASM which

*
Corresponding author. Tel./fax: +82 42 869 5432.
E-mail addresses: [email protected] (W. Kim), [email protected] (J.-J. Lee).

0957-4158/$ - see front matter
2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.mechatronics.2004.09.002
372 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

is modularly constructed on the basis of individual PDMs. As a result, a model-based visual

tracker, the MASM, is designed to overcome the problems of the ACM, while the ideas of
designer are systematically integrated in this MASM to include the expected variations on
the target.

2004 Elsevier Ltd. All rights reserved.

Keywords: Visual tracking; Snake; Active contour model; Active shape model; Point distribution model

1. Introduction

Visual tracking is one of the interesting fields in robotics, automation systems and
intelligent information systems as it provides rich and integrated information com-
pared to simple sensors that give a few kinds of specialized information. Tracking
in visual space means continuous extractions of features or information in a stream
of images generated by the vision systems. Tracking feature targets could be useful to
understand geometrical information of the environments, in which the systems are
operating, and to know the positional information in 2-dimensional or 3-dimen-
sional space. Furthermore, shape or pattern information of the target can also be
easily obtained by applying the active contour models (ACMs) or the active shape
models (ASMs). Using these tracking concepts, a computer system has been devel-
oped with the hand mouse as a pointing device. In this system the tip of a finger
is tracked in a visual tracking system. Consequently finger tip can act like a conven-
tional pointing device as the mouse [13]. In an automated traffic monitoring system,
a visual tracking system supplies shape information of moving cars and the related
traffic information of highways. Some researchers have tried to design their ACMs
for automatic interpretation of facial expressions. In medical applications, active
contours are used to segment the important organs in MR images, and to analyze
the body motion by tracking movements of arms and legs of humans. In understand-
ing of audio–visual speech, ASMs have been applied to supply the mode informa-
tion by fitting directly the contours into the boundaries of the lips in the image
space [21].

1.1. Motivations

In image processing, until now the amount of information of images has been suf-
ficient to be processed even though the computing technologies are developing rap-
idly. In this context, the tracking of only important local areas are used in real-time
visual tracking, leaving other area of the image unprocessed. Active contour model is
one of these feature-based or local tracking algorithms [5,6]. This model is designed
to track edge segments or edge contours such as boundaries of targets. The edge
information is a more invariant feature than color information against the changes
of illumination conditions. Therefore the active contour models can provide the geo-
metrical information such as the shape and the positions of the targets being tracked
as a fast and efficient method. However, in the real application of ACMs, there are
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 373

Fig. 1. The comparisons of images. (a) Clean background. (b) Cluttered background. (c) The viewing edge
map of (b) image.

many surrounding edges, and the tracking system can easily get confused what to
follow, particularly in cluttered environments.
In Fig. 1 there are three kinds of images, (a) is a sample image in a clean back-
ground, and (b) in a cluttered background. Assume a face-like object to be the target
in this example. Active contour model should locate the boundaries of this target to
track it in a sequence of images. In the cluttered background there are many back-
ground edges as shown in (c) which is the edge map of image (b). The ACM finds
difficulties to locate the boundaries of the target due to surrounding edges in this
case.
In the case of strong surrounding edges the nodes of active contour model, known
as the snake in the visual tracking field, may face the local minima problem. In Fig.
2(a), the target has a triangular shape, and the circles refer to the snake elements, or
snaxels. The two lines to the left of the image act as strong edges. Let us imagine one
of the snaxels lie on a line as shown in Fig. 2(b), causing the snaxel on the line to fall
in the energy of the strong line. Therefore the shape of active contour may break and
may not be able to maintain the triangle shape. A failure in a frame naturally affects
the overall tracking performance as the initial points of snaxels of succeeding are re-
placed with the results of active contours of the previous frame. As a result tracking
errors accumulate in processing the image sequence. In this local minima point of
view, robust tracking abilities are needed in designing visual tracking algorithms.

1.2. Features in this paper

There are problems of tracking when using the active contour model, and they
are related with the illumination condition, the variations of edge condition, the

(a) (b)

ta
rg
et
target

Fig. 2. Target tracking in strong surrounding edges.

374 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

surrounding edges near targets and so on. At present more robust tracking model are
required to overcome these problems. The proposed tracking model of this paper
possess following important features.
A shape-based or model-based tracking model is tried to overcome the surround-
ing edges, the noisy edges, and the drifts of snaxels on contours. This is more robust
than node-based model trackers. Due to the inclusion of ASM scheme, the corre-
sponding relations of this model between two contours in different frames are better
compared to the conventional model. The PDMs are generated in the consideration
of the projective relationship, and this method is reasonable because almost all pla-
nar rigid objects are under the projective transformation. The designerÕs concept on
the variations of shape are individually implemented in each PDMs. The individually
generated PDMs are integrated in the modular active shape model (MASM) by
selecting the effective eigenvectors of each PDM.

1.3. Related topics

Active contour models are developed as useful tools for segmentation and track-
ing of rigid or deformable objects. Snake is another name of the active contour mod-
els (ACMs), introduced by Kass et al. in 1988 [1]. They have designed three kinds of
Snake energies such as the internal energy, the image energy, and the external energy.
The process of energy minimization naturally provides the results of the segmenta-
tion and tracking of the target objects in important images. They tried to solve their
energy minimization problem by variational approaches. Amini et al. [2] have pre-
sented dynamic programming method to solve the energy minimization problem.
Leymarie and Levine [3] tried to make segmentation and tracking deformable objects
like ameba, and proved the convergence of the motions of their Snake. Xu and
Prince [4] used Snake for the purpose of modelling the three dimensional objects.
Cootes et al. have proposed the active shape models (ASMs) [7,9], in which the con-
cepts of point distribution model (PDM) are used as the database of the variations of
shapes and the principal component analysis (PCA) has been applied to derive the
eigenvectors and the modes of the PDMs of the important shapes. In 1998 Cootes
et al. [8] proposed the active appearance models (AAMs) to includes the pattern
information of targets in their PDMs. To include the model information of the shape
of the target, the knowledge on the shape should be used as a priori information. In
this case an affine transformation is used to match the model shape to the current
shape being extracted. For this, the model shape should have affine invariant char-
acteristics. In 1998 Ip and Shen proposed an affine invariant Snake, in which trian-
gular areas that are invariant under affine transformations are used to design the
model energy [14]. Shen and Davatzikos [15] have designed the adaptive-focus
deformable model, in which the concept of the attribute vector is used to design
the model energy. The attribute vector consists of a set of triangular areas on a seg-
ment of target. Mexatas et al. proposed the elastically adaptive deformable model in
2002 [22]. In their paper the elastic parameters are adaptively controlled on the
distances from the real edges to make the contours fit the real boundaries of the
target.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 375

1.4. Organization

In Section 1 introductory statements are provided with the motivations of this re-
search and the literature survey on the related topics. The preliminary studies on the
related topics are included in Section 2. The concepts on the ACMs and the ASMs
are briefly explained, and the analysis on the AI-snake and the adaptive-focus
deformable model are discussed to derive the requirements of this research. Problem
statements on this research are expressed in Section 3, in which the main problems in
the streams on the ACMs and the ASMs are investigated to make motivations to this
research. In Section 4 new tracking scheme is proposed to overcome the problems
explained in Section 2. In the proposed scheme, the concepts on the generations of
individual PDMs are explained, and they are the ones for the implementations of
the designerÕs ideas on the expected variations of the shape of the desired target.
The modular active shape model (MASM) is also explained, which is modularly con-
structed on the results of PCA of the individually generated PDMs. The simulation
results are included in Section 5 to emphasize the effectiveness of the proposed
scheme. The concluding remarks are summarized in Section 6, which goes over
the important features on this research.

2. Preliminary study

In this chapter the related topics are briefly explained to provide the required
backgrounds for this research work. Firstly, the ACMs and the ASMs are dealt with
the brief explanations followed by the concepts on the AI-Snake and the adaptive-
focus deformable model, with the analysis to emphasize the motivations to identify
the main problems of this research field.

2.1. Active contour models [1,2,11,12]

The ACM has been proposed to represent images in terms of energy surface, in
which the boundaries of the target object have the minimal or maximal values in
their surface.
In Fig. 3 the concept of a transformation of an image into the energy surface is
conceptually explained. A simple gradient operation such as
j$I(v(s))j2 is used to
generate the energy surface. By the characteristics of the gradient operators, the
boundary pixels of the object in the image have lower gradient values compared
to the pixels in other areas. As a result of this transformation, segmentations of
the object in consideration can be related to the optimization problems.
A deformable curve v(s, t) is defined as follows
vðs; tÞ ¼ ðxðs; tÞ; yðs; tÞÞ ð1Þ
where x and y are image coordinates, and the parameters s and t represent the spatial
index and time, respectively. The Snake nodes or elements, called snaxels are located
in the field of active contour. The ith snaxel of a discrete active contour is expressed
376 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Fig. 3. The transformation of an image to the energy surface.

as V(i). The energies should be designed to make the contour shrink naturally into
the boundaries of the target in the image, and finally is trapped in the valley of
the energy surface for the boundaries.
Kass et al. [1] proposed three kinds of energies: the continuous energy, the curva-
ture energy and the image energy. The continuous energy, Econtinuous, is related with
the first derivative of v(s) along spatial index s. Therefore Econtinuous is defined as
2
Econtinuous ðvðsÞÞ ¼ aðvðsÞÞ j vs j ð2Þ
where a(v(s)) is a weighting parameter on v(s), and vs  ov os
. The characteristics of this
equation is such that, it maintains the space between the neighboring snaxels in aver-
age distance for all the neighboring snaxel pairs. This fact can be easily confirmed by
its first derivative, expressed in the discrete version. In discrete version, snaxels feel
vector forces to make them move to maintain the average distance between neigh-
boring snaxels.
The curvature energy, Ecurvature, is related with the second derivative of v(s) along
spatial index s. Therefore Ecurvature is defined as
2
Ecurvature ðvðsÞÞ ¼ bðvðsÞÞ j vss j ð3Þ
o2 v
where b(v(s)) is a weighting parameter on v(s) and vss  .
This energy due to its sec-
os2
ond derivative nature makes the snaxels shrink into the internal direction of the tar-
get object.
The image energy depends on the gradient values of a pixel point of the image
frame to give snaxels higher energy values on the edges of the target. By this energy
term, those snaxels around edges would fall into the valleys of the image energy
around the edges, and remain in the boundaries of the target. The image energy,
Eimage, is defined as
Eimage ðvðsÞÞ ¼
cðvðsÞÞ j rIðvðsÞÞj2 ð4Þ
where c(v(s)) is a weighting parameter on v(s), $ is the gradient operator, and I(v(s))
is the pixel value of v(s). The total energy potential of the snake, Esnake, would be
obtained by aggregating snake energies for an interval X of the spatial index s, as
follows:
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 377

Fig. 4. The segmentation and tracking process of the ACMs.

Z
Esnake ðvðsÞÞ ¼ fEcontinuous ðvðsÞÞ þ Ecurvature ðvðsÞÞ þ Eimage ðvðsÞÞg ds: ð5Þ
X

Therefore if the active contours are placed around the target, it will shrink into the
internal region as described in Eq. (3), maintaining the spaces in average distance
according to Eq. (2), and finally the snaxles will be trapped in the boundaries of
the target as given in Eq. (4) which are also the valleys of energy as characterized
by the gradient operations. This segmentation and tracking concepts is illustrated
in Fig. 4.

2.2. Active shape models [7,8]

An active shape model is a kind of iterative fitting algorithm that uses the related
shape information. To constrain shape information in this algorithm, a point distri-
bution model (PDM), which is a sort of database containing the shape information
of targets, should be built. There may be many candidates of the shape models that
depend on the motions of the target, the variations of viewing geometry, etc. To con-
struct a PDM in a systematic way, all landmarks, which are the feature points of the
shape should maintain their corresponding locations for each various shape model.
Let the ith shape model be defined as
xi ¼ ðxi1 ; y i1 ; xi2 ; y i2 ; . . . ; xiN ; y iN ÞT ð6Þ
where i is the shape model index, (xij, yij) is the x–y coordinate in the image plane for
the jth landmark point of ith shape model, and N is the number of total landmarks.
Here xi is a (2N · 1) vector. Another type of definition for xi is x ~i given by
 
xi1 xi2 . . . xiN
~i ¼
x ð7Þ
y i1 y i2 . . . y iN
whose dimension is (2 · N).
Given N aligned model shapes, xi (i = 1, . . ., N), the mean shape, x
, and the devi-
ations from x
 are calculated using
1 XN
¼
x xi ð8Þ
N i¼1
and
dxi ¼ xi
x
: ð9Þ
378 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Fig. 5. Shape models for an object. (a) Test image in which the object exists in the circle. (b) The
extraction of the shape of the target. (c) The variations along the X-axis. (d) The variations along the Y-
axis.

The schematic explanation on shape models is depicted in Fig. 5. In this test image
there is a target object marked in a circle in Fig. 5(a). The extraction of the boundary
shape of the target using ACM is illustrated in Fig. 5(b). Using the variations of the
shape along the X and Y axes, the shape models can be constructed as shown in Fig.
5(c) and (d), respectively. By obtaining the deviations, dxi (i = 1, . . ., N), the covari-
ance matrix is calculated as
1 XN
S¼ dxi  dxTi : ð10Þ
N i¼1
By analyzing S as a linear system, the eigenvectors of S can be calculated. Conse-
quently an active shape model (ASM) for a shape can be constructed as follows:
x¼x
þPb ð11Þ
where (2N · t) matrix P is comprised of t eigenvectors ordered in magnitude as
follows:
P ¼ ½p1 ; p2 ; . . . ; pt ; ð12Þ
(t · 1) vector b is a set of modes for the ASM, and t should be decided to make
this ASM spans the entire range of variations of the shape given by the designer
(Fig. 6).
For the shape models expressed in Fig. 5(c) and Fig. 5(d), PCA is applied to ob-
tain the eigenvector and modes for the database. It is natural to obtain two main
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 379

Fig. 6. The shape variations on modes of the PCA. (a) The variations on the mode b1. (b) The variations
on the mode b2.

modes and the related eigenvectors with them because the shape models have the
variations along only two directions. Therefore, two modes and the related eigenvec-
tors are adequate to fully express the variations of the shape along X and Y axes. The
shape equation is expressed as follows for these models:
x¼x
þPB ð13Þ
where (2N · 2) matrix P is comprised of two eigenvectors ordered in magnitude as
follows:
P ¼ ½p1 ; p2 ; ð14Þ
and (2 · 1) vector B has only two modes of b1 and b2.
Let the two modes vary in a certain range to know the effects on the shape vari-
ations expressed in (13) and (14). In this example the facts are easily observed that
the mode b1 is related with the shape variations along Y-axis, while the mode b2
being related with X-axis as shown in Fig. 6.

2.3. AI-Snake [14]

The AI-Snake has the meanings of the affine-invariant active contour model be-
cause the model energy of this snake has an invariant characteristic as the affine
transformations. Ip and Shen [14] proposed three kinds of snake energies such as
the smoothing energy, the model energy and the data energy. Therefore, total snake
energy is the sum of these energies as
Esnake ¼ Esmooth þ Emodel þ Edata : ð15Þ
Firstly, the smoothing energy is designed to make snaxels move into the internal
directions of target, and is define as
X
N
Esmooth ¼ esmooth ðV i Þ ð16Þ
i¼1
380 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

and

j areai j AREAmdl
esmooth ðV i Þ ¼ a   1 2
; ð17Þ
AREAnow 2
d
where areai is a triangular area of (Vi
1, Vi, Vi+1), AREAnow is the total area of the
snake, AREAmdl is the total area of the shape model. Symbol a is a weighting factor
and d is the average inter-snaxel distance.
Secondly, the model energy is designed to make snaxels move to the affine-trans-
formed model shape. In the model energy in (19), W aligned i is the ith aligned node
point of the model shape by using affine transformation, and Vi is the corresponding
ith snaxel of the active contour. If Vi moves close to W aligned i , the areas of
DW aligned V i W aligned and DW aligned V i W aligned approach zero according to (19). In regard to this
i
1 i i iþ1

concept, the model energy, Emodel, is designed as

X
N
Emodel ¼ emodel ðV i Þ ð18Þ
i¼1

and
j S W aligned V i W aligned j þ j S W aligned V i W aligned j AREA
i
1 i i iþ1 mdl
emodel ðV i Þ ¼ b   1 2
; ð19Þ
AREAnow 2
d

where b is a weighting factor, jS W aligned V i W aligned j and jS W aligned V i W aligned j are the areas of the
i
1 i i iþ1
triangles DW aligned V i W aligned and DW aligned V i W aligned , respectively. Finally, the data energy is ex-
i
1 i i iþ1
pressed by the same function as the image energy of the conventional active contours
in [1].
The AI-Snake has robust characteristics in the cluttered environment because the
model shape information is included in the energy formulation. However, the track-
ing scheme has a significant dependence on the affine transformation. In the interme-
diate state of every iteration, there are possibilities of snaxels to fall into the
surrounding edges in the cluttered environment. This distortion of shape, dA, in-
duces the disturbance in the affine transformation, and causes distorted aligned
model shapes. Therefore, the stabilities of this snake is not guaranteed. However,
analyzing their simulation result, it can be guaranteed that the boundary shape of
the target object shrinks gradually into the center. In case of including translational
motion, there may be severe surrounding edges in the direction of motion, and these
may induce larger disturbance term, dA.

2.4. Adaptive-focus deformable model [15]

For a contour C there is a set of node points, {Vi = (xi, yi)ji = 1, 2, . . . , N}. For the
ith snaxel Vi, its attribute is defined on R, which is the number of triangles related
with this node, as
T
F i ¼ ½fi;1 fi;2    fi;R ð20Þ
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 381

where fi,m is the area of a triangle comprised of three snaxels, Vi
m, Vi and Vi+m and
m is a variable index. In the adaptive-focus deformable model, the concept of this
attribute vector is used to design the model energy because of the affine-invariant
characteristics.
In this model, snake energy is comprised of two kinds of energies, a model energy
Emodel
i and a data energy Edata
i as follows:
XN
Esnake ¼ xi E i ð21Þ
i¼1
X
N
¼ xi ðEmodel
i þ Edata
i Þ; ð22Þ
i¼1

where xi is a weighting factor for the ith snaxel.

The Emodel
i is designed to include the degree of similarity between the snake and the
model. Therefore Emodel
i is defined as
X N  Snk 2
Emodel ¼ d vs
^
f^ Mdl
f ð23Þ
i i;vs i;vs
vs¼1
Snk Mdl
where f^ i;vs and f^ i;vs are the normalized attribute elements for the snake and the
model, respectively, and dvs is a weighting parameter for the vs-th triangle area.
The process of fitting this model to the real boundaries is the method of segment-
based fitting on the basis of the affine transformation. Therefore, a priori knowledge
on the shape and deformation patterns is needed to apply this model. This model is
adequate to segment biological objects.

3. Problem statements

There are many problems that make the direct application of active contours to
the real world unstable. The problems are related with the cluttered environments,
the condition of illumination and the distortions in shape due to noise. The items
of problems are listed below:

• surrounding edges,
• strong edges in motion direction,
• snaxel drift due to illumination change, and
• weak edges.

The physical concepts on these problems are dealt in this chapter to derive the
requirements naturally for the new tracking scheme.

3.1. Surrounding edges

In cluttered environments there are a lot of surrounding edges around the target.
The surrounding edges comprise the edges which exist in the background or on the
382 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Next
Target
Target Target
Frame
Frame

Fig. 7. The problem of surrounding edges: the snaxels have difficulties in classifying the edges in cluttered
environment.

other objects, particularly the edges around the real targets. These edges create local
minima in the optimization, therefore the snaxels having small search windows may
have difficulties in classifying the edges in cluttered environments. This belongs to the
category of aperture problems.
In Fig. 7 the problem upon this is depicted schematically. Almost all snakes use
the previous tracking results as the initial starting points for the next frame. In the
figure the target slightly moves to the right. The snaxels are expressed as small dots
in the figure, and they are confused to find real edges. As a result there are possibil-
ities for snaxels to fall in the local minima, that are not the real edges but the sur-
rounding edges. There are possibilities to solve this problem if the fitting process
is carried out globally.

3.2. Strong edges in motion direction

In the case of the existence of strong edges in the motion direction of targets, it is
difficult for snaxels to overcome these strong edges because they are deep valleys for
snaxels in the energy space. The concepts on the problems on the strong edges are
depicted in Fig. 8. The snaxels have much difficulties to avoid this thick line because
it has large image energy. Therefore, the structure of the ACMs may break if the fit-
ting is carried out locally (node-based).

Next
Frame
Frame

Fig. 8. The problem of strong edges: the snaxels fall naturally into the strong local minima.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 383

3.3. Snaxel drift due to illumination change

The corresponding relations on contours are important to recover the motion

information or to reconstruct depth information. For high quality image processing,
it is necessary that the snaxel maintain the relation with the corresponding snaxels on
the contour, despite the changes of the environment.
In Fig. 9 the concept of 3D reconstruction is illustrated. A 3D object is projected
on the different image planes. If the corresponding points on the contours maintain
on these two image planes, despite the changes of the size and the position for the
same object, then the quality of reconstruction will be high.
In Fig. 10 there are two energy diagrams for the two image frames, in which there
are three axes; X, Y, and image energy (Eimage). Between two frames there are no
changes such as the motion of target, viewing of camera, etc., except the change
of illumination. The illumination change affects the variations of the pixel values
of image space, and finally on the surface of image energy because the gradient func-
tion is a function of pixel values as given in (4). As seen in Fig. 10, different illumi-
nation conditions induce different motions for snaxels on the same contour. In the
figure, the two snaxels of ith frame moves to the left direction due to the gradient
of Eimage, while the center snaxel of (i + 1)th frame moves to the right direction
due to the different Eimage gradient induced by the change of illumination. This

Fig. 9. The concept of 3D reconstruction.

y y

x x

-E image -E image

(i)-th Frame (i+1)-th Frame

Fig. 10. The effect of the change of illumination condition.

384 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Fig. 11. The occurrence of weak edge.

phenomenon of snaxel drift due to the environmental change have significant effects
on the quality of 3D reconstruction or motion estimation. The major reason being
the corresponding relations between two contours of two successive frames is not
guaranteed on the occurrences of these drifts.

3.4. Weak edge

Weak edge refers to a phenomenon on local segments, where the edges of contours
are temporarily weak due to the illumination condition, or geometrical characteristics.
The effects of weak edges on the contour tracking is depicted in Fig. 11, in which
weak edge occurs locally and temporarily on the upper portion of the target in the ith
frame. Due to this weak edges, Eimage is weak on this segment, and therefore the
Econtinuous and Ecurvature become dominant over Eimage. As a result Ecurvature will
make the segment of active contour move into the internal area of the target in
the next frame. At this point the problem is that a instant failure affects on the over-
all tracking performance, because the recovery to the real edges is not guaranteed for
the conventional snake that falls temporarily into the local edge segments.

4. Tracking scheme using the modular active shape model

Based on the analysis of the problems of the active contour models, robustness are
required in the design of visual tacking systems. Most problems are related with the
fitting method of ACMs, in which the fitting is tried by node-based method. Node-
based method is sensitive to the local minima of the environments, and furthermore
the malfunctions of few individual snaxels would lead to serious shape distortions.
To overcome the problems described in the previous chapter, model information
should be included in the design stage. Many researchers have recently tried to
consider the methods to include model information in energy functions, and the
AI-Snake and the adaptive-focus deformable model have been designed based on
invariant features such as triangular areas [14,15]. The AI-Snake is greatly dependent
on the affine transformation and noise. Furthermore surrounding edges may induce
distorted shape in intermediate state, therefore adaptive-focus deformable model is
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 385

appropriate to segment biological objects. The method of using the ASMs to refine
the results of an ACM is explained in this chapter. Especially in this method the
modular ASM (MASM) is constructed to integrated the individually generated
information on the variations of target shapes.

4.1. Design requirements

To overcome the problems discussed in the previous chapter, the design

requirements listed below should be considered in the design of visual tracking
systems:

• model-based or segment-based motion of the active contour,

• provide the ability of recovery to the real boundaries upon occurrence of shape
distortion in intermediate stages,
• design the tracker in consideration of the expected shape variations.

Firstly, the motion of active contours should be controlled in the model or seg-
ment-based method. As explained earlier, node-based motion is sensitive to noise
and surrounding edges. Especially when designing the model-based active contours,
the variations of the reference shape should be fully considered in the construction of
model database.
Secondly, there is a possibility of recovery to the real edges for snaxels falling into
the local edges. This means that providing of the possibilities of recovery to the real
boundaries upon occurrence of shape distortions in intermediate stage. For the con-
ventional active contours, an instant failure affects on the overall tracking perform-
ance, and the possibility of recovery is very low.
Finally, the visual tracking algorithm should be designed in consideration of the
expected shape variations. In robotics, the vision systems are utilized as sensors, that
detect the target objects and provide the geometric information to controllers. Most
of the target are rigid objects and the variations of them are under projective rela-
tionships. Therefore, the variations of the target in consideration can be fairly ex-
pected using the projective transformation.

4.2. Tracking scheme using the modular ASM

A tracking scheme using the modular active shape model (MASM), proposed in
this paper, is illustrated in Fig. 12. The explanations on the MASM will be given in
the following sections. In this scheme the steps to make the overall tracking system
are as follows:

• initialization of active contour,

• grabbing new image,
• the active contour as a preprocessor,
• the fitting step of MASM, and
• the refinement of the active contour model.
386 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Projective Projective Modular

PDM PDM PCA
ASM
Generation Generation

Initial Affine
Shape Transform
Initialization Preprocessor Postprocessor
(Open Problem) Refinement
Tracking
Grabbing New to
ASM
E continous New Position Real Edge
Fitting
E curvature Image
E continous Shape E continous
E image E image E image

Fig. 12. The tracking scheme using the modular active shape model (MASM).

The initialization of the active contour model should be firstly applied to extract
the basic shape information of the target in consideration. However, it is not easy
for the engineers in visual tracking field to extract the shape of target at the first frame
without any a priori information such as the rough position information. Because this
initialization problem on active contours is an open problem, it is assumed that the
tentative position information of the target is given at the first frame and the place-
ment of the active contour can be done around the target depending on this assump-
tion. After tentative placing of the initial active contour around the target, the active
contour will contract into the internal direction of the target while spacing the snaxels
in average distance because of the energy terms such as Econtinuous and Ecurvature, and
finally approach the boundaries of the target by the effect of Eimage as explained in (5).
To process the next image, the vision system will grab new image frame after
extracting the boundary information from the first frame. In this image the variations
on the shape and the position of the target must be very small due to the assumption
that the operating frequency of the vision system is much faster than the one of the
target motion. This assumption is effective for most of the real automatic systems.
The active contour starts to detect the boundary of the target in new images from
the starting points obtained from the tracking results of the previous image. In this
step only two energies such as Econtinuous and Eimage are used to construct a snake
energy because Ecurvature gives the effects on the internal motion for the snaxels.
The results of the application of the active contour, which have the new position
and shape information on the boundaries of the target, also possess the distortion
components induced by noise or the surrounding edges.
Therefore, the ASM is applied to fit the model shape on the results including the
distortion terms by the active contour. At this moment, the ASM in this paper are
constructed on the basis of modular concept. According to the modular concept,
the eigenvectors from the results of different PCAs are integrated modularly to cover
the variations of shapes.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 387

The results of fitting the model to the current data of the ACM show imperfec-
tions at the real boundaries of the target because the ASM is operated only on the
model data. Therefore, it is necessary to refine the fitting result of the ASM to make
the snaxels find the real boundaries, which is near from the model data. The ACM as
a post-processor is applied for this refinement, with smaller search window than the
window of the ACM in the preprocessor.
In the following sections explanations on the MASM will be given, especially on
the motivations and the methods to make the projective PDMs and on the construc-
tion of the modular ASM.

4.3. Projective generation of PDMs

PDM is a kind of database that holds of information on the variations of the

shape of the target. To make an ASM, the PDM on the target shape should be pre-
pared first, and then the PCA is applied to extract the eigenvectors related to the var-
iances of the database. For this reason the generation of PDM is very important and
the preparation of the PDM should be done in consideration of the expected varia-
tions of the shape. At this moment the designer should expect the type of the shape
variations. It is natural to expect that the rigid object must be under the projective
relationships. As a result, PDMs in this research is generated on the projective
transformation.
There are six kinds of PDMs, which are generated on the selections of the rotating
references. The PDM1 is generated by rotating the object contour, which is seg-
mented at the first frame, along the X-axis and Y-axis as depicted in Fig. 13. The pro-
jection relationship between image coordinate, [u, v, w]T, and 3-D coordinate,
[x, y, z, 1]T, is described as
2 3
2 3 x
u 6y7
6 7 6 7
4 v 5 ¼ P6 7 ð24Þ
4z5
w
1
where P is the camera projection matrix.

Fig. 13. The concept on the generation of PDM1.

388 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Camera projection matrix, P, is the multiplication of internal camera parameter,

K, with a Euclidean transformation,M as
P ¼K M ð25Þ
2 3
fu 0 u0  
R t
¼ 4 0 fv v0 5  ð26Þ
000 1
0 0 1
2 3
2 3 r11 r12 r13 tx
fu 0 u0 6 r21 r22 r23 ty 7
¼ 4 0 fv v0 5  6 4 r31 r32 r33 tz 5
7 ð27Þ
0 0 1
0 0 0 1
where fu and fv are conversion factors from metric scale to pixel unit for u and v coor-
dinate, respectively. The qualities u0 and v0 are the center points of the u–v coordi-
nated to the metric coordinate, respectively, and R and t are rotation matrix and
translation matrix between the world coordinate and camera coordinate.
The PDM1 is constructed by combining the results that are obtained by rotating
the projection plane along the X-axis and the Y-axis separately. The two components
for constructing PDM1 is shown in Fig. 14. For the generation of PDM1 in the
geometry of Fig. 13, the projection matrix, P, of (25) is applied in the forms of:
2 3
2 3 1 0 0 0
f u 0 u0 6 7
6 7 6 0 cos /
sin / 0 7
P ¼ 4 0 f v v0 5  6 7 ðX -axis rotationÞ; ð28Þ
4 0 sin / cos / z 5
0 0 1
0 0 0 1
2 3
2 3 cos h 0 sin h 0
fu 0 u0 6 0
6 7 6 1 0 077
P ¼40 fv v0 5  6 7 ðY -axis rotationÞ; ð29Þ
4
sin h 0 cos h z 5
0 0 1
0 0 0 1
where / and h are rotation angles.

Fig. 14. The two components for the construction of PDM1.

 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 389

For the generation of database including skewing variations, the projection plane
should be rotated with respect to center points. In this research four points are se-
lected to provide skewing effects on the constructions of PDMs as shown in Fig.
15. The resulting shape models are shown in Fig. 16. In that, the four set of the shape
models are the skewing variations for distinct points. PDM2 is constructed by com-
bining four kinds of shape models including skewing variations. From PDM3 to
PDM6 each PDM is constructed for each skewing shape model. The method on
the generation of this projective PDMs is summarized in Table 1.

Fig. 15. The distinct points for the generation of the skewing PDMs.

Fig. 16. The shape models including skewing variations on four distinct points.
390 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Table 1
The type of projective PDMs
PDM Description Remarks
PDM1 Combination of the results along the X- and Y-axis
PDM2 Combination of the skewing results for four distinct points
PDM3 The shape model on the skewing result for a distinct point Number 1
PDM4 The shape model on the skewing result for a distinct point Number 2
PDM5 The shape model on the skewing result for a distinct point Number 3
PDM6 The shape model on the skewing result for a distinct point Number 4

For these individually generated PDMs in the base of projective relationships,

PCA is applied to obtain the important eigenvectors and the corresponding modes.
After this analysis the important modes can be discriminated by changing the modes
within certain ranges. The ASM will make different shapes if the modes are impor-
tant while there are very small variations in shape for minor modes.
Let the ith shape model of a PDM be defined as
T
xi ¼ ðxi1 ; y i1 ; xi2 ; y i2 ; . . . ; xi25 ; y i25 Þ ð30Þ
where i is the index of the shape model, (xij, yij) is the x–y coordinate in image plane for
the jth landmark point of ith shape model. The total number of landmark points is 25.
Given N model shapes, xi (i = 1, . . . , N) which are generated from projective
method for each PDM, the mean shape, x , is calculated as
1 XN
¼
x xi : ð31Þ
N i¼1
The deviation for each model shape from x
 is calculated by
dxi ¼ xi
x
: ð32Þ
In this procedure the total number of shape models, N = 8.
After obtaining the deviations, dxi (i = 1, . . . , 8) for each PDM, the covariance
matrix is calculated by
1 XN
S¼ dxi  dxTi : ð33Þ
N i¼1
By analyzing S, its eigenvectors can be calculated, and an active shape model (ASM)
for a PDM can be constructed as follows:
x¼x
þPb ð34Þ
where (2N · 6) matrix P is comprised of 6 eigenvectors ordered in magnitude as
follows:
P ¼ ½p1 ; p2 ; . . . ; p6 : ð35Þ
In that, modes are defined by a (6 · 1) vector,
T
b ¼ ½b1 ; b2 ; . . . ; b6 : ð36Þ
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 391

The results on PCA are shown in Figs. 17–19. From the result of the PCA on PDM1,
the mode 1 and the mode 2 are important, and related with the shape rotations along
X-axis and Y-axis, respectively. For the case of PDM2, four modes are related as,

Variations for Data Set1 along b1 Variations for Data Set1 along b1 Variations for Data Set1 along b1
400 600 400
b1 b2 b3
b1- b2- b3-
b1+ 400 b2+ b3+
200 200

200
0 0

0
-200 -200
-200

-400 -400
-400

-600 -600
-600

-800 -800 -800

-400 -200 0 200 400 600 800 1000 -200 0 200 400 600 800 -200 0 200 400 600 800

Variations for Data Set1 along b1 Variations for Data Set1 along b1 Variations for Data Set1 along b1
400 400 400
b4 b5 b6
b4- b5- b6-
200 b4+ 200 b5+ 200 b6+

0 0 0

-200 -200 -200

-400 -400 -400

-600 -600 -600

-800 -800 -800

-200 0 200 400 600 800 -200 0 200 400 600 800 -200 0 200 400 600 800

(a)

Variations for Data Set2 along b1 Variations for Data Set2 along b1 Variations for Data Set2 along b1
-50 -60 -60
b1 b2 b3
-60 b1- b2- b3-
-70 -70
b1+ b2+ b3+
-70
-80 -80
-80
-90 -90
-90

-100 -100 -100

-110
-110 -110
-120
-120 -120
-130
-130 -130
-140

-150 -140 -140

190 200 210 220 230 240 250 190 200 210 220 230 240 250 260 190 200 210 220 230 240 250

Variations for Data Set2 along b1 Variations for Data Set2 along b1 Variations for Data Set2 along b1
-60 -60 -60
b4 b5 b6
b4- b5- b6-
-70 -70 -70
b4+ b5+ b6+

-80 -80 -80

-90 -90 -90

-100 -100 -100

-110 -110 -110

-120 -120 -120

-130 -130 -130

-140 -140 -140

190 200 210 220 230 240 250 190 200 210 220 230 240 250 190 200 210 220 230 240 250

(b)

Fig. 17. The plots of the shapes of ASM for PDM1 (a) and PDM2 (b), along the variations of the modes.
392 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

translation along X, translation along Y, rotation along X and rotation along Y. For
the cases from PDM3 to PDM6, only one mode is important for each PDM. For

Variations for Data Set3 along b1 Variations for Data Set3 along b1 Variations for Data Set3 along b1
-50 -50 -50
b1 b2 b3
b1- b2- b3-
-100 b1+ b2+ b3+
-100 -100

-150
-150 -150

-200
-200 -200
-250

-250 -250
-300

-350 -300 -300

-400 -350 -350

240 260 280 300 320 340 240 260 280 300 320 340 240 260 280 300 320 340

Variations for Data Set3 along b1 Variations for Data Set3 along b1 Variations for Data Set3 along b1
-50 -50 -50
b4 b5 b6
b4- b5- b6-
-100 b4+ -100 b5+ -100 b6+

-150 -150 -150

-200 -200 -200

-250 -250 -250

-300 -300 -300

-350 -350 -350

240 260 280 300 320 340 240 260 280 300 320 340 240 260 280 300 320 340

(a)

Variations for Data Set4 along b1 Variations for Data Set4 along b1 Variations for Data Set4 along b1
-80 -80 -80
b1 b2 b3
b1- b2- b3-
b1+ b2+ b3+
-100 -100 -100

-120 -120 -120

-140 -140 -140

-160 -160 -160

-180 -180 -180

-200 -200 -200

250 300 350 400 450 250 300 350 400 250 300 350 400

Variations for Data Set4 along b1 Variations for Data Set4 along b1 Variations for Data Set4 along b1
-80 -80 -80
b4 b5 b6
b4- b5- b6-
-100 b4+ -100 b5+ -100 b6+

-120 -120 -120

-140 -140 -140

-160 -160 -160

-180 -180 -180

-200 -200 -200

250 300 350 400 250 300 350 400 250 300 350 400

(b)

Fig. 18. The plots of the shapes of ASM for PDM3 (a) and PDM4 (b), along the variations of the modes.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 393

each PDM3, 4, 5 and 6, the mode is related with the skewing for the distinct point 1,
2, 3 and 4. The results of PCAs are summarized in Table 2.

Variations for Data Set5 along b1 Variations for Data Set5 along b1 Variations for Data Set5 along b1
100 50 50
b1 b2 b3
b1- b2- b3-
b1+ b2+ b3+
50
0 0

0
-50 -50

-50

-100 -100
-100

-150 -150
-150

-200 -200 -200

240 260 280 300 320 340 240 260 280 300 320 340 240 260 280 300 320 340

Variations for Data Set5 along b1 Variations for Data Set5 along b1 Variations for Data Set5 along b1
50 50 50
b4 b5 b6
b4- b5- b6-
b4+ b5+ b6+
0 0 0

-50 -50 -50

-100 -100 -100

-150 -150 -150

-200 -200 -200

240 260 280 300 320 340 240 260 280 300 320 340 240 260 280 300 320 340

(a)

Variations for Data Set6 along b1 Variations for Data Set6 along b1 Variations for Data Set6 along b1
-80 -80 -80
b1 b2 b3
b1- b2- b3-
b1+ b2+ b3+
-100 -100 -100

-120 -120 -120

-140 -140 -140

-160 -160 -160

-180 -180 -180

-200 -200 -200

150 200 250 300 350 150 200 250 300 350 150 200 250 300 350

Variations for Data Set6 along b1 Variations for Data Set6 along b1 Variations for Data Set6 along b1
-80 -80 -80
b4 b5 b6
b4- b5- b6-
-100 b4+ -100 b5+ -100 b6+

-120 -120 -120

-140 -140 -140

-160 -160 -160

-180 -180 -180

-200 -200 -200

150 200 250 300 350 150 200 250 300 350 150 200 250 300 350

(b)

Fig. 19. The plots of the shapes of ASM for PDM5 (a) and PDM6 (b), along the variations of the modes.
394 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Table 2
The summarized results of PCAs on the projective PDMs
PDM Mode1 Mode2 Mode3 Mode4
PDM1 Rotation along X Rotation along Y
PDM2 Translation Translation Scaling Rotation
along X along Y along Y
along X along Y (Reverse to PDM1)
PDM3 Skewing on Point 1
PDM4 Skewing on Point 2
PDM5 Skewing on Point 3
PDM6 Skewing on Point 4

4.4. Construction of the MSAM

The MASM (modular active shape model) is constructed with four stages, such as
concept, database, analysis, and selection of eigenvalues as shown in Fig. 20. In the
concept stage, designers make the concepts on the expected variations of target
shapes. In the database stage, the PDMs are individually generated on the basis of
each concept. In the analysis stage, the PCA is applied to analyze the PDMs, and
obtain the eigenvalues and the corresponding modes related with the variations of
data. Finally the designers will select the effective eigenvectors to construct the mod-
ular ASM.
Assume a contour C in which there are a set of node points, {Vi = (xi, yi)ji =
1, 2, . . . , n}. In this definition the number n is the total number of snaxels in the con-
tour. At the first frame the segmentation results by the conventional snake are as-
sumed to be good, and this assumption is somewhat natural because the

Concept Database Analysis Selection

Expected Projective
P 1 , P 2 ,…, P n
Shape 1 PDM 1

P
Expected Projective
P n+1 , P n+2 ,…, P m
Shape 2 PDM 2
C
……
……

……

Expected Projective
P o+1 , P o+2 ,…, P N
Shape n PDM n

Fig. 20. The construction of the MASM (modular active shape model).
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 395

initialization is an open problem in the applications of active contour models. Let the
initial segmentation results be XI as

XI ¼ ðxI1 ; y I1 ; xI2 ; y I2 ; . . . ; xIn ; y In ÞT ð37Þ

where (xIi, yIi) is the x–y coordinate in image plane of the ith snaxel, and n is the
number of total snaxels in the contour. Similarly, the output vector of a modular ac-
tive shape model (MASM), Xo, is defined as
T
Xo ¼ ðxo1 ; y o1 ; xo2 ; y o2 ; . . . ; xon ; y on Þ ð38Þ

where (xoi, yoi) is the x–y coordinate in image plane of the ith output node of the
modular ASM.
From these definitions a modular active shape model (MASM) can be constructed
as follows:
2 3
B1
6B 7
6 27
Xo ¼ XI þ ½P1 ; P2 ; . . . ; PN 6 7 ð39Þ
45
BN

where Pi is the ith eigenvector selected from the results of the PCA on the PDMs, Bi
is the ith mode corresponding to the Pi, and N is the effective mode number.

4.4.1. Affine transformation between model and current data

In the formulation of the PDMs a designerÕs concepts on the variations of the tar-
get shape can be expressed by rotating the projection plane about a few reference
axes, and this method make some individual PDMs depending on the selection of
reference rotating axes. On the database of these individual PDMs, some distinct
modes on the variations of the shape are obtained to design the modular active shape
model (MASM).
An affine transformation is a kind of linear mapping and can be expressed as a
(3 · 3) matrix as follows:
2 3
a11 a12 b1
6 7
Haffine ¼ 4 a21 a22 b2 5; ð40Þ
0 0 1

which maps a contour in the MASM to the image space. Given the two contours, Qa
and Qb,
2 3
x1 x2 . . . xn
6 7
Qa ¼ 4 y 1 y 2 . . . y n 5 ; ð41Þ
1 2 ... n
396 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402
2 3
u1 u2 . . . un
6 7
Q b ¼ 4 v1 v2 ... vn 5; ð42Þ
1 2 ... n
a linear affine transformation between these two contours is determined by minimiz-
ing the following cost function, e,
e ¼ kQb
Haffine  Qa k: ð43Þ
The solution of e is related with the pseudo inverse because the problem is over-
determined, and by using this fact the affine transformation is obtained as follows:

1
Haffine ¼ ½Qb  QTa  ½Qa  QTa : ð44Þ
Using this result a contour in MASM space, XMASM, is mapped to the aligned con-
tour, Xaffine
MASM in the image space as
2 3
a11 a12 b1
6 7
Xaffine
MASM ¼ 4 a21 a22 b2 5  XMASM : ð45Þ
0 0 1

4.4.2. Shape fitting of the MASM

For fitting the MASM to current boundaries of the target, a cost function, J, is
proposed as
J ¼ a  Eimage þ b  kB
Bprev k ð46Þ
where Eimage is the image energy, B and Bprev are (N · 1) vectors of MASM for the
current modes and for the modes in the previous step, respectively, and a and b are
the weighting factors.

5. Simulations

5.1. A planar object tracking

In this simulation, tracking of an artificial object is tried. It is natural to think that

the differences between two successive frames are very small because the operating
frequency of the vision system is considered to be substantially faster compared to
the motion of the target. The number of frames is 60, and there is a strong edge line
in the middle of the pictures, in which a face-like planar object moves from right to
left in the stream of frames. The tracking results are displayed in Fig. 21, and the
extraction shapes for some frames are plotted in Fig. 22. The invariant profiles are
obtained by using the results in [18]. By comparing these invariant profiles [16,17],
the quality for the corresponding relationship between snaxels are checked as in
Fig. 23. The tracking performance for the proposed case is much better than the con-
ventional method.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 397

Fig. 21. The tracking results with the comparison between the conventional snake and the proposed
model for simulation 1.

Conventional
Conventional
Motion of Shape for Conventional Snake

200
Y- a xi s

150
Y-Aix s

100

50

0
0 50 100
X-axis
150
X-axis
200 250 300

Proposed
Proposed
Motion of Shape for Proposed Model

200
Y- a xi s

150
Y-Aix s

100

50

0 150
0 50 100 200 250 300
X-axis X-axis

Fig. 22. The tracking shapes with the comparison between the conventional snake and the proposed
model for simulation 1.
398 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

Invariant Profile of conventional snake : Frame 2

0 .8

Frame 2
I n v a r ia n t V a lu e

Invariant value

0 .7
P r o fi l e

0 .6

0 .5

0 .4
0 5 10 15 20 25
Node Index Node Number
Invariant Profile : Frame 30
1 .5

Frame 30
In v a ria n t V a lu e

1
Invariant value
P r o fi l e

0 .5

-0 .5
0 5 10 15 20 25
Node Index Node Number
12
x 10 Invariant
I Profile : Frame 59
3

2
Frame 59
Invariant value
P r o fi l e

-1

-2
0 5 10
Node Index
15 20 Node Number 25

(a)

Invariant Profile of Proposed Model: Frame2

0. 8

Frame 2
Invariant value
P r o fi l e

0. 7

0. 6

0. 5

0. 4
0 5 10
Node Index
15 20 Node Number 2 5
Invariant Profile of Proposed Model: Frame2
0. 8

Frame 30
0. 7
Invariant value
P r o fi l e

0. 6

0. 5

0. 4
0 5 10 15 20
Node Index Node Number2 5
Invariant Profile of Proposed Model: Frame2
0. 8

Frame 59
0. 7
Invariant value
P r o fi l e

0. 6

0. 5

0. 4
0 5 10 15 20 25
Node Index Node Number
(b)
Fig. 23. The invariant profile for simulation 1. (a) Conventional snake; (b) Proposed model.

5.2. Feature tracking on a wall

In this simulation, a feature object tracking is tried on a wall. The number of

frames is 99, in the stream of frames the scene moves due to the motion of the cam-
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 399

Fig. 24. The tracking results with the comparison between the conventional snake and the proposed
model for simulation 3.

Conventional
Motion of S ha pe for Conventional S na ke

200
Y- a xi s

150
Y-A i x s

100

50

0
0 50 100 X-axis
150
X-a xis
200 250 300

Proposed
Motion of S ha pe for P ropos ed Model

200
Y- a xi s

150
Y-Aix s

100

50

0
150
0 50 100
X-axis X-a xis
200 250 300

Fig. 25. The tracking shapes with the comparison between the conventional snake and the proposed
model for simulation 3.
400 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

era. The tracking results are shown in Fig. 24, and the extraction shapes for some
frames are plotted in Fig. 25. The invariant profiles are obtained by using the results

Invariant Profile of conventional snake : Frame 2

0.9
Frame 2
Invariant Value

0.8
P r o fi l e

0.7

0.6

0.5

0.4 Node Number

0 5 10 Node Index 15 20 25

Invariant Profile : Frame 50

x 1 011
1
Frame 50
0.5
Invariant Value
P r o fi l e

-0.5

-1 Node Number
0 5 10 Node Index 15 20 25

Invariant Profile : Frame 98

1
Frame 99
0.8
Invariant Value
P r o fi l e

0.6

0.4

0.2

0
0 5 10 15 20 Node Number 25
Node Index

(a)

Invariant Profile of Proposed Model: Frame 2

0.8 Frame 2
0 .75
Invariant Value
P r o fi l e

0 .7
0 .65
0 .6
0 .55
0 .5
0 .45
0 5 10 15 20
Node Number 25
Node Index
Invariant Profile of Proposed Model: Frame 50

0.8 Frame 50
0 .75
Invariant Value
P r o fi l e

0 .7
0 .65
0 .6
0 .55
0 .5
0 .45 Node Number
0 5 10 15 20 25
Node Index
Invariant Profile of Proposed Model: Frame 98
0.8
0 .75
Frame 99
0 .7
Invariant Value
P r o fi l e

0 .65
0 .6
0 .55
0 .5
0 .45
0 5 10
Node Index
15 20 Node Number 25

(b)
Fig. 26. The invariant profile for simulation 3. (a) Conventional snake; (b) Proposed model.
 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402 401

in [18]. By comparing these invariant profiles, the quality for the corresponding rela-
tionship between snaxels are checked and shown in Fig. 26. The tracking perform-
ance for the proposed case is much better than the conventional method.

6. Conclusion

There are problems of tracking when using the active contour model, and they are
related with the illumination condition, the variations of edge condition, the sur-
rounding edges near targets and so on. At present more robust tracking model are
required to overcome these problems. The proposed tracking model of this paper
possess following important features.

• A shape-based or model-based tracking model is tried to overcome the surround-

ing edges, the noisy edges, and the drifts of snaxels on contours. This is more
robust than node-based model trackers.
• Due to the inclusion of ASM scheme, the corresponding relations of this model
between two contours in different frames are better compared to the conventional
model.
• The PDMs are generated in the consideration of the projective relationship, and
this method is reasonable because almost all planar rigid objects are under the
projective transformation.
• The designerÕs concept on the variations of shape are individually implemented in
each PDMs.
• The individually generated PDMs are integrated in the modular active shape
model (MASM) by selecting the effective eigenvectors of each PDM.
• As a result, the proposed MASM is a systematic approach to integrate the indi-
vidual ideas on the different expected variations of a target shape.
• Finally, it is natural that processing times and complexities might be increased due
to integration of ASM and ACM to the construction of the proposed MASM.
Therefore, the designers should consider these complexities in applying the
MASM to their tracking system.

References

[1] Kass M, Witkin A, Terzopoulos D. Snake: Active contour models. Int J Comput Vision 1988:321–31.
[2] Amini AA, Weymouth TE, Jain RC. Using dynamic programming for solving variational problems in
vision. IEEE Trans Pattern Anal Mach Intell 1990;12:855–67.
[3] Leymarie F, Levine MD. Tracking deformable objects in the plane using an active contour model.
IEEE Trans Pattern Anal Mach Intell 1993;15(6):617–34.
[4] Xu C, Prince JL. Snakes, shapes, and gradient vector flow. IEEE Trans Image Process
1998;7(3):359–69.
[5] Jain AK. Fundamentals of digital image processing. NJ: Prentice-Hall; 1989.
[6] Jain R, Kasturi R, Schunck BG. Machine vision. McGraw-Hill; 1995.
402 W. Kim, J.-J. Lee / Mechatronics 15 (2005) 371–402

[7] Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models—their training and application.
Comput Vision Image Understand 1995;61(1):38–59.
[8] Cootes TF, Edwards GJ, Taylor CJ. Active appearance models. Proc Euro Conf Comput Vision
1998;61(Jun):484–98.
[9] Hill A, Cootes TF, Taylor CJ. Active shape models and the shape approximation problems. Image
Vision Comput 1996;14:601–7.
[11] Blake A, Yuille A. Active vision. MIT-Press; 1992.
[12] Trucco E, Verri A. Introductory techniques for 3-D computer vision. Prentice Hall; 1998.
[13] Blake A, Isard M. Active contours. Springer; 1998.
[14] Ip H, Shen D. An affine-invariant active contour model (AI-Snake) for model-based segmentation.
Image Vision Comput 1998;16:135–46.
[15] Shen D, Davatzikos C. An adaptive-focus deformable model using statistical and geometric
information. IEEE Trans Pattern Anal Mach Intell 2000;22(8):906–13.
[16] Forsyth D, Mundy L, Zisserman A, Coelho C, Heller A, Rothwell C. Invariant descriptor for 3-D
object recognition and pose. IEEE Trans Pattern Anal Mach Intell 1991;13(10):971–91.
[17] Arbter K, Snyder WE, Burkhardt H, Hirzinger G. Application of affine-invariant Fourier descriptors
to recognition of 3-D objects. IEEE Trans Pattern Anal Mach Intell 1990;12(7):640–7.
[18] Roh KS, Kweon IS. 2-D object recognition using invariant contour descriptor and projective
refinement. Pattern Recog 1998;31(4):441–55.
[21] Matthews I, Cootes TF, Bangham JA, Cox S, Harvey R. Extraction of visual features for lipreading.
IEEE Trans Pattern Anal Mach Intell 2002;24(2):198–213.
[22] Metaxas DN, Kakadiaris IA. Elastically adaptive deformable models. IEEE Trans Pattern Anal
Mach Intell 2002;24(10):1310–21.