Data Driven Model Acquisition using Minimum Description Length

2001

An approach is presented to automatically segment and label a continuous observation sequence of hand gestures for a complete unsupervised model acquisition. The method is based on the assumption that gestures can be viewed as repetitive sequences of atomic components, similar to phonemes in speech, starting and ending in a rest position and governed by a high level structure controlling the temporal sequence. It is shown that the generating processes for the atomic components and derived gesture models can be described by a mixture of Gaussian in their respective component and gesture space. Mixture components modelling atomic components and gestures respectively are determined using a standard EM approach, while the determination of the number of mixture components and therefore the number of atomic components and gestures is based on an information criterion, the Minimum Description Length (MDL).

Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001

gesture space by a mixture of Gaussian, with each mixture component tied to one atomic behavioul: Mixture components are determined using a standard EM approach while the determination of the number of components is based on an information criteria, the Minimum Description Length.

2019

Sequence modeling for signs and gestures is an open research problem. In that direction, there is a sustained effort towards modeling signs and gestures as a sequence of subunits. In this paper, we develop a novel approach to infer movement subunits in a data-driven manner to model signs and gestures in the framework of hidden Markov models (HMM) given the skeleton information. This approach involves: (a) representation of position and movement information with measurement of hand positions relative to body parts (head, shoulders, hips); (b) modeling these features to infer a sign-specific left-to-right HMM; and (c) clustering the HMM states to infer states or subunits that are shared across signs and updating the HMM topology of signs. We investigate the application of the proposed approach on sign and gesture recognition tasks, specifically on Turkish signs HospiSign database and Italian gestures Chalearn 2014 task. On both databases, our studies show that, while yielding competit...

2006 International Conference on Image Processing, 2006

Hidden Markov models using the Fully-Connected, Left-Right and Left-Right Banded model structures are applied to the problem of alphabetical letter gesture recognition. We examine the effect of training techniques, in particular the Baum-Welch and Viterbi Path Counting techniques, on each of the model structures. We show that recognition rates improve when moving from a Fully-Connected model to a Left-Right model and a Left-Right Banded 'staircase' model with peak recognition rates of 84.8%, 92.31% and 97.31% respectively. The Left-Right Banded model in conjunction with the Viterbi Path Counting present the best performance. Direct calculation of model parameters from analysis of the physical system was also tested, yielding a peak recognition rate of 92%, but the simplicity and efficiency of this approach is of interest.

2010

We investigate the automatic phonetic modeling of sign language based on phonetic sub-units, which are data driven and without any prior phonetic information. Visual processing is based on a probabilistic skin color model and a framewise geodesic active contour segmentation; occlusions are handled by a forward-backward prediction component leading finally to simple and effective region-based visual features. For sign-language modeling we propose a modeling structure for data-driven sub-unit construction. This utilizes the cue that is considered crucial to segment the signal into parts; at the same time we also classify the segments by implicitly assigning labels of Dynamic or Static type. This segmentation and classification step disentangles Dynamic from Static parts and allows us to employ for each type of segment the appropriate cue, modeling and clustering approach. The constructed Dynamic segments are exploited at the model level via hidden Markov models (HMMs). The Static segments are exploited via k-means clustering. Each Dynamic or Static part, exploits the appropriate cue related to the movement. We propose that the movement cues are normalized in order to be translation and scale invariant. We apply the proposed modeling for further combination of the movement trajectory individual cues. The proposed approaches are evaluated in recognition experiments conducted on the continuous sign language corpus of Boston University (BU-400) showing promising preliminary results.

Ninth International Workshop on Frontiers in Handwriting Recognition, 2004

Hidden Markov models using the Fully-Connected, Left-Right and Left-Right Banded model structures are applied to the problem of alphabetical letter gesture recognition. We examine the effect of training techniques, in particular the Baum-Welch and Viterbi Path Counting techniques, on each of the model structures. We show that recognition rates improve when moving from a Fully-Connected model to a Left-Right model and a Left-Right Banded 'staircase' model with peak recognition rates of 84.8%, 92.31% and 97.31% respectively. The Left-Right Banded model in conjunction with the Viterbi Path Counting present the best performance. Direct calculation of model parameters from analysis of the physical system was also tested, yielding a peak recognition rate of 92%, but the simplicity and efficiency of this approach is of interest.

Frontiers in Handwriting …, 2004

Hidden Markov models using the Fully-Connected, Left-Right and Left-Right Banded model structures are applied to the problem of alphabetical letter gesture recognition. We examine the effect of training techniques, in particular the Baum-Welch and Viterbi Path Counting techniques, on each of the model structures. We show that recognition rates improve when moving from a Fully-Connected model to a Left-Right model and a Left-Right Banded 'staircase' model with peak recognition rates of 84.8%, 92.31% and 97.31% respectively. The Left-Right Banded model in conjunction with the Viterbi Path Counting present the best performance. Direct calculation of model parameters from analysis of the physical system was also tested, yielding a peak recognition rate of 92%, but the simplicity and efficiency of this approach is of interest.

Hand gesture recognition from visual images has a nuniber of potential application in HCI (human coniputer interaction), machine vision, VR(virtua1 reality), machine control in the industryfield, and so on. Most conventional approaches to hand gesture recognition have employed datagloves. But, for more natural interface, hand gesture must be recognized from visual images as in the communication between humans withouf using any external devices. Our research is intended to draw and edit graphic elements by hand gesture. Up to now, many methods for hand gesture recognition have been proposed such as syntactical analysis, neural based approach, HMM (hidden Markov model) based recognition. As gesture is the continuous motion on the sequential time series, HMM must be a prominent recognition tool. Though each analysis method has me?fts and denierits, the most important thing in hand gesture recognition is what the input features are that represent very well the characteristics of moving hand gesture. In our research, we consider the planar hand gesture in front of camera and therefore 8-directional chain codes as input vectors. For training an HMM network, a simple context modeling nlethod is embedded as training on "left-to-right" HMM model. This model is applied to draw graphic elements such as triangle, rectangular, circle, arc, horizontal line, vertical line and edit the specified graphic elements such as copy, delete, move, swap, undo, close. Therefore, the overall objectives are 12 dynamic gestures. In our experiments, we have good recognition results on a pre-confined test environnient : 1) the spotting time is synchronized at the static state of a hand, 2) other limb parts except hands is motionless, 3) the change of hand posture during movement is meaningless. Our system will be advanced by adopting more diverse input features representing well dynamic features of hand gestures . 0-7803-4053-1/97/$10.00 e 1997 IEEE

Cognitive Technologies, 2006

Humans make often conscious and unconscious gestures, which reflect their mind, thoughts and the way these are formulated. These inherently complex processes can in general not be substituted by a corresponding verbal utterance that has the same semantics (McNeill, 1992). Gesture, which is a kind of body language, contains important information on the intention and the state of the gesture producer. Therefore, it is an important communication channels in human computer interaction. In the following we describe first the state of the art in gesture recognition. The next section describes the gesture interpretation module. After that we present the experiments and results for recognition of user states. We summarize our results in the last section. 1 State of the Art 1.1 Applications of Gesture Gesture can be used in a wide range of applications: gesture in conventional human computer interaction (HCI), interaction through linguistic gesture and manipulation through physical contact. We cover each of these in the following. Gesture in Conventional HCI Under the window, icon, menu and pointing device (WIMP) paradigm, the use of mouse and pen of a graphic tablet such as that of the Wacom 1 Company are typical example applications of gesture. This kind of gestures with the help of pointing devices is intensively employed in computer aided design (CAD) (Sachs, 1990) and online handwriting recognition (Buxton et al., 1985). In the literature this category of gestures is called pen-based gesture. Rubine introduced the GRANDMA system (Rubine, 1991), in which the user is allowed to define arbitrary gestures interactively. These user-defined gestures can be input either through a mouse or with the help of a pen. The system is able to learn the static and dynamic properties of the gestures on the basis of some training data and subsequently analyzes them in real time.