Skip to main content

Steve Beet

Followers

2

Following

3

Co-author

1

Public Views

Arnab Bhattacharya

IIT Kanpur

Alexander R Jensenius

University of Oslo

UNIKOM

Roshan Chitrakar

Nepal College of Information Technology

UPM - Universiti Putra Malaysia

Graduate Center of the City University of New York

George Christodoulides

Université de Mons

German Research Center for Artificial Intelligence

Dr Zulfiqar Habib

COMSATS Institute of Information Technology Lahore

Nidhal El Abbadi

University of Kufa

Interests

Uploads

Papers by Steve Beet

Adaptive flux interpolation, flow-based prediction, delta or delta-delta coefficients: which is best?

4th European Conference on Speech Communication and Technology (Eurospeech 1995)

Combining auditory representations using fuzzy sets

3rd International Conference on Spoken Language Processing (ICSLP 1994)

Adaptive flux interpolation, flow-based prediction, delta or delta-delta coefficients: which is best?

4th European Conference on Speech Communication and Technology (Eurospeech 1995)

Computationally efficient methods of calculating instantaneous frequency for auditory analysis

3rd European Conference on Speech Communication and Technology (Eurospeech 1993)

Combining auditory representations using fuzzy sets

3rd International Conference on Spoken Language Processing (ICSLP 1994)

Analysis of non-linear speech generating dynamics

3rd International Conference on Spoken Language Processing (ICSLP 1994)

... descending order. The embedding transformation of sk onto M, ek > mj, is achieved by proj... more

Computationally efficient methods of calculating instantaneous frequency for auditory analysis

3rd European Conference on Speech Communication and Technology (Eurospeech 1993)

Speech Production and Perception

SpringerBriefs in Electrical and Computer Engineering, 2013

'A periodic signal repeats a pattern at a regular interval or frequency. Aperiodic signals do not... more 'A periodic signal repeats a pattern at a regular interval or frequency. Aperiodic signals do not exhibit such patterns. Wind instruments produce periodic sounds, while percussion instruments are generally aperiodic. 21 dB, short for decibel, corresponds to ajust noticeable difference in sound energy. 'Hz, or Hertz, means cycles (or periods) per second. 1kHz, or one kiloHertz, is one thou sand cycles per second. 1 MHz, or MegaHertz, is one million cycles per second.

Analysis of non-linear speech generating dynamics

3rd International Conference on Spoken Language Processing (ICSLP 1994)

... descending order. The embedding transformation of sk onto M, ek > mj, is achieved by proj... more

Automatic segmentation: data-driven units of speech

5th European Conference on Speech Communication and Technology (Eurospeech 1997)

Published in Visual Representations of Speech Signals

The human auditory system has an amazing ability to separate and understand sounds. We believe th... more The human auditory system has an amazing ability to separate and understand sounds. We believe that temporal information plays a key role in this ability, more important than the spectral information that is traditionally emphasized in hearing science. In many hearing tasks, such as describing or classifying single sound sources, the underlying mathematical equivalence makes the temporal versus spectral argument moot. We show how the nonlinearity of the auditory system breaks this equivalence, and is especially important in analyzing complex sounds from multiple sources of different characteristics. The auditory system is inherently nonlinear. In a linear system, the component frequencies of a signal are unchanged, and it is easy to characterize the amplitude and phase changes caused by the system. The cochlea and the neural processing that follow are more interesting. The bandwidth of a cochlear "filter" changes at different sound levels, and neurons change their sensitivity as they adapt to sounds. Inner Hair Cells (IHC) produce nonlinear rectified versions of the sound, generating new frequencies such as envelope components. All of these changes make it difficult to describe auditory perception in terms of the spectrum or Fourier transform of a sound. One characteristic of an auditory signal that is undisturbed by most nonlinear transformations is the periodicity information in the signal. Even if the bandwidth, amplitude, and phase characteristics of a signal are changing, the repetitive characteristics do not. In addition, it is very unlikely that a periodic signal could come from more than one source. Thus the auditory system can safely assume that sound fragments with a consistent periodicity can be combined and assigned to a single source. Consider, for example, a sound formed by opening and closing the glottis four times and filtering the resulting puffs of air with the vocal resonances. After nonlinear processing the lower auditory nervous system will still detect four similar events which will be heard and integrated as coming from a voice. The duplex theory of pitch perception, proposed by Licklider in 1951 [11] as a unifying model of pitch perception, is even more useful as a model for the extraction and representation of temporal structure for both periodic and non-periodic signals. This theory produces a movie-like image of sound which is called a correlogram. We believe that the correlogram, like other representations that summarize the temporal information in a signal, is an important tool for understanding the auditory system. The correlogram represents sound as a three dimensional function of time, frequency, and periodicity. A cochlear model serves to transform a one dimensional acoustic pressure into a two dimensional map of neural firing rate as a function of time and place along the cochlea. A third dimension is added to the representation by measuring the periodicities in the output from the cochlear model. These three dimensions are shown in Fig. 1. While most of our own work has concentrated on the correlogram, the important message in this chapter is that time and periodicity cues should be an important part of an auditory representation. This chapter describes two cochlear models and explores a structure which we believe can be used to represent and interpret the temporal information in an acoustic signal. Section 2 of this chapter describes two nonlinear models of the cochlea we use in our work. These two models differ in their computational approach and are used to illustrate the robustness of the

VoiScan: Telephone Voice Analysis for Health and Biometric Applications

by Steve Beet and Ladan Baghai-Ravary

Speech and Computer, 2017

The telephone, whether mobile, landline, or VoIP, is probably the most widely used form of long-d... more The telephone, whether mobile, landline, or VoIP, is probably the most widely used form of long-distance communication. The most common use of voice biometrics is in telephone-based speaker verification, so the ability to operate effectively over the telephone is crucial. Similarly, access to vocal health monitoring, and other voice analysis technology, would benefit enormously if it were available over the telephone, via an automatic system. This paper describes a set of voice analysis algorithms, designed to be robust against the kinds of distortion and signal degradation encountered in modern telephone communication. The basis of the algorithms in traditional analysis is discussed, as are the design choices made in order to ensure robustness. The utility of these algorithms is demonstrated in a number of target domains.

Statistical LIP modelling for visual speech recognition

1996 8th European Signal Processing Conference (EUSIPCO 1996), 1996

We describe a speechreading (lipreading) system purely based on visual features extracted from gr... more We describe a speechreading (lipreading) system purely based on visual features extracted from grey level image sequences of the speaker's lips. Active shape models are used to track the lip contours while visual speech information is extracted from the shape of the contours. The distribution and temporal dependencies of the shape features are modelled by continuous density Hidden Markov Models. Experiments are reported for speaker independent recognition tests of isolated digits. The analysis of individual feature components suggests that speech relevant information is embedded in a low dimensional space and fairly robust to inter- and intra-speaker variability.

Recognition of phonemes from estimation errors

1996 8th European Signal Processing Conference (EUSIPCO 1996), 1996

Speech recognition systems generally use delta and delta-delta (velocity and acceleration) coeffi... more Speech recognition systems generally use delta and delta-delta (velocity and acceleration) coefficients to characterise the dynamics apparent in frame-based representations of speech. These coefficients can be thought of as the errors of simple predictors. This paper describes the use of error coefficients derived from more advanced (and accurate) forms of prediction and interpolation. Both overall recognition accuracy and the detailed confusions observed are compared with those of the ‘traditional’ methods. The task used is speaker-independent phoneme recognition using a subset of the TIMIT database, and four different speech representations. The error coefficient performance on this task appears to be directly related to the robustness of the estimator used, with the best of the new methods out-performing delta-delta coefficients by around 10%.

Improved speech recognition using a reduced auditory representation

ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing

ABSTRACT The information produced by a straightforward auditory model, like the original speech s... more ABSTRACT The information produced by a straightforward auditory model, like the original speech signal, is subject to extreme local variability and is not suitable for direct presentation to speech recognition algorithms. Therefore, an analysis is required which will make the important features of the neural firing patterns explicit, but which will describe them concisely and without disruption by known characteristics of speech. Several attempts have been made to develop such an analysis, however, such analyses ignore significant aspects of neural firing patterns, and consequently information is lost (usually that associated with fine temporal structure). The authors describe a `reduced auditory representation&#39;, which retains more information, and presents the results of preliminary recognition experiments using this data

Speachreading using shape and intensity information

Proceedings of the 4th International Conference on Spoken Language Processing, 1996

We describe a speechreading system that uses both, shape information from the lip contours and in... more We describe a speechreading system that uses both, shape information from the lip contours and intensity information from the mouth area. Shape information is obtained by tracking and parameterising the inner and outer lip boundary in an image sequence. Intensity information is extracted from a grey level model, based on principal component analysis. In comparison to other approaches, the intensity area deforms with the shape model to ensure that similar object features are represented after non-rigid deformation of the lips. We describe speaker independent recognition experiments based on these features and Hidden Markov Models. Preliminary results suggest that similar performance can be achieved by using either shape or intensity information and slightly higher performance by their combined use.

Visual speech recognition using active shape models and hidden Markov models

1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings

This paper describes a novel approach for visual speech recognition. The shape of the mouth is mo... more This paper describes a novel approach for visual speech recognition. The shape of the mouth is modelled by an Active Shape Model which is derived from the statistics of a training set and used to locate, track and parameterise the speaker's lip movements. The extracted parameters representing the lip shape are modelled as continuous probability distributions and their temporal dependencies are modelled by Hidden Markov Models. We present recognition tests performed on a database of a broad variety of speakers and illumination conditions. The system achieved an accuracy of 85.42 % for a speaker independent recognition task of the first four digits using lip shape information only.

Locating and tracking facial speech features

Proceedings of 13th International Conference on Pattern Recognition, 1996

This paper describes a robust method for extracting visual speech information from the shape of l... more This paper describes a robust method for extracting visual speech information from the shape of lips to be used for an automatic speechreading (lipreading) systems. Lip deformation is modelled by a statistically based deformable contour model which learns typical lip deformation from a training set. The main difficulty in locating and tracking lips consists of finding dominant image features for representing the lip contours. We describe the use of a statistical profile model which learns dominant image features from a training set. The model captures global intensity variation due to different illumination and different skin reflectance as well as intensity changes at the inner lip contour due to mouth opening and visibility of teeth and tongue. The method is validated for locating and tracking lip movements on a database of a broad variety of speakers.

Speechreading using shape and intensity information

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

We describe a speechreading system that uses both, shape information from the lip contours and in... more We describe a speechreading system that uses both, shape information from the lip contours and intensity information from the mouth area. Shape information is obtained by tracking and parameterising the inner and outer lip boundary in an image sequence. Intensity information is extracted from a grey level model, based on principal component analysis. In comparison to other approaches, the intensity area deforms with the shape model to ensure that similar object features are represented after non-rigid deformation of the lips. We describe speaker independent recognition experiments based on these features and Hidden Markov Models. Preliminary results suggest that similar performance can be achieved by using either shape or intensity information and slightly higher performance by their combined use.

Speaker identification by lipreading

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

This paper describes a new approach for speaker identification based on lipreading. Visual featur... more This paper describes a new approach for speaker identification based on lipreading. Visual features are extracted from image sequences of the talking face and consist of shape parameters which describe the lip boundary and intensity parameters which describe the grey-level distribution of the mouth area. Intensity information is based on principal component analysis using eigenspaces which deform with the shape model. The extracted parameters account for both, speech dependent and speaker dependent information. We built spatio-temporal speaker models based on these features, using HMMs with mixtures of Gaussians. Promising results were obtained for text dependent and text independent speaker identification tests performed on a small video database.

Adaptive flux interpolation, flow-based prediction, delta or delta-delta coefficients: which is best?

4th European Conference on Speech Communication and Technology (Eurospeech 1995)

Combining auditory representations using fuzzy sets

3rd International Conference on Spoken Language Processing (ICSLP 1994)

Adaptive flux interpolation, flow-based prediction, delta or delta-delta coefficients: which is best?

4th European Conference on Speech Communication and Technology (Eurospeech 1995)

Computationally efficient methods of calculating instantaneous frequency for auditory analysis

3rd European Conference on Speech Communication and Technology (Eurospeech 1993)

Combining auditory representations using fuzzy sets

3rd International Conference on Spoken Language Processing (ICSLP 1994)

Analysis of non-linear speech generating dynamics

3rd International Conference on Spoken Language Processing (ICSLP 1994)

... descending order. The embedding transformation of sk onto M, ek > mj, is achieved by proj... more

Computationally efficient methods of calculating instantaneous frequency for auditory analysis

3rd European Conference on Speech Communication and Technology (Eurospeech 1993)

Speech Production and Perception

SpringerBriefs in Electrical and Computer Engineering, 2013

'A periodic signal repeats a pattern at a regular interval or frequency. Aperiodic signals do not... more 'A periodic signal repeats a pattern at a regular interval or frequency. Aperiodic signals do not exhibit such patterns. Wind instruments produce periodic sounds, while percussion instruments are generally aperiodic. 21 dB, short for decibel, corresponds to ajust noticeable difference in sound energy. 'Hz, or Hertz, means cycles (or periods) per second. 1kHz, or one kiloHertz, is one thou sand cycles per second. 1 MHz, or MegaHertz, is one million cycles per second.

Analysis of non-linear speech generating dynamics

3rd International Conference on Spoken Language Processing (ICSLP 1994)

... descending order. The embedding transformation of sk onto M, ek > mj, is achieved by proj... more

Automatic segmentation: data-driven units of speech

5th European Conference on Speech Communication and Technology (Eurospeech 1997)

Published in Visual Representations of Speech Signals

The human auditory system has an amazing ability to separate and understand sounds. We believe th... more The human auditory system has an amazing ability to separate and understand sounds. We believe that temporal information plays a key role in this ability, more important than the spectral information that is traditionally emphasized in hearing science. In many hearing tasks, such as describing or classifying single sound sources, the underlying mathematical equivalence makes the temporal versus spectral argument moot. We show how the nonlinearity of the auditory system breaks this equivalence, and is especially important in analyzing complex sounds from multiple sources of different characteristics. The auditory system is inherently nonlinear. In a linear system, the component frequencies of a signal are unchanged, and it is easy to characterize the amplitude and phase changes caused by the system. The cochlea and the neural processing that follow are more interesting. The bandwidth of a cochlear "filter" changes at different sound levels, and neurons change their sensitivity as they adapt to sounds. Inner Hair Cells (IHC) produce nonlinear rectified versions of the sound, generating new frequencies such as envelope components. All of these changes make it difficult to describe auditory perception in terms of the spectrum or Fourier transform of a sound. One characteristic of an auditory signal that is undisturbed by most nonlinear transformations is the periodicity information in the signal. Even if the bandwidth, amplitude, and phase characteristics of a signal are changing, the repetitive characteristics do not. In addition, it is very unlikely that a periodic signal could come from more than one source. Thus the auditory system can safely assume that sound fragments with a consistent periodicity can be combined and assigned to a single source. Consider, for example, a sound formed by opening and closing the glottis four times and filtering the resulting puffs of air with the vocal resonances. After nonlinear processing the lower auditory nervous system will still detect four similar events which will be heard and integrated as coming from a voice. The duplex theory of pitch perception, proposed by Licklider in 1951 [11] as a unifying model of pitch perception, is even more useful as a model for the extraction and representation of temporal structure for both periodic and non-periodic signals. This theory produces a movie-like image of sound which is called a correlogram. We believe that the correlogram, like other representations that summarize the temporal information in a signal, is an important tool for understanding the auditory system. The correlogram represents sound as a three dimensional function of time, frequency, and periodicity. A cochlear model serves to transform a one dimensional acoustic pressure into a two dimensional map of neural firing rate as a function of time and place along the cochlea. A third dimension is added to the representation by measuring the periodicities in the output from the cochlear model. These three dimensions are shown in Fig. 1. While most of our own work has concentrated on the correlogram, the important message in this chapter is that time and periodicity cues should be an important part of an auditory representation. This chapter describes two cochlear models and explores a structure which we believe can be used to represent and interpret the temporal information in an acoustic signal. Section 2 of this chapter describes two nonlinear models of the cochlea we use in our work. These two models differ in their computational approach and are used to illustrate the robustness of the

VoiScan: Telephone Voice Analysis for Health and Biometric Applications

by Steve Beet and Ladan Baghai-Ravary

Speech and Computer, 2017

The telephone, whether mobile, landline, or VoIP, is probably the most widely used form of long-d... more The telephone, whether mobile, landline, or VoIP, is probably the most widely used form of long-distance communication. The most common use of voice biometrics is in telephone-based speaker verification, so the ability to operate effectively over the telephone is crucial. Similarly, access to vocal health monitoring, and other voice analysis technology, would benefit enormously if it were available over the telephone, via an automatic system. This paper describes a set of voice analysis algorithms, designed to be robust against the kinds of distortion and signal degradation encountered in modern telephone communication. The basis of the algorithms in traditional analysis is discussed, as are the design choices made in order to ensure robustness. The utility of these algorithms is demonstrated in a number of target domains.

Statistical LIP modelling for visual speech recognition

1996 8th European Signal Processing Conference (EUSIPCO 1996), 1996

We describe a speechreading (lipreading) system purely based on visual features extracted from gr... more We describe a speechreading (lipreading) system purely based on visual features extracted from grey level image sequences of the speaker's lips. Active shape models are used to track the lip contours while visual speech information is extracted from the shape of the contours. The distribution and temporal dependencies of the shape features are modelled by continuous density Hidden Markov Models. Experiments are reported for speaker independent recognition tests of isolated digits. The analysis of individual feature components suggests that speech relevant information is embedded in a low dimensional space and fairly robust to inter- and intra-speaker variability.

Recognition of phonemes from estimation errors

1996 8th European Signal Processing Conference (EUSIPCO 1996), 1996

Speech recognition systems generally use delta and delta-delta (velocity and acceleration) coeffi... more Speech recognition systems generally use delta and delta-delta (velocity and acceleration) coefficients to characterise the dynamics apparent in frame-based representations of speech. These coefficients can be thought of as the errors of simple predictors. This paper describes the use of error coefficients derived from more advanced (and accurate) forms of prediction and interpolation. Both overall recognition accuracy and the detailed confusions observed are compared with those of the ‘traditional’ methods. The task used is speaker-independent phoneme recognition using a subset of the TIMIT database, and four different speech representations. The error coefficient performance on this task appears to be directly related to the robustness of the estimator used, with the best of the new methods out-performing delta-delta coefficients by around 10%.

Improved speech recognition using a reduced auditory representation

ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing

ABSTRACT The information produced by a straightforward auditory model, like the original speech s... more ABSTRACT The information produced by a straightforward auditory model, like the original speech signal, is subject to extreme local variability and is not suitable for direct presentation to speech recognition algorithms. Therefore, an analysis is required which will make the important features of the neural firing patterns explicit, but which will describe them concisely and without disruption by known characteristics of speech. Several attempts have been made to develop such an analysis, however, such analyses ignore significant aspects of neural firing patterns, and consequently information is lost (usually that associated with fine temporal structure). The authors describe a `reduced auditory representation&#39;, which retains more information, and presents the results of preliminary recognition experiments using this data

Speachreading using shape and intensity information

Proceedings of the 4th International Conference on Spoken Language Processing, 1996

We describe a speechreading system that uses both, shape information from the lip contours and in... more We describe a speechreading system that uses both, shape information from the lip contours and intensity information from the mouth area. Shape information is obtained by tracking and parameterising the inner and outer lip boundary in an image sequence. Intensity information is extracted from a grey level model, based on principal component analysis. In comparison to other approaches, the intensity area deforms with the shape model to ensure that similar object features are represented after non-rigid deformation of the lips. We describe speaker independent recognition experiments based on these features and Hidden Markov Models. Preliminary results suggest that similar performance can be achieved by using either shape or intensity information and slightly higher performance by their combined use.

Visual speech recognition using active shape models and hidden Markov models

1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings

This paper describes a novel approach for visual speech recognition. The shape of the mouth is mo... more This paper describes a novel approach for visual speech recognition. The shape of the mouth is modelled by an Active Shape Model which is derived from the statistics of a training set and used to locate, track and parameterise the speaker's lip movements. The extracted parameters representing the lip shape are modelled as continuous probability distributions and their temporal dependencies are modelled by Hidden Markov Models. We present recognition tests performed on a database of a broad variety of speakers and illumination conditions. The system achieved an accuracy of 85.42 % for a speaker independent recognition task of the first four digits using lip shape information only.

Locating and tracking facial speech features

Proceedings of 13th International Conference on Pattern Recognition, 1996

This paper describes a robust method for extracting visual speech information from the shape of l... more This paper describes a robust method for extracting visual speech information from the shape of lips to be used for an automatic speechreading (lipreading) systems. Lip deformation is modelled by a statistically based deformable contour model which learns typical lip deformation from a training set. The main difficulty in locating and tracking lips consists of finding dominant image features for representing the lip contours. We describe the use of a statistical profile model which learns dominant image features from a training set. The model captures global intensity variation due to different illumination and different skin reflectance as well as intensity changes at the inner lip contour due to mouth opening and visibility of teeth and tongue. The method is validated for locating and tracking lip movements on a database of a broad variety of speakers.

Speechreading using shape and intensity information

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

We describe a speechreading system that uses both, shape information from the lip contours and in... more We describe a speechreading system that uses both, shape information from the lip contours and intensity information from the mouth area. Shape information is obtained by tracking and parameterising the inner and outer lip boundary in an image sequence. Intensity information is extracted from a grey level model, based on principal component analysis. In comparison to other approaches, the intensity area deforms with the shape model to ensure that similar object features are represented after non-rigid deformation of the lips. We describe speaker independent recognition experiments based on these features and Hidden Markov Models. Preliminary results suggest that similar performance can be achieved by using either shape or intensity information and slightly higher performance by their combined use.

Speaker identification by lipreading

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

This paper describes a new approach for speaker identification based on lipreading. Visual featur... more This paper describes a new approach for speaker identification based on lipreading. Visual features are extracted from image sequences of the talking face and consist of shape parameters which describe the lip boundary and intensity parameters which describe the grey-level distribution of the mouth area. Intensity information is based on principal component analysis using eigenspaces which deform with the shape model. The extracted parameters account for both, speech dependent and speaker dependent information. We built spatio-temporal speaker models based on these features, using HMMs with mixtures of Gaussians. Promising results were obtained for text dependent and text independent speaker identification tests performed on a small video database.