IEEE Transactions on Audio, Speech, and Language Processing, 2000
In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likeli... more In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likelihood evaluation algorithm for graphical processing units (GPUs). The evaluation of these likelihoods is one of the most computationally intensive parts of automatic speech recognizers, but it can be parallelized and offloaded to GPU devices. Our approach offers a significant speed-up over the recently published approaches, because it utilizes the GPU architecture in a more effective manner. All the recent implementations have been intended only for NVIDIA graphics processors, programmed either in CUDA or OpenCL GPU programming frameworks. We present results for both CUDA and OpenCL. Further, we have developed an OpenCL implementation optimized for ATI/AMD GPUs. Results suggest that even very large acoustic models can be used in real-time speech recognition engines on computers equipped with a lowend GPU or laptops. In addition, the completely asynchronous GPU management provides additional CPU resources for the decoder part of the LVCSR. The optimized implementation enables us to apply fusion techniques together with evaluating many (10 or even more) speaker-specific acoustic models. We apply this technique to a real-time parliamentary speech recognition system where the speaker changes frequently.
In the paper we introduce the on-line captioning system developed by our teams and used by the Cz... more In the paper we introduce the on-line captioning system developed by our teams and used by the Czech Television (CTV), the public service broadcaster in the Czech Republic. The research project is targeted at incorporation of speech technologies into the CTV environment. One of the key missions is the development of captioning system supporting captioning of a “live” acoustic track. It can be either the real audio stream or the audio stream produced by a shadow speaker. Another key mission is to develop software tools and techniques usable for training the shadow speakers. During the initial phases of the project we concluded that the broadcasting of the Parliamentary meetings of the Chamber of Deputies fulfills the necessary conditions that enable it to be captioned without the aid of the shadow speaker. We developed a fully automatic captioning pilot system making the broadcasting of Parliamentary meetings of the Chamber of Deputies accessible to the hearing impaired viewers. The pilot run enabled us and our partners in the Czech TV to develop and evaluate the complete captioning infrastructure and collect, review and possibly implement opinions and suggestions of the targeted audience. This paper presents our experience gathered during first years of the project to the public audience.
IEEE Transactions on Audio, Speech & Language Processing, 2012
In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likeli... more In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likelihood evaluation algorithm for graphical processing units (GPUs). The evaluation of these likelihoods is one of the most computationally intensive parts of automatic speech recognizers, but it can be parallelized and offloaded to GPU devices. Our approach offers a significant speed-up over the recently published approaches, because it utilizes the GPU architecture in a more effective manner. All the recent implementations have been intended only for NVIDIA graphics processors, programmed either in CUDA or OpenCL GPU programming frameworks. We present results for both CUDA and OpenCL. Further, we have developed an OpenCL implementation optimized for ATI/AMD GPUs. Results suggest that even very large acoustic models can be used in real-time speech recognition engines on computers equipped with a lowend GPU or laptops. In addition, the completely asynchronous GPU management provides additional CPU resources for the decoder part of the LVCSR. The optimized implementation enables us to apply fusion techniques together with evaluating many (10 or even more) speaker-specific acoustic models. We apply this technique to a real-time parliamentary speech recognition system where the speaker changes frequently.
This paper presents initial experiments with the identification and automatic detection of parasi... more This paper presents initial experiments with the identification and automatic detection of parasitic sounds in speech signals. The main goal of this study is to identify such sounds in the source recordings for unit-selection-based speech synthesis systems and thus to avoid their unintended usage in synthesised speech. The first part of the paper describes the phonetic analysis and identification of parasitic phenomena in recordings of two Czech speakers. In the second part, experiments with the automatic detection of parasitic sounds using HMM-based and BVM classifiers are presented. The results are encouraging, especially those for glottalization phenomena.
In the paper we present two techniques improving the recognition accuracy of multilayer perceptro... more In the paper we present two techniques improving the recognition accuracy of multilayer perceptron neural networks (MLP ANN) by means of adopting Speaker Adaptive Training. The use of the MLP ANN, usually in combination with the TRAPS parametrization, includes applications in speech recognition tasks, discriminative features production for GMM-HMM and other. In the first SAT experiments, we used the VTLN as a speaker normalization technique. Moreover, we developed a novel speaker normalization technique called Minimum Error Linear Transform (MELT) that resembles the cMLLR/fMLLR method [1] with respect to the possible application either on the model or features.
In this paper we present three novel posterior transforms with the primary goal to achieve a high... more In this paper we present three novel posterior transforms with the primary goal to achieve a high reduction of a feature vector size. The presented methods transform the posteriors to 1 D or 2 D space. For such a high reduction ratio the usually applied methods fail to keep the discriminative information. Contrary, the presented methods were specifically designed to retain most of the discriminative information. In our experiments, we used several different combinations of feature extraction methods nowadays commonly used, i.e. the PLP features (augmented with delta and acceleration coefficients) and two kinds of MLP-ANN features: the bottleneck (BN) and posterior estimates (PE). The experiments were designed with special attention to the assessment of possible improvements of the performance when the PLP features are combined either with the BN features or with the PE features whose dimensionality was reduced using the proposed feature transforms. The performance of the designed transforms was tested on two different speech corpora: a telephone speech SpeechDat-East corpus and multimodal Czech Audio-Visual corpus.
In this paper we present a novel method for adaptation of a multi-layer perceptron neural network... more In this paper we present a novel method for adaptation of a multi-layer perceptron neural network (MLP ANN). Nowadays, the adaptation of the ANN is usually done as an incremental retraining either of a subset or the complete set of the ANN parameters. However, since sometimes the amount of the adaptation data is quite small, there is a fundamental drawback of such approach – during retraining, the network parameters can be easily overfitted to the new data. There certainly are techniques that can help overcome this problem (early-stopping, cross-validation), however application of such techniques leads to more complex and possibly more data hungry training procedure. The proposed method approaches the problem from a different perspective. We use the fact that in many cases we have an additional knowledge about the problem. Such additional knowledge can be used to limit the dimensionality of the adaptation problem. We applied the proposed method on speaker adaptation of a phoneme recognizer based on traps (Temporal Patterns) parameters. We exploited the fact that the employed traps parameters are constructed using log-outputs of mel-filter bank and by virtue of reformulating the first layer weight matrix adaptation problem as a mel-filter bank output adaptation problem, we were able to significantly limit the number of free variables. Adaptation using the proposed method resulted in a substantial improvement of phoneme recognizer accuracy.
The EuroMISE Center focuses on new approaches in the field of electronic health record (EHR). Amo... more The EuroMISE Center focuses on new approaches in the field of electronic health record (EHR). Among others, the structured health documentation in dentistry in the form of an EHR is being systematically studied. This paper describes the evolution of the EHR developed at the EuroMISE Center named MUDRLite and its graphical component for dentists called DentCross. The summary of features of the DentCross component is followed by a brief description of automatic speech recognition and an ASR module. The problems with data insertion into EHR during examination of a dental patient lead to further research in the area of the automatic speech recognition in medical practice. Cooperation of engineers, informaticians and dental physicians resulted in an application called DentVoice which is a successful application of the ASR module and the DentCross component of the MUDRLite EHR. The junction of voice control and graphical representation of dental arch makes hand-busy activities in dental praxis easier, quicker and more comfortable. This will result in a better quality of the data stored in a structured form in dental EHR, thus enabling better decision making and use of decision support systems.
In the paper we demonstrate a complex supervised learning method based on a binary decision graph... more In the paper we demonstrate a complex supervised learning method based on a binary decision graphs. This method is employed in construction of a silence/speech detector. Performance of the resulting silence/speech detector is compared with performance of common silence/speech detectors used in telecommunications and with a detector based on HMM and a bigram silence/speech language model. Each non-leaf node of a decision graph has assigned a question and a sub-classifier answering this question. We test three kinds of these sub-classifiers: linear classifier, classifier based on separating quadratic hyper-plane (SQHP), and Support Vector Machines (SVM) based classifier. Moreover, besides usage of a single decision graph we investigate application of a set of binary decision graphs.
In this paper we describe a HMM-based sign language recognition (SLR) system for isolated signs. ... more In this paper we describe a HMM-based sign language recognition (SLR) system for isolated signs. In the first part we describe the image parametrization method producing features used for recognition. Our goal was to find the best combination of a feature space dimension reduction method and an HMM structure.
This paper describes ANN based posterior estimates and their application to speech recognition. W... more This paper describes ANN based posterior estimates and their application to speech recognition. We replaced the standard back-propagation with the L-BFGS quasi-Newton method. We have focused only on posterior based feature vector extraction. Our goal was a feature vector dimension reduction. Thus we designed three posterior transforms to space with dimensionality 1 or 2. The designed transforms were tested on the SpeechDat-East corpus. We also applied the introduced method on a Czech audio-visual corpus. In both cases the methods leads to significant word error rate decrease.
IEEE Transactions on Audio, Speech, and Language Processing, 2000
In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likeli... more In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likelihood evaluation algorithm for graphical processing units (GPUs). The evaluation of these likelihoods is one of the most computationally intensive parts of automatic speech recognizers, but it can be parallelized and offloaded to GPU devices. Our approach offers a significant speed-up over the recently published approaches, because it utilizes the GPU architecture in a more effective manner. All the recent implementations have been intended only for NVIDIA graphics processors, programmed either in CUDA or OpenCL GPU programming frameworks. We present results for both CUDA and OpenCL. Further, we have developed an OpenCL implementation optimized for ATI/AMD GPUs. Results suggest that even very large acoustic models can be used in real-time speech recognition engines on computers equipped with a lowend GPU or laptops. In addition, the completely asynchronous GPU management provides additional CPU resources for the decoder part of the LVCSR. The optimized implementation enables us to apply fusion techniques together with evaluating many (10 or even more) speaker-specific acoustic models. We apply this technique to a real-time parliamentary speech recognition system where the speaker changes frequently.
In the paper we introduce the on-line captioning system developed by our teams and used by the Cz... more In the paper we introduce the on-line captioning system developed by our teams and used by the Czech Television (CTV), the public service broadcaster in the Czech Republic. The research project is targeted at incorporation of speech technologies into the CTV environment. One of the key missions is the development of captioning system supporting captioning of a “live” acoustic track. It can be either the real audio stream or the audio stream produced by a shadow speaker. Another key mission is to develop software tools and techniques usable for training the shadow speakers. During the initial phases of the project we concluded that the broadcasting of the Parliamentary meetings of the Chamber of Deputies fulfills the necessary conditions that enable it to be captioned without the aid of the shadow speaker. We developed a fully automatic captioning pilot system making the broadcasting of Parliamentary meetings of the Chamber of Deputies accessible to the hearing impaired viewers. The pilot run enabled us and our partners in the Czech TV to develop and evaluate the complete captioning infrastructure and collect, review and possibly implement opinions and suggestions of the targeted audience. This paper presents our experience gathered during first years of the project to the public audience.
IEEE Transactions on Audio, Speech & Language Processing, 2012
In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likeli... more In this paper, we describe an optimized version of a Gaussian-mixture-based acoustic model likelihood evaluation algorithm for graphical processing units (GPUs). The evaluation of these likelihoods is one of the most computationally intensive parts of automatic speech recognizers, but it can be parallelized and offloaded to GPU devices. Our approach offers a significant speed-up over the recently published approaches, because it utilizes the GPU architecture in a more effective manner. All the recent implementations have been intended only for NVIDIA graphics processors, programmed either in CUDA or OpenCL GPU programming frameworks. We present results for both CUDA and OpenCL. Further, we have developed an OpenCL implementation optimized for ATI/AMD GPUs. Results suggest that even very large acoustic models can be used in real-time speech recognition engines on computers equipped with a lowend GPU or laptops. In addition, the completely asynchronous GPU management provides additional CPU resources for the decoder part of the LVCSR. The optimized implementation enables us to apply fusion techniques together with evaluating many (10 or even more) speaker-specific acoustic models. We apply this technique to a real-time parliamentary speech recognition system where the speaker changes frequently.
This paper presents initial experiments with the identification and automatic detection of parasi... more This paper presents initial experiments with the identification and automatic detection of parasitic sounds in speech signals. The main goal of this study is to identify such sounds in the source recordings for unit-selection-based speech synthesis systems and thus to avoid their unintended usage in synthesised speech. The first part of the paper describes the phonetic analysis and identification of parasitic phenomena in recordings of two Czech speakers. In the second part, experiments with the automatic detection of parasitic sounds using HMM-based and BVM classifiers are presented. The results are encouraging, especially those for glottalization phenomena.
In the paper we present two techniques improving the recognition accuracy of multilayer perceptro... more In the paper we present two techniques improving the recognition accuracy of multilayer perceptron neural networks (MLP ANN) by means of adopting Speaker Adaptive Training. The use of the MLP ANN, usually in combination with the TRAPS parametrization, includes applications in speech recognition tasks, discriminative features production for GMM-HMM and other. In the first SAT experiments, we used the VTLN as a speaker normalization technique. Moreover, we developed a novel speaker normalization technique called Minimum Error Linear Transform (MELT) that resembles the cMLLR/fMLLR method [1] with respect to the possible application either on the model or features.
In this paper we present three novel posterior transforms with the primary goal to achieve a high... more In this paper we present three novel posterior transforms with the primary goal to achieve a high reduction of a feature vector size. The presented methods transform the posteriors to 1 D or 2 D space. For such a high reduction ratio the usually applied methods fail to keep the discriminative information. Contrary, the presented methods were specifically designed to retain most of the discriminative information. In our experiments, we used several different combinations of feature extraction methods nowadays commonly used, i.e. the PLP features (augmented with delta and acceleration coefficients) and two kinds of MLP-ANN features: the bottleneck (BN) and posterior estimates (PE). The experiments were designed with special attention to the assessment of possible improvements of the performance when the PLP features are combined either with the BN features or with the PE features whose dimensionality was reduced using the proposed feature transforms. The performance of the designed transforms was tested on two different speech corpora: a telephone speech SpeechDat-East corpus and multimodal Czech Audio-Visual corpus.
In this paper we present a novel method for adaptation of a multi-layer perceptron neural network... more In this paper we present a novel method for adaptation of a multi-layer perceptron neural network (MLP ANN). Nowadays, the adaptation of the ANN is usually done as an incremental retraining either of a subset or the complete set of the ANN parameters. However, since sometimes the amount of the adaptation data is quite small, there is a fundamental drawback of such approach – during retraining, the network parameters can be easily overfitted to the new data. There certainly are techniques that can help overcome this problem (early-stopping, cross-validation), however application of such techniques leads to more complex and possibly more data hungry training procedure. The proposed method approaches the problem from a different perspective. We use the fact that in many cases we have an additional knowledge about the problem. Such additional knowledge can be used to limit the dimensionality of the adaptation problem. We applied the proposed method on speaker adaptation of a phoneme recognizer based on traps (Temporal Patterns) parameters. We exploited the fact that the employed traps parameters are constructed using log-outputs of mel-filter bank and by virtue of reformulating the first layer weight matrix adaptation problem as a mel-filter bank output adaptation problem, we were able to significantly limit the number of free variables. Adaptation using the proposed method resulted in a substantial improvement of phoneme recognizer accuracy.
The EuroMISE Center focuses on new approaches in the field of electronic health record (EHR). Amo... more The EuroMISE Center focuses on new approaches in the field of electronic health record (EHR). Among others, the structured health documentation in dentistry in the form of an EHR is being systematically studied. This paper describes the evolution of the EHR developed at the EuroMISE Center named MUDRLite and its graphical component for dentists called DentCross. The summary of features of the DentCross component is followed by a brief description of automatic speech recognition and an ASR module. The problems with data insertion into EHR during examination of a dental patient lead to further research in the area of the automatic speech recognition in medical practice. Cooperation of engineers, informaticians and dental physicians resulted in an application called DentVoice which is a successful application of the ASR module and the DentCross component of the MUDRLite EHR. The junction of voice control and graphical representation of dental arch makes hand-busy activities in dental praxis easier, quicker and more comfortable. This will result in a better quality of the data stored in a structured form in dental EHR, thus enabling better decision making and use of decision support systems.
In the paper we demonstrate a complex supervised learning method based on a binary decision graph... more In the paper we demonstrate a complex supervised learning method based on a binary decision graphs. This method is employed in construction of a silence/speech detector. Performance of the resulting silence/speech detector is compared with performance of common silence/speech detectors used in telecommunications and with a detector based on HMM and a bigram silence/speech language model. Each non-leaf node of a decision graph has assigned a question and a sub-classifier answering this question. We test three kinds of these sub-classifiers: linear classifier, classifier based on separating quadratic hyper-plane (SQHP), and Support Vector Machines (SVM) based classifier. Moreover, besides usage of a single decision graph we investigate application of a set of binary decision graphs.
In this paper we describe a HMM-based sign language recognition (SLR) system for isolated signs. ... more In this paper we describe a HMM-based sign language recognition (SLR) system for isolated signs. In the first part we describe the image parametrization method producing features used for recognition. Our goal was to find the best combination of a feature space dimension reduction method and an HMM structure.
This paper describes ANN based posterior estimates and their application to speech recognition. W... more This paper describes ANN based posterior estimates and their application to speech recognition. We replaced the standard back-propagation with the L-BFGS quasi-Newton method. We have focused only on posterior based feature vector extraction. Our goal was a feature vector dimension reduction. Thus we designed three posterior transforms to space with dimensionality 1 or 2. The designed transforms were tested on the SpeechDat-East corpus. We also applied the introduced method on a Czech audio-visual corpus. In both cases the methods leads to significant word error rate decrease.
Uploads
Papers by Jan Trmal