Defenses against security threats have been an interest of recent studies. Recent works have show... more Defenses against security threats have been an interest of recent studies. Recent works have shown that it is not difficult to attack a natural language processing (NLP) model while defending against them is still a cat-mouse game. Backdoor attacks are one such attack where a neural network is made to perform in a certain way on specific samples containing some triggers while achieving normal results on other samples. In this work, we present a few defense strategies that can be useful to counter against such an attack. We show that our defense methodologies significantly decrease the performance on the attacked inputs while maintaining similar performance on benign inputs. We also show that some of our defenses have very less runtime and also maintain similarity with the original inputs.
2018 2nd International Conference on Data Science and Business Analytics (ICDSBA), 2018
This work compares the performance of three prominent Neural Network based classifiers for speech... more This work compares the performance of three prominent Neural Network based classifiers for speech emotion recognition (SER). The classifiers such as the Multilayer Perceptron (MLP), the Radial Basis Function Network (RBFN) and the Probabilistic Neural Network (PNN) have been tested for their effectiveness in recognizing speech emotions such as angry, sad, bore and happy states from Berlin (EMO-DB) database. The self-learning ability of these classifiers to capture complex input and output relationship effectively makes them versatile in the field of SER. Hence, these parallel structured networks are expected to provide the intended accuracy for the proposed task as speech frequencies occur in parallel. Extensive simulation of these classifiers has been carried out using the popular Vector Quantized based Linear Predictive Cepstral Coefficients (LPCC_VQ) and the excitation source Hurst components. The PNN has shown to outperform all others in classifying the chosen emotional states using the extracted feature sets. The LPCC_VQ remains more discriminating for the proposed SER system than the pH as our results reveal. The average classification error using the LPCC-VQ feature sets has been 0.173 for the PNN as compared to 0.185 with RBFN and 0.227 with MLP as observed from our results.
This paper presents our progress towards deploying a versatile communication platform in the task... more This paper presents our progress towards deploying a versatile communication platform in the task of highly multilingual live speech translation for conferences and remote meetings live subtitling. The platform has been designed with a focus on very low latency and high flexibility while allowing research prototypes of speech and text processing tools to be easily connected, regardless of where they physically run. We outline our architecture solution and also briefly compare it with the ELG platform. Technical details are provided on the most important components and we summarize the test deployment events we ran so far.
ELITR (European Live Translator) project aims to create a speech translation system for simultane... more ELITR (European Live Translator) project aims to create a speech translation system for simultaneous subtitling of conferences and online meetings targetting up to 43 languages. The technology is tested by the Supreme Audit Office of the Czech Republic and by alfaview®, a German online conferencing system. Other project goals are to advance document-level and multilingual machine translation, automatic speech recognition, and automatic minuting.
2018 2nd International Conference on Data Science and Business Analytics (ICDSBA)
This work compares the performance of three prominent Neural Network based classifiers for speech... more This work compares the performance of three prominent Neural Network based classifiers for speech emotion recognition (SER). The classifiers such as the Multilayer Perceptron (MLP), the Radial Basis Function Network (RBFN) and the Probabilistic Neural Network (PNN) have been tested for their effectiveness in recognizing speech emotions such as angry, sad, bore and happy states from Berlin (EMO-DB) database. The self-learning ability of these classifiers to capture complex input and output relationship effectively makes them versatile in the field of SER. Hence, these parallel structured networks are expected to provide the intended accuracy for the proposed task as speech frequencies occur in parallel. Extensive simulation of these classifiers has been carried out using the popular Vector Quantized based Linear Predictive Cepstral Coefficients (LPCC_VQ) and the excitation source Hurst components. The PNN has shown to outperform all others in classifying the chosen emotional states using the extracted feature sets. The LPCC_VQ remains more discriminating for the proposed SER system than the pH as our results reveal. The average classification error using the LPCC-VQ feature sets has been 0.173 for the PNN as compared to 0.185 with RBFN and 0.227 with MLP as observed from our results.
The work attempts to characterize and classify speech emotions using the spectrogram. Initially, ... more The work attempts to characterize and classify speech emotions using the spectrogram. Initially, it extracts the individual Red, Green, and Blue parameters from the raw speech spectrogram image of every individual emotional utterance. Further, it computes the statistical parameters of individual RGB components to characterize the chosen emotional states. The utterances of anger, happiness, neutral, and sad emotional states from the standard Berlin (EMO-DB) database has been used for this purpose. The individual statistical R, G, and B spectrogram parameters are found to be different within an emotion as well as across emotional states. Thus, these values have been used as different feature sets to classify the designated emotional states using the popular Multilayer Perceptron Neural Network (MLPNN).
This paper presents an automatic speech translation system aimed at live subtitling of conference... more This paper presents an automatic speech translation system aimed at live subtitling of conference presentations. We describe the overall architecture and key processing components. More importantly, we explain our strategy for building a complex system for end-users from numerous individual components, each of which has been tested only in laboratory conditions. The system is a working prototype that is routinely tested in recognizing English, Czech, and German speech and presenting it translated simultaneously into 42 target languages.
Proceedings of the 17th International Conference on Spoken Language Translation
In this paper, we present our submission to the Non-Native Speech Translation Task for IWSLT 2020... more In this paper, we present our submission to the Non-Native Speech Translation Task for IWSLT 2020. Our main contribution is a proposed speech recognition pipeline that consists of an acoustic model and a phoneme-tographeme model. As an intermediate representation, we utilize phonemes. We demonstrate that the proposed pipeline surpasses commercially used automatic speech recognition (ASR) and submit it into the ASR track. We complement this ASR with off-the-shelf MT systems to take part also in the speech translation track.
Proceedings of the 17th International Conference on Spoken Language Translation
This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020... more This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020. We describe systems for offline ASR, real-time ASR, and our cascaded approach to offline SLT and real-time SLT. We select our primary candidates from a pool of pre-existing systems, develop a new end-toend general ASR system, and a hybrid ASR trained on non-native speech. The provided small validation set prevents us from carrying out a complex validation, but we submit all the unselected candidates for contrastive evaluation on the test set.
This paper presents a Bayesian multilingual topic model for learning language-independent documen... more This paper presents a Bayesian multilingual topic model for learning language-independent document embeddings. Our model learns to represent the documents in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. We propagate the learned uncertainties through linear classifiers for zero-shot cross-lingual topic identification. Our experiments on 5 language Europarl and Reuters (MLDoc) corpora show that the proposed model outperforms multi-lingual word embedding and BiLSTM sentence encoder based systems with significant margins in the majority of the transfer directions. Moreover, our system trained under a single day on a single GPU with much lower amounts of data performs competitively as compared to the state-of-the-art universal BiLSTM sentence encoder trained on 93 languages. Our experimental analysis shows that the amount of parallel data improves the overall performance of embeddings. Nonetheless, exploiting the uncertainties is always beneficial.
Defenses against security threats have been an interest of recent studies. Recent works have show... more Defenses against security threats have been an interest of recent studies. Recent works have shown that it is not difficult to attack a natural language processing (NLP) model while defending against them is still a cat-mouse game. Backdoor attacks are one such attack where a neural network is made to perform in a certain way on specific samples containing some triggers while achieving normal results on other samples. In this work, we present a few defense strategies that can be useful to counter against such an attack. We show that our defense methodologies significantly decrease the performance on the attacked inputs while maintaining similar performance on benign inputs. We also show that some of our defenses have very less runtime and also maintain similarity with the original inputs.
2018 2nd International Conference on Data Science and Business Analytics (ICDSBA), 2018
This work compares the performance of three prominent Neural Network based classifiers for speech... more This work compares the performance of three prominent Neural Network based classifiers for speech emotion recognition (SER). The classifiers such as the Multilayer Perceptron (MLP), the Radial Basis Function Network (RBFN) and the Probabilistic Neural Network (PNN) have been tested for their effectiveness in recognizing speech emotions such as angry, sad, bore and happy states from Berlin (EMO-DB) database. The self-learning ability of these classifiers to capture complex input and output relationship effectively makes them versatile in the field of SER. Hence, these parallel structured networks are expected to provide the intended accuracy for the proposed task as speech frequencies occur in parallel. Extensive simulation of these classifiers has been carried out using the popular Vector Quantized based Linear Predictive Cepstral Coefficients (LPCC_VQ) and the excitation source Hurst components. The PNN has shown to outperform all others in classifying the chosen emotional states using the extracted feature sets. The LPCC_VQ remains more discriminating for the proposed SER system than the pH as our results reveal. The average classification error using the LPCC-VQ feature sets has been 0.173 for the PNN as compared to 0.185 with RBFN and 0.227 with MLP as observed from our results.
This paper presents our progress towards deploying a versatile communication platform in the task... more This paper presents our progress towards deploying a versatile communication platform in the task of highly multilingual live speech translation for conferences and remote meetings live subtitling. The platform has been designed with a focus on very low latency and high flexibility while allowing research prototypes of speech and text processing tools to be easily connected, regardless of where they physically run. We outline our architecture solution and also briefly compare it with the ELG platform. Technical details are provided on the most important components and we summarize the test deployment events we ran so far.
ELITR (European Live Translator) project aims to create a speech translation system for simultane... more ELITR (European Live Translator) project aims to create a speech translation system for simultaneous subtitling of conferences and online meetings targetting up to 43 languages. The technology is tested by the Supreme Audit Office of the Czech Republic and by alfaview®, a German online conferencing system. Other project goals are to advance document-level and multilingual machine translation, automatic speech recognition, and automatic minuting.
2018 2nd International Conference on Data Science and Business Analytics (ICDSBA)
This work compares the performance of three prominent Neural Network based classifiers for speech... more This work compares the performance of three prominent Neural Network based classifiers for speech emotion recognition (SER). The classifiers such as the Multilayer Perceptron (MLP), the Radial Basis Function Network (RBFN) and the Probabilistic Neural Network (PNN) have been tested for their effectiveness in recognizing speech emotions such as angry, sad, bore and happy states from Berlin (EMO-DB) database. The self-learning ability of these classifiers to capture complex input and output relationship effectively makes them versatile in the field of SER. Hence, these parallel structured networks are expected to provide the intended accuracy for the proposed task as speech frequencies occur in parallel. Extensive simulation of these classifiers has been carried out using the popular Vector Quantized based Linear Predictive Cepstral Coefficients (LPCC_VQ) and the excitation source Hurst components. The PNN has shown to outperform all others in classifying the chosen emotional states using the extracted feature sets. The LPCC_VQ remains more discriminating for the proposed SER system than the pH as our results reveal. The average classification error using the LPCC-VQ feature sets has been 0.173 for the PNN as compared to 0.185 with RBFN and 0.227 with MLP as observed from our results.
The work attempts to characterize and classify speech emotions using the spectrogram. Initially, ... more The work attempts to characterize and classify speech emotions using the spectrogram. Initially, it extracts the individual Red, Green, and Blue parameters from the raw speech spectrogram image of every individual emotional utterance. Further, it computes the statistical parameters of individual RGB components to characterize the chosen emotional states. The utterances of anger, happiness, neutral, and sad emotional states from the standard Berlin (EMO-DB) database has been used for this purpose. The individual statistical R, G, and B spectrogram parameters are found to be different within an emotion as well as across emotional states. Thus, these values have been used as different feature sets to classify the designated emotional states using the popular Multilayer Perceptron Neural Network (MLPNN).
This paper presents an automatic speech translation system aimed at live subtitling of conference... more This paper presents an automatic speech translation system aimed at live subtitling of conference presentations. We describe the overall architecture and key processing components. More importantly, we explain our strategy for building a complex system for end-users from numerous individual components, each of which has been tested only in laboratory conditions. The system is a working prototype that is routinely tested in recognizing English, Czech, and German speech and presenting it translated simultaneously into 42 target languages.
Proceedings of the 17th International Conference on Spoken Language Translation
In this paper, we present our submission to the Non-Native Speech Translation Task for IWSLT 2020... more In this paper, we present our submission to the Non-Native Speech Translation Task for IWSLT 2020. Our main contribution is a proposed speech recognition pipeline that consists of an acoustic model and a phoneme-tographeme model. As an intermediate representation, we utilize phonemes. We demonstrate that the proposed pipeline surpasses commercially used automatic speech recognition (ASR) and submit it into the ASR track. We complement this ASR with off-the-shelf MT systems to take part also in the speech translation track.
Proceedings of the 17th International Conference on Spoken Language Translation
This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020... more This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020. We describe systems for offline ASR, real-time ASR, and our cascaded approach to offline SLT and real-time SLT. We select our primary candidates from a pool of pre-existing systems, develop a new end-toend general ASR system, and a hybrid ASR trained on non-native speech. The provided small validation set prevents us from carrying out a complex validation, but we submit all the unselected candidates for contrastive evaluation on the test set.
This paper presents a Bayesian multilingual topic model for learning language-independent documen... more This paper presents a Bayesian multilingual topic model for learning language-independent document embeddings. Our model learns to represent the documents in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. We propagate the learned uncertainties through linear classifiers for zero-shot cross-lingual topic identification. Our experiments on 5 language Europarl and Reuters (MLDoc) corpora show that the proposed model outperforms multi-lingual word embedding and BiLSTM sentence encoder based systems with significant margins in the majority of the transfer directions. Moreover, our system trained under a single day on a single GPU with much lower amounts of data performs competitively as compared to the state-of-the-art universal BiLSTM sentence encoder trained on 93 languages. Our experimental analysis shows that the amount of parallel data improves the overall performance of embeddings. Nonetheless, exploiting the uncertainties is always beneficial.
Uploads
Papers by SANGEET SAGAR