We study robust pitch synchronous parameters that are derived from envelope and instantaneous fre... more We study robust pitch synchronous parameters that are derived from envelope and instantaneous frequencies estimated via a bank of cochlear filters. Closed set Speaker Identification experiments are performed on the SPIDRE corpus with matched and mis- matched handsets conditions. The recognizer is based on a hybrid Linear Vector Quantization and Single Layer Perceptron (LVQ- SLP). Experiments are reported with different
Usually, speaker recognition systems do not take into account the dependence between the vocal so... more Usually, speaker recognition systems do not take into account the dependence between the vocal source and the vocal tract. A feasi- bility study that retains this dependence is presented here. A model of joint probability functions of the pitch and the feature vectors is proposed. Three strategies are designed and compared for all female speakers taken from the SPIDRE corpus.
A paradigm for automatic speech recognition using networks of actions performing variable depth a... more A paradigm for automatic speech recognition using networks of actions performing variable depth analysis is presented. The paradigm produces descriptions of speech properties that are related to speech units through Markov models representing system performance. Preliminary results in the recognition of isolated letters and digits are presented.
Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005
A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three c... more A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three capabilities are essential for robot audition; sound source localization, separation, and recognition of separated sounds. While the first two are frequently addressed, the last one has not been studied so much. We present a system that gives a humanoid robot the ability to localize, separate and recognize simultaneous sound sources. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation (GSS) and a multi-channel postfilter that gives us a further reduction of interferences from other sources. An automatic speech recognizer (ASR) based on the Missing Feature Theory (MFT) recognizes separated sounds in real-time by generating missing feature masks automatically from the post-filtering step. The main advantage of this approach for humanoid robots resides in the fact that the ASR with a clean acoustic model can adapt the distortion of separated sound by consulting the post-filter feature masks. Recognition rates are presented for three simultaneous speakers located at 2m from the robot. Use of both the post-filter and the missing feature mask results in an average reduction in error rate of 42% (relative).
Neuronal assemblies typically synchronize within the gamma band (30-80 Hz) and are fundamental to... more Neuronal assemblies typically synchronize within the gamma band (30-80 Hz) and are fundamental to information processing. Despite numerous investigations, the exact mechanisms and origins of gamma oscillations are yet to be known. Here, through multiunit recordings in the primary visual cortex of cats we show that the strength of gamma power (20-40 Hz and 60-80 Hz) is significantly stronger between the functionally connected units than the unconnected units within an assembly. Furthermore, there is increased frequency coherence in the gamma range between the connected units than the unconnected units. Lastly, the higher gamma rhythms (60-80 Hz) are mostly linked to the fast-spiking neurons. These results led us to postulate that gamma activity is intrinsically generated between the connected units within cell-assemblies (microcircuits) in relation to the stimulus within an emergent '50 ms-temporal window-of-opportunity'. This article is protected by copyright. All rights res...
Cortical organization rests upon the fundamental principle that neurons sharing similar propertie... more Cortical organization rests upon the fundamental principle that neurons sharing similar properties are co-located. In the visual cortex, neurons are organized into orientation columns. In a column, most neurons respond optimally to the same axis of an oriented edge, that is, the preferred orientation. This orientation selectivity is believed to be absolute in adulthood. However, in a fully mature brain, it has been established that neurons change their selectivity following sensory experience or visual adaptation. Here, we show that after applying an adapter away from the tested cells, neurons whose receptive fields were located remotely from the adapted site also exhibit a novel selectivity in spite of the fact that they were not adapted. These results indicate a robust reconfiguration and remapping of the orientation domains with respect to each other thus removing the possibility of an orientation hole in the new hypercolumn. These data suggest that orientation columns transcend anatomy, and are almost strictly functionally dynamic.
Usually, speaker recognition systems do not take into account the short-term dependence between t... more Usually, speaker recognition systems do not take into account the short-term dependence between the vocal source and the vocal tract. A feasibility study that re- tains this dependence is presented here. A model of joint probability functions of the pitch and the feature vectors is proposed. Three strategies are designed and compared for all female speakers taken from the SPIDRE
The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditor... more The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditory signal representations in high-dimensional spaces, so as to define the components of auditory objects. The concept of an auditory object and its neural representation is introduced. An illustrative application then follows, consisting in the analysis of various auditory signals : speech, music and natural outdoor environments. A new automatic speech recognition (ASR) system is then proposed and compared to a conventional statistical system. The proposed system clearly shows that an object-based analysis introduces a great flexibility and robustness for the task of speech recognition. The integration of knowledge from neuroscience and acoustic signal processing brings new ways of thinking to the field of classification of acoustic signals.
The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditor... more The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditory signal representations in high-dimensional spaces, so as to define the components of auditory objects. The concept of an auditory object and its neural representation is introduced. An illustrative application then follows, consisting in the analysis of various auditory signals : speech, music and natural outdoor environments. A new automatic speech recognition (ASR) system is then proposed and compared to a conventional statistical system. The proposed system clearly shows that an object-based analysis introduces a great flexibility and robustness for the task of speech recognition. The integration of knowledge from neuroscience and acoustic signal processing brings new ways of thinking to the field of classification of acoustic signals.
Sensory neurons exhibit remarkable adaptability in acquiring new optimal selectivity to unfamilia... more Sensory neurons exhibit remarkable adaptability in acquiring new optimal selectivity to unfamiliar features when a new stimulus becomes prevalent in the environment. In conventionally prepared adult anesthetized cats, we used visual adaptation to change the preferred orientation selectivity in V1 neurons. Cortical circuits are dominated by complex and intricate connections between neurons. Cross-correlation of cellular spike-trains discloses the putative functional connection between two neurons. We sought to investigate changes in these links following a 12 min uninterrupted application of a specific, usually non-preferred, orientation. We report that visual adaptation, mimicking training, modulates the magnitude of crosscorrelograms suggesting that the strength of inter-neuronal relationships is modified. While individual cell-pairs exhibit changes in their response correlation strength, the average correlation of the recorded cell cluster remains unchanged. Hence, visual adaptation induces plastic changes that impact the connectivity between neurons.
2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA), 2012
ABSTRACT The purpose of this research is to develop a sensorial substitution system from vision t... more ABSTRACT The purpose of this research is to develop a sensorial substitution system from vision to audition. The intention is to provide a noninvasive solution for people with visual impairment to compensate their disability using brain plasticity. The “prosthesis” is a signal processing system that analyses the visual scene to identify objects of interest and encode them into sound. The encoding should be based on characteristics of the human auditory system so that the generated sounds provide an overview of the visual scene in front of the patient, enabling him to locate each identified object. This should allow people with visual disabilities to move around more easily, even in cluttered environments. This paper describes the image processing to enhance objects from images and gives an overview of the sound encoding.
IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, 2004
Jean-Marc Valin, Franlois Michaud, Brahim Hadjou, Jean Rouat LABORIUS - Research Laboratory on Mo... more Jean-Marc Valin, Franlois Michaud, Brahim Hadjou, Jean Rouat LABORIUS - Research Laboratory on Mobile Robotics and Intelligent Systems Department of Electrical Engineering and Computer Engineering Universitk de Sherbrooke, Sherbrooke (Quebec) CANADA, JI K 2R1 { Jean-...
2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005
A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three c... more A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three capabilities are essential for robot audition; sound source localization, separation, and recognition of separated sounds. We have adopted the missing feature theory (MFT) for automatic recognition of separated speech, and developed the robot audition system. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation (GSS) and a multi-channel post-filter that gives us a further reduction of interferences from other sources. The automatic speech recognition based on MFT recognizes separated sounds by generating missing feature masks automatically from the post-filtering step. The main advantage of this approach for humanoid robots resides in the fact that the ASR with a clean acoustic model can adapt the distortion of separated sound by consulting the postfilter feature masks. In this paper, we used the improved Julius as an MFT-based automatic speech recognizer (ASR). The Julius is a real-time large vocabulary continuous speech recognition (LVCSR) system. We performed the experiment to evaluate our robot audition system. In this experiment, the system recognizes a sentence, not an isolated word. We showed the improvement in the system performance through three simultaneous speech recognition on the humanoid SIG2.
Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), 2003
The hearing sense on a mobile robot is important because it is omnidirectional and it does not re... more The hearing sense on a mobile robot is important because it is omnidirectional and it does not require direct line-of-sight with the sound source. Such capabilities can nicely complement vision to help localize a person or an interesting event in the environment. To do so the robot auditory system must be able to work in noisy, unknown and diverse environmental conditions. In this paper we present a robust sound source localization method in three-dimensional space using an array of 8 microphones. The method is based on time delay of arrival estimation. Results show that a mobile robot can localize in real time different types of sound sources over a range of 3 meters and with a precision of 3 • .
We study robust pitch synchronous parameters that are derived from envelope and instantaneous fre... more We study robust pitch synchronous parameters that are derived from envelope and instantaneous frequencies estimated via a bank of cochlear filters. Closed set Speaker Identification experiments are performed on the SPIDRE corpus with matched and mis- matched handsets conditions. The recognizer is based on a hybrid Linear Vector Quantization and Single Layer Perceptron (LVQ- SLP). Experiments are reported with different
Usually, speaker recognition systems do not take into account the dependence between the vocal so... more Usually, speaker recognition systems do not take into account the dependence between the vocal source and the vocal tract. A feasi- bility study that retains this dependence is presented here. A model of joint probability functions of the pitch and the feature vectors is proposed. Three strategies are designed and compared for all female speakers taken from the SPIDRE corpus.
A paradigm for automatic speech recognition using networks of actions performing variable depth a... more A paradigm for automatic speech recognition using networks of actions performing variable depth analysis is presented. The paradigm produces descriptions of speech properties that are related to speech units through Markov models representing system performance. Preliminary results in the recognition of isolated letters and digits are presented.
Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005
A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three c... more A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three capabilities are essential for robot audition; sound source localization, separation, and recognition of separated sounds. While the first two are frequently addressed, the last one has not been studied so much. We present a system that gives a humanoid robot the ability to localize, separate and recognize simultaneous sound sources. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation (GSS) and a multi-channel postfilter that gives us a further reduction of interferences from other sources. An automatic speech recognizer (ASR) based on the Missing Feature Theory (MFT) recognizes separated sounds in real-time by generating missing feature masks automatically from the post-filtering step. The main advantage of this approach for humanoid robots resides in the fact that the ASR with a clean acoustic model can adapt the distortion of separated sound by consulting the post-filter feature masks. Recognition rates are presented for three simultaneous speakers located at 2m from the robot. Use of both the post-filter and the missing feature mask results in an average reduction in error rate of 42% (relative).
Neuronal assemblies typically synchronize within the gamma band (30-80 Hz) and are fundamental to... more Neuronal assemblies typically synchronize within the gamma band (30-80 Hz) and are fundamental to information processing. Despite numerous investigations, the exact mechanisms and origins of gamma oscillations are yet to be known. Here, through multiunit recordings in the primary visual cortex of cats we show that the strength of gamma power (20-40 Hz and 60-80 Hz) is significantly stronger between the functionally connected units than the unconnected units within an assembly. Furthermore, there is increased frequency coherence in the gamma range between the connected units than the unconnected units. Lastly, the higher gamma rhythms (60-80 Hz) are mostly linked to the fast-spiking neurons. These results led us to postulate that gamma activity is intrinsically generated between the connected units within cell-assemblies (microcircuits) in relation to the stimulus within an emergent '50 ms-temporal window-of-opportunity'. This article is protected by copyright. All rights res...
Cortical organization rests upon the fundamental principle that neurons sharing similar propertie... more Cortical organization rests upon the fundamental principle that neurons sharing similar properties are co-located. In the visual cortex, neurons are organized into orientation columns. In a column, most neurons respond optimally to the same axis of an oriented edge, that is, the preferred orientation. This orientation selectivity is believed to be absolute in adulthood. However, in a fully mature brain, it has been established that neurons change their selectivity following sensory experience or visual adaptation. Here, we show that after applying an adapter away from the tested cells, neurons whose receptive fields were located remotely from the adapted site also exhibit a novel selectivity in spite of the fact that they were not adapted. These results indicate a robust reconfiguration and remapping of the orientation domains with respect to each other thus removing the possibility of an orientation hole in the new hypercolumn. These data suggest that orientation columns transcend anatomy, and are almost strictly functionally dynamic.
Usually, speaker recognition systems do not take into account the short-term dependence between t... more Usually, speaker recognition systems do not take into account the short-term dependence between the vocal source and the vocal tract. A feasibility study that re- tains this dependence is presented here. A model of joint probability functions of the pitch and the feature vectors is proposed. Three strategies are designed and compared for all female speakers taken from the SPIDRE
The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditor... more The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditory signal representations in high-dimensional spaces, so as to define the components of auditory objects. The concept of an auditory object and its neural representation is introduced. An illustrative application then follows, consisting in the analysis of various auditory signals : speech, music and natural outdoor environments. A new automatic speech recognition (ASR) system is then proposed and compared to a conventional statistical system. The proposed system clearly shows that an object-based analysis introduces a great flexibility and robustness for the task of speech recognition. The integration of knowledge from neuroscience and acoustic signal processing brings new ways of thinking to the field of classification of acoustic signals.
The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditor... more The emphasis is put on the hierarchical structure, independence and sparseness aspects of auditory signal representations in high-dimensional spaces, so as to define the components of auditory objects. The concept of an auditory object and its neural representation is introduced. An illustrative application then follows, consisting in the analysis of various auditory signals : speech, music and natural outdoor environments. A new automatic speech recognition (ASR) system is then proposed and compared to a conventional statistical system. The proposed system clearly shows that an object-based analysis introduces a great flexibility and robustness for the task of speech recognition. The integration of knowledge from neuroscience and acoustic signal processing brings new ways of thinking to the field of classification of acoustic signals.
Sensory neurons exhibit remarkable adaptability in acquiring new optimal selectivity to unfamilia... more Sensory neurons exhibit remarkable adaptability in acquiring new optimal selectivity to unfamiliar features when a new stimulus becomes prevalent in the environment. In conventionally prepared adult anesthetized cats, we used visual adaptation to change the preferred orientation selectivity in V1 neurons. Cortical circuits are dominated by complex and intricate connections between neurons. Cross-correlation of cellular spike-trains discloses the putative functional connection between two neurons. We sought to investigate changes in these links following a 12 min uninterrupted application of a specific, usually non-preferred, orientation. We report that visual adaptation, mimicking training, modulates the magnitude of crosscorrelograms suggesting that the strength of inter-neuronal relationships is modified. While individual cell-pairs exhibit changes in their response correlation strength, the average correlation of the recorded cell cluster remains unchanged. Hence, visual adaptation induces plastic changes that impact the connectivity between neurons.
2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA), 2012
ABSTRACT The purpose of this research is to develop a sensorial substitution system from vision t... more ABSTRACT The purpose of this research is to develop a sensorial substitution system from vision to audition. The intention is to provide a noninvasive solution for people with visual impairment to compensate their disability using brain plasticity. The “prosthesis” is a signal processing system that analyses the visual scene to identify objects of interest and encode them into sound. The encoding should be based on characteristics of the human auditory system so that the generated sounds provide an overview of the visual scene in front of the patient, enabling him to locate each identified object. This should allow people with visual disabilities to move around more easily, even in cluttered environments. This paper describes the image processing to enhance objects from images and gives an overview of the sound encoding.
IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, 2004
Jean-Marc Valin, Franlois Michaud, Brahim Hadjou, Jean Rouat LABORIUS - Research Laboratory on Mo... more Jean-Marc Valin, Franlois Michaud, Brahim Hadjou, Jean Rouat LABORIUS - Research Laboratory on Mobile Robotics and Intelligent Systems Department of Electrical Engineering and Computer Engineering Universitk de Sherbrooke, Sherbrooke (Quebec) CANADA, JI K 2R1 { Jean-...
2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005
A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three c... more A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three capabilities are essential for robot audition; sound source localization, separation, and recognition of separated sounds. We have adopted the missing feature theory (MFT) for automatic recognition of separated speech, and developed the robot audition system. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation (GSS) and a multi-channel post-filter that gives us a further reduction of interferences from other sources. The automatic speech recognition based on MFT recognizes separated sounds by generating missing feature masks automatically from the post-filtering step. The main advantage of this approach for humanoid robots resides in the fact that the ASR with a clean acoustic model can adapt the distortion of separated sound by consulting the postfilter feature masks. In this paper, we used the improved Julius as an MFT-based automatic speech recognizer (ASR). The Julius is a real-time large vocabulary continuous speech recognition (LVCSR) system. We performed the experiment to evaluate our robot audition system. In this experiment, the system recognizes a sentence, not an isolated word. We showed the improvement in the system performance through three simultaneous speech recognition on the humanoid SIG2.
Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), 2003
The hearing sense on a mobile robot is important because it is omnidirectional and it does not re... more The hearing sense on a mobile robot is important because it is omnidirectional and it does not require direct line-of-sight with the sound source. Such capabilities can nicely complement vision to help localize a person or an interesting event in the environment. To do so the robot auditory system must be able to work in noisy, unknown and diverse environmental conditions. In this paper we present a robust sound source localization method in three-dimensional space using an array of 8 microphones. The method is based on time delay of arrival estimation. Results show that a mobile robot can localize in real time different types of sound sources over a range of 3 meters and with a precision of 3 • .
Uploads
Papers by Jean Rouat