ACM Transactions on Speech and Language Processing, 2011
This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improv... more This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improve science learning by students in 3 rd , 4 th , and 5 th grades (7 to 11 years old) through conversational dialogs with a virtual science tutor. In our study, individual students engage in spoken dialogs with the virtual tutor Marni during 15 to 20 minute sessions following classroom science investigations to discuss and extend concepts embedded in the investigations. The spoken dialogs in MyST are designed to scaffold learning by presenting open-ended questions accompanied by illustrations or animations related to the classroom investigations and the science concepts being learned. The focus of the interactions is to elicit self-expression from students. To this end, Marni applies some of the principles of Questioning the Author , a proven approach to classroom conversations, to challenge students to think about and integrate new concepts with prior knowledge to construct enriched menta...
Proceedings of ACL-2016 System Demonstrations, 2016
This paper presents a conversational, multimedia, virtual science tutor for elementary school stu... more This paper presents a conversational, multimedia, virtual science tutor for elementary school students. It is built using state of the art speech recognition and spoken language understanding technology. This virtual science tutor is unique in that it elicits self-explanations from students for various science phenomena by engaging them in spoken dialogs and guided by illustrations, animations and interactive simulations. There is a lot of evidence that self-explanation works well as a tutorial paradigm, Summative evaluations indicate that students are highly engaged in the tutoring sessions, and achieve learning outcomes equivalent to expert human tutors. Tutorials are developed through a process of recording and annotating data from sessions with students, and then updating tutor models. It enthusiastically supported by students and teachers. Teachers report that it is feasible to integrate into their curriculum.
this report. A small fee is requiredto cover our costs. The database may be freely copied and dis... more this report. A small fee is requiredto cover our costs. The database may be freely copied and distributed.9 Speaker Information
2014 IEEE-RAS International Conference on Humanoid Robots, 2014
This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to... more This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to support verbal and non-verbal communication between the robot and humans, with the goal of closely modeling the dynamics of natural faceto-face communication. The proposed robotic head consists of two major components: 1) a hardware component that contains a small projector, a fish-eye lens, a custom-designed mask and a neck system with 3 degrees of freedom; 2) a facial animation system, projected onto the robotic mask, that is capable of presenting facial expressions, realistic eye movement, and accurate visual speech. We present three studies that compare Human-Robot Interaction with Human-Computer Interaction with a screen-based model of the avatar. The studies indicate that the robotic face is well accepted by users, with some advantages in recognition of facial expression and mutual eye gaze contact.
Speech recognition is the process of converting an acoustic signal, captured by a microphone or a... more Speech recognition is the process of converting an acoustic signal, captured by a microphone or a telephone, to a set of words. The recognized words can be the final results, for such applications as commands & control, data entry, and document preparation. They can also serve as the input to further linguistic processing in order to achieve speech understanding, a subject covered in section 1.8.
After describing the belief that speech is special, empirical and theoretical research is reviewe... more After describing the belief that speech is special, empirical and theoretical research is reviewed undermining the tenets of this belief. A new framework is presented as a theoretical framework for language learning. Central to this framework is the natural ease of multimodal perception, particularly the value of visible speech. The value of synthetic talking heads is described along with their
1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1997
... Spoken letter recognition. In RP Lippman, J. Moody, and D. S. Touretzky, editors, Advances in... more ... Spoken letter recognition. In RP Lippman, J. Moody, and D. S. Touretzky, editors, Advances in Neural Information Processing Systems 3. San Mateo, CA: Morgan Kaufmann, 1991. V.Fontaine, C. Ris, H. Leich, J. Vantieghen, S. Ac-caino, and D. Compernolle. ...
This article talks about how advances in human language technology can help overcomesome of the b... more This article talks about how advances in human language technology can help overcomesome of the barriers that prevent community participation in cyberspace. Human languagetechnology refers to the set of technologies, such as speech recognition and speech synthesisthat are used to create spoken language systems---systems that allow people to communicatewith machines using speech.A significant advantage of using speech as an interface
The purpose of this chapter is to describe the vocabulary development and promising, evidence-bas... more The purpose of this chapter is to describe the vocabulary development and promising, evidence-based vocabulary interventions for English learners (ELs) from preschool through second grade. To achieve this purpose, we have taken six steps. First, we describe the elements of language development in the native language (L1) and a second language (L2) and how these elements relate to three phases of reading development (i.e., the prereading phase, the learning to read phase, and the reading to learn phase). We contend that in order for ELs to succeed in school, they need a strong language foundation prior to entering kindergarten. This language foundation needs to continue developing during the "learning to read" and "reading to learn" phases. Second, we describe the limitations of current practice in preschool for ELs related to vocabulary instruction and to family involvement to support children's language development. Third, we report curricular challenges fac...
Proceedings of the workshop on Human Language Technology - HLT '94, 1994
This paper describes eight telephone-speech corpora at various stages of development at the Cente... more This paper describes eight telephone-speech corpora at various stages of development at the Center for Spoken Language Understanding. For each corpus, we describe data collection procedures, methods of soliciting callers, protocol used to collect the data, transcriptions that accompany the speech data, and the expected release date. The corpora are available at no charge to academic institutions.
ACM SIGCAPH Computers and the Physically Handicapped, 1998
This report describes a three-year project, now eight months old, to develop interactive learning... more This report describes a three-year project, now eight months old, to develop interactive learning tools for language training with profoundly deaf children. The tools combine four key technologies: speech recognition, developed at the Oregon Graduate Institute; speech synthesis, developed at the University of Edinburgh and modified at OGI; facial animation, developed at University of California, Santa Cruz; and face tracking and speech reading, developed at Carnegie Mellon University. These technologies are being combined to create an intelligent conversational agent; a three-dimensional face that produces and understands auditory and visual speech. The agent has been incorporated into CSLU Toolkit, a software environment for developing and researching spoken language systems. We describe our experiences in bringing interactive learning tools to classrooms at the Tucker-Maxon Oral School in Portland, Oregon, and the technological advances that are required for this project to succeed.
ACM Transactions on Speech and Language Processing, 2011
This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improv... more This article describes My Science Tutor (MyST), an intelligent tutoring system designed to improve science learning by students in 3 rd , 4 th , and 5 th grades (7 to 11 years old) through conversational dialogs with a virtual science tutor. In our study, individual students engage in spoken dialogs with the virtual tutor Marni during 15 to 20 minute sessions following classroom science investigations to discuss and extend concepts embedded in the investigations. The spoken dialogs in MyST are designed to scaffold learning by presenting open-ended questions accompanied by illustrations or animations related to the classroom investigations and the science concepts being learned. The focus of the interactions is to elicit self-expression from students. To this end, Marni applies some of the principles of Questioning the Author , a proven approach to classroom conversations, to challenge students to think about and integrate new concepts with prior knowledge to construct enriched menta...
Proceedings of ACL-2016 System Demonstrations, 2016
This paper presents a conversational, multimedia, virtual science tutor for elementary school stu... more This paper presents a conversational, multimedia, virtual science tutor for elementary school students. It is built using state of the art speech recognition and spoken language understanding technology. This virtual science tutor is unique in that it elicits self-explanations from students for various science phenomena by engaging them in spoken dialogs and guided by illustrations, animations and interactive simulations. There is a lot of evidence that self-explanation works well as a tutorial paradigm, Summative evaluations indicate that students are highly engaged in the tutoring sessions, and achieve learning outcomes equivalent to expert human tutors. Tutorials are developed through a process of recording and annotating data from sessions with students, and then updating tutor models. It enthusiastically supported by students and teachers. Teachers report that it is feasible to integrate into their curriculum.
this report. A small fee is requiredto cover our costs. The database may be freely copied and dis... more this report. A small fee is requiredto cover our costs. The database may be freely copied and distributed.9 Speaker Information
2014 IEEE-RAS International Conference on Humanoid Robots, 2014
This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to... more This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to support verbal and non-verbal communication between the robot and humans, with the goal of closely modeling the dynamics of natural faceto-face communication. The proposed robotic head consists of two major components: 1) a hardware component that contains a small projector, a fish-eye lens, a custom-designed mask and a neck system with 3 degrees of freedom; 2) a facial animation system, projected onto the robotic mask, that is capable of presenting facial expressions, realistic eye movement, and accurate visual speech. We present three studies that compare Human-Robot Interaction with Human-Computer Interaction with a screen-based model of the avatar. The studies indicate that the robotic face is well accepted by users, with some advantages in recognition of facial expression and mutual eye gaze contact.
Speech recognition is the process of converting an acoustic signal, captured by a microphone or a... more Speech recognition is the process of converting an acoustic signal, captured by a microphone or a telephone, to a set of words. The recognized words can be the final results, for such applications as commands & control, data entry, and document preparation. They can also serve as the input to further linguistic processing in order to achieve speech understanding, a subject covered in section 1.8.
After describing the belief that speech is special, empirical and theoretical research is reviewe... more After describing the belief that speech is special, empirical and theoretical research is reviewed undermining the tenets of this belief. A new framework is presented as a theoretical framework for language learning. Central to this framework is the natural ease of multimodal perception, particularly the value of visible speech. The value of synthetic talking heads is described along with their
1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1997
... Spoken letter recognition. In RP Lippman, J. Moody, and D. S. Touretzky, editors, Advances in... more ... Spoken letter recognition. In RP Lippman, J. Moody, and D. S. Touretzky, editors, Advances in Neural Information Processing Systems 3. San Mateo, CA: Morgan Kaufmann, 1991. V.Fontaine, C. Ris, H. Leich, J. Vantieghen, S. Ac-caino, and D. Compernolle. ...
This article talks about how advances in human language technology can help overcomesome of the b... more This article talks about how advances in human language technology can help overcomesome of the barriers that prevent community participation in cyberspace. Human languagetechnology refers to the set of technologies, such as speech recognition and speech synthesisthat are used to create spoken language systems---systems that allow people to communicatewith machines using speech.A significant advantage of using speech as an interface
The purpose of this chapter is to describe the vocabulary development and promising, evidence-bas... more The purpose of this chapter is to describe the vocabulary development and promising, evidence-based vocabulary interventions for English learners (ELs) from preschool through second grade. To achieve this purpose, we have taken six steps. First, we describe the elements of language development in the native language (L1) and a second language (L2) and how these elements relate to three phases of reading development (i.e., the prereading phase, the learning to read phase, and the reading to learn phase). We contend that in order for ELs to succeed in school, they need a strong language foundation prior to entering kindergarten. This language foundation needs to continue developing during the "learning to read" and "reading to learn" phases. Second, we describe the limitations of current practice in preschool for ELs related to vocabulary instruction and to family involvement to support children's language development. Third, we report curricular challenges fac...
Proceedings of the workshop on Human Language Technology - HLT '94, 1994
This paper describes eight telephone-speech corpora at various stages of development at the Cente... more This paper describes eight telephone-speech corpora at various stages of development at the Center for Spoken Language Understanding. For each corpus, we describe data collection procedures, methods of soliciting callers, protocol used to collect the data, transcriptions that accompany the speech data, and the expected release date. The corpora are available at no charge to academic institutions.
ACM SIGCAPH Computers and the Physically Handicapped, 1998
This report describes a three-year project, now eight months old, to develop interactive learning... more This report describes a three-year project, now eight months old, to develop interactive learning tools for language training with profoundly deaf children. The tools combine four key technologies: speech recognition, developed at the Oregon Graduate Institute; speech synthesis, developed at the University of Edinburgh and modified at OGI; facial animation, developed at University of California, Santa Cruz; and face tracking and speech reading, developed at Carnegie Mellon University. These technologies are being combined to create an intelligent conversational agent; a three-dimensional face that produces and understands auditory and visual speech. The agent has been incorporated into CSLU Toolkit, a software environment for developing and researching spoken language systems. We describe our experiences in bringing interactive learning tools to classrooms at the Tucker-Maxon Oral School in Portland, Oregon, and the technological advances that are required for this project to succeed.
Uploads
Papers by Ron Cole