One of the most common cues in human relations is the reaction when someone approaches for a touc... more One of the most common cues in human relations is the reaction when someone approaches for a touch interaction. While the "before touch" distance has been investigated in daily life scenarios, it has not been studied for virtual environments. The measurements of a pre-touch distance in virtual reality can be applied to study social interactions especially for haptic interactions in virtual spaces where virtual agents interact autonomously with human participants. In the first stage of this study, we collected data to define a pre-touch distance when a virtual agent tries to touch the participant's face. On the basis of these results, we then classified participants into two groups based on preferred pre-touch distance: a "Near" group and a "Far" group. Next, we experimentally investigated the relationship between the participant's perception of an avatar's reaction to touch interaction and their preferred pre-touch distance. The results indicated that the participants felt friendliness to the agent who reacts with shorter pre-touch distance. We also found that the participant's pre-touch distance defined their preferences regarding the agent's behavior: those with a shorter pre-touch distance preferred agents with a closer interaction distance, and those with a longer pre-touch distance preferred agents with a longer interaction distance.
This paper investigates the effects of group interaction in a storytelling situation for children... more This paper investigates the effects of group interaction in a storytelling situation for children using two robots: a reader robot and a listener robot as a side-participant. We developed a storytelling system that consists of a reader robot, a listener robot, a display, a gaze model, a depth sensor, and a human operator who responds and provides easily understandable answers to the children’s questions. We experimentally investigated the effects of using a listener robot and either one or two children during a storytelling situation on the children’s preferences and their speech activities. Our experimental results showed that the children preferred storytelling with the listener robot. Although two children obviously produced more speech than one child, the listener robot discouraged the children’s speech regardless of whether one or two were listening.
In recent years, communication robots aiming to offer mental support to the elderly have attracte... more In recent years, communication robots aiming to offer mental support to the elderly have attracted increasing attention. Dialogue systems consisting of two robots could provide the elderly with opportunities to hold longer conversations in care homes. In this study, we conducted an experiment to compare two types of scenario-based dialogue systems with different types of bodies—physical and virtual robots—to investigate the effects of embodying such dialogue systems. Forty elderly people aged from 65 to 84 interacted with either an embodied desktop-sized humanoid robot or computer graphic agent displayed on a monitor. The elderly participants were divided into groups depending on the success of the interactions. The results revealed that (i) in the group where the robots responded more successfully with the expected conversation flow, the elderly are more engaged in the conversation with the physical robots than the virtual robots, and (ii) the elderly in the group in which robots r...
This paper reports the effects of communication cues on robot-initiated touch interactions at clo... more This paper reports the effects of communication cues on robot-initiated touch interactions at close distance by focusing on two factors: gaze-height for making eye contact and speech timing before and after touches. Although both factors are essential to achieve acceptable touches in human-human touch interaction, their effectiveness remains unknown in human-robot touch interaction contexts. To investigate the effects of these factors, we conducted an experiment whose results showed that being touched with before-touch timing is preferred to being touched with after-touch timing, although gaze-height did not significantly improve the feelings of robot-initiated touch.
Many social robots have emerged in public places to serve people. For these services, the robots ... more Many social robots have emerged in public places to serve people. For these services, the robots are assumed to be able to present internal aspects (i.e., mind, sociability) to engage and interact with people over the long term. In this paper, we propose a novel dialogue structure called experience-based dialogue to help a robot present and maintain a good interaction over the long term. This dialogue structure contains a piece of knowledge and a story about how the robot gained this knowledge, which are used to compose the robot's experience-related utterances for sharing experiences of interacting with previous users other than just the current user and help it present its internal aspects. We conducted an experiment to test the effects of our proposed dialogue structure and measure them with some published subjective scales. The results showed that experience-based dialogue can help a robot obtain better evaluations in terms of perceived intelligence, sociability, mind, anthropomorphism, animacy, likability, level of acceptance, and positive user reaction.
As agents, social robots are expected to increase opportunities for dialogue with the elderly. Ho... more As agents, social robots are expected to increase opportunities for dialogue with the elderly. However, it is difficult to sustain a dialogue with an elderly user because speech recognition frequently fails during the dialogue. Here, to overcome this problem, regardless of speech recognition failure, we developed a question–answer–response dialogue model. In this model, a robot took initiative in the dialogue by asking the user various questions. Moreover, to improve user experience during dialogue, we extended the model such that two robots could participate in the dialogue. Implementing these features, we conducted a field trial in a nursing home to evaluate the twin-robot dialogue system. The average word error rate of speech recognition was 0.778. Despite the frequently high number of errors, participants talked for 14 min in a dialogue with two robots and felt slightly strange during the dialogue. Although we found no significant difference between a dialogue with one robot and...
The recognition of indicated objects by interacting people is an essential function for robots th... more The recognition of indicated objects by interacting people is an essential function for robots that act in daily environments. However, due to ambiguous references by them, accurate recognition of indicated objects have difficulties for the robots. For example, people sometimes use the words which did not contain in the robots' databases, or they did not use enough words to identify the object. Therefore, to improve recognition accuracy, we must decrease such ambiguity of indicating behaviors of people. For this purpose, we experimentally compared two kinds of interaction strategies to decrease the ambiguity: explicitly providing requests to people about how to refer to objects, or implicitly aligning with people's indicating behaviors. The experimental results showed that participants evaluated the implicit strategy to be more natural than the explicit strategy, and the recognition performances of the two strategies were not significantly different.
SICE Journal of Control, Measurement, and System Integration
The popularization of video-viewing systems enables both adults and children to continuously watc... more The popularization of video-viewing systems enables both adults and children to continuously watch countless video clips. However, such extended video viewing might cause health problems, particularly for children. Rule-making tendencies are weaker for video-viewing systems than they are for television programming. Children struggle with selfmotivated video-viewing stops because of attractive, rich clips. In this study, we propose a video recommendation system that arranges video clips based on a pre-set time, to support self-motivated video-viewing stops. Our proposed system enables parents to limit video-viewing time in advance and provides video clips that are arranged to finish exactly at a pre-set time. We conducted two experiments. The first experiment targeted adults to confirm the effectiveness of our approach. Our results from the first experiment suggest that our proposed system creates a good video-viewing ending time and could support self-motivated video-viewing stops. Our second experiment targeted children and aimed to obtain children's reactions to the use of our developed system, as well as feedback from their parents. We observed the children's self-motivated video-viewing stops. In addition, we obtained good impressions of the system from parents.
SICE Journal of Control, Measurement, and System Integration
In this paper, we develop a text chat system to support graphical interaction for on-line group d... more In this paper, we develop a text chat system to support graphical interaction for on-line group discussion and investigate whether our graphical text chat system increases a speaker's feeling of conveying and a listener's feeling of understanding in on-line group discussion. For on-line communication, in particular at group discussion, text chat is a very popular and widely used communication tool. Not only text messages but also graphical ones are available in recent chat systems, and many graphical text chat systems exist. However, whether and how such graphical text chat systems contribute to both feelings of conveying and understanding between users remains unknown. In this study, we developed a graphical text chat system that enables users to exchange interpretations of the concepts contained in each other's text messages as images. Our experimental results demonstrate that the developed system increased both conveying and understanding feelings between users from both the speaker and listener viewpoints.
This paper proposes a multimodal interactive approach to improving recognition performance of obj... more This paper proposes a multimodal interactive approach to improving recognition performance of objects a person indicates to a robot. We considered two phenomena in human-human and human-robot interaction to design the approach: alignment and alignment inhibition. Alignment is a phenomenon that people tend to use the same words or gestures as their interlocutor uses; alignment inhibition is an opposite phenomenon, which people tend to decrease the amount of information in their words and gestures when their interlocutor uses excess information. Based on the phenomena, we designed robotic behavior policies that a robot should use enough information without being excessive to identify objects so that people would use similar information with the robot to refer to those objects, which would contribute to improve recognition performance. To verify our design, we developed a robotic system to recognize the objects to which people referred and conducted an experiment in which we manipulated the redundancy of information used in the confirmation behavior. The results showed that proposed approach improved recognition performance of objects to which referred by people.
We developed an autonomous human-like guide robot for a science museum. Its identifies individual... more We developed an autonomous human-like guide robot for a science museum. Its identifies individuals, estimates the exhibits at which visitors are looking, and proactively approaches them to provide explanations with gaze autonomously, using our new approach called speak-and-retreat interaction. The robot also performs such relation-building behaviors as greeting visitors by their names and expressing a friendlier attitude to repeat visitors. We conducted a field study in a science museum at which our system basically operated autonomously and the visitors responded quite positively. First-time visitors on average interacted with the robot for about 9 min, and 94.74% expressed a desire to interact with it again in the future. Repeat visitors noticed its relation-building capability and perceived a closer relationship with it.
This paper reports how robot-assisted language learning (RALL) impacts Japanese adults' English s... more This paper reports how robot-assisted language learning (RALL) impacts Japanese adults' English speaking skills. With existing research on RALL focusing on children, there is little evidence indicating RALL's effects on adults. We developed a RALL system comprising a robot, a tablet, and designed learning materials for speaking practice. To explore the effects of this system on the speaking skills of adults, we conducted a pretest and posttest designed pilot experiment. The participants in this study were nine female Japanese university students. The participants used the system for speaking practice for a total of 30 min per day for 7 days. The results of comparing the pretests with the posttests demonstrated that the adults' English speaking skills were enhanced; in particular, their speaking accuracy, fluency, and pronunciation improved greatly. This study contributes to providing clear evidence of the effects of RALL on Japanese adults' English speaking skills.
Journal of Advanced Computational Intelligence and Intelligent Informatics
In human-robot conversation in a real environment, low speech recognition and unnatural response ... more In human-robot conversation in a real environment, low speech recognition and unnatural response generation are critical issues. Most autonomous conversational robotic systems avoid these issues by restricting user input and robot responses. However, such restrictions often render the interaction boring because the conversation becomes predictable. In this study, we propose the use of multiple robots as a solution for this problem. To explore the effect of multiple robots on a conversation, we developed an autonomous conversational robotic system and conducted a field trial in a real event. Our system adopted a button interface, which restricted user input within positive or negative intention, and maintained a conversation by choosing the most suitable of the prepared static scenarios. Through the field trial, we found that visitors who conversed with multiple robots continued their conversation for a more prolonged period, and their experience improved their impression on the conv...
One of the most common cues in human relations is the reaction when someone approaches for a touc... more One of the most common cues in human relations is the reaction when someone approaches for a touch interaction. While the "before touch" distance has been investigated in daily life scenarios, it has not been studied for virtual environments. The measurements of a pre-touch distance in virtual reality can be applied to study social interactions especially for haptic interactions in virtual spaces where virtual agents interact autonomously with human participants. In the first stage of this study, we collected data to define a pre-touch distance when a virtual agent tries to touch the participant's face. On the basis of these results, we then classified participants into two groups based on preferred pre-touch distance: a "Near" group and a "Far" group. Next, we experimentally investigated the relationship between the participant's perception of an avatar's reaction to touch interaction and their preferred pre-touch distance. The results indicated that the participants felt friendliness to the agent who reacts with shorter pre-touch distance. We also found that the participant's pre-touch distance defined their preferences regarding the agent's behavior: those with a shorter pre-touch distance preferred agents with a closer interaction distance, and those with a longer pre-touch distance preferred agents with a longer interaction distance.
This paper investigates the effects of group interaction in a storytelling situation for children... more This paper investigates the effects of group interaction in a storytelling situation for children using two robots: a reader robot and a listener robot as a side-participant. We developed a storytelling system that consists of a reader robot, a listener robot, a display, a gaze model, a depth sensor, and a human operator who responds and provides easily understandable answers to the children’s questions. We experimentally investigated the effects of using a listener robot and either one or two children during a storytelling situation on the children’s preferences and their speech activities. Our experimental results showed that the children preferred storytelling with the listener robot. Although two children obviously produced more speech than one child, the listener robot discouraged the children’s speech regardless of whether one or two were listening.
In recent years, communication robots aiming to offer mental support to the elderly have attracte... more In recent years, communication robots aiming to offer mental support to the elderly have attracted increasing attention. Dialogue systems consisting of two robots could provide the elderly with opportunities to hold longer conversations in care homes. In this study, we conducted an experiment to compare two types of scenario-based dialogue systems with different types of bodies—physical and virtual robots—to investigate the effects of embodying such dialogue systems. Forty elderly people aged from 65 to 84 interacted with either an embodied desktop-sized humanoid robot or computer graphic agent displayed on a monitor. The elderly participants were divided into groups depending on the success of the interactions. The results revealed that (i) in the group where the robots responded more successfully with the expected conversation flow, the elderly are more engaged in the conversation with the physical robots than the virtual robots, and (ii) the elderly in the group in which robots r...
This paper reports the effects of communication cues on robot-initiated touch interactions at clo... more This paper reports the effects of communication cues on robot-initiated touch interactions at close distance by focusing on two factors: gaze-height for making eye contact and speech timing before and after touches. Although both factors are essential to achieve acceptable touches in human-human touch interaction, their effectiveness remains unknown in human-robot touch interaction contexts. To investigate the effects of these factors, we conducted an experiment whose results showed that being touched with before-touch timing is preferred to being touched with after-touch timing, although gaze-height did not significantly improve the feelings of robot-initiated touch.
Many social robots have emerged in public places to serve people. For these services, the robots ... more Many social robots have emerged in public places to serve people. For these services, the robots are assumed to be able to present internal aspects (i.e., mind, sociability) to engage and interact with people over the long term. In this paper, we propose a novel dialogue structure called experience-based dialogue to help a robot present and maintain a good interaction over the long term. This dialogue structure contains a piece of knowledge and a story about how the robot gained this knowledge, which are used to compose the robot's experience-related utterances for sharing experiences of interacting with previous users other than just the current user and help it present its internal aspects. We conducted an experiment to test the effects of our proposed dialogue structure and measure them with some published subjective scales. The results showed that experience-based dialogue can help a robot obtain better evaluations in terms of perceived intelligence, sociability, mind, anthropomorphism, animacy, likability, level of acceptance, and positive user reaction.
As agents, social robots are expected to increase opportunities for dialogue with the elderly. Ho... more As agents, social robots are expected to increase opportunities for dialogue with the elderly. However, it is difficult to sustain a dialogue with an elderly user because speech recognition frequently fails during the dialogue. Here, to overcome this problem, regardless of speech recognition failure, we developed a question–answer–response dialogue model. In this model, a robot took initiative in the dialogue by asking the user various questions. Moreover, to improve user experience during dialogue, we extended the model such that two robots could participate in the dialogue. Implementing these features, we conducted a field trial in a nursing home to evaluate the twin-robot dialogue system. The average word error rate of speech recognition was 0.778. Despite the frequently high number of errors, participants talked for 14 min in a dialogue with two robots and felt slightly strange during the dialogue. Although we found no significant difference between a dialogue with one robot and...
The recognition of indicated objects by interacting people is an essential function for robots th... more The recognition of indicated objects by interacting people is an essential function for robots that act in daily environments. However, due to ambiguous references by them, accurate recognition of indicated objects have difficulties for the robots. For example, people sometimes use the words which did not contain in the robots' databases, or they did not use enough words to identify the object. Therefore, to improve recognition accuracy, we must decrease such ambiguity of indicating behaviors of people. For this purpose, we experimentally compared two kinds of interaction strategies to decrease the ambiguity: explicitly providing requests to people about how to refer to objects, or implicitly aligning with people's indicating behaviors. The experimental results showed that participants evaluated the implicit strategy to be more natural than the explicit strategy, and the recognition performances of the two strategies were not significantly different.
SICE Journal of Control, Measurement, and System Integration
The popularization of video-viewing systems enables both adults and children to continuously watc... more The popularization of video-viewing systems enables both adults and children to continuously watch countless video clips. However, such extended video viewing might cause health problems, particularly for children. Rule-making tendencies are weaker for video-viewing systems than they are for television programming. Children struggle with selfmotivated video-viewing stops because of attractive, rich clips. In this study, we propose a video recommendation system that arranges video clips based on a pre-set time, to support self-motivated video-viewing stops. Our proposed system enables parents to limit video-viewing time in advance and provides video clips that are arranged to finish exactly at a pre-set time. We conducted two experiments. The first experiment targeted adults to confirm the effectiveness of our approach. Our results from the first experiment suggest that our proposed system creates a good video-viewing ending time and could support self-motivated video-viewing stops. Our second experiment targeted children and aimed to obtain children's reactions to the use of our developed system, as well as feedback from their parents. We observed the children's self-motivated video-viewing stops. In addition, we obtained good impressions of the system from parents.
SICE Journal of Control, Measurement, and System Integration
In this paper, we develop a text chat system to support graphical interaction for on-line group d... more In this paper, we develop a text chat system to support graphical interaction for on-line group discussion and investigate whether our graphical text chat system increases a speaker's feeling of conveying and a listener's feeling of understanding in on-line group discussion. For on-line communication, in particular at group discussion, text chat is a very popular and widely used communication tool. Not only text messages but also graphical ones are available in recent chat systems, and many graphical text chat systems exist. However, whether and how such graphical text chat systems contribute to both feelings of conveying and understanding between users remains unknown. In this study, we developed a graphical text chat system that enables users to exchange interpretations of the concepts contained in each other's text messages as images. Our experimental results demonstrate that the developed system increased both conveying and understanding feelings between users from both the speaker and listener viewpoints.
This paper proposes a multimodal interactive approach to improving recognition performance of obj... more This paper proposes a multimodal interactive approach to improving recognition performance of objects a person indicates to a robot. We considered two phenomena in human-human and human-robot interaction to design the approach: alignment and alignment inhibition. Alignment is a phenomenon that people tend to use the same words or gestures as their interlocutor uses; alignment inhibition is an opposite phenomenon, which people tend to decrease the amount of information in their words and gestures when their interlocutor uses excess information. Based on the phenomena, we designed robotic behavior policies that a robot should use enough information without being excessive to identify objects so that people would use similar information with the robot to refer to those objects, which would contribute to improve recognition performance. To verify our design, we developed a robotic system to recognize the objects to which people referred and conducted an experiment in which we manipulated the redundancy of information used in the confirmation behavior. The results showed that proposed approach improved recognition performance of objects to which referred by people.
We developed an autonomous human-like guide robot for a science museum. Its identifies individual... more We developed an autonomous human-like guide robot for a science museum. Its identifies individuals, estimates the exhibits at which visitors are looking, and proactively approaches them to provide explanations with gaze autonomously, using our new approach called speak-and-retreat interaction. The robot also performs such relation-building behaviors as greeting visitors by their names and expressing a friendlier attitude to repeat visitors. We conducted a field study in a science museum at which our system basically operated autonomously and the visitors responded quite positively. First-time visitors on average interacted with the robot for about 9 min, and 94.74% expressed a desire to interact with it again in the future. Repeat visitors noticed its relation-building capability and perceived a closer relationship with it.
This paper reports how robot-assisted language learning (RALL) impacts Japanese adults' English s... more This paper reports how robot-assisted language learning (RALL) impacts Japanese adults' English speaking skills. With existing research on RALL focusing on children, there is little evidence indicating RALL's effects on adults. We developed a RALL system comprising a robot, a tablet, and designed learning materials for speaking practice. To explore the effects of this system on the speaking skills of adults, we conducted a pretest and posttest designed pilot experiment. The participants in this study were nine female Japanese university students. The participants used the system for speaking practice for a total of 30 min per day for 7 days. The results of comparing the pretests with the posttests demonstrated that the adults' English speaking skills were enhanced; in particular, their speaking accuracy, fluency, and pronunciation improved greatly. This study contributes to providing clear evidence of the effects of RALL on Japanese adults' English speaking skills.
Journal of Advanced Computational Intelligence and Intelligent Informatics
In human-robot conversation in a real environment, low speech recognition and unnatural response ... more In human-robot conversation in a real environment, low speech recognition and unnatural response generation are critical issues. Most autonomous conversational robotic systems avoid these issues by restricting user input and robot responses. However, such restrictions often render the interaction boring because the conversation becomes predictable. In this study, we propose the use of multiple robots as a solution for this problem. To explore the effect of multiple robots on a conversation, we developed an autonomous conversational robotic system and conducted a field trial in a real event. Our system adopted a button interface, which restricted user input within positive or negative intention, and maintained a conversation by choosing the most suitable of the prepared static scenarios. Through the field trial, we found that visitors who conversed with multiple robots continued their conversation for a more prolonged period, and their experience improved their impression on the conv...
Uploads
Papers by Takamasa Iio