Talking with Conversational Agents in Collaborative Action

Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 2021

More and more voice-user interfaces (VUIs), such as smart speakers like Amazon Alexa or social robots like Jibo or Cozmo, are entering multi-user environments including homes. VUIs can utilize multimodal cues such as graphics, expressive sounds, and movement to convey social engagement, afecting how users perceive agents as social others. Reciprocal relationships with VUIs, i.e., relationships with give-and-take between the VUI and user, are of key interest as they are more likely to foster rapport and emotional engagement, and lead to successful collaboration. Through an elicitation study with three commercially available VUIs, we explore small group interactions (n = 33 participants) focused on the behaviors participants display to various VUIs to understand (1) reciprocal interactions between VUIs and participants and among small groups and (2) how participants engage with VUIs as the interface's embodiment becomes more socially capable. The discussion explores (1) theories of sociability applied to the users' behaviors seen with the VUIs, and (2) the group contexts where VUIs that build reciprocal relationships with users can become a powerful persuasive technology and a collaborative companion. We conclude the discussion with recommendations for promoting reciprocity from participants and, therefore, fostering rapport and emotional engagement in VUI interactions. • Human-centered computing → User studies; Natural language interfaces.

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

Personal assistants using a command-dialogue model of speech recognition, such as Siri and Cortana, have become increasingly powerful and popular for individual use. In this paper we explore whether similar techniques could be used to create a speech-based agent system which, in a group meeting setting, would similarly monitor spoken dialogue, pro-actively detect useful actions, and carry out those actions without specific commands being spoken. Using a low-fi technical probe, we investigated how such a system might perform in the collaborative work setting and how users might respond to it. We recorded and transcribed a varied set of nine meetings from which we generated simulated lists of automated 'action items', which we then asked the meeting participants to review retrospectively. The low rankings given on these discovered items are suggestive of the difficulty in applying personal assistant technology to the group setting, and we document the issues emerging from the study. Through observations, we explored the nature of meetings and the challenges they present for speech agents.

International Journal of Speech Technology, 1998

This paper proposes a method of designing human-computer speech interfaces based on principles of human conversation. It argues that conversation is the primary mode of language use and that it is fundamentally collaborative. Speech interfaces should therefore be designed to recreate the collaborative nature of natural conversations. The paper presents five strategies for designing collaborative conversational intelfaces, and it describes the principles of human-language use that underly these strategies. The paper also argues that collaborative conversational interfaces have a crucial advantage over other kinds of interfaces in that they are readily adaptive to different levels of experience and styles of use. The paper gives examples of collaborative conversational interfaces that we have developed, and discusses the ways in which these interfaces have been made adaptive.

The burgeoning field of speech–based user interfaces, pushed heavily by most major technology vendors, including Google (with Assistant), Apple (Siri), Amazon (Alexa), and Microsoft (Cortana), presents a new challenge in designing end user experiences; one where we cannot rely on there being a visual element at all. With the individual interests of the vendors, we have seen a growth of very distinct guidelines and platforms, resulting in a lack of consistency across the field. There is an opportunity to consider these platforms and this modality of interaction, and how we can design for it more generally. By reviewing the current array of literature on voice and conversational user interfaces, as well as general speech and user interface metaphors, an understand- ing and framing for the potential of this field is to be achieved. The different core vendors and their corporate attitudes and business goals are examined to find issues that may affect building for them. Thematic analysi...

Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing

This virtual workshop seeks to bring together the burgeoning communities centred on the design, development, application, and study of so-called Conversational User Interfaces (CUIs). CUIs are used in myriad contexts, from online support chatbots through to entertainment devices in the home. In this workshop, we will examine the challenges involved in transforming CUIs into everyday computing devices capable of supporting collaborative activities across space and time. Additionally, this workshop seeks to establish a cohesive CUI community and research agenda within CSCW. We will examine the roles in which CSCW research can contribute insights into understanding how CUIs are or can be used in a variety of settings, from public to private, and how they can be brought into a potentially unlimited number of tasks. This proposed workshop will bring together researchers from academia and practitioners from industry to survey the state-of-the-art in terms of CUI design, use, and understanding, and will map new areas for work including addressing the technical, social, and ethical challenges that lay ahead. By bringing together existing researchers and new ideas in this space, we intend to foster a strong community and enable potential future collaborations. CCS CONCEPTS • Human-centered computing → Natural language interfaces; Collaborative and social computing systems and tools.

CUI 2021 - 3rd Conference on Conversational User Interfaces

Although smart speakers support increasingly complex multi-turn dialogues, they still play a mostly reactive role, responding to user's questions or requests. With rapid technological advances, they are becoming more capable of initiating conversations by themselves. However, before developing such proactive features, it is important to understand how people perceive different types of agent-initiated interactions. We conducted an online survey in which participants (= 47) rated 8 scenarios around proactive smart speakers on different aspects. Despite some controversy around proactive systems,

Lecture Notes in Computer Science, 2002

The development of speech tools suitable for use in real world environments requires collaboration between computational linguistics and new implementation fields e.g. robotics, and the incorporation of new AI techniques to improve overall system performance. In this paper we present the core development concepts of SAID (Speaking Autonomous Intelligent Devices). The work presented centres around four key strands of research, namely the recasting of the Time Map model as a Multi-Agent System (MAS), the development of a MAS based audio feature extraction system, the deployment of a BDI based dialog agent and the design of a MAS based social robot architecture.

2012

We report about design and characteristics of the LAST MINUTE corpus. The recordings in this data collection are taken from a WOZ experiment that allows to investigate how users interact with a companion system in a mundane situation with the need for planning, re-planning and strategy change. The resulting corpus is distinguished with respect to aspects of size (e.g. number of subjects, length of sessions, number of channels, total length of records) as well as quality (e.g. balancedness of cohort, well designed scenario, standard based transcripts, psychological questionnaires, accompanying in-depth interviews).

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers, 2021

Despite the increasing popularity of novel conversational services for smart speakers, the current ones have supported limited proactive interactions. In this work, we focus to understand opportune moments for proactive conversational interactions in domestic contexts. To achieve our goal, we built a speech-based experience sampling device and performed a 1-week field study with 40 students living in university dormitories. From 3,572 in-situ experiences, we identified eleven categories to find contextual features related to opportune moments. We showed that the key determinants for interruptibility are relevant to personal contextual factors, user mobility, and social presence. Through considering the aforementioned factors, we envision that a smart speaker can intelligently manage the timing of conversations. CCS CONCEPTS • Human-centered computing → Ubiquitous and mobile computing; User interface management systems.