Papers by Takashi Yamauchi
Proceedings of the 2021 International Conference on Multimodal Interaction, 2021
This paper explores the psycho-physiological impact of remote touch in conjunction with conversat... more This paper explores the psycho-physiological impact of remote touch in conjunction with conversations shared over video telecommunication. Interlocutors sent upper arm squeezes to each other through paired touch input and output devices in real time. Committed couples were recruited to use the devices in semi-structured discussions after they watched a video clip that contained emotionally charged moments. Users were not told how to use the devices and were free to define the purpose of their use. We examined how remote touch may impact skin conductance and heart rate variability during emotionally charged conversations. Our results revealed that irrespective of speech sentiment, co-verbal touch reduced arousal when compared to discourse without touch. However, touch was used more frequently in utterances that were judged negative and its reduction in arousal was more pronounced than co-verbal touch in positive utterances. These findings suggest that remote touch can be used to attenuate negative conversations or similar circumstances. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI).
Sensors, 2020
This article explores the affective impact of remote touch when used in conjunction with video te... more This article explores the affective impact of remote touch when used in conjunction with video telecon. Committed couples were recruited to engage in semi-structured discussions after they watched a video clip that contained emotionally charged moments. They used paired touch input and output devices to send upper-arm squeezes to each other in real-time. Users were not told how to use the devices and were free to define the purpose of their use. We examined how remote touch was used and its impact on skin conductance and affective response. We observed 65 different touch intents, which were classified into broader categories. We employed a series of analyses within a framework of behavioral and experiential timescales. Our findings revealed that remote touches created a change in the overall psychological affective experience and skin conductance response. Only remote touches that were judged to be affective elicited significant changes in EDA measurements. Our study demonstrates th...
Mathematics, 2018
Using a multisensory interface system, we examined how people’s emotional experiences change as t... more Using a multisensory interface system, we examined how people’s emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense (“seeing” their touch). Our system (the Interactive Plant system) senses the electrical capacitance of the human body and visualizes users’ tactile information on a flat screen (when the touch is gentle, the program draws small and thin roots around the pot; when the touch is more harsh or abrupt, big and thick roots are displayed). We contrasted this multimodal combination (touch + vision) with a unimodal interface (touch only or watch only) and measured the impact of the multimodal interaction on participants’ emotion. We found significant emotional gains in the multimodal interaction. Participants’ self-reported positive affect, joviality, attentiveness and self-assurance increased dramatically in multimodal interaction relative to unimodal interaction; participants’ electrodermal activity (EDA) increased in the mul...
PLOS ONE, 2019
The accurate detection of attention-deficit/hyperactivity disorder (ADHD) symptoms, such as inatt... more The accurate detection of attention-deficit/hyperactivity disorder (ADHD) symptoms, such as inattentiveness and behavioral disinhibition, is crucial for delivering timely assistance and treatment. ADHD is commonly diagnosed and studied with specialized questionnaires and behavioral tests such as the stop-signal task. However, in cases of late-onset or mild forms of ADHD, behavioral measures often fail to gauge the deficiencies well-highlighted by questionnaires. To improve the sensitivity of behavioral tests, we propose a novel version of the stop-signal task (SST), which integrates mouse cursor tracking. In two studies, we investigated whether introducing mouse movement measures to the stop-signal task improves associations with questionnaire-based measures, as compared to the traditional (keypressbased) version of SST. We also scrutinized the influence of different parameters of stop-signal tasks, such as the method of stop-signal delay setting or definition of response inhibition failure, on these associations. Our results show that a) SSRT has weak association with impulsivity, while mouse movement measures have strong and significant association with impulsivity; b) machine learning models trained on the mouse movement data from "known" participants using nested cross-validation procedure can accurately predict impulsivity ratings of "unknown" participants; c) mouse movement features such as maximum acceleration and maximum velocity are among the most important predictors for impulsivity; d) using preset stop-signal delays prompts behavior that is more indicative of impulsivity.
International Journal of Human-Computer Interaction, 2016
The Editors would like to thank the following external reviewers who, in addition to the members ... more The Editors would like to thank the following external reviewers who, in addition to the members of the Editorial Advisory Board, have kindly agreed to review manuscripts submitted to the IJHCI during 2015. All reviewers are recognized for having contributed towards the highest possible quality of the journal and their efforts are very much appreciated.
The reversed association theory (Dunn & Kirsner, 1988) provides a powerful procedure for studying... more The reversed association theory (Dunn & Kirsner, 1988) provides a powerful procedure for studying the link between cognitive processes and task performance. It helps find whether two behavioral tasks involve the same or different cognitive processes. However, this theory has not been fully utilized. Finding reversed association requires a large, single study consisting of at least six independent manipulations. Furthermore, the statistical procedure to verify reversed association has not been fully developed. This article presents practical solutions for these problems by investigating the recent controversy over categorization and similarity judgments. First, a contrast analysis is illustrated in a case study to statistically verify reversed association. Second, empirical experiments and computer simulations are presented to verify the reliability of the reversed association test. Combined together, this study reveals that there is a non- monotonic relationship (i.e., reversed asso...
Journal of Experimental Psychology: Learning, Memory, and Cognition, 2005
PSYCHOLOGIA, 2008
When a person is characterized categorically with a noun label (e.g., Linda is a feminist), peopl... more When a person is characterized categorically with a noun label (e.g., Linda is a feminist), people tend to think that the attributes associated with that person are central and long-lasting (i.e., labeling effect). This bias, which is related to category-based induction and social misattributions such as stereotyping, has been known to occur because we associate the person with prototypical attributes represented in the category. One experiment described in this article indicates that the labeling effect can occur separately from the attributes represented in the category. The experiment suggests that labeling bolsters not only the perception of prototypical attributes but also the awareness of unrelated or even irrelevant attributes. The results from the experiments suggest that some generic information inherent in noun labels play a crucial role in category-based reasoning.
Lecture Notes in Computer Science, 2007
For the development of Semantic Web technology, researchers and developers in the Semantic Web co... more For the development of Semantic Web technology, researchers and developers in the Semantic Web community need to focus on the areas in which human reasoning is particularly difficult. Two studies in this paper demonstrate that people are predisposed to use class-inclusion labels for inductive judgments. This tendency appears to stem from a general characteristic of human reasoning-using heuristics to solve problems. The inference engines and interface designs that incorporate human reasoning need to integrate this general characteristic underlying human induction.
When two objects carry the same category label, we tend to perceive that these objects are simila... more When two objects carry the same category label, we tend to perceive that these objects are similar. When two objects carry different category labels, we perceive that these objects are dissimilar. How does this happen? In an attempt to clarify the effect of category labels on similarity judgments, pictures of animal tissues were presented with fictitious labels and participants judged the similarity of the pictures. The perceived similarity increased when the labels highlighted the interrelatedness of features; the effect of labels was absent when the labels did not highlight the interrelatedness of features. The results indicate that category labels help clarify inter-relationships of features, and modify our perception of similarity. When two objects carry the same category labels, people tend to think that they have features in common. When they carry different labels, people perceive that these objects have different features. How does this happen? One theory claims that verbal ...
Though a traditional assumption in similarity judgment is that people selectively attend to certa... more Though a traditional assumption in similarity judgment is that people selectively attend to certain features, few studies have explored the actual method of identifying “salient features” in visual stimuli. In this study, we used complex, realistic images to examine whether people selectively process salient features. Stimuli were triads of original and morphed animal face pictures. In the behavioral experiment, participants viewed two original pictures and a morphed composite of the originals and decided which original picture was more similar to the morphed picture. In the computational analysis, we employed Gabor function and wavelets, Gray-Level Cooccurrence Metrics (GLCM) combined with principal component analysis to extract candidate visual features, such as Gabor texture, brightness, size, and contour for the entire face as well as parts of the face. The simulated annealing algorithm was applied to behavioral data to determine possible weight distributions for the candidate f...
The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavio... more The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior - task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals - gaze motion, electroencephalogram (E...
The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavio... more The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior—task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals—gaze motion, electroencephalogram (EEG),...
Frontiers in psychology, 2018
Attention deficit/hyperactivity disorder (ADHD) is frequently characterized as a disorder of exec... more Attention deficit/hyperactivity disorder (ADHD) is frequently characterized as a disorder of executive function (EF). However, behavioral tests of EF, such as go/No-go tasks, often fail to grasp the deficiency in EF revealed by questionnaire-based measures. This inability is usually attributed to questionnaires and behavioral tasks assessing different constructs of EFs. We propose an additional explanation for this discrepancy. We hypothesize that this problem stems from the lack of assessment of decision-making (e.g., continuous monitoring of motor behavior such as velocity and acceleration in choice reaching) in classical versions of behavioral tasks. We test this hypothesis by introducing dynamic assessment in the form of mouse motion in a go/No-go task. Our results indicate that, among healthy college students, self-report measures of ADHD symptoms become strongly associated with performance in behavioral tasks when continuous assessment (e.g., acceleration in the mouse-cursor m...
Cognitive science, Jan 13, 2017
Affective computing research has advanced emotion recognition systems using facial expressions, v... more Affective computing research has advanced emotion recognition systems using facial expressions, voices, gaits, and physiological signals, yet these methods are often impractical. This study integrates mouse cursor motion analysis into affective computing and investigates the idea that movements of the computer cursor can provide information about emotion of the computer user. We extracted 16-26 trajectory features during a choice-reaching task and examined the link between emotion and cursor motions. Participants were induced for positive or negative emotions by music, film clips, or emotional pictures, and they indicated their emotions with questionnaires. Our 10-fold cross-validation analysis shows that statistical models formed from "known" participants (training data) could predict nearly 10%-20% of the variance of positive affect and attentiveness ratings of "unknown" participants, suggesting that cursor movement patterns such as the area under curve and dir...
PloS one, 2017
Recent evidence suggests that top-down attention facilitates unconscious semantic processing. To ... more Recent evidence suggests that top-down attention facilitates unconscious semantic processing. To clarify the role of attention in unconscious semantic processing, we traced trajectories of the computer mouse in a semantic priming task and scrutinized the extent to which top-down attention enhances unconscious semantic processing in four different stimulus-onset asynchrony (SOA: 50, 200, 500, or 1000ms) conditions. Participants judged whether a target digit (e.g., "6") was larger or smaller than five, preceded by a masked priming digit (e.g., "9"). The pre-prime duration changed randomly from trial to trial to disrupt participants' top-down attention in an uncued condition (in a cued condition, a green square cue was presented to facilitate participants' top-down attention). The results show that top-down attention modifies the time course of subliminal semantic processing, and the temporal attention window lasts more than 1000ms; attention facilitated by ...
Cognition and Emotion, 2016
Nonlinguistic signals in the voice and musical instruments play a critical role in communicating ... more Nonlinguistic signals in the voice and musical instruments play a critical role in communicating emotion. Although previous research suggests a common mechanism for emotion processing in music and speech, the precise relationship between the two domains is unclear due to the paucity of direct evidence. By applying the adaptation paradigm developed by Bestelmeyer, Rouger, DeBruine, and Belin [2010. Auditory adaptation in vocal affect perception. Cognition, 117(2), 217-223.
2015 International Conference on Affective Computing and Intelligent Interaction (ACII), 2015
The perception of emotion is critical for social interactions. Nonlinguistic signals such as thos... more The perception of emotion is critical for social interactions. Nonlinguistic signals such as those in the human voice and musical instruments are used for communicating emotion. Using an adaptation paradigm, this study examines the extent to which common mental mechanisms are applied for emotion processing of instrumental and vocal sounds. In two experiments we show that prolonged exposure to affective nonlinguistic vocalizations elicits auditory after effects when participants are tested on instrumental morphs (Experiment 1a), yet no aftereffects are apparent when participants are exposed to affective instrumental sounds and tested on non-linguistic voices (Experiment 1b). Specifically, results indicate that exposure to angry vocal sounds made participants perceive instrumental sounds as angrier and less fearful, but not vice versa. These findings suggest that there is a directionality for emotion perception in vocal and instrumental sounds. Significantly, this unidirectional relationship reveals that mechanisms used for emotion processing is likely to be shared from vocal sounds to instrumental sounds, but not vice versa.
Consciousness and cognition, Jan 15, 2015
The role of attention in subliminal semantic priming remains controversial: some researchers argu... more The role of attention in subliminal semantic priming remains controversial: some researchers argue that attention is necessary for subliminal semantic priming, while others suggest that subliminal semantic processing is free from the influence of attention. The present study employs a cursor motion method to measure priming and evaluate the influence of attention. Specifically, by employing a semantic priming task developed by Naccache, Blandin, and Dehaene (2002), we investigate the extent to which top-down attention influences semantic priming. Results indicate that, consistent with the Naccache et al. (2002) results, attention facilitates priming. However, inconsistent with their theory, significant priming is still observed even in near absence of attention. We suggest that top-down attention helps but is not necessary for subliminal semantic processing.
Uploads
Papers by Takashi Yamauchi