The mallet percussion instruments are gaining attention in modern music arrangements, especially ... more The mallet percussion instruments are gaining attention in modern music arrangements, especially in movie soundtracks and synthesized furniture music. Using a combination of these instruments, a reasonable variety of timbres and pitches are possible. Players can further choose mallets of different makes, which in turn significantly alter the timbre of the sound produced. How can these sounds be used to suggest various moods and emotions in musical compositions? This study compares the sounds of the marimba, vibraphone, xylophone, and glockenspiel, with different mallet hardness over various pitch registers, and ten emotional
Recent research has shown that different musical instrument sounds have strong emotional characte... more Recent research has shown that different musical instrument sounds have strong emotional characteristics. It has also shown how these emotional characteristics change with different pitch and dynamics for the piano and solo bowed strings. This paper investigates how the emotional characteristics of the section bowed strings vary with pitch and dynamics, comparing and contrasting the solo and section string results. We conducted listening tests where listeners compared the section string sounds pairwise over ten emotional categories. The section and solo string results were somewhat similar overall, but with some notable differences as well. The emotional characteristics Happy, Heroic, Romantic, Comic, Calm, and Shy generally increased with pitch in an arching shape that peaked at C5 and decreased at the highest pitches. The characteristics Angry and Sad generally decreased with pitch. Scary was somewhat U-shaped and especially strong in the extreme high register. In terms of dynamics, the results showed that Happy, Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Shy, and Sad were stronger for soft notes. These results provide audio engineers and musicians with possible suggestions for emphasizing various emotional characteristics of the section strings in orchestral recordings and performances.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the emotional characteristics were even more clearly distinguished for 250 ms tones. We also found that the density of significant harmonics, a feature that has particular relevance to non-sustaining tones, was observed to achieve especially strong correlation for all tested emotion categories in these tones with early release.
The goals of this paper are the following: (1) To determine the failure rates using Major and Min... more The goals of this paper are the following: (1) To determine the failure rates using Major and Minor as a predictor for positive and negative moods, respectively, (2) To determine the musical features that contributed to making an example a fail case and (3) To determine whether there are significant differences between Natural, Melodic and Harmonic Minor. We addressed these issues by conducting a listening test and manually analysing fail cases among 252 low-arousal classical piano excerpts. We found that Major-Minor categorisation was 90% accurate and high-register Natural Minor excerpts accounted for nearly half of the residual fail cases.
Journal of The Audio Engineering Society, Apr 2, 2015
Recent research has shown the connection between timbre and music emotion and has focused on sust... more Recent research has shown the connection between timbre and music emotion and has focused on sustaining instruments. This paper undertakes listening tests to investigate the connection for nonsustaining instruments including plucked string, pitched percussion, and keyboard sounds compared pairwise by listening test subjects over eight emotional categories. Different durations were studied to investigate the effect of early release on the emotional characteristics. The experiments showed that the harpsichord, marimba, vibraphone, and xylophone were highly rated for positive emotional characteristics; the guitar, harp, and plucked violin were highly rated for negative emotional characteristics; and the piano was rated emotionally neutral. There were differences in emotional characteristics due to early release, such as the harpsichord decreasing in ranking for positive emotional characteristics on sounds with early releases. By correlating timbral features with the listener rankings, we found that decay slope and density of significant harmonics were significant timbral features for many emotional characteristics.
Journal of The Audio Engineering Society, Apr 28, 2017
Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with... more Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with categories such as Happy, Sad, Angry, and Calm. Previous research has shown that both listeners and automated systems often have difficulty distinguishing low-arousal categories such as Calm and Sad. This paper seeks to explore what makes the categories Calm and Sad so difficult to distinguish. We used 300 low-arousal excerpts from the classical piano repertoire to determine the coverage of the categories Calm and Sad in the low-arousal space, their overlap, and their balance to one another. Our results show that Calm was 40% bigger in terms of coverage than Sad, but that on average Sad excerpts were significantly more negative in mood than Calm excerpts were positive. Calm and Sad overlapped in nearly 20% of the excerpts, meaning 20% of the excerpts were about equally Calm and Sad. Calm and Sad covered about 92% of the low-arousal space, where 8% of the space were holes that were not-at-all Calm or Sad. The largest holes were for excerpts considered Mysterious and Doubtful, but there were smaller holes among positive excerpts as well. Due to the holes in the coverage, the overlaps, and imbalances the Calm-Sad model adds about 6% more errors when compared to asking users directly whether the mood of the music is positive or negative. Nevertheless, the Calm-Sad model is still useful and appropriate for applications in music emotion recognition and recommendation such as when a simple and intuitive interface is preferred or when categorization is more important than precise differentiation.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
Journal of The Audio Engineering Society, Dec 1, 2016
The piano is an instrument extensively used in classical, jazz, and pop music since its broad pit... more The piano is an instrument extensively used in classical, jazz, and pop music since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of isolated one-second piano sounds. Listeners compared the sounds pairwise over ten emotion categories. The results showed that all ten emotional categories were significantly affected by pitch and nine of them by dynamics. In particular, the emotional characteristics Happy, Romantic, Comic, Calm, Mysterious, and Shy generally increased with pitch but sometimes decreased at the highest pitches. The characteristics Heroic, Angry, and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers. In terms of dynamics, the results showed that the characteristics Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Mysterious, Shy, and Sad were stronger for soft notes. Surprisingly, Happy was not affected by dynamics. These results help quantify the emotional characteristics of piano sounds.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the emotional characteristics were even more clearly distinguished for 250 ms tones. We also found that the density of significant harmonics, a feature that has particular relevance to non-sustaining tones, was observed to achieve especially strong correlation for all tested emotion categories in these tones with early release.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
The mallet percussion instruments are gaining attention in modern music arrangements, especially ... more The mallet percussion instruments are gaining attention in modern music arrangements, especially in movie soundtracks and synthesized furniture music. Using a combination of these instruments, a reasonable variety of timbres and pitches are possible. Players can further choose mallets of different makes, which in turn significantly alter the timbre of the sound produced. How can these sounds be used to suggest various moods and emotions in musical compositions? This study compares the sounds of the marimba, vibraphone, xylophone, and glockenspiel, with different mallet hardness over various pitch registers, and ten emotional
Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with... more Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with categories such as Happy, Sad, Angry, and Calm. Previous research has shown that both listeners and automated systems often have difficulty distinguishing low-arousal categories such as Calm and Sad. This paper seeks to explore what makes the categories Calm and Sad so difficult to distinguish. We used 300 low-arousal excerpts from the classical piano repertoire to determine the coverage of the categories Calm and Sad in the low-arousal space, their overlap, and their balance to one another. Our results show that Calm was 40% bigger in terms of coverage than Sad, but that on average Sad excerpts were significantly more negative in mood than Calm excerpts were positive. Calm and Sad overlapped in nearly 20% of the excerpts, meaning 20% of the excerpts were about equally Calm and Sad. Calm and Sad covered about 92% of the low-arousal space, where 8% of the space were holes that were not-at-all Calm or Sad. The largest holes were for excerpts considered Mysterious and Doubtful, but there were smaller holes among positive excerpts as well. Due to the holes in the coverage, the overlaps, and imbalances the Calm-Sad model adds about 6% more errors when compared to asking users directly whether the mood of the music is positive or negative. Nevertheless, the Calm-Sad model is still useful and appropriate for applications in music emotion recognition and recommendation such as when a simple and intuitive interface is preferred or when categorization is more important than precise differentiation.
The piano is an instrument extensively used in classical, jazz, and pop music since its broad pit... more The piano is an instrument extensively used in classical, jazz, and pop music since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of isolated one-second piano sounds. Listeners compared the sounds pairwise over ten emotion categories. The results showed that all ten emotional categories were significantly affected by pitch and nine of them by dynamics. In particular, the emotional characteristics Happy, Romantic, Comic, Calm, Mysterious, and Shy generally increased with pitch but sometimes decreased at the highest pitches. The characteristics Heroic, Angry, and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers. In terms of dynamics, the results showed that the characteristics Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Mysterious, Shy, and Sad were stronger for soft notes. Surprisingly, Happy was not affected by dynamics. These results help quantify the emotional characteristics of piano sounds.
Previous research has shown that different musical instrument sounds have strong emotional charac... more Previous research has shown that different musical instrument sounds have strong emotional characteristics. This paper investigates how emotional characteristics vary with pitch and dynamics within the bowed string instrument family. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of the violin, viola, cello, and double bass. Listeners compared the sounds pairwise over ten emotional categories. The results showed that the emotional characteristics Happy, Heroic, Romantic, Comic, and Calm generally increased with pitch, but decreased at the highest pitches. The characteristics Angry and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers, while Shy was relatively unaffected by pitch. In terms of dynamics, the results showed that Heroic, Comic, and Angry were stronger for loud notes, while Romantic, Calm, Shy, and Sad were stronger for soft notes. Surprisingly, Scary was least affected by dynamics. These results provide audio engineers and musicians with possible suggestions for emphasizing various emotional characteristics of the bowed strings in sound recordings and performances.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the...
The mallet percussion instruments are gaining attention in modern music arrangements, especially ... more The mallet percussion instruments are gaining attention in modern music arrangements, especially in movie soundtracks and synthesized furniture music. Using a combination of these instruments, a reasonable variety of timbres and pitches are possible. Players can further choose mallets of different makes, which in turn significantly alter the timbre of the sound produced. How can these sounds be used to suggest various moods and emotions in musical compositions? This study compares the sounds of the marimba, vibraphone, xylophone, and glockenspiel, with different mallet hardness over various pitch registers, and ten emotional
Recent research has shown that different musical instrument sounds have strong emotional characte... more Recent research has shown that different musical instrument sounds have strong emotional characteristics. It has also shown how these emotional characteristics change with different pitch and dynamics for the piano and solo bowed strings. This paper investigates how the emotional characteristics of the section bowed strings vary with pitch and dynamics, comparing and contrasting the solo and section string results. We conducted listening tests where listeners compared the section string sounds pairwise over ten emotional categories. The section and solo string results were somewhat similar overall, but with some notable differences as well. The emotional characteristics Happy, Heroic, Romantic, Comic, Calm, and Shy generally increased with pitch in an arching shape that peaked at C5 and decreased at the highest pitches. The characteristics Angry and Sad generally decreased with pitch. Scary was somewhat U-shaped and especially strong in the extreme high register. In terms of dynamics, the results showed that Happy, Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Shy, and Sad were stronger for soft notes. These results provide audio engineers and musicians with possible suggestions for emphasizing various emotional characteristics of the section strings in orchestral recordings and performances.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the emotional characteristics were even more clearly distinguished for 250 ms tones. We also found that the density of significant harmonics, a feature that has particular relevance to non-sustaining tones, was observed to achieve especially strong correlation for all tested emotion categories in these tones with early release.
The goals of this paper are the following: (1) To determine the failure rates using Major and Min... more The goals of this paper are the following: (1) To determine the failure rates using Major and Minor as a predictor for positive and negative moods, respectively, (2) To determine the musical features that contributed to making an example a fail case and (3) To determine whether there are significant differences between Natural, Melodic and Harmonic Minor. We addressed these issues by conducting a listening test and manually analysing fail cases among 252 low-arousal classical piano excerpts. We found that Major-Minor categorisation was 90% accurate and high-register Natural Minor excerpts accounted for nearly half of the residual fail cases.
Journal of The Audio Engineering Society, Apr 2, 2015
Recent research has shown the connection between timbre and music emotion and has focused on sust... more Recent research has shown the connection between timbre and music emotion and has focused on sustaining instruments. This paper undertakes listening tests to investigate the connection for nonsustaining instruments including plucked string, pitched percussion, and keyboard sounds compared pairwise by listening test subjects over eight emotional categories. Different durations were studied to investigate the effect of early release on the emotional characteristics. The experiments showed that the harpsichord, marimba, vibraphone, and xylophone were highly rated for positive emotional characteristics; the guitar, harp, and plucked violin were highly rated for negative emotional characteristics; and the piano was rated emotionally neutral. There were differences in emotional characteristics due to early release, such as the harpsichord decreasing in ranking for positive emotional characteristics on sounds with early releases. By correlating timbral features with the listener rankings, we found that decay slope and density of significant harmonics were significant timbral features for many emotional characteristics.
Journal of The Audio Engineering Society, Apr 28, 2017
Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with... more Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with categories such as Happy, Sad, Angry, and Calm. Previous research has shown that both listeners and automated systems often have difficulty distinguishing low-arousal categories such as Calm and Sad. This paper seeks to explore what makes the categories Calm and Sad so difficult to distinguish. We used 300 low-arousal excerpts from the classical piano repertoire to determine the coverage of the categories Calm and Sad in the low-arousal space, their overlap, and their balance to one another. Our results show that Calm was 40% bigger in terms of coverage than Sad, but that on average Sad excerpts were significantly more negative in mood than Calm excerpts were positive. Calm and Sad overlapped in nearly 20% of the excerpts, meaning 20% of the excerpts were about equally Calm and Sad. Calm and Sad covered about 92% of the low-arousal space, where 8% of the space were holes that were not-at-all Calm or Sad. The largest holes were for excerpts considered Mysterious and Doubtful, but there were smaller holes among positive excerpts as well. Due to the holes in the coverage, the overlaps, and imbalances the Calm-Sad model adds about 6% more errors when compared to asking users directly whether the mood of the music is positive or negative. Nevertheless, the Calm-Sad model is still useful and appropriate for applications in music emotion recognition and recommendation such as when a simple and intuitive interface is preferred or when categorization is more important than precise differentiation.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
Journal of The Audio Engineering Society, Dec 1, 2016
The piano is an instrument extensively used in classical, jazz, and pop music since its broad pit... more The piano is an instrument extensively used in classical, jazz, and pop music since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of isolated one-second piano sounds. Listeners compared the sounds pairwise over ten emotion categories. The results showed that all ten emotional categories were significantly affected by pitch and nine of them by dynamics. In particular, the emotional characteristics Happy, Romantic, Comic, Calm, Mysterious, and Shy generally increased with pitch but sometimes decreased at the highest pitches. The characteristics Heroic, Angry, and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers. In terms of dynamics, the results showed that the characteristics Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Mysterious, Shy, and Sad were stronger for soft notes. Surprisingly, Happy was not affected by dynamics. These results help quantify the emotional characteristics of piano sounds.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the emotional characteristics were even more clearly distinguished for 250 ms tones. We also found that the density of significant harmonics, a feature that has particular relevance to non-sustaining tones, was observed to achieve especially strong correlation for all tested emotion categories in these tones with early release.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pi... more The piano is an instrument extensively used in classical, jazz, and pop music, since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. Previous work has found the piano to be emotionally neutral among eight tested non-sustaining instruments. This paper further explores the emotional characteristics of piano sounds with a listening test comparing isolated one-second sounds of different pitches and dynamics on the piano over ten emotional categories: Happy, Sad, Heroic, Scary, Comic, Shy, Romantic, Mysterious, Angry, and Calm. In the experiment, the loud bass was ranked the most Angry and Heroic, and the effect dropped with increasing pitch. The soft treble was ranked the most Calm and Shy, and the effect dropped with decreasing pitch. The trend was clear across the octaves. Both loud and soft sounds have distinguishing emotional characteristics. In contrast, the emotional categories Mysterious and Happy were not much affected by dynamics.
The mallet percussion instruments are gaining attention in modern music arrangements, especially ... more The mallet percussion instruments are gaining attention in modern music arrangements, especially in movie soundtracks and synthesized furniture music. Using a combination of these instruments, a reasonable variety of timbres and pitches are possible. Players can further choose mallets of different makes, which in turn significantly alter the timbre of the sound produced. How can these sounds be used to suggest various moods and emotions in musical compositions? This study compares the sounds of the marimba, vibraphone, xylophone, and glockenspiel, with different mallet hardness over various pitch registers, and ten emotional
Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with... more Music emotion recognition and recommendation systems often use a simplified 4-quadrant model with categories such as Happy, Sad, Angry, and Calm. Previous research has shown that both listeners and automated systems often have difficulty distinguishing low-arousal categories such as Calm and Sad. This paper seeks to explore what makes the categories Calm and Sad so difficult to distinguish. We used 300 low-arousal excerpts from the classical piano repertoire to determine the coverage of the categories Calm and Sad in the low-arousal space, their overlap, and their balance to one another. Our results show that Calm was 40% bigger in terms of coverage than Sad, but that on average Sad excerpts were significantly more negative in mood than Calm excerpts were positive. Calm and Sad overlapped in nearly 20% of the excerpts, meaning 20% of the excerpts were about equally Calm and Sad. Calm and Sad covered about 92% of the low-arousal space, where 8% of the space were holes that were not-at-all Calm or Sad. The largest holes were for excerpts considered Mysterious and Doubtful, but there were smaller holes among positive excerpts as well. Due to the holes in the coverage, the overlaps, and imbalances the Calm-Sad model adds about 6% more errors when compared to asking users directly whether the mood of the music is positive or negative. Nevertheless, the Calm-Sad model is still useful and appropriate for applications in music emotion recognition and recommendation such as when a simple and intuitive interface is preferred or when categorization is more important than precise differentiation.
The piano is an instrument extensively used in classical, jazz, and pop music since its broad pit... more The piano is an instrument extensively used in classical, jazz, and pop music since its broad pitch range and ample dynamic levels allow the instrument to become self-contained and versatile for various kinds of music. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of isolated one-second piano sounds. Listeners compared the sounds pairwise over ten emotion categories. The results showed that all ten emotional categories were significantly affected by pitch and nine of them by dynamics. In particular, the emotional characteristics Happy, Romantic, Comic, Calm, Mysterious, and Shy generally increased with pitch but sometimes decreased at the highest pitches. The characteristics Heroic, Angry, and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers. In terms of dynamics, the results showed that the characteristics Heroic, Comic, Angry, and Scary were stronger for loud notes, while Romantic, Calm, Mysterious, Shy, and Sad were stronger for soft notes. Surprisingly, Happy was not affected by dynamics. These results help quantify the emotional characteristics of piano sounds.
Previous research has shown that different musical instrument sounds have strong emotional charac... more Previous research has shown that different musical instrument sounds have strong emotional characteristics. This paper investigates how emotional characteristics vary with pitch and dynamics within the bowed string instrument family. We conducted listening tests to compare the effects of pitch and dynamics on the emotional characteristics of the violin, viola, cello, and double bass. Listeners compared the sounds pairwise over ten emotional categories. The results showed that the emotional characteristics Happy, Heroic, Romantic, Comic, and Calm generally increased with pitch, but decreased at the highest pitches. The characteristics Angry and Sad generally decreased with pitch. Scary was strong in the extreme low and high registers, while Shy was relatively unaffected by pitch. In terms of dynamics, the results showed that Heroic, Comic, and Angry were stronger for loud notes, while Romantic, Calm, Shy, and Sad were stronger for soft notes. Surprisingly, Scary was least affected by dynamics. These results provide audio engineers and musicians with possible suggestions for emphasizing various emotional characteristics of the bowed strings in sound recordings and performances.
Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent ... more Timbre evokes emotion in music, as do loudness, pitch, rhythm, and other music qualities. Recent research has confirmed the correlation between timbral features with emotion in isolated musical instrument tones. Attack time, spectral centroid and its deviation have significant correlation with emotional characteristics in tones of sustaining (e.g., trumpet) and non-sustaining instruments (e.g., harp). Previous work has considered instrument tones of one to two seconds duration. This paper presents a similar experiment with pairwise comparison of very short (250 ms) isolated tones of nonsustaining instruments, including plucked string, pitched percussion, and keyboard instruments. The tones were investigated for the emotion categories Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. The results agreed with those for one-second tones, with plucked string tones evoking negative emotional categories and pitched percussion tones evoking positive categories. Surprisingly, the...
Uploads
Papers by Chuck-jee Chau