This paper describes the implementation of a digital audio / visual feedback system for an extemp... more This paper describes the implementation of a digital audio / visual feedback system for an extemporaneous dance performance. The system was designed to automatically synchronize aesthetically with the dancers. The performance was premiered at the Slingshot festival in Athens Georgia on March 9, 2013.
Bodytext is a performance work involving speech, movement and sound. A dancer's movement and spok... more Bodytext is a performance work involving speech, movement and sound. A dancer's movement and spoken description of a dance are re-mediated within an augmented environment, where acquired speech is rewritten and re-sonified through physical interaction. The displayed textual objects recombine, creating the score for an emergent and ever evolving performance. Bodytext seeks insight into the relations between kinaesthetic experience, memory, agency and language. Dancer, audience and machine are enmeshed in a recursive dynamic they must both follow to its (il)logical conclusion, the innate entropy of the system giving structure to the overall work.
As an artist, I have created a number of interactive sonic environments that seek to provide a tr... more As an artist, I have created a number of interactive sonic environments that seek to provide a truly unique outcome for each person. The audible domain provides an intuitive, experiential, visceral, instinctive, spontaneous and intimate perceptual habitat where the human body is central, where the visceral engagement with sonic architectures dismisses the western mind-body split as hopelessly inadequate. Do we really only listen with the mind and not the body? The interactive sound environment presents a unique articulation of space and place, one in which the fluidity of the human body can be empowered to find new ways of engaging with environment. This unique and many faceted approach appoints space itself as a performative medium, instructing the user in new ways of listening, whilst simultaneously casting the listener as performative agent. The interactive sound environment establishes a causal loop in which the individual comes to consider the way in which the environment condi...
This issue of Organised Sound explores Interactivity in Musical Instruments. Contributors were as... more This issue of Organised Sound explores Interactivity in Musical Instruments. Contributors were asked to consider questions such as: Are we really in control of our musical instruments, both from a pragmatic and theoretical perspective? The notion of control places the instrument itself in a passive position -that is, it becomes simply a machine/mechanism to be conquered. In reality, the instrument has a set of qualities that play an active role in the music-making process. In sound-based or electroacoustic music, complex interpolations of timbral characteristics are often required. Indeed, the morphology of an entire 'orchestra' of algorithms and/or sample re-synthesis may be under the control or manipulation of a single performer. Design approaches that allow a quick, accurate and intuitive engagement with the sound material are paramount. Acoustic musicians often discuss the ways in which a fine instrument speaks, a turn of phrase that summarises the ease and speed at which the instrument sounds and the effort the musician must invest not only to excite the instrument to sound, but to produce a rich and sonorous tone. Such an instrument, easily manipulable and dynamically variable, is by nature somewhat unstable. High performer skill levels guarantee the desired musical outcome; however, the musician is also aware that the instrument teeters on the brink of chaos. This quality, while seeming counter-intuitive, is a crucial counterpoint in virtuosic performance, where the musician moves past a conscious application of technique into a dynamic relationship with the instrument, the conduit being a cybernetic-like action-response loop, described by the response/timbral qualities of the instrument, the performer's musical intent and other conditioning factors such as the acoustics of the performance space, received performance practice, and so on. The notion of control is then potentially problematic. Perhaps the electroacoustic music performer is not so much 'in control' when navigating the potentials inherent in the work. If this is so, then performance gestures take on a very different function; their designation moves from an event-based classification to encompass the notion of gesture as form and timbre as inter-relationships, influencing orchestration, focus or structural evolution as the performance/musical work evolves.
This article is intended to raise some points of interest and mark out some pointers for alternat... more This article is intended to raise some points of interest and mark out some pointers for alternative approaches to the design and execution of interactive music systems and artworks which pursue interaction that:[bull ] does not include any pre-defined pathways,[bull ] takes dynamic morphology as its foundation, and[bull ] implements dynamic software infrastructures, built on the object-oriented model, providing dynamic instrument instantiation, orchestration and timbral control.It is intended that such design would be a precursor to a new approach to interactivity that responds more directly and uniquely to those who engage in the work, and in so doing rewards them more richly for their time, energy and enthusiasm.
This chapter investigates design strategies for developing digital musical instruments (DMIs) for... more This chapter investigates design strategies for developing digital musical instruments (DMIs) for participatory music. In particular, we present strategies to enhance collaborative musical skills such as rhythmic entrainment and listening/responding to other participants: building skills of this kind has the capacity to motivate long-term usage and adoption of the DMI by a broad range of communities. The design strategies described here address the problems of developing DMIs for long-term use, both in collaborative, mixed skill level contexts, and in established musical and dance traditions. Interactive Tango Milonga—presented here as a case study—is an interactive dance system allowing social tango dancers to drive musical outcomes in real-time via their dance movement. Motion sensors are attached to dancers, and the signals from these sensors are sent to a computer, where an algorithm transforms them into tango music. The impact of the interactive tango system on the musical list...
The frequency f of periodic, or oscillatory, motion is the number of oscillations per second. In ... more The frequency f of periodic, or oscillatory, motion is the number of oscillations per second. In the SI system, it is measured in hertz: (15-1)
Lambeosaurine hadrosaurs are duck-billed dinosaurs known for their large head crests, which resea... more Lambeosaurine hadrosaurs are duck-billed dinosaurs known for their large head crests, which researchers hypothesize were resonators for vocal calls. This paper describes the motivation and process of iteratively designing a musical instrument and interactive sound installation based on imagining the sounds of this extinct dinosaur. We used scientific research as a starting point to create a means of sound production and resonator, using a 3D model obtained from Computed Topology (CT) scans of a Corythosaurus skull and an endocast of its crest and nasal passages. Users give voice to the dinosaur by blowing into a mouthpiece, exciting a larynx mechanism and resonating the sound through the hadrosaur's full-scale nasal cavities and skull. This action allows an embodied glimpse into an ancient past. Users know the dinosaur through the controlled exhalation of their breath, how the compression of the lungs leads to a whisper or a roar.
ACM SIGGRAPH 98 Electronic art and animation catalog on - SIGGRAPH '98, 1998
Background Story: A MASSIVE meteorite is coming, you are in great DANGER and you need to leave pl... more Background Story: A MASSIVE meteorite is coming, you are in great DANGER and you need to leave planet "**" as soon as you can. Your scientists spotted a safer place "not too far", planet "%%" should be a great place for your colony. To reach planet "%%", you first need to escape your current planet. For that, you need to figure out the escape velocity!
International Conference on Auditory-Visual Speech Processing 2008, 2008
The Thinking Head project has as it aims to develop (i) a new generation Talking Thinking Head th... more The Thinking Head project has as it aims to develop (i) a new generation Talking Thinking Head that embodies human attributes, and improves human-machine interaction; and (ii) a plug-and-play research platform for users to test software in an interactive real-time environment. Here, project progress is discussed in terms of the four teams: 1. Head Building–(i) Plug-and-Play architecture,(ii) Thinking Media Framework, and (iii) Animation; 2. Human-Head Interaction (HHI)–(i) Wizard of Oz studies, and (ii) joint attention by human ...
This paper discusses the responsive sound installation Reeds. The Melbourne International Festiva... more This paper discusses the responsive sound installation Reeds. The Melbourne International Festival of the Arts commissioned the Reeds project in 2000, for exhibition on the Ornamental Lake at the Royal Botanic Gardens, Melbourne. It consists of twenty-one large floating sculptures 1 , modeled to represent clusters of river reeds in immaculate man-made plantings. Each reed pod 2 contained a collection of electronics for either the gathering of weather information or the reception and dispersion of sound. The sound installation gathered data from two realtime weather stations, and produced eight channels of musical output by interpreting the machine unit pulses of the weather data as pulse inputs to Inverse Fast Fourier Transform (IFFT) algorithms. The Reeds project focused on a consideration of multiple streams of chaotic and constantly varying sound. I was interested in exploring whether the sonic environment would remain homogenous even though, unlike a musical ensemble, the control inputs varied randomly and independently of each other. The sound installation was site specific, reflecting directly upon the environment it inhabited, both in terms of its visual quality, and aesthetic of the sound.
Interactivity has become a major consideration in the development of a contemporary art practice ... more Interactivity has become a major consideration in the development of a contemporary art practice that engages with the proliferation of computer based technologies. Keywords
Laptop performance of computer music has become wide spread in the electronic music community. It... more Laptop performance of computer music has become wide spread in the electronic music community. It brings with it many issues pertaining to the communication of musical intent. Critics argue that performances of this nature fail to engage audiences as many performers use the mouse and keyboard to control their musical works, leaving no visual cues to guide the audience as to the correlation between performance gestures and musical outcomes. Interfaces need to communicate something of their task. The author will argue that cognitive affordances associated with the performance interface become paramount if the musical outcomes are to be perceived as clearly tied to realtime performance gestures, ie. That the audience is witnessing the creation of the music in that moment as distinct to the manipulation of pre-recorded or pre-sequenced events.
New Interfaces for Musical Expression must speak to the nature of `instrument', that is, it ... more New Interfaces for Musical Expression must speak to the nature of `instrument', that is, it must always be understood that the interface binds to a complex musical phenomenon. This paper explores the nature of engagement, the point of performance that occurs when a human being engages with a computer based instrument. It asks questions about the nature of the instrument in computer music and offers some conceptual models for the mapping of gesture to sonic outcomes.
This paper discusses approaches to realtime motion tracking in contemporary dance. It outlines so... more This paper discusses approaches to realtime motion tracking in contemporary dance. It outlines some problems with current techniques and proposes, through previous research, some alternative approaches that could provide much richer data sets for realtime sonification and visualization of choreographic patterns. 1. Background. While undertaking my PhD (Paine, 2002b), I developed a number of approaches to the mapping of gesture (tracked using video based systems, VNS1 and Cyclops), which identify issues involved in the sonification of gesture (Mulder et al., 1997) in a manner that creates a visceral engagement with the quality of outcome. The following are excerpts from that research as a way of contextualizing the need for the AI research discussed below. During 2000, while I was the Australia Council for the Arts, New Media Arts fellow at RMIT University, I began some research that attempted to expand the interactive environment research (Moser & MacLeod, 1996) I had been doing ove...
For a recently designed telescopic didjeridu, Capybara and Wacom interface. The work explores the... more For a recently designed telescopic didjeridu, Capybara and Wacom interface. The work explores the shifting fundamentals and overtones of the didjeridu and the possibilities of interactive synthesis. Traditional playing techniques are extended and morphed by and in response to electronic elaboration. The performers explore shifting dronal material, vocalisations, and additive rhythmic patterns to create dramatic shifts in timbre, density and
In this article, we discuss some of our research with Local Area Networks (LAN) in the context of... more In this article, we discuss some of our research with Local Area Networks (LAN) in the context of sound installations or musical performances. Our systems, built on top of Web technologies, enable novel possibilities of collective and collaborative interaction, in particular by simplifying public access to the artwork by presenting the work through the web browser of their smartphone/tablet. Additionally, such a technical framework can be extended with so-called nano-computers, microprocessors and sensors. The infrastructure is completely agnostic as to how many clients are attached, or how they connect, which means that if the work is available in a public space, groups of friends, or even informally organised flash mobs, may engage with the work and perform the contents of the work at any time, and if available over the Internet, at any place. More than the technical details, the specific artistic directions or the supposed autonomy of the agents of our systems, this article focus...
This paper describes the implementation of a digital audio / visual feedback system for an extemp... more This paper describes the implementation of a digital audio / visual feedback system for an extemporaneous dance performance. The system was designed to automatically synchronize aesthetically with the dancers. The performance was premiered at the Slingshot festival in Athens Georgia on March 9, 2013.
Bodytext is a performance work involving speech, movement and sound. A dancer's movement and spok... more Bodytext is a performance work involving speech, movement and sound. A dancer's movement and spoken description of a dance are re-mediated within an augmented environment, where acquired speech is rewritten and re-sonified through physical interaction. The displayed textual objects recombine, creating the score for an emergent and ever evolving performance. Bodytext seeks insight into the relations between kinaesthetic experience, memory, agency and language. Dancer, audience and machine are enmeshed in a recursive dynamic they must both follow to its (il)logical conclusion, the innate entropy of the system giving structure to the overall work.
As an artist, I have created a number of interactive sonic environments that seek to provide a tr... more As an artist, I have created a number of interactive sonic environments that seek to provide a truly unique outcome for each person. The audible domain provides an intuitive, experiential, visceral, instinctive, spontaneous and intimate perceptual habitat where the human body is central, where the visceral engagement with sonic architectures dismisses the western mind-body split as hopelessly inadequate. Do we really only listen with the mind and not the body? The interactive sound environment presents a unique articulation of space and place, one in which the fluidity of the human body can be empowered to find new ways of engaging with environment. This unique and many faceted approach appoints space itself as a performative medium, instructing the user in new ways of listening, whilst simultaneously casting the listener as performative agent. The interactive sound environment establishes a causal loop in which the individual comes to consider the way in which the environment condi...
This issue of Organised Sound explores Interactivity in Musical Instruments. Contributors were as... more This issue of Organised Sound explores Interactivity in Musical Instruments. Contributors were asked to consider questions such as: Are we really in control of our musical instruments, both from a pragmatic and theoretical perspective? The notion of control places the instrument itself in a passive position -that is, it becomes simply a machine/mechanism to be conquered. In reality, the instrument has a set of qualities that play an active role in the music-making process. In sound-based or electroacoustic music, complex interpolations of timbral characteristics are often required. Indeed, the morphology of an entire 'orchestra' of algorithms and/or sample re-synthesis may be under the control or manipulation of a single performer. Design approaches that allow a quick, accurate and intuitive engagement with the sound material are paramount. Acoustic musicians often discuss the ways in which a fine instrument speaks, a turn of phrase that summarises the ease and speed at which the instrument sounds and the effort the musician must invest not only to excite the instrument to sound, but to produce a rich and sonorous tone. Such an instrument, easily manipulable and dynamically variable, is by nature somewhat unstable. High performer skill levels guarantee the desired musical outcome; however, the musician is also aware that the instrument teeters on the brink of chaos. This quality, while seeming counter-intuitive, is a crucial counterpoint in virtuosic performance, where the musician moves past a conscious application of technique into a dynamic relationship with the instrument, the conduit being a cybernetic-like action-response loop, described by the response/timbral qualities of the instrument, the performer's musical intent and other conditioning factors such as the acoustics of the performance space, received performance practice, and so on. The notion of control is then potentially problematic. Perhaps the electroacoustic music performer is not so much 'in control' when navigating the potentials inherent in the work. If this is so, then performance gestures take on a very different function; their designation moves from an event-based classification to encompass the notion of gesture as form and timbre as inter-relationships, influencing orchestration, focus or structural evolution as the performance/musical work evolves.
This article is intended to raise some points of interest and mark out some pointers for alternat... more This article is intended to raise some points of interest and mark out some pointers for alternative approaches to the design and execution of interactive music systems and artworks which pursue interaction that:[bull ] does not include any pre-defined pathways,[bull ] takes dynamic morphology as its foundation, and[bull ] implements dynamic software infrastructures, built on the object-oriented model, providing dynamic instrument instantiation, orchestration and timbral control.It is intended that such design would be a precursor to a new approach to interactivity that responds more directly and uniquely to those who engage in the work, and in so doing rewards them more richly for their time, energy and enthusiasm.
This chapter investigates design strategies for developing digital musical instruments (DMIs) for... more This chapter investigates design strategies for developing digital musical instruments (DMIs) for participatory music. In particular, we present strategies to enhance collaborative musical skills such as rhythmic entrainment and listening/responding to other participants: building skills of this kind has the capacity to motivate long-term usage and adoption of the DMI by a broad range of communities. The design strategies described here address the problems of developing DMIs for long-term use, both in collaborative, mixed skill level contexts, and in established musical and dance traditions. Interactive Tango Milonga—presented here as a case study—is an interactive dance system allowing social tango dancers to drive musical outcomes in real-time via their dance movement. Motion sensors are attached to dancers, and the signals from these sensors are sent to a computer, where an algorithm transforms them into tango music. The impact of the interactive tango system on the musical list...
The frequency f of periodic, or oscillatory, motion is the number of oscillations per second. In ... more The frequency f of periodic, or oscillatory, motion is the number of oscillations per second. In the SI system, it is measured in hertz: (15-1)
Lambeosaurine hadrosaurs are duck-billed dinosaurs known for their large head crests, which resea... more Lambeosaurine hadrosaurs are duck-billed dinosaurs known for their large head crests, which researchers hypothesize were resonators for vocal calls. This paper describes the motivation and process of iteratively designing a musical instrument and interactive sound installation based on imagining the sounds of this extinct dinosaur. We used scientific research as a starting point to create a means of sound production and resonator, using a 3D model obtained from Computed Topology (CT) scans of a Corythosaurus skull and an endocast of its crest and nasal passages. Users give voice to the dinosaur by blowing into a mouthpiece, exciting a larynx mechanism and resonating the sound through the hadrosaur's full-scale nasal cavities and skull. This action allows an embodied glimpse into an ancient past. Users know the dinosaur through the controlled exhalation of their breath, how the compression of the lungs leads to a whisper or a roar.
ACM SIGGRAPH 98 Electronic art and animation catalog on - SIGGRAPH '98, 1998
Background Story: A MASSIVE meteorite is coming, you are in great DANGER and you need to leave pl... more Background Story: A MASSIVE meteorite is coming, you are in great DANGER and you need to leave planet "**" as soon as you can. Your scientists spotted a safer place "not too far", planet "%%" should be a great place for your colony. To reach planet "%%", you first need to escape your current planet. For that, you need to figure out the escape velocity!
International Conference on Auditory-Visual Speech Processing 2008, 2008
The Thinking Head project has as it aims to develop (i) a new generation Talking Thinking Head th... more The Thinking Head project has as it aims to develop (i) a new generation Talking Thinking Head that embodies human attributes, and improves human-machine interaction; and (ii) a plug-and-play research platform for users to test software in an interactive real-time environment. Here, project progress is discussed in terms of the four teams: 1. Head Building–(i) Plug-and-Play architecture,(ii) Thinking Media Framework, and (iii) Animation; 2. Human-Head Interaction (HHI)–(i) Wizard of Oz studies, and (ii) joint attention by human ...
This paper discusses the responsive sound installation Reeds. The Melbourne International Festiva... more This paper discusses the responsive sound installation Reeds. The Melbourne International Festival of the Arts commissioned the Reeds project in 2000, for exhibition on the Ornamental Lake at the Royal Botanic Gardens, Melbourne. It consists of twenty-one large floating sculptures 1 , modeled to represent clusters of river reeds in immaculate man-made plantings. Each reed pod 2 contained a collection of electronics for either the gathering of weather information or the reception and dispersion of sound. The sound installation gathered data from two realtime weather stations, and produced eight channels of musical output by interpreting the machine unit pulses of the weather data as pulse inputs to Inverse Fast Fourier Transform (IFFT) algorithms. The Reeds project focused on a consideration of multiple streams of chaotic and constantly varying sound. I was interested in exploring whether the sonic environment would remain homogenous even though, unlike a musical ensemble, the control inputs varied randomly and independently of each other. The sound installation was site specific, reflecting directly upon the environment it inhabited, both in terms of its visual quality, and aesthetic of the sound.
Interactivity has become a major consideration in the development of a contemporary art practice ... more Interactivity has become a major consideration in the development of a contemporary art practice that engages with the proliferation of computer based technologies. Keywords
Laptop performance of computer music has become wide spread in the electronic music community. It... more Laptop performance of computer music has become wide spread in the electronic music community. It brings with it many issues pertaining to the communication of musical intent. Critics argue that performances of this nature fail to engage audiences as many performers use the mouse and keyboard to control their musical works, leaving no visual cues to guide the audience as to the correlation between performance gestures and musical outcomes. Interfaces need to communicate something of their task. The author will argue that cognitive affordances associated with the performance interface become paramount if the musical outcomes are to be perceived as clearly tied to realtime performance gestures, ie. That the audience is witnessing the creation of the music in that moment as distinct to the manipulation of pre-recorded or pre-sequenced events.
New Interfaces for Musical Expression must speak to the nature of `instrument', that is, it ... more New Interfaces for Musical Expression must speak to the nature of `instrument', that is, it must always be understood that the interface binds to a complex musical phenomenon. This paper explores the nature of engagement, the point of performance that occurs when a human being engages with a computer based instrument. It asks questions about the nature of the instrument in computer music and offers some conceptual models for the mapping of gesture to sonic outcomes.
This paper discusses approaches to realtime motion tracking in contemporary dance. It outlines so... more This paper discusses approaches to realtime motion tracking in contemporary dance. It outlines some problems with current techniques and proposes, through previous research, some alternative approaches that could provide much richer data sets for realtime sonification and visualization of choreographic patterns. 1. Background. While undertaking my PhD (Paine, 2002b), I developed a number of approaches to the mapping of gesture (tracked using video based systems, VNS1 and Cyclops), which identify issues involved in the sonification of gesture (Mulder et al., 1997) in a manner that creates a visceral engagement with the quality of outcome. The following are excerpts from that research as a way of contextualizing the need for the AI research discussed below. During 2000, while I was the Australia Council for the Arts, New Media Arts fellow at RMIT University, I began some research that attempted to expand the interactive environment research (Moser & MacLeod, 1996) I had been doing ove...
For a recently designed telescopic didjeridu, Capybara and Wacom interface. The work explores the... more For a recently designed telescopic didjeridu, Capybara and Wacom interface. The work explores the shifting fundamentals and overtones of the didjeridu and the possibilities of interactive synthesis. Traditional playing techniques are extended and morphed by and in response to electronic elaboration. The performers explore shifting dronal material, vocalisations, and additive rhythmic patterns to create dramatic shifts in timbre, density and
In this article, we discuss some of our research with Local Area Networks (LAN) in the context of... more In this article, we discuss some of our research with Local Area Networks (LAN) in the context of sound installations or musical performances. Our systems, built on top of Web technologies, enable novel possibilities of collective and collaborative interaction, in particular by simplifying public access to the artwork by presenting the work through the web browser of their smartphone/tablet. Additionally, such a technical framework can be extended with so-called nano-computers, microprocessors and sensors. The infrastructure is completely agnostic as to how many clients are attached, or how they connect, which means that if the work is available in a public space, groups of friends, or even informally organised flash mobs, may engage with the work and perform the contents of the work at any time, and if available over the Internet, at any place. More than the technical details, the specific artistic directions or the supposed autonomy of the agents of our systems, this article focus...
Uploads
Papers by Garth Paine