Live Brain-Computer Cinema & Neurocinematics by Polina Zioga
The fields of neural prosthetic technologies and Brain-Computer Interfaces (BCIs) have witnessed ... more The fields of neural prosthetic technologies and Brain-Computer Interfaces (BCIs) have witnessed in the past 15 years an unprecedented development, bringing together theories and methods from different scientific fields, digital media, and the arts. More in particular, artists have been amongst the pioneers of the design of relevant applications since their emergence in the 1960s, pushing the boundaries of applications in real-life contexts. With the new research, advancements, and since 2007, the new low-cost commercial-grade wireless devices, there is a new increasing number of computer games, interactive installations, and performances that involve the use of these interfaces, combining scientific, and creative methodologies. The vast majority of these works use the brain-activity of a single participant. However, earlier, as well as recent examples, involve the simultaneous interaction of more than one participants or performers with the use of Electroencephalography (EEG)-based multi-brain BCIs. In this frame, we discuss and evaluate “Enheduanna—A Manifesto of Falling,” a live brain-computer cinema performance that enables for the first time the simultaneous real-time multi-brain interaction of more than two participants, including a performer and members of the audience, using a passive EEG-based BCI system in the context of a mixed-media performance. The performance was realised as a neuroscientific study conducted in a real-life setting. The raw EEG data of seven participants, one performer and two different members of the audience for each performance, were simultaneously recorded during three live events. The results reveal that the majority of the participants were able to successfully identify whether their brain-activity was interacting with the live video projections or not. A correlation has been found between their answers to the questionnaires, the elements of the performance that they identified as most special, and the audience's indicators of attention and emotional engagement. Also, the results obtained from the performer's data analysis are consistent with the recall of working memory representations and the increase of cognitive load. Thus, these results prove the efficiency of the interaction design, as well as the importance of the directing strategy, dramaturgy and narrative structure on the audience's perception, cognitive state, and engagement.
The new commercial-grade Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) have... more The new commercial-grade Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) have led to a phenomenal development of applications across health, entertainment and the arts, while an increasing interest in multi-brain interaction has emerged. In the arts, there is already a number of works that involve the interaction of more than one participants with the use of EEG-based BCIs. However, the field of live brain-computer cinema and mixed-media performances is rather new, compared to installations and music performances that involve multi-brain BCIs. In this context, we present the particular challenges involved. We discuss Enheduanna – A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance that enables the real-time brain-activity interaction of one performer and two audience members; and we take a cognitive perspective on the implementation of a new passive multi-brain EEG-based BCI system to realise our creative concept. This article also presents the preliminary results and future work.
Interactive new media art and games belong to distinctive fields, but nevertheless share common g... more Interactive new media art and games belong to distinctive fields, but nevertheless share common grounds, tools, methodologies, challenges, and goals, such as the use of applications and devices for engaging multiple participants and players, and more recently electroencephalography (EEG)-based brain-computer interfaces (BCIs). At the same time, an increasing number of new neuroscientific studies explore the phenomenon of brain-to-brain coupling, the dynamics and processes of the interaction and synchronisation between multiple subjects and their brain activity. In this context, we discuss interactive works of new media art, computer and serious games that involve the interaction of the brain-activity, and hypothetically brain-to-brain coupling, between multiple performer/s, spectator/s, or participants/players. We also present Enheduanna – A Manifesto of Falling (2015), a new live brain-computer cinema performance, with the use of an experimental passive multi-brain BCI system under development. The aim is to explore brain-to-brain coupling between performer/s and spectator/s as means of controlling the audio-visual creative outputs.
Although the use of Brain-Computer Interfaces (BCIs) in the arts originates in the 1960s, there i... more Although the use of Brain-Computer Interfaces (BCIs) in the arts originates in the 1960s, there is a limited number of known applications in the context of real-time audio-visual and mixed-media performances and accordingly the knowledge base of this area has not been developed sufficiently. Among the reasons are the difficulties and the unknown parameters involved in the design and implementation of the BCIs. However today, with the dissemination of the new wireless devices, the field is rapidly growing and changing. In this frame, we examine a selection of representative works and artists, in comparison to the current scientific evidence. We identify important performative and neuroscientific aspects, issues and challenges. A model of possible interactions between the performers and the audience is discussed and future trends regarding liveness and interconnectivity are suggested.
The majority of the applications of Brain-Computer Interfaces (BCIs) in the arts and entertainmen... more The majority of the applications of Brain-Computer Interfaces (BCIs) in the arts and entertainment use the brain activity of a single participant, although an increasing number involve the interaction of more than one user.
This dissertation investigates the design and implementation of multi-brain electroencephalography (EEG)–based BCIs in the context of live cinema and mixed-media performances. The aim is to identify the interdisciplinary challenges involved and to develop an effective model for the simultaneous multi-brain interactions of performers and audiences.
By combining scientific and practice-based methodologies, a new passive multi-brain EEG-based BCI system was developed and implemented in 'Enheduanna—A Manifesto of Falling', the first demonstration of a live brain–computer cinema performance (CCA Glasgow, July 2015). This new work enabled the simultaneous, real-time interaction—with the use of EEG-based BCIs—of more than two participants, including both a performer and members of the audience, in the context of a mixed-media performance. The analysis of the participants’ data reveals a correlation between the elements of the performance that they identified as most special and their indicators of attention and emotional engagement, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live braincomputer cinema performance, as a novel format of performative work and as a complete combination of creative and scientific solutions.
Artists have been interested in the human brain’s anatomy and physiology since at least the Renai... more Artists have been interested in the human brain’s anatomy and physiology since at least the Renaissance, while in the twentieth century, the technological revolution enabled them to include in their practices methods adopted from the sciences and engineering, like Brain-Computer Interfaces (BCIs). The use of BCIs originates in the 1960s, with musicians, performers and artists being amongst the pioneers in the design of BCI applications. In recent years, after a period of little progress in the field, the introduction of new commercial-grade Electroencephalography (EEG)-based BCIs has led to a phenomenal development of applications across health, entertainment and the arts. At the same time, in the fields of neuroscience and experimental psychology, has emerged a new increasing interest in the mechanisms and processes of the interaction between multiple subjects and their brain-activity, referred to as multi-brain interaction. Although the vast majority of the applications in the arts and entertainment use the brain-activity of a single participant, there are earlier as well as an increasing number of recent examples that involve the simultaneous interaction of more than one participants, mainly in the context of installations, computer games and music performances.
This dissertation investigates the use of multi-brain EEG-based BCIs in the context of live cinema and mixed-media performances, which is a rather new field bearing distinct characteristics. Using an interdisciplinary approach, a critical overview of the development of the main BCI hardware, software and modes of interaction is presented and relevant works are examined. The aim is to identify the neuroscientific, computational, creative, performative and experimental challenges of the design and implementation of multi-brain BCIs in mixed-media performances, which leads to the main research question:
What might be an effective model for the simultaneous multi-brain interaction of performers and audiences using EEG-based BCIs in the context of live cinema and mixed-media performances?
In order to address the main research enquiry, scientific and practice-based methodologies were combined and a new passive multi-brain EEG-based BCI system was developed. The system was further implemented in the context of the research case study, Enheduanna – A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance (CCA Glasgow 29-31 July 2015). This new work enabled for the first time the simultaneous real-time interaction with the use of EEG-based BCIs of more than two participants, including both a performer as well as members of the audience in the context of a mixed-media performance. The analysis of the participants’ data has most interestingly revealed a correlation between the elements of the performance, which they identified as most special, and their indicators of attention and emotional engagement that were increased during the last two scenes, when their brain-activity was interacting with the live visuals, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live brain-computer cinema performance as a new format of performative work and as a complete combination of creative and scientific solutions. This dissertation also presents the new trends in the field, such as hybrid BCIs, the combination with virtual and mixed reality systems, together with future work.
ECREA Film Studies Section Conference ‘Research Methods in Film Studies: Challenges and Opportunities’, Oct 18, 2019
The beginning of the 21st century saw the emergence of neurocinematics, the nneuroscience of film... more The beginning of the 21st century saw the emergence of neurocinematics, the nneuroscience of film, as a new interdisciplinary field that investigates the effect of free viewing of films on the spectators’ brain-activity, searching for similarities in their spatiotemporal responses. This new methodological approach has opened new opportunities for audience research and has provided us with insight on the level of control that aspects of the film language, such as the narrative and the editing, can have over the viewer’s brain states. At the same time, the proliferation of new brain-imaging techniques and technologies, such as the Brain-Computer Interfaces (BCIs), that enable the interaction, individually or collectively, of the brain of audiences and/or performers with the moving image, has led to the production of interactive videos, films and works of live cinema. A growing number of researchers and filmmakers use these interfaces, not only as creative and storytelling tools, but also as methods of conducting neuroscientific audience research in real-life settings. In this context, this paper presents a review and methodological analysis of relevant studies, following categorisations that are crucial for the interpretation of the results obtained, such as the brain-imaging techniques applied, the environment of the study, the number, type and role of the participants (audience versus performers), and the film form among others. The paper also focuses on how the use of BCIs is changing previous conventions, including discourses of authorship/co-authorship, mise-en-scène and editing, and the reinvention of the film language as embodiment of the audience’s cognitive state. Lastly, a way forward is proposed by looking at the potential of live brain-computer cinema as a new film form that combines live cinema, neurocinematics and the use of BCIs for studying and enabling collective audience interactions in a cinema theatre environment.
Over the past few years, we have seen the extraordinary development of neural prosthetic technolo... more Over the past few years, we have seen the extraordinary development of neural prosthetic technologies that can replace or enhance functions of our central nervous system. For example, devices like Brain-Computer Interfaces (BCIs) allow the direct communication of the brain with a computer. The most common technique applied in these devices, is Electroencephalography (EEG)a recording of the electrical activity along the scalp. These technologies are mainly used in health, but our new research shows how they are changing the future of cinema too. This is no coincidence. Artists have been among the pioneers in the use of these technologies, developing creative applications since their first emergence in the 1960s. Early examples include Music For Solo Performer (1965) by Alvin Lucier, which is considered the first performance using EEG technology. Interactive artworks, like the Brainwave Drawings (1972) by Nina Sobell and installations like Alpha Garden (1973) by Jaqueline Humbert also illustrate how the art world paved the way.
The Guardian, 2018
Charlie Chaplin, known for the slapstick humour of his films, was part of a generation of actors ... more Charlie Chaplin, known for the slapstick humour of his films, was part of a generation of actors who managed to continue working through the transition from the era of silent film to one filled with dialogue and sound.
The introduction of sound wasn’t the only way that people revolutionised cinema, and it won’t be the last.
So what does the history of cinema tell us about the evolution of technology in the arts?
And how can brain activity be used to change the plot of a film? Is there a place for the traditionally passive experience of watching a film to become more interactive, or will that detract from what we value about cinema?
This week, Jordan Erica Webber is joined by the chief curator of the Museum of the Moving Image in New York, David Schwartz, and Dr Polina Zioga, director the Interactive Filmmaking Lab, at Staffordshire University.
Interactive Filmmaking & Cinema by Polina Zioga
The Formation of Film Audiences: Conference Proceedings, Mar 11, 2022
As in the broader cultural sector, COVID-19 has created unprecedented challenges for cinema exhib... more As in the broader cultural sector, COVID-19 has created unprecedented challenges for cinema exhibitors in the UK and worldwide. Venues have been forced to close for months, operate with substantial uncertainty, or re-open with reduced capacity. Still, their position remains precarious and the challenges to the sustainability of the sector are ongoing: changing restrictions and associated increased costs, reduction in production and distribution pipelines. The British Film Institute highlighted that the pandemic poses an existential threat, especially to independent exhibitors and those operating in remote or deprived areas. Thus, COVID-19 has also highlighted existing inequalities, the digital divide, and the need to expand the audiences’ diversity. Meanwhile, in sectoral events, panels have reflected on how exhibitors and audiences have become more accustomed to accessing media experiences online, and how this new digital literacy will support cinemas’ efforts to attract cinemagoers in their reopening. Prior to COVID-19, the use of interaction design for new cinematic experiences had attracted the interest of festivals, filmmakers and researchers. In this position paper, we argue that interaction design and technologies can help independent cinemas to engage and galvanise new audiences to patronise COVID-safe venues. From low-end online platforms to high-end immersive experiences, new technologies are transforming connectivity across society, and have the potential to support access for D/deaf, neurodivergent, and disabled audiences, but adoption by exhibitors is so far limited. We outline the research needs and priorities in this field. These include identifying facilitators and obstacles to industry adoption of interactive forms, and mapping experiences and attitudes across the sector. Together with directions for immediate practical solutions, it is crucial to gather critical data for future research use, in order to pave the way for long-term solutions and design innovation, so that the sector can build resilience, recover and reach underserved audiences.
Interactive Film & Media Journal, 2022
To reach younger audiences, museums worldwide have incorporated interactive and hands-on activiti... more To reach younger audiences, museums worldwide have incorporated interactive and hands-on activities, while some venues specialise in children as their main audience. Videos, in particular, can be easily integrated in the museum space and provide a variety of application possibilities. Their use creates a hybrid experience for the visitor in which the interaction between physical and digital elements transforms and enriches their experience of the exhibits. Furthermore, the use of interactive technologies has been proven to increase visitor numbers and interactions on- and off-site. In this context, our practice-based research focuses on the use of interactive video technologies, and factors that can lead into the design of engaging and user-friendly museum experiences for children, with the aim of investigating their application through the production of a new interactive film for young museum visitors. To achieve this, a museum was chosen as a case study and a survey was conducted. The results indicated that the creation of an interactive video could benefit the areas that were visited less; the preferable length is rather short; while hands-on and video installations promote and prolong the engagement of young visitors, and are favoured by both younger and older children. Additionally, fictional or dramatised stories are attractive to children compared to documentaries; and it would be preferrable to access the interactive content on their mobile devices. These have led to the production of Paintings Alive, an interactive film for children, featuring and reenacting the paintings in the museum’s art gallery, and accessible on the visitors’ mobile devices. Our article also discusses the findings of the project, alongside the challenges and limitations imposed by the COVID-19 pandemic, and offers recommendations for future work.
Interactive Film and Media Conference 2021: New Narratives, Racialization, Global Crises, and Social Engagement, Jan 30, 2022
In recent years, the increasing number of interactive films being released, has highlighted the n... more In recent years, the increasing number of interactive films being released, has highlighted the need for further development of methods and criteria that can guide the earlier stages of development, such as the scriptwriting process. Following the framework of interactive storytelling as a spectrum, it is acknowledged that writing a script for an interactive narrative that involves branching path options for navigating through the story, or multiple endings, is becoming more common and presents its own challenges. In this context, this paper examines established criteria used for assessing narrative quality and examines currently available software for interactive scriptwriting, identifying their affordances and limitations. Accordingly, we present Scapegoat, a short interactive crime drama, based on the model of British homicide investigations, and with the objective to investigate in practice the application of the criteria for narrative quality, together with the processes and elements of scriptwriting that can lead to a strong engaging story. We propose an approach that can efficiently incorporate crucial information of the interaction design, it can be effectively communicated to the crew and cast and used throughout the production lifecycle of the film. We highlight the crucial role of the on-set script supervisor for ensuring the interaction design is not compromised, and continuity is retained. We also discuss recommendations for further developments, including the importance of engaging the crew and cast early in the development process, together with future work into the requirements of interactive commissioners for television and film, and the need for standarisation in the industry.
IFM Journal Vol.02 N.04, 2022
This special issue of the Interactive Film and Media Journal constitutes the first of two issues ... more This special issue of the Interactive Film and Media Journal constitutes the first of two issues dedicated to the IV International Interactive Film and Media Conference, held online in June 2022 and co-organized by Hudson Moura (Toronto Metropolitan University, Canada), Heidi Rae Cooley (University of Texas at Dallas, USA), Anna Wiehl (University of Bayreuth, Germany) and Stefano Odorico (Technological University of the Shannon, Ireland / Leeds Trinity University, UK). As tangential as it may seem, the IFM conferences are an opportunity to reflect on the scholarly field of interactivity, offering a platform for academics and practitioners to broaden the field across disciplines. The conference titled: Interactive Epistemology, Listening, and Ecomedia clearly demonstrated that interactivity does not just have a newly rediscovered popularity in film and media, but it is also omnipresent in a multidisciplinary context, from cultural productions such as games, films and books to audience perception such as streaming, virtual reality and real-time; from intellectual publications such as news, hypertexts, social media to virtual world economy such as cryptocurrencies.
Based on the conference’s call for papers, we “are witnessing a shift in all levels of cultural production. As a result, interactivity gains more space and conceptual independence with specific terminology, questions, promises, and challenges. Moreover, interactivity is growing as an academic discipline, a field of study, and an area of research in several spheres.” In a post-pandemic society, we feel that now is an excellent time to be returning to these debates, as the computer-based aesthetic of the multiple such as that of the Zoom interface and similar (which we have all mastered during periods of isolation) have become a newly established fixture of working life.
Audiences beyond the multiplex: understanding the value of a diverse film culture, Mar 3, 2021
As in the broader cultural sector, COVID-19 has created unprecedented challenges for cinema exhib... more As in the broader cultural sector, COVID-19 has created unprecedented challenges for cinema exhibitors in the UK and world-wide. Venues have been forced to close for months or operate with substantial uncertainty and reduced capacity. Still, their position remains precarious and the challenges to the sustainability of the sector are ongoing: changing restrictions and associated increased costs, reduction in production and distribution pipelines. The British Film Institute highlighted that the pandemic poses an existential threat especially to the independent exhibitors, and those operating in remote or deprived areas. Thus, COVID-19 has also highlighted existing inequalities, the digital divide, and the need to expand the audiences’ diversity.
Meanwhile, in sectoral events, panels have reflected on how exhibitors and audiences have become more accustomed to accessing media experiences online, and this new digital literacy will support cinemas’ efforts to attract cinemagoers in their reopening. Prior to COVID-19, the use of interaction design for new cinematic experiences had attracted the interest of festivals, filmmakers and researchers. In this position paper, we argue that interaction design and technologies can help independent cinemas to engage and galvanise new audiences to patronise COVID-safe venues. From low-end online platforms, to high-end immersive experiences, new technologies are transforming connectivity across society, and have the potential to support access for D/deaf, neurodivergent, and disabled audiences, but adoption by exhibitors is so far limited. We outline the research needs and priorities in this field. These include identifying facilitators and obstacles to industry adoption of interactive forms, and mapping experiences and attitudes across the sector. Together with directions for immediate practical solutions, it is crucial to gather critical data for future research use, in order to pave the way for long-term solutions and design innovation, so that the sector can build resilience, recover and reach underserved audiences.
---
URL: https://www.beyondthemultiplex.net/session-9-audiences-accessibility-and-responding-to-the-pandemic/
Interactive Film & Media Journal, 2022
To reach younger audiences, museums worldwide have incorporated interactive and hands-on activiti... more To reach younger audiences, museums worldwide have incorporated interactive and hands-on activities, while some venues specialise in children as their main audience. Videos, in particular, can be easily integrated in the museum space and provide a variety of application possibilities. Their use creates a hybrid experience for the visitor in which the interaction between physical and digital elements transforms and enriches their experience of the exhibits. Furthermore, the use of interactive technologies has been proven to increase visitor numbers and interactions on- and off-site. In this context, our research focuses on the use of interactive video technologies, and factors that can lead into the design of engaging and user-friendly museum experiences for children. To achieve this, a museum was chosen as a case study and a survey was conducted. The results indicated that the creation of an interactive video could benefit the areas that were visited less; the preferable length is rather short; while hands-on and video installations promote and prolong the engagement of young visitors, and are favoured by both younger and older children. Additionally, fictional or dramatised stories are attractive to children compared to documentaries; and it would be preferable to access the interactive content on their mobile devices. These have led to the production of Paintings Alive, an interactive film for children, based on the museum’s art gallery, and accessible on the visitors’ mobile devices. Our paper also discusses the findings of the project, alongside the challenges and limitations imposed by the COVID-19 pandemic, and offers recommendations for future work.
Interactive Film and Media Conference 2021: New Narratives, Racialization, Global Crises, and Social Engagement, Aug 6, 2021
In recent years, the increasing number of interactive films being released, has highlighted the n... more In recent years, the increasing number of interactive films being released, has highlighted the need for further development of methods and criteria that can guide the earlier stages of development, such as the scriptwriting process. Following the framework of interactive storytelling as a spectrum, it is acknowledged that writing a script for an interactive narrative that involves branching path options, or multiple endings, is becoming more common and presents its own challenges. In this context, this paper examines established criteria used for assessing narrative quality and examines currently available software for interactive scriptwriting, identifying their affordances and limitations. We present Scapegoat, a short interactive crime drama, with the objective to investigate in practice the application of the criteria for narrative quality, together with the processes and elements of scriptwriting that can lead to a strong engaging story. We propose an approach that can efficiently incorporate crucial information of the interaction design, it can be effectively communicated to the crew and cast and used throughout the production lifecycle of the film. We also discuss recommendations for further developments and future work.
---
URL: https://interactivefilm.blogspot.com/2021/07/scriptwriting-for-interactive-crime.html
MeCCSA Brighton 2020 ‘Media Interactions and Environments’ Conference, Jan 8, 2020
It is widely considered that technology gave birth and continues to play a vital role in film pro... more It is widely considered that technology gave birth and continues to play a vital role in film production around the globe. Technological advancements have shaped film language and the audience experience throughout history. With the emergence of interactivity and the proliferation of new digital media and technologies, new opportunities, discourses and challenges have arisen. In this context, this paper discusses that current classification approaches are not appropriate for capturing in a comprehensive manner the multidimensional aspects of interactive film history. It is proposed that a new classification system is needed in order to map the history of interactive filmmaking and cinema, alongside the development and affordances of the different interactive technologies in use, like the Hypertext Markup Language (HTML) and web technologies, 360o video, Virtual and Augmented Reality, and Artificial Intelligence (AI) among others. Other prominent characteristics and practices that are considered include the enablement of either single- and/or collective multi-interaction of the audience with the moving image; the role of complex algorithms and the type of data analysis performed; technologies that make direct use of the audience’s physiology and cognitive state, such as the Brain-Computer Interfaces (BCIs); as well as the gamification of the film viewing experience. Accordingly, a new approach to identifying a suitable systematic structure is outlined, as opposed to recycling the conventional methods of classification, chronological order and genre categorisation.
Interactive Media & Networks by Polina Zioga
DCAC 2023 5th International Conference on Digital Culture & AudioVisual Challenges, May 12, 2023
Summary:
This paper presents ‘This Place Has Its Own Air’, a data-driven video-installation and ... more Summary:
This paper presents ‘This Place Has Its Own Air’, a data-driven video-installation and proof of concept for urban air quality data visceralisation. Its title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd, and is concerned with the effect of the city’s air (man-made polluted environment) on the body. The video-installation uses the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience; enabling viewers to have a real-time glimpse of the air they breathe and reflect on the need for reducing the pollution of their environment.
Background:
The quality of the air we breathe, and the air pollution of our environment, are inextricably linked. Air pollutants can spread across long distances and are principally the products of combustion from space heating, power generation, or from motor vehicle traffic. They are monitored and regulated using human health- and environmentally-based criteria for setting permissible levels, as they can cause both short- and long-term negative health effects. In UK and in Scotland, air quality information, such as real-time data, are used to forecast pollution, and are made freely available to the public, often accompanied by health advice and recommended actions for short-term exposure to air pollution. However, despite the improvements in the air quality in recent years, air pollution remains the biggest environmental threat to health in the UK.
Objective & Methods:
The project’s title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd (1893-1981), a Scottish Modernist writer and poet, born in Aberdeenshire. In her work, Shepherd was concerned with the effect of the mountain (natural environment) on the body, and the bodily experience of being in the world (nature). Similarly, our data-driven video-installation is concerned with the effect of the city’s air (man-made polluted environment) on the body. It involves the projection of a live animated moving image that is being processed and rendered using the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience (visceralisation). To achieve this, digital medical images of the human body are used to construct an animated environment that is ambiguous in terms of its origin; and which expands and contracts, as if breathing, following the human adult respiratory rate. The urban air quality data are acquired using the OpenWeather Air Quality API, obtaining hourly readings of the levels of the nitrogen dioxide (NO2) and the fine particles (PM2.5 and PM10), which are the two types of pollutants primarily produced by motor vehicle traffic. Then, the real-time digital processing of the air quality data is performed by a custom software program written in Processing. The levels of the pollutants are scaled and used to control in real-time elements of the live animated moving image. They are mapped to RGB colour values, based on the UK AIR Daily Air Quality Index, which is divided into four bands, from low, to moderate, high, and very high; and generate a real-time colour filter, shifting from cooler to warmer tints. While a particle system appears and overlays different parts of the image, as the concentration of hazardous pollutants rises and falls throughout the day.
The final output, which is silent and designed to have immediacy for presentation in a transient space with a limited dwell time, aims at a ‘meaning-making’ of the environmental themes, which are central to its concept. The man-made intervention (pollution) to the natural environment (air) affects the human bodily experience (breathing), which in turn can prompt a new man-made intervention (reduction of pollution). The resulting video is designed to be displayed on a large vertical screen in a central public building, thus serving as visceralisation of the city’s real-time air quality data.
Conclusion:
The aim of the project is to create an evocative visualisation of the air pollution effect on the body, in order to aesthetically immerse the audiences, promote their critical engagement, and increase the awareness of the need for reducing the environmental pollution. The video-installation will give the city’s residents, and passers-by a real-time glimpse of the air they breathe, evident through the changing colour and the presence of particles in the animated moving image that correspond to the real-time pollutant levels. As a result, the audience is immersed in an evocative imagery that prompts them to reflect on the vital need for reducing the air pollution of their environment, and in the longer-term consider the actions they can take to achieve this change.
Interconnected devices and objects pervade our everyday lives with an increasing trend. These dig... more Interconnected devices and objects pervade our everyday lives with an increasing trend. These digital objects, connected through wireless and cable networks, exchange information in various levels, producing diverse types of interactions between them and their users. The paper explores the aspect of interconnectivity as a key-attribute of the contemporary digital artefacts that populate our everyday environments. It explores the notion of home, place and network by focusing on the effects of digital interconnectivity in the way we perceive private and public space. In this context, “HOME network”, a collaborative project, is presented. The project is a portable, netless (without an Internet connection) Wi-Fi network, a free access unlocked digital platform, transmitting within the urban environment of various cities during a series of specific time periods and events. The artists carry the network through space and time, as an unrevealed, private, performative act within the realm of the public urban space, leaving only temporary transmitted digital traces/data within the vicinity of the transmission range, thus producing interactions and exploring the boundaries between the private and the public, the physical and the digital space and challenging the notion of surveillance in urban environments. New action patterns are introduced, which look into new ways of performing the physicality of the body within a digital nomadism. At the same time the artists address themes and invite the visitors of the network to engage, participate in and reflect on commonly shared experiences and contemporary questions regarding our sense of belonging in both the private and the public sphere.
New Media Art by Polina Zioga
The bilingual publication “Polina Zioga - The Shelter”, in Greek and in English, was released by ... more The bilingual publication “Polina Zioga - The Shelter”, in Greek and in English, was released by Pavlidis Creative publications on the occasion of Polina Zioga’s solo exhibition “The Shelter”, at Fournos Center for Digital Culture, from 18th until 26th of June 2012.
It includes visual material, video stills and details, from the homonym video installation, but also unreleased material taken from the creative process. Inside the publication's pages can be read Heraclitus’ fragment, which consists the central core and reference of Polina Zioga’s work, the critical text by the curator Zoe Fragoulopoulou, and also a “public-private dialogue” between them, concerning the individual/biological body as an adaptation of a social/collective one, but also the theatrical stage as another space for the presentation of the visual artwork and the representation of the human correlations.
It includes 48 color pages, in 150gr. velvet paper, dimensions 12x18cm, with soft cover.
ISBN: 978-618-80091-0-3
Uploads
Live Brain-Computer Cinema & Neurocinematics by Polina Zioga
This dissertation investigates the design and implementation of multi-brain electroencephalography (EEG)–based BCIs in the context of live cinema and mixed-media performances. The aim is to identify the interdisciplinary challenges involved and to develop an effective model for the simultaneous multi-brain interactions of performers and audiences.
By combining scientific and practice-based methodologies, a new passive multi-brain EEG-based BCI system was developed and implemented in 'Enheduanna—A Manifesto of Falling', the first demonstration of a live brain–computer cinema performance (CCA Glasgow, July 2015). This new work enabled the simultaneous, real-time interaction—with the use of EEG-based BCIs—of more than two participants, including both a performer and members of the audience, in the context of a mixed-media performance. The analysis of the participants’ data reveals a correlation between the elements of the performance that they identified as most special and their indicators of attention and emotional engagement, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live braincomputer cinema performance, as a novel format of performative work and as a complete combination of creative and scientific solutions.
This dissertation investigates the use of multi-brain EEG-based BCIs in the context of live cinema and mixed-media performances, which is a rather new field bearing distinct characteristics. Using an interdisciplinary approach, a critical overview of the development of the main BCI hardware, software and modes of interaction is presented and relevant works are examined. The aim is to identify the neuroscientific, computational, creative, performative and experimental challenges of the design and implementation of multi-brain BCIs in mixed-media performances, which leads to the main research question:
What might be an effective model for the simultaneous multi-brain interaction of performers and audiences using EEG-based BCIs in the context of live cinema and mixed-media performances?
In order to address the main research enquiry, scientific and practice-based methodologies were combined and a new passive multi-brain EEG-based BCI system was developed. The system was further implemented in the context of the research case study, Enheduanna – A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance (CCA Glasgow 29-31 July 2015). This new work enabled for the first time the simultaneous real-time interaction with the use of EEG-based BCIs of more than two participants, including both a performer as well as members of the audience in the context of a mixed-media performance. The analysis of the participants’ data has most interestingly revealed a correlation between the elements of the performance, which they identified as most special, and their indicators of attention and emotional engagement that were increased during the last two scenes, when their brain-activity was interacting with the live visuals, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live brain-computer cinema performance as a new format of performative work and as a complete combination of creative and scientific solutions. This dissertation also presents the new trends in the field, such as hybrid BCIs, the combination with virtual and mixed reality systems, together with future work.
The introduction of sound wasn’t the only way that people revolutionised cinema, and it won’t be the last.
So what does the history of cinema tell us about the evolution of technology in the arts?
And how can brain activity be used to change the plot of a film? Is there a place for the traditionally passive experience of watching a film to become more interactive, or will that detract from what we value about cinema?
This week, Jordan Erica Webber is joined by the chief curator of the Museum of the Moving Image in New York, David Schwartz, and Dr Polina Zioga, director the Interactive Filmmaking Lab, at Staffordshire University.
Interactive Filmmaking & Cinema by Polina Zioga
Based on the conference’s call for papers, we “are witnessing a shift in all levels of cultural production. As a result, interactivity gains more space and conceptual independence with specific terminology, questions, promises, and challenges. Moreover, interactivity is growing as an academic discipline, a field of study, and an area of research in several spheres.” In a post-pandemic society, we feel that now is an excellent time to be returning to these debates, as the computer-based aesthetic of the multiple such as that of the Zoom interface and similar (which we have all mastered during periods of isolation) have become a newly established fixture of working life.
Meanwhile, in sectoral events, panels have reflected on how exhibitors and audiences have become more accustomed to accessing media experiences online, and this new digital literacy will support cinemas’ efforts to attract cinemagoers in their reopening. Prior to COVID-19, the use of interaction design for new cinematic experiences had attracted the interest of festivals, filmmakers and researchers. In this position paper, we argue that interaction design and technologies can help independent cinemas to engage and galvanise new audiences to patronise COVID-safe venues. From low-end online platforms, to high-end immersive experiences, new technologies are transforming connectivity across society, and have the potential to support access for D/deaf, neurodivergent, and disabled audiences, but adoption by exhibitors is so far limited. We outline the research needs and priorities in this field. These include identifying facilitators and obstacles to industry adoption of interactive forms, and mapping experiences and attitudes across the sector. Together with directions for immediate practical solutions, it is crucial to gather critical data for future research use, in order to pave the way for long-term solutions and design innovation, so that the sector can build resilience, recover and reach underserved audiences.
---
URL: https://www.beyondthemultiplex.net/session-9-audiences-accessibility-and-responding-to-the-pandemic/
---
URL: https://interactivefilm.blogspot.com/2021/07/scriptwriting-for-interactive-crime.html
Interactive Media & Networks by Polina Zioga
This paper presents ‘This Place Has Its Own Air’, a data-driven video-installation and proof of concept for urban air quality data visceralisation. Its title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd, and is concerned with the effect of the city’s air (man-made polluted environment) on the body. The video-installation uses the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience; enabling viewers to have a real-time glimpse of the air they breathe and reflect on the need for reducing the pollution of their environment.
Background:
The quality of the air we breathe, and the air pollution of our environment, are inextricably linked. Air pollutants can spread across long distances and are principally the products of combustion from space heating, power generation, or from motor vehicle traffic. They are monitored and regulated using human health- and environmentally-based criteria for setting permissible levels, as they can cause both short- and long-term negative health effects. In UK and in Scotland, air quality information, such as real-time data, are used to forecast pollution, and are made freely available to the public, often accompanied by health advice and recommended actions for short-term exposure to air pollution. However, despite the improvements in the air quality in recent years, air pollution remains the biggest environmental threat to health in the UK.
Objective & Methods:
The project’s title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd (1893-1981), a Scottish Modernist writer and poet, born in Aberdeenshire. In her work, Shepherd was concerned with the effect of the mountain (natural environment) on the body, and the bodily experience of being in the world (nature). Similarly, our data-driven video-installation is concerned with the effect of the city’s air (man-made polluted environment) on the body. It involves the projection of a live animated moving image that is being processed and rendered using the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience (visceralisation). To achieve this, digital medical images of the human body are used to construct an animated environment that is ambiguous in terms of its origin; and which expands and contracts, as if breathing, following the human adult respiratory rate. The urban air quality data are acquired using the OpenWeather Air Quality API, obtaining hourly readings of the levels of the nitrogen dioxide (NO2) and the fine particles (PM2.5 and PM10), which are the two types of pollutants primarily produced by motor vehicle traffic. Then, the real-time digital processing of the air quality data is performed by a custom software program written in Processing. The levels of the pollutants are scaled and used to control in real-time elements of the live animated moving image. They are mapped to RGB colour values, based on the UK AIR Daily Air Quality Index, which is divided into four bands, from low, to moderate, high, and very high; and generate a real-time colour filter, shifting from cooler to warmer tints. While a particle system appears and overlays different parts of the image, as the concentration of hazardous pollutants rises and falls throughout the day.
The final output, which is silent and designed to have immediacy for presentation in a transient space with a limited dwell time, aims at a ‘meaning-making’ of the environmental themes, which are central to its concept. The man-made intervention (pollution) to the natural environment (air) affects the human bodily experience (breathing), which in turn can prompt a new man-made intervention (reduction of pollution). The resulting video is designed to be displayed on a large vertical screen in a central public building, thus serving as visceralisation of the city’s real-time air quality data.
Conclusion:
The aim of the project is to create an evocative visualisation of the air pollution effect on the body, in order to aesthetically immerse the audiences, promote their critical engagement, and increase the awareness of the need for reducing the environmental pollution. The video-installation will give the city’s residents, and passers-by a real-time glimpse of the air they breathe, evident through the changing colour and the presence of particles in the animated moving image that correspond to the real-time pollutant levels. As a result, the audience is immersed in an evocative imagery that prompts them to reflect on the vital need for reducing the air pollution of their environment, and in the longer-term consider the actions they can take to achieve this change.
New Media Art by Polina Zioga
It includes visual material, video stills and details, from the homonym video installation, but also unreleased material taken from the creative process. Inside the publication's pages can be read Heraclitus’ fragment, which consists the central core and reference of Polina Zioga’s work, the critical text by the curator Zoe Fragoulopoulou, and also a “public-private dialogue” between them, concerning the individual/biological body as an adaptation of a social/collective one, but also the theatrical stage as another space for the presentation of the visual artwork and the representation of the human correlations.
It includes 48 color pages, in 150gr. velvet paper, dimensions 12x18cm, with soft cover.
ISBN: 978-618-80091-0-3
This dissertation investigates the design and implementation of multi-brain electroencephalography (EEG)–based BCIs in the context of live cinema and mixed-media performances. The aim is to identify the interdisciplinary challenges involved and to develop an effective model for the simultaneous multi-brain interactions of performers and audiences.
By combining scientific and practice-based methodologies, a new passive multi-brain EEG-based BCI system was developed and implemented in 'Enheduanna—A Manifesto of Falling', the first demonstration of a live brain–computer cinema performance (CCA Glasgow, July 2015). This new work enabled the simultaneous, real-time interaction—with the use of EEG-based BCIs—of more than two participants, including both a performer and members of the audience, in the context of a mixed-media performance. The analysis of the participants’ data reveals a correlation between the elements of the performance that they identified as most special and their indicators of attention and emotional engagement, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live braincomputer cinema performance, as a novel format of performative work and as a complete combination of creative and scientific solutions.
This dissertation investigates the use of multi-brain EEG-based BCIs in the context of live cinema and mixed-media performances, which is a rather new field bearing distinct characteristics. Using an interdisciplinary approach, a critical overview of the development of the main BCI hardware, software and modes of interaction is presented and relevant works are examined. The aim is to identify the neuroscientific, computational, creative, performative and experimental challenges of the design and implementation of multi-brain BCIs in mixed-media performances, which leads to the main research question:
What might be an effective model for the simultaneous multi-brain interaction of performers and audiences using EEG-based BCIs in the context of live cinema and mixed-media performances?
In order to address the main research enquiry, scientific and practice-based methodologies were combined and a new passive multi-brain EEG-based BCI system was developed. The system was further implemented in the context of the research case study, Enheduanna – A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance (CCA Glasgow 29-31 July 2015). This new work enabled for the first time the simultaneous real-time interaction with the use of EEG-based BCIs of more than two participants, including both a performer as well as members of the audience in the context of a mixed-media performance. The analysis of the participants’ data has most interestingly revealed a correlation between the elements of the performance, which they identified as most special, and their indicators of attention and emotional engagement that were increased during the last two scenes, when their brain-activity was interacting with the live visuals, proving the efficiency of the interaction design, the importance of the directing strategy, dramaturgy and narrative structure. Accordingly, the original contributions of the research include the new passive multi-brain EEG-based BCI system, the live brain-computer cinema performance as a new format of performative work and as a complete combination of creative and scientific solutions. This dissertation also presents the new trends in the field, such as hybrid BCIs, the combination with virtual and mixed reality systems, together with future work.
The introduction of sound wasn’t the only way that people revolutionised cinema, and it won’t be the last.
So what does the history of cinema tell us about the evolution of technology in the arts?
And how can brain activity be used to change the plot of a film? Is there a place for the traditionally passive experience of watching a film to become more interactive, or will that detract from what we value about cinema?
This week, Jordan Erica Webber is joined by the chief curator of the Museum of the Moving Image in New York, David Schwartz, and Dr Polina Zioga, director the Interactive Filmmaking Lab, at Staffordshire University.
Based on the conference’s call for papers, we “are witnessing a shift in all levels of cultural production. As a result, interactivity gains more space and conceptual independence with specific terminology, questions, promises, and challenges. Moreover, interactivity is growing as an academic discipline, a field of study, and an area of research in several spheres.” In a post-pandemic society, we feel that now is an excellent time to be returning to these debates, as the computer-based aesthetic of the multiple such as that of the Zoom interface and similar (which we have all mastered during periods of isolation) have become a newly established fixture of working life.
Meanwhile, in sectoral events, panels have reflected on how exhibitors and audiences have become more accustomed to accessing media experiences online, and this new digital literacy will support cinemas’ efforts to attract cinemagoers in their reopening. Prior to COVID-19, the use of interaction design for new cinematic experiences had attracted the interest of festivals, filmmakers and researchers. In this position paper, we argue that interaction design and technologies can help independent cinemas to engage and galvanise new audiences to patronise COVID-safe venues. From low-end online platforms, to high-end immersive experiences, new technologies are transforming connectivity across society, and have the potential to support access for D/deaf, neurodivergent, and disabled audiences, but adoption by exhibitors is so far limited. We outline the research needs and priorities in this field. These include identifying facilitators and obstacles to industry adoption of interactive forms, and mapping experiences and attitudes across the sector. Together with directions for immediate practical solutions, it is crucial to gather critical data for future research use, in order to pave the way for long-term solutions and design innovation, so that the sector can build resilience, recover and reach underserved audiences.
---
URL: https://www.beyondthemultiplex.net/session-9-audiences-accessibility-and-responding-to-the-pandemic/
---
URL: https://interactivefilm.blogspot.com/2021/07/scriptwriting-for-interactive-crime.html
This paper presents ‘This Place Has Its Own Air’, a data-driven video-installation and proof of concept for urban air quality data visceralisation. Its title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd, and is concerned with the effect of the city’s air (man-made polluted environment) on the body. The video-installation uses the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience; enabling viewers to have a real-time glimpse of the air they breathe and reflect on the need for reducing the pollution of their environment.
Background:
The quality of the air we breathe, and the air pollution of our environment, are inextricably linked. Air pollutants can spread across long distances and are principally the products of combustion from space heating, power generation, or from motor vehicle traffic. They are monitored and regulated using human health- and environmentally-based criteria for setting permissible levels, as they can cause both short- and long-term negative health effects. In UK and in Scotland, air quality information, such as real-time data, are used to forecast pollution, and are made freely available to the public, often accompanied by health advice and recommended actions for short-term exposure to air pollution. However, despite the improvements in the air quality in recent years, air pollution remains the biggest environmental threat to health in the UK.
Objective & Methods:
The project’s title is based on the seminal memoir The Living Mountain (1977) by Nan Shepherd (1893-1981), a Scottish Modernist writer and poet, born in Aberdeenshire. In her work, Shepherd was concerned with the effect of the mountain (natural environment) on the body, and the bodily experience of being in the world (nature). Similarly, our data-driven video-installation is concerned with the effect of the city’s air (man-made polluted environment) on the body. It involves the projection of a live animated moving image that is being processed and rendered using the real-time air quality data of the city where it is presented, to create an evocative visualisation of the bodily experience (visceralisation). To achieve this, digital medical images of the human body are used to construct an animated environment that is ambiguous in terms of its origin; and which expands and contracts, as if breathing, following the human adult respiratory rate. The urban air quality data are acquired using the OpenWeather Air Quality API, obtaining hourly readings of the levels of the nitrogen dioxide (NO2) and the fine particles (PM2.5 and PM10), which are the two types of pollutants primarily produced by motor vehicle traffic. Then, the real-time digital processing of the air quality data is performed by a custom software program written in Processing. The levels of the pollutants are scaled and used to control in real-time elements of the live animated moving image. They are mapped to RGB colour values, based on the UK AIR Daily Air Quality Index, which is divided into four bands, from low, to moderate, high, and very high; and generate a real-time colour filter, shifting from cooler to warmer tints. While a particle system appears and overlays different parts of the image, as the concentration of hazardous pollutants rises and falls throughout the day.
The final output, which is silent and designed to have immediacy for presentation in a transient space with a limited dwell time, aims at a ‘meaning-making’ of the environmental themes, which are central to its concept. The man-made intervention (pollution) to the natural environment (air) affects the human bodily experience (breathing), which in turn can prompt a new man-made intervention (reduction of pollution). The resulting video is designed to be displayed on a large vertical screen in a central public building, thus serving as visceralisation of the city’s real-time air quality data.
Conclusion:
The aim of the project is to create an evocative visualisation of the air pollution effect on the body, in order to aesthetically immerse the audiences, promote their critical engagement, and increase the awareness of the need for reducing the environmental pollution. The video-installation will give the city’s residents, and passers-by a real-time glimpse of the air they breathe, evident through the changing colour and the presence of particles in the animated moving image that correspond to the real-time pollutant levels. As a result, the audience is immersed in an evocative imagery that prompts them to reflect on the vital need for reducing the air pollution of their environment, and in the longer-term consider the actions they can take to achieve this change.
It includes visual material, video stills and details, from the homonym video installation, but also unreleased material taken from the creative process. Inside the publication's pages can be read Heraclitus’ fragment, which consists the central core and reference of Polina Zioga’s work, the critical text by the curator Zoe Fragoulopoulou, and also a “public-private dialogue” between them, concerning the individual/biological body as an adaptation of a social/collective one, but also the theatrical stage as another space for the presentation of the visual artwork and the representation of the human correlations.
It includes 48 color pages, in 150gr. velvet paper, dimensions 12x18cm, with soft cover.
ISBN: 978-618-80091-0-3