See discussions, stats, and author profiles for this publication at: [Link]

net/publication/318845097

Avoid Setup: Insights and Implications of Generative Cinema

Article in Leonardo · August 2017

DOI: 10.1162/LEON_a_01456

CITATIONS READS
3 539

1 author:

Dejan Grba
University of the Arts in Belgrade
34 PUBLICATIONS 123 CITATIONS

SEE PROFILE

All content following this page was uploaded by Dejan Grba on 24 October 2020.

The user has requested enhancement of the downloaded file.

 Avoid Setup: Insights and Implications of
Generative Cinema

Dejan Grba
New Media Department, Faculty of Fine Arts
University of the Arts in Belgrade
Dejan Grba
Pariska 16
11000 Belgrade, Serbia
[Link]@[Link]
ABSTRACT
Generative artists engage the poetic and expressive potentials of film playfully and efficiently, with
See <[Link]/toc/leon/50/4>
for supplemental files associated with this issue. explicit or implicit critique of cinema in a broader cultural context. This paper looks at the incentives,
insights, and implications of generative cinema, which significantly expands the creative realm for
artists working with film, but also incites critical assessment of the business-oriented algorithmic
strategies in the film industry. The poetic divergence, technical fluency, and conceptual cogency
of generative cinema successfully demonstrate that authorship evolves toward ever more abstract
reflection and cognition which equally treat existing creative achievements as inspirations, sources of
knowledge, and tools.

This paper is motivated by the observation that there are complex connections between the
creativity of cinematography and the procedural fluency which is essential to generative art.
These connections have been targeted implicitly or explicitly by the artists of generative cinema
but remain virtually untouched in theoretical discourse. Film studies have traditionally been
focused on the historical, narrative, formal, aesthetic, and political aspects of the relations
among cinema, technology, culture, media, and other art forms. Theoretical studies in new
media art primarily address these relations on the conceptual, material, and phenomenological
levels, investigating and comparing how the different references of information are captured,
stored, manipulated, retrieved, and perceived in film and digital media. In Cinema and the Code,
Gene Youngblood anticipates the creative potentials of the algorithmic foundation of code-based
processing of the formal elements in film, but never explicates them [1].

This paper explores generative cinema by discussing the successful and thought-provoking projects
that represent relevant approaches toward cinema in generative art and exemplify the artists’
abilities to transcend the conceptual, expressive, and aesthetic limits of code-based art. The theme
is observed primarily from the aspect of the artists’ creative thinking and critical evaluation, with
the aim to show that the cognitive tensions between film and generative art have significant
expressive, intellectual, and ethical implications that could benefit both fields. The goal of the
paper is also to encourage further theoretical and practical research in generative cinema.

Generative Cinema
The immense poetic and expressive potentials of film have been barely realized within the
cinematic cultural legacy, mainly due to industrialization, commercialization, politicization,
and consequent adherence to the pop-cultural paradigms [2]. Unrestrained by commercial
imperatives, motivated by unconventional views of film, animation, and art in general,
generative artists have started to engage these potentials playfully and efficiently, with
explicit or implicit critique of cinema in a broader cultural, economic, and political context.

The conceptions of generative art in contemporary discourse vary in inclusiveness [3–8].

In this paper, generative art is perceived broadly, as a heterogeneous realm of artistic approaches

384 © 2017 Dejan Grba | Leonardo, Vol. 50, No. 4, pp. 384–393, 2017 doi:10.1162/LEON_a_01456
based upon combining the predefined elements with different factors of unpredictability in
conceptualizing, producing, and presenting the artwork, thus formalizing the uncontrollability
of the creative process and underlining the contextual nature of art [9,10]. Consequently,
generative cinema is understood as the development and application of generative art
methodologies in working with film both as a medium and as the source material.

Generative cinema has been an emerging field in digital art for the past 20 years. Before that,
generative techniques had seldom been explored in both conventional and experimental film
[11–13]. As a logical extension of generative animation [14], generative cinema in digital art
became feasible with the introduction of affordable tools for digital recording and editing
of video and film. It expanded technically, methodologically, and conceptually with the
development of computational techniques for manipulating large numbers of images, audio
samples, indexes, and other types of relevant film data. Diversifying beyond purely computation-
based generativity—which drew considerable and well-deserved criticism [15]—the production
of generative cinema unfolds into a number of practices with different poetics and incentives.
Here are some examples.

Supercut
Cristian Marclay’s Telephones (1995) used supercut as a generative mixer of conventional
cinematic situations involving phone calls. Supercut is an edited set of short video and/or film
sequences selected and extracted from their sources according to at least one recognizable
criterion. Focusing on specific words, phrases, scene blockings, visual compositions, camera
dynamics, etc., supercuts often accentuate the repetitiveness of narrative and technical clichés
in film and television.

With the explosion of online video sharing, supercut became a pop-cultural genre but remained
a potent artistic device, for example in work by Jennifer and Kevin McCoy, Tracey Moffatt,
Marco Brambilla, and Kelly Mark. It was charged with political and meta-political critique in
R. Luke DuBois’s brilliant Acceptance projects (2012 and 2016) (Figure 1), the two-channel video
installations in which the acceptance speeches given by the two major-party presidential
candidates (Obama and Romney in 2012, Clinton and Trump in 2016) are continuously
synchronized to the words and phrases each of them speaks, which are 75–80% identical but
distributed differently.

The conceptual and technical logic of supercut received a fundamental critical assessment
with Sam Lavigne’s Python applications Videogrep (2014), which generates supercuts by using
the semantic analysis of video subtitles to match the segments with selected words, and
Audiogrep (2015), which transcribes audio files and creates audio supercuts based on the input
search phrases.

Statistical
Classification, indexing, and systematic quantification of formal qualities in time-based media
allow for building databases that can be handled and manipulated with statistical tools. This
enables artists to make alternative visualizations and temporal mappings that reveal the overall
visual and structural logic of popular films.

The idea of unconventional editing and presentation of film has been explored in a number of
projects. Soft Cinema: Navigating the Database (2002–2003), by Lev Manovich and Andreas
Kratky, demonstrates Manovich’s view of the cinema as a digital (discrete) medium and of the
film as a database. The project was based on classifying and tagging a set of stored video clips,

Avoid Setup: Insights and Implications of Generative Cinema | Grba 385

 Figure 1. R. Luke DuBois, Acceptance 2016, 2016. (© R. Luke DuBois)

algorithmically creating the editing scenarios in real time, and on devising a user interface for
arranging, navigating, and playing the material [16].

Programmed manipulation of digitized film also enables artists to statistically process films
frame by frame, for example in Ben Fry’s Disgrand (1998), Ryland Wharton’s Palette Reduction
(2009), and Jim Campbell’s Illuminated Average Series (2000–2009), which averages and merges
all the frames from Orson Welles’s Citizen Kane (1941) and Hitchcock’s Psycho (1960) [17].

In Portrait (2013) (Figure 2), Shinseungback Kimyonghun used computer vision in the statistical
style of Jim Campbell and Jason Salavon. The software detects faces in every 24th frame of a
selected movie, averages and blends them into one composite with the dominant facial identity
of a movie, stressing the figurative paradigm in mainstream cinema.

The classic conceptual, formal, and experiential form of the infographic processing of film was
achieved in Frederic Brodbeck’s Cinemetrics (2011). Its core is a Python-based online application

Figure 2. Shinseungback Kimyonghun, Portrait, 2013, with images from Bourne Identity (2002) and Taxi Driver (1975).
(© Shinseungback Kimyonghun)

386 Grba | Avoid Setup: Insights and Implications of Generative Cinema

for interactive visualization and analysis of loaded films according to a number of criteria such as
duration, average luminance and chromatic values, number of cuts, dynamics of movement in
sequences, comparisons between different genres, original film versions vs. remakes, films by the
same director, films by different directors, etc. [18].

Crowdsourced
As an old method for outsourcing complex, iterative, or otherwise demanding projects to many
participants who are expected to make relatively small contributions, crowdsourcing has
significantly evolved with the internet (and has often been skillfully exploited), from the SETI@
home screensaver in the early WWW, to FoldIt, Kickstarter, Wikipedia, CAPTCHA, social
networking, and social media platforms.

In Man with a Movie Camera: The Global Remake (2008), Perry Bard combines online participation
with automatic selection of crowdsourced video clips to make a shot-by-shot interpretation of
Dziga Vertov’s seminal eponymous film, Man with a Movie Camera (1929). A similar idea, the
surrealist “exquisite corpse” method for sequential collaging of found video clips, is behind João
Henrique Wilbert’s Exquisite Clock (2009), which constructs a digital clock with six screens
showing the uploaded users’ free-style photographic interpretations of decimal digits.

With The Pirate Cinema (2012–2014) (Figure 3) Nicolas Maigret brings real-time robotic sampling
of film to the world of peer-to-peer exchange. The installation uses a computer that constantly
downloads the 100 most-viewed torrents on a tracker website, intercepts the currently
downloading video/audio snippets, projects them on the screen with the information on their
origins and destinations, discards them and repeats the process with the next stream in the
download queue [19].

The idea of expanding the conventional film structure with crowdsourced, programmatically
arranged, and interactively manipulable content was polished up and designed to consequently
reflect the logic of online video sharing in Jono Brandell and George Michael Brower’s Life in a
Day Touchscreen Gallery (2011). It is a highly configurable platform for organizing, sorting, and

Figure 3. Nicolas Maigret, The Pirate Cinema, 2012–2014. (© Nicolas Maigret)

Avoid Setup: Insights and Implications of Generative Cinema | Grba 387

 screening the clip selections of all the 80,000 short video submissions to a traditionally scripted
and edited crowdsourcing film, Life in a Day (2010), directed by Kevin Macdonald, which used
around 10,000 selected video clips. The fact that Touchscreen Gallery was a sideshow instead of
being central to the Life in a Day project reflects the dominant ideology of mainstream cinema.

Deanimated
One of the most impressive critical deconstructions of the structural and audio-visual
conventions in cinema was achieved by Martin Arnold with Deanimated (2002) (Figure 4).
He successively removed both visual and sonic manifestations of the actors in Joseph H. Lewis’s
B-movie thriller The Invisible Ghost (1941), and then consistently retouched the image and sound
so that the film’s final 15 minutes show only empty spaces accompanied by the crackling of the
soundtrack [20].

Figure 4. Martin Arnold, Deanimated, 2002, with corresponding stills from Invisible Ghost (1941) (left) and Deanimated (right).
(© Martin Arnold)

Similarly motivated to avoid the figurative and narrative dictates of film tradition, Vladimir
Todorović combines generative animations with voiceover narration and ambient soundtrack
in The Snail on the Slope (2009), Silica-esc (2010), and 1985 (2013). 1985 (Figure 5) is an abstract
rendition of the fictional activities of the ministries of Peace, Love, Plenty, and Truth that govern
Oceania one year after the events in George Orwell’s 1984 (1949). Its uncanny ambience relies on
sudden changes of sound and image, triggered by the random walk algorithm that was modified
with cosine function, accelerated and decelerated.

Documentary narrative structure can also be transcended, for example in Jonathan Minard and
James George’s computer film CLOUDS (2015), which dynamically links real-time generative
animations and sound with prerecorded documentary footage.

388 Grba | Avoid Setup: Insights and Implications of Generative Cinema

Figure 5. Vladimir Todorović, 1985, 2013. (© Vladimir Todorović)

Condensed
In Fast Film (2003) (Figure 6), Virgil Widrich intelligently expanded the possibilities for
reproducing and interpreting film snippets in order to accentuate the fascinations, obsessions,
and stereotypes of conventional cinema. Fast Film was created by paper-printing the frames
from selected film sequences, reshaping, warping, and tearing them up into new animated
compositions. In its exciting 14 minutes of runtime, Fast Film provides an elegant and
engaging critical condensation of key cinematic themes, such as romance, abduction, chase,
fight, and deliverance.

Nine years later, György

Pálfi exploited this
narrative methodology,
along with the
achievements of supercut
art and culture, to produce
the feature-length movie
Final Cut: Ladies and
Gentlemen (2012) out of
short sequences from 450
popular films and cartoons.
Although it proved to be
Figure 6. Virgil Widrich, Fast Film, 2003. (© Virgil Widrich)
barely watchable in
continuity due to the fundamental incompatibility between rapid editing of incoherent imagery
and its long running time, film critics praised it as “an ode to cinema” [21].

Synthesized
The concept of real-time procedural audiovisual synthesis from an arbitrary sample pool, in
contrast, elevates the film structure by following the essential logic of cinema. This was achieved
by Sven König in sCrAmBlEd?HaCkZ! (2006), which uses psychoacoustic techniques to calculate
the spectrum signatures of the sound snippets from stored video materials and saves them in a
multidimensional database that is searched in real time to mimic any input sound by playing the
matching audio snippets and their corresponding videos [22]. Perhaps this innovative project was

Avoid Setup: Insights and Implications of Generative Cinema | Grba 389

 largely overlooked because König used the sCrAmBlEd?HaCkZ! software mainly for VJing rather
than for developing complex artworks by establishing the specific relations between the sources
of stored and input materials.

Procedural audiovisual synthesis was advanced through the application of neural networking
and machine learning by Parag Kumar Mital in YouTube Smash Up (Figure 7). Each week, this
online software takes the #1 YouTube video of the week and resynthesizes it using an algorithm
that collages appropriate fragments of sonic and visual material coming only from the remaining
Top 10 YouTube videos [23]. It produces a surreal animated effect, visually resembling
Arcimboldo’s grotesque pareidolic compositions [24].

Figure 7. Parag Kumar Mital, YouTube Smash Up: Emotional Baby! Too Cute!, 2012. (© Parag Kumar Mital)

The more demanding, machine-based synthesis of coherent film structure and plausible narrative
was tackled by Oscar Sharp and Ross Goodwin in Sunspring (2016) (Figure 8), which was their
entry in the 48-Hour Film Challenge of the Sci-Fi London film festival. Experienced in natural
language processing and neural networks, Goodwin programmed a long short-term memory
recurrent neural network and, for the learning stage, supplied it with a number of 1980s and
1990s sci-fi movie screenplays found on the internet. The software, which appropriately “named”
itself Benjamin, generated the screenplay as well as the screen directions around the given
prompts, and Sharp produced Sunspring accordingly.

The film brims with awkward lines and plot inconsistencies, but it qualified among the top 10
festival entries and inspired one of the judges to remark, “I’ll give them top marks if they
promise never to do this again” [25]. Sunspring playfully reverses the “Deep Content” technology
of the [Link] web service, which analyzes transcripts, audiovisual patterns, and
any form of data feed that describes the video content itself, and automatically converts it into
advanced metadata which is then processed by a machine learning system that matches the
metadata with the natural language queries [26].

A Void Setup
All these approaches in generative cinema point to the powerful algorithmic concepts for freely,
parametrically, and/or analytically generating cinematic structure, narrative, composition,

390 Grba | Avoid Setup: Insights and Implications of Generative Cinema

Figure 8. Oscar Sharp and Ross Goodwin, Sunspring, 2016. (© Oscar Sharp and Ross Goodwin)

editing, presentation, and interaction. One such concept proposes a flexible system for automatic
arrangement of manually tagged film clips, or their arrangement according to input parameters
[27]. A more complex one would be able to combine computer vision, semantic analysis, and
machine learning to recognize various categories and reconstruct plots from a set of arbitrarily
collected shots, sequences, or entire films, and to transform and reconfigure these elements
according to a wide range of artist-defined criteria that substantially surpass those in
conventional film.

The algorithmic tools of generative cinema significantly expand the realm of creative
methodologies for artists working with film and animation. They provide artists with new
insights into conceptual, formal, and expressive elements of film and animation, which can be
enhanced through experimentation. Furthermore, the algorithmic principles of the successful
generative cinema artworks, regardless of their technical transparency, can be inferred,
repurposed, and developed into new projects with radically different poetic identities and
outcomes. These creative capacities also provide a specific context for the critical assessment
of conventional film.

Just as it clumsily borrowed or repurposed ideas from the avant-garde, mainstream cinema has
been systematically exploiting some aesthetic effects and themes of digital generative art, with
little understanding of the intellectual values behind generative methodologies. This superficial
exploitation is revealed in goofs spotted by informed viewers. When the commercial film tries
to utilize algorithms as creative tools, it does so ineptly and ineffectually, reflecting its rigid
ideology, as exemplified by Macdonald’s Life in a Day and Pálfi’s Final Cut.

The algorithmic strategies that the film industry applies successfully are those for
conceptualization, script evaluation, box-office assessment, and other business-related aspects of
production, distribution, and marketing. Major production companies, such as Relativity Media
in Hollywood, use statistical processing of screenplay drafts, while consulting services, such as
Epagogix, offer their clients the big-data–based predictions of their films’ market performance
[28–30]. The outcry over the ultimate loss of creativity, provoked by media disclosures on these
practices is, however, either naive or cynical because business-related algorithms have always
been integral to big-budget filmmaking [31].

Struggling with competitive new media and art forms, the film industry today is unable to
transcend and unwilling to hide its reliance on communicating a subset of human universals [32].

Avoid Setup: Insights and Implications of Generative Cinema | Grba 391

 Therefore, it runs its business more consciously and rationally, focusing the algorithms on market
analysis, target group research, risk assessment, and screenplay design, all the way to the test-
screening evaluations corresponding to the debugging procedures in computer coding. While
this pragmatic algorithmization seems logical, it is creatively counterproductive. A global
mass-market film industry could benefit from generative cinema only if it takes certain
commercial risks and opens up to the experimental incentives of its creative talents.

Unrestrained by commercial imperatives, motivated by unconventional views on film, animation,

and art in general, generative artists develop new approaches and methodologies that can be
advanced and repurposed by other artists, stir our amazement with the moving image, and
broaden our critical understanding of the cinema as cultural product. In this regard, the poetic
divergence, the technical fluency, and the conceptual cogency of generative cinema successfully
demonstrate that authorship evolves toward ever more abstract reflection and cognition that
equally treat existing creative achievements as inspirations, sources of knowledge, and tools.

References and Notes

1. G. Youngblood, “Cinema and the Code,” Leonardo, Computer Art in Context Supplemental Issue,
27–30 (1989).
2. W. Benjamin, “The Work of Art in the Age of Its Technological Reproducibility,” The Work of Art in
the Age of Its Technological Reproducibility, and Other Writings on Media (Cambridge, MA/London:
The Belknap Press of Harvard University Press, 2008).
3. P. Galanter, “What Is Generative Art? Complexity Theory as a Context for Art Theory,” VI Generative
Art Conference Proceedings (Milan: Domus Argenia Publisher, 2003) pp. 225–245.
4. I. Arns, “Read_me, run_me, execute_me. Code as Executable Text: Software Art and its Focus on
Program Code as Performative Text,” Medien Kunst Netz, <[Link]
generative-tools/read_me/1/>.
5. D. Quaranta, “Generative (Inter)Views: Recombinant Conversation With Four Software Artists” in
D. Quaranta, ed., [Link]. Art Electronic Systems and Software Art (Turin: Teknemedia, 2006).
6. M. Boden and E. Ernest, “What Is Generative Art?”, Digital Creativity 20, No. 1–2, 21–46 (2009).
7. M. Watz, “Closed Systems: Generative Art and Software Abstraction” in E. de Lavandeyra Schöffer,
M. Watz, and A. Doms, eds., MetaDeSIGN - LAb[au] (Dijon: Les presses du reel, 2010).
8. M. Pearson, Generative Art (Shelter Island, NY: Manning Publications, 2011).
9. A. Dorin et al., “A Framework for Understanding Generative Art,” Digital Creativity 23, No. 3–4,
239–259 (2012).
10. D. Grba, “Get Lucky: Cognitive Aspects of Generative Art,” in XIX Generative Art Conference
Proceedings (Venice: Fondazione Bevilacqua La Masa, 2015) pp. 200–213.
11. M.B.N. Hansen, “Digital Technics Beyond the ‘Last Machine’: Thinking Digital Media with Hollis
Frampton (Modulating Movement-Perception)” in E. Røssaak, ed., Between Stillness and Motion: Film,
Photography, Algorithms (Amsterdam: Amsterdam University Press, 2015) pp. 45–72.
12. M. Leggett, “Generative Systems and the Cinematic Spaces of Film and Installation Art,” Leonardo 40,
No. 2, 123–128 (2007).
13. Youngblood [1].
14. N. Montfort et al., 10 PRINT CHR$(205.5+RND(1));: GOTO 10 (Cambridge, MA: The MIT Press,
2012).
15. Arns [4] and Watz [7].
16. L. Manovich and A. Kratky, Soft Cinema: Navigating the Database (Cambridge, MA: The MIT Press,
2005).
17. J. Campbell, “Illuminated Average Series,” <[Link]
illuminated_averages/[Link]>.
18. F. Brodbeck, “Cinemetrics,” <[Link]

392 Grba | Avoid Setup: Insights and Implications of Generative Cinema

19. G. Cox, “Real-Time for Pirate Cinema,” PostScriptUM, No. 20 (2015).
20. G. Matt and T. Miessgang, Martin Arnold: Deanimated (Vienna: Kunsthalle Wien / Springer
Verlag, 2002).
21. Q.P., “Final Cut—Ladies and Gentlemen, an Ode to Cinema,” Festival de Cannes (2012), <http://
[Link]/en/theDailyArticle/[Link]>.
22. S. König, “sCrAmBlEd?HaCkZ!,” YouTube, <[Link]
23. P.K. Mital, “Computational Audiovisual Scene Synthesis,” PhD thesis (Arts and Computational
Technologies Goldsmiths, University of London, 2014).
24. S.F. Pagden, Arcimboldo (Milano: Skira, 2007).
25. A. Newitz, “The Multiverse / Explorations & Meditations on Sci-Fi,” Ars Technica, Jun 9 (2016),
<[Link]
26. Valossa, “Description of Research,” <[Link]
27. Q.C. Berga, “Code as a Medium to Reflect, Act and Emancipate: Case Study of Audiovisual Tools
that Question Standardised Editing Interfaces,” Interface Politics: 1st International Conference (2016).
28. C. Jones, “Ryan Kavanaugh Uses Math to Make Movies,” Esquire, Nov 19 (2009), <[Link]
[Link]/news-politics/a6641/ryan-kavanaugh-1209/>.
29. B. Barnes, “Solving Equation of a Hit Film Script, with Data,” The New York Times, May 5 (2013),
<[Link]
html>.
30. S.V. Smith, “What’s Behind the Future of Hit Movies? An Algorithm,” Marketplace, July 19 (2013),
<[Link]
31. L. Buñuel, My Last Breath (London: Fontana Paperbacks, 1985) pp. 131–133.
32. D.E. Brown, Human Universals (New York, NY: McGraw-Hill, 1991).

Avoid Setup: Insights and Implications of Generative Cinema | Grba 393

View publication stats