Infinite Regress:

The Blurring of an Architectural Game-Space

By Dariusz Jacob Boron, B.A.S

A thesis submitted to

The Faculty of Graduate Studies and Research in partial fulfillment of

the requirements for the degree of

Master of Architecture

School of Architecture
Carleton University
Ottawa, Ontario
2004-2006

© Dariusz Jacob Boron 2006.

i
 Abstract

Architecture and gaming have always been unconnected. Architecture uses a vast array

of tools as a means to an end, to show information. The computer gaming industry

however, creates virtual worlds for its experiential potential. While this disjunction is

explicit I believe that the gaming industry presents architecture with a medium from

which to expand its expression. I explore the redundant nature of the space created when

two or more spatial conditions are imbedded/layered within each other. By investigating

the separate industries of architecture and gaming, I hope to blur and possibly alleviate

the distinctions between the tools that architects and game designers use. With growth in

gaming technologies and superior compatibility, architects should have the capacity to

work between these two worlds. With such technologies architects or designers can

further enhance their design by experiencing their designs through a virtual, immersive

and interactive real-time environments.

ii
 Acknowledgements

I would like to thank quite a few people for your help and support.
Kate for your love and patience with me.
Ben for your extensive help and support in all of the stages of the thesis.
Mike Getz for your help with the in-school online Quake game.
Tom Bonhomme for helping with editing.
Friends and Family who supported me.
All support staff from Carleton University and the School ofArchitecture.

in
dla m a m y i taty

iv
 Table of Contents

Chapter 1 1
1.1 Introduction 1
1.2 Thesis Statement 1
1.3 Structure of the Thesis 3

Chapter 2. Background 1: Games 6

2.1 Introduction 6
2.2 Space and Immersivity in Computer Games 8
Text-only Space (no pictorial dimension) 10
Contained 2-dimensional space 10
Wrap-around space 10
Spaces that scroll along one axis 11
Scrolling along two axes 11
Adjacent spaces displayed in one room at a time 14
Scrolling with multiple background layers 14
Limited 3-dimensional space - the use ofperspective in the scene... 16
Two spaces on one screen or two separate screens 16
Early 3d space - one horizontal plane 16
Full 3d Space — horizontal/vertical movement 18
Mapped space - pre-rendered panoramas 18
2.3 Viewpoints: First-Person, Third-Person, and Overhead Views 20
2.4 Gaming Technology and Terminology 22
Game Engines 22
Polygons 24
Texture and image mapping 24
Real-time light and shadow mapping 24
Normal Mapping and other technologies 25
Dynamics and in-game physics 26
Artificial intelligence 27
The Internet's influence on the gaming industry 27
Mega MultiPlayer Role Playing Games (MMRPGs) 29
2.5 Conclusion 31

Chapter 3: Background 2: Architecture 32

3.1 Introduction 32
3.2 Digital Tools in the A E C industry 33
BIM - Building Information Modeling 34
GIS - Geographical Information Systems 35
Computer Simulations - structural performance, acoustics,
environmental factors, movement and lighting 35
3.3 G a m e Engines 37
3.4 Conclusion 39

v
Chapter 4: Crossover Environments 42
4.1 Introduction 42
4.2 Meta-Communication 43
Immersion and Interactivity 44
Immersive Fallacy & Immediacy 44
The virtual as a threshold into the real 46
Disembodied consciousness/Virtual bodies 47
Sentient environments 47
Parallel and hybrid worlds 50
Category 1: Non-parallel; non-interactive 49
A painting on the wall 49
An LCD surface 49
A television or book in crowded room 49
Category 2: Parallel and non-interactive 50
A mirror on a wall 50
Category 3: Non-parallel; interactive 50
A phone call in crowded room 50
Mega-multiplayer environments 50
Category 4: Parallel; interactive 50
Video conferencing 51
Doug Back's gallery installation 51
Supercity and Karlskrona2 52
4.3 Conclusion 55

Chapter 5: Experimentation 56
5.1 Introduction 57
5.2 Theory and Terminology 60
Information and Noise 60
Redundancy 60
Convergence 61
5.3 Experiments, Scenarios and Case Studies 63
Quake III Arena - School ofArchitecture real-time model 63
The QTVR medium 66
TDR - virtual transformation 69
Real, Virtual and the Intermediate 71
Surrounding projection 77
Surrounding virtual projection using VR goggles 80

5.4 Conclusion 82

Chapter 6: Conclusion 83

Endnotes 90
Bibliography 91

vi
 List of Illustrations

Fig. 1 Pong. From: Atari games (5 Oct 2005)

<[Link]

Fig. 2 Fig. 2 Pacman. From: Quest [Link] (5 Feb 2005)

<[Link]

Fig. 3 Super Mario Bros. 3 From: [Link] (5 Feb 2005)

<[Link]

Fig. 4 Simcity. From: [Link] (5 Feb 2005)

<[Link]

Fig. 5 Berzerk From: [Link] (5 Feb 2005)

<[Link]

Fig. 6 Double Dragon From: [Link] (5 Feb 2005)

<[Link]

Fig. 7 Tempest. From: [Link] (5 Feb 2005)

<[Link]

Fig. 8 Spy vs. Spy. From: [Link] (5 Feb 2005)

<[Link]

Fig. 9 Screenshot of Doom. From: [Link] (5 Feb 2005)

< [Link] >

Fig. 10 Screenshot of Quake. From: [Link] (5 Feb 2005)

<[Link]

Fig. 11 Screenshot of Myst IV. From: [Link] (5 Feb 2005)

<[Link]

Fig. 12 Screenshot of Half-life2. From: [Link] (5 Feb 2005)

<[Link]

Fig. 13 A character of likeness is created in Uru, from the Myst series.

Fig. 14 Screenshot of Age of Empire 3. From: [Link] (5 Feb 2005)

<[Link]

Fig. 15 Screenshot of Half Life 2. From: [Link] (5 Feb 2005)

<[Link] .html>

vii
Fig. 16 World of Warcraft Screenshot. From [Link] (5 Feb 2005)
<[Link]

Fig. 17 World of Warcraft Screenshot. From [Link] (5 Feb 2005)

<[Link]

Fig. 18 Screenshot from Autodesk's AutoCad. Using 3d modeling to examine buildings

as a whole.

Fig. 19 Karlskrona2 From: <[Link] (5 Feb 2005)

<[Link]

Fig. 20 Karlskrona2 From: <[Link] (5 Feb 2005)

<[Link]

Fig. 21 Full-Circle 360° Camera Module From: [Link] (5 Feb 2005)

<[Link]

Fig. 22 Partial Panorama created by merging many photographs together.

Fig. 23 Combining Quake with windows into the real via webcam.

Fig. 24 Real looking into a virtual screen, seeing a virtual representation of a real per
captured in real space, projecting that into the virtual.

Fig. 25 Real photography and stitching.

Fig. 26 Digital panorama.

Fig. 27 Combining real photography and digital panorama.

Fig. 28 School of Architecture & digital model played within the Quake game.

Fig. 29 School of Architecture model as seen through the character within Quake.

Fig. 30 Merging the real within a virtual context.

Fig. 31 Real and Virtual. Interchangeable?

Fig. 32 A real 360 surround screen, projecting one's presence into another.

Fig. 33 A virtual 360 surround screen.

 1

Chapter One

1.1 Introduction

Where realistic environments, interaction and immersivity are concerned, game worlds

have begun to encroach on the realm of architecture - blurring distinctions between

physical and virtual experience. Moreover, as a function of the drive for realism, more

and more game spaces are presented perceptively in "first person," as one would

experience an actual built environment. When you factor in the real-time interaction

between individuals supported by Internet-based gaming, game environments have the

potential to perform key functions we associate with architecture, namely the creation of

meaningful places to stage and support social exchange. Finally, when these virtual

places are connected back into the real via real-time digital video feeds, virtual

environments can function as thresholds or portals back into the real.

1.2 Thesis Statement

Despite the increasing degree of overlap between gaming and architecture, architects are

mostly ignorant of developments in the gaming industry - often willfully so. That being

said, architects use a vast array of digital tools as a means to an end, namely to model

buildings that are intended to be constructed. While the digital model is also a means to

an end for a game developer, the end is different. For the architect the virtual model is

(only) a representation, whereas a game developer creates virtual worlds to support

inhabitation and experience. While architects may legitimately believe that the real

trumps the virtual, the existence of the real may not obviate the usefulness of the virtual

version. Indeed the digital model can perform a different function once the building
 2

modeled has been designed and constructed. In this respect the digital model can

transcend its function as a mere tool or means to an end - expanding and enhancing the

experience of the actual building.

It is the overlap and redundancy between the real (as represented by architecture) and the

sophisticated virtual environment (as manifested in contemporary computer games) that I

propose to explore in this thesis - both from the perspective of the design professional

and of the user/player. Using a number of spaces, scenarios and digital media, I intend to

explore the ambiguous, provisional and redundant nature of space created when real and

virtual spaces are embedded/layered within each other. In so doing, I also hope to

encourage a dialogue between architects and game developers, to promote awareness and

use of new technologies among architects, and to expand the definition of architecture.

This thesis explores the potential of redundancy, overlap and blurring between the realms

the real and the virtual, of gaming and architecture.

In recent years game companies have encouraged the gaming public to participate in the

creation of levels (game environments) by placing their level builders in the public

domain. While researching this thesis I've come across several studies assessing the

potentials of game engines for architectural modeling.1 While I, too, use a level builder

(Quake 3 Arena) to model an actual building (the School of Architecture at Carleton

University), the thrust of the thesis is more theoretical than practical. Again, I wish to go

beyond the use of the model or level as an architectural design/presentation tool.

 3

1.3 Structure of the thesis

The thesis comprises both written and design work - design being used as a form of

research. I begin by introducing computer-based games in Chapter 2, looking at key

developments, categories and technologies. In Chapter 3 I look at the use of digital tools

by architects - the ways their means and ends differ from those of game developers.

Chapter 4 examines "crossover environments" - digital environments that, whether by

their function or the technologies they incorporate, challenge distinctions between the

real and the virtual.

Chapter 5 describes the design portion of the thesis - explorations into embedding the

virtual within the real. This includes the use of real-time video feeds into virtual spaces,

the incorporation of photography, the use of QuickTime VR panoramas, etc. The aim is

to discover new spaces, merge/blur the virtual and the real, expand the definition of space

in the realm of architectural, and to displace and redefine the experience of the viewing

subject within redundant and parallel spaces.

Architects, as noted, are comparatively conservative when it comes to conceptualizing

the potentials of virtual environments. While they use many of the same tools as game

developers, the digital models they create remain provisional and defer to the real

environments they represent. Generally digital models are abandoned once the

architectural design moves into the construction documents phase. More often than not

buildings are modeled as objects (to assess their impact on the city or adjacent

environment) rather than as spaces to be explored or navigated. Interactivity is, for a

 4

variety of reasons, not a priority for architects, who generally use the model to generate

rendered stills or pre-rendered fly-throughs.

At the same time, sophisticated computer games have begun to transcend their function

as entertainment and have become spaces for interaction. In this sense game

environments increasingly resemble the environments architects create. Massively

Multiplayer Online Role-Playing Game (MMORPGs) bring thousands of people together

for interaction, via characters. When players interact, play may halt altogether,

transforming the game into a "chat room" or vehicle for communication. Such game

environments are tantamount to real world spaces both in their function and in their

qualities (sound, light and shadow, physical constraints, and architectural detail).

When virtual environments are examined outside their function as design and

presentation tools and/or gaming levels, they take on a different, potentially redundant

quality. The design portion of this thesis attempts to blur the distinction between

architecture and game design by embedding virtual versions of real environments within

those environments then feeding the real back in the virtual. Existing simultaneously

within real and virtual versions of the same environment, the viewing subject is displaced

- watching himself watch himself through various layers of mediation- simultaneously

positioned in the roles of person, player and character. Such redundancy and "feedback"

is facilitated by technologies such as web cams that permit the virtual to come in contact

with real space.

 5

Leveraging gaming technologies to bridge between the real and the virtual would allow

architects to examine their work in new ways. Real-time feeds into photorealistic and

immersive virtual worlds can vastly expand the quantity and quality of spaces that

architects design - breaking down boundaries, blurring distinctions, and opening up new

possibilities. As the future brings more options for how and where we interact as human

beings and as the line between the real and the virtual continues to erode, architecture's

role can significantly transform to allow humans to interact in a broader choice of

settings - whether real, virtual or indeterminate.

 6

Chapter 2: Background One - Games

"Gaming is now a bigger and more profitable enterprise than the film and
television industries combined ($6.3 billion in video games sold worldwide in
1999, and online gaming is a $250 million industry). "

2.1 Introduction

A game is an entertaining and participatory activity for one or more persons. Academics

Katie Salen and Eric Zimmerman note that games are "capable of addressing the most

profound themes of human existence in a manner unlike any other form of

communication."3 Games confirm our desires, evoke pleasures, anxieties, release,

wonder and knowledge. Whether open-ended, procedural and/or collaborative, games

provide a framework that structures choices and responds to the actions of the player(s).

The rules of the game also structure interaction between players - who play both against

the game and each other. Games can challenge our problem-solving skills, enhance

physical coordination, and help to nurture relationships.

To help games appeal to the imagination, especially computer games, a narrative is often

employed. A narrative is a story or a tale communicated via word of mouth or written

text. Early version computer games of Dungeons and Dragons were text-based. The

narrative would structure the choices presented to players and provides a context within

which to make decisions. In the case of Dungeons and Dragons the narrative is different

for every player. This kind of game is called a role (or roll) playing game because

players select a character and play in turns.

 7

Games may be categorized in a variety of ways: single vs. multiple player, card games,

board games, role playing, adventure, action, strategy, simulation, sports, god games,

educational, puzzle, online, single player, etc. Given the digital focus of this thesis,

however, we will look specifically at categories and characteristics of computer games.

One of the major goals for which computer game companies strive is realism. Realistic

environments can enhance the immersive experience of games by more effectively

evoking feelings of fright, shock, anxiety, etc. Similarly, many computer games support

"pure playability" by fantasy-based visuals. In this form, the most important value is

entertainment through imagination. Rather than creating highly realistic environments,

these games deploy abstraction and color to foster imagination and problem solving.

Early in their development computer games only represented reality through

diagrammatic symbols (e.g., the arcade console games of the 1970s). As computer

gaining developed, however, realism improved in all genres and highly detailed three-

dimensional models replaced 8-bit sprites (2D, flat, animated images). Today, layered-

image backgrounds have been supplanted by richly detailed, three-dimensional cities and

landscapes.

The drive for realism has characterized computer gaming from its inception and has, to

some degree, become synonymous with the industry. Arguably realism is implicit in the

medium ~ as is the assumption that greater realism equals more immersivity. Equally

important considerations such as playability, music, narrative, challenge and character

development appear secondary to graphical output. Rather than encouraging players to

 8

decipher symbolism within a game, they are increasingly presented with photo-realistic

environments.

To better understand relevant parallels between architecture and game development, this

chapter will outline key themes, many technical. I will define games and trace the

evolution of computer gaming using categories such as spatial depth, point of view, level

of immersion, and realism. I'll also discuss current technologies, tools and benchmarks

in the gaming industry, including photographic image mapping and real-time video feeds.

Tracing a chronology of digital gaming will help us better understand the hyper-realism

that characterizes the industry today (in the form of first-person navigation and highly

detailed massive-multiplayer environments). The term immersive fallacy describes the

phenomenon of empathetic transposition between the individual player and a character

within in the game. Analogous to the concept of "suspension of disbelief' in films, it

describes the level of immersion and absorption into the action and the degree to which

one forgets that he is playing and/or is different than the character. Realism, first-person

viewpoints, perspectival space and technological innovation all serve the immersive

fallacy in game design.

Understanding the state of gaming technologies, in turn, will inform our assessment of

the use of digital modeling in architecture and will set the stage for the experiments

described in Chapter 4.

2.2 Space and Immersivity in Computer Games

Although games can be categorized in a numbers of ways, it is perhaps most relevant to

categorize computer games with respect to their immersive qualities. The representation
 9

of space (boundaries, depth and perspective) has a direct impact on the degree of

immersion. While it is certainly not the only factor, it is an important one - and is

particularly relevant to the relationship between gaming and architecture.

When discussing space and the movement (of objects, characters, etc.) though space, it is

important to differentiate pre-rendered scenes from scenes that are rendered on the fly.

Free movement though perspectival space is extremely computation-intensive. Until

quite recently there was an inverse relationship between freedom of movement and the

sophistication of the scene. A hybrid model (used extensively in the Myst series) is

movement thorough linked, pre-rendered QuickTimeVirtualReality (QTVR) panoramas.

Unlike first-person shooter (FPS) games like Quake and Far Cry, the Myst player

"jumps" from panorama to panorama, each of which is rendered in great detail. Although

the viewer is stationary in each panorama, his viewpoint can rotate 360 degrees in all

directions. In recent versions of Myst, these panoramas include video clips of characters

moving though the space while the player looks around.

With faster processors, more efficient texture mapping and better rendering algorithms, it

is now possible to support both full movement through perspectival space and high levels

of detail. The Myst cousin Uru is an example of such a game - extremely rich in detail

(all of which the player must scrutinize for clues to the unfolding drama) but supporting

real-time rendering and fluid movement in both first and third person (see section 2.3

below).
 10

Text-only Space (no pictorial dimension)

Text-space, which is a form of immersion, was introduced in the 1970s with games such

as Dungeons and Dragons. In this form, the player simply had to read and interpret a text

within a character/computer interactive system, and imagine a world while being guided

through the text.

Contained 2-dimensional space

Following text-based games, graphical games such as Pong and Space Invaders used

pixels to draw graphics as representations of objects. In Pong, for example, a spot

represented a ball and vertical bands of pixels represent paddles. The interaction between

the spot and bands obeyed the laws of simple physics - allowing the player to relate the

play to tennis or ping-pong. From their inception graphical games deployed symbolism

and metaphor; graphics were imputed with a metaphorical value to foster an immersive

relationship to play. The associational metaphor enabled the user to suspend disbelief

and intuitively relate the game world to the real world. Metaphors included gravity,

grade, danger, friend, foe, objects, ice and so on. Without the use of real world metaphors

game-play would be vague and ambiguous. To encourage immersion in the game,

players needed to relate the digital events to real world experience.

Wrap-around space

This graphical interface of games like Asteroids and Pacman warped space by permitting

objects to disappear off the edge of the screen only to reappear directly opposite to where

they vanished. This gave the impression that the game space was continuous and

permitted the subconscious to extrapolate and travel through an invisible space. Such
 11

continuity of space is understandable in the game environment yet impossible in real

scenarios.

Spaces that scroll along one axis

Games such as Defender, Super Mario Brothers, and Atari's football game used scrolling

to give the impression of continuous movement through a two dimensional space. The

Super Mario Bros series featured horizontal movement while games such as Spy Hunter

privileged vertical movement. Continuous horizontal movement was common in

animations such as the Flintstones where characters stayed within one plane and

background repeated behind them as they moved left and right.

Scrolling along two axes

Gauntlet and Simcity permit scrolling along both the x and y axis in a two dimensional

plane.
 12

Fig. 1 Pong. From: Atari games (5 Feb 2005)

<[Link]

Fig 2 Pacman. From: Quest [Link] (5 Feb 2005)

<[Link]
Fig. 3 Super Mario Bros. 3 From: [Link] (5 Feb 2005)
<[Link]

______________•
EteflpliorofiitMtastfndotwHelp
Funds: * 20.000 Aug 1372
Rendentiat SI 00 Bool«^|T|[_|[_j[_j[_jjaa«ic

@TlHtlMTlMa **DiaBP[fa»t____ m

^ - E f t l T X * 6:35PM
__tart|i;Am..iec...|g]N...|m*\\m^ i i e f i * i * a ^

Fig. 4 Simcity. From: [Link] (5 Feb 2005)

' <[Link]
 14

Adjacent spaces displayed in one room at a time

Games like Berzerk take place in multiple rooms where the user is able to imagine that

his/her character is invisible to other game features. If the user's character is hidden f

view from the enemy, the user will not be seen, therefore not engaged. The user will

have to maneuver to stay hidden and/or accomplish desired tasks.

Scrolling with multiple background layers

Double Dragon used scrolling to simulate a deep and dense background. The depth effect

was created by the gradation of movement of the various layers of background images.

The background of the first level of Double Dragon contains an image of a city that

imputes a character and feel to the game.

 15

Fig. 5 Berzerk From: [Link] (5 Feb 2005)

<http ://www. digitpress. com/dp sightz/atari5 200/[Link]>

Fig. 6 Double Dragon From: [Link] (5 Feb 2005)

<[Link] >
 16

Limited 3-dimensional space - the use of perspective in the scene

The spaces used in Tempest and Night Driver were made to simulate a 3 dimensional

environment. By using vector lines and/or rules of perspective, a simple perspectival

scene was implied.

Two spaces on one screen or two separate screens

Multiplayer games such as Spy vs. Spy were characterized by having players control their

character while attempting to find others by searching through a labyrinth of rooms. The

location of one's character and environment (as well as those of the enemy) had to be

envisioned before one could become an effective, intuitive player; hence, the game

required players to quickly master mental mapping skills.

Early 3d space - one horizontal plane

Wolfenstein 3D is one of the earliest "3D" first person shooter games. The player is in

the first person view mode and must explore a series of dungeons in an immersive world.

Drawing on these advances gaming technology, the game Doom presented a two-and-a-

half dimensional (2.5d) game. The floor level changed heights and characters could be

engaged from above or below.

 17

Fig. 7 Tempest. From: [Link] (5 Feb 2005)

<[Link]

Fig. 8 Spy vs. Spy. From: [Link] (5 Feb 2005)

<[Link]
 18

Full 3d Space - horizontal/vertical movement

Quake, FarCry and most current first-person shooter games (Doom3, Half-Life2, etc.) use

full, three-dimensional space. In this game space, a player is allowed full freedom of

movement. One can walk along the ground plane or look and shoot anywhere that the

game allows. Players also have the ability to fly or jump in the environment since the

plane of movement is no longer limited to the ground and the scene is rendered on the fly.

Mapped space — pre-rendered panoramas

Instead of real-time, on-the-fly rendering, mapped space games such as Myst present a

slower-paced game with pre-rendered environments. Every animation is pre-recorded

and every viewpoint is pre-calculated. In this mode, the computer does not need to run a

game engine that will calculate its visuals in real-time. It needs only to store the

information that is needed for each particular scene. This way the animation is smoother

and realistic, unfortunately at the expense of repealing fast-paced (inter)action.

Fig.9 Screenshot of D o o m . From: [Link] (5 Feb 2005)
< [Link] >

Fig. 10 Screenshot of Quake. From: [Link] (5 Feb 2005)

<[Link]

Fig. 11 Screenshot of Myst IV. From: [Link] (5

[Link]
 20

2.3 Viewpoints: First-Person, Third-Person, and Overhead Views

First-person viewing means that the view on the monitor is the view that the character in

the game will see. It is the most immersive viewpoint as it permits players to project

themselves into their characters. Most gun-oriented or shooting games utilize this mode

(thus the moniker "first person shooter"). Game developers with an artistic bent are able

to create frightening "horror-film" experiences when they know the positions from which

the character will be looking. Since the first-person view imputes an empathetic and

experiential aspect to a game it can also serve as a useful tool for architectural spatial

explorations. Depending on the circumstance, the digital building model should not only

be used to output still images or pre-rendered animations, but accommodate a viewer's

unrestricted movement within the scene.

Third person viewing implies that the player controls a character, whom he watches on

the screen. In recent games like Uru players must create a character (or avatar) that they

control throughout the game. It can be a fantasy-based character or one that resembles

the person controlling the character. The case of Uru, this character is watched "over the

shoulder" from behind.

 21

Fig. 12 Screenshot of Half-life2. From: [Link] (5 Feb 2005)

< [Link] .html>

Fig. 13 A character of likeness is created in Uru, from the Myst series.

The overhead or "god" m o d e displays game characters from above - whether in plan

(map), plan perspective, or at a slight angle with respect to the ground plane. Simcity,

Civilization and Age of Empire are all examples of overhead games. These games focus

on the large-scale interaction of buildings and cities in a constructed environment, rath

than on individual characters.

2.4 Gaming Technology and Terminology

In this section I will introduce key terms, tools, techniques and technologies used in

computer game design. Topics include polygons, real-time versus pre-rendered

environments, bump and normal mapping, dynamics, in-game physics, game engines,

and architectural implications.

Game Engines

A game engine is the piece of computer software that controls most facets of the game.

Game engine components include dynamics that control the way objects move, and the

manner in which the environment incorporates physical forces like gravity, mass and

friction, and various modes like swimming, driving vehicles, parachuting, etc. Multi-

faceted games incorporate multiple dynamic movements. If a particular game becomes

too multi-faceted, the depth of the simulation will go down and play may be

compromised. Meaningful play is said to stem from players making knowledgeable

decisions with limited options.

 23

Fig. 14 Screenshot of Age of Empire 3. From: [Link] (5 Feb 2005)

<[Link]

tri l \Gga i m

— . ;.•

_P_______. *«*<

Fig. 15 Screenshot of Half Life 2. From: [Link] (5 Feb 2005)

<[Link] .html>
Polygons

Polygons are two-dimensional shapes with three or more sides. They are the building

blocks of scenes and characters. In the gaming industry, polygons normally refer to

triangles because triangular shapes are efficient to calculate and are, by construction,

always planar. When designing games, programmers attempt to optimize the number of

polygons used in a scene (a combination of the environment, objects and characters).

Too many polygons means computation-intensive rendering resulting in low frame rates

and a breakdown in response times. Sluggish response times compromise interactivity

and work against the immersive fallacy.

Texture and image mapping

Texture and image mapping refers to polygons having an image overlaid onto them.

Otherwise polygons would remain single colors, only changed by an altered position to

the light source(s).

Real-time light and shadow mapping

Real-time shadow mapping (as the name implies) is used in predominately in real-time

gaining. It is one thing to render in real time a digitized model; to scale, move and rotat

a player around a dynamically rendered environment. It is quite another to light the scene

with complex real-time lighting algorithms that reflect reality. Real-time shadows

dynamically calculate shadows on surfaces in a realistic fashion, following the rules of

optics. New games present fresh real-time lighting ideas in realistic ways (e.g., light
 25

sources that sway and cast shadows that distort as they move, adding drama or suspense

to a scene.

Normal Mapping and other technologies

Bump mapping, a technique similar to normal mapping, adds a displacement or height

value to a surface map. This bump map is a flat greyscale image in which white

represents high bumps and black depressions. When rendered bump mapped surfaces not

only have color/pattern but appear to have raised surface detail. Bump mapping can

create beautiful details on surfaces when rendered in still image format. However, when

it is used in animations, pre-rendered bump maps imply lighting from one direction only,

making real-time lighting on a moving figure quite unrealistic.

By contrast, normal mapping simulates shadows in all three Cartesian directions. Rather

than a grayscale image mapped to the surface, normal mapping uses a three-colored

palette (one for each direction in a three-dimensional virtual space). The resulting real-

time rendering creates highly realistic, polygon efficient models/objects/environments for

gaming. Normal mapping allows game designers to create highly detailed low-polygon

models or meshes that have the appearance of a high-polygon model.

Similar to bump-mapping, normal-mapping encodes both the height of a pixel and

the direction a pixel faces in a normal bitmap (standard bump maps just encode
the height.) Using this normal map, a 3D engine can add detail to a low-poly
model.

A wire frame shows a large number of polygons to produce a scene. Texturing and lighting

will produce amazing results.

 26

Dynamics and in-game physics

Until quite recently games had limited physics built into their real-time engine, limiting

what a user could do in the environment. "The ability to effect the environment that is so

proudly foreground is essentially limited to being able to inflict damage upon it, rather

then to have truly multiple choice."5 In-game physics, however, are absolutely integral to

current first-person shooter games. In-game physics and dynamics mean that games have

the ability to reproduce real time gravity effects on objects allowing objects to move in

realistic ways. Each object is imputed, for example, properties such as mass, friction and

buoyancy. Moreover players now have the opportunity to use these various objects in the

game environment in order to complete certain objectives of the game. These real-time

physical properties make environments more dynamic and believable. As game theorist

Barry Atkins states, a deformable environment "is part of a basic strategy of'immersion'.

The concept that one might somewhat 'forget' that one is playing because there is no

collapse into illogical lack of cause and effect."6 This means that the objects change and

react as the user interacts with them

In the game Half Life 2, for example, one must climb onto a platform and control a

powerful electromagnet in order to pass a certain level. The objective is to transport the

character's vehicle to another location. When a player experiences the interactive fun

that comes with lifting and dropping the vehicle, the experience on the rig becomes an

exciting portion of the game.

Artificial intelligence

Artificial Intelligence, or AI, is the ability of computer software to mimic, through

programming code, a genre of behavior that may be considered intelligent. Games use

AI technologies to program the user's characters and to give the human players a

challenge. In the past, characters were programmed by simple code which produced

simple and often predictable outcomes. Players could quickly and easily learn the

patterns and overcome the challenges. Today Al-programmed characters receive certain

characteristics such as speed, aim accuracy and physical fortitude as the game progresses.

The Internet's influence on the gaming industry

As theorist Katie Salen notes, "meaningful play emerges from the interaction between

players and the system of the game, as well as from the context in which the game is

played.'" Today, meaningful play can also emerge from a multiplayer community

environment. A game is played when players 'play' or make choices. The choices are

actions usually followed by outcomes. Thus, this meaningful play resides in the

relationships between these players, actions and events (outcomes) and their integration

into the larger context of the game.

The gaming industry has leveraged increased speeds of communication over the Internet

to accommodate physically disparate payers within the same game environment. In the

1970s and up to the early 1990s, primitive, text-based MUD (Multi-User Dungeon)

games came alive only in the player's imagination. Now, by using the Internet to connect

to a common server, multitudes of players can interact in a detail-rich virtual world —the

design of which encourages players to challenge and inspire each other, fostering bonds
and relationships. N o w the story-telling space has become a social space where the

actions of other players can influence the setting. Tellingly, "Game experience may

change during online play" is written on the packaging of game titles such as Half Life 2.

Augmented by artificial intelligence, online games foster a new kind of shared virtual

consciousness.

When virtual spaces become social spaces we have the opportunity to (re)design our

bodies and revisit social conventions. Are these bodies metaphors for cyber-existence?

Can they be considered metaphors at all? Consider what the components or

characteristics of our physical bodies tell others about our culture, nationality, lifestyle,

fashion awareness, job title, age, experience, status, sex, size, blood type, etc. Are these

same characteristics important in the cyber world? If not, why are they important in the

real world? Is cyberspace potentially a more equal ground for human discussion and

human evolution? Is it a place devoid of discrimination or is it just different?

Virtual social spaces suggest a dualism between physical and non-physical realms. This

kind of dualism existed in the Middle Ages supported by religious beliefs in a world

beyond the physical, outside of the material domain. "This non-physical space

metaphorically paralleled the material world, but it was not contained within physical

space." Medieval spiritual space was a separate and unique part of reality yet, to the

extent that God was imminent in corporeal reality, shared characteristics with it.

Similarly, today's virtual social spaces are not entirely separate from our reality. Online

gaming communities, for example, have meta-game aspects — forces that bring the two

worlds together. Gerhard Spiller states that we are "beginning to see that those
organizations once called metaphorically alive are truly alive, but animated by a life of a

larger scope and wider definition. [He] call[s] this greater life 'hyperlife'."9 A kind of

hyperlife is said to exist in the busy organization of a post office, a city like New York,

Simcity, in the Aids virus, as well as in online gaming communities. This hyper-life's

diverse dimensions will be explored in Chapter 5 where associations will be fashioned

from reality, from game spaces and from other places, between and beyond.

Massively Multiplayer Online Role Playing Games (MMORPGs)

As suggested above, Massively Multiplayer Online Role Playing Games (MMORPGs)

are growing in popularity. In the past multiplayer games were restricted to a single

level/map with a limited number of players. First-person shooter games such as Quake

and Half-Life featured small levels with quick action that accommodate games lasting

between 5 and 30 minutes. MMORPGs have recently broken records for the number of

users that were online, playing the game at one time.

As an example we might briefly consider World of Warcraft (WOW) which went online

in November, 2004. In this game format, each player chooses his own character and

develops individual characteristics. According to developer of the game (Blizzard) and

[Link], World of Warcraft is the fastest-growing massively multiplayer

online role playing game and has shattered all previous concurrency records in by

supporting over 200,000 simultaneous players during the holiday period.

Fig. 16 World of Warcraft Screenshot. [Link] (5 Feb 2005)
<[Link]

Fig. 17 World of Warcraft Screenshot. From [Link] (5 Feb 2005)

<[Link]
 31

2.5 Conclusion

With that said we must still question why the gaming industry is so driven toward realism

and why realism and immersion seem to be synonymous. Perhaps this realism is a call

for a hidden architecture. The clever abstraction and symbolic representation of early

computer games has given way to a reality factor - supported by the techniques and

technologies described above. Will users bore of hyper-realism of a game and find it

redundant? Perhaps, in reaction to architecturally rich virtual spaces, the real would

become less real than the game, thus reversing the theoretical positions of the real and

virtual. What impact might this have on architecture or the ways in which architects

define what they do?

With the knowledge of the tools and techniques of gaming we are well positioned to

make connections with the tools used by the Architecture, Engineering and Construction

(AEC) industry. Parallel items can be noted when considering connections between the

goals and aspirations of architecture and the gaming industry.

Chapter 3: Background 2 - Architecture

"Since the dawn of CAD, the AEC community has sought the uber-
intelligent building model - a single database that contains geometric
information about a building's physical form, data on its components
and their behavior, as well as cost, erection schedules and other non-
design information.10

3.1 Introduction

This chapter briefly describes digital tools used by architects - by way of better

understanding how their use of digital technologies differs from that of game developers.

Implicit in this discussion is the belief that architects would benefit from a better

knowledge of the tools and techniques of the gaming industry- to expand both what they

do and the ways they go about it. In particular, I believe that gaming engines have the

potential to significantly enhance the design palette of the architect. To test (on a very

preliminary level) this hypothesis, I produced a three-dimensional model of the School of

Architecture building at Carleton University and used an engine from the popular game

Quake to permit users to move through the space. The model accommodated real-time

movement through a series of rooms containing diverse material and lighting conditions.

This experiment is discussed further in Chapter 5.

To the extent that architects (or their clients) would benefit from the ability to immerse

themselves in the designed space during its conceptual or schematic development, a tool

such as this would be increasingly useful as the technology improves. With

improvements in technology users will have the ability to make virtual environments

increasingly realistic. Should we reach the point where users cannot or do not

differentiate between the virtual and the real, we will have crossed a threshold into a new
 33

way of thinking about our designed world. Until recently visual realism and interactivity

were mutually exclusive in the digital realm - the more polygons the model contained,

the less interactive it was. Realistic visualization required scenes to be pre-rendered,

limiting their ability to support real-time interactivity. Current technologies, however,

accommodate both high levels of detail and interactivity, providing the means to bridge

the experiential disjuncture between the real and virtual worlds.

With this in mind, a parallel yet diverging line of inquiry explores the impact of highly

realistic virtual environments on the definition of architecture itself. When working with

real-time technologies, a potential conflict arises when either the architect or game

developer creates a space whose function and experiential qualities allow it to replace the

real — the digital version being in some way superior to the real. Chapter 4 will further

explore these "crossover" environments while Chapter 5 will present the reader with

some unique spaces and scenarios.

3.2 Digital Tools in the AEC industry

Numerous Computer Aided Design (CAD) programs are in use in the architectural

profession today. The most common is AutoDesk's AutoCAD, used primarily for drafted

(2-dimensional) drawing. While most CAD programs accommodate both 2 and 3-

dimensional drawing, drafting and modeling are discreet undertakings and represent

relatively independent data sets.

In Hybrid Space: New Forms in Digital Architecture, however, Peter Zellner states that

"buildings [will no longer] need the plan/section/elevation convention, but rather [will
 34

be] fully formed in 3d modeling, profiling, prototyping and manufacturing software."11

Zeliner believes that detail-rich models will enable the next generation of designers,

consultants, clients, contractors and fabricators to articulate the design and to

interactively explore the building before it is built. Thomas Mayne, founder of

Morphosis, agrees with Zellner. He contends that "in the future there will no longer be a

need to draw plans; everything will be done in three-dimensional model format."12 This

idea is more radical than it sounds given the efficiency and effectiveness two-dimensional

drawings as abstractions of data. Zellner and Mayne envision that orthographic

projections will be generated, as needed, as views or sections through models.

Presentation, working, shop and construction drawings would be derived from a single,

multi-layered, virtual model that could be shared between users. Given the predilection

for complexity in architecture today (to some degree fueled by the availability of digital

tools), intricate calculations are required for building systems and architectural

components. Digital visualization facilitates communication between the architects,

engineers, fabricators and those constructing the building.

BIM— Building Information Modeling

The use of technology to visualize the process of construction is known as Building

Information Modeling (BIM), which is structured on the premise that a building is created

as a total package in the form of a digital, three-dimensional model. This model

functions beyond the design and construction phase of the building; it is used to control

building systems once the building is finished and facilitates life-cycle maintenance.
A s noted, B I M software aspires to a "uber" level of integration. It can incorporate

simulations and programs that calculate unseen forces that an architect should know

about in order to operate more effectively. Individual computer software packages track

only a fraction of the total information involved in the construction and operation of a

building. The vision of BIM is that the integration of these systems into a single package

will allow architects to move to the next level of design — where technical, code-related,

and long-term maintenance issues can inform the design process from its inception. By

joining individual subsets of information, designers will have access to all interpretations

of the building, many of which normally come from structural, mechanical, and electrical

engineers.

GIS — Geographical Information Systems

GIS, or Geographic Information Systems, is a powerful mapping system that combines a

vast array of data in a map format. The system includes information on sewage, subway,

cabling networks, high-rise buildings, crime patterns between jurisdictions, emergency

routes and other connecting fabrics. GIS systems allow designers to connect proposed

buildings into the larger infrastructural context. They feed into and expand the

effectiveness of the Building Information Model.

Computer Simulations - structural performance, acoustics, environmental factors,

movement and lighting

Engineers use a variety of simulation tools to aid in complex calculations. Advanced

visualization in structural engineering includes Progressive Collapse Analysis (helpful

for calculating the safest entry and exit points of a building) and Buckling Analysis
 36

(which visualizes the maximum safety of structures). Similarly, Piping Systems can help

the architect to understand just how much plumbing is required. Together these tools

facilitate visualization, integration and installation of HVAC systems.

Acoustics is the study of the physical properties of sound ~ whose characteristics can be

measured, quantified and charted. Using sophisticated software, architects and acoustical

engineers can better control sound, considering the impact of a building's volume, shape,

and materiality on acoustic performance. While, in the past, physical models were

needed to test acoustics, "until now, there has been no guarantee that the new space will

have any aural relationship to the reference space."13 Today digital sound 'particles' -

representations of the total energy produced by the sound source - are sent in all

directions. Acoustic simulations show how sound energy flows, helping to identify

echoes and improve the effectiveness of absorption placement. "Auralization is the

ability to render accurately the sound of a space, whether built or not, while immersed in

a controlled listening environment."14 We now have the technology to actually hear an

un-built space, and create amazing sounding spaces.

Water streaming and wind simulations can also benefit the design of virtually any

building. Wind simulation algorithms permit designers to predict and mitigate harsh,

windy conditions in urban areas. Similarly, large bodies of water can be studied to

determine the environmental impact of a building on the water's edge, and various scales

of currents can be studied to predict their impact on shorelines.

 37

It is now also possible to simulate the movement of individuals through space. This

software works by placing objects in space and using visualization techniques to interpret

movement around them. Such simulations can be useful in determining the placement of

emergency exits and in the design of urban public spaces. Vehicular simulations are at

the forefront of highway and transportation hub design.

Lighting simulation is a component of most major 3D modeling programs — whether as

simple shade and shadow or sophisticated ray tracing and radiosity algorithms. To

accurately simulate lighting, proper empirical data must be inputted into the digital

model. Glare on surfaces can be calculated before the project is finished permitting, for

example, materials to be modified to avoid problems.

Among the other forces affecting the design of architecture which can now be simulated

are glare studies in airports, budget organization, material specification, lighting patterns

shadows, radiant heat and hot spots studies, water runoff, thermal drifts on the inside and

outside of a building, etc.

3.3 Game Engines

As noted above, game engines could be a useful tool for the AEC industry - should the

Building Information Model be adopted. Their multi-faceted properties would enable

designers to interact with, examine and present design proposals in innovative ways. In

addition to fly-throughs and eye-level walk-throughs, engines such as Ubisoft/Crytek's

FarCry engine support movement through the scene in various vehicles. The night

mission portion of FarCry requires players to don infrared glasses to detect heat-spots of
enemies. It's not difficult to imagine h o w such a function could be adapted to permit

design professionals to switch from a normal to a "temperature detection view" — to

observe air flows, thermal and material properties (acoustic information, light/radiance),

electrical work, plumbing, HVAC systems, and structural stress levels (variable colors).

Modern games also feature vast multiplayer options/modes, permitting disparate

individuals to inhabit the same virtual space at the same time. Consulting engineers

might, for example, congregate in the digital space with the architect to point out and

discuss areas of interest.

Current game engines operate using parameters that coordinate game operations; these

parameters include gravity, physics, AI, and the behavior of objects within the

environment. If programmed with information relevant to architectural projects, these

digital environments would follow virtual physics and permit forces, information, etc. to

be realistically demonstrated.

Because of the improvements in inputting/importing models from AutoCAD, 3DStudio,

Maya and other modeling programs used by architects, game engines could be deployed

at an early stage in the design process. The designer could inhabit the schematic model

to assess the proportions of spaces, the influence of light quality/quantity, potential

program layouts, etc. As basic decisions were made, detailed information such as

color/material, lighting, etc. could be added and the model could be shared by way of

soliciting feedback from others involved in the design or construction process.

 39

Because of their familiarity with 3D design programs such as Maya and 3dStudioMax -

many of which are used by game developers — I believe architects could easily make the

transition to the use of game environments. In this fashion, architects can be considered

to be level builders.

3.4 Conclusion

Using the scenarios presented above I've attempted to make a case for the use of gaming

tools in the making of architecture. These tools would allow architects to simulate

various forces/decisions and provide them with an interactive way of addressing results,

using causality as a means. The combination of architectural and gaming tools, coupled

with the ability to undertake simulations in interactive, highly realistic environments,

would change not only the way architects work, but their understanding of architecture

itself. Vast multi-player virtual spaces may begin to look and act "real" - becoming

indistinguishable from the real as our concept of the real changes. Roles may also

change.

Architects currently use virtual models as a means to an end, namely to produce a work

of architecture. Even in this capacity the enhanced digital model would generate

additional synergy between the user (the designer) and the work. When the model

transcends its function as mere model, however, it can augment the function of the

building - whether used independently (in place of) or in addition to (in parallel with) the

building itself. At the limit, depending on its ability to support functions in a meaningful

way, the existence of a quality-rich virtual environment may obviate the need for the

building itself, redefining architecture in the process.

 40

The goal of gamers, in this same instance, is also event oriented. Events and interaction

between users make the gaming environment similar to real-world environments. But

rather than literal realization (with all of its advantages and limitations), the game is

focused on a virtual materialization that fosters immersion and accommodates interaction

with the environment, the parameters of the game, and with other users. These

observations lead me to explore the varying degrees of interaction between the real world

and the gaming world. Using a variety of media, I can investigate new spatial conditions

that equate, merge and separate the real and the (gaming) virtual. The following chapter

will focus on the immersive aspects of these tools ~ envisioning things from a user's

perspective, namely from the inside of a model and creating buildings within an

experiential realm.
 41

>l.'>.f!'i'n?[Link]:Tr*:^mi:i>L-!,v,'m'i'iU i n n \ m i ^ i ^ i l — ^ 1 — — ^ — — — — M M
O *t I* Van frowt l o * Haw Cxnxun rum* NXVT t>»«u JL- Kfcfc im/i wtik*. Kb
J*J«J
:
. M S it V S ;} W tv rri if a? ? A -4 p) I»'---"••"• J |~— • »>««» 3. ' >& £S iy — % f • t ft *•
: -c^»•>»Ps
» -• »*s /e >a '*-do r
-C < -, ;»q* « * a a * :i> B % O •.- * »<*

Sdta 1 1 * MwlunOfUi- 3 . \ .

s«»e Hi I T i m i w i ]

SHtf1 & n C laHIMO TOUA \WiWf OlfUO LWT JHODil

n
Fig. 18 Screenshot from Autodesk's AutoCad. Using 3d modeling to examine buildings
as a whole.
 42

Chapter 4: Crossover Environments

4.1 Introduction

This chapter will explore immersion and social interaction within virtual spaces - beyond

their use as gaming environments and/or design-related tools. In order to test some of the

observations I make in this chapter, it was necessary to experiment with various media.

These experiments are described in Chapter 5.

Gaming author Eugene Provenzo observes that "When you play a video game, you enter

into the world of the programmers who made it. You have to do more than identify with a

character on the screen. You must act for it. Identification through action has a special

kind of hold. For many people, what is being pursued in a game is not just a high score,

but an altered state. This altered state depends on the game's ability to alter a player'

perception, to entice them into spaces and places beyond the norm, to make them forget

that they are playing. Paraphrasing Katie Salen, in Rules of Play, interactivity is

something to be experienced, rather than observed.

Level designers do not directly design play but design the context and structure to

accommodate play . A level is a skeleton with obstacles, vantage points, and so on.

While, in the single player games, the level is not exactly arbitrary, play requires the

addition of rules, opposing characters, etc.. In a multiplayer mode, play results primaril

from interaction between players, detaching it even further from the environment. In a

similar fashion, architects do not design experience but rather the surroundings that

accommodate various functions and experiences. Both level builders and architects

design "spaces of possibility" in which meanings can emerge. However "purpose built,"
 43

these spaces might be, they inevitably transcend the functions and uses for which they are

designed.

On this basis, we can distinguish a game from the interactive virtual environment in

which it is staged. Games will always attempt to challenge the player; there will be goals

to reach, opposing forces with which to grapple, and criteria against which to evaluate

actions and outcomes. In support of game-related interaction, today's virtual

environments accommodate multiple players in immersive, detail-rich spaces. In an

appropriately designed environment, however, users can develop their own rules of

interaction, whether in addition to or in place of the rules of the game. I believe

architects should pay attention to such environments. Beyond their usefulness for the

design and life-cycle maintenance of a building (as discussed in Chapter 3), digital spaces

can encroach on and overlap with the "real world."

4.2 Meta-Communication

Interaction and exchange between players that transcends the more literal aspects of the

game may be termed as "meta-communication." Online games, for example, are

fascinating for the deep and meaningful relationships they promote between characters,

players and persons, cultures and communities. On-line games are occasions for open

narrative, self-narrative, self-discovery, and immersion into new kinds of experiences.

The following terms describe some known relationships between the self, the virtual and

the beyond.

Immersion and Interactivity

 44

Immersion and interactivity go hand in hand with game experience. A s Barry Atkins

notes, interaction must exist between computer and human user for a game to be truly

immersive. Cause and effect contribute to the experience; without it, states Atkins, there

is no learning, no knowing:

"The essential characteristic of what is termed interactivity in relation to

the computer game is that is must watch the reader. We act. It reacts.
We act again. It reacts again. It rewards our attention with attention of
its own. This might be presented to us in 'real-time' but we are locked in
a complex dialogue or dance with the machine that amounts to a sequence
of exchange that goes both ways. Even not to act is an act, and signifies.
And in that dialogue of absence and action rests the fundamental claim to
interactivity of the computer game. [...] The text we read watches us over
time, it presents the illusion of 'knowing' us as we come to 'know' it, of
'reading' us as we 'read' it. "16

Immersive Fallacy & Immediacy

In Chapter 2,1 introduced the concept of Immersive Fallacy - analogous to the concept of

"suspension of disbelief' in film. As Salen and Zimmerman state, "The immersive

fallacy is the idea that the pleasure of a media experience lies in its ability to sensuall

transport the participant into an illusory simulated reality,"17

Immediacy is the ability to authentically reproduce the world and create an alternative

reality. While on some level every medium aspires to bring us closer to a more

immediate and authentic experience, for every medium there is a different point where

"immediacy leads to hypermediacy."18 Take the example of using a webcam to

communicate via live video to someone in another part of the world. The context in

which we view the video image (within a web page, etc.) will have an impact on how

immediate or artificial the experience will seem.

 Immersive fallacy would assert that a player has an immersive
relationship with the character, that to play the character is to become the
character. In the immersive fallacy's ideal game, the player would identify
completely with the character, the game's frame would drop away and
the player would lose him or herself totally with the game character. "19

Today's top immersive games shuffle the player's awareness between person, player and

character. Salen and Zimmerman explain: "The phenomenon of meta-communication

implies that game players are aware of the frame of a game and that a player's state of

mind embodies a kind of double-consciousness that both accepts and refutes that

frame. " This play with double-consciousness asserts that the player is immersed within

the character of the game while at the same time being aware that he or she is controlling

a puppet, a tool or object. The player both knows and "forgets" the artificial nature of the

game, being both player and character simultaneously.

Imagine a game that encourages players to constantly switch frames, questioning the

inside and the outside, perhaps simultaneously. The immersive fallacy suggests that

one's pleasure with a medium relates to the degree to which the medium transports the

player into an illusory environment. Historically, these illusory worlds were created as

paintings, photographs, moving pictures, color moving pictures (with sound), video

(delivered via television), and most recently as immersive games like Quake. Is this

progression inevitable? Will it continue, or are we approaching end of the line with

respect to realism? Although the drive for increased realism seems to characterize the

gaming industry, it doesn't necessarily take into consideration the meta-communicative

nature of play. Indeed, immersion is not only a question of the "reality effect," nor is

there a simple relationship between player/character, player/game, game/outside world.

 46

The virtual as a threshold into the real

The next level in gaming (and/or interactive digital environments) may be to appropriate

and incorporate the real - the presence of which would add yet another layer to the

person/player/character relationship. A player could exit the digital realm into the real

through a device such as a webcam. A character within the game environment could

encounter a familiar image (whether on a screen, in a frame or in some other way mapped

to a surface). As the character approaches, it becomes evident that what he/she is seeing

is a real-time web feed of him/herself qua player playing the game. As a result, the

character's vision completes a reversal in the loop by entering the physical person.

Similarly, the player inhabits a virtual character to observe him/herself. This genre of

inversion of the person/player/character relationship has not yet been explored in the

gaming industry. As a research and theoretical proposition, however, this spilt could be

quite interesting ~ as it transcends familiar modes of game play and introduces a new

viewpoint.

An example of the meta-game is in the title Far Cry. Here the clock on the wall shows

the actual time. Realizing that game time can exist in the exact time as your own can be

quite uncanny.
 47

Disembodied consciousness/Virtual bodies

Human consciousness has always been characterized by dualism and dualities. The

duality between body and mind is key into understanding immersion within digital

environments, as is our ability to experience things empirically while simultaneously

reflecting on that experience. We are always concurrently subjects and objects of our

own consciousness.

As players we project ourselves into the game. Through thought, dreaming and all means

of artistic creation we, as humans, attempt to transcend our physical existence and to

project ourselves into a virtual or conceptual realm. Moreover we are always trying to

resolve the physical and conceptual realms, to move transcendentally toward the

conceptual without abandoning our empirical grounding in reality. Sophisticated digital

environments permit us to transport our disembodied consciousness into a virtual realm

where we can regain a body. Individuals become players, players become characters and,

through characters, we inhabit new bodies. While only consciousness is required to

immerse ourselves in the game, the body is recreated to permit the consciousness to act

and to interact. The person/ player/ character relationship transcends its already-complex

definition and aspires to additional dimensions of location and being. The virtual body is

capable of being in two or more places simultaneously, of seeing more than one thing at

time.

Sentient environments

Supported by in-game physics and artificial intelligence, sophisticated game

environments can be said to have a consciousness of their own. They are "aware" of and
respond to the position/actions of the players/characters. Cues to this consciousness m a y

come from the surroundings as when the sun location and time of day adjust to the

location and time of day at which the game is being played. Game environments may be

darker at night (e.g., Uru) forcing us either to reset the date, time and location parameters

of the operating system or to wait until daylight to accomplish a task. Another example

can emerge from a currency within a game and its value throughout the virtual terrain.

This occurs because a meta-game occurs within the game, interacting with the players,

their characters and the environments that they come into contact with. In SimCity the

environment has an explicit consciousness of its own. Here, invisible forces, values,

cultures, and languages envelop the inhabitants, whether or not they are aware of it.

While it is more difficult for an architect to design the same level of consciousness into a

building, lighting and HVAC systems can be designed to recognize users, adjust to time

of day, etc. Moreover, the BIM paradigm suggests that there is a parallel, digital version

of the building that is used as a means by which to interact with building systems.

Parallel and hybrid worlds

When considering possible manifestations of layering, combinations, and overlaps

between the real and the virtual, it is helpful to remind ourselves that redundancy has

always existed. This section identifies and classifies examples of such overlaps,

involving different levels of technology. Classifications range from "non-parallel and

non-interactive" to "parallel and interactive." The goal is to better position the crossover

environment (e.g., the MMORPG) within a history of artistic and technological

endeavors. Neither the categories nor the examples are intended to be definitive.

Category 1: Non-parallel; non-interactive

A painting on the wall

A painting on a wall is an example of a non-parallel, non-interactive environment.

Within the painting an image is presented to the viewer through the virtual viewpoint of

the painting's frame. More often than not, the scene depicted is different than the setting

in which the painting is displayed. Interactivity is limited to the act of looking,

interpreting and/or reflecting on the image. Conceptually projecting oneself into the

scene does not normally have an impact on the painting, although the scene is staged for

the viewer's benefit and thus anticipates the act of viewing.

An LCD surface

Images (moving or still) displayed on a LCD or computer monitor are not necessarily

interactive or linked with the real world. Like the painting on a wall, the environment

from which we look is normally different from the environment at or into which we look.

Not until the digital (LCD) and the real (captured by a video camera) are linked, do the

environments become parallel.

A television or book in crowded room

Television programming depicts events and stages stories that captivate the attention of

the viewer, drawing them into the narrative. Similarly, books bring non-parallel worlds to

life in the reader's imagination. Stories and events may run concurrently with events

happening in the reader/viewer's environment, dividing their attention between real and

the virtual. To the extent that these narratives are self-contained and are not changed by

being watch or read, we can classify them as non-interactive.

Category 2: Parallel and non-interactive

 50

A mirror on a wall

Mirrors present us with parallel and reversed worlds. While we can use the mirror to see

into our own world we can not alter the reflected version except by changing the real one.

Using two or more mirrors, one can create a kind of infinite regress, overlapping reality

and reflections with reflections.

Category 3: Non-parallel; interactive

A phone call in crowded room

Telephony virtually links its participants together, facilitating interaction through voice

The (virtual realm) in which communication occurs is, by and large, independent of the

spaces that the individual parties occupy. Via telephony, parties can be two 'places' at

one time.

Massively-multiplayer environments

MMORPG environments accommodate interaction both with the environment and with

other players. By and large, however, they are non-parallel because the virtual

environments in which players meet are independent from the physical environments that

individual players occupy.

Category 4: Parallel; interactive

Parallel environments accommodate more than just simple communication between two

parties. They overlay and intersect with the physical space that each of the participants
occupy. B y appropriating reality (via video feeds, etc.), they blur the distinction between

real and virtual and challenge concepts of time and space.

Video conferencing

Video conferencing allows multiple users to communicate through voice and image.

Signals from each user overlap and create a hybrid 'physical' setting. Even though users

occupy separate locations they are able to come together and share their virtual identity

and presence with one another.

Doug Back's gallery installation21

Doug Back's gallery piece, Small Artist Pushing Technology, is comprised of a rotating

camera atop a monitor on a pedestal in the middle of an empty gallery. The video feed is

played back on the monitor in such a way as to give the appearance of a real-time feed.

Viewers position themselves in front of the monitor and wait for the camera to rotate to

their location in order to see themselves on the screen. The video feed, however,

randomly changes from real-time to delayed feed. Thus periodically the viewer is

presented with an empty room (i.e., the absence of self) when the camera is pointing at

them. The feeling can be quite unsettling. One must then think twice the relationship

between the real and the virtual environment (the one of the screen) and one's assumption

about the veracity of video images.

Supercity and Karlskrona2

Superflex is a c o m p a n y that carries out experiments within the design and architecture

fields. Supercity, which was designed by Superflex, is a virtual city designed as a

research experiment and used over the Internet as a local network. The primary project,

Karlskrona2, was a virtual city that was created for the real city's citizens. The project's

website describes it as follows:

The [KARLSKRONA]2 project is to create a digital copy of the city karlskrona in

Sweden, to be called Karlskrona2. this copy will be accessible to the citizens of the city
via the Internet, where digital representations of themselves (avatars) and the streets and
buildings of the city centre will create a virtual version of the structure and personal
relationships in Karlskrona (the real city).

Karlskrona2 will initially be an exact replica of the city but as the virtual citizens meet
and interact things will change, buildings will redefine their function, social hierarchies
will alter, laws will be reconstituted and renewed. The virtual Karlskrona will be visible
to the real city through a large-scale video projection in the main square. Here citizens
can gather in real space to follow the activities of their avatars and consider the
divergences between Karlskrona and Karlskrona2

The project is designed as a research experiment, using the Internet as a local network
rather than a global communication tool. Karlskrona2 is a "free space", in the sense of
not having to obey the legal, economic or social rules of Karlskrona. At the same time, it
is inhabited by at least part of the same community of individuals as the real city,
bringing the challenge of virtual reality into the lives of neighbours andfriends.

To what extent will be a fulfillment of individual or collective fantasies? To what extent

will it conform to the pattern of Karlskrona? What new possibilities does the Internet's
"free space" offer to an existing community?

Endeavors like Karlskrona2 present opportunities to study many social phenomena.

Real-world class hierarchies have been broken down in this virtual landscape, giving rise

to new virtual classes and positions. In its virtual form, alterative societies were

permitted to form both in relation to and independent of the "real" city.

Karlskrona is an excellent example of an overlapping parallel environment. Although a

screen was set up in the main square to display the virtual version of the city, however,

no real-time video connections were made between the real and the virtual. This one

could see the virtual while in the real but one could not use the virtual as a means of

(re)accessing the real.

Fig. 19 Karlskrona2 From: <[Link] (5 Feb 2005)
<[Link]

Fig. 20 Karlskrona2 From: <[Link] (5 Feb 2005)

<[Link]
4.3 Conclusion

The intention of this chapter has been to introduce categories, examples and terminology

to describe crossover environments. These form the basis of the experimentation I

describe in the following chapter. By way of setting the context, it is important to

understand the numerous ways in which we project ourselves into virtual spaces and how

we routinely occupy both virtual and real realms simultaneously. We also need to

consider the ways and degrees to which crossover spaces promote immersion and overlap

with the physical spaces we occupy. The Meta-game, immersive fallacy and immediacy

each describe the influence of gaming environments on people. From a

phenomenological standpoint, these immersive worlds are as real as the physical. After

all, communication and interaction take many forms and employ many media. Perhaps

the sentient nature of a virtual space can reflect, shape and augment a community's

consciousness - virtual versions can promote a stronger engagement with and

appreciation of their physical counterparts. Perhaps, too, architecture can overcome its

adversarial relationship to virtual space by using and appropriating it to enhance and

extend the meaning of the built environment. Indeed, architects should have a stake in

resolving the virtual with the real.

Chapter 5: Experimentation

5.1 Introduction

In this chapter I describe the various media and technologies with which I have

experimented — by way of exploring possibilities of overlap between the real and the

virtual. As I am interested in embedding the real into the virtual (and vice versa), I

looked particularly at photography, QTVR panoramas, and real-time QTVR-type video

feeds. I hope to demonstrate how the use of these media can strengthen connections

between the worlds of architecture and the gaming industry. While each medium can be

used effectively on its own, using media together allows them to reinforce each other.

These experiments focus on immersion and on the displacement of the subjective self.

Photography is the de facto technique for capturing images of the real world.

Photographic media play an important role in architectural projects, as, for example,

when demonstrating how proposed buildings fit into the surroundings. Photography is

also used for site examination, documentation, and presentations. Panoramic

photography captures images in a wide format - extending the frame in the horizontal

direction. While photography an essentially two-dimensional medium, panoramic

images can be digitally adjusted or stitched together to form a virtual space around the

viewer. Apple's Quicktime player has been the forerunner of this technology and works

both with photographs and digital models. Saving a digital model in QTVR (Quicktime

Virtual Reality) flattens out a 360 view into a long image, then overlays the image to

form an interactive virtual box when viewed within a QTVR player. Using QTVR
 57

technology, environments are taken from 3D capture to 2D map to 3D virtual box which

is viewed interactively on a 2D screen. Interactivity, too, is an additional dimension.

 58

Fig. 21 Full-Circle 360° Camera Module From: [Link] (5 Feb 2005)

<[Link]

Fig. 22 Partial Panorama created by merging many photographs together.

 59

Real-time video feeds can also be channeled though a QTVR-like interface. Using a

spherical lens, the video camera captures a 360 degree view of a scene. The distorted

image is translated by either hardware or software into a scrollable, cylindrical image that

permits us to interact with the real time video output. This live feed does not have to be

pre-animated or pre-loaded since it captures what is already there. Imagine a live telecast

from another country viewed in this form; because the viewpoint is not constrained, the

viewer is free to explore the scene in all directions (the viewer is stationary but the view

is not). Moreover, combining live (real-time) video feed with pre-rendered, hyper-

realistic elements in a virtual environment allows us to blur the real and the virtual. The

layering of the real, corporeal world with the virtual world represents tremendous

possibilities for rich social spaces.

We have already noted that pre-rendered QTVR panoramas are used in games such as

Myst IV. The game immersion is smooth and the interactivity with surrounding objects

and characters is seamless. The surrounding space is pre-rendered, pre-animated and it

appears as if you're surrounded by a movie, engulfed in the action. In this particular

game, the object of the game is closely scrutinizing and manipulation the environment to

discover the object of the game. QTVRs, as noted, would not be as effective in a first

person shooter game.

 60

5.2 Theory and Terminology

Information and Noise

As a discipline Information Theory studies the movement of information in a system,

how it is transferred and received. Information can be said to be divorced from meaning

as it is a measure of uncertainty in a situation. For example, if a situation is completely

predictable, there is no information present. "It is a concept bound only to the

quantification of stimuli or signals in a situation."23 In this respect, information will

always have an inherent noise. In many devices this noise is seen as a negative element.

In the game domain, noise is not necessarily negative. "Sometimes entire games are

premised on the notion of putting noise into a channel of communication."24 Simple

games like charades will have an innate noise, which is part of the game. This noise

muffles the information that is being presented and, in turn, challenge the players.

Redundancy

Where noise affects the successful transmission of a signal, there exists a counterbalance

called redundancy. Redundancy refers to the idea in information theory that information

can be successfully transmitted even if some of it is lost. This occurs because redundant

patterns compensate for gaps within the information. In the communication systems such

as spoken language, more information than we need is transmitted. According to Warren

Weaver, an information theorist, the redundancy of the English language is about 50

percent. He states that "half of the letters or words we choose in writing or speaking are

under our free choice, and about half (although we are not ordinarily aware of it) are
 61

really controlled by the statistical structure of the language."25 We could therefore

eliminate half of the words we use and still transmit the necessary information, in the

English language. Noise and redundancy are both integral to the way games transmit

information. Redundancy is particularly important in games filled with "modular,

interlocking informational elements;"26 the game becomes more flexible, allowing many

of its parts to be filled in. The system of redundancy found in language also applies to

media such as the Internet (a somewhat text-based interface), through which it also makes

its way into computer games.

Like games, works of architecture include noise and redundancy. This is explicit during

the design stages when "modular, interlocking" elements are pieced together in a variety

of permutations. Even in the final design, however, building codes require redundancy of

functional elements like doors, exit stairs, elevators and windows. Similarly, noise can

create interesting spatial conditions, such as accidental, unintentional or uncalculated

spaces. To some extent noise describes the degree to which the whole is greater than the

sum of its parts.

Convergence

Games are becoming increasingly comprehensive, featuring real-to-life environments in

interactive, virtual three-dimensional space. In parallel with computer performance,

gaming developers are pushing software to new limits. As performance-related

impediments are overcome, software can emerge as a powerful tool for the architecture

profession, in many facets of the discipline.

 62

Convergence, as I envision it, involves the overlap and interpenetration of real and virtual

spaces and being; in the context of this thesis, this means that integration of built and

digital versions of the same architectural space. Using a webcam to pull the real into the

virtual version of a space, permits the viewer to engage the real space on which the game

space is modeled ~ as in my experiments with the School of Architecture building. If

the player logs into the virtual space from within the real space, video feedback can

permit the player's character to confront the player as character. Through the eyes of the

character in the virtual space, the player sees him/herself playing.

Contemporary games have not yet incorporated visual input from the real via devices

such as webcams, although, as noted in Chapter 2, computer games frequently leverage

the date/time and location settings of the user's operating system to synchronize the game

with the user's environment. Games such as Myst and Uru also incorporate pre-recorded

video footage into the digital environment, permitting players to engage with "real"

characters as they navigate around. Sound, too, greatly enhances game environments,

promoting deeper immersion and facilitating interaction between users. Today, five and

seven-speaker surround systems make spatial recognition even more realistic and

environments more immersive. In the game Half-life 2, the person/player/character

relationship is fluid. This game includes the ability to view a computer screen as an

object within the game. Made to look like a television signal in real-time, the screen

displays what the camera in another part of the level is seeing, often a space visited

before. Often these areas will be populated with enemies, giving the player something

else to stress about. This technique projects the character's vision into another space,

making it difficult at times to determine where the character is located. In most cases,
 63

one must recall and recreate the recently visited space in one's mind in order to recognize

the position and direction of the enemy soldiers. In a (human user) multiplayer mode,

being the voyeur for too long may have fatal implications, virtually speaking of course.

5.3 Experiments, Scenarios and Case Studies

In this section I will present several scenarios suggesting how gaming technologies and

architecture might converge, merging the tools of visualization, experience and

construction. Virtual environments resulting from such a convergence could be

redundant, complimentary to, or be substitutions for built spaces. By and large the social

spaces of MMORPGs accommodate individuals who would otherwise never be able (to

afford) to meet. As both occasions and venues, MMORPGs do not present a direct threat

to architecture. Beyond MMORPGs, however, I also envision redundant spaces in which

the physical and virtual more directly overlap. The phenomenon of overlap also refers to

the over-lapping and layering of being, body and consciousness - scattered between the

real and the virtual.

Quake III Arena - School of Architecture real-time model

To explore the convergence of information, noise and redundancy I constructed a digital

model of the School of Architecture building at Carleton University using the Quake III

Arena level builder. I then placed an interface/computer/window to provide access to this

model with a room (the Hub) inside the School. This allowed "players" to enter the

virtual version of the School from within the School itself. I also placed a webcam at the

computer to allow users to see themselves on a screen in the game, which ultimately,
 64

created an infinite regress. The computer hosting the interactive model was also setup as

a public Internet server to permit people outside the School to log in and explore the

virtual model. This allowed students and staff (physically in the School) to interact with

alumni (outside the School) as they explored the School together, using their virtual

bodies.

The Quake experiment permitted users to experience the meta-game in a multi-layered

environment. As noted, while in the virtual world, players have the opportunity to see a

screen in the game that has a live stream of the real world via a webcam situated in the

Hub. The user sits at a computer in the Hub, looking at a virtual self looking back at

him/herself sitting at a computer in the Hub. The relationship between player/character,

player/game, game/outside world continually oscillates. Embedding a real-time video

stream into the virtual 3D model blurs the context in which the player views him/herself.
 65

Fig. 23 Combining Quake with windows into the real via webcam.

Fig. 24 Real looking into a virtual screen, seeing a virtual representation of a real person,
captured in real space, projecting that into the virtual.
 66

The QTVR medium

QTVR is used extensively on tourism and automobile manufacturers' websites to show

products and places in a more meaningful ways. Historians and architects are also

beginning to use this technology to describe their work - both built and digitally

modeled/reconstructed. Embedding real-time video into a digital environment using

QTVR, however, results in a conceptually confusing convergence of the real and the

virtual, rife with possibilities!

I quickly explored convergence between a 3D model and panoramic photography in the

central "street" of the Architecture Building. Various inputs and outputs can be placed in

such virtual spaces; slippages occur and redundancies abound. The images below

illustrate the potentials layering and juxtaposition of various elements. Perhaps, as in

Doug Back installation, time could be become the ambiguous frame. At times these

worlds could be overlaid and become synonymous with one other, only experienced with

reference to the game reality, the virtual game reality.

The combined images could be a 3D model or game space, with the other being

potentially a real time feed from cameras in the real space. Now imagine playing the

game where at various points in the level one would encounter a window though which

one saw a real-time, real-world feed. Like Karlskrona2, the line between the real and the

virtual would be transgressed. What were once distinct realms would unite, and

overlapping elements would reinforce the redundant aspects in both.

 67

Embedding a real QTVR image within a digital image is one form of overlap. The digital

presents real-time movements of characters with the real as a still image. Only the area

in which characters cross the QTVR does the real give way to the digital. A person in the

real world could walk around the environment and with the use of spatial locators, while

their virtual self repeats those movements in the virtual world. The virtual level or map

would determine where the user could or could not move, as would the real world's

physical boundaries. The QTVR of the real would relay this information so that no

irregular or physically-impossible events occurred.

In this scenario the digital presents real-time movements of characters, like in a game,

while the virtual camera captures the surrounding area. The real is a live feed animated

image. This kind of mode presents real-time feeds to both worlds. A camera would

capture the position of the real players and project them into the digital game world. In

the absence of support, the real players would not be able to interact with invisible game

players ~ the real players would only appear as walking ghosts in the virtual game world.

In order to see the layering of these opposites, one would need a portal or virtual looking

glass. As with Karlskrona2, the virtual city would be visible to real onlookers though a

digital interface. A physical screen in the real world would give individuals a glimpse of

the virtual world. Here, the transition of real to virtual is instantaneous, onlookers and

the looked-upon will have a plane of transfer, which they are unable to completely cross.
 68

Fig. 25 Real photography and stitching

Fig. 26 Digital panorama

—
^J

Fig. 27 Combining real photography and digital panorama

TDR — virtual transformation

This study led me to an exploration of the space of the Technical Data Room (TDR), also

in the School of Architecture at Carleton. In this scenario, I first photographed the spa

then created a virtual model of the space using a game engine.

Technical Data Room - School of Architecture - Panorama photography - many photos

stitched together to form horizontal "panorama". Similarly to Fig. 25.

Technical Data Room - School of Architecture - Digital panorama

The above images demonstrate a layering and overlapping of images to form a sense of

cohesion between the photographs and the model. Wherever applicable, the user or

programmer can manipulate the scene to determine where "tears" or changes of layer

should occur. Real or virtual, what is important will determine how the media should be

manipulated.
Fig. 28 School of Architecture & digital model played within the Quake game

Fig. 29 School of Architecture model as seen through the character within Quake.
 71

While the original model of the School created in Quake (see above) was a massing study

of a more extensive space, I built a separate model of the TDR using Ubisoft's FarCry

editor in order to explore the space in greater detail. The objective of the second TDR

exercise was to become familiar with the software and to test the use of a game engine to

explore blurring, embedding, subjectivity, and distinctions between the real and the

virtual. In this scenario, game players enter the game mode in real-time, allowing for

instant reaction and change based on cause and effect.

Real, Virtual and the Intermediate

Now, with the above as a preamble on multi-viewing environments, we can further

explore new kinds of spatial conditions involving the real, the virtual, and spaces in

between. One of these conditions involves the user being within the Game space,

watching others watch him or her in a multiplayer environment. This kind of interaction

occurs when a player projects him/herself into a digital character - who may be designed

to resemble the player to strengthen the bond between player and character. In the first-

person mode the character would come into contact with other (human) users. An

immersive field captures the characters without consideration for person context

interaction, i.e., purely character to character. In the third-person mode, players can see

their character and thus, project their individual characteristics into the character.

Another level of imagination and visualization occurs when one can see another body

mimicking what the mind thinks, rather than control through the body.
 72

As discussed with respect to HalfLife2, one can watch a digital monitor within a game

space and see the real-time positions of others in a different room or places. Such views

or portals between virtual spaces add yet another dimension to the already complex

laying of real and virtual. One can also be within a game space, watching a digital

monitor, seeing oneself, watching the monitor, digitally or virtually. Such uncanny

experiences prompt one to rethink the relationship between position, location and

identity. Being neither singular nor unique, the concept of location becomes devalued,

spawning parallel identities and subjectivities. But we have to question the parallel-ness

of these split subjects. Are they in any way equal with respect to the character and the

player? Or do they have a split-character. For a few seconds, while watching the

monitor, you become aware of another cohesive layer of existence. One can think about

it, by stretching the person/player/character relationship into a void where character

extends itself by becoming a player and controlling another character or a pseudo

character. Imagine a pair of virtual hands controlling what you are controlling, even

though those hands are controlling what appears above them - a computer game. Who is

controlling this game; you or the hands that you're controlling?

A user immersed in a game space watches a digital monitor and sees him/herself

watching the monitor through a webcam. This experience may be even more bizarre

since, rather than involving your virtual figure, the person/player/character relationship

reverses and loops back to the player. Again, the person is playing a game and the player

becomes the character. The character moves about the virtual space, and discovers a

virtual monitor or screen where, with the touch of a button, the view zooms in to reveal a

real-time video feed of a webcam showing the player in real space. The players character
 73

confronts the player as a character, playing. Who's in control? Who is doing the

looking?
 74

Fig. 30 Merging the real within a virtual context.

 75

Fig. 31 Real and Virtual. Interchangeable?

 76

Another scenario involves watching a digital monitor within a game space on which you

see yourself watching looking into a monitor on which sits a webcam (obviously) that can

be controlled by the character in the game. This transcends beyond the physical-less

limit of the previous mode where one would only look through a webcam. In this

capacity the character, controlled by the player, has the ability to move objects within the

real, most importantly the webcam. The character will be able to control what it sees, to

interact with the real. This might be considered to be an overly complex or redundant

interface: real world into virtual, through game (character) controlling the real. Why not

use a simple real-time monitor to control a robot-like device in real-time? Well, it

depends on the goal of the interaction. Perhaps the potential of one's imagination is

better tapped in an imaginative world like cyberspace or game-space. Perhaps one would

make different choices as a character (within a virtual environment) than one might make

in the real world ~ free of the effects of gravity, oxygen, friction, mass, etc.

Another scenario involves a game in which a digital canvas displays a live feed image of,

for example, the School of Architecture (e.g., the TDR or Pit). Rather than looking at an

in-game monitor ~ a virtual representation of a monitor - the viewer here is presented

with a screen. In both its position and what it displays, the screen presents the viewer

with the real image of what they would expect to see within the virtual world. The screen

introduces us to a different kind of view of the space - one that is fragmented and/or

artistically composed. Multiple cameras focused on a particular object or event, for

instance, could present a cubist-like version of the space. Reversing the scenario where

multiple characters view a single object, this scenario permits the player qua viewer to

see the same scene from multiple viewpoints simultaneously.

Surrounding projection

Along the same lines, imagine that a live-feed image of yourself is projected onto this

digital canvas within the game environment. The uncanny feeling that one might

experience when seeing oneself on a computer screen inside a game is amplified by the

scale of screen within the digital space - surrounding and immersing the character in an

overwhelming way. The screen displays the person as the same scale as the character,

permitting the character to develop a more intense relationship with the image. In this

form, subtleties such as facial features can be leveraged and experienced by others in the

virtual space. The focus is on the immersive aspect.

Similarly, a large-scale digital canvas could be placed in (or projected onto a surface

within) a real space and produce different effects. This screen could be located, for

instance, in the central street of the Carleton Architecture Building and images could be

projected to correspond to the viewpoint of the viewer. When looking into this digital

window, then, one might see digital characters located within the digital level/map.

These might appear as custom characters, representing alumni, professors and virtual

visitors to the architecture school space. Given the scale and position of the screen and

the fact that the viewpoints of the real and virtual are calibrated to correspond, these

virtual beings would appear to be present in real space and time. When the image of the

real, in turn, is projected onto a similar screen in the virtual world (as if from the opposit

side) this screen becomes a life-size threshold between the real and virtual versions of the

same world. Characters on both sides of the screen have an idea of who (and how many)

are watching them. This fascinating intersection of parallel realities suggests a number of
theoretical and technical ways to use the screen as an interactive medium. Touch screens,

for example, could be used to support such explorations given that they accommodate

drawing, painting, pointing, and expressing feelings.

Imagine being in real space and observing a 360 canvas on which is projected a QTVR

image. This is yet another opportunity for exploration. This event involves a physically

overpowering structure - a version of the old cycloramas. A virtual structure like this,

created in the TDR of the Architecture building, explores how one might interact with

such a screen. Surrounding the person is the digital model, including real-time characters

imbedded into the curvature of the screen. Rather than looking into a screen, which is

showing a selected portion of virtual space, this curved screen displays a near 360 degree

real-time moving image (video feed) of another location and time. The video will not lie,

but rather display what is there and surrounding the viewer with another place in the

world.

As previously noted, the impression of a three-dimensional space can be created on a

two-dimensional surface using a two-dimensional receptor (camera) and a two

dimensional projector (e.g., a monitor). What would happen if the receptor and the

projector worked in a three dimensional system? Rather than capturing an image this

projector would be generating one, using the same idea as the spherical lens except that

here the input and output would be inverted. A spherical input and output device would

create a sort of surrounding real-time sphere of infinity. There would be no proper

perspective in the traditional notion. The meeting of lines, parallel lines, and various

layered images and spaces would be quite surreal. Depending on the viewer's position
and on the position of the capturing device, the view could change and create something

unexplainable.

Another interesting situation presents itself when someone in one space (e.g., the

Architecture building) sees a real-time feed from another place in the world (e.g., a studio

in Spain) projected onto a screen. Within the same space as the screen, a camera shows

the people in Spain an image (of their image in the space?) of the Architecture building.

In this fashion an infinite space can be presented to both parties if they both have the

same equipment. Essentially, this equipment consists of a spherical projection screen, a

360 degree-lens camera to capture the broadcasting people, and an opposite spherical lens

to project the incoming images onto the spherical screen.

People inside the sphere will see themselves on the canvas in the other location. This is

because the image that is broadcasted will be seen again, repeating itself. This

phenomenon is similar to that of feedback that musicians may encounter during concerts.

When musicians place their microphones too close to the speakers, a loop of sound will

eventually produce a screeching noise. The users of these surrounding screens will

experience something similar. Users will not be able to overcome this infinite regress

unless they make the canvas that they are watching invisible so that the viewers on the

other side cannot see themselves on the screen. The screen must be eliminated or made

virtual. A way to make the canvas invisible would be to create the same kind of canvas

in VR. The redundancy that is produced is a direct feedback. Similarly, when an

individual points a camera into the monitor where it is sending signals, the image

overlaps and the image of an infinite space appears in the monitor. When examining this
 80

phenomenon closely there may be a slight, almost unnoticeable lag within the overlaying

"windows". This is due to the speed of transmission and reception and equipment.

Surrounding virtual projection using VR goggles.

In a variation on the preceding scenario, users could wear VR goggles rather than seeing

images projected on to a 360 degree screen. Participants would wear translucent or

opaque VR goggles or a headset that would virtually surround them with a sphere of the

real-time video feed of the distant space. In this capacity, the distant viewers can actually

see the opposite space via the spherical lens real-time camera without interruption from

an opaque sphere. Both parties could project themselves into the adjacent world through

this virtual medium. Their presence would transcend the physical, being transmitted into

the other realm in a digital format.

The situation suggests the potential incorporation of movement. The VR goggle wearer

might be given the opportunity to move the camera viewpoint along, for instance, a track.

Participants in the distant place may follow the moving camera, as if being led.

Similarly, a distant virtual user may walk with a crowd of people in another location,

experiencing the space in real time as if he were actually moving though it.
 81

Fig. 32 A real 360 surround screen, projecting one's presence into another.

Fig. 33 A virtual 360 surround screen.

5.4 Conclusion

In this chapter I've attempted to describe a realm of possibilities for extending the real

into the virtual (and vice versa) through the use of real-time video feeds. The details

have been left to the imagination of the reader, for it is only though experience that the

implications of these scenarios can be truly appreciated. Given architecture's stake in

the real, this technology has the potential to challenge and expand our understanding of

architectural space.
Chapter 6: Conclusion

There are many innovations in the gaming industry of which the architecture community

may not be aware. Beyond the opportunities that new tools and techniques might

represent (e.g., in the presentation of projects to clients), these innovations continue to

redefine the boundary between and definitions of the real and the virtual. This thesis

envisions a fluid and complementary relationship between real environments and their

virtual counterparts. One way to explore/exploit such redundancies is to embed real-time

video feeds into highly realistic, interactive, multi-user, virtual versions of the same

environment, permitting subjects (players) to move fluidly between parallel and

interconnected realms. Such environments would have profound implications on our

understanding and experience of subjectivity.

Chapter 2 explored key aspects of the gaming industry. Various game spaces were

described to present a chronological (re)definition of immersion in games. Among the

newer technologies identified were real-time lighting, normal mapping, physics, Artificial

Intelligence, immersion, interactivity and internet online gaming. Chapter 2 also

explored the logic behind games, explaining what makes them attractive to players,

describing the various viewpoints that one can assume within an immersive environment,

and discussing the many similarities between the game level and the digital architectural

model. While architectural models are used primarily to generate perspectival views, we

can imagine major overlaps between the architectural digital model and the game - once

the model is inserted into the game environment and the viewer becomes a kind of a

player. Once immersed in the scene, information becomes experience; by contrast

traditional forms of architectural representation privilege information over experience.

 84

The distinction between information and experience is one of the major differences that

this thesis hopes to help to overcome.

Furthermore, in the case of Massively-Multiplayer Online Role Playing Games

(MMORPGs), gaming events include social interaction. Multiplayer environments now

have the potential to incorporate webcams and voice over IP (sound) to present a more

multidimensional immersion than is available though telephones and chat rooms today.

Instead of facilitating interaction within a virtual cyber "nowhere," game environments

support interaction in an experiential "somewhere" — a veritable place. This place is

created by a virtual architecture in concert with characters that inhabit it. While the

aspect of place is intricate by definition, complex to describe, and best experienced in

person. Online multiplayer environments come in only a few "flavors," the most popular

being the detailed digital three-dimensional world in which character interaction is

shaped through text and the movements (dances, gestures, attacks) of the characters. In

the last chapter of the thesis I explored alternative spatial conditions that could be

supported within a gaming environment; perhaps even incorporated into architectural

visualization.

Many of the media explored, especially photography, are visual. To perceive space,

users require tools that support the use of their senses. As noted, Immanuel Kant's

writings address the idea that the only way that humans perceive the natural world and its

phenomena is empirically, through the senses. The limiting aspects of our bodily senses,

cannot, however apprehend an unlimited world of matter and knowledge.

 85

"What is delivered to our consciousness is the product of our bodily

apparatus, and takes the forms it does because of the nature of that
apparatus: there cannot be visual representation without eyes, or
aural representation without ears; there cannot be thoughts or ideas
without brains. But sights, sounds, ideas and so on, are not the
objects external to us, they are representations of these objects.
They exist only in the human apparatus that produces them. "27

With this in mind, I investigated tools that might expand the ways architects work as well

as the spaces on and with which they work. Some of these tools help them engage

invisible forces and phenomena (e.g., infrared light, sound visualization, various types of

crop coloring in satellite images etc.). Chapter 3 comments on visualization tools used in

the AEC industry.

Chapter 4 described various modes of being — real, virtual, between and beyond.

Combining real and theoretical phenomena, I attempted to describe that which is best

understood through experience. Using the concept of meta-communication and the

person/player/character relationships that typifies the gaming industry, I identified

categories and characteristics of gaming spaces and suggested how they might be used in

the formulation of a theoretical gaming design tool. The power of contemporary game

engines has improved significantly in recent years. Using a game engine alongside

current architectural tools suggests interesting possibilities.

Chapter 5 outlined the design experimentation I undertook as a part of this research. The

scenarios presented promote a technological marriage between gaming and architectural

visualization. By utilizing Quake, Quicktime, FarCry, and real-time video feeds, I

explored how hybrid environments might challenge modes of being (real, virtual, and in

between), allow us to leverage the redundant nature of overlapping spaces, objects,

entities, and to extend our concepts of subjectivity. Using the g a m e engine to support

various modes of immersion and interaction, I was able integrate gaming environments

with architectural models - taking the digital architectural model well beyond its

current, limited use for presentations purposes. These varying spatial conditions suggest

new kinds of interactions that redefine concepts like presence and reality.

The current focus in the gaming industry is on making products increasingly realistic.

Game developers would find it difficult to market games with/aAre-looking models due to

gamers' expectations regarding realism, in-game dynamics and physics, real sounds and

natural effects (fog, rain). A hidden architecture emerges in the gaming world when real-

world's rules (passage of time, lighting, gravity, friction, etc.) are re-enacted in the

virtual. For some people (players), this strong virtual realism creates a reality more

realistic than the realities they inhabit. For them, the virtual is a more authentic

experience than real life.

In the gaming industry, the imagination is often limited to trying to imitate the real.

Rather than games reaching beyond the imaginable, they take on a subdued role in

replicating the real in the digital format, presenting us with a virtual experience based

solely on the real.

"C.S. Peirce, defines the virtual as follows: "A virtual X (where X is a common
noun) is something, not an X, which has the efficiency (virtus) of an X." His
definition does not adhere to either of the OED definitions strictly, as it indicates
an "external form or embodiment," which does not reference itself and which
stands in for something else ("a virtual (as opposed to actual) thing"), but in such
a way as to be as effective in terms of representation as the actual thing.'"
 87

"The farthest extension of the modern usage ofvirtuality is manifest in

Baudri Ilard's notion of the collapse of the distinction between the real and the
virtual. For Baudrillard, the enormous circulation of images that marks the last
half of the 20th century (and is marked by television more than any other
technology) introduces an ontological confusion in terms of the image that is not
present even for Peirce; for Baudrillard, the virtual does not indicate a difference
across orders, but indeed the complete collapse of orders, so that the real and the
virtual become indistinguishable (on an ontological level). For Baudrillard,
images have lost their ability to be virtual because a real referent can no longer
be distinguished - what once was designated as virtual is now more present than
reality itself (or reality is so imbricated with these images that the difference
becomes moot). Against virtuality, Baudrillard posits the hyper-real. "29

Perhaps the gaming industry is striving for the hyper-real, motivated by a television-like

concept of the authentic - the logical conclusion of which (as noted) would be increased

incorporation of real-time video feeds. The virtual can however relate to the real in other

than a mimetic fashion. The virtual has the power to distinguish itself from (transform,

transcend) the real, by presenting a different set of rules (physics, friction, colors, light

patterns, time systems, etc.) — an infinite world of possibility emerging from (but not

numerically bound by) the real.

In terms of the impact of n e w techniques and technologies on architecture, n e w design

software may take environmentally responsive architecture to new heights. Where

building form is concerned orthogonal designs may give way to forms associated with

nature. Perhaps new designs will take into consideration the function of a particular

space and shape its geometry according to the dynamics already existing in adjacent

spaces or the natural world. Newly developed materials or ways of assembling, can only

be assembled using the computer; Frank Gehry's museum at Bilbao is an example of

organic form, the use of new materials (titanium), and computer-based fabrication of

building components. New materials and assemblages may become so complex that they
can only be visualized and realized with the use of the computer, and will require gaming

technologies to test functions, forces, etc.

Sentient virtual environments immerse players within rich spaces, leveraging cause-and-

effect interaction to deepen engagement. Real world architecture is becoming sentient as

well. Room lighting and temperature controls deploy sensors to automatically adjust to

the presence and preferences of inhabitants. Perhaps walls, windows, doors and the

systems of a building will become similarly automated in such a way as to respond to

individual users' needs and desires. Building overhangs may automatically enlarge and

change position according to the position of the sun and the user in the space. A fully

conscious physical space would create a new kind of causality between spaces and

inhabitants, and allow architects to conceptualize form in a more dynamic and organic

way.

We've observed that architects use a vast array of digital tools as a means to an end,

namely to model buildings that are intended to be constructed. For the architect, the

virtual model is primarily a representation. Architectural design focuses on creating

spaces that are appropriate their context, budget and to the functions they accommodate;

form often tells about function and vice versa. More often than not, architects design

spaces that accommodate events and social gatherings. This thesis has gone to great

lengths to emphasize that contemporary game designers now also design social spaces

that foster events and support shifting scenarios. Similar to the real, game environments

promote responses to stimuli such as color of a space, material of walls, shapes of

particular space and so on - they are neither functionally nor architecturally neutral.
 89

Game and architecture design relate because they both promote the use of space for

events supporting inhabitation and experience. Whether or not architects support what is

happening in the game industry (or indeed are even aware of it), overlap between the

disciplines is happening.

We observed that the digital model can perform a different function once the building is

designed and constructed - extending and enhancing the experience of the actual

building. With a diverse palette of virtual possibilities, a digital architecture can be

explored in another way. Time and space can be adjusted, and buildings systems opened

and analyzed in a new way. Anything that cannot be allowed or possible in the real

world, could potentially be accommodated in the virtual. Like the real architecture,

virtual architecture should be designed to inspire, support human activity and interaction,

increase our range of experience, enhance relationships, promote a deeper understanding

and appreciation of the built environment, and augment the quality of life; virtual or real.

I hope that the knowledge gained from this thesis will encourage students and design

professionals to expand the tools they use to include gaming, photography and video.

Exploring the subversive potential of new media will surely present, perhaps by accident,

a new way of seeing ourselves, and of understanding and appreciating spaces, places, and

the world around us. The kinds of visuals and experiences that gaming brings to non-

gaming scenarios are astonishing. The incorporation of gaming into the architectural

process has the potential to transform the way architects work - and transform

architecture in the process. Whatever else they might be, the new tools and techniques

represent a word of possibilities.

 90

Endnotes:

Quake blows away design problems. From [Link]. (17 Jan 2006)
<[Link]
2
Gunnar Liestl, Andrew Morrison, and Terje Rasmussen. Digital Media Revisited. (Cambridge, Mass.: M I T Press,
2003). Introduction.
3
Katie Salen and Eric Zimmerman, Rules of Play (Cambridge: The M I T Press, 2004) Introduction.
4
Steve Klett 3d G a m e Engines Exposed Maximum PC Fall 2004, 66.
Barry Atkins, More than a game : the computer game as fictional form (New York : Manchester University Press, 2003) 68
Barry Atkins, More than a game : the computer game as fictional form (New York : Manchester University Press, 2003) 66.
7
Salen 32.
8
Neil Spiller, Cyber reader: Critical Writings for the Digital Era (New York: Phaidon Press, 2002) 300.
9
Spiller218.
10
Deborah Snoonian, "Digital Practice: Software interoperability makes strides, but hurdles still remain," Architectural
Record 191.9 (2003): 177.
11
Peter Zellner, Hybrid Space: N e w Forms in Digital Architecture (New York: Rizzoli, 1999) 167.
12
Thomas Mayne, "Morphosis: Works," Symposium Forum, David Azrieli Theatre, Carleton University, Ottawa, 23 Apr. 2004.
13
Hart 154.
14
Hart 152.
15
Eugene F Provenzo Jr., Video Kids (Cambridge: Harvard University Press, 1991) 22.
16
Atkins 146.
17
Katie Salen and Eric Zimmerman, Rules of Play (Cambridge: The M I T Press, 2004) 450.
18
Salen 452.
19
Salen 452.
20
Salen 450.
21
"Small Artist Pushing Technology", Version 2.3, computer controlled video installation, Josh Baer Gallery, N e w
York, N e w York, 1991
22
Bradley, Will. Karlskrona2. From: [Link] (20 Feb 2005) <[Link]
23
Katie Salen and Eric Zimmerman, Rules of Play (Cambridge: The M I T Press, 2004) 193.
24
Salen 196.
25
Salen 198.
26
Salen 198.
27
Bryan Magee, The Story of Philosophy. (Montreal: The Reader's Digest Association Ltd., 1998) 134.
28
Virtuality. From [Link] (18 Jan 2006)
<[Link]
29
Virtuality. From [Link] (18 Jan 2006)
<[Link]
 91

Bibliography

Albrecht, Donald. Design Culture Now. New York: Princeton Architectural Press, 2000.

Atkins, Barry. More than a game : the computer game as fictional form. New York :
Manchester University Press, 2003.

Baudrillard Jean. Simulacra and Simulation. Michigan: The University of Michigan

Press, 1997.

Benedikt, Michael. Cyberspace: First Steps. Cambridge: MIT Press., 1993.

Bertol, Daniela. Designing Digital Space: An Architect's Guide to Virtual Reality.

Toronto: John Wiley & Sons, 1997.

Blau, Brian & Clark Dodsworth. Visual Proceedings: The Art and Interdisciplinary
Programs of S I G G R A P H 96. N e w York: Association for Computing Machinery Inc.,
1996.

Catinella, Rita F. "The Future: The Next Generation of Architects will Design
Completely Within a Virtual World. What will be the Architectural Language of
the Future?" Architectural Record 187.12 (1999): 276.

Jacobs, Jane. The Death and Life of Great American Cities. London: Jonathan Cape,
1962.

Kerlow, Isaac Victor. The Art of 3-D: Computer Animation and Imaging. Toronto: John
Wiley & Sons, 2000.

Klett, Steve. (Editor) 3d Game Engines Exposed Maximum PC Fall 2004, 66.

Liestl, Gunnar, Andrew Morrison, and Terje Rasmussen. Digital Media Revisited.
Cambridge, Mass.: M I T Press, 2003.

Lubell, Sam. "Building with Clicks, Not Bricks." Architectural Record 190.12 (2002):
170-175.

Magee, Bryan. ThP Story of Philosophy. Montreal: The Reader's Digest Association
Ltd., 1998.

Mayne, Thomas. "Morphosis: Work" Symposium Forum, David Azrieli Theatre,

Carleton University, Ottawa. 23 Apr. 2004.

Postmodernism and the Three Types of Immersion - By Ernest Adams - Gamasutra, (9

July 2004). <[Link]
Provenzo, Eugene F. Jr. Video Kids. Cambridge: Harvard University Press, 1991.

Raph Koster, Online World Timeline. From [Link] (22 Mar. 2005)
<[Link]

Salen, Katie and Eric Zimmerman. Rules of Play. Cambridge: The MIT Press, 2004.

Schmitt, Gerhard. Information Architecture. Boston: Birkhauser, 1999.

Spiller, Neil. Cyber reader : Critical Writings for the Digital Era. New York: Phaidon
Press, 2002.

Sprague, Elmer and Paul W. Taylor. Knowledge and Value. New York: Harcourt,
Brace and Company, 1959.

Superflex, supercity. From: KARLSKRONA2: (5 Feb 2005)

<[Link]

Wexelblat, Alan. Virtual Reality: Applications and Explorations. Toronto: Academic

Press Professional, 1993.

Whyte, Jennifer. Virtual Reality and the Built Environment. New York: Architectural
Press, 2002.

Zellner, Peter. Hybrid Space: New Forms in Digital Architecture. New York: Rizzoli,
1999.

Zhai, Philip. Get Real: A Philosophical Adventure in Virtual Reality. New York:
R o w m a n & Litttlefield, 1998.