This is a raw outline for a fresh start, based on Michael's notes for the CCA-NSERC new media program last year.
1 WHAT is it?
1.1 Einstein's Dreams is an environment in which visitors encounter performers in responsive fields of video, light, and spatialized sound, in a set of tableaus Each tableau is inspired by a vignette from Alan Lightman's novel, Einstein's Dreams, set in Berne Switzerland, in 1904, the year that Albert Einstein received the Nobel prize. Or rather, a set of parallel 1904's, each of which is a different kind of time: in one, time slows to a halt as you approach a particular place; in another there is no future; in third, time sticks and slips; in a fourth age reverses and what is rotten becomes fresh as time passes.
1.2 In one concept, a large theatrical space (24 x 20 x 8m) will contain multiple tableaux, each with room for 6-12 people in its own pool of light and sound. Visitors and perhaps performers can move from tableaux to tableaux. The performers' actions will evolve in concert with the dynamics of lighting, sound, and visitors' expectations in order to create different kinds of time, inspired by the novel's vignettes. Sometimes a performer will walk from a location, dragging the pool of conditioning light and sound. The pool mutates or merges into another pool with a different type of time.
2 A literature review and/or historical contemporary understanding of the context for the research
2.1 One hundred years after the two epochal advents in physics of relativity theory and quantum mechanics, we are still reverberating with the consequences. ED is not a biography or a didactic allegory, but a poetic exploration of our consciousness in time.
2.2 There is huge volume of elaborations and didactic works about the theory of relativity and quantum mechanics, including Einstein's own popular essays, and the canonical scientific / philosophical works by Hermann Weyl, Alfred N. Whitehead Principle of Relativity, Henri Bergson Duration and Simultaneity. In the arts, there have been some distinguished works that treat the theories and the theorists in an externalist way, as icons or as social phenomena. But what we propose is to work directly with the spectators' felt experiences of time.
2.3 What Alan Lightman evoked in his novel was a poetic variation around the felt experience of time, not the "actual" physics of time, but alternatives of time, those hypothesized modes of living in time that could have been imagined, or that never were. By foregrounding how time works in these worlds, the vignettes foreground movement, which is the temporalization of the body. These movements are embedded in everyday life, made marvelous by poetic conceits of time. This fits perfectly with both Sha and Montanaro's own artistic research into the charging of movement and gesture, of finding or evoking marvelous configurations of movement in the everyday.
2.4 Scientifically ED will create an experimental apparatus for inducing perceptibly different senses of temporal passage, chance and order, mortality and anisotropy -- the arrow of time. Our goal is to use the poetic techniques of theater and dance and responsive media to evoke these sharply different ways of temporal experience so that the visitors will feel this for themselves. Our experimental goal will be to discover ways to not just depict different kinds of temporal processes, but to condition a physical setting to yield first-person experiences of these different kinds of being in time.
2.5
Re-interpreting C. Geertz's term "thick," the Topological Media Lab designs thick experimental events environments sensitive to the effect of the whole ensemble, rather than neutralizing all perceptual features but one measurable parameter such as the end-to-end latency of a force-feedback controller.
One scientific and phenomenological application of TML's holistic --
thick -- design approach to responsive environments is a collaboration with Dr. Tore Nielsen's Dream and Nightmare Laboratory (U. de Montréal) investigating the phenomenon of felt presence. In that alternative to a minimalist psychological experiment, we built a whole room furnished like a small parlor, but whose furniture, lighting, and sound fields were subtly "animated" according to precisely fashioned scripts. Another is a project with Prof. David Morris (Philosophy, Concordia) about the relation between memory, identity, and how a body comports in a physical place.
The Topological Media Lab was founded in 2001 on the premise that the disciplines with the most experience in the design of responsive environments, media, and "interaction" are in fact the performing arts.
As an artist, Montanaro has been concerned with shaping whole events, not as the sum of parts like special effects, or technical action and reaction. This meshes very well with the Sha's guiding concern with creating "thick" events that are "rich, but not complicated." In fact the TML's methodological research has been to discover ways to understand how to condition a physical space using "thick" design techniques: "media choreography." [EXPLAIN]
2.6 Montonaro's aesthetic values _____________ (symbolic power, joy, magic, ...), and prefers not to fetishize the computational technology, which accords perfectly with Sha's aesthetic as well. Following analogous design strategies in the engineering community like Mark Weiser's calm technology, or the European Union's ___-th Framework: "Disappearing Computer," we intend to create events in which the computational technology works as unobstrusively as well-designed lighting.
2.7 The core concerns of ED circle around the relationships set up in a performance, or an event in which at least momentarily the intents of creator (author, composer, choreographer) entangle with the intents of the performer and observer. For Montanaro, performance at its heart is a relational triangle: performer - choreographer/director - observer. For Sha this suggests fresh insights into questions raised in relativity and quantum mechanics about the how the observer is implicated in the construction of phenomena. In relativity, the fundamental relation is between two frames of reference, one at rest with respect to the observer, and another in motion. In quantum mechanics, the observed parameter is replaced by the notion of an expectation value which is the symmetrical inner product of the state of the observer with the Hermitian acting on the state of the observed. We can never directly measure the state of the observer or observed.
[FIGURE 1. Performance is relational triangle: performer - choreographer/director - observer]
[FIGURE 2. compare with relativity: observer - rest-frame - moving-frame; or quantum mechanics: <observer state, apparatus (hermitian operator), observed state>]
2.8 Remove if-then logic at macro (human) scale as well as CPU scale =>
realtime concurrent programming vs "object-oriented programming" (OOP)
texture programming vs. OOP.
2.9 Performer <-- act-on --> environment; but also self-act-on as well. concurrency and distributed agency =>
need new movement structures <----> continuous state-based media choreography. the experiment is to test the reach and adequacy of the Ozone as a compositional technique
2.10 Superposing, overlaying, imposing ritual/ formalized / artificially enhanced action on top of pedestrian, everyday actions generates interesting tensions of intent ===> Einsteins Dreams: apply temporal operators as a suite of estranging , magicking "devices" using temporal textures (parallel , prior research at TML)
2.11 For performance, we can always "cheat" and depict. - i.e. allegory. but instead of thinking of that as cheating, think of it as "Turk"-- ie theater:
2.12 The question is how can we bring the observer inside the event? how can we induce altered, or alter ... temporal consciousness, according to Einsteins Dreams?
2.13 The role of the observer
In an interactive performing arts event a large part of the observer’s perception, understanding and appreciation is third person reliant, resting heavily on the performer’s interpretation of the creative artists dialogue as well as the performer’s physical and emotional relationship to the environment. Even though the observer and performer have access to the same audio and visual information the observer’s physical connection with the material is, experientially speaking, vicarious. Though this separation exists in traditional live performance the difference is amplified many times over with the introduction of the technologically enhanced or intelligent space. It is here, at the point of divergence between performance and non-performance based forms of interactive artistic expression, that my interest and current research lie. Performer or voyeur, the separation runs deep
2.14 Parallel with relativity and modern physics:
The idea of the observer that both relativity and QM brought explicitly into the frame : see next sections
3 The underlying technical/scientific issues or complexities, along with an explanation of how they will be resolved 3.1 Temporal textures
3.2 Think of
filters rather than projections of image or sound-field. We create realtime systems that sample the existing sound or lightfields (eg from arrays of microphones and cameras), and re-synthesize them and send them with meaningful spatial or temporal transforms back into the given tableaus. Simple techniques, such as temporal delays that we can regionalize down to individual pixels or fixtures with resolution from milleseconds to hours, based on high resolution cameras with high frame rates (120 to 600 FPS) already can provide rich effects.
But additionally we propose to leverage the TML's 5 years of experience adapting computational physics techniques to create temporal textures that have the qualities of physical materials -- smoke, water, syrup, ice, etc. We are investigating how to leverage the person's tacit knowledge of physical materials to give a sense of tangibility that cuts across sense modalities. This is an example of user interaction with "interfaces" so dense (like cloth or elastic controllers) as to require continuous models and a phenomenological rather than cognitive approach.
3.3
Continuous State-based Media Choreography
4 The names and titles of the members of the project team, their roles in the collaboration and how the project will be managed 4.1 Michael Montanaro
4.2 Sha Xin Wei
5 The integrated work plan for the research and/or production activities
5.1 WHERE will it be done?
Montreal Usine-C
Tate Modern 2011
Harvard Biochem building (Galison)
Under Brooklyn Bridge
San Francisco Presidio
6 The nature of the industrial participation, if any, and its contribution to the project (including a letter of support and an NSERC Form 183A for each industry partner) 7 How participation in this project will enrich the professional experience of the individual artist, the scientist/engineer and other participants
7.1 Montanaro has worked for years with dance and
fresh ways to choreograph, other than usual action (dancer) reaction (system)
7.2 Xin Wei: new ways to understand time and sense of time, ways to create temporal texture : field-based ways to
7.3 Opportunity to working together experimentally,. Have done it voluntarily on our own time for 5 years, now wish to do it funded.
8 Plans to manage intellectual property and copyright issues related to the proposed project
9 How the collaborators will evaluate their collaboration and research outcomes
9.1 production
9.2 publication in relevant journals: ACM Multimedia, Smart Graphics, Ubicomp, TDR,
10 How the project results will be disseminated
10.1 performance
10.2 workshops with students in dance and media systems
10.3 in publications
11 The project’s impact on the broader artistic and scientific communities, and the social, cultural, economic or environmental benefits to Canada. 11.1 lighting and movement