Elemental Agency (2017)

Elemental Agency is a piece concerned with an emergent gestural language that manifests across movement, sound and light. Rather than sound/light articulating movement, or movement driving media, the approach builds outward from gestural metaphors as a point of shared intersection for these phenomena. The work draws upon the metaphorical constructs of Japanese Godai and Indian Vaastu Shastra theories of five elements: earth, water, wind, fire and void/space. Working with conceptual metaphors drawn from these traditions, constraints are provided to the dancers of embodying a given element as a collective – a texture of movement, rather than a singular human entity, that manifests the non-human agency of a given element. Motivations related to world, body, motion, emotion and enaction are given: earth as stubbornness and resistance to change, wind as expansive, elusive and compassionate, fire as energetic and forceful, etc. Using machine learning methods, the sonic interaction designers seek to capture moments of gestural expression and fuse this with sounds that similarly embody a given elemental quality/profile. Negotiation between art forms leads to a collective choreography and sound design that emerges from the embodiment of the nonhuman elements and the mediation of the machine agents. Visual projection functions as both lighting and enhancement of embodied experience in space, rather than screen-oriented media experience. Spotlights both enhance elemental qualities as well as embodying behaviours that align with the constraints of the element metaphors: rigid tracking of movement within earth state, amorphous following within water state, tendency towards consumption in fire state, etc.

The pieces moves through short movements of distinct elements: water swells and then recedes into earth, which tectonically shifts before dispersing to the wind, dying down to invoke fire. Gestures and movement are recognized and mapped to sound and light output. The collective texture of these elements then shifts to the individual gesturality of each dancer, who singularly choose to embody a given element: moving from the collective texture of mono-element to the collision and transformation between elements, passing between dancers. The pervasive aether of machine listening recognizes gestures captured throughout the four-elements sections and re-inject loops of these into the performance: this aether ever-listening to elements, and re-performing its understanding of each dancers elemental offerings. Increased density and destabilization of dancers’ elemental transformations lead to an ascendance into the void: pure energy, beyond the everyday, a distribution/suspension/expansion of presence and expression. Void is devoid of any object-hood or gestural points of reference.

The piece rests in void state during its installation period. If audience members interact (browser via laptop or mobile device) with the elemental portal, they may bring the piece back into the world of a given elemental state. The performance space then invites interaction from audience.


Doug Van Nort: conception, direction, system composition, void state sound/interaction design 
Vanessa Boutin, Holly Buckridge, Shaelynn Lobbezoo, Joshua Murphy, Paige Sayles, Marie-Victoria de Vera: emergent and collective choreography, dance
Yirui Fu: spotlight visual/behaviour design
Akeem Glasgow: kinect tracking and mapping to visuals
Rory Hoy: earth state sound/interaction design
Ian Jarvis: fire state sound/interaction design
Kieran Maraj: water, wind states sound/interaction design
Michael Palumbo: mobile audience interaction,network architecture programming 
Mingxin Zhang: spotlight visual/behaviour design

Additional References:

Van Nort, D. “Gestural Metaphor and Emergent Human/Machine Agency in Two Contrasting Interactive Dance/Music Pieces”. Proceedings of the 2020 International Symposium of Electronic Arts (ISEA), 2020.

Jarvis, I., D. Van Nort. Posthuman Gesture, in Proc. of the International Conference on Movement and Computing (MOCO), 2018.