Exploring Gestural Affordances in Audio Latent Space Navigation

Shuoyang Jasper Zheng^1*Anna Xambó Sedó¹Nick Bryan-Kinns²

• ¹Queen Mary University of London, London, United Kingdom
• ²University of the Arts London, London, United Kingdom

The latent space of generative AI models affords unique creative possibilities and broad design space for AI-enhanced digital music instruments. While interface designs for latent space navigation typically rely on sound-producing gestures that involve bodily motions and movements, the underlying subjective perception of these gestures remains underexplored. To understand how musicians perceive sound-producing gestures and tailor performance techniques in audio latent space, we present a user study workshop with an AI-enhanced digital music instrument with a tablet interface. Eighteen musicians were recruited to test out open-ended gestures and tasked to create musical scores. We report how they use sound-producing gestures in the latent space and develop performance techniques. We contribute findings from an embodied music cognition perspective of how subjective perception of gestures shapes musicians' technique development in audio latent space navigation. We discuss the implications of new gestural affordances discovered by participants in our workshop, aiming to elucidate new opportunities for digital musical instruments with audio latent space navigation.