Impact Factor 3.566

The Frontiers in Neuroscience journal series is the 1st most cited in Neurosciences

Original Research ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Neurosci. | doi: 10.3389/fnins.2018.00197

Interactive Sonification Exploring Emergent Behavior Applying Models for Biological Information and Listening

  • 1School of Arts and Media, University of Salford, United Kingdom

Sonification is an open-ended design task to construct sound informing a listener of data. Understanding application context is critical for shaping design requirements for data translation into sound. Sonification requires methodology to maintain reproducibility when data sources exhibit nonlinear properties of self-organization and emergent behaviour. This research formalizes interactive sonification in an extensible model to support reproducibility when data exhibits emergent behaviour. In the absence of sonification theory, extensibility demonstrates relevant methods across case studies. The interactive sonification framework foregrounds three factors: reproducible system implementation for generating sonification; interactive mechanisms enhancing a listener’s multisensory observations; and reproducible data from models that characterize emergent behavior. Supramodal attention research suggests interactive exploration with auditory feedback can generate context for recognizing irregular patterns and transient dynamics. The sonification framework provides circular causality as a signal pathway for modelling a listener interacting with emergent behavior. The extensible sonification model adopts a data acquisition pathway to formalize functional symmetry across three subsystems: Experimental Data Source, Sound Generation, and Guided Exploration. To differentiate time criticality and dimensionality of emerging dynamics, tuning functions are applied between subsystems to maintain scale and symmetry of concurrent processes and temporal dynamics. Tuning functions accommodate sonification design strategies that yield order parameter values to render emerging patterns discoverable as well as rehearsable, to reproduce desired instances for clinical listeners. Case studies are implemented with two computational models, Chua’s circuit and Swarm Chemistry social agent simulation, generating data in real-time that exhibits emergent behaviour. Heuristic Listening is introduced as an informal model of a listener’s clinical attention to data sonification through multisensory interaction in a context of structured inquiry. Three methods are introduced to assess the proposed sonification framework: Listening Scenario classification, data flow Attunement, and Sonification Design Patterns to classify sound control. Case study implementations are assessed against these methods comparing levels of abstraction between experimental data and sound generation. Outcomes demonstrate the framework performance as a reference model for representing experimental implementations, also for identifying common sonification structures having different experimental implementations, identifying common functions implemented in different subsystems, and comparing impact of affordances across multiple implementations of listening scenarios.

Keywords: Sonification, listening, emergent behavior, Interaction design, biological information, cognitive cycle, supramodal attention, chaotic dymamics, social agents

Received: 04 Oct 2017; Accepted: 12 Mar 2018.

Edited by:

Diego Minciacchi, Università degli Studi di Firenze, Italy

Reviewed by:

Hiroaki Wagatsuma, Kyushu Institute of Technology, Japan
Waldemar Karwowski, University of Central Florida, United States  

Copyright: © 2018 Choi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Prof. Insook Choi, University of Salford, School of Arts and Media, MediaCityUK Room 2.08C, City of Salford, M50 2HE, United Kingdom,