Events in context -the HED framework for the study of brain, experience and behavior (rev 4) Provisionally Accepted

Scott Makeig¹ Kay A. Robbins^2*

• ¹Swartz Center for Computational Neurosciences, Institute for Neural Computation, University of California, San Diego, United States
• ²Department of Computer Science, University of Texas at San Antonio, United States

The brain is a complex dynamic system whose current state is inextricably coupled to awareness of past, current, and anticipated future threats and opportunities that continually affect awareness and behavioral goals and decisions. Brain activity is driven on multiple time scales by an ever-evolving flow of sensory, proprioceptive, and idiothetic experience. Neuroimaging experiments seek to isolate and focus on some aspect of these complex dynamics to better understand how human experience, cognition, behavior, and health are supported by brain activity. Here we consider an event-related data modeling approach that seeks to parse experience and behavior into a set of time-delimited events. We distinguish between event processes themselves, that unfold through time, and event markers that record the experiment timeline latencies of event onset, offset, and any other event phase transitions. Precise descriptions of experiment events (sensory, motor, or other) allow participant experience and behavior to be interpreted in the context either of the event itself or of all or any experiment events. We discuss how events in neuroimaging experiments have been, are currently, and should best be identified and represented with emphasis on the importance of modeling both events and event context for meaningful interpretation of relationships between brain dynamics, experience, and behavior. We show how text annotation of time series neuroimaging data using the system of Hierarchical Event Descriptors (HED; https://www.hedtags.org) can more adequately model the roles of both events and their ever-evolving context than current data annotation practice and can thereby facilitate data analysis, meta-analysis, and mega-analysis. Finally, we discuss ways in which the HED system must continue to expand to serve the evolving needs of neuroimaging research.