Time-frequency feature calculation of multi-stage audiovisual neural processing via electroencephalogram microstates

Yang Xi^1*Lu Zhang²Cunzhen Li²Xiaopeng Lv³Zhu Lan²

• ¹School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, China
• ²Northeast Electric Power University, Jilin, China
• ³Jilin City Hospital of Chemical Industry, Jilin, China

Audiovisual (AV) perception, an effective, commonly used modality for environmental cognition and social communication, involves multisensory processing of large-scale neuronal activities exhibiting nonlinear characteristics modulated by attentional mechanisms; however precise characterization of AV processing remains elusive. We designed an AV semantic discrimination experiment to acquire electroencephalogram (EEG) data under attended and unattended conditions. We developed an EEG microstate-based method for segmenting AV processing into functional sub-stages (temporally resolved neural signatures). Through hierarchical clustering of global field power-peak topographic maps and the Krzanowski-Lai Global Explained Variance-based evaluation, we identified distinct temporally continuous microstate sequences characterizing attended/unattended processing. By analyzing filtered EEG data across frequency bands, we quantified microstate attributes to derive time-frequency features that achieved 97.8% accuracy in classifying attended/unattended states and 98.6% accuracy in discriminating unimodal (visual/auditory) versus multimodal processing using multiple machine learning models. Our method effectively characterizes AV processing dynamics and provides neurophysiologically interpretable explanations for classification outcomes.