AUTHOR=Qiao Kai , Chen Jian , Wang Linyuan , Zhang Chi , Zeng Lei , Tong Li , Yan Bin 

TITLE=Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 13 - 2019

YEAR=2019

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2019.00692

DOI=10.3389/fnins.2019.00692

ISSN=1662-453X

ABSTRACT=Recently, visual encoding and decoding based on functional Magnetic Resonance Imaging (fMRI) have made many great achievements with the rapid development of deep network computation. In spite of hierarchically similar representations of deep network and human vision, the visual information flows from primary visual cortices to high visual cortices, and conversely, from high visual cortices to primary visual cortices respectively based on the bottom-up manner and top-down manner. Inspired by the bidirectional information flows, we proposed the bidirectional recurrent neural network (BRNN) based method to decode the category from fMRI data. The forward and backward directions in the BRNN module characterized the bottom-up and top-down manners respectively. The proposed method regarded selected voxels of each visual cortex region (V1, V2, V3, V4 and LO) as one node in the sequence that feed into the BRNN module, and combined the output of BRNN module to decode the category with the subsequent fully connected layer. In this way, not only hierarchical information representations but also bidirectional information flows in human visual cortices can be efficiently used. Experiment results demonstrated that our method improved the accuracy on the three-level category decoding task than other methods, which implicitly validated the hierarchical and bidirectional human visual representations. Through comparison, we also analyzed and concluded that the category representations of human visual cortices were hierarchical, distributed, complementary and correlative.