AUTHOR=Chase Henry W. 

TITLE=A novel technique for delineating the effect of variation in the learning rate on the neural correlates of reward prediction errors in model-based fMRI

JOURNAL=Frontiers in Psychology

VOLUME=Volume 14 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2023.1211528

DOI=10.3389/fpsyg.2023.1211528

ISSN=1664-1078

ABSTRACT=Computational models play an increasingly important role in describing variation in neural activation in human neuroimaging experiments, including evaluating individual differences in the context of psychiatric neuroimaging. In particular, reinforcement learning (RL) techniques have been widely adopted to examine neural responses to reward prediction errors and stimulus or action values, and how these might vary as a function of clinical status. However, there is a lack of consensus around the importance of the precision of free parameter estimation for these methods, particularly with regard to the learning rate. In the present study, I introduce a novel technique which may be used within a general linear model (GLM) to model the effect of misestimation of the learning rate on reward prediction error (RPE)-related neural responses. Initially, I provide some theoretical background and simulations showing that the conventional approach to fitting RL models to RPE responses is more likely to reflect individual differences in a reinforcement efficacy construct (lambda) rather than learning rate (alpha). The proposed method, which is essentially adding a derivative regressor provides a second regressor which reflects the learning rate. Validation analyses were performed including examining two other equivalent methods which yield highly similar results, and a demonstration of sensitivity of the method in presence of fMRI-like noise. Overall, findings from the simulations underscore the importance of the lambda parameter for interpreting individual differences in RPE-coupled neural activity, and validate a novel neural metric of the modulation of such activity by individual differences in the learning rate. The method is expected to find application in understanding aberrant reinforcement learning across different psychiatric patient groups including major depression and substance use disorder.