Humans perseverate on punishment avoidance goals in multigoal reinforcement learning


P. B. Sharp, E. M. Russek, Q. J. M. Huys, R. J. Dolan, and E. Eldar. 2022. “Humans perseverate on punishment avoidance goals in multigoal reinforcement learning.” Elife, 11.


Managing multiple goals is essential to adaptation, yet we are only beginning to understand computations by which we navigate the resource demands entailed in so doing. Here, we sought to elucidate how humans balance reward seeking and punishment avoidance goals, and relate this to variation in its expression within anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). We constructed a computational model of multigoal pursuit to quantify the degree to which participants could disengage from the pursuit goals when instructed to, as well as devote less model-based resources toward goals that were less abundant. In general, participants (n = 192) were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, participants often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. In a similar vein, participants showed no significant downregulation of avoidance when punishment avoidance goals were less abundant in the task. Importantly, we show preliminary evidence that individuals with chronic worry may have difficulty disengaging from punishment avoidance when instructed to seek reward. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward. Future studies should test in larger samples whether a difficulty to disengage from punishment avoidance contributes to chronic worry.


2050-084xSharp, Paul BOrcid: 0000-0003-4949-1501Russek, Evan MHuys, Quentin J MDolan, Raymond JOrcid: 0000-0001-9356-761xEldar, Eran1094/20/Israel Science Foundation/098362/Z/12/Z/WT_/Wellcome Trust/United KingdomPS00318453/Fulbright Association/R01MH125564/NH/NIH HHS/United States2019801/Israel Binational Science Foundation/WT_/Wellcome Trust/United KingdomR01MH124092/NIH Blueprint for Neuroscience Research/R01 MH124092/MH/NIMH NIH HHS/United StatesR01 MH125564/MH/NIMH NIH HHS/United StatesR01MH124092/NH/NIH HHS/United StatesOpen Access Funding/2Max Planck UCL Centre for Computational Psychiatry and Ageing Research/Journal Article2022/02/25Elife. 2022 Feb 24;11:e74402. doi: 10.7554/eLife.74402.