reinforcement learning and decision making
Reversal learning and Compulsivity: Role of Dopamine
Brain dopamine is probably best known for its implication in reinforcement learning. We study the contribution of dopamine to not only reward, but also punishment learning, in the context of reversal learning paradigms, as models of compulsivity. Indeed dopamine-related compulsivity, as seen in addiction or in medicated Parkinson’s disease, has often been attributed to dopamine-induced increases in the weight on the benefits versus the costs of actions. In the lab, we combine psychopharmacology with computational reinforcement learning modeling, genetics, electrophysiology and/or neuroimaging (EEG, PET and fMRI) to increase our understanding of the paradoxical relationship between dopamine and compulsivity.
For example, we have shown, using computational genetics, that effects of the dopamine transporter polymorphism on perseveration after a reversal are better accounted for by increased reliance on previous reinforcement, corresponding computationally to a reduction in the learning rate, than by reduced sensitivity to punishment (den Ouden et al., 2013). Furthermore, in a different line of work, we have demonstrated that the same dopaminergic drug can have diametrically opposite effects on reversal learning, in subjects with high and low baseline dopamine synthesis capacity in the striatum (Cools et al. 2009). Moreover, the use of a pretreatment design enabled us to conclude that such effects are mediated by the dopamine D2 receptor. That study also showed that it was accompanied by modulation of BOLD signal in the striatum and can be predicted from individual variation in baseline working memory capacity (Van der Schaaf et al., 2014). In ongoing work, we acquire PET and psychopharmacological (fMRI) data from large samples of volunteers (n±100 ) to account for the large individual variability in drug effects on reward learning, thus incidentally also contributing to precision psychiatry.
Relevant publications
Van der Schaaf ME, Van Schouwenburg MR, Geurts D, Schellekens AFA, Buitelaar J, Verkes RJ, Cools R (2014). Establishing the dopamine-dependency of human striatal signals during reward and punishment reversal learning. Cereb Cortex 24(3):633-42
Den Ouden H, Fernandez G, Elshout J, Rijpkema M, Hoogman M, Franke B, Daw ND, Cools R (2013). Dissociable effects of dopamine and serotonin on reversal learning. Neuron 80(4):1090-100
Van der Schaaf ME, Van Schouwenburg MR, Geurts D, Schellekens AFA, Buitelaar J, Verkes RJ, Cools R (2014). Establishing the dopamine-dependency of human striatal signals during reward and punishment reversal learning. Cereb Cortex 24(3):633-42
Van der Schaaf ME, Fallon SJ, ter Huurne N, Buitelaar J, Cools R (2013). Working memory capacity predicts effects of methylphenidate on reversal learning. Neuropsychopharmacology 38(10):2011-8
Van der Schaaf ME, Zwiers MP, van Schouwenburg MR, Geurts DEM, Schellekens AFA, Buitelaar JK, Verkes RJ, Cools R (2013). Dopaminergic drug effects during reversal learning depend on anatomical connections between the orbitofrontal cortex and the amygdala. Front Neurosci. Aug 14;7:142
Von Borries K, Verkes RJ, Bulten BH, Cools R, de Bruijn ERA (2013). Feedback-related negativity codes outcome valence, but not outcome expectancy during reversal learning. Cogn Affect Behav Neurosci 13(4):737-46
Smittenaar P, Chase HW, Aarts E, Nusselein B, Bloem BR, Cools R (2012). Decomposing effects of dopaminergic medication in Parkinson’s disease on probabilistic action selection: learning or performance? Eur J Neurosci 35(7):1144-51
Van der Schaaf M, Warmerdam E, Crone E, Cools R (2012). Distinct linear and non-linear trajectories of reward and punishment reversal learning during development: Relevance for dopamine's role in adolescent decision making. Developmental Cognitive Neuroscience 1(4):578-90
Chase HW, Swainson R, Durham L, Benham L, Cools R (2011). Feedback-related negativity codes prediction error, but not behavioural adjustment during probabilistic reversal learning. J Cogn Neurosci. 23(4):936-946 [5.7]
Robinson OJ, Frank MJ, Sahakian BJ, Cools R (2010). Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage 51:1459-1467
Robinson OJ, Standing HR, DeVito EE, Cools R, Sahakian BJ (2010). Dopamine precursor depletion improves punishment prediction during reversal learning in healthy females but not males. Psychopharm 211(2):187-95
Cools R, Frank MF, Gibbs SE, Miyakawa A, Jagust W, D'Esposito M (2009). Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J Neurosci 29: 1538-1543
Clatworthy PL, Lewis SJ, Brichard L, Hong YT, Izquierdo D, Clark L, Cools R, Aigbirhio FI, Baron JC, Fryer TD, Robbins TW (2009). Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory. J Neurosci 29(15):4690-6
Dodds CM, Müller U, Clark L, van Loon A, Cools R, Robbins TW (2008). Methylphenidate has differential effects on blood oxygenation level-dependent signal related to cognitive subprocesses of reversal learning. J Neurosci 28(23):5976-82
Cools R, Lewis SGJ, Clark L, Barker RA, Robbins TW (2007). L-DOPA disrupts activity in the nucleus accumbens during reversal learning in Parkinson’s disease. Neuropsychopharmacology 32 (1): 180-189
Cools R, Altamirano L, D’Esposito M (2006). Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia 44 (10):1663-1673
Cools R, Clark L, Owen AM, Robbins TW (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional MRI. J Neurosci 22: 4563-4567
Cools R, Barker RA, Sahakian BJ, Robbins TW (2001). Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex 11:1136-1143