Model-based decision making in the human brain
Alan Hampton2007
Sign up for access to the world's latest research
Abstract
also want to thank friends outside of Caltech, who contributed to this work in pretty much the same way. Last of all, I want to thank my parents and sisters for the support and patience they have given during the years. Much of what I am now I owe to them. And to my dearest Allison, for her love and patience, and for knowing how to encourage me when things got tough, and tone me down when I was too excited! v ABSTRACT Many real-life decision making problems incorporate higher-order structure, involving interdependencies between different stimuli, actions, and subsequent rewards. It is not known whether brain regions implicated in decision making, such as ventromedial prefrontal cortex, employ a stored model of the task structure to guide choice (model-based decision making) or merely learn action or state values without assuming higher-order structure, as in standard reinforcement learning. To discriminate between these possibilities we scanned human subjects with fMRI while they performed two different decision making tasks with higher-order structure: probabilistic reversal learning, in which subjects had to infer which of two choices was the more rewarding and then flexibly switch their choice when contingencies changed; and the inspection game, in which subjects had to successfully compete against an intelligent adversary by mentalizing the opponent's state of mind in order to anticipate the opponent's behavior in future. For both tasks we found that neural activity in a key decision making region: ventromedial prefrontal cortex, was more consistent with computational models that exploit higher-order structure, than with simple reinforcement learning. Moreover, in the social interaction game, subjects were found to employ a sophisticated strategy whereby they used knowledge of how their actions would influence the actions of their opponent to guide their choices. Specific computational signals required for the implementation of such a strategy were present in medial prefrontal cortex and superior temporal sulcus, providing insight into the basic computations underlying competitive strategic interactions. These results suggest that brain regions such as ventromedial prefrontal cortex employ an abstract model of task structure to guide behavioral choice, computations that may underlie the human capacity for complex social interactions and abstract strategizing. vi TABLE OF CONTENTS
