Our behavior is motivated by rewards of different nature among which we
frequently need to choose. Because there is no single sense organ transducing
rewards of different types, our brain must integrate and compare them to
choose the options with the highest subjective value. Neuroimaging, electrophysiological,
behavioral, and pharmacological techniques, in combination with
molecular and genetic tools have begun to elucidate the neuronal mechanisms
underlying reward processing and decision-making. This book highlights some
of these advancements that have led to the current understanding of the brain’s
involvement on reward and decision-making processes. The book addresses one
fundamental question about the nature of behavior: how does the brain process
reward and makes decisions when facing multiple options?
A reward is an object that generates approach behavior and consumption, produces
learning, and represents a basis for decision-making. Reward is fundamental
to motivation, learning, cognition, and the organization of behavior. It is also
often a goal for behavior, allowing appropriate selection of sequences of responses
and decisions in order to reach that goal. The brain has developed a network,
known as the reward system, whose major components include dopaminergic
midbrain neurons, the ventral striatum, the prefrontal cortex (especially its orbitofrontal
part), and the amygdala. Dysfunction of this system is observed in several
neurological and psychiatric disorders as schizophrenia, Parkinson’s disease,
eating disorders, and drug addiction.
Although the capacity to predict reward information is essential for cognitive
functions, such as learning and motivation, the reward system is also involved in
more complex cognitive functions. For example, brain dopamine circuitry underlies
a number of behavioral functions that depend on sequential ordering, such
as sequential choice behavior. Moreover, dopamine reward signals influence processes
in both cortical and sub-cortical structures that modulate cognitive functions,
such as economic decision-making and working memory. Because the reward system
is involved in these different aspects, ranging from motivation to the organization
of behavior, a number of researches have started to combine brain imaging
and computational approaches to address problems related to classical conditioning,
decision making under risk or uncertainty and weighting between costs and
benefits.
It is still unclear whether specific functions can be ascribed to particular structures
of the reward system. Such functions include computing the value signals
Preface
of cues announcing potentially rewarded outcomes or of rewarded outcomes
(incentive and hedonic values), holding them in working memory to plan and
organize behavior toward an outcome, comparing actual and potential outcomes,
determining how well the outcome satisfies current needs and evaluating
the overall action schemes assessing the trade-offs between cost/efforts and benefits.
A general
framework remains to be developed to integrate these functions
and to explain how specific brain regions ensure our behavior to be most efficient
in satisfying our needs.
The neural bases of reward and decision-making processes are of great interest
to a broad readership because of the fundamental role of reward in a number of
behavioral processes (such as motivation, learning, and cognition) and because
of their theoretical and clinical implications for understanding dysfunctions of
the dopaminergic system and pathologies of reward processing. Findings in this
research field are also important to basic neuroscientists interested in the reward
system, cognitive psychologists working on conditioning/reinforcement, as well
as computational neuroscientists studying probabilistic models of brain functions.
Reward and decision-making cover a wide range of topics and levels of analysis,
from molecular mechanisms to neural systems dynamics and to computational
models. The contributions to this book are forward-looking assessments of the
current and future issues faced by researchers. We were fortunate to assemble an
outstanding collection of experts who addressed different aspects of reward and
decision-making processes. Five current topics are specifically addressed.
Part One is devoted to basic monkey anatomical and electrophysiological
data. In their natural environment, animals face a multitude of stimuli, very
few of which are likely to be useful as predictors of reward. It is thus crucial
that the brain learns to predict rewards, providing a critical evolutionary advantage
for survival. This first part of the book offers a comprehensive view on
the specific contributions of different brain structures as the dopaminergic midbrain
neurons, the ventral striatum, and the prefrontal cortex, including the lateral
prefrontal cortex and the orbitofrontal cortex, to the component processes
underlying reinforcement-guided decision-making, such as the representation of
instructions, expectations, and outcomes, the updating of action values and the
evaluation process guiding choices between prospective rewards. Special emphasize
is made on the neuroanatomy of the reward system and the fundamental
roles of dopaminergic neurons in learning stimuli–reward associations. In particular,
recent electrophysiological data in monkeys are reviewed showing that
the activity of dopaminergic neurons carries information about two statistical
parameters of reward: a transient reward prediction error signal that codes a
discrepancy between the reward actually delivered and the anticipated reward,
and a sustained signal covarying with reward uncertainty.
Part Two broadly covers brain imaging studies on reward and decisionmaking.
There is a current explosion of fMRI studies on reward and decisionmaking.
This area of research encompasses a broad range of issues, such as the
neural substrates of different value-related signals involved when processing and
deciding between rewards of different nature, such as prediction error, uncertainty,
Preface xi
subjective value of each option, goal, and decision-value. This part of the book
also addresses the study of the neural correlates of perceptual decision-making
in humans using non-invasive methods such as fMRI and EEG and investigate
the neural substrates underlying processes described by psychological theories,
such as instrumental versus Pavlovian conditioning, goals, habits, and learning
with different discount factors. Another question addressed in this part of
the book is to know whether similar neural substrates underlie rewards of different
types. In particular, primary (such as drink, food, and sex) and secondary
rewards (such as money, power, …) may both be processed by similar and
distinct neural substrates and may also share neurobiological mechanisms subserved
by non-natural (drug) rewards. Finally, the relationships between the two
fields of neuroscience and economics is addressed in a number of fMRI studies
investigating the neural substrates of different cognitive biases demonstrating
that individuals do not operate as optimal decision-makers in maximizing utility
(direct violation of the assumptions of the “standard economic model”). For
example, people generally overestimate the loss they will derive from anticipated
future events and discount future rewards relative to immediate ones.
Part Three of the book focuses on pathophysiology (lesion data and neuroimaging
findings) involving disorders of decision-making and of the reward system
that link together basic research areas, including systems, cognitive, and clinical
neuroscience. Dysfunction of the reward system and decision-making is present
in a number of neurological and psychiatric disorders, such as Parkinson’s disease,
schizophrenia, drug addiction, and focal brain lesions. In particular, the
study of patients with Parkinson’s disease and models of this disease in nonhuman
primates indicate that dopamine depletion impair reward-based learning,
but not punishment avoidance. Moreover, patients with Parkinson’s disease
treated with dopamine receptor agonists, may develop impulsive behavior such
as pathological gambling, compulsive shopping or hypersexuality, providing a
window in understanding how dopamine treatment influences reward mechanisms
leading to impulsivity.
Part Four is devoted to the roles of hormones and different genes involved in
dopamine transmission on the reward system and on decision-making processes.
The combination of molecular genetic, endocrinology and neuroimaging has
provided a considerable amount of data that helps understanding the biological
mechanisms influencing reward processing. These studies have demonstrated
that genetic and hormonal variations affecting dopaminergic transmission have
an impact on the physiological response of the reward system and on its associated
cognitive functions. These variations may account for some of the interand
intra-individual behavioral differences observed in reward processing and
social cognition. These findings are important because they point to the neural
influence of genes conferring vulnerability to develop neuropathologies related
to compulsive behavior.
Finally, Part Five presents computational models on decision-making, including
Bayesian models of decision-making, models of decision under risk and uncertainty
taking into account the roles of social learning, regret, and disappointments
xii Preface
and models of the basal ganglia. New bridges have recently been made between
animal neurophysiology, human neuroimaging, and behavior and theoretical
models that provide a formal and quantitative account of reward and decisionmaking.
For example, computational approaches have linked learning mechanisms
with the underlying neurophysiology of reinforcements, using temporal
difference learning approaches. Other types of models have shown how decisions
based on uncertain information may benefit from an accumulation of information
over time. These models show how neurons may compute the time integral
of sensory signals as evidence for or against a proposition and propose that the
decision is made when the integrated evidence reaches a certain threshold. These
types of models explain a variety of behavioral and physiological measurements
obtained from monkey electrophysiological data.
We anticipate that while some readers may read the volume from the first to
the last chapter, other readers may read only one or more chapters at a time, and
not necessarily in the order presented in this book. This is why we encouraged an
organization of this volume whereby each chapter can stand alone, while making
references to others and minimizing redundancies across the volume. Given the
consistent acceleration of advances in the different approaches described in this
book on reward and decision-making, you are about to be dazzled by a first-look
at the new stages of an exciting era in brain research. Enjoy!
Dr Jean-Claude Dreher
CNRS
Reward and decision making team
Cognitive Neuroscience Center
67 Bd Pinel 69675 Bron
France