Feb. 17, 11:45, Wei Ji Ma. Optimality and Probabilistic Computation in Visual Categorization

Matthew Botvinick

Princeton University

Hierarchical reinforcement learning and human behavior

Research on human and animal behavior has long emphasized its

hierarchical structure,  according to which tasks are comprised of

subtask sequences, which are themselves built of simple actions. The

hierarchical structure of behavior has also been of enduring interest

within neuroscience, where it has been widely considered to reflect

prefrontal cortical functions. In recent work, we have been

reexamining behavioral hierarchy and its neural substrates from the

point of view of recent developments in computational reinforcement

learning. Specifically, we've been considering at a set of approaches

known collectively as hierarchical reinforcement learning, which

extend the reinforcement learning paradigm by allowing the learning

agent to aggregate actions into reusable subroutines or skills. A

close look at the components of hierarchical reinforcement learning

suggests how they might map onto neural structures, in particular

regions within the dorsolateral and orbital prefrontal cortex. It

also suggests specific ways in which hierarchical reinforcement

learning might provide a complement to existing psychological models

of hierarchically structured behavior. A particularly important

question that hierarchical reinforcement learning brings to the fore

is that of how learning identifies new action routines that are

likely to provide useful building blocks in solving a wide range of

future problems. Here and at many other points, hierarchical

reinforcement learning offers an appealing framework for

investigating the computational and neural underpinnings of

hierarchically structured behavior. In addition to introducing the

theoretical framework, I'll describe a first set of neuroimaging and

behavioral studies, in which we have begun to test specific predictions.