Adaptive exploration in reinforcement learning

Thumbnail Image
Patrascu, Relu-Eugen
Journal Title
Journal ISSN
Volume Title
University of Guelph

Reinforcement learning is a computational approach to learning from direct interaction with the environment. In order to ensure survival, an agent must adapt to new situations and challenges. Using trial and error the agent learns which actions lead to higher returns in the long run and exploits the accumulated knowledge to behave optimally. The trade-off between exploration and exploitation is the principal subject of this document. A simple yet powerful novel technique named past success directed exploration is introduced and investigated, together with its implications on the implementation of reinforcement algorithms. In particular, the Sarsa algorithm is implemented in both a traditional connectionist way, based on a backpropagation neural network, and a novel way, based on the fuzzy ARTMAP neural architecture. A number of experiments demonstrating advantages and better performance of the fuzzy ARTMAP implementation of Sarsa and its augmentation with the new exploration method are described.

reinforcement learning, adaptive exploration, past success directed exploration, fuzzy ARTMAP, neural architecture