Prediction Error and Actor-Critic Hypotheses in the Brain -- Reviewing on-policy / off-policy critic learning in the context of Temporal Differences and Residual Learning -- Reward Function Design in Reinforcement Learning -- Exploration Methods In Sparse Reward Environments -- A Survey on Constraining Policy Updates Using the KL Divergence -- Fisher Information Approximations in Policy Gradient Methods -- Benchmarking the Natural gradient in Policy Gradient Methods and Evolution Strategies -- Information-Loss-Bounded Policy Optimization -- Persistent Homology for Dimensionality Reduction -- Model-free Deep Reinforcement Learning - Algorithms and Applications -- Actor vs Critic -- Bring Color to Deep Q-Networks -- Distributed Methods for Reinforcement Learning -- Model-Based Reinforcement Learning -- Challenges of Model Predictive Control in a Black Box Environment -- Control as Inference?

This book reviews research developments in diverse areas of reinforcement learning such as model-free actor-critic methods, model-based learning and control, information geometry of policy searches, reward design, and exploration in biology and the behavioral sciences. Special emphasis is placed on advanced ideas, algorithms, methods, and applications. The contributed papers gathered here grew out of a lecture course on reinforcement learning held by Prof. Jan Peters in the winter semester 2018/2019 at Technische Universität Darmstadt. The book is intended for reinforcement learning students and researchers with a firm grasp of linear algebra, statistics, and optimization. Nevertheless, all key concepts are introduced in each chapter, making the content self-contained and accessible to a broader audience.