Mathematical Foundations of Reinforcement Learning
Authors: Shiyu Zhao

DOI: https://doi.org/10.1007/978-981-97-3944-8

ISBN: 978-981-97-3943-1

Published: 22 January 2025

This book provides a mathematical yet accessible introduction to the fundamental concepts, core challenges, and classic reinforcement learning algorithms. It aims to help readers understand the theoretical foundations of algorithms, providing insights into their design and functionality. Numerous illustrative examples are included throughout. The mathematical content is carefully structured to ensure readability and approachability. The book is divided into two parts. The first part is on the mathematical foundations of reinforcement learning, covering topics such as the Bellman equation, Bellman optimality equation, and stochastic approximation. The second part explicates reinforcement learning algorithms, including value iteration and policy iteration, Monte Carlo methods, temporal-difference methods, value function methods, policy gradient methods, and actor-critic methods. With its comprehensive scope, the book will appeal to undergraduate and graduate students, post-doctoral researchers, lecturers, industrial researchers, and anyone interested in reinforcement learning.

Shiyu Zhao is currently an Associate Professor and Director of the Intelligent Unmanned Systems Laboratory in the School of Engineering at Westlake University, Hangzhou, China. He received his Ph.D. degree in Electrical and Computer Engineering from the National University of Singapore in 2014. Before joining Westlake University in 2019, he was a Lecturer in the Department of Automatic Control and Systems Engineering at the University of Sheffield, UK. His primary research interest lies in decision-making and sensing of multi-robot systems.