附件包含本书第一版与第二版的PDF文档。

Partially Observed Markov Decision Processes - Filtering, Learning, and Controlled Sensing 2nd Edition
Author: Vikram Krishnamurthy, Cornell University

Published: June 2025
ISBN: 9781009449434

Covering formulation, algorithms and structural results and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. In light of major advances in machine learning over the past decade, this edition includes a new Part V on inverse reinforcement learning as well as a new chapter on non-parametric Bayesian inference (for Dirichlet processes and Gaussian processes), variational Bayes and conformal prediction.

Links theory to real-world applications in controlled sensing

Consolidates results from across the literature of multiple different disciplines into a centralized resource

Presents the key ideas underpinning Bayesian filtering, POMDPs, reinforcement learning, and inverse reinforcement learning in an accessible way