Contents
Table of Contents i
Schedules iv
1 Definitions, basic properties, the transition matrix
2 Calculation of n-step transition probabilities, class structure, absorption, and irreducibility
3 Hitting probabilities and mean hitting times
4 Survival probability for birth and death chains, stopping times and strong Markov property
5 Recurrence and transience
6 Random walks in dimensions one, two and three
7 Invariant distributions 25 7.1 Examples of invariant distributions
8 Existence and uniqueness of invariant distribution, mean return time, positive and null recurrence
9 Convergence to equilibrium for ergodic chains
10 Long-run proportion of time spent in given state
11 Time reversal, detailed balance, reversibility, random walk on a graph
12 Concluding problems and recommendations for further study