《马尔科夫过程导论》讲述了：To some extent, it would be accurate to summarize the contents of this book as an intolerably protracted description of what happens when either one raises a transition probability matrix P (i.e., all entries (P)o are nonnegative and each row of P sums to 1) to higher and higher powers or one exponentiates R(P - I), where R is a diagonal matrix with non-negative entries. Indeed, when it comes right down to it, that is all that is done in this book. However, I, and others of my ilk, would take offense at such a dismissive characterization of the theory of Markov chains and processes with values in a countable state space, and a primary goal of mine in writing this book was to convince its readers that our offense would be warranted
Preface.
    Chapter1 RandomWalksAGoodPlacetoBegin
    1.1.NearestNeighborRandomWlalksonZ
    1.1.1.DistributionatTimen
    1.1.2.PassageTimesviatheReflectionPrinciple
    1.1.3.SomeRelatedComputations
    1.1.4.TimeofFirstReturn
    1.1.5.PassageTimesviaFunctionalEquations
    1.2.RecurrencePropertiesofRandomWalks
    1.2.1.RandomWalksonZd
    1.2.2.AnElementaryRecurrenceCriterion
    1.2.3.RecurrenceofSymmetricRandomWalkinZ2
    1.2.4.nansienceinZ3
    1.3.Exercises
   
    Chapter2 Doeblin'STheoryforMarkovChains
    2.1.SomeGeneralities
    2.1.1.ExistenceofMarkovChains
    2.1.2.TransionProbabilities&ProbabilityVectors
    2.1.3.nansitionProbabilitiesandFunctions
    2.1.4.TheMarkovProperty
    2.2.Doeblin'STheory
    2.2.1.Doeblin'SBasicTheorem
    2.2.2.ACoupleofExtensions
    2.3.ElementsofErgodicTheory
    2.3.1.TheMeanErgodicTheorem
    2.3.2.ReturnTimes
    2.3.3.Identificationofπ
    2.4.Exercises
   
    Chapter3 MoreabouttheErgodicTheoryofMarkovChains
    3.1.ClassificationofStates
    3.1.1.Classification,Recurrence,andTransience
    3.1.2.CriteriaforRecurrenceandTransmnge
    3.1.3.Periodicity
    3.2.ErgodicTheorywithoutDoeblin
    3.2.1.ConvergenceofMatrices
    3.2.2.AbelConvergence
    3.2.3.StructureofStationaryDistributions
    3.2.4.ASmallImprovement
    3.2.5.TheMcanErgodicTheoremAgain
    3.2.6.ARefinementinTheAperiodicCase
    3.2.7.PeriodicStructure
    3.3.Exercises
   
    Chapter4 MarkovProcessesinContinuousTime
    4.1.PoissonProcesses
    4.1.1.TheSimplePoissonProcess
    4.1.2.CompoundPoissonProcessesonZ
    4.2.MarkovProcesseswithBoundedRates
    4.2.1.BasicConstruction
    4.2.2.TheMarkovProperty
    4.2.3.TheQ-MatrixandKolmogorov'SBackwardEquation
    4.2.4.Kolmogorov'SForwardEquation
    4.2.5.SolvingKolmogorov'SEquation
    4.2.6.AMarkovProcessfromitsInfinitesimalCharacteristics..
    4.3.UnboundedRates
    4.3.1.Explosion
    4.3.2.CriteriaforNon.explosionorExplosion
    4.3.3.WhattoDoWhenExplosionOccurs
    4.4.ErgodicProperties
    4.4.1.ClassificationofStates
    4.4.2.StationaryMeasuresandLimitTheorems
    4.4.3.Interpretingπii
    4.5.Exercises
   
    Chapter5 ReversibleMarkovProeesses
    5.1.R,eversibleMarkovChains
    5.1.1.ReversibilityfromInvariance
    5.1.2.MeasurementsinQuadraticMean
    5.1.3.TheSpectralGap
    5.1.4.ReversibilityandPeriodicity
    5.1.5.RelationtoConvergenceinVariation
    5.2.DirichletFormsandEstimationofβ
    5.2.1.TheDirichletFormandPoincar4'SInequality
    5.2.2.Estimatingβ+
    5.2.3.Estimatingβ-
    5.3.ReversibleMarkovProcessesinContinuousTime
    5.3.1.CriterionforReversibility
    5.3.2.ConvergenceinL2(π)forBoundedRates
    5.3.3.L2(π)ConvergenceRateinGeneral
    5.3.4.Estimating
    5.4.GibbsStatesandGlauberDynamics
    5.4.1.Formulation
    5.4.2.TheDirichletForm
    5.5.SimulatedAnnealing
    5.5.1.TheAlgorithm
    5.5.2.ConstructionoftheTransitionProbabilities
    5.5.3.DescriptionoftheMarkovProcess
    5.5.4.ChoosingaCoolingSchedule
    5.5.5.SmallImprovements
    5.6.Exercises
   
    Chapter6 SomeMildMeasureTheory
    6.1.ADescriptionofLebesgue'sMeasureTheory
    6.1.1.MeasureSpaces
    6.1.2.SomeConsequencesofCountableAdditivity
    6.1.3.Generatinga-Algebras
    6.1.4.MeasurableFunctions
    6.1.5.LebesgueIntegration
    6.1.6.StabilityPropertiesofLebesgueIntegration
    6.1.7.LebesgueIntegrationinCountableSpaces
    6.1.8.Fubini'sTheorem
    6.2.ModelingProbability
    6.2.1.ModelingInfinitelyManyTossesofaFairCoin
    6.3.IndependentRandomVariables
    6.3.1.ExistenceofLotsofIndependentRandomVariables
    6.4.ConditionalProbabilitiesandExpectations
    6.4.1.ConditioningwithRespecttoRandomVariables
    Notation
    References
    Index
    ……

DJVU格式 很清晰