Markov Chains

Höfundur: J. R. Norris (Útgáfa: 1)
Markov Chains

Kaup valmöguleikar

Markov chains are central to the understanding of random processes. This is not only because they pervade the applications of random processes, but also because one can calculate explicitly many quantities of interest. This textbook, aimed at advanced undergraduate or MSc students with some background in basic probability theory, focuses on Markov chains and quickly develops a coherent and rigorous theory whilst showing also how actually to apply it.

Both discrete-time and continuous-time chains are studied. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and exercises and examples drawn both from theory and practice. It will therefore be an ideal text either for elementary courses on random processes or those that are more oriented towards applications.

Nánar um bókina

Útgefandi
Cambridge University Press
ISBN
9781107385085
Print ISBN
9780521633963
Format
ePub
Útgáfa
1
Höfundar
J. R. Norris
Tungumál
English
Útgefið
1998-07-28
Prent takmörkun á líftíma
10
Prent takmörkun
5
Afritunar takmörkun
5

Kaflar

  • Cover
  • Title
  • Copyright
  • Dedication
  • Preface
  • Introduction
  • 1. Discrete-time Markov chains
  • 1.1 Definition and basic properties
  • 1.2 Class structure
  • 1.3 Hitting times and absorption probabilities
  • 1.4 Strong Markov property
  • 1.5 Recurrence and transience
  • 1.6 Recurrence and transience of random walks
  • 1.7 Invariant distributions
  • 1.8 Convergence to equilibrium
  • 1.9 Time reversal
  • 1.10 Ergodic theorem
  • 1.11 Appendix: recurrence relations
  • 1.12 Appendix: asymptotics for n!
  • 2. Continuous-time Markov chains I
  • 2.1 Q-matrices and their exponentials
  • 2.2 Continuous-time random processes
  • 2.3 Some properties of the exponential distribution
  • 2.4 Poisson processes
  • 2.5 Birth processes
  • 2.6 Jump chain and holding times
  • 2.7 Explosion
  • 2.8 Forward and backward equations
  • 2.9 Non-minimal chains
  • 2.10 Appendix: matrix exponentials
  • 3. Continuous-time Markov chains II
  • 3.1 Basic properties
  • 3.2 Class structure
  • 3.3 Hitting times and absorption probabilities
  • 3.4 Recurrence and transience
  • 3.5 Invariant distributions
  • 3.6 Convergence to equilibrium
  • 3.7 Time reversal
  • 3.8 Ergodic theorem
  • 4. Further theory
  • 4.1 Martingales
  • 4.2 Potential theory
  • 4.3 Electrical networks
  • 4.4 Brownian motion
  • 5. Applications
  • 5.1 Markov chains in biology
  • 5.2 Queues and queueing networks
  • 5.3 Markov chains in resource management
  • 5.4 Markov decision processes
  • 5.5 Markov chain Monte Carlo
  • 6. Appendix: probability and measure
  • 6.1 Countable sets and countable sums
  • 6.2 Basic facts of measure theory
  • 6.3 Probability spaces and expectation
  • 6.4 Monotone convergence and Fubini’s theorem
  • 6.5 Stopping times and the strong Markov property
  • 6.6 Uniqueness of probabilities and independence of σ-algebras
  • Further reading
  • Index