University of Wales, Bangor - School of Informatics Mathematics H2 Modules

Lecturer: Dr Neil Rymer

Recommended reference books:
There is no one recommended book for this course.
The following list of books can be used for reference purposes, along with the duplicated notes.
Many of these books are available in the College Library.

• Stochastic Processes, Emanuel Parzen, Holden Day, (QA273.P37).
• A First Course in Stochastic Processes, Samuel Karlin, Academic Press, (QA273.K3215).
• Probability and Random Processes, A Leon-Garcia, Addison Wesley, 1989, ISBN 020112906X, (TK153.L425).
• Introduction to Stochastic Models, Roe Goodman, Benjamin Cummings, 1988, ISBN 0805360115, (QA273.G655).
• Denumerable Markov Chains, J Kemeny, J Snell and A Knapp, Van Nostrand, (QA273.K4).
• Introduction to Stochastic Processes, D Cox and H Miller, Chapman & Hall.
• Introduction to Probability Theory & Applications, VOL 1. Feller, Wiley, (QA273.F37).
• Theory of Markov Processes, Bharucha and Reid, (QA273.B4).
• Probability and its Engineering uses, Fry, (QA273.F7).
• Probability, Random Variables and Stochastic Processes, Papoulis, (QA273.P3).

Aims
To describe and classify discrete Markov chains.
To show how to model systems by Markov chains.
To discuss questions of absorption and recurrence.

Objectives
At the end of the course students should be able to:

• Apply the techniques of matrix manipulation and series expansion necessary to compute transition and recurrence probabilities, stationary distributions and expectations.
• Model situations as discrete Markov processes.
• Classify individual states.

Summary
This course introduces the general principles of Stochastic Processes which are models of systems which evolve with time according to probabilistic laws. Thus we assume a process - be it in Biology, Physics, Engineering or the Social Sciences - to be in one of a set of possible states and assign probabilities to its moving to another state at the next time interval. We further assume that time moves in discrete jumps or generations, and restrict to Markov processes where the probabilities are taken to be independent of time and past events. They only depend on the present situation.
The course describes and classifies the various types of Markov chains and shows how information on future events can be deduced. It provides answers to such questions as: Can I return to this state? Must I return? Will I return at regular intervals? How long on average will it take to return? What final states are possible? What is the probability of reaching a particular state after n generations?

Pre-requisites: Be registered for a Single or Joint Honours degree in Mathematics.

Syllabus

Stochastic and Markov Processess
Representative examples. Transition matrix. n-step transitions. Chapman-Kolmogorov equation. Calculation techniques. Generating functions. Occupation probabilities. Stationary distributions.

Classification of states
Accessible, closed, absorbing. Communicating states. Periodic and recurrent states.

Renewal equation
Time to recurrence. Probability of recurrence. Absorption. Time to absorption. Random walk and ruin problems.

Assessment
2 hour end of semester closed book examination 100%

School of Informatics home page Mathematics home page U. W. Bangor home page Latest modification to this page: 01/07/02