Basic Probability and Statistics

Lecture set II. Probability: Lecture notes and problem sets

Basic Probability

First try to solve the problems in the first section of the lecture notes. If you cannot, then please study the notes before trying again. This material will be discussed in a lecture only if there are specific questions. However, the notions of event sets, probabilities (and the rules for combining them), conditional probabilities, independence and Bayes' theorem (a quick intro and another example) will be used repeatedly.

Distributions, Generating functions, Moments and Cumulants

You are expected to have met the various simple distributions already. The remainder of the material will be discussed briefly in a lecture. The distinction between moments and cumulants, and the various generating functions will crop up repeatedly in the course. Learn them carefully. In case of difficulty, you can consult web pages from courses on ocean engineering and optics. A quick look at these sources will also tell you how widely these concepts are used.

The Central Limit Theorem

The proof of the central limit theorem is the first example we examine of a renormalisation group flow. You can also consult web sources such as (this and this) for a statement and proof of the theorem. It may be fun to watch Java animations such as rolling dice, a pinball machine, and the quincunx. But as you can see when you run them, there is one crucial element of the proof which is missing from the animations. What is it? How could it be animated?

What is a random sequence?

Very pragmatically, you can call a sequence of numbers random if they pass enough tests of "randomness". Is this a circular definition? Read Donald Knuth's The Art of Computer Programming (Vol II, Chap 3) to find out.

Alternative definitions can be given in terms of complexity. See a popular article by G.J.Chaitin in Scientific American (May 1975) called Randomness and Mathematical Proof. Complexity lies outside the course content, but if you are interested you can follow it up in The Computer Journal: special issue on Kolmogorov complexity. As an exercise, read a New York Times essay on random numbers, and criticise it.

Generating random numbers

Knuth's book, The Art of Computer Programming (Vol II, Chap 3) has the classic algorithms for generating pseudo-random numbers on a computer. More modern algorithms are given in Press et al's book Numerical Recipes. Other sources are given below.

Mersenne Twister: this random number generator is recommended for later work in the course. It is available for download in both C and FORTRAN versions.
Random numbers: industry standard
Random numbers: links
The definitive bibliography

General References

Probability and its Engineering Uses by T.C. Fry, Van Nostrand, 1928. This is a carefully written book which contains all the basic material, with lots of worked out examples and problems. Available in the library.
Probability and Statistics by M.H. DeGroot, Addison Wesley, 1986. A very well written book which starts from absolutely basic material and progresses to a point which includes most things anybody might need. Lots of examples and problems.
Introduction to Probability Models by S.M. Ross, Academic Press, 1989. Lots of examples and problems make this an useful book. However, the text is not suited to self-instruction and the terminology is non-standard.
Random Processes by M. Rosenblatt, Oxford University Press, 1962.
Probability and Measure by P. Billingsley, John Wiley and Sons, 1986.

Stat Mech 2002

Course

Basic Probability and Statistics