Forest ErrAna StatDist

Average and Variance

Average

The word "average" is used to describe a property of a probability distribution or a set of observations/measurements made in an experiment which gives an indication of a likely outcome of an experiment.

The symbol

[math]\mu[/math]

is usually used to represent the average.

Other notations are

[math]\bar{x}[/math]

Definition of the average

[math]\mu \equiv \lim_{N\rightarrow \infty} \frac{\sum x_i}{N}[/math]

Here the above average of a parent distribution is defined as being equal to taking an adding up infinite number of observations (x_i) of an observable x and dividing by the observations. This definition uses the assumption the results of experiment asymptotically approach the average of the parent distribution.

Variance

The word "variance" is used to describe a property of a probability distribution or a set of observations/measurements made in an experiment which gives an indication how much an observation will deviate from and average value.

A deviation [math](d_i)[/math] of any measurement [math](x_i)[/math] from a parent distribution with a mean [math]\mu[/math] can be defined as

[math]d_i\equiv x_i - \mu[/math]

the deviations should average to ZERO for an infinite number of observations by definition of the mean.

Definition of the average

[math]\mu \equiv \lim_{N\rightarrow \infty} \frac{\sum x_i}{N}[/math]

[math]\lim_{N\rightarrow \infty} \frac{\sum (x_i - \mu)}{N}[/math]

[math]= \left ( \lim_{N\rightarrow \infty} \frac{\sum (x_i }{N}\right ) - \mu[/math]

[math]= \left ( \lim_{N\rightarrow \infty} \frac{\sum (x_i }{N}\right ) - \lim_{N\rightarrow \infty} \frac{\sum x_i}{N} = 0[/math]

But the AVERAGE DEVIATION [math](\bar{d})[/math] is given by an average of the magnitude of the deviations given by

[math]\bar{d} = \lim_{N\rightarrow \infty} \frac{\sum \left | (x_i - \mu)\right |}{N}[/math] = a measure of the dispersion of the expected observations about the mean

Taking the absolute value though is cumbersome when performing a statistical analysis so one may express this dispersion in terms of the variance

A typical variable used to denote the variance is

[math]\sigma^2[/math]

and is defined as

[math]\sigma^2 = \lim_{N\rightarrow \infty}\left [ \frac{\sum (x_i-\mu)^2 }{N}\right ][/math]

Standard Deviation

The standard deviation is defined as the square root of the variance

S.D. = [math]\sqrt{\sigma}[/math]

The mean should be thought of as a parameter which characterizes the observations we are making in an experiment. In general the mean specifies the probability distribution that is representative of the observable we are trying to measure through experimentation.

The variance characterizes the uncertainty associated with our experimental attempts to determine the "true" value. Although the mean and true value may not be equal, their difference should be less than the uncertainty given by the governing probability distribution.

Average for an unknown probability distribution (parent population)

If the "Parent Population" is not known, you are just given a list of numbers with no indication of the probability distribution that they were drawn from, then the average and variance may be calculate as shown below.

Arithmetic Mean and variance

If [math]n[/math] observables are mode in an experiment then the arithmetic mean of those observables is defined as

[math]\bar{x} = \frac{\sum_{i=1}^{i=N} x_i}{N}[/math]

The "unbiased" variance of the above sample is defined as

[math]s^2 = \frac{\sum_{i=1}^{i=N} (x_i - \bar{x})^2}{N-1}[/math]

If you were told that the average is [math]\bar{x}[/math] then you can calculate the

"true" variance of the above sample as

[math]\sigma^2 = \frac{\sum_{i=1}^{i=N} (x_i - \bar{x})^2}{N}[/math]

Weighted Mean and variance

If each observable ([math]x_i[/math]) is accompanied by an estimate of the uncertainty in that observable ([math]\delta x_i[/math]) then weighted mean is defined as

[math]\bar{x} = \frac{ \sum_{i=1}^{i=n} \frac{x_i}{\delta x_i}}{\sum_{i=1}^{i=n} \frac{1}{\delta x_i}}[/math]

The variance of the distribution is defined as

[math]\bar{x} = \sum_{i=1}^{i=n} \frac{1}{\delta x_i}[/math]

Probability Distributions

Mean and variance

Mean of Discrete Probability Distribution

In the case that you know the probability distribution you can calculate the mean and standard deviation as

For a Discrete probability distribution

[math]\mu = \lim_{N \rightarrow \infty} \frac{\sum_{i=1}^n x_i N P(x_i)}{N}[/math]

where

[math]N=[/math] number of observations

[math]n=[/math] number of different possible observable variables

[math]x_i =[/math] ith observable quantity

[math]P(x_i) =[/math] probability of observing [math]x_i[/math] = Probability Mass Distribution

for a continuous probability distribution

Expectation Value

The average of a sample drawn from any probability distribution is defined in terms of the expectation value E(x) such that

The expectation value for a discrete probability distribution is given by

[math]E(x) = \sum_i x_i P(x_i)[/math]

The expectation value for a continuous probability distribution is calculated as

[math]E(x) = \int x dP(x)[/math]

Binomial Distribution

Binomial random variable describes experiments in which the outcome has only 2 possibilities. The two possible outcomes can be labeled as "success" or "failure". The probabilities may be defined as

p

the probability of a success

and

q

the probability of a failure.

If we let [math]X[/math] represent the number of successes after repeating the experiment [math]n[/math] times

Experiments with [math]n=1[/math] are also known as Bernoulli trails.

Then [math]X[/math] is the Binomial random variable with parameters [math]n[/math] and [math]p[/math].

The number of ways in which the [math]x[/math] successful outcomes can be organized in [math]n[/math] repeated trials is

[math]\frac{n !}{ \left [ (n-x) ! x !\right ]}[/math] where the [math] ![/math] denotes a factorial such that [math]5! = 5\times4\times3\times2\times1[/math].

The expression is known as the binomial coefficient and is represented as

[math]{n\choose x}=\frac{n!}{x!(n-x)!}[/math]

The probability of any one ordering of the success and failures is given by

[math]P( \mbox{experimental ordering}) = p^{x}q^{n-x}[/math]

This means the probability of getting exactly k successes after n trials is

[math]P(x=k) = {n\choose x}p^{x}q^{n-x} [/math]

It can be shown that the Expectation Value of the distribution is

[math]\mu = n p[/math]

and the variance is

[math]\sigma^2 = npq[/math]

Examples

The number of times a coin toss is heads.

The probability of a coin landing with the head of the coin facing up is

[math]P = \frac{\mbox{number of desired outcomes}}{\mbox{number of possible outcomes}} = \frac{1}{2}[/math]

Suppose you toss a coin 4 times. Here are the possible outcomes

order Number	Trial #				# of Heads
	1	2	3	4
1	t	t	t	t	0
2	h	t	t	t	1
3	t	h	t	t	1
4	t	t	h	t	1
5	t	t	t	h	1
6	h	h	t	t	2
7	h	t	h	t	2
8	h	t	t	h	2
9	t	h	h	t	2
10	t	h	t	h	2
11	t	t	h	h	2
12	t	h	h	h	3
13	h	t	h	h	3
14	h	h	t	h	3
15	h	h	h	t	3
16	h	h	h	h	4

The probability of order #1 happening is

P( order #1) = [math]\left ( \frac{1}{2} \right )^0\left ( \frac{1}{2} \right )^4 = \frac{1}{16}[/math]

P( order #2) = [math]\left ( \frac{1}{2} \right )^1\left ( \frac{1}{2} \right )^3 = \frac{1}{16}[/math]

The probability of observing the coin land on heads 3 times out of 4 trials is.

[math]P(x=3) = \frac{4}{16} = \frac{1}{4} = {n\choose x}p^{x}q^{n-x} = \frac{4 !}{ \left [ (4-3) ! 3 !\right ]} \left ( \frac{1}{2}\right )^{3}\left ( \frac{1}{2}\right )^{4-3} = \frac{24}{1 \times 6} \frac{1}{16} = \frac{1}{4}[/math]

Count number of times a 6 is observed when rolling a die

p=1/6

Expectation value :

The expected (average) value from a single roll of the dice is

[math]E({\rm Roll\ With\ 6\ Sided\ Die}) =\sum_i x_i P(x_i) = 1 \left ( \frac{1}{6} \right) + 2\left ( \frac{1}{6} \right)+ 3\left ( \frac{1}{6} \right)+ 4\left ( \frac{1}{6} \right)+ 5\left ( \frac{1}{6} \right)+ 6\left ( \frac{1}{6} \right)=\frac{1 + 2 + 3 + 4 + 5 + 6}{6} = 3.5[/math]

Poisson Distribution

[math]P(x) = \frac{\left ( \lambda s \right)^x e^{-\lambda s}}{x!}[/math]

where

[math]\lambda[/math] = probability for the occurrence of an event per unit interval [math]s[/math]

Homework Problem (Bevington pg 38)

Derive the Poisson distribution assuming a small sample size

1.) Assume that the average rate of an event is constant over a given time interval and that the events are randomly distributed over that time interval.

2.) The probability of NO events occuring over the time interval t is exponential such that

[math]P(0,t,\tau) = \exp^{-t/\tau}[/math]

where \tau is a constant of proportionality associated with the mean time

the change in the probability as a function of time is given by

[math]dP(0,t,\tau) = - P(0,t,\tau) \frac{dt}{\tau}[/math]

t-Distributioon

[1] Forest_Error_Analysis_for_the_Physical_Sciences