maths.free › Probability Theory › 10. Probability › Central Tendency

Central Tendency

Consider the following two situations: Situation 1. A small town decides to hold a lottery to raise funds for charitable purposes.

Central Tendency

Consider the following two situations:

Situation 1. A small town decides to hold a lottery to raise funds for charitable purposes. A total of \(10,001\) tickets are sold, and the tickets are labeled with numbers from the set \(\{0,1,2,\dots,10,000\}\). At a public ceremony, duplicate tickets are placed in a big box, and the mayor draws the winning ticket from out of the box. Just to heighten the suspense as to who has actually won the prize, the mayor reports that the winning number is at least \(7,500\). The citizens ooh and aah and they can't wait to see who among them will be the final winner.
Situation 2. Behind a curtain, a fair coin is tossed \(10,000\) times, and the number of heads is recorded by an observer, who is reputed to be honest and impartial. Again, the outcome is an integer in the set \(\{0,1,2,\dots,10,000\}\). The observer then emerges from behind the curtain and announces that the number of heads is at least than \(7,500\). There is a pause and then someone says What? Are you out of your mind?

So we have two probability spaces, both with sample space \(S=\{0,1,2,\dots,10,000\}\). For each, we have a random variable \(X\), the winning ticket number in the first situation, and the number of heads in the second. In each case, the expected value, \(E(X)\), of the random variable \(X\) is \(5,000\). In the first case, we are not all that surprised at an outcome far from the expected value, while in the second, it seems intuitively clear that this is an extraordinary occurrence. The mathematical concept here is referred to as central tendency, and it helps us to understand just how likely a random variable is to stray from its expected value.

For starters, we have the following elementary result.

To make Markov's inequality more concrete, we see that on the basis of this trivial result, the probability that either the winning lottery ticket or the number of heads is at least \(7,500\) is at most \(5000/7500=2/3\). So nothing alarming here in either case. Since we still feel that the two cases are quite different, a more subtle measure will be required.

Variance and Standard Deviation

Again, let \((S,P)\) be a probability space and let \(X\) be a random variable. The quantity \(E((X-E(X))^2)\) is called the variance of \(X\) and is denoted \(\var(X)\). Evidently, the variance of \(X\) is a non-negative number. The standard deviation of \(X\), denoted \(\sigma_X\) is then defined as the quantity \(\sqrt{\var(x)}\), , \(\sigma_X^2 =\var(X)\).

Example

For the spinner shown at the beginning of the chapter, let \(X(i)=i^2\) when the pointer stops in region\(i\). Then we have already noted that the expectation \(E(X)\) of the random variable \(X\) is \(109/8\). It follows that the variance \(\var(X)\) is: \[\begin{aligned}\var(X) =\amp(1^2-\frac{109}{8})^2\frac{1}{8}+(2^2-\frac{109}{8})^2\frac{1}{4}+ (3^2-\frac{109}{8})^2\frac{1}{8}+(4^2-\frac{109}{8})^2\frac{1}{8} \\ \amp+ (5^2-\frac{109}{8})^2\frac{3}{8} \\ =\amp (108^2+105^2+100^2+93^2+84^2)/512 \\ =\amp 48394/512\end{aligned}\] It follows that the standard deviation \(\sigma_X\) of \(X\) is then \(\sqrt{48394/512}\approx 9.722\).

Variance (and standard deviation) are quite useful tools in discussions of just how likely a random variable is to be near its expected value. This is reflected in the following theorem.

Theorem: Chebyshev's Inequality

Let \(X\) be a random variable in a probability space \((S,P)\), and let \(k>0\) be a positive real number. If the expectation \(E(X)\) of \(X\) is \(\mu\) and the standard deviation is \(\sigma_X\), then \[\begin{aligned}\end{aligned}\]

Proof

Let \(A=\{r\in \reals:|r-\mu|>k\sigma_X\}\).

Then we have: \[\begin{aligned}\var(X) \amp = E((X-\mu)^2) \\ \amp = \sum_{r\in \reals}(r-\mu)^2\prob(X=r) \\ \amp \ge \sum_{r\in A}(r-\mu)^2\prob(X=r) \\ \amp \ge k^2\sigma_X^2\sum_{r\in A}\prob(X=r) \\ \amp \ge k^2\sigma_X^2\prob(|X-\mu|>k\sigma_X).\end{aligned}\] Since \(\var(X)=\sigma_X^2\), we may now deduce that \(1/k^2\geq \prob(|X-\mu|)>k\sigma_X)\). Therefore, since \(\prob(|X-\mu|\le k\sigma_X)=1-\prob(|X-\mu|> k\sigma_X)\), we conclude that \[\begin{aligned}\end{aligned}\]

Condensed — the full section is in Keller & Trotter, Applied Combinatorics.

Symbols used here

\sum_{k=1}^{n} a_k

summation
Add a_k for k = 1 up to n.

\sqrt{x},\ \sqrt[n]{x}

square root, n-th root
The non-negative number whose square (n-th power) is x.

x \in A,\ A \subseteq B

element of, subset
x belongs to A; every element of A is in B.

\sigma,\ s,\ \sigma^2

standard deviation, sample s.d., variance
Typical distance from the mean; its square.

\bar{x},\ \mu

sample mean, population mean
Average of the data; average of the whole population.

P(A),\ P(A \mid B)

probability, conditional probability
Chance of A; chance of A given that B happened.

imaginary unit
i² = −1.

\approx

approximately equal
Equal to the precision shown, not exactly.

\leq,\ \geq

less/greater than or equal
Inequalities that allow equality; < and > exclude it.

factorial
n × (n−1) × … × 1; the number of orderings of n things. 0! = 1.

\binom{n}{k}

binomial coefficient, "n choose k"
Number of k-element subsets of n things: n!/(k!(n−k)!).

\int f(x)\,dx,\ \int_a^b

integral
Antiderivative (indefinite) or signed area from a to b (definite).

A \cup B,\ A \cap B,\ A \setminus B

union, intersection, difference
In either; in both; in A but not B.

E[X],\ \operatorname{Var}(X)

expected value, variance
Probability-weighted average of X; its spread.

N(\mu, \sigma^2),\ z

normal distribution, z-score
The bell curve with mean μ and variance σ²; (x − μ)/σ.

\mu(A),\ \sigma\text{-algebra}

measure of A
Size of a set; the family of sets that can be measured.

Questions people ask

What is the difference between probability and statistics?

Probability goes from a known model to what the data should look like; statistics goes from data back to the model. Probability theory is the deductive half.

What does the law of large numbers promise?

That the average of many independent samples converges to the expected value. It says nothing about any single trial.

మీ సొంత ప్రయత్నించండి

Parts of this page are adapted from Keller & Trotter, Applied Combinatorics (CC BY-SA 4.0). Condensed and re-explained here; errors are ours.

ఇంకా Probability Theory

Sample spaces and the axioms Random variables and expectation The common distributions The law of large numbers and the central limit theorem