Probability space

In probability theory, a probability space or a probability triple \((\Omega, \mathcal{F}, P)\) is a mathematical construct that provides a formal model of a random process or experiment.

A probability space consists of three elements:

1. A sample space, \(\Omega\), which is the set of all possible outcomes of a random process under consideration.
2. An event space, \(\mathcal{F}\), which is a set of events, where an event is a subset of outcomes in the sample space.
3. A probability function, \(P\), which assigns, to each event in the event space, a probability, which is a number between 0 and 1 (inclusive).

In order to provide a model of probability, these elements must satisfy probability axioms.

For example, one can define a probability space which models the throwing of a dice:

1. The sample space \(\Omega\) is typically the set \(\{1, 2, 3, 4, 5, 6\}\) where each element in the set is a label which represents the outcome of the die landing on that label. For example, \(1\) represents the outcome that the die lands on 1.
2. The event space \(\mathcal{F}\) could be the set of all subsets of the sample space, which would then contain simple events such as \(\{5\}\) ("the die lands on 5"), as well as complex events such as \(\{2, 4, 6\}\) ("the die lands on an even number").
3. The probability function \(P\) would then map each event to the number of outcomes in that event divided by 6, so for example, \(\{5\}\) would be mapped to \(1/6\), and \(\{2, 4, 6\}\) would be mapped to \(3/6 = 1/2\).

When an experiment is conducted, it results in exactly one outcome \(\omega\) from the sample space \(\Omega\). All the events in the event space \(\mathcal{F}\) that contain the selected outcome \(\omega\) are said to "have occurred". The probability function \(P\) must be so defined that if the experiment were repeated arbitrarily many times, the number of occurrences of each event as a fraction of the total number of experiments, will most likely tend towards the probability assigned to that event.

The Soviet mathematician Andrey Kolmogorov introduced the notion of a probability space and the axioms of probability in the 1930s. In modern probability theory, there are alternative approaches for axiomatization, such as the algebra of random variables.

A probability space is a mathematical triplet \((\Omega, \mathcal{F}, P)\) that presents a model for a particular class of real-world situations. As with other models, its author ultimately defines which elements \(\Omega\), \(\mathcal{F}\), and \(P\) will contain.

• The sample space \(\Omega\) is the set of all possible outcomes. An outcome is the result of a single execution of the model. Outcomes may be states of nature, possibilities, experimental results and the like. Every instance of the real-world situation (or run of the experiment) must produce exactly one outcome. If outcomes of different runs of an experiment differ in any way that matters, they are distinct outcomes. Which differences matter depends on the kind of analysis we want to do. This leads to different choices of sample space.
• The σ-algebra \(\mathcal{F}\) is a collection of all the events we would like to consider. This collection may or may not include each of the elementary events. Here, an "event" is a set of zero or more outcomes; that is, a subset of the sample space. An event is considered to have "happened" during an experiment when the outcome of the latter is an element of the event. Since the same outcome may be a member of many events, it is possible for many events to have happened given a single outcome. For example, when the trial consists of throwing two dice, the set of all outcomes with a sum of 7 pips may constitute an event, whereas outcomes with an odd number of pips may constitute another event. If the outcome is the element of the elementary event of two pips on the first die and five on the second, then both of the events, "7 pips" and "odd number of pips", are said to have happened.
• The probability measure \(P\) is a set function returning an event's probability. A probability is a real number between zero (impossible events have probability zero, though probability-zero events are not necessarily impossible) and one (the event happens almost surely, with almost total certainty). Thus \(P\) is a function \(P : \mathcal{F} \to [0,1].\) The probability measure function must satisfy two simple requirements: First, the probability of a countable union of mutually exclusive events must be equal to the countable sum of the probabilities of each of these events. For example, the probability of the union of the mutually exclusive events \(\text{Head}\) and \(\text{Tail}\) in the random experiment of one coin toss, \(P(\text{Head}\cup\text{Tail})\), is the sum of probability for \(\text{Head}\) and the probability for \(\text{Tail}\), \(P(\text{Head}) + P(\text{Tail})\). Second, the probability of the sample space \(\Omega\) must be equal to 1 (which accounts for the fact that, given an execution of the model, some outcome must occur). In the previous example the probability of the set of outcomes \(P(\{\text{Head},\text{Tail}\})\) must be equal to one, because it is entirely certain that the outcome will be either \(\text{Head}\) or \(\text{Tail}\) (the model neglects any other possibility) in a single coin toss.

Condensed: the full section is in Wikipedia.

In short, a probability space is a measure space such that the measure of the whole space is equal to one.

The expanded definition is the following: a probability space is a triple \((\Omega,\mathcal{F},P)\) consisting of:

• the sample space \(\Omega\), an arbitrary non-empty set,
• the σ-algebra \(\mathcal{F} \subseteq 2^\Omega\) (also called σ-field), a set of subsets of \(\Omega\), called events, such that:
  • \(\mathcal{F}\) contains the sample space: \(\Omega \in \mathcal{F}\),
  • \(\mathcal{F}\) is closed under complements: if \(A\in\mathcal{F}\), then also \((\Omega\setminus A)\in\mathcal{F}\),
  • \(\mathcal{F}\) is closed under countable unions: if \(A_i\in\mathcal{F}\) for \(i=1,2,\dots\), then also \((\bigcup_{i=1}^\infty A_i)\in\mathcal{F}\)
    • The corollary from the previous two properties and De Morgan's law is that \(\mathcal{F}\) is also closed under countable intersections: if \(A_i\in\mathcal{F}\) for \(i = 1,2,\dots\), then also \((\bigcap_{i=1}^\infty A_i)\in\mathcal{F}\)
• the probability measure \(P:\mathcal{F}\to[0,1]\), a function on \(\mathcal{F}\) such that:
  • P is countably additive (also called σ-additive): if \(\{A_i\}_{i=1}^\infty \subseteq \mathcal{F}\) is a countable collection of pairwise disjoint sets, then \(P(\bigcup_{i=1}^\infty A_i)=\sum_{i=1}^\infty P(A_i),\)
  • the measure of the entire sample space is equal to one: \(P(\Omega)=1\).

For a countable sample space \(\Omega\), probabilities can be ascribed to elements of \(\Omega\) by a discrete probability distribution \(p:\Omega\to[0,1]\) such that \(\sum_{\omega\in\Omega} p(\omega)=1,\) whereby all subsets of \(\Omega\) can be treated as events (thus, \(\mathcal{F}=2^\Omega\) is the power set) and the probability measure has the form

The greatest σ-algebra \(\mathcal{F}=2^\Omega\) describes the complete information. In general, a σ-algebra \(\mathcal{F}\subseteq2^\Omega\) corresponds to a finite or countable partition \(\Omega=B_1\cup B_2\cup\dots\), the general form of an event \(A\in\mathcal{F}\) being \(A=B_{k_1}\cup B_{k_2}\cup\dots\).

The case \(p(\omega)=0\) is permitted by the definition, but rarely used, since such \(\omega\) can safely be excluded from the sample space.

If Ω is uncountable, still, it may happen that P(ω) ≠ 0 for some ω; such ω are called atoms. They are an at most countable (maybe empty) set, whose probability is the sum of probabilities of all atoms. If this sum is equal to 1 then all other points can safely be excluded from the sample space, returning us to the discrete case. Otherwise, if the sum of probabilities of all atoms is between 0 and 1, then the probability space decomposes into a discrete (atomic) part (maybe empty) and a non-atomic part.

If P(ω) = 0 for all ω ∈ Ω, then Ω must be uncountable (otherwise the axiom P(Ω) = 1 would be rendered impossible) and equation (1) does not apply: the probability of a set is not necessarily the sum over the probabilities of its elements, as summation is only defined for countable numbers of elements. This makes the probability space theory much more technical. A formulation stronger than summation, measure theory is applicable. Initially the probabilities are ascribed to some "generator" sets (see the examples). Then a limiting procedure allows assigning probabilities to sets that are limits of sequences of generator sets, or limits of limits, and so on. All these sets are the σ-algebra \(\mathcal{F}\). For technical details see Carathéodory's extension theorem. Sets belonging to \(\mathcal{F}\) are called measurable. In general they are much more complicated than generator sets, but much better than non-measurable sets.

A probability space \((\Omega,\; \mathcal{F},\; P)\) is said to be a complete probability space if for all \(B \in \mathcal{F}\) with \(P(B) = 0\) and all \(A\; \subset \;B\) one has \(A \in \mathcal{F}\). Often, the study of probability spaces is restricted to complete probability spaces.

A random variable X is a measurable function X: Ω → S from the sample space Ω to another measurable space S called the state space.

If A ⊂ S, the notation Pr(X ∈ A) is a commonly used shorthand for \(P(\{\omega \in \Omega: X(\omega) \in A\})\).

If Ω is countable, we almost always define \(\mathcal{F}\) as the power set of Ω, i.e. \(\mathcal{F} = 2^\Omega\) which is trivially a σ-algebra and the biggest one we can create using Ω. We can therefore omit \(\mathcal{F}\) and just write (Ω,P) to define the probability space.

On the other hand, if Ω is uncountable and we use \(\mathcal{F} = 2^\Omega\) we get into trouble defining our probability measure P because \(\mathcal{F}\) is too "large", i.e. there will often be sets to which it will be impossible to assign a unique measure. In this case, we have to use a smaller σ-algebra \(\mathcal{F}\), for example the Borel algebra of Ω, which is the smallest σ-algebra that makes all open sets measurable.

Kolmogorov's definition of probability spaces gives rise to the natural concept of conditional probability. Every set A with non-zero probability (that is, P(A) > 0) defines another probability measure \[P(B \mid A) = {P(B \cap A) \over P(A)}\] on the space. This is usually pronounced as the "probability of B given A".

For any event A such that P(A) > 0, the function Q defined by Q(B) = P(B | A) for all events B is itself a probability measure.

Two events, A and B are said to be independent if P(A ∩ B) = P(A) P(B).

Two random variables, X and Y, are said to be independent if any event defined in terms of X is independent of any event defined in terms of Y. Formally, they generate independent σ-algebras, where two σ-algebras G and H, which are subsets of F are said to be independent if any element of G is independent of any element of H.

Two events, A and B are said to be mutually exclusive or disjoint if the occurrence of one implies the non-occurrence of the other, i.e., their intersection is empty. This is a stronger condition than the probability of their intersection being zero.

If A and B are disjoint events, then P(A ∪ B) = P(A) + P(B). This extends to a (finite or countably infinite) sequence of events. However, the probability of the union of an uncountable set of events is not the sum of their probabilities. For example, if Z is a normally distributed random variable, then P(Z = x) is 0 for any x, but P(Z ∈ R) = 1.

The event A ∩ B is referred to as "A and B", and the event A ∪ B as "A or B".