Lecture 1. C) Domain of Probability Function
Contents
Domain of the Probability Function
Our choice of domain for the probability function seems to work well. However, mathematicians encountered a difficulty, when they considered the power set of sets with uncountable elements, for example, when determining the probability of a number falling in an interval in the real line. For starters, the power set of the reals seems very complex. There is another, more serious issue: It is possible to prove that if the power set of the reals is used as the domain of [math]P\left(\cdot\right)[/math], one can then partition the real line into an infinite number of sets, each of which can be assigned equal probability. This is not good; now, either each set has strictly positive probability, so that the sum of probabilities is infinity, or each set has zero probability, such that the sum of probabilities equals zero. (If you'd like to know how to create such a problematic partition of sets, see the example in the Math blog InfinityPlusOneMath.)
[math]\sigma[/math]-algebra
In trying to find a convenient domain for the probability function, mathematicians isolated the properties of a set of sets that does not suffer from the paradox mentioned above. This is called a [math]\sigma[/math]-algebra, and it’s the typical domain for probability functions. In case you haven’t heard the term ‘algebra’ in this context before, an algebra is a definition of a set and of the operations that can be applied to it. A [math]\sigma[/math]-algebra [math]\mathcal{B}[/math] w.r.t. [math]S[/math] is a collection of events with the following properties:
- [math]\emptyset\in \mathcal{B}.[/math]
- [math]B\in\mathcal{B}\Rightarrow B^{C}\in\mathcal{B}.[/math]
- [math]B_{1},B_{2},...\in\mathcal{B}\Rightarrow\bigcup_{i=1}^{\infty}B_{i}\in\mathcal{B}.[/math]
Above, notice that even though [math]\emptyset[/math] and [math]B[/math] are sets, we use notation [math]\in[/math] rather than [math]\subseteq[/math], because [math]\mathcal{B}[/math] is a set of sets (i.e., each of its elements is a set).
So, [math]\mathcal{B}[/math] is a [math]\sigma[/math]-algebra if the empty set belongs to it; the complement of each element is also in [math]\mathcal{B}[/math], and if some sets belong to [math]\mathcal{B}[/math], then so does the union of those sets. We won’t be getting into how this definition solves the continuity issues explained above.
For the discrete case, it is easy to write a few [math]\sigma[/math]-algebras down:
- The discrete [math]\sigma[/math]-algebra is given by the power set of [math]S[/math]:
[math]\mathcal{B}=2^{S}=\left\{ \emptyset,\left\{ Heads\right\} ,\left\{ Tails\right\} ,\left\{ Heads,Tails\right\} \right\}.[/math]
- The trivial [math]\sigma[/math]-algebra is given by
[math]\mathcal{B}=\emptyset\cup S=\left\{ \emptyset,\left\{ Heads,Tails\right\} \right\}[/math].
You can verify that these are indeed sigma algebras.
Remark
You may be a little confused now. Notice that when we defined the sample space, we said that [math]Heads[/math] and [math]Tails[/math] were its elements in the typical coin tossing case. However, we did not list the empty set. The reason is that the empty set is not an element of [math]S[/math] in that case. This wouldn't make sense, since [math]S[/math] is not a set of sets, and [math]\emptyset[/math] is a set. In sum, [math]\emptyset[/math] is a subset of [math]S[/math], and it is an element of [math]\mathcal{B}[/math].
Borel [math]\sigma[/math]-algebra
When [math]S[/math] contains uncountable sets, the standard choice for the domain of the probability function is the Borel [math]\sigma[/math]-algebra (we will denote it as [math]\mathcal{B}(R)[/math], but will not define it here). The Borel [math]\sigma[/math]-algebra contains all open intervals in R, closed intervals, half-open, singletons, unions of intervals, etc. So, the Borel [math]\sigma[/math]-algebra allows one to ask questions about the probability of most ‘reasonable’ sets. ‘Unreasonable’ sets are hard to come up with (see Borel Set for more information).
