Lecture 6. B) Conditional PMF/PDF
Conditional PMF/PDF
The conditional pmf/pdf of [math]X[/math] given [math]Y=y[/math], [math]f_{\left.X\right|Y}\left(x,y\right)[/math], is given by
[math]f_{\left.X\right|Y}\left(x,y\right)=\frac{f_{X,Y}\left(x,y\right)}{f_{Y}\left(y\right)}.[/math]
If [math]f_{Y}\left(y\right)\gt 0[/math], then all of the properties of pmfs/pdfs apply to the conditional pmf/pdf.
This conditional function is a pmf/pdf in its own right. It adds up to one, and is positive (bounded at 1 in the case of the pmf).
The interpretation is intuitive. If [math]\left(X,Y\right)'[/math] is discrete and [math]f_{Y}\left(y\right)\gt 0[/math], then
[math]f_{\left.X\right|Y}\left(x,y\right)=\frac{f_{X,Y}\left(x,y\right)}{f_{Y}\left(y\right)}=\frac{P\left(X=x,Y=y\right)}{P\left(Y=y\right)}=P\left(\left.X=x\right|Y=y\right)[/math] i.e., [math]f_{\left.X\right|Y}\left(x,y\right)[/math] is the conditional probability of [math]X=x[/math] given that [math]Y=y[/math].
Because the value of a pdf does not correspond to a probability, the interpretation is trickier in the continuous case. If we would like to describe the density of [math]X[/math] when [math]Y=5[/math], for example, then [math]f_{\left.X\right|Y}\left(x,5\right)[/math] gives us the intended object. However, a problem may arise. How can we ask about the pdf [math]f_{\left.X\right|Y}\left(x,5\right)[/math], for example, given that [math]P\left(Y=5\right)=0[/math] when [math]Y[/math] is continuous? A related issue is that one can actually obtain different conditional pdfs [math]f_{\left.X\right|Y}\left(x,y\right)[/math], depending on how they are calculated! In the next section we briefly mention the Borel paradox.
Conditioning on Sets
From the discrete case, above, it follows that in the continuous case we can condition on an interval over [math]y[/math]. For example,
[math]f_{\left.X\right|Y}\left(x,y\in\left[\underline{y},\overline{y}\right]\right)[/math]=[math]\frac{\int_{\underline{y}}^{\overline{y}}f_{X,Y}\left(x,y\right)dy}{P\left(y\in\left[\underline{y},\overline{y}\right]\right)}=\frac{\int_{\underline{y}}^{\overline{y}}f_{X,Y}\left(x,y\right)dy}{\int_{-\infty}^{\infty}\int_{\underline{y}}^{\overline{y}}f_{X,Y}\left(x,y\right)dydx}.[/math]
Borel Paradox
The Borel paradox is that it is possible to construct two distinct but equally valid conditional pdfs, when we condition on a measure zero event (i.e., an event with probability zero, such as a point in the continuous case).
A resolution for this ambiguity is to instead take the limit of a pdf conditional on a set of values with positive probability that converges to the measure zero event as [math]n[/math] increases.
For example, let [math]\overline{y}_{n}^{1}\gt y\gt \underline{y}_{n}^{1}[/math] and [math]\overline{y}_{n}^{2}\gt y\gt \underline{y}_{n}^{2}[/math] be different sequences of upper and lower bounds, such that
[math]\lim_{n\rightarrow\infty}\overline{y}_{n}^{1}=\lim_{n\rightarrow\infty}\underline{y}_{n}^{1}=\lim_{n\rightarrow\infty}\overline{y}_{n}^{2}=\lim_{n\rightarrow\infty}\underline{y}_{n}^{2}=y[/math]
and
[math]P\left(y\in\left[\underline{y}_{n}^{1},\overline{y}_{n}^{1}\right]\right)\gt 0[/math] and [math]P\left(y\in\left[\underline{y}_{n}^{2},\overline{y}_{n}^{2}\right]\right)\gt 0\,\forall n\in\mathbb{N}.[/math]
Then, one could calculate
[math]\lim_{n\rightarrow \infty}f_{\left.X\right|Y}\left(x,y\in\left[\underline{y}_{n}^{1},\overline{y}_{n}^{1}\right]\right)[/math]
and
[math]\lim_{n\rightarrow n\rightarrow \infty}f_{\left.X\right|Y}\left(x,y\in\left[\underline{y}_{n}^{2},\overline{y}_{n}^{2}\right]\right).[/math]
Although this procedure can yield two different results, each limit can only yield one conditional probability. In practice, this issue is often ignored or assumed away.
For more information on the Borel paradox, see Proschan and Presnell (1998), this Wikipedia page, and this blog post for an intuitive explanation.
