Lecture 16. C) A More General Example

From Significant Statistics
Jump to navigation Jump to search

A more general example

  • Likelihood: [math]f\left(\left.x\right|p\right)=p^{x}\left(1-p\right)^{1-x}1\left(x\in\left\{ 0,1\right\} \right)[/math], before.
  • Prior Belief: [math]f\left(p\right)=Beta\left(\alpha,\beta\right)=\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\Gamma\left(\beta\right)}p^{\alpha-1}\left(1-p\right)^{\beta-1}[/math]. (note that [math]Beta\left(1,1\right)=U\left(0,1\right).[/math])

Recall that [math]E\left(p\right)=\frac{\alpha}{\alpha+\beta}[/math] and [math]Var\left(p\right)=\frac{\alpha\beta}{\left(\alpha+\beta\right)^{2}\left(1+\alpha+\beta\right)}[/math].

The posterior probability when [math]x=1[/math] is given by:

[math]f\left(\left.p\right|x=1\right)=\frac{\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\Gamma\left(\beta\right)}p^{\alpha-1}\left(1-p\right)^{\beta-1}p}{f_{X}\left(1\right)=\int_{0}^{1}f_{\left.X\right|p}f_{p}dp}[/math]

Notice that [math]f_{X}\left(x\right)[/math] does not depend on the random variable of [math]f\left(\left.p\right|x=1\right)[/math], which is [math]p[/math]. In fact, the denominator is only required to ensure that [math]f\left(\left.p\right|x=1\right)[/math] integrates to one.

The implication is that we can ignore the denominator and focus on the part that does depend on [math]p[/math], the kernel of [math]f_{\left.p\right|X}[/math]. We end up with the following identity:

[math]f\left(\left.p\right|X\right)\propto p^{\alpha}\left(1-p\right)^{\beta-1}[/math]

where [math]\propto[/math] can be read as “is proportional to”, and [math]p^{\alpha}\left(1-p\right)^{\beta-1}[/math] is the kernel.

Inspection of the Kernel implies that [math]f\left(\left.p\right|x=1\right)[/math] is a Beta distribution with parameters [math]\alpha+1[/math] and [math]\beta[/math].

The mean and variance of [math]\left.p\right|x=1[/math] are given by

[math]\begin{aligned} E\left(\left.p\right|x=1\right) & =\frac{\alpha+1}{\alpha+\beta+1}.\\ Var\left(\left.p\right|x=1\right) & =\frac{\left(\alpha+1\right)\beta}{\left(\alpha+\beta+1\right)^{2}\left(\alpha+\beta+2\right)}.\end{aligned}[/math]

It is often easy to identify the distribution by its kernel. In our example, it is clear that

[math]f_{X}\left(1\right)=\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\Gamma\left(\beta\right)}.\frac{\Gamma\left(\alpha+1\right)\Gamma\left(\beta\right)}{\Gamma\left(\alpha+\beta+1\right)}=\frac{\alpha}{\alpha+\beta},[/math]

so that we end up with a Beta distribution that integrates to one.