Lecture 16. E) Normal Distribution

From Significant Statistics
Jump to navigation Jump to search

Example: Normal Distribution

Let [math]\sigma^{2}[/math] be known, and

  • Likelihood: [math]f_{\left.X\right|\mu}=N\left(\mu,1\right)[/math].
  • Prior: [math]f_{\mu}=N\left(0,100\right)[/math].

Focusing on the Kernels, we know that

[math]\begin{aligned} f_{\left.\mu\right|X} & \propto\exp\left(-\frac{1}{2}\left(x-\mu\right)^{2}\right)\exp\left(-\frac{1}{2.100}\mu^{2}\right)\\ & =\exp\left\{ -\frac{1}{2}\left(x^{2}+\mu^{2}-2x\mu+\frac{\mu^{2}}{100}\right)\right\} \\ & \propto\exp\left\{ -\frac{1}{2}\left[\frac{101}{100}\left(\mu^{2}-2\mu\frac{100}{101}x+\left(\frac{100}{101}x\right)^{2}-\left(\frac{100}{101}x\right)^{2}\right)\right]\right\} \\ & \propto\exp\left\{ -\frac{\left(\mu-\frac{100}{101}x^{2}\right)}{2.\frac{100}{101}}\right\} \end{aligned}[/math]

Where the proportional signs keep track of expressions that depend solely on [math]x[/math] being added or removed from the exponential. Notice that we can multiply or divide the pdf by functions of [math]x[/math] at will, because [math]x[/math] does not belong to the kernel (after all, this is a pdf of [math]p[/math]). Instead, [math]x[/math] belongs to the normalizing constant. We already know that the posterior integrates to one, so our goal is to find the right kernel, and we can always adjust the constant later.

The result above implies that

[math]\begin{aligned} f_{\left.\mu\right|X} & \propto\exp\left\{ -\frac{\left(\mu-\frac{100}{101}x^{2}\right)}{2.\frac{100}{101}}\right\} \\ \Downarrow\\ f_{\left.\mu\right|X} & =N\left(\frac{100}{101}x^{2},\frac{100}{101}\right).\end{aligned}[/math]

The fact that we ended up with a normal distribution is a consequence of the fact that the normal distribution is a conjugate prior with a normal likelihood function.