Lecture 12. H) Equivalence Between LRT and Wald Tests
Equivalence Between LRT and Wald Tests
In this case, we expand the log-likelihood function around [math]\widehat{\theta}_{ML}[/math]:
[math]\begin{aligned} l\left(\theta\right) & \simeq l\left(\widehat{\theta}_{ML}\right)+\underset{=0}{\underbrace{l^{'}\left(\widehat{\theta}_{ML}\right)}}\left(\theta-\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta-\widehat{\theta}_{ML}\right)^{2}\\ & =l\left(\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta-\widehat{\theta}_{ML}\right)^{2}\end{aligned}[/math]
Now, plugging this result evaluated at [math]\theta_{0}[/math] into the LRT yields:
[math]\begin{aligned} 2\left[l\left(\widehat{\theta}_{ML}\right)-l\left(\theta_{0}\right)\right] & \simeq2\left[l\left(\widehat{\theta}_{ML}\right)-\left(l\left(\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}\right)\right]\\ & =-l^{''}\left(\widehat{\theta}_{ML}\right)\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}\\ & =\frac{\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}}{-l^{''}\left(\widehat{\theta}_{ML}\right)^{-1}}\end{aligned}[/math]
which yields the Wald test.
Notice that the log-likelihood of the Normal distribution is quadratic in [math]\mu[/math], such that these 3 procedures produce exactly the same test for [math]\mu[/math] (when [math]\sigma^{2}[/math] is known), because we have used quadratic approximations to the log-likelihood function.
Finally, when the null hypothesis is composite, it is usually possible to construct an LM and Wald test.
