Lecture 12. G) Equivalence Between LRT and LM Tests
Equivalence Between LRT and LM Tests
One can expand the log-likelihood function around [math]\theta_{0}[/math]:
[math]l\left(\theta\right)\simeq l\left(\theta_{0}\right)+l'\left(\theta_{0}\right)\left(\theta-\theta_{0}\right)+\frac{l''\left(\theta_{0}\right)}{2}\left(\theta-\theta_{0}\right)^{2}[/math]
We will show that this by using this approximation, the LRT yields the LM test.
In order to calculate the maximum log-likelihood, we maximize w.r.t. [math]\theta[/math], by use of the first-order condition:
[math]\begin{aligned} foc\left(\theta\right):\,\frac{d}{d\theta}l\left(\theta\right)=0 & \Leftrightarrow l'\left(\theta_{0}\right)+l''\left(\theta_{0}\right)\left(\theta-\theta_{0}\right)=0\\ & \Leftrightarrow\theta^{*}=\theta_{0}-\frac{l'\left(\theta_{0}\right)}{l''\left(\theta_{0}\right)}\end{aligned}[/math]
Such that the maximized log-likelihood is approximately equal to
[math]\begin{aligned} l\left(\theta^{*}\right) & =l\left(\theta_{0}\right)+l'\left(\theta_{0}\right)\left(\theta^{*}-\theta_{0}\right)+\frac{l''\left(\theta_{0}\right)}{2}\left(\theta^{*}-\theta_{0}\right)^{2}\\ & =l\left(\theta_{0}\right)+l'\left(\theta_{0}\right)\left(-\frac{l'\left(\theta_{0}\right)}{l''\left(\theta_{0}\right)}\right)+\frac{l''\left(\theta_{0}\right)}{2}\left(-\frac{l'\left(\theta_{0}\right)}{l''\left(\theta_{0}\right)}\right)^{2}\\ & =l\left(\theta_{0}\right)-\frac{l'\left(\theta_{0}\right)^{2}}{l''\left(\theta_{0}\right)}+\frac{l'\left(\theta_{0}\right)^{2}}{2l''\left(\theta_{0}\right)}\\ & =l\left(\theta_{0}\right)-\frac{l'\left(\theta_{0}\right)^{2}}{2l''\left(\theta_{0}\right)}\end{aligned}[/math]
The [math]LRT[/math] test is given by [math]2\left[l\left(\widehat{\theta}_{ML}\right)-l\left(\theta_{0}\right)\right][/math].
Plugging in our approximation of the maximum log-likelihood yields
[math]\begin{aligned} & 2\left[l\left(\widehat{\theta}_{ML}\right)-l\left(\theta_{0}\right)\right]\\ \simeq & 2\left[l\left(\theta^{*}\right)-l\left(\theta_{0}\right)\right]\\ = & 2\left[l\left(\theta_{0}\right)-\frac{l'\left(\theta_{0}\right)^{2}}{2l''\left(\theta_{0}\right)}-l\left(\theta_{0}\right)\right]\\ = & -\frac{l'\left(\theta_{0}\right)^{2}}{l''\left(\theta_{0}\right)}\end{aligned}.[/math]
which is indeed the LM test.
Notice that using the “2” in the LRT test was useful to establish the approximate equality between the LRT and the LM test.
