Lecture 13. E) p-value
p-value
Consider the test [math]H_{0}:\mu=\mu_{0}[/math] vs. [math]H_{1}:\mu\lt \mu_{0}[/math], that rejects [math]H_{0}[/math] if [math]\overline{x}\lt c.[/math]
Up to now, we have decided to reject [math]H_{0}[/math] by choosing [math]c[/math] to satisfy a given probability of type 1 error. Specifically, we have often selected
[math]c:\,P_{\mu_{0}}\left(\overline{X}\lt c\right)=.05.[/math]
Let's define critical value [math]c[/math] above that establishes a type 1 error probability of 5% as [math]c_{0.05}.[/math]
Now, suppose we replaced the generic critical value [math]c[/math] by the realized statistic in the data, [math]\overline{x}[/math]. In this case, the probability of committing a type 1 error when [math]c=\overline{x}[/math] can be written as
[math]P_{\mu_{0}}\left(\overline{X}\lt \overline{x}\right)[/math]
Why would we want to replace the critical value with the realization of the test statistic?
Consider what happens, for example, if we plugged in the realized value of [math]\overline{x}[/math], and obtained [math]P_{\mu_{0}}\left(\overline{X}\lt \overline{x}\right)=0.03.[/math] Clearly, this means that [math]\overline{x}\lt c_{0.05}[/math], and we should reject the null hypothesis. This is equivalent to comparing [math]P_{\mu_{0}}\left(\overline{X}\lt \overline{x}\right)=0.03[/math] to [math]P_{\mu_{0}}\left(\overline{X}\lt c_{0.05}\right)=0.05[/math], and noticing that we reject the null hypothesis as long as [math]P_{\mu_{0}}\left(\overline{X}\lt \overline{x}\right)\lt \alpha=0.05.[/math]
In other words, if the type 1 error probability associated with critical value [math]\overline{x}[/math] falls below the probability threshold [math]\alpha[/math], we reject the null hypothesis.
As you may have suspected, the p-value is a statistic. One advantage of using p-values is that we obtain a quantitative measure of “by how much our hypothesis was rejected.” For example, a p-value of [math]0.01[/math] provides more evidence against the null hypothesis than a p-value of [math]0.04.[/math]
We now provide a working definition of p-value (more formal and general definitions exist, for example for composite [math]H_{0}[/math]).
Definition
A p-value is the type 1 error probability of test statistic [math]T\left(X\right)[/math] with critical value [math]T\left(x\right)[/math], i.e.,
[math]p-value=P_{\theta_{0}}\left(T\left(X\right)\lt T\left(x\right)\right)[/math]
for the case where [math]T\left(X\right)\lt T\left(x\right)[/math] implies that [math]\theta_{0}[/math] is rejected.
Again, consider the test [math]H_{0}:\mu=\mu_{0}[/math] vs. [math]H_{1}:\mu\lt \mu_{0}[/math], that rejects [math]H_{0}[/math] if [math]\overline{x}\lt c.[/math]
We can calculate the p-value as:
[math]\begin{aligned} & P_{\mu_{0}}\left(\overline{X}\lt \overline{x}\right)\\ = & P_{\mu_{0}}\left(\underset{\sim N\left(0,1\right)}{\underbrace{\frac{\overline{X}-\mu}{\frac{\sigma}{\sqrt{n}}}}}\lt \frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}\right)\\ = & \Phi\left(\frac{\overline{x}-\mu_{0}}{\frac{\sigma}{\sqrt{n}}}\right).\end{aligned}[/math]
