Lecture 9. F) Factorization Theorem
Factorization Theorem
As we saw in the example of the Normal distribution, it can be tedious to find a sufficient statistic for a parameter. Luckily, the factorization theorem makes it easy, provided the pmf/pdf of the sample is available:
Let [math]X_{1}..X_{n}[/math] be a random sample from a distribution with pmf/pdf [math]f\left(\left.\cdot\right|\theta\right)[/math], where [math]\theta\in\Theta[/math] is unknown.
A statistic [math]T=T\left(X_{1}..X_{n}\right)[/math] is sufficient for [math]\theta[/math] if and only if there exist functions [math]g\left(\cdot\right)[/math] and [math]h\left(\cdot\right)[/math] s.t.
[math]\Pi_{i=1}^{n}f\left(\left.x_{i}\right|\theta\right)=g\left(\left.T\left(x_{1},...,x_{n}\right)\right|\theta\right).h\left(x_{1},...,x_{n}\right),[/math]
for every [math]\left(x_{1}..x_{n}\right)\in\mathbb{R}^{n}[/math] and every [math]\theta\in\Theta.[/math]
One way to understand the result above, is that if we were to maximize the likelihood above, only the first factor would be relevant, since the second equals a constant that is independent of the parameters.
Example: Uniform
Suppose [math]X_{i}\sim U\left(0,\theta\right)[/math] such that the joint pdf equals
[math]\Pi_{i=1}^{n}f\left(\left.x_{i}\right|\theta\right)=\underset{g\left(\left.x_{\left(n\right)}\right|\theta\right)}{\underbrace{\frac{1}{\theta^{n}}.1\left(x_{\left(n\right)}\leq\theta\right)}}.\underset{h\left(x_{1}..x_{n}\right)}{\underbrace{1\left(x_{\left(1\right)}\geq0\right)}}[/math]
From the factorization theorem, [math]x_{\left(n\right)}[/math] is a sufficient statistic for [math]\theta[/math]. Again, notice that the maximization of the likelihood function w.r.t. [math]\theta[/math] will only depend on [math]x_{\left(n\right)}[/math], since [math]h\left(x_{1}..x_{n}\right)[/math] is a constant that will not affect the estimator.
