Lecture 9. F) Factorization Theorem

From Significant Statistics
Jump to navigation Jump to search

Factorization Theorem

As we saw in the example of the Normal distribution, it can be tedious to find a sufficient statistic for a parameter. Luckily, the factorization theorem makes it easy, provided the pmf/pdf of the sample is available:

Let [math]X_{1}..X_{n}[/math] be a random sample from a distribution with pmf/pdf [math]f\left(\left.\cdot\right|\theta\right)[/math], where [math]\theta\in\Theta[/math] is unknown.

A statistic [math]T=T\left(X_{1}..X_{n}\right)[/math] is sufficient for [math]\theta[/math] if and only if there exist functions [math]g\left(\cdot\right)[/math] and [math]h\left(\cdot\right)[/math] s.t.

[math]\Pi_{i=1}^{n}f\left(\left.x_{i}\right|\theta\right)=g\left(\left.T\left(x_{1},...,x_{n}\right)\right|\theta\right).h\left(x_{1},...,x_{n}\right),[/math]

for every [math]\left(x_{1}..x_{n}\right)\in\mathbb{R}^{n}[/math] and every [math]\theta\in\Theta.[/math]

One way to understand the result above, is that if we were to maximize the likelihood above, only the first factor would be relevant, since the second equals a constant that is independent of the parameters.

Example: Uniform

Suppose [math]X_{i}\sim U\left(0,\theta\right)[/math] such that the joint pdf equals

[math]\Pi_{i=1}^{n}f\left(\left.x_{i}\right|\theta\right)=\underset{g\left(\left.x_{\left(n\right)}\right|\theta\right)}{\underbrace{\frac{1}{\theta^{n}}.1\left(x_{\left(n\right)}\leq\theta\right)}}.\underset{h\left(x_{1}..x_{n}\right)}{\underbrace{1\left(x_{\left(1\right)}\geq0\right)}}[/math]

From the factorization theorem, [math]x_{\left(n\right)}[/math] is a sufficient statistic for [math]\theta[/math]. Again, notice that the maximization of the likelihood function w.r.t. [math]\theta[/math] will only depend on [math]x_{\left(n\right)}[/math], since [math]h\left(x_{1}..x_{n}\right)[/math] is a constant that will not affect the estimator.