Lecture 4. F) Gamma
Contents
Gamma Distribution
A r.v. [math]X[/math] follows a Gamma distribution with parameters [math]\alpha,\beta\gt 0[/math] if [math]X[/math] has continuous pdf
[math]f_{X}\left(x\right)=\begin{cases} \frac{1}{\Gamma\left(\alpha\right)\beta^{\alpha}}x^{\alpha-1}\exp\left(-\frac{x}{\beta}\right), & x\gt 0\\ 0, & otherwise \end{cases}[/math]
where [math]\Gamma\left(x\right)[/math] is the gamma function, given by
[math]\Gamma\left(x\right)=\int_{0}^{\infty}t^{\alpha-1}\exp\left(-t\right)dt,\,\alpha\gt 0[/math].
The Gamma function is a natural extension of the factorial operation, because [math]\Gamma\left(\alpha+1\right)=\alpha\Gamma\left(\alpha\right)[/math] and [math]\Gamma\left(1\right)=1[/math], which implies that [math]\Gamma\left(n+1\right)=n!\,\forall n\in\mathbb{N}.[/math] The Gamma distribution is especially useful for Bayesian estimation, which we will cover later.
This is a good time to describe a common property of pdfs. Notice that over its support, function [math]\frac{1}{\Gamma\left(\alpha\right)\beta^{\alpha}}x^{\alpha-1}\exp\left(-\frac{x}{\beta}\right)[/math] has some factors that depend on [math]x[/math], and others that do not:
[math]f_{X}\left(x\right)=\frac{1}{\Gamma\left(\alpha\right)\beta^{\alpha}}x^{\alpha-1}\exp\left(-\frac{x}{\beta}\right)=\underset{normalizing\,constant}{\underbrace{\frac{1}{\Gamma\left(\alpha\right)\beta^{\alpha}}}}.\underset{kernel\,of\,pdf}{\underbrace{x^{\alpha-1}\exp\left(-\frac{x}{\beta}\right)}}.[/math]
The normalizing constant does not depend on [math]x[/math]. It is there simply to make sure that the function integrates to one. From this, we immediately learn that [math]\int_{0}^{\infty}x^{\alpha-1}\exp\left(-\frac{x}{\beta}\right)dx=\Gamma\left(\alpha\right)\beta^{\alpha}.[/math]
Mean
Consider first r.v. [math]\widetilde{X}\sim Gam\left(\alpha,1\right)[/math], with the information that [math]X=\beta\widetilde{X}\sim Gam\left(\alpha,\beta\right)[/math]. Then,
[math]E\left(\widetilde{X}\right)=\int_{0}^{\infty}xf_{X}\left(\left.x\right|\alpha,1\right)dx=\int_{0}^{\infty}x\frac{1}{\Gamma\left(\alpha\right)}x^{\alpha-1}\exp\left(-x\right)dx=\frac{1}{\Gamma\left(\alpha\right)}\int_{0}^{\infty}x^{\alpha}\exp\left(-x\right)dx\frac{\Gamma\left(\alpha+1\right)}{\Gamma\left(\alpha+1\right)}=\frac{\Gamma\left(\alpha+1\right)}{\Gamma\left(\alpha\right)}\underset{\int_{0}^{\infty}f_{\widetilde{X}}\left(\left.x\right|\alpha+1,1\right)dx=1}{\underbrace{\int_{0}^{\infty}\frac{1}{\Gamma\left(\alpha+1\right)}x^{\alpha}\exp\left(-x\right)dx}}=\alpha[/math].
So, [math]E\left(X\right)=E\left(\beta\widetilde{X}\right)=\beta E\left(\widetilde{X}\right)=\alpha\beta[/math].
Variance
[math]Var\left(\widetilde{X}\right)=E\left(\widetilde{X}^{2}\right)-E\left(\widetilde{X}\right)^{2}=...=\alpha,[/math] so [math]Var\left(X\right)=\alpha\beta^{2}[/math].
MGF
[math]M_{X}\left(t\right)=\left(1-\frac{t}{\beta}\right)^{-\alpha},t\lt \beta[/math]
