Lecture 3. A) Expected Value

From Significant Statistics
Jump to navigation Jump to search

Expected Value

The expected value of r.v. [math]X[/math], usually written as [math]E\left(X\right)[/math], is defined as

[math]E\left(X\right)=\sum_{x\in\mathbf{R}}x\,f_{X}\left(x\right)[/math] if [math]X[/math] is discrete.

[math]E\left(X\right)=\int_{-\infty}^{+\infty}t\,f_{X}\left(t\right)dt[/math] if [math]X[/math] is continuous.

In general, suppose we would like to calculate [math]E\left(g\left(X\right)\right)[/math] where [math]g\left(\cdot\right)[/math] is a function. Then we would obtain

[math]E\left(g\left(X\right)\right)=\sum_{x\in\mathbf{R}}g\left(x\right)\,f_{X}\left(x\right)[/math] if [math]X[/math] is discrete.

[math]E\left(g\left(X\right)\right)=\int_{-\infty}^{+\infty}g\left(t\right)\,f_{X}\left(t\right)dt[/math] if [math]X[/math] is continuous.

Existence of the Expected Value

Unlike in the discrete case, it is possible to obtain [math]E\left(X\right)=\infty[/math] in the continuous case. This is surprising if we think of expectations as averages, because no average of finite numbers turns out to be infinite. However, this is not assured in the case where [math]X[/math] is continuous. To see this, notice that we already know that some integrals yield infinity rather than a real number.

For example, [math]\int_{1}^{\infty}\frac{1}{x}dx=\infty[/math]. The reason is that, while [math]\frac{1}{x}[/math] is decreasing when [math]x\gt 1[/math], it approximates the x-axis ‘too slowly’, such that the area underneath grows fast enough so its sum is infinite.

Suppose instead that r.v. [math]X[/math] has pdf [math]\frac{1}{x^{2}}[/math], defined in domain [math]\left[1,\infty\right][/math]. We can show that this function is indeed a pdf, since [math]\int_{1}^{\infty}\frac{1}{x^{2}}dx=1[/math], and the function is positive over its domain. In this case, [math]E\left(X\right)=\int_{1}^{\infty}x\frac{1}{x^{2}}dx=\int_{1}^{\infty}\frac{1}{x}dx=\infty[/math]; We have discovered a r.v. that does not have an expected value (i.e., it’s infinite). Intuitively, the expected value exists as long as the pdf approaches zero fast enough.

You will usually read the statement “if [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math], then [math]E\left(X\right)[/math] does not exist”, and may wonder about the absolute value. This is just a compact way to write the non-existence of the expected value. To see this, first suppose [math]X[/math] were always positive. In this case, the absolute value would be redundant, but the statement would remain correct. If [math]X[/math] were always negative and with [math]E\left[X\right]=-\infty[/math], then the statement would still apply, because [math]E\left[\left|X\right|\right]=\infty[/math] remains true. Finally, suppose [math]X[/math] spans [math]\left(-\infty,\infty\right)[/math] and [math]\int_{-\infty}^{0}\left|t\right|\,f_{X}\left(t\right)dt=-\infty[/math] and [math]\int_{0}^{\infty}\left|t\right|\,f_{X}\left(t\right)dt=+\infty[/math]. Then, the expectation of [math]X[/math] does not exist, as it is indeterminate, but the statement [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math] still holds.

So, [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math] is an efficient way to summarize the cases that may lead a r.v. to not have an expectation.

Alternative notation

You may sometimes see the statement [math]E\left(X\right)=\int_{-\infty}^{+\infty}t dF_{X}\left(t\right)[/math] instead. This notation usually refers to the Lebesgue integral, where [math]F_{X}\left(t\right)[/math] refers to a 'measure.' We do not cover the distinction here here. Notice that if we are ok canceling differentials (we'll avoid the long technicalities), we can obtain [math]\int_{-\infty}^{+\infty}\left|t\right|dF_{X}\left(t\right)=\int_{-\infty}^{+\infty}\left|t\right|dF_{X}\frac{dt}{dt}=\int_{-\infty}^{+\infty}\left|t\right|\frac{dF_{X}\left(t\right)}{dt}dt[/math], and finally by noting that [math]\frac{dF_{X}\left(t\right)}{dt}=f_{X}\left(t\right)[/math], we obtain the familiar expression [math]\int_{-\infty}^{+\infty}t\,f_{X}\left(t\right)dt[/math].

Basic properties of expectations

  • Linearity: [math]E\left(ag\left(X\right)+bh\left(X\right)\right)=aE\left(g\left(X\right)\right)+bE\left(h\left(X\right)\right)[/math]
  • Order-preserving: [math]g\left(X\right)\leq h\left(X\right),\,\forall x\in\mathbb{R}\Rightarrow E\left(g\left(X\right)\right)\leq E\left(h\left(X\right)\right)[/math] (and equality holds if [math]g\left(X\right)=h\left(X\right)[/math])