Lecture 3. A) Expected Value

From Significant Statistics
Jump to navigation Jump to search

Expected Value

The expected value of r.v. [math]X[/math], usually written as [math]E\left(X\right)[/math], is defined as

[math]E\left(X\right)=\sum_{x\in\mathbf{R}}x\,f_{X}\left(x\right)[/math] if [math]X[/math] is discrete

[math]E\left(X\right)=\int_{-\infty}^{+\infty}t\,f_{X}\left(t\right)dt[/math] if [math]X[/math] is continuous

In general, suppose we would like to calculate [math]E\left(g\left(X\right)\right)[/math] where [math]g\left(\cdot\right)[/math] is a function. Then we would obtain

[math]E\left(g\left(X\right)\right)=\sum_{x\in\mathbf{R}}g\left(x\right)\,f_{X}\left(x\right)[/math] if [math]X[/math] is discrete

[math]E\left(g\left(X\right)\right)=\int_{-\infty}^{+\infty}g\left(t\right)\,f_{X}\left(t\right)dt[/math] if [math]X[/math] is continuous

Existence of the Expected Value

Unlike in the discrete case, it is possible to obtain [math]E\left(X\right)=\infty[/math] in the continuous case. This is surprising if we think of expectations as averages, because no average of finite numbers turn out to be infinite. However, this is not assured in the case where [math]X[/math] is continuous. To see this, notice that we already know that some integrals yield infinity rather than a real number. For example, [math]\int_{1}^{\infty}\frac{1}{x}dx=\infty[/math]. The reason is that, while [math]\frac{1}{x}[/math] is decreasing when [math]x\gt 1[/math], it approximates the x-axis ‘too slowly’, such that the area underneath grows fast enough so its sum is infinite. Suppose r.v. [math]X[/math] has pdf [math]\frac{1}{x^{2}}[/math], defined on domain [math]\left[1,\infty\right][/math]. We can show that this function is indeed a pdf, since [math]\int_{1}^{\infty}\frac{1}{x^{2}}dx=1[/math], and the function is positive over its domain. In this case, [math]E\left(X\right)=\int_{1}^{\infty}x\frac{1}{x^{2}}dx=\int_{1}^{\infty}\frac{1}{x}dx=\infty[/math]. We have discovered an r.v. that does not have an expected value (i.e., it’s infinite). Intuitively, the expected value exists as long as the pdf approaches zero fast enough.

You will usually read the statement “if [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math], then [math]E\left(X\right)[/math] does not exist”, and may wonder about the absolute value. This is just a quick way to write the condition. To see this, first suppose [math]X[/math] were always positive. In this case, the absolute value would be redundant, but the statement would remain correct. If [math]X[/math] were always negative and with [math]E\left[X\right]=-\infty[/math], then the statement would still apply, because [math]E\left[\left|X\right|\right]=\infty[/math] remains true. Finally, suppose [math]X[/math] spans [math]\left(-\infty,\infty\right)[/math]. In this case, [math]\int_{-\infty}^{0}\left|t\right|\,f_{X}\left(t\right)dt=-\infty[/math] and [math]\int_{0}^{\infty}\left|t\right|\,f_{X}\left(t\right)dt=+\infty[/math]. Then, the expectation of [math]X[/math] does not exist, as it is indeterminate, but the statement [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math] still holds. So, [math]\int_{-\infty}^{+\infty}\left|t\right|\,f_{X}\left(t\right)dt=\infty[/math] is an efficient way to summarize the cases that may lead a r.v. to not have an expectation.

Alternative notation

You may sometimes see the statement [math]E\left(X\right)=\int_{-\infty}^{+\infty}t dF_{X}\left(t\right)[/math] instead. This notation usually refers to the Lebesgue integral, where [math]dF_{X}\left(t\right)[/math] refers to a 'measure.' Rather than going down that rabbit hole, which we do not cover here, simply notice that by our previous rule of canceling differentials, [math]\int_{-\infty}^{+\infty}\left|t\right|dF_{X}\left(t\right)=\int_{-\infty}^{+\infty}\left|t\right|dF_{X}\left(t\right)\frac{dF_{X}\left(t\right)}{dt}\frac{dt}{dF_{X}\left(t\right)}=\int_{-\infty}^{+\infty}\left|t\right|\frac{dF_{X}\left(t\right)}{dt}dt[/math], and finally by noting that [math]\frac{dF_{X}\left(t\right)}{dt}=f_{X}\left(t\right)[/math], we obtain the familiar expression [math]\int_{-\infty}^{+\infty}t\,f_{X}\left(t\right)dt[/math].

Basic properties of expectations

  • Linearity: [math]E\left(ag\left(X\right)+bh\left(X\right)\right)=aE\left(g\left(X\right)\right)+bE\left(h\left(X\right)\right)[/math]
  • Order-preserving: [math]g\left(X\right)\leq h\left(X\right),\,\forall x\in\mathbb{R}\Rightarrow E\left(g\left(X\right)\right)\leq E\left(h\left(X\right)\right)[/math] (and equality holds if [math]g\left(X\right)=h\left(X\right)[/math])