Tutorial on Applications of Expectation and Variance Doron ZEILBERGER

Version of Nov, 8, 2015 (thanks to Tamar Zeilberger, who won a dollar, for finding a typo in p.5) Example I: Two classes, one consisting of ONLY boys, that has 30 students, and another consisting of ONLY girls, that has 20 students, go to a field trip. Each class had one teacher. One of the students is randomly selected, each student with equal probability. Let random variable X denote the number of children in that chosen student’s class. One of the teachers of these two classes is randomly selected, each with the same probability. Let the random variable Y denote the number of students of the chosen teacher’s class.

(a) What is larger E(X) or E(Y)? Explain why these quantities are different.

(b) Compute V ar(X) and V ar(Y) Solution to Example I: (a) There are 30 + 20 = 50 students, 30 are boys and 20 are girls, and since each student is equally likely (with prob. \frac{1}{50}) to be picked:

P (Boy) = \frac{30}{50} = \frac{3}{5} , P (Girl) = \frac{20}{50} = \frac{2}{5} .

Also, for the random variable X

X(Boy) = 30 , X(Girl) = 20 .

Hence

E(X) = P (Boy) · X(Boy) + P (Girl) · X(Girl) = \frac{3}{5} · 30 + \frac{2}{5} · 20 = \frac{130}{5} = 26 .

For the random variable Y, the values are analogous

Y (T eacherOfBoys) = 30 , Y (T eacherOfGirls) = 20 ,

but the probability distribution is different:

P (T eacherOfBoys) = \frac{1}{2} , P (T eacherOfGrils) = \frac{1}{2} .

Hence E(Y) = P (T eacherOfBoys) · Y (T eacherOfBoys) + P (T eacherOfGirls) · Y (T eacherOfGirls)

= \frac{1}{2} · 30 + \frac{1}{2} · 20 = \frac{50}{2} = 25 .

Answer to I(a): E(X) = 26 and E(Y) = 25. Hence E(X) is larger. This is as it should be, since in this case the “coin” is loaded in favor of the boys, since there are more of them. (b)

(b) regarding X We already know from part (a) that E(X), let’s call it μ, equals 26. So

V ar(X) = E((X−μ)^{2}) = \frac{3}{5}·(30−26)^{2}+\frac{2}{5}·(20−26)^{2} = \frac{3}{5}·(4)^{2}+\frac{2}{5}·(−6)^{2} = \frac{3}{5}·16+\frac{2}{5}·36 = \frac{120}{5} = 24 .

(b) regarding Y We already know from part (a) that E(Y), let’s call it μ, equals 25. So

V ar(Y) = E((Y −μ)^{2}) = \frac{1}{2} ·(30−25)^{2} + \frac{1}{2} ·(20−25)^{2} = \frac{1}{2} ·(5)^{2} + \frac{1}{2} ·(−5)^{2} = \frac{1}{2} ·25+ \frac{1}{2} ·25 = 25 .

Answer to I(b): V ar(X) = 24 and V ar(Y) = 25. Do Right Now: Problem 1 of Homework 5-PolySci 200A. Important Properties of Expectation Linearity of Expectation: For any random variables X, Y (with the same unerlying probability space of course) [Warning: In Example I, this is not the case!], we have

E(X + Y) = E(X) + E(Y) .

(In words: whenever you have to take the expectation of a sum, you can take the sum of the expectations, in other words, you can distribute) Warning: E(XY) is NOT E(X)E(Y). It is only true if X and Y are independent random variables (they do not influence each other). In fact it is true, more generally, if they are uncorellated.

”Taking a Constant Out” Rule for Expectation For any constant c

E(cX) = cE(X) .

(In words: You can take constants out of the E symbol). Note: This makes perfect sense. If everything gets, e.g., doubled, then the expectation is also doubled. The expectation of a constant random variable is that constant! If c is a constant, then

E(c) = c .

Of course, if it is always the same (very boring!) the ‘expectation’ is guaranteed and is equal to that constant. Important special cases:

E(1) = 1 , E(0) = 0 .

Taking a Constant Out Rule for Variance V ar(cX) = c^{2} V ar(X) .

Warning: Note the c^{2} on the right side, e.g. V ar(3X) = 9V ar(X) NOT 3V ar(X). Adding a constant Rule Variance: If c is any constant, then

V ar(c + X) = V ar(X) .

Comment: Adding a constant does not change the “variability”, it gets cancelled out! Example II: If E(X) = 3 and E(X^{2}) = 1, and E(X^{3}) = −1. Find

(a) E((X + 2)(X − 3))

(b) E(X(X + 2)(X − 3)) Solution to Example II: (a):

Using highschool algebra (X + 2)(X − 3) = X^{2} + 2X − 3X − 6 = X^{2} − X − 6 .

Applying the expecation operation, E, to both sides, we get E( (X + 2)(X − 3) ) = E( X^{2} − X − 6 ) .

Using the linearity of expectation, we get that the above equals

E(X^{2}) − E(X) − 6E(1) .

From the data of the problem (see problem above), E(X^{2}) = 1, E(X) = 3, and of course (always!) E(1) = 1, so this equals

1 − 3 − 6 = −8 .

Answer to II(a): −8. (Note: For this part, E(X^{3}) was not needed). (b): Using highschool algebra

X(X + 2)(X − 3) = X(X^{2} + 2X − 3X − 6) = X(X^{2} − X − 6) = X^{3} − X^{2} − 6X .

Applying the expecation operation, E, to both sides, we get E( X(X + 2)(X − 3) ) = E( X^{3} − X^{2} − 6X) .

Using the linearity of expectation, we get that the above equals

E(X^{3}) − E(X^{2}) − 6E(X) .

From the data of the problem (see problem above), E(X^{3}) = −1, E(X^{2}) = 1, E(X) = 3, so this equals

(−1) − (1) − 6 · (3) = −1 − 1 − 18 = −20 .

Answer to II(b): −20.

Example III: If E(X) = 3 and V ar(X) = 2, find

(a) E( (2X + 1)(X − 2) )

(b) V ar(−11 + 5X) Solution to Example III(a): This is a multi-step problem. Before starting, we must find E(X^{2}), and then we can proceed as in Example II. By the famous formula

V ar(X) = E(X^{2}) − E(X)^{2} ,

we get the equivalent statement (also worth memorizing)

E(X^{2}) = V ar(X) + E(X)^{2} .

Hence, in this problem

E(X^{2}) = V ar(X) + E(X)^{2} = 2 + 3^{2} = 2 + 9 = 11 .

Now that we have both E(X) = 3, and E(X^{2}) = 11, we can proceed to answer III(a). Using highschool algebra

(2X + 1)(X − 2) = 2X^{2} + X − 4X − 2 = 2X^{2} − 3X − 2 .

Hence, applying the E operation to both sides, and using the linearity of expectation, and taking out constants rules

E( (2X + 1)(X − 2) ) = E( 2X^{2} − 3X − 2 ) = 2E(X^{2}) − 3E(X) − 2E(1) =

= 2 · 11 − 3 · 3 − 2 · 1 = 22 − 9 − 2 = 11 .

Answer to III(a): 11. Solution to III(b):

V ar(−11 + 5X) = V ar(5X) = 5^{2}V ar(X) = 25 · 2 = 50 .

Answer to III(b): 50. Note: Part (b) is much easier and does not depend on part (a).

Do Right Now: Problem 2 of Homework 5-PolySci 200A. Note on Problem 3 of Homework 5-PolySci 200A: This is a very wordy, multi-step problem, but you have all the technical know-how to tackle it. Please do your best to do it all by yourself! Example IV: Verify, directly, that

E(X + Y) = E(X) + E(Y) ,

if X can only take the values 2 and 5, and Y can only take the values −1 and 4, and it is given that

P (X = 2 AND Y = −1) = \frac{1}{8} ,

P (X = 2 AND Y = 4) = \frac{1}{4} ,

P (X = 5 AND Y = −1) = \frac{1}{8} ,

P (X = 5 AND Y = 4) = \frac{1}{2} .

Solution to Example IV: Let’s figure out the probability distribution for the new random variable X + Y

P (X + Y = 1) = \frac{1}{8}

P (X + Y = 6) = \frac{1}{4} ,

P (X + Y = 4) = \frac{1}{8} ,

P (X + Y = 9) = \frac{1}{2} .

Hence

E(X + Y) = P (X + Y = 1) · 1 + P (X + Y = 6) · 6 + P (X + Y = 4) · 4 + P (X + Y = 9) · 9

= \frac{1}{8} · 1 + \frac{1}{4} · 6 + \frac{1}{8} · 4 + \frac{1}{2} · 9 = \frac{53}{8} .

So, by doing it directly, we got E(X + Y) = \frac{53}{8}. Now let’s find E(X) and E(Y). Using the above data:

P (X = 2) = P (X = 2 AND Y = −1) + P (X = 2 AND Y = 4) = \frac{1}{8} + \frac{1}{4} = \frac{3}{8} ,

P (X = 5) = P (X = 5 AND Y = −1) + P (X = 5 AND Y = 4) = \frac{1}{8} + \frac{1}{2} = \frac{5}{8} .

Hence

E(X) = P (X = 2) · 2 + P (X = 5) · 5 = \frac{3}{8} · 2 + \frac{5}{8} · 5 = \frac{31}{8}

Similarly

P (Y = −1) = P (X = 2 AND Y = −1) + P (X = 5 , AND Y = −1) = \frac{1}{8} + \frac{1}{8} = \frac{1}{4}

P (Y = 4) = P (X = 2 AND Y = 4) + P (X = 5 AND Y = 4) = \frac{2}{8} + \frac{1}{2} = \frac{3}{4}

Hence

E(Y) = P (Y = −1) · (−1) + P (Y = 4) · 4 = \frac{1}{4} · (−1) + \frac{3}{4} · 4 = \frac{11}{4}

Hence

E(X) + E(Y) = \frac{31}{8} + \frac{11}{4} = \frac{53}{8} .

The same as E(X + Y) computed above! YEA!: We verified the theorem E(X + Y) = E(X) + E(Y) for this particular special case. Warning: The example below is very advanced and abstract. Example V: Prove that

E(X + Y) = E(X) + E(Y)

for the special case where X and Y can each only take two different values. Solution to Example V: Let the two values that X can take be x_{1} and x_{2}, and let the two values that Y can take be y_{1} and y_{2}. Also, let us denote the following:

P (X = x_{1} AND Y = y_{1}) = p_{11} ,

P (X = x_{1} AND Y = y_{2}) = p_{12} ,
P (X = x_{2} AND Y = y_{1}) = p_{21} ,
P (X = x_{2} AND Y = y_{2}) = p_{22} ,

where of course, p_{11} +p_{12} +p21+p_{22} = 1 (they must add-up to 1, since this covers all possibilities), and they are each between 0 and 1 (inclusive), as probabilities should be.

Let’s figure out the probability distribution for the new random variable X + Y P (X + Y = x_{1} + y_{1}) = p_{11}

P (X + Y = x_{1} + y_{2}) = p_{12} ,
P (X + Y = x_{2} + y_{1}) = p_{21} ,
P (X + Y = x_{2} + y_{2}) = p_{22} .

Hence

E(X + Y) = P (X + Y = x_{1} + y_{1}) · (x_{1} + y_{1}) + P (X + Y = x_{1} + y_{2}) · (x_{1} + y_{2})

+P (X + Y = x_{2} + y_{1}) · (x_{2} + y_{1}) + P (X + Y = x_{2} + y_{2}) · (x_{2} + y_{2})
= p_{11}(x_{1} + y_{1}) + p_{12}(x_{1} + y_{1}) + p_{21}(x_{2} + y_{1}) + p_{22}(x_{2} + y_{2}) .

Now let’s find E(X) and E(Y). Using the above data:

P (X = x_{1}) = P (X = x_{1} AND Y = y_{1}) + P (X = x_{1} AND Y = y_{2}) = p_{11} + p_{12} , P (X = x_{2}) = P (X = x_{2} AND Y = y_{1}) + P (X = x_{2} AND Y = y_{2}) = p_{21} + p_{22} .

Hence

E(X) = P (X = x_{1}) · x_{1} + P (X = x_{2}) · x_{2} = (p_{11} + p_{12})x_{1} + (p_{21} + p_{22})x_{2} .

Similarly, P (Y = y_{1}) = P (X = x_{1} AND Y = y_{1}) + P (X = x_{2} AND Y = y_{1}) = p_{11} + p_{21} , P (Y = y_{2}) = P (X = x_{1} AND Y = y_{2}) + P (X = x_{2} AND Y = y_{2}) = p_{12} + p_{22} .

Hence

E(Y) = P (Y = y_{1}) · y_{1} + P (Y = y_{2}) · y_{2} = (p_{11} + p_{21})y_{1} + (p_{12} + p_{22})y_{2} .

Adding up, we get:

E(X) + E(Y) = (p_{11} + p_{12})x_{1} + (p_{21} + p_{22})x_{2} + (p_{11} + p_{21})y_{1} + (p_{12} + p_{22})y_{2} , and this the same as E(X + Y) computed above! (check the algebra!) QED.

Note: This is a more general version of Problem 4 in Homework 5. There it is assumed that the two random variables are independent.

There the inputs are

P (X = x_{1}) (and P (X = x_{2}), that is necessarily 1−P (X = x_{1}), since there are only two possibilities for the values of X),

and

P (Y = y_{1}) (and P (Y = y_{2}), that is necessarily 1 − P (Y = y_{1}), since there are only two possibilities for the values of Y).

Calling P (X = x_{1}) = α and P (Y = y_{1}) = β, to do Probelm 4 of Homework 5, you do the above, but with the simplifying values

p_{11} = αβ , p_{12} = α(1 − β) ,
p_{21} = (1 − α)β , p_{22} = (1 − α)(1 − β) , .

Example VI: Let X be a random variable with expected value μ and variance σ^{2}. Find the expected values of the following functions of X

Y = \frac{3X−5μ}{7σ}

Solution to Example VI:

E(Y) = E(\frac{3X − 5μ}{7σ}) = \frac{3E(X) − 5μ}{7σ} = \frac{3μ − 5μ}{7σ} = −\frac{2μ}{7σ}

Do right now: Problem 5 of Homework 5.

Doron Zeilberger, Department of Mathematics, Rutgers University (New Brunswick), Hill Center-Busch Campus, 110 Frelinghuysen Rd., Piscataway, NJ 08854-8019, USA.

Email: zeilberg at math dot rutgers dot edu .