%%a Plain TeX file by Doron Zeilberger (x pages)

%begin macros
\def\L{{\cal L}}
\baselineskip=14pt
\parskip=10pt
\def\halmos{\hbox{\vrule height0.15cm width0.01cm\vbox{\hrule height
  0.01cm width0.2cm \vskip0.15cm \hrule height 0.01cm width0.2cm}\vrule
  height0.15cm width 0.01cm}}
\font\eightrm=cmr8 \font\sixrm=cmr6
\font\eighttt=cmtt8
\magnification=\magstephalf
\def\P{{\cal P}}
\def\Q{{\cal Q}}
\def\1{{\overline{1}}}
\def\2{{\overline{2}}}
\parindent=0pt
\overfullrule=0in
\def\Tilde{\char126\relax}
\def\frac#1#2{{#1 \over #2}}
%\headline={\rm  \ifodd\pageno  \RightHead  \else  \LeftHead  \fi}
%\def\RightHead{\centerline{
%Title
%
%\def\LeftHead{ \centerline{Doron Zeilberger}}
%end macros

\centerline
{\bf Tutorial on Applications of Expectation and Variance }
\bigskip
\centerline
{\it Doron ZEILBERGER}

Version of Nov, 8, 2015 (thanks to Tamar Zeilberger, who won a dollar, for finding a typo in p.5)

{\bf Example I}: Two classes, one consisting of ONLY boys,  that has 30 students,
and another consisting of ONLY girls, that has 20 students, go to a field trip. 
Each class had one teacher.

One of the students
is randomly selected, each student with equal probability. Let random variable $X$ denote
the number of children in that chosen student's class. 

One of the teachers of these two classes is randomly selected, each with the same probability.
Let the random variable $Y$ denote the number of students of the chosen teacher's class.

{\bf (a)} What is larger $E(X)$ or $E(Y)$? Explain why these quantities are different.

{\bf (b)} Compute $Var(X)$ and $Var(Y)$


{\bf Solution to Example I}:

{\bf (a) }

There are $30+20=50$ students, $30$ are boys and $20$ are girls, and since each student is equally likely (with prob. $\frac{1}{50}$)
to be picked:
$$
P(Boy)=\frac{30}{50}=\frac{3}{5} \quad, \quad
P(Girl)=\frac{20}{50}=\frac{2}{5} \quad .
$$
Also, for the random variable X
$$
X(Boy)=30 \quad, \quad X(Girl)=20 \quad .
$$

Hence
$$
E(X)=P(Boy)\cdot X(Boy) + P(Girl)\cdot X(Girl) =
\frac{3}{5} \cdot 30 + \frac{2}{5} \cdot 20= \frac{130}{5}= 26 \quad .
$$


For the random variable $Y$, the values are analogous
$$
Y(TeacherOfBoys)= 30 \quad, \quad
Y(TeacherOfGirls)= 20 \quad ,
$$
but the {\bf probability distribution} is different:
$$
P(TeacherOfBoys)= \frac{1}{2} \quad, \quad
P(TeacherOfGrils)= \frac{1}{2} \quad .
$$

Hence
$$
E(Y)=P(TeacherOfBoys)\cdot Y(TeacherOfBoys) + P(TeacherOfGirls)\cdot Y(TeacherOfGirls) 
$$
$$
=
\frac{1}{2} \cdot 30 + \frac{1}{2} \cdot 20= \frac{50}{2}= 25 \quad .
$$

{\bf Answer to I(a)}: $E(X)=26$ and $E(Y)=25$. Hence $E(X)$ is larger. This is as it should be, since
in this case the ``coin'' is {\bf loaded} in favor of the boys, since there are more of them.

{\bf (b) }

{\bf (b) regarding $X$} 

We already know from part (a) that $E(X)$, let's call it $\mu$, equals $26$.

So

$$
Var(X)= E( (X-\mu)^2) =
\frac{3}{5} \cdot (30-26)^2 + \frac{2}{5} \cdot (20-26)^2= 
\frac{3}{5} \cdot (4)^2 + \frac{2}{5} \cdot (-6)^2=
\frac{3}{5} \cdot 16 + \frac{2}{5} \cdot 36= \frac{120}{5}=24 \quad .
$$

{\bf (b) regarding $Y$} 

We already know from part (a) that $E(Y)$, let's call it $\mu$, equals $25$.

So

$$
Var(Y)= E( (Y-\mu)^2) =
\frac{1}{2} \cdot (30-25)^2 + \frac{1}{2} \cdot (20-25)^2= 
\frac{1}{2} \cdot (5)^2 + \frac{1}{2} \cdot (-5)^2=
\frac{1}{2} \cdot 25 + \frac{1}{2} \cdot 25= 25 \quad .
$$

{\bf Answer to I(b)}: $Var(X)=24$ and $Var(Y)=25$.


{\bf Do Right Now}: Problem 1 of Homework 5-PolySci 200A.



{\bf Important Properties of Expectation}

{\bf Linearity of Expectation}: For {\it any} random variables $X$, $Y$ (with the same unerlying probability space of course)

[Warning: In Example I, this is not the case!], we have

$$
E(X+Y)= E(X)+ E(Y) \quad.
$$

(In words: whenever you have to take the expectation of a sum, you can take the sum of the expectations, in other words,
you can distribute)

{\bf Warning}: $E(XY)$ is {\bf NOT} $E(X)E(Y)$. It is only true if $X$ and $Y$ are {\it independent} random variables
(they do not influence each other). In fact it is true, more generally, if they are {\it uncorellated}.

{\bf "Taking a Constant Out'' Rule for Expectation}

For any {\bf constant} $c$
$$
E(cX)=c E(X) \quad .
$$
(In words: You can take {\bf constants} out of the $E$ symbol).

Note: This makes perfect sense. If everything gets, e.g., doubled, then the expectation is also doubled.

{\bf The expectation of a constant random variable is that constant!}

If $c$ is a constant, then
$$
E(c)=c \quad .
$$

Of course, if it is always the same (very boring!) the `expectation' is {\it guaranteed} and is equal to that constant.

{\it Important special cases}:

$$
E(1)=1 \quad, \quad  E(0)=0 \quad .
$$

{\bf Taking a Constant Out Rule for Variance}
$$
Var(cX)=c^2 \, Var(X) \quad .
$$

{\bf Warning}: Note the $c^2$ on the right side, e.g. $Var(3X)=9Var(X)$ {\bf NOT} $3Var(X)$.

{\bf Adding a constant Rule Variance}: If $c$ is any constant, then
$$
Var(c+X)= Var(X) \quad .
$$

{\bf Comment:} Adding a constant does not change the ``variability'', it gets cancelled out!



{\bf Example II:}

If $E(X)=3$ and $E(X^2)=1$, and $E(X^3)=-1$. Find

(a) $E((X+2)(X-3))$

(b) $E(X(X+2)(X-3))$

{\bf Solution to Example II}:

{\bf (a)}: 

Using {\it highschool algebra}
$$
(X+2)(X-3)=X^2+2X-3X-6= X^2-X-6 \quad .
$$

Applying the {\bf expecation operation}, $E$, to both sides, we get
$$
E( \, (X+2)(X-3) \,  ) = E (\, X^2-X-6 \, ) \quad .
$$

Using the {\bf linearity of expectation}, we get that the above equals
$$
E(X^2)-E(X)- 6 E(1) \quad .
$$
From the data of the problem (see problem above), $E(X^2)=1$, $E(X)=3$, and of course (always!) $E(1)=1$, so this equals
$$
1-3-6=-8 \quad.
$$

{\bf Answer to II(a)}: $-8$.

(Note: For this part, $E(X^3)$ was not needed).


{\bf (b)}: 

Using {\it highschool algebra}
$$
X(X+2)(X-3)=X(X^2+2X-3X-6)=  X(X^2-X-6)=X^3-X^2-6X \quad .
$$

Applying the {\bf expecation operation}, $E$, to both sides, we get
$$
E( \, X(X+2)(X-3) \, ) = E (\, X^3-X^2-6X \, ) \quad .
$$

Using the {\bf linearity of expectation}, we get that the above equals
$$
E(X^3)-E(X^2)- 6 E(X) \quad .
$$
From the data of the problem (see problem above), 

$E(X^3)=-1$, $E(X^2)=1$, $E(X)=3$, 

so this equals
$$
(-1)- (1) - 6\cdot (3)= -1-1-18= -20 \quad .
$$

{\bf Answer to II(b)}: $-20$.



{\bf Example III}: If $E(X)=3$ and $Var(X)=2$, find

(a) $E( \, (2X+1)(X-2) \, )$

(b) $Var(-11+5X)$

{\bf Solution to Example III(a)}:

This is a {\bf multi-step problem}. Before starting, we must find $E(X^2)$, and then we can proceed as in Example II.

By the famous formula
$$
Var(X)= E(X^2)-E(X)^2 \quad,
$$
we get the equivalent statement (also worth memorizing)
$$
E(X^2)=Var(X)+ E(X)^2 \quad.
$$
Hence, in this problem
$$
E(X^2)=Var(X)+ E(X)^2 = 2+ 3^2= 2+9= 11 \quad.
$$

Now that we have {\it both} $E(X)=3$, and $E(X^2)=11$, we can proceed to answer III(a).

Using {\it highschool algebra}

$$
 (2X+1)(X-2)= 2X^2+X-4X-2= 2X^2- 3X-2 \quad.
$$
Hence, applying the $E$ operation to {\bf both sides}, and using the {\it linearity of expectation}, and
{\it taking out constants} rules
$$
E(\,  (2X+1)(X-2) \, ) = 
E(\,   2X^2- 3X-2 \, )=
2E(X^2) - 3 E(X) -2 E(1)=
$$
$$
= 2 \cdot 11 -3 \cdot 3 -2 \cdot 1=
22-9-2=11 \quad .
$$

{\bf Answer to III(a)}: $11$.

{\bf Solution to III(b)}:

$$
Var(-11+5X)= Var(5X)= 5^2 Var(X)= 25 \cdot 2= 50 \quad .
$$

{\bf Answer to III(b)}: $50$.

{\bf Note}: Part (b) is much easier and does {\bf not} depend on part (a).

{\bf Do Right Now}: Problem 2 of Homework 5-PolySci 200A.


Note on Problem 3 of Homework 5-PolySci 200A: This is a very wordy, multi-step problem, but
you have all the technical know-how to tackle it. Please do your best to do it
{\it all by yourself!}

{\bf Example IV}:

Verify, directly,  that
$$
E(X+Y)=E(X)+E(Y) \quad ,
$$
if $X$ can only take the values $2$ and $5$, and $Y$ can only take the values $-1$ and $4$, and it is given that
$$
P(X=2 \quad AND \quad   Y=-1)=\frac{1}{8} \quad ,
$$
$$
P(X=2 \quad AND \quad  Y=4)=\frac{1}{4} \quad ,
$$
$$
P(X=5 \quad  AND \quad  Y=-1)=\frac{1}{8} \quad ,
$$
$$
P(X=5 \quad AND \quad  Y=4)=\frac{1}{2} \quad .
$$

{\bf Solution to Example IV}:

Let's figure out the {\bf probability distribution} for the new random variable $X+Y$
$$
P(X+Y=1)= \frac{1}{8}
$$
$$
P(X+Y=6)=\frac{1}{4} \quad ,
$$
$$
P(X+Y=4)=\frac{1}{8} \quad ,
$$
$$
P(X+Y=9)=\frac{1}{2} \quad .
$$

Hence

$$
E(X+Y)=P(X+Y=1) \cdot 1 + P(X+Y=6) \cdot 6 + P(X+Y=4) \cdot 4 + P(X+Y=9) \cdot 9 
$$
$$
= \frac{1}{8} \cdot 1 + \frac{1}{4} \cdot 6 + \frac{1}{8} \cdot 4 + \frac{1}{2} \cdot 9=\frac{53}{8} \quad.
$$

So, by doing it directly, we got $E(X+Y)=\frac{53}{8}$.

Now let's find $E(X)$ and $E(Y)$.
Using the above data:
$$
P(X=2)=P(X=2 \quad AND \quad  Y=-1)+ P(X=2 \quad AND \quad  Y=4) = \frac{1}{8} + \frac{1}{4}= \frac{3}{8} \quad ,
$$
$$
P(X=5)=P(X=5 \quad AND \quad  Y=-1)+ P(X=5 \quad AND \quad  Y=4) = \frac{1}{8} + \frac{1}{2}= \frac{5}{8} \quad .
$$
Hence
$$
E(X)= P(X=2) \cdot 2 + P(X=5) \cdot 5= \frac{3}{8} \cdot 2 + \frac{5}{8} \cdot 5 =\frac{31}{8}
$$

Similarly
$$
P(Y=-1)= P(X=2 \quad AND \quad  Y=-1)+ P(X=5 \quad , AND \quad  Y=-1) = \frac{1}{8} + \frac{1}{8}= \frac{1}{4}
$$
$$
P(Y=4)= P(X=2 \quad AND \quad  Y=4)+ P(X=5 \quad AND \quad  Y=4) = \frac{2}{8} + \frac{1}{2}= \frac{3}{4}
$$
Hence
$$
E(Y)= P(Y=-1) \cdot (-1) + P(Y=4) \cdot 4= \frac{1}{4} \cdot (-1) + \frac{3}{4} \cdot 4 =\frac{11}{4}
$$

Hence
$$
E(X)+ E(Y)= \frac{31}{8}+ \frac{11}{4}= \frac{53}{8} \quad .
$$

The same as $E(X+Y)$ computed above!

{\bf YEA!}: We verified the theorem $E(X+Y)=E(X)+E(Y)$ for this {\it particular} {\bf special case}.



{\bf Warning}: The example below is very advanced and abstract.

{\bf Example V}:

Prove that
$$
E(X+Y)=E(X)+E(Y) \quad
$$
for the special case where $X$ and $Y$ can each only take two different values.


{\bf Solution to Example V}:

Let the two values that $X$ can take be $x_1$ and $x_2$, and
let the two values that $Y$ can take be $y_1$ and $y_2$.

Also, let us denote the following:
$$
P(X=x_1 \quad AND \quad  Y=y_1) = p_{11} \quad ,
$$
$$
P(X=x_1 \quad AND \quad  Y=y_2)=p_{12} \quad ,
$$
$$
P(X=x_2 \quad AND \quad  Y=y_1)=p_{21} \quad ,
$$
$$
P(X=x_2 \quad AND \quad  Y=y_2)=p_{22} \quad ,
$$
where of course, $p_{11}+p_{12}+p{21}+p_{22}=1$ (they must add-up to $1$, since this covers {\bf all} possibilities),
and they are each between $0$ and $1$ (inclusive), as probabilities should be.

Let's figure out the {\bf probability distribution} for the new random variable $X+Y$
$$
P(X+Y=x_1+ y_1 )= p_{11}
$$
$$
P(X+Y=x_1+y_2)= p_{12} \quad ,
$$
$$
P(X+Y=x_2 + y_1 )=p_{21} \quad ,
$$
$$
P(X+Y= x_2 + y_2)=p_{22} \quad .
$$

Hence

$$
E(X+Y)=P(X+Y=x_1+y_1) \cdot (x_1+y_1) + P(X+Y=x_1+y_2) \cdot (x_1+y_2) 
$$
$$
+ P(X+Y=x_2+y_1) \cdot (x_2+y_1) + P(X+Y=x_2+y_2) \cdot (x_2+y_2)
$$
$$
=p_{11}(x_1+y_1)+  p_{12}(x_1+y_1)+  p_{21}(x_2+y_1)+  p_{22}(x_2+y_2)  \quad .
$$

Now let's find $E(X)$ and $E(Y)$.

Using the above data:
$$
P(X=x_1)=P(X=x_1 \quad AND \quad  Y=y_1)+ P(X=x_1 \quad AND \quad  Y=y_2) = p_{11}+p_{12} \quad ,
$$
$$
P(X=x_2)=P(X=x_2 \quad AND \quad  Y=y_1)+ P(X=x_2 \quad AND \quad  Y=y_2) = p_{21}+p_{22} \quad .
$$

Hence

$$
E(X)= P(X=x_1) \cdot x_1 + P(X=x_2) \cdot x_2= (p_{11}+p_{12}) x_1 + (p_{21}+p_{22})x_2 \quad .
$$

Similarly,
$$
P(Y=y_1)=P(X=x_1 \quad AND \quad  Y=y_1)+ P(X=x_2 \quad AND \quad  Y=y_1) = p_{11}+p_{21} \quad ,
$$
$$
P(Y=y_2)=P(X=x_1 \quad AND \quad  Y=y_2)+ P(X=x_2 \quad AND \quad  Y=y_2) = p_{12}+p_{22} \quad .
$$
Hence
$$
E(Y)= P(Y=y_1) \cdot y_1 + P(Y=y_2) \cdot y_2= (p_{11}+p_{21}) y_1 + (p_{12}+p_{22})y_2 \quad .
$$
Adding up, we get:
$$
E(X)+ E(Y)= (p_{11}+p_{12}) x_1 + (p_{21}+p_{22})x_2+ 
(p_{11}+p_{21}) y_1 + (p_{12}+p_{22})y_2 \quad ,
$$
and this the same as $E(X+Y)$ computed above! (check the algebra!) QED.


{\bf Note:} This is a more general version of Problem 4 in Homework 5. There it is assumed that
the two random variables are {\it independent}.

There the inputs are 

$P(X=x_1)$ (and $P(X=x_2)$, that is necessarily $1-P(X=x_1)$,  since there are only two possibilities for the values of $X$), 

and 

$P(Y=y_1)$ (and $P(Y=y_2)$, that is necessarily $1-P(Y=y_1)$,  since there are only two possibilities for the values of $Y$).

Calling $P(X=x_1)=\alpha$ and $P(Y=y_1)=\beta$, to do Probelm 4 of Homework 5, you do the above, but with the
simplifying values

$$
p_{11}= \alpha \beta \quad , \quad
p_{12}= \alpha (1-\beta) \quad , \quad
$$
$$
p_{21}= (1-\alpha) \beta \quad , \quad
p_{22}= (1- \alpha) (1-\beta) \quad , \quad .
$$



{\bf Example VI}: Let $X$ be a random variable with expected value $\mu$ and variance $\sigma^2$. Find the expected
values of the following functions of $X$

  $Y=\frac{3X - 5 \mu}{7 \sigma}$


{\bf Solution to Example VI}:

$$
E(Y) = E(\frac{3X - 5 \mu}{7 \sigma} )= \frac{ 3 E(X)- 5 \mu}{7 \sigma}= \frac{ 3 \mu- 5 \mu}{7 \sigma}= -\frac{2\mu}{7\sigma} \quad
$$

{\bf Do right now}: Problem 5 of Homework 5.

\bigskip
\hrule
\bigskip
Doron Zeilberger, Department of Mathematics, Rutgers University (New Brunswick), Hill Center-Busch Campus, 110 Frelinghuysen
Rd., Piscataway, NJ 08854-8019, USA. \hfill\break
Email: {\tt zeilberg at math dot rutgers dot edu}   \quad .


\end



