%%a Plain TeX file by Doron Zeilberger (x pages)

%begin macros
\def\L{{\cal L}}
\baselineskip=14pt
\parskip=10pt
\def\halmos{\hbox{\vrule height0.15cm width0.01cm\vbox{\hrule height
  0.01cm width0.2cm \vskip0.15cm \hrule height 0.01cm width0.2cm}\vrule
  height0.15cm width 0.01cm}}
\font\eightrm=cmr8 \font\sixrm=cmr6
\font\eighttt=cmtt8
\magnification=\magstephalf
\def\P{{\cal P}}
\def\Q{{\cal Q}}
\def\1{{\overline{1}}}
\def\2{{\overline{2}}}
\parindent=0pt
\overfullrule=0in
\def\Tilde{\char126\relax}
\def\frac#1#2{{#1 \over #2}}
%\headline={\rm  \ifodd\pageno  \RightHead  \else  \LeftHead  \fi}
%\def\RightHead{\centerline{
%Title
%
%\def\LeftHead{ \centerline{Doron Zeilberger}}
%end macros

\centerline
{\bf Tutorial on Expectation and Variance }
\bigskip
\centerline
{\it Doron ZEILBERGER}

{\bf IMPORTANT REMINDER} (Probability)

If you have a finite number of scenarios, say two, tossing a coin, where there are only two outcomes, Heads (H) and Tail (T),
we say that the set of scenarios is the {\bf sample space}. If it is a fair coin, then Heads is supposed, in the
long run to come up roughly as often as Tails (in real life it is hardly ever exact), so we say that
the probability of Heads and Probability of Tails are both $\frac{1}{2}$.

In  this case the sample space is $\{H,T\}$ and
$$
P(H)=\frac{1}{2} \quad, \quad  P(T)=\frac{1}{2}  \quad ,
$$

But if the coin is loaded, say in the long run, Heads only shows up 1 in ten times, then
$$
P(H)=\frac{1}{10} \quad, \quad  P(T)=\frac{9}{10} 
$$


If you roll a die, the sample space is:
$$
\{1, 2, 3, 4, 5, 6\} \quad .
$$

The individual scenarios (namely, landing on 1, landing on 2, etc.) are called {\it atomic events}.
If it is a fair coin then all the probabilities are the same, and hence
$$
P(1)=\frac{1}{6} \quad, \quad  P(2)=\frac{1}{6} \quad , \quad
P(3)=\frac{1}{6} \quad, \quad  P(4)=\frac{1}{6}  \quad , \quad
P(5)=\frac{1}{6} \quad, \quad  P(6)=\frac{1}{6} \quad .
$$

But if it is a loaded coin, then the probabilities may be different. For example
$$
P(1)=\frac{1}{12} \quad, \quad  P(2)=\frac{1}{4} \quad , \quad
P(3)=\frac{1}{20} \quad, \quad  P(4)=\frac{17}{60} \quad , \quad
P(5)=\frac{1}{18} \quad, \quad  P(6)=\frac{5}{18} \quad .
$$

{\bf Important Check}: The sum of all the probabilities of the atomic events must be 1.

An {\bf event} is any subset of the set of atomic scenarios, including that nothing happened, the empty set,
$\emptyset$ whose probability is $0$ and the {\bf Universal Set}, whose probability is $1$.

To compute the probability of an event, you simply add up the probabilities of its atoms.

{\bf Example I}: With the above loaded coin, find

(a) The probability of the event $\{1,3,6\}$

(b) The probability of the event $\{2,6\}$

(c) The probability that it landed on an even face.

(d) The probability that it landed on an odd face.

{\bf Solution to Example I}:

(a) $P(\{1,3,6\})=P(1)+P(3)+P(6)=\frac{1}{12}+\frac{1}{20} + \frac{5}{18}=\frac{37}{90}$

(b) $P(\{2,6\})=P(2)+P(6)=\frac{1}{4}+ \frac{5}{18}=\frac{19}{36}$

(c) The even faces are $\{2,4,6\}$, so

$P(EvenFace)=P(\{2,4,6\}=P(2)+P(4)+P(6)=\frac{1}{4}+ \frac{17}{60} + \frac{5}{18}=\frac{73}{90}$

(d) The long way is to do the same for $\{1,3,5\}$, but since the complement of the event ``odd'' is ``even''
we can just do

$P(OddFace)=1-P(EvenFace)=1- \frac{73}{90}= \frac{17}{90}$.



{\bf Do Right Now} 

{\bf Exercise 1}:

A certain tetrahedral die (with four faces) labeled $1,2,3,4$, has the following probabilities for the
atomic events
$$
P(1)=\frac{1}{8} \quad, \quad  P(2)=\frac{3}{8} \quad , \quad
P(3)=\frac{1}{16} \quad, \quad  P(4)=\frac{7}{16}  \quad .
$$
Find

(a) The probability of the event $\{1,4\}$

(b) The probability of the event $\{1,3,4\}$

(c) The probability that it landed on an even face.

(d) The probability that it landed on an odd face.


{\bf Important Concept: Random Variable}

Given our ``universal set'' of all atomic events, a {\it random variable} (a stupid name) is really
an assigning of a value to any atomic scenario. For example  if we have a coin and you are promised that
if it lands Heads you get \$10 dollars, and if lands Tails you lose \$5 dollars, then
the ``random variable'' is ``gain'', and we have
$$
X(Heads)= 10 \quad, \quad  X(Tail)=-5 \quad .
$$

Another example, in our family, the random variable is height (in inches) then
$$
X(Doron)=71 \quad, \quad X(Jane)=63 \quad, \quad X(Celia)=63 \quad, \quad X(Tamar)=64 \quad, \quad X(Hadas)=69 \quad .
$$


{\bf Example II}: In a six-faced die, labeled $\{1,2,3,4,5, 6\}$, if you are promised $20-i^2$ dollars if
it lands on a face with $i$ dots, spell out the random variable, let's call it $X$, for all possible scenarios.

{\bf Solution to Example II}:

$$
X(1)=20-1^2=19 \quad, \quad
X(2)=20-2^2=16 \quad, \quad
X(3)=20-3^2=11 \quad, \quad
X(4)=20-4^2=4 \quad, \quad
$$
$$
X(5)=20-5^2=-5 \quad, \quad
X(6)=20-6^2=-16 \quad, \quad .
$$

{\bf Do Right Now!}

{\bf Exercise 2}: 
In a four-faced die, labeled $\{1,2,3,4\}$ 
if you are promised $10-3i$ dollars if it lands on a face with $i$ dots, 
spell out the random variable, let's call it $X$ for all possible scenarios.


{\bf Important note}: Once you have a random variable $X$ defined on your sample space, you get many new ones
$2X$, $3X+2$, $X^2$, $X^3$ etc.


{\bf Example III}: In a six-faced die, labeled $\{1,2,3,4,5, 6\}$ if you are promised $20-i^2$ dollars if
it lands on a face with $i$ dots, let's call it $X$.
Spell out the random variables (a)$2X$ (b) $X^2$ (c) $X^3$

{\bf Solution to Example III}:


(a)

$$
(2X)(1)=2 \cdot 19=38 \quad, \quad
(2X)(2)=2 \cdot 16 =32\quad, \quad
(2X)(3)=2 \cdot 11=22 \quad, \quad
$$
$$
(2X)(4)=2 \cdot 4 =8 \quad, \quad
(2X)(5)=2 \cdot (-5)=-10 \quad, \quad
(2X)(6)=2 \cdot (-16)=-32 \quad, \quad .
$$

(b)
$$
X^2(1)=19^2=361 \quad, \quad
X^2(2)=16^2=256 \quad, \quad
X^2(3)=11^2=121 \quad, \quad
$$
$$
X^2(4)=4^2 \quad, \quad
X^2(5)=(-5)^2=25 \quad, \quad
X^2(6)=(-16)^2=256 \quad, \quad .
$$

(c)

$$
X^3(1)=19^3=6859 \quad, \quad
X^3(2)=16^3=4096 \quad, \quad
X^3(3)=11^3=1331 \quad, \quad
$$
$$
X^3(4)=4^3=64 \quad, \quad
X^3(5)=(-5)^3= -125 \quad, \quad
X^3(6)=(-16)^3 = -4096 \quad, \quad .
$$


{\bf Exercise 3}: 
In a four-faced die, labeled $\{1,2,3,4\}$ if you are promised $10-3i$ dollars if
it lands on a face with $i$ dots, let's call the random variable describing the gain $X$.
Spell out the random variables (a)$2X$ (b) $X^2$ (c) $X^3$.


 
{\bf Important Concept: Expectation of random variable}

Suppose that the {\it sample space} is $\{H,T\}$ and it is a fair coin, $P(H)=\frac{1}{2}$ and $P(T)=\frac{1}{2}$.
You are promised \$100 every time it lands on Heads and nothing if it lands on Tails. How much would you expect
to gain in the long-run per toss? Since in the long-run it would, roughly lands Heads every other time,
your {\bf expected gain} (per toss) is \$50, which is $100 \cdot \frac{1}{2}=50$.

If it is a loaded coin, with $P(H)=\frac{1}{10}$ and $P(T)=\frac{9}{10}$, then you would only get \$100
one-tenth of the time, so he would get, roughly, in the long-run, $100 \cdot \frac{1}{10}=10$ dollars.


If it is a loaded coin, with $P(H)=\frac{1}{10}$ and $P(T)=\frac{9}{10}$, 
but now you have to pay $10$ dollars if it lands Tails
then you would only get \$100 one-tenth of the time, but have to pay $10$ dollars, nine-tenth of the time,
so in the long-run you should expect, per toss,
$100 \cdot \frac{1}{10} - 10 \cdot \frac{9}{10}=10-9=1$ dollars.

This motivates the following

{\bf VERY IMPORTANT DEFINITION (FORMULA)}

If $X$ is a random variable on a sample space whose atomic events are $\{x_1, x_2, \dots , x_n\}$,
with respective probabilities $P(x_1), \dots, P(x_n)$, and $X$ is any random variable, then
the {\bf expectation } of $X$, denoted by $E(X)$, (often also by $\mu$) is given by (in sigma notation)
$$
E(X)=\sum_{i=1}^{n} P(x_i)X(x_i) \quad
$$
In more concrete $\dots$ notation
$$
E(X)= P(x_1)X(x_1)+ P(x_2)X(x_2)+ \dots + P(x_n)X(x_n) \quad .
$$


{\bf Example IV}: Using the same probability distribution as Example I, namely
$$
P(1)=\frac{1}{12} \quad, \quad  P(2)=\frac{1}{4} \quad , \quad
P(3)=\frac{1}{20} \quad, \quad  P(4)=\frac{17}{60}  \quad , \quad 
P(5)=\frac{1}{18} \quad, \quad  P(6)=\frac{5}{18} 
$$
and the same random variable,X, as in  Example II, compute $E(X)$

{\bf Solution to Example IV}:
$$
E(X)= \frac{1}{12} \cdot  19 +
\frac{1}{4} \cdot  16 +
\frac{1}{20} \cdot  11 +
$$
$$
\frac{17}{60} \cdot  4 +
\frac{1}{18} \cdot  (-5) +
\frac{5}{18} \cdot  (-16)=229/90=2.544\dots \quad.
$$

{\bf Answer to Example IV}: The expectation $E(X)$, is $229/90=2.544\dots$ .

{\bf Do Right Now}

{\bf Exercise 4}: Using the same probability distribution as Exercise 1, namely
$$
P(1)=\frac{1}{8} \quad, \quad  P(2)=\frac{3}{8} ,
P(3)=\frac{1}{16} \quad, \quad  P(4)=\frac{7}{16} 
$$
and the same random variable,X, as in  Exercise 2, namely

if you are promised $10-3i$ dollars if it lands on a face with $i$ dots, 
compute the expectation $E(X)$.

{\bf Example IV'}: Using the same probability distribution as in Example I, namely
$$
P(1)=\frac{1}{12} \quad, \quad  P(2)=\frac{1}{4} \quad , \quad 
P(3)=\frac{1}{20} \quad, \quad  P(4)=\frac{17}{60} \quad , \quad 
P(5)=\frac{1}{18} \quad, \quad  P(6)=\frac{5}{18}  \quad ,
$$
and the same random variable,$X$, as in  Example II, compute the expectation of $X^2$, namely $E(X^2)$

{\bf Solution to Example IV'}:
$$
E(X)= \frac{1}{12} \cdot  (19)^2 +
\frac{1}{4} \cdot  (16)^2 +
\frac{1}{20} \cdot  (11)^2 +
$$
$$
\frac{17}{60} \cdot  4^2 +
\frac{1}{18} \cdot  (-5)^2 +
\frac{5}{18} \cdot  (-16)^2=\frac{1063}{6}=177.16667\dots  \quad .
$$
{\bf Answer to Example IV'}: The expectation of $X^2$, $E(X^2)$, is $\frac{1063}{6}=177.16667\dots$

{\bf Do Right Now}

{\bf Exercise 4'}: Using the same probability distribution as Exercise 1, namely
$$
P(1)=\frac{1}{8} \quad, \quad  P(2)=\frac{3}{8} ,
P(3)=\frac{1}{16} \quad, \quad  P(4)=\frac{7}{16} 
$$
and the same random variable,X, as in  Exercise 2, namely

if you are promised $10-3i$ dollars if it lands on a face with $i$ dots, 

compute the expectation of $X^2$, $E(X^2)$.

{\bf Important Concept: Variance}

The {\bf Variance} of a random variable, denoted by $Var(X)$ (often also $\sigma^2$) is
the expectation of the square of the ``deviation from expectation'', namely, calling
$E(X)=\mu$
$$
E((X-\mu)^2 ) \quad .
$$
There are two ways to compute it, the first way is to first figure out, explicitly, the
values of the new random variable $(X-\mu)^2$  on each of the atomic events, and then
take the expectation of that, but a better way is via the

{\bf Important Formula}
$$
Var(X)= E(X^2) - E(X)^2 \quad .
$$
(alias $E(X^2)-\mu^2$)

{\bf Note:} $E(X^2)$ is NOT the same as $E(X)^2$, so the difference is not 0!
In $E(X^2)$ you {\bf first} square $X$ getting a brand-new random variable, $X^2$ (like we did above),
and in $E(X)^2$, you {\bf first} find the expectation $E(X)$ (alias $\mu$) and then square it.

In the extreme case where $X$ is the same on every member of our sample space, the expectation equals to that
common value, and then the variance is $0$.

{\bf Proof of the Important Formula}

$$
Var(X)=E((X-\mu)^2)= E(X^2- 2X \mu + \mu^2)= E(X^2)- E(2X\mu)+ E(\mu^2)=
$$
$$
E(X^2)- 2\mu E(X)+ \mu^2 E(1)= E(X^2) -2\mu^2 + \mu^2 \dot 1= E(X^2)- \mu^2 \quad .
$$



{\bf Example IV''}: Using the same probability distribution as Example I, namely
$$
P(1)=\frac{1}{12} \quad, \quad  P(2)=\frac{1}{4} \quad , \quad
P(3)=\frac{1}{20} \quad, \quad  P(4)=\frac{17}{60}  \quad , \quad
P(5)=\frac{1}{18} \quad, \quad  P(6)=\frac{5}{18}  \quad , \quad
$$
and the same random variable,$X$, as in  Example II, compute the variance of $X$, namely $Var(X)$

{\bf Solution to Example IV''}: From Example IV, and Example IV'
$$
E(X)=229/90 \quad , \quad 
E(X^2)=\frac{1063}{6} \quad
$$
Hence
$$
Var(X)=E(X^2)-E(X)^2=
\frac{1063}{6}- (\frac{229}{90})^2=\frac{1382609}{8100}=170.6924691\dots \quad
$$

{\bf Answer to Example IV''}: The variance of $X$, $Var(X)$, is $\frac{1382609}{8100}=170.6924691\dots$ \quad .




{\bf Do Right Now}

{\bf Exercise 4''}: Using the same probability distribution as Exercise 1, namely
$$
P(1)=\frac{1}{8} \quad, \quad  P(2)=\frac{3}{8} ,
P(3)=\frac{1}{16} \quad, \quad  P(4)=\frac{7}{16} 
$$
and the same random variable,X, as in  Exercise 2, namely

if you are promised $10-3i$ dollars if it lands on a face with $i$ dots, 

compute the variance of $X$, $Var(X)$.

{\bf Note:} You may use the answers that you got for Exercises 4 and 4'.


{\bf Do Right Now}

Exercise 5:

In our family, consider the random variable, $X$,  height (in inches) then
$$
X(Celia)=63 \quad, \quad X(Tamar)=64 \quad, \quad X(Hadas)=69 \quad .
$$

If you pick a girl {\bf uniformly} at random (each with probability $\frac{1}{3}$) find the
expectation and variance of the Zeilberger girls.


Exercise 5':

In our family, consider the random variable, $X$,  height (in inches) then
$$
X(Celia)=63 \quad, \quad X(Tamar)=64 \quad, \quad X(Hadas)=69 \quad .
$$

If I pick Celia  with probability $0.1$, Tamar with probability $0.4$ and Hadas with probability $.5$,
find the expectation and variance of the height with this probability distribution.




\bigskip
\hrule
\bigskip
Doron Zeilberger, Department of Mathematics, Rutgers University (New Brunswick), Hill Center-Busch Campus, 110 Frelinghuysen
Rd., Piscataway, NJ 08854-8019, USA. \hfill\break
Email: {\tt zeilberg at math dot rutgers dot edu}   \quad .


\end





















