Let $X_1, X_2, \dots$ be independent and identically distributed random variables with mean 0 and common variance $\sigma^2 = \operatorname{Var}(X_1),$ and assume the common moment-generating function $M_{X_1}(t)$ exists. Let \[Z_n = \frac{X_1 + \dots + X_n}{\sigma\sqrt{n}}. \]Prove the central limit theorem by following the outline:
Goal: Show that the limit of the moment generating function of $Z_n$ is $e^{\frac{1}{2}t^2}.$ Since the limit of the moment generating function matches that of the normal distribution with mean 0 and variance 1, the limit of $Z_n$ must be a normal random variable with mean 0 and variance 1 by Lévy’s continuity theorem.

1. Show that the moment generating function of $Z_n$ is \[M_{Z_n}(t) = \left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right) \right)^n. \]
2. Show that the moment generating function has a Taylor series expansion around $t=0$ of the form \[ M_{X_1}(t) = a + bt + ct^2 + o(t^2), \] where $o(t^2)$ is a function that satisfies $\lim_{t\to0} o(t^2)/t^2 =0.$ Find $a, b,$ and $c$ in terms of $\sigma^2$.
3. Show that \[\lim_{n\to\infty}M_{Z_n}(t) = e^{\frac{1}{2}t^2}. \]

---

Proof:

1. Using the independence of $X_i$, we have \begin{align} M_{Z_n}(t) &= \mathbf{E}\left[e^{tZ_n}\right]\\ &= \mathbf{E}\left[e^{(\frac{t}{\sigma\sqrt{n}})(X_1 + \dots + X_n)}\right]\\ &= \mathbf{E}\left[e^{(\frac{t}{\sigma\sqrt{n}})X_1}\times\dots\times e^{(\frac{t}{\sigma\sqrt{n}})X_n}\right]\\ &= \mathbf{E}\left[e^{(\frac{t}{\sigma\sqrt{n}})X_1}\right]\times \dots\times \mathbf{E}\left[e^{(\frac{t}{\sigma\sqrt{n}})X_n}\right]&\text{if $X$ and $Y$ are independent, then $\mathbf{E}[f(X)g(Y)] = \mathbf{E}[f(X)]\mathbf{E}[g(Y)]$.}\\ &= \left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right) \right)^n. \end{align}
2. For a given random variable $X$, the function $e^{tX}$ has the power series expansion \[ e^{tX} = 1 + tX + \frac{(tX)^2}{2!} + \dots. \] Then by applying linear of expectation, the moment-generating function $\mathbf{E}\left[e^{tX}\right]$ can be written as \[\mathbf{E}\left[e^{tX}\right] = 1 + \mathbf{E}\left[X\right]t + \mathbf{E}\left[X^2\right]\frac{t^2}{2} + \dots. \] So for case of $X_i$, we have that \[M_{X_i}(t) = a + bt + ct^2 + o(t^2), \] where \[a = 1, \qquad b = \mathbf{E}[X_i] = 0,\] and \[c = \frac{\mathbf{E}[X_i^2]}{2} = \frac{\sigma^2}{2}. \]
3. Combining the previous two results, we get \[ M_{Z_n}(t) = \left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right) \right)^n = \left( a + \frac{bt}{\sigma\sqrt{n}} + \frac{ct^2}{\sigma^2n} + o\left(\frac{t^2}{\sigma^2 n}\right)\right)^n. \] Knowing what $a, b,$ and $c$ are, we get \[ M_{Z_n}(t) = \left(1 +\frac{t^2}{2n} + o\left(\frac{t^2}{\sigma^2 n}\right)\right)^n. \] We can throw away $o\left(\frac{t^2}{\sigma^2 n}\right)$ since $\lim_{n\to\infty}o\left(\frac{t^2}{\sigma^2 n}\right) = 0$, so we take the limit \[ \lim_{n\to\infty}\left(1 +\frac{t^2}{2n}\right)^n. \] Since $\left(1+\frac{x}{n}\right)^n = e^x,$ we find that the limit is \[ \lim_{n\to\infty} M_{Z_n}(t) = e^{\frac{1}{2}t^2}. \]

   ---

   Longer version:
   This section is for those who did not like that I threw away $o\left(\frac{t^2}{\sigma^2 n}\right)$ without detailed justification.
   The moment-generating function $M_{X_1}(t)$ has the power series expansion \[M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right) = 1 +\frac{t^2}{2n} + \sum_{k=3}^\infty a_k \frac{t^k}{n^{k/2}} \] for some constants $a_k.$ Let us consider $M_{Z_n}(t) = \left(M_{X_1}\left(\tfrac{t}{\sigma\sqrt{n}}\right)\right)^n.$ We write \[\lim_{n\to\infty}\left(M_{X_1}\left(\tfrac{t}{\sigma\sqrt{n}}\right)\right)^n = \lim_{n\to\infty} e^{n\ln\left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)\right)}.\] Since the exponential function is continuous, we pass the limit into the exponent. Now we consider \[ \lim_{n\to\infty} n\ln\left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)\right). \] This is an indeterminate form, so we need to apply l'Hopital's rule. We get \begin{align} \lim_{n\to\infty}n\ln\left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)\right) &= \lim_{n\to\infty} \frac{\ln\left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)\right)}{1/n} \\ &= \lim_{n\to\infty}\frac{\frac{d}{dn}\left[M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)\right]}{M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)}\times \frac{1}{-1/n^2}& \text{l'Hopital's rule}\\ &= \lim_{n\to\infty}\frac{-n^2\left(-\frac{t^2}{2n^2} - \sum_{k=3}^\infty a_k \frac{k}{2} \frac{t^{k}}{n^{(k+2)/2}} \right)}{M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)}&\text{differentiate $M_{X_1}\left(\tfrac{t}{\sigma\sqrt{n}}\right)$ term by term}\\ &= \lim_{n\to\infty}\frac{\left(\frac{t^2}{2} + \sum_{k=3}^\infty a_k \frac{k}{2}\frac{t^{k}}{n^{(k-2)/2}} \right)}{M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right)}\\ &= \frac{t^2/2+0}{1}.& M_{X_i}(0) = 1 \end{align} Putting it all together, we have that \[\lim_{n\to\infty} \left(M_{X_1}\left(\frac{t}{\sigma\sqrt{n}}\right) \right)^n = e^{\frac{1}{2}t^2}. \]