Definition (Characteristic function)
For X X X rv with X ∼ μ X\sim\mu X ∼ μ characteristic function is defined as φ X ( t ) = E [ e i t X ] = E [ cos t X ] + i E [ sin t X ] = ∫ R e i t x μ ( d x ) \varphi_{X}(t)=\mathbb{E}[e^{itX}]=\mathbb{E}[\cos tX]+i\mathbb{E}[\sin tX]=\int\limits _{\mathbb{R}}e^{itx} \, \mu(dx) φ X ( t ) = E [ e i tX ] = E [ cos tX ] + i E [ sin tX ] = R ∫ e i t x μ ( d x )
Proposition
For a ∈ R , b > 0 a\in \mathbb{R},b>0 a ∈ R , b > 0 rv X X X φ a + b X ( t ) = e i a t φ b X ( t ) \varphi_{a+bX}(t)=e^{iat}\varphi_{bX}(t) φ a + b X ( t ) = e ia t φ b X ( t )
\begin{proof} φ a + b X ( t ) = E [ e i t a e i t b X ] = e i t a E [ e i t b X ] = e i t a φ b X ( t ) \varphi_{a+bX}(t)=\mathbb{E}[e^{ita}e^{itbX}]=e^{ita}\mathbb{E}[e^{itbX}]=e^{ita}\varphi_{bX}(t) φ a + b X ( t ) = E [ e i t a e i t b X ] = e i t a E [ e i t b X ] = e i t a φ b X ( t ) \end{proof}
Theorem
If X 1 ⊥ ⊥ X 2 X_{1}\perp\!\!\!\perp X_{2} X 1 ⊥ ⊥ X 2 φ X 1 + X 2 ( t ) = φ X 1 ( t ) ⋅ φ X 2 ( t ) \varphi_{X_{1}+X_{2}}(t)=\varphi_{X_{1}}(t)\cdot\varphi_{X_{2}}(t) φ X 1 + X 2 ( t ) = φ X 1 ( t ) ⋅ φ X 2 ( t )
\begin{proof} φ X 1 + X 2 = E [ e i t X 1 e i t X 2 ] = E [ e i t X 1 ] E [ e i t X 2 ] = φ X 1 ( t ) φ X 2 ( t ) \varphi_{X_{1}+X_{2}}=\mathbb{E}[e^{itX_{1}}e^{itX_{2}}]=\mathbb{E}[e^{itX_{1}}]\mathbb{E}[e^{itX_{2}}]=\varphi_{X_{1}}(t)\varphi_{X_{2}}(t) φ X 1 + X 2 = E [ e i t X 1 e i t X 2 ] = E [ e i t X 1 ] E [ e i t X 2 ] = φ X 1 ( t ) φ X 2 ( t ) independence \end{proof}
Definition (Fourier Inverse)
For f ∈ L 1 ( R , B , λ ) f\in L^{1}(\mathbb{R},\mathcal{B},\lambda) f ∈ L 1 ( R , B , λ ) Fourier transform is defined as f ^ ( t ) = ∫ R e i t u f ( u ) d u \hat{f}(t)=\int\limits_{\mathbb{R}}e^{itu}f(u) \, du f ^ ( t ) = R ∫ e i t u f ( u ) d u g ∈ L 1 ( R , B , λ ) g\in L^{1}(\mathbb{R},\mathcal{B},\lambda) g ∈ L 1 ( R , B , λ ) Fourier inverse is defined as g ˇ ( x ) = 1 2 π ∫ R e − i t x g ( t ) d t \check{g}(x)=\frac{1}{2\pi}\int\limits _{\mathbb{R}}e^{-itx}g(t) \, dt g ˇ ( x ) = 2 π 1 R ∫ e − i t x g ( t ) d t
Proposition
If h ( u ) = h μ , σ ( u ) = 1 2 π σ e − ( u − μ ) 2 / 2 σ 2 h(u)=h_{\mu,\sigma}(u)=\frac{1}{\sqrt{ 2\pi }\sigma}e^{-(u-\mu)^{2}/2\sigma^{2}} h ( u ) = h μ , σ ( u ) = 2 π σ 1 e − ( u − μ ) 2 /2 σ 2 ( h ^ ) ˇ ( x ) = h ( x ) ∀ x ∈ R \check{(\hat{h})}(x)=h(x)\quad\forall x\in \mathbb{R} ( h ^ ) ˇ ( x ) = h ( x ) ∀ x ∈ R
\begin{proof} First note that for Z ∼ N ( 0 , 1 ) Z\sim \mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) φ Z ( t ) = E Z [ e i t Z ] = ∫ R e i t u e − u 2 2 π d u = e t 2 / 2 ∫ R e − ( u − i t ) 2 / 2 2 π d u = e t 2 2 ∫ R e − u 2 2 2 π d u = e t 2 2 \begin{align*}
\varphi_{Z}(t)&= \mathbb{E}_{Z}[e^{itZ}]=\int\limits _{\mathbb{R}}e^{itu} \frac{e^{-u^{2}}}{\sqrt{ 2\pi }} \, du\\
&= e^{t^{2}/2}\int\limits_{\mathbb{R}} \frac{e^{-(u-it)^{2}/2}}{\sqrt{ 2\pi }} \, du\\
&= e^{\frac{t^{2}}{2}}\int\limits _{\mathbb{R}} \frac{e^{\frac{-u^{2}}{2}}}{\sqrt{ 2\pi }} \, du \\
&= e^{\frac{t^{2}}{2}}
\end{align*} φ Z ( t ) = E Z [ e i tZ ] = R ∫ e i t u 2 π e − u 2 d u = e t 2 /2 R ∫ 2 π e − ( u − i t ) 2 /2 d u = e 2 t 2 R ∫ 2 π e 2 − u 2 d u = e 2 t 2
Then note that W ∼ N ( μ , σ 2 ) ≡ μ + σ N ( 0 , 1 ) W\sim \mathcal{N}(\mu,\sigma^{2})\equiv\mu+\sigma \mathcal{N}(0,1) W ∼ N ( μ , σ 2 ) ≡ μ + σ N ( 0 , 1 ) φ W ( t ) = φ μ + σ Z = e i μ t e − σ 2 t 2 2 \varphi_{W}(t)=\varphi_{\mu+\sigma Z}=e^{i\mu t}e^{-\frac{\sigma^{2}t^{2}}{2}} φ W ( t ) = φ μ + σ Z = e i μ t e − 2 σ 2 t 2
so h ^ ( t ) = ∫ R e i t u 1 2 π e − ( u − μ ) 2 / 2 σ 2 d u = φ N ( μ , σ 2 ) ( t ) = e i μ t e − σ 2 t 2 2 \begin{align*}
\hat{h}(t)&= \int\limits _{\mathbb{R}}e^{itu} \frac{1}{\sqrt{ 2\pi }}e^{-(u-\mu)^{2}/2\sigma^{2}} \, du \\
&=\varphi_{\mathcal{N}(\mu,\sigma^{2})}(t)\\
&= e^{i\mu t}e^{-\frac{\sigma^{2}t^{2}}{2}}
\end{align*} h ^ ( t ) = R ∫ e i t u 2 π 1 e − ( u − μ ) 2 /2 σ 2 d u = φ N ( μ , σ 2 ) ( t ) = e i μ t e − 2 σ 2 t 2 h ^ ˇ ( x ) = 1 2 π ∫ R e i μ t e − σ 2 t 2 2 e − i t x d t = 1 2 π σ ∫ R e i t ( μ − x ) e − t 2 2 σ − 2 2 π σ − 1 d x = 1 2 π σ φ N ( 0 , σ − 2 ) ( μ − x ) = 1 2 π σ e − ( x − μ ) 2 2 σ 2 \begin{align*}
\check{\hat{h}}(x)&= \frac{1}{2\pi}\int\limits _{\mathbb{R}}e^{i\mu t}e^{\frac{-\sigma^{2}t^{2}}{2}}e^{-itx} \, dt\\
&= \frac{1}{\sqrt{ 2\pi }\sigma}\int\limits_{\mathbb{R}} e^{it(\mu-x)}\frac{e^{\frac{-t^{2}}{2\sigma^{-2}}}}{\sqrt{ 2\pi }\sigma^{-1}} \, dx \\
&= \frac{1}{\sqrt{ 2\pi }\sigma}\varphi_{\mathcal{N}(0,\sigma^{-2})}(\mu-x)\\
&= \frac{1}{\sqrt{ 2\pi }\sigma}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}
\end{align*} h ^ ˇ ( x ) = 2 π 1 R ∫ e i μ t e 2 − σ 2 t 2 e − i t x d t = 2 π σ 1 R ∫ e i t ( μ − x ) 2 π σ − 1 e 2 σ − 2 − t 2 d x = 2 π σ 1 φ N ( 0 , σ − 2 ) ( μ − x ) = 2 π σ 1 e − 2 σ 2 ( x − μ ) 2
\end{proof}
Cor
Let V = span { h μ , σ : μ ∈ R , σ > 0 } V=\text{span}\{ h_{\mu,\sigma}:\mu \in \mathbb{R},\sigma>0 \} V = span { h μ , σ : μ ∈ R , σ > 0 } f ∈ V f\in V f ∈ V f ^ ˇ = f and f ˇ ^ = f \check{\hat{f}}=f\text{ and }\hat{\check{f}}=f f ^ ˇ = f and f ˇ ^ = f
Theorem (Levy continuity theorem (special case))
For X 1 , X 2 , … , X X_{1},X_{2},\dots,X X 1 , X 2 , … , X rv s. X n → d X ⟺ lim n → ∞ φ X n ( t ) = φ X ( t ) ∀ t ∈ R X_{n}\xrightarrow{d}X\iff\lim_{ n \to \infty } \varphi_{X_{n}}(t)=\varphi_{X}(t)\quad \forall t\in \mathbb{R} X n d X ⟺ n → ∞ lim φ X n ( t ) = φ X ( t ) ∀ t ∈ R
\begin{proof} We first state and prove this lemma: >[!lem] >For any ψ ∈ C c 2 ( R ) \psi \in C_{c}^{2}(\mathbb{R}) ψ ∈ C c 2 ( R ) f ∈ V f\in V f ∈ V ∀ ϵ > 0 \forall\epsilon>0 ∀ ϵ > 0 ∣ ψ ( x ) − f ( x ) ∣ < ϵ ∀ x ∈ R |\psi(x)-f(x)|<\epsilon\quad \forall x\in \mathbb{R} ∣ ψ ( x ) − f ( x ) ∣ < ϵ ∀ x ∈ R
then by Portmanteau’s Theorem we have X n → d X ⟹ E [ e i t X n ] → E [ e i t X ] X_{n}\xrightarrow{d}X\implies \mathbb{E}[e^{itX_{n}}]\to \mathbb{E}[e^{itX}] X n d X ⟹ E [ e i t X n ] → E [ e i tX ] E [ e i t X n ] → E [ e i t X ] , ∀ t \mathbb{E}[e^{itX_{n}}]\to \mathbb{E}[e^{itX}],\,\forall t E [ e i t X n ] → E [ e i tX ] , ∀ t f ∈ V f\in V f ∈ V E [ f ( X n ) ] → E [ f ( X ) ] \mathbb{E}[f(X_{n})]\to \mathbb{E}[f(X)] E [ f ( X n )] → E [ f ( X )] g ( t ) = f ˇ ( t ) g(t)=\check{f}(t) g ( t ) = f ˇ ( t ) f ( x ) = ∫ R e i t x g ( t ) d t f(x)=\int\limits _{\mathbb{R}} e^{itx}g(t) \, dt f ( x ) = R ∫ e i t x g ( t ) d t ∫ R g ( t ) E [ e i t X n ] d t = E [ ∫ e i t X n g ( t ) d t ] = E [ f ( X n ) ] \int\limits _{\mathbb{R}}g(t)\mathbb{E}[e^{itX_{n}}] \, dt =\mathbb{E}\left[ \int\limits e^{itX_{n}}g(t) \, dt \right]=\mathbb{E}[f(X_{n})] R ∫ g ( t ) E [ e i t X n ] d t = E [ ∫ e i t X n g ( t ) d t ] = E [ f ( X n )] Fubini-Tonelli and ∫ g ( t ) E [ e i t X ] d t = E [ f ( X ) ] \int\limits g(t)\mathbb{E}[e^{itX}] \, dt =\mathbb{E}[f(X)] ∫ g ( t ) E [ e i tX ] d t = E [ f ( X )] ∣ g ( t ) E [ e i t X n ] ∣ ≤ ∣ g ( t ) ∣ ∈ L 1 ( R ) ∀ n \left| g(t)\mathbb{E}[e^{itX_{n}}] \right| \le |g(t)|\in L^{1}(\mathbb{R})\quad\forall n g ( t ) E [ e i t X n ] ≤ ∣ g ( t ) ∣ ∈ L 1 ( R ) ∀ n Dominated Convergence Theorem we have lim n → ∞ E [ f ( X n ) ] = E [ f ( X ) ] \lim_{ n \to \infty } \mathbb{E}[f(X_{n})]=\mathbb{E}[f(X)] n → ∞ lim E [ f ( X n )] = E [ f ( X )] ψ ∈ C c 2 ( R ) \psi \in C_{c}^{2}(\mathbb{R}) ψ ∈ C c 2 ( R ) lim n → ∞ E [ ψ ( X n ) ] = E [ ψ ( X ) ] \lim_{ n \to \infty }\mathbb{E}[\psi(X_{n})]=\mathbb{E}[\psi(X)] lim n → ∞ E [ ψ ( X n )] = E [ ψ ( X )] ϵ > 0 \epsilon>0 ϵ > 0 f ∈ V f\in V f ∈ V ∣ ψ ( x ) < f ( x ) ∣ < ϵ ∀ x |\psi(x)<f(x)|<\epsilon\quad\forall x ∣ ψ ( x ) < f ( x ) ∣ < ϵ ∀ x ∣ E [ ψ ( X ) ] − E [ f ( X ) ] ∣ < ϵ \left| \mathbb{E}[\psi(X)]-\mathbb{E}[f(X)] \right|<\epsilon ∣ E [ ψ ( X )] − E [ f ( X )] ∣ < ϵ X n X_{n} X n E [ ψ ( X n ) ] − E [ ψ ( X ) ] = E [ f ( X n ) ] − E [ f ( X ) ] + O ≤ 2 ( ϵ ) → 0 \mathbb{E}[\psi(X_{n})]-\mathbb{E}[\psi(X)]=\mathbb{E}[f(X_{n})]-\mathbb{E}[f(X)]+\mathcal{O}_{\le 2}(\epsilon)\to0 E [ ψ ( X n )] − E [ ψ ( X )] = E [ f ( X n )] − E [ f ( X )] + O ≤ 2 ( ϵ ) → 0 Portmanteau’s Theorem we have X n → d X X_{n}\xrightarrow{d}X X n d X \end{proof}
Cor
If X , Y X,Y X , Y rv s such that φ Y ( t ) = φ X ( t ) , ∀ t \varphi_{Y}(t)=\varphi_{X}(t),\forall t φ Y ( t ) = φ X ( t ) , ∀ t Y ∼ X Y\sim X Y ∼ X
\begin{proof} Set X n = Y X_{n}=Y X n = Y n n n \end{proof}
Proposition
For rv X ∈ L k X\in L^{k} X ∈ L k k ≥ 1 k\ge 1 k ≥ 1 d ℓ d t ℓ φ X ( t ) = i ℓ E [ X ℓ e i t X ] ℓ ≤ k \frac{d^{\ell}}{dt^{\ell}}\varphi_{X}(t)=i^{\ell}\mathbb{E}[X^{\ell}e^{itX}]\quad \ell\le k d t ℓ d ℓ φ X ( t ) = i ℓ E [ X ℓ e i tX ] ℓ ≤ k
\begin{proof}
\end{proof}
Proposition
If X ∈ L k X\in L^{k} X ∈ L k k k k φ X ( k ) ( t ) \varphi_{X}^{(k)}(t) φ X ( k ) ( t )
\begin{proof}
\end{proof}
Theorem (Central Limit Theorem)
For X 1 , X 2 , … X_{1},X_{2},\dots X 1 , X 2 , … iid rv s with E [ X i ] = μ Var ( X i ) = σ 2 ∀ i \mathbb{E}[X_{i}]=\mu\quad \text{Var}(X_{i})=\sigma^{2}\quad \forall i E [ X i ] = μ Var ( X i ) = σ 2 ∀ i S n = ∑ i = 1 n X i S_{n}=\sum_{i=1}^{n}X_{i} S n = ∑ i = 1 n X i S n − n μ σ n → d Y for Y ∼ N ( 0 , 1 ) \frac{S_{n}-n\mu}{\sigma \sqrt{ n }}\xrightarrow{d}Y\quad\text{for }Y\sim \mathcal{N}(0,1) σ n S n − n μ d Y for Y ∼ N ( 0 , 1 ) 1 n ∑ i = 1 n X i → d N ( μ , σ 2 ) \frac{1}{\sqrt{ n }}\sum_{i=1}^{n}X_{i}\xrightarrow{d}\mathcal{N}(\mu,\sigma^{2}) n 1 i = 1 ∑ n X i d N ( μ , σ 2 )