Maximal Likelihood Estimator

Introduction

Let’s say we observe a realization of $X_{0},X_{1},\ldots,X_{N}$ from a $\mbox{Markov}(\lambda,P)$ . How do we estimate the transition matrix $P$ ? Well lets define some things:

Likelihood

Definition (Log-likelihood)

Given $N+1$ RVs $X_{0},\ldots,X_{N}$ from $\mbox{Markov}(\lambda,P)$ the log-likelihood $L$ is defined as $\log(L)=\log(\lambda_{X_{0}})+\sum\limits_{i,j\in S}\left(\sum\limits_{k=0}^{N-1}\mathbb{1}_{\{X_{k}=i,X_{k+1}=j\}} \right)\log(p_{ij})$

Definition (Maxmal likelihood estimator)

For $i,j\in S$ the maximal likelihood estimator for $P$ is $\hat P_{ij}=\frac{\sum\limits_{k=0}^{N-1}\mathbb{1}_{\{X_{k}=i,X_{k+1}=j\}}}{\sum\limits_{k=0}^{N-1}\mathbb{1}_{\{X_{k}=i\}}}$

Lemma

Assume $P$ is irreducible and positive recurrent. Then for any $i,j\in S$ $P\left(\lim_{n\to\infty}\hat P_{ij}=P_{ij}\right)=1$