Construction

Recalling our Q-Matrix $Q$. We can construct our CTMC using the following intuition. Given a continuous time-process $\{X_{t}:t\ge0\}$ once $X_{t}$ enters the state $i$, we start a clock $\mathcal{C}^{i,j}$ for each state $j\not=i$. These clocks are independent and $\mathcal{C}^{i,j}\sim\mbox{Exp}(q_{ij})$. If the $k^{th}$ clock expires first then the process jumps to state $k$. Now we can denote $$\begin{align*} K=\arg\min_{j\not=i}\mathcal{C}^{i,j}, \ \mathcal{C}=\inf_{j\not=i}\mathcal{C}^{i,j} \end{align*}$$We know that $$\mathcal{C}\sim\mbox{Exp}(\nu_{i}), \ P(K=k)=\frac{q_{ik}}{\nu_{i}}$$and $\mathcal{C}\perp\!\!\!\perp K$, i.e. $\mathcal{C}$ represents the shortest timer length or the time taken to leave state $i$, while $K$ is the first timer to ring.

Developing the idea further

For a continuous-time MC we need to define some initial distribution $\lambda$ and a Q-Matrix $Q$ of the state space $I$. Let us define the following: $$\begin{align*} \{T_{n}:n\ge1\}&\stackrel{iid}\sim\mbox{Exp}(1)\\\{Y_{n}:n\ge0\}&\sim\mbox{Markov}(\lambda,\Pi) \end{align*}$$where $Y_{i}$ is the jump chain for $X_{t}$ and $\Pi$ is the transition matrix associated with $Q$. Then for $n\ge1$ we define the holding times to be $$S_{n}=\frac{T_{n}}{\nu(Y_{n-1})}$$where $\nu(Y_{n-1})=\nu_{Y_{n-1}}$. Conditional on $\{Y_{n-1}=i\}$ we have that $S_{n}\sim\mbox{Exp}(\nu_{i})$. Now let our jump times be defined as $J_{0}=0$ and $J_{n}=\sum\limits_{k=1}^{n}S_{k}$. Then for $t\ge0$, $$X_{t}=\begin{cases}Y_{n}&J_{n}\le t<J_{n+1}\\\infty&\mbox{otherwise} \end{cases}$$ >[!def] Markov Process — continuous time >Assume the Q-Matrix is non-explosive, i.e. $$P_{i}\left(\sum\limits_{n=1}^{\infty} \frac{T_{n}}{\nu(Y_{n-1})}=\infty\right)=1$$ We will refer to $\{X_{t}:t\ge0\}$ as a continuous-time Markov chain with the initial distribution $\lambda$ and generator $Q$, and write $$\{X_{t}:t\ge0\}\sim\mbox{Markov}(\lambda,Q)$$$Q$ is called the generator of $\{X_{t}:t\ge0\}$.

Transition probabilities