Trying to grasp how to compute a $n$-step transition probability
As the title suggests, I am trying to understand how to compute a $n$-step transition probability given a transition probability matrix.
Please understand that this is purely for me to prepare for an exam and not a homework question (It might be too trivial to be a homework question anyway).
Suppose we have simple transition probability matrix, P denoted by:
\begin{bmatrix} 0 & 2/3 & 1/3 \\ 1/2 & 0 & 1/2 \\ 1/2 & 1/2 & 0 & \end{bmatrix}
with three states, $X_{n}=\{0,1,2\}$. So, for example, $P_{01}$, which denotes the one step probability that a state at $0$ will be at state $1$ in the next step is $\frac{2}{3}$ as given in the matrix above.
My professor gave us a formula to calculate the $n$-step transition probability which is: $$P^{(n)}_{ij}=\sum_{l=0}^{\infty} P^{(m)}_{il}P^{(n-m)}_{lj}.$$
Suppose I want to calculate $P^{(3)}_{00}$ and say, $P^{(2)}_{23}$. How can I apply the formula to calculate the above probabilities?
I have also referred to:
but it only had one numerical example of computing a 2-step transition probability. Can someone show me how to do it, step-by-step? Your help is much appreciated!
