Asymptotic Theorems for Discrete Markov Chains

Let Xₙ be a discrete time Markov chain with state space S(countably infinite, in general) and initial probability distribution μ⁽⁰⁾= (P(X₀=i1),P(X₀=i2),···,). Can we compute or at least estimate the probabilities P(Xₙ=j|X₀=i) and P(Xₙ=j) for largen? We will discuss this question and give some answers even if there exists periodic states. We will also relate the limiting probabilities with the ergodic type of limits and prove that the computation of the limiting probabilities are a stronger result than that of the ergodic theorem. Finally, we will mention some open problems regarding these limiting probabilities.