## Oldies but goodies: Ergodic Theorem

The goal of this blog post is to provide a self-contained proof of the so-called fundamental theorem
of Markov chains.
It states that provided some *basic* properties (spoiler: ergodicity) any Markov chain
random walk converges to the same limiting distribution—irrespective of the starting state.
A hidden goal is to introduce some important tools
for the analysis of the average-reward criterion in MDPs.