The Derivation of Markov Chain Properties Using Generalized Matrix Inverses

Abstract

In many stochastic models a Markov chain is present either directly or indirectly through some form of embedding. The analysis of many problems of interest associated with these models, eg. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, often requires the solution of a system of linear equations involving I – P, where P is the transition matrix of a finite, irreducible, discrete time Markov chain. Generalized matrix inverses play an important role in the solution of such singular sets of equations. In this presentation we survey the application of generalized inverses to the aforementioned problems focussing primarily on Markov chains.