This paper provides a comprehensive literature survey on the evolving Spiking Neural Network (eSNN) architecture since its introduction in 2006 as a further extension of the ECoS paradigm introduced by Kasabov in 1998. We summarize the functioning of the method, discuss several of its extensions and present a number of applications in which the eSNN method was employed. We focus especially on some proposed extensions that allow the processing of spatio-temporal data and for feature and parameter optimisation of eSNN models to achieve better accuracy on classification/prediction problems and to facilitate new knowledge discovery. Finally, some open problems are discussed and future directions highlighted.