LMS: Past, Present and Future

Abstract

We sketch some aspects of LMS particularly focussing on some puzzles, problems and potentials. We explain the 'independence heuristic' by means of averaging theory for which we give a simple expose'. We suggest an explanation for the 'urban myth' that white noise only performance formulae can be used as surrogates for the correct performance formulae which involve autocorrelations.We then extend the discussion to more recent network versions such as diffusion LMS. We show that the sharing aspect of network LMS algorithms induces a two time scale structure (something not yet widely known) and exhibit its consequences. We comment on the recent upsurge of interest in `online learning' in machine learning and its failure to reference the adaptive signal processing literature. Finally we speculate on future developments of LMS.