Lorenzo Giambagli

How a Student becomes a Teacher: learning and forgetting through Spectral methods

In theoretical Machine Learning, the teacher-student paradigm is often employed as an effective metaphor for real-life tuition. A student network is trained on data generated by a fixed teacher network until it matches the instructor’s ability to cope with the assigned task. The above scheme proves…