When Can We Approximate Wide Contrastive Models with Neural Tangent Kernels and Principal Component Analysis?
Contrastive learning is a paradigm for learning representations from unlabelled data and several recent works have claimed that such models effectively learn spectral embeddings and show relations between (wide) contrastive models and kernel principal component analysis (PCA). However, it is not kno…