Lingjuan Ge

Compressed Self-Attention for Deep Metric Learning with Low-Rank Approximation

In this paper, we apply self-attention (SA) mechanism to boost the performance of deep metric learning. However, due to the pairwise similarity measurement, the cost of storing and manipulating the complete attention maps makes it infeasible for large inputs. To solve this problem, we propose a com…