Dionysis Christopoulos

Attention, Please! Revisiting Attentive Probing Through the Lens of Efficiency

As fine-tuning becomes impractical at scale, probing is emerging as the preferred evaluation protocol. However, standard linear probing can understate the capability of models whose pre-training optimizes local representations rather than an explicit global representation. This motivates attentive p…