Leyla Candogan

Spatial Priors via Space Filling Curves for Small and Limited Data Vision Transformers

Though Vision Transformers (ViTs) have become the dominant backbone in many computer vision tasks, due to permutation invariance, their attention mechanism lacks explicit spatial inductive biases. This become particularly important in two common settings: when model capacity is small or training dat…