Greyson Brothers

Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs

We investigate the design of pooling methods used to summarize the outputs of transformer embedding models, primarily motivated by reinforcement learning and vision applications. This work considers problems where a subset of the input vectors contains requisite information for a downstream task (si…