← Search

Will Aitken

2 accepted papers

2024

Collaboration or Corporate Capture? Quantifying NLP’s Reliance on Industry Artifacts and Contributions

ACL 2024long

Impressive performance of pre-trained models has garnered public attention and made news headlines in recent years. Almost always, these models are produced by or in collaboration with industry. Using them is critical for competing on natural language processing (NLP) benchmarks and correspondingly…

2023

Prefix Propagation: Parameter-Efficient Tuning for Long Sequences

ACL 2023short

Parameter-efficient tuning aims to mitigate the large memory requirements of adapting pretrained language models for downstream tasks. For example, one popular method, prefix-tuning, prepends trainable tokens to sequences while freezing the rest of the model’s parameters. Although such models attain…