2024
ETAS: Zero-Shot Transformer Architecture Search via Network Trainability and Expressivity
ACL 2024findings
Transformer Architecture Search (TAS) methods aim to automate searching for the optimal Transformer architecture configurations for a given task. However, they are impeded by the prohibitive cost of evaluating Transformer architectures. Recently, several Zero-Shot TAS methods have been proposed to m…