ALPS: Attention Localization and Pruning Strategy for Efficient Adaptation of Large Language Models
Aligning general-purpose large language models (LLMs) to downstream tasks often incurs significant training adjustment costs. Prior research has explored various avenues to enhance alignment efficiency, primarily through minimal-data training or data-driven activations to identify key attention head…