Liuzhicheng

DiJiang: Efficient Large Language Models through Compact Kernelization

In an effort to reduce the computational load of Transformers, research on linear attention has gained significant momentum. However, the improvement strategies for attention mechanisms typically necessitate extensive retraining, which is impractical for large language models with a vast array of pa…