2026
MAR: EFFICIENT LARGE LANGUAGE MODELS VIA MODULE-AWARE ARCHITECTURE REFINEMENT
ICASSP 2026poster
Large Language Models (LLMs) excel across diverse domains but suffer from high energy costs due to quadratic attention and dense Feed-Forward Network (FFN) operations. To address these issues, we propose Module-aware Architecture Refinement (MAR), a two-stage framework that integrates State Space Mo…