Mohammad Erfan Sadeghi

MambaExtend: A Training-Free Approach to Improve Long Context Extension of Mamba

The inherent quadratic complexity of the attention mechanism in transformer models has driven the research community to explore alternative architectures with sub-quadratic complexity, such as state-space models. Mamba has established itself as a leading model within this emerging paradigm, achievin…