2025
Lambda-Skip Connections: the architectural component that prevents Rank Collapse
ICLR 2025poster
Rank collapse, a phenomenon where embedding vectors in sequence models rapidly converge to a uniform token or equilibrium state, has recently gained at- tention in the deep learning literature. This phenomenon leads to reduced expres- sivity and potential training instabilities due to vanishing grad…