LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking
Deep learning models particularly those based on transformers often employ numerous stacked structures which possess identical architectures and perform similar functions. While effective this stacking paradigm leads to a substantial increase in the number of parameters pos- ing challenges for pract…