2024
BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models
NeurIPS 2024poster
This work presents BAdam, an optimization method that leverages the block coordinate descent (BCD) framework with Adam's update rule. BAdam offers a memory efficient approach to the full parameter finetuning of large language models. We conduct a theoretical convergence analysis for BAdam in the…