CompMTL: Layer-Wise Competitive Multi-Task Learning

Abstract

It is challenging to simultaneously address multiple related tasks using a unified multi-task model and consistently balance conflicts across these tasks. The conflicts arise because each task competes to update the shared module in a manner that can better align with its own requirements. To address the conflicts, existing multi-task learning methods primarily balance task losses or gradients of the shared module, but frequently overlook the differences in layer-wise conflicts, which can enable a more fine-grained conflict-averse. For details, each task establishes its competitive influence over a particular layer by assigning varying degrees of importance to it. Attenuating the gradients of tasks with relatively lower importance during updates in specific layers may not only balance gradient conflicts but also facilitate the progression of other tasks. Based on this, we proposed Layer-wise Competitive Multi-task Learning, wherein multiple tasks compete for gradient update weights within the shared modules of a multi-task model. This approach aims to achieve layer-specific gradient balance by considering each task’s relative importance at different layers within the shared module. Tasks of relatively lower importance for a specific layer will receive smaller gradient updates, thereby facilitating faster convergence for more important tasks.