To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning
Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow
Abstract
In low-resource settings, model transfer can help to overcome a lack of labeled data for many tasks and domains. However, predicting useful transfer sources is a challenging problem, as even the most similar sources might lead to unexpected negative transfer results. Thus, ranking methods based on task and text similarity — as suggested in prior work — may not be sufficient to identify promising sources. To tackle this problem, we propose a new approach to automatically determine which and how many sources should be exploited. For this, we study the effects of model transfer on sequence labeling across various domains and tasks and show that our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.
BibTeX
@inproceedings{lange-etal-2021-share,
title = "To Share or not to Share: {P}redicting Sets of Sources for Model Transfer Learning",
author = {Lange, Lukas and
Str{\"o}tgen, Jannik and
Adel, Heike and
Klakow, Dietrich},
editor = "Moens, Marie-Francine and
Huang, Xuanjing and
Specia, Lucia and
Yih, Scott Wen-tau",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2021",
address = "Online and Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.emnlp-main.689/",
doi = "10.18653/v1/2021.emnlp-main.689",
pages = "8744--8753"
}