XAI-Lyricist: Improving the Singability of AI-Generated Lyrics with Prosody Explanations
Qihao Liang, Xichu Ma, Finale Doshi-Velez, Brian Lim, Ye Wang
Abstract
Explaining the singability of lyrics is an important but missing ability of language models (LMs) in song lyrics generation. This ability allows songwriters to quickly assess if LM-generated lyrics can be sung harmoniously with melodies and helps singers align lyrics with melodies during practice. This paper presents XAI-Lyricist, leveraging musical prosody to guide LMs in generating singable lyrics and providing human-understandable singability explanations. We employ a Transformer model to generate lyrics under musical prosody constraints and provide demonstrations of the lyrics' prosody patterns as singability explanations. XAI-Lyricist is evaluated by computational metrics (perplexity, prosody-BLEU) and a human-grounded study (human ratings, average time and number of attempts for singing). Experimental results show that musical prosody can significantly improve the singability of LM-generated lyrics. A controlled study with 14 singers also confirms the usefulness of the provided explanations in helping them to interpret lyrical singability faster than reading plain text lyrics.
BibTeX
@inproceedings{ijcai2024p872,
title = {XAI-Lyricist: Improving the Singability of AI-Generated Lyrics with Prosody Explanations},
author = {Liang, Qihao and Ma, Xichu and Doshi-Velez, Finale and Lim, Brian and Wang, Ye},
booktitle = {Proceedings of the Thirty-Third International Joint Conference on
Artificial Intelligence, {IJCAI-24}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
editor = {Kate Larson},
pages = {7877--7885},
year = {2024},
month = {8},
note = {Human-Centred AI},
doi = {10.24963/ijcai.2024/872},
url = {https://doi.org/10.24963/ijcai.2024/872},
}