Refining App Reviews: Dataset, Methodology, and Evaluation
Amrita Singh, Chirag Jain, Mohit Chaudhary, Preethu Rose Anish
Abstract
With the growing number of mobile users, app development has become increasingly lucrative. Reviews on platforms such as Google Play and Apple App Store provide valuable insights to developers, highlighting bugs, suggesting new features, and offering feedback. However, many reviews contain typos, spelling errors, grammar mistakes, and complex sentences, hindering efficient interpretation and slowing down app improvement processes. To tackle this, we introduce RARE (Repository for App review REfinement), a benchmark dataset of 10,000 annotated pairs of original and refined reviews from 10 mobile applications. These reviews were collaboratively refined by humans and large language models (LLMs). We also conducted an evaluation of eight state-of-the-art LLMs for automated review refinement. The top-performing model (Flan-T5) was further used to refine an additional 10,000 reviews, contributing to RARE as a silver corpus.
BibTeX
@inproceedings{singh-etal-2024-refining,
title = "Refining App Reviews: Dataset, Methodology, and Evaluation",
author = "Singh, Amrita and
Jain, Chirag and
Chaudhary, Mohit and
Anish, Preethu Rose",
editor = "Dernoncourt, Franck and
Preo{\c{t}}iuc-Pietro, Daniel and
Shimorina, Anastasia",
booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track",
month = nov,
year = "2024",
address = "Miami, Florida, US",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.emnlp-industry.44/",
doi = "10.18653/v1/2024.emnlp-industry.44",
pages = "595--608"
}