Punctuation Restoration: A Case Study of BERT-Based Models' Task-Specific Excellence

Abstract

Large Language Models (LLMs) have made remarkable strides in various tasks, yet their suitability for restoring punctuation in ASR-generated transcripts remains under-explored. Through extensive experiments, we demonstrate that LLMs tend to repeatedly use the same punctuation marks and alter input text tokens, in addition to incurring high computational costs. In contrast, a simple two-stage BERT-based method—which first identifies punctuation positions and then predicts the correct punctuation types—achieves the best accuracy with at least a 10x speed improvement.