Nicholas Egan

Play the Shannon Game with Language Models: A Human-Free Approach to Summary Evaluation

The goal of a summary is to concisely state the most important information in a document. With this principle in mind, we introduce new reference-free summary evaluation metrics that use a pretrained language model to estimate the information content shared between a document and its summary. These…