Andrius Buinovskij

The Role of n-gram Smoothing in the Age of Neural Networks

For nearly three decades, language models derived from the n-gram assumption held the state of the art on the task. The key to their success lay in the application of various smoothing techniques that served to combat overfitting. However, when neural language models toppled n-gram models as the bes…