Neural Document Segmentation Using Weighted Sliding Windows with Transformer Encoders
Saeed Abbasi, Aijun An, Heidar Davoudi, Ron Di Carlantonio, Gary Farmaner
Abstract
We introduce a novel Transformer-based method for document segmentation, tailored for practical, real-world applications. This method utilizes overlapping text sequences with a unique position-aware weighting mechanism to enhance segmentation accuracy. Through comprehensive experiments on both public and proprietary datasets, we demonstrate significant improvements, establishing new state-of-the-art standards by achieving up to a 10% increase in segmentation F1 score compared to existing methods. Additionally, we explore the application of our segmentation method in downstream retrieval-augmented question answering tasks, where it improves the quality of generated responses by 5% while achieving up to four times greater efficiency. These results underscore our model’s potential as a robust and scalable solution for real-world text segmentation challenges.
BibTeX
@inproceedings{abbasi-etal-2025-neural,
title = "Neural Document Segmentation Using Weighted Sliding Windows with Transformer Encoders",
author = "Abbasi, Saeed and
An, Aijun and
Davoudi, Heidar and
Di Carlantonio, Ron and
Farmaner, Gary",
editor = "Rambow, Owen and
Wanner, Leo and
Apidianaki, Marianna and
Al-Khalifa, Hend and
Eugenio, Barbara Di and
Schockaert, Steven and
Darwish, Kareem and
Agarwal, Apoorv",
booktitle = "Proceedings of the 31st International Conference on Computational Linguistics: Industry Track",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-industry.67/",
pages = "807--816"
}