A Continued Pretrained LLM Approach for Automatic Medical Note Generation
Dong Yuan, Eti Rastogi, Gautam Naik, Sree Prasanna Rajagopal, Sagar Goyal, Fen Zhao, Bharath Chintagunta, Jeffrey Ward
Abstract
LLMs are revolutionizing NLP tasks. However, the use of the most advanced LLMs, such as GPT-4, is often prohibitively expensive for most specialized fields. We introduce HEAL, the first continuously trained 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing. Our results demonstrate that HEAL outperforms GPT-4 and PMC-LLaMA in PubMedQA, with an accuracy of 78.4%. It also achieves parity with GPT-4 in generating medical notes. Remarkably, HEAL surpasses GPT-4 and Med-PaLM 2 in identifying more correct medical concepts and exceeds the performance of human scribes and other comparable models in correctness and completeness.
BibTeX
@inproceedings{yuan-etal-2024-continued,
title = "A Continued Pretrained {LLM} Approach for Automatic Medical Note Generation",
author = "Yuan, Dong and
Rastogi, Eti and
Naik, Gautam and
Rajagopal, Sree Prasanna and
Goyal, Sagar and
Zhao, Fen and
Chintagunta, Bharath and
Ward, Jeffrey",
editor = "Duh, Kevin and
Gomez, Helena and
Bethard, Steven",
booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)",
month = jun,
year = "2024",
address = "Mexico City, Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.naacl-short.47/",
doi = "10.18653/v1/2024.naacl-short.47",
pages = "565--571"
}