Jason T. Stillerman

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code

Pretrained language models are integral part of AI applications, but their high computational cost for training limits accessibility. Initiatives such as Bloom and StarCoder aim to democratize access to pretrained models for collaborative community development. Despite these efforts, such models enc…