Simon Roux

The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modeling

Biological language model performance depends heavily on pretraining data quality, diversity, and size. While metagenomic datasets feature enormous biological diversity, their utilization as pretraining data has been limited due to challenges in data accessibility, quality filtering and deduplicatio…