Tatsuya Sagawa

How Well Do Large-Scale Chemical Language Models Transfer to Downstream Tasks?

Chemical Language Models (CLMs) pre-trained on large-scale molecular data are widely used for molecular property prediction. However, the common belief that increasing training resources, such as model size, dataset size, and training compute, improves both pre-training loss and downstream task perf