← Search

Dominic Masters

3 accepted papers

2024

Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets

ICLR 2024poster

Recently, pre-trained foundation models have enabled significant advancements in multiple fields. In molecular machine learning, however, where datasets are often hand-curated, and hence typically small, the lack of datasets with labeled features, and codebases to manage those datasets, has hindered…

2023

Generating QM1B with PySCF$_{\text{IPU}}$

NeurIPS 2023poster

The emergence of foundation models in Computer Vision and Natural Language Processing have resulted in immense progress on downstream tasks. This progress was enabled by datasets with billions of training examples. Similar benefits are yet to be unlocked for quantum chemistry, where the potential o…

Cited by 0SourcePDFScholar
2021

Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence

NeurIPS 2021poster

We investigate the reasons for the performance degradation incurred with batch-independent normalization. We find that the prototypical techniques of layer normalization and instance normalization both induce the appearance of failure modes in the neural network's pre-activations: (i) layer normaliz…

Cited by 25SourcePDFScholar