← Search

Tom Joy

5 accepted papers

2024

What Makes and Breaks Safety Fine-tuning? A Mechanistic Study

NeurIPS 2024poster

Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment. To better understand the underlying factors that make models safe via safety fine-tuning, we design a synthetic data generation framework that captures salient aspects of an unsafe input by…

2023

Sample-Dependent Adaptive Temperature Scaling for Improved Calibration

AAAI 2023technical

It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the confidences of the predictions on any input by scaling the logi…

2023

Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration

CVPR 2023poster

The current approach for testing the robustness of object detectors suffers from serious deficiencies such as improper methods of performing out-of-distribution detection and using calibration metrics which do not consider both localisation and classification quality. In this work, we address these…

2022

Learning Multimodal VAEs through Mutual Supervision

ICLR 2022spotlight

Multimodal VAEs seek to model the joint distribution over heterogeneous data (e.g.\ vision, language), whilst also capturing a shared representation across such modalities. Prior work has typically combined information from the modalities by reconciling idiosyncratic representations directly in the…

2021

Capturing Label Characteristics in VAEs

ICLR 2021poster

We present a principled approach to incorporating labels in variational autoencoders (VAEs) that captures the rich characteristic information associated with those labels. While prior work has typically conflated these by learning latent variables that directly correspond to label values, we argue t…