Jerry Ngo

What Do Language Models Hear? Probing for Auditory Representations in Language Models

This work explores whether language models encode meaningfully grounded representations of sounds of objects. We learn a linear probe that retrieves the correct text representation of an object given a snippet of audio related to that object, where the sound representation is given by a pretrained a…