Kirsten Morehouse

Position: Rethinking LLM Bias Probing Using Lessons from the Social Sciences

The proliferation of LLM bias probes introduces three challenges: we lack (1) principled criteria for selecting appropriate probes, (2) a system for reconciling conflicting results across probes, and (3) formal frameworks for reasoning about when and why experimental findings will generalize to real…