Bob M. Jacobs

Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans’ expressed preferences over multiple outputs…