Towards a Theory of AI Personhood
Abstract
I am a person and so are you. Philosophically we sometimes grant personhood to non-human animals, and entities such as sovereign states or corporations can legally be considered persons. But when, if ever, should we ascribe personhood to AI systems? In this paper, we outline necessary conditions for AI personhood, focusing on agency, theory-of-mind, and self-awareness. We discuss evidence from the machine learn- ing literature regarding the extent to which contemporary AI systems, such as language models, satisfy these conditions, finding the evidence surprisingly inconclusive. If AI systems can be considered persons, then typical fram- ings of AI alignment may be incomplete. Whereas agency has been discussed at length in the literature, other aspects of per- sonhood have been relatively neglected. AI agents are often assumed to pursue fixed goals, but AI persons may be self- aware enough to reflect on their aims, values, and positions in the world and thereby induce their goals to change. We highlight open research directions to advance the understand- ing of AI personhood and its relevance to alignment. Finally, we reflect on the ethical considerations surrounding the treat- ment of AI systems. If AI systems are persons, then seeking control and alignment may be ethically untenable.
BibTeX
@article{Ward_2025, title={Towards a Theory of AI Personhood}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/34982}, DOI={10.1609/aaai.v39i26.34982}, abstractNote={I am a person and so are you. Philosophically we sometimes
grant personhood to non-human animals, and entities such
as sovereign states or corporations can legally be considered
persons. But when, if ever, should we ascribe personhood to
AI systems? In this paper, we outline necessary conditions
for AI personhood, focusing on agency, theory-of-mind, and
self-awareness. We discuss evidence from the machine learn-
ing literature regarding the extent to which contemporary AI
systems, such as language models, satisfy these conditions,
finding the evidence surprisingly inconclusive.
If AI systems can be considered persons, then typical fram-
ings of AI alignment may be incomplete. Whereas agency has
been discussed at length in the literature, other aspects of per-
sonhood have been relatively neglected. AI agents are often
assumed to pursue fixed goals, but AI persons may be self-
aware enough to reflect on their aims, values, and positions
in the world and thereby induce their goals to change. We
highlight open research directions to advance the understand-
ing of AI personhood and its relevance to alignment. Finally,
we reflect on the ethical considerations surrounding the treat-
ment of AI systems. If AI systems are persons, then seeking
control and alignment may be ethically untenable.}, number={26}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Ward, Francis Rhys}, year={2025}, month={Apr.}, pages={27680-27688} }