Answers are grounded in this paper and cite their sources.
A feature of the kind of agency that evolved in animals is that their capacity for goal‐directed behaviour is exercised in the service of persistence and reproduction. These are the purposes (in a specific sense) for which behaviour has been produced. Further, there is feedback between having these purposes and goal‐directed behaviour becoming increasingly sophisticated. Artificial intelligence (AI) systems are unlikely ever to have biological purposes. However, the same kind of feedback process could arise in AI systems if they acquire the right kind of organisation, and it promotes their individual persistence. That possibility is a significant new AI safety consideration.
No takes yet. Share an insight, caveat, or question.
Nicholas J. Shea (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: