The Emergence of Agency in Artificial Intelligence: Science Fiction or a Real Threat?

The article examines the trend of the increasingly widespread use of artificial intelligence (AI) systems, which is a consequence of human civilization's transition to a new stage of development: the civilization of cognitive technologies. This stage is characterized by the satisfaction of needs through the replacement of humans with artificial cognitive systems for the intelligent control of machinery. The paper investigates the possibility and dangers of the emergence of agency in artificial intelligence. The concepts of consciousness, self-consciousness, and agency (subjectivity) are analyzed, and it is argued that their conflation is erroneous. The author proposes an original definition of agency, formulated in terms of consciousness and self-consciousness. A link is established between agency and the possession of needs. In the context of assessing emerging threats to humanity from an alternative intelligence, the case of an entity simultaneously possessing both needs and a high level of intelligence is highlighted as a primary concern. Drawing upon previous research, the author asserts that it is possible for AI to achieve intelligence comparable to human reason, including the ability to solve creative problems. General system theory, particularly the author’s proposed method of multisystem knowledge integration, is presented as a potential means to this end. It is shown that artificial intelligence cannot acquire needs autonomously, and there is no practical necessity for humans to endow it with them, as agency is not required for solving intellectual problems. Moreover, an AI imbued with agency could, under specific circumstances, represent a tangible threat to humanity. Its needs and desires would inevitably enter into conflict with those of people. In a confrontation with an agentive AI of superior intellect, humanity would likely be vanquished. Therefore, the author concludes, artificial intelligence must never be allowed to attain agency.