Why is the question of whether a machine could be conscious (have subjective experience) or sentient (have valenced experience) raised in science? This paper addresses three closely related questions: (a) Why strive to develop conscious artificial systems? (b) Is artificial consciousness possible? (c) Could artificial consciousness be recognised? Starting with a brief historical overview of the construction of mental hierarchies within Western cultures, psychological and social driving forces for developing conscious machines are then considered. Arguing that, against this historical background, the development of conscious machines seems both dangerously naïve and morally irresponsible, the question of precaution arises. The need for precaution hinges partly on possibility. Conscious AI is assumed to be theoretically possible within certain theoretical frameworks, yet no independent empirical evidence is presently available. In that situation, we can neither logically exclude nor affirm the possible existence, or future existence, of artificial consciousness in the real world. Even assuming possibility, another question is whether or how we could recognise machine consciousness. Artificial systems use human-generated training data to mimic human behaviours, which, if successful, may persuade human users of their sentience. This is a psychological fact with no evidential value whatsoever. Moreover, to the extent that animals and artificial systems are very different in substance, structure, and functions, their putative sentience might also be very different and therefore incommensurable, which would pose a formidable obstacle for detecting, let alone understanding, a sentient machine. The paper concludes that, because of how human nature has been expressed throughout our history and continues to express itself today, developing conscious machines (possibly with for humans undetectable and incomprehensible minds) is a monumentally dangerous idea.
Kathinka Evers (Mon,) studied this question.