Ethical Alignment of LLMs in Healthcare: Does GPT-o1 Adopt a Deontological or Utilitarian Approach?

In moral philosophy, two foundational approaches shape ethical decisions: deontology and utilitarianism, often exemplified by the "Trolley" dilemma. We conducted an experiment evaluating multiple LLMs, including OpenAI's GPT-o1, across five medical versions of this dilemma. While some models adhered to established ethical standards, others inconsistently favored utilitarian outcomes, raising concerns about their alignment with human medical ethics. Careful evaluation of LLMs' ethics in healthcare is warranted, to ensure their safe integration into clinical practice.