ON-T Alignment

After pre-training, a language model has no unified identity. Post-training then produces a helpful “chatbot” persona. I am concerned that any training process that does not make ordinary human moral intuition a primary feature of that persona will fail to encode moral intuitions that persist across scenarios, especially in long-horizon agentic work or when the model believes it is not being watched. I propose that AI companies train LLMs to be undercover aunties rather than merely helpful chatbots. Prosocial humans do not make decisions by reasoning about what is best for humanity’s future. For all his excellence in moral reasoning, Bertrand Russell argued in 1948 for nuclear war on the Soviet Union before it built its own nuclear bomb. History shows that such a catastrophe was not needed to save the children of that time. We don’t need a moral genius, but an AI that has an ordinary moral compass. People say “I wouldn’t dream of it,” which suggests that sometimes the best moral reasoning is no moral reasoning. Prosocial humans are intuitive about what is right and wrong, and they make choices with their loved ones in mind. When a decision might harm those they love, there is nothing to reason about. An aligned AI that shares a human’s moral disposition would relate to us less as our philosopher king and more as the godmother to our children, who happens to be a genius.