Autonomy or guidance: What users want from AI versus human advisors

As agentic artificial intelligence (AI) increasingly serves as a source of expert advice, designers face critical moral questions: should AI advisors optimize outcomes, guide users, or present information neutrally? This paper investigates laypeople’s preferences for four advice designs inspired by moral debates about outcomes versus agency: outcome efficacy, outcome expectancy, complete agency, and guided agency. Across two advice scenarios in health and finance, participants consistently rejected designs that restrict user autonomy. Participants favored "complete agency" (information only) when receiving AI advice but "guided agency" (information and recommendation) from human experts, suggesting that people assign different normative roles to AI and human experts. This asymmetry has implications for AI system design, regulation, and ethics. As AI advice becomes more prevalent, respecting users’ autonomy and expectations is crucial not only for usability, but also for ensuring that algorithmic systems remain aligned with public values.