Mathematical modeling study demonstrates that reduced trust in AI enhances perceived search effectiveness in collaborative cybersecurity tasks, highlighting systemic risks of user overreliance.
We investigate human overreliance on artificial intelligence (AI), a phenomenon in which users fail to critically verify AI outputs despite knowing the system is imperfect. To analyze this interaction, we model a cybersecurity scenario in which a human user and an AI chatbot collaborate to locate a vulnerable node in a large-scale distributed network. We consider a search process in which the probability distribution of the node’s location is updated using Bayes’ rule. In each round, the chatbot sequentially scans the nodes and suggests a potentially vulnerable node; the user then decides whether to accept the detection and end the search or refine the query and continue searching. The goal is to find an efficient scanning algorithm that maximizes the user’s expected satisfaction with the search results. Finally, we provide a formal proof of the “more for less” paradox, demonstrating that reducing trust in AI can, under certain conditions, increase perceived search effectiveness.
No takes yet. Share an insight, caveat, or question.
Kriheli et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: