Brittleness in the design of cooperative problem-solving systems: the effects on user performance

One of the critical problems in the design and use of advanced decision-support systems is their potential "brittleness". This brittleness can arise because of the inability of the designer to anticipate and design for all of the scenarios that could arise during the use of the system. The typical "safety valve" to deal with this problem is to keep a person "in the loop", requiring that person to apply his/her expertise in making the final decision on what actions to take. This paper provides empirical data on how the role of the decision support system can have a major impact on the effectiveness of this design strategy. Using flight planning for commercial airlines as a testbed, three alternative designs for a graphical flight planning tool were evaluated, using 27 dispatchers and 30 pilots as subjects. The results show that the presentation of a suggestion or recommendation by the computer early in the person's own problem evaluation can have a significant impact on that person's decision processes, influencing situation assessment and the evaluation of alternative solutions.