Mondrian

Mondrian is an object-oriented graphical editor that can learn new graphical procedures through programming by demonstration. A user can demonstrate a sequence of graphical editing commands on a concrete example to illustrate how the new procedure should work. An interface agent records the steps of the procedure in a symbolic form, using machine learning techniques, tracking relationships between graphical objects and dependencies among the interface operations. The agent generalizes a program that can then be used on “analogous” examples. The generalization heuristics set it apart from conventional “macros” that can only repeat an exact sequence of steps. The system represents user-defined operations using pictorial “storyboards” of examples. By bringing the power of procedural programming to easy-to-use graphical interfaces, we hope to break down the “Berlin Wall” that currently exists between computer users and computer programmers.