Media Streams: an iconic visual language for video annotation

In order to enable the search and retrieval of video from large archives, we need a representation of video content. Although some aspects of video can be automatically parsed, a detailed representation requires that video be annotated. We discuss the design criteria for a video annotation language with special attention to the issue of creating a global, reusable video archive. Our prototype system, Media Streams, enables users to create multi-layered, iconic annotations of streams video data. Within Media Streams, the organization and categories of the Director's Workshop allow users to browse and compound over 2200 iconic primitives by means of a cascading hierarchical structure which supports compounding icons across branches of the hierarchy. The problems of creating a representation of action for video are given special attention, as well as describing transitions in video.>