Watching and talking

New multimedia applications, such as community-created video repositories and tools for synchronous sharing, have revolutionized the ways that media is watched and shared. Effective instrumentation of these applications can enable researchers and system designers to better understand how video is being consumed: that is, how it is being watched, shared, augmented with annotations and otherwise experienced by individuals, by groups, and crowds. In addition to consuming content, however, people also talk about it. Conversational exchanges around video content can take place within applications (e.g., in chat spaces) and/or using separate communication channels (e.g., microblogs). We propose that the actions of these video viewers, with the video object itself and/or with each other around the video object, provide rich data for understanding the semantics and social relevance of various pieces of video content. We illustrate this approach with current research and a novel taxonomy of social multimedia interaction.