Unlocking the Archive

This poster presents a zero-shot, multimodal AI pipeline designed to automatically segment and catalogue digitised broadcast video heritage. Developed in consultation with partner archives, the tool addresses the challenge of "dark data", large collections of preserved video files that lack the reliable metadata needed for basic access and discoverability. Unlike traditional shot boundary detection tools that identify technical cuts, this pipeline targets semantic boundaries (where one topic ends and another begins) without requiring any labelled training data. It combines three AI modalities, designed to run entirely locally on consumer hardware: - Visual Analysis: Moondream2 generates natural language descriptions of sampled frames to capture visual cues like studio settings and on-screen text. - Audio Transcription: OpenAI Whisper extracts timestamped speech to trace narrative threads. - LLM Reasoning: Open-weight language models (such as Gemma-3-4B or Qwen) analyse the aligned audio-visual evidence using a sliding window to detect semantic content shifts. The tool transforms opaque video files into structured, searchable data, outputting time-coded segments, free-text summaries, topic labels, and detected on-screen metadata. Crucially, its local-first architecture ensures that sensitive or copyright-restricted archival material never leaves the institution's infrastructure. The poster outlines the methodology, system architecture, performance findings, and an offline human-in-the-loop verification dashboard, demonstrating that meaningful semantic analysis of audiovisual archives is highly accessible for cultural heritage institutions without relying on expensive cloud infrastructure. Code Availability: The associated codebase is publicly available under an MIT licence at: https://github.com/kingsdigitallab/issa/tree/main/prototypes/segment