Building a time machine for efficient recording and retrieval of high-volume network traffic

There are times when it would be extraordinarily convenient to record the entire contents of a high-volume network traffic stream, in order to later travel back in time and inspect activity that has only become interesting in retrospect. Two examples are security forensics---determining just how an attacker compromised a given machine---and network trouble-shooting, such as inspecting the precursors to a fault after the fault. We describe the design and implementation of a Time Machine to efficiently support such recording and retrieval. The efficiency of our approach comes from leveraging the heavy-tailed nature of network traffic: because the bulk of the traffic in high-volume streams comes from just a few connections, by constructing a filter that records only the first bytes of each connection we can greatly winnow down the recorded volume while still retaining both small connections in full, and the beginnings of large connections (which often suffices) .