Archiving and Temporal Analysis of Behavioral Web Data - Tales from the Inside

Behavioral web data such as social web activity streams, query logs or behavioral traces from web search and navigation are crucial to understand the temporal evolution of the web and the human interactions that produce web data and models trained on such data. Thus, behavioral web data empowers research in various fields, such as (temporal) information retrieval, computational social science or cognitive and behavioral modeling of users over time. On the other hand, archiving and using such kind of data is associated with a number of technical and non-technical challenges, e.g. legal and ethical concerns, fast decay of data over time, as well as dependency on 3rd party gatekeepers, such as Twitter/X or Google. We present case studies from many years of research into archiving and temporal analysis of behavioral web data, including large-scale social web archives such as TweetsKB, based on an archive of 14 bn tweets harvested continuously since 2013, and web search and navigation behavior tracked through user studies, longitudinal panels or crowdsourcing-based quasi experiments.