Beyond Self-Report: Making AI Use Observable at Scale: Behavioral Traces, Adaptive Interviews, and Distributed Hermeneutic Inquiry

Research on AI use often relies on what people report about their attitudes, intentions, and practices. These accounts describe participants' perspectives on their work. Behavioral logs show prompting, copying, revision, rejection, and disengagement. Linking both forms of evidence connects situated action with participants' reasons for it and supports grounded recommendations for practice and system design. We developed and deployed a research platform that links instrumented AI-assisted work to immediate qualitative recall. A deterministic tracker converts activity across the chat and workspace into a typed event stream. A language model selects potentially consequential moments from that stream and generates timestamped questions, which an adaptive interviewer asks within minutes of task completion. The same interviewing infrastructure can operate independently for research questions about experiences, expectations, or practices that cannot be directly instrumented. Independent interviewer instances allow interviews and recall sessions to run concurrently or asynchronously without assigning a researcher to each participant. One researcher applied the method across 13 teacher-education studies over 14 weeks, producing 147 completed sessions. Four workspace studies recorded 92 completed sessions and 47.8 hours of task work; one asynchronous study produced 57 completed sessions without appointments or researcher presence. In a comparison of 12 lesson-planning sessions, documents with similarly high shares of AI-generated text resulted from different processes of acceptance, constraint-setting, and rejection. Linked traces and recall accounts showed where participants applied pedagogical judgment. A separate audit of 361 interviewer turns found interpretive reflection in every interview and later use of earlier answers in 25 of 39 interviews. We adopt a functionalist account of hermeneutic agency and a functional self-model account of experience within this distributed hermeneutic arrangement. Hermeneutic agency consists in the causal organization of interpretation across turns. The model constructs a provisional participant model, retains it, integrates later material, and uses the resulting interpretation to select a question. The self-model account treats experience reports as access-available interpretations shaped by memory, attention, biography, embodiment, and the interview situation. Participants can qualify these interpretations, while researchers remain responsible for the scientific claims derived from the resulting record. Human validation of the model's recall-anchor selection remains outstanding.