LLMs as Research Instruments for Problems in Engineering Management: A Context Modeling Framework

In engineering management research, large language models (LLMs) are increasingly used as analytical tools, yet their application often lacks a systematic and scientifically verifiable methodological foundation. This study introduces Context Modeling, a structured methodology that transforms LLMs from heuristic tools into scientifically verifiable research methods. The framework provides a unified process for prompt structuring, semantic constraint design, and rigorous output evaluation within a traceable dual-layer architecture. The structural layer assesses reliability through fidelity, robustness, consistency, and reproducibility, while the mechanistic layer applies feature attribution to reveal decision logic and feature dependencies. The framework is demonstrated using the Rethinking I-94 infrastructure redevelopment case led by the Minnesota Department of Transportation. Based on public survey data, a synthetic population of 3,154 agents is constructed to simulate attitudes toward two contrasting policy alternatives. Results show that the LLM reproduces stable structural distributions across scenarios: overall Fidelity exceeds 98%, Robustness ranges from 87% to 91%, Consistency from 87% to 93%, and Reproducibility exceeds 60% across both policy alternatives. At the subgroup level, Fidelity remains high (approximately 98%–99%), and most Robustness and Consistency values exceed 90%, with localized reductions under one scenario reflecting context-dependent semantic sensitivity rather than structural instability. These findings indicate that Context Modeling enables structurally stable and mechanistically interpretable LLM-based decision patterns grounded in real data. Through quantitative validation and feature-level interpretation, the results indicate that the proposed Context Modeling framework can provide a reliable research method in the field of engineering management based on LLMs and problem context simulation.