Answers are grounded in this paper and cite their sources.
Cyber-physical systems are developed using heterogeneous modeling frameworks that address different engineering concerns and abstraction levels. In practice, models from mechanical, electrical, and software engineering coexist across development phases, creating the need to specify inter-model consistency despite differences in terminology, structure, and execution semantics. A first step of such a specification is to identify which modeling constructs of two frameworks correspond, that is, whose instances have to be kept consistent, and along which dimensions. In this paper, we investigate how well Large Language Models (LLMs) perform this correspondence identification for pairs of modeling constructs, compared with expert references. We use two gold-standard datasets: 90 automotive construct pairs across AUTOSAR, ROS 2, CAD, Simulink, and a brake specification metamodel, annotated by three experts, and 32 smart-grid construct pairs across three IEC metamodels, whose reference labels are derived from a published rule-based integration. Each pair is labeled on an ordinal scale along three dimensions: syntactic, semantic, and behavioral. Six large language models from OpenAI and Google are evaluated under zero-shot, one-shot, and few-shot prompting, together with two agentic systems, OpenAI Codex and Anthropic Claude Code, run with web access allowed or disabled. We report Quadratic Weighted Kappa, Mean Absolute Error, accuracy, and macro-F1 with bootstrap confidence intervals and paired tests. On average, the LLMs agree with the references beyond chance and beyond string- and embedding-based baselines, but ordinal agreement remains weak to moderate. On the automotive pairs with traceable independent annotations, the agreement of the best configurations with individual experts does not differ significantly from the agreement among the experts, whereas most configurations agree less in the semantic dimension and, on the AUTOSAR–ROS 2 pairs, also in the syntactic dimension. Over both datasets, semantic correspondence is judged more reliably than syntactic correspondence. Vendor differences are small and dimension-specific, and in-context examples improve agreement only in selected dimensions. On the automotive pairs, the ordinal agreement of the agentic systems does not exceed that of the best direct configurations. On the smart-grid pairs, for which their prompts, unlike the direct prompts, define the dimensions in the terms used to derive the reference labels, they reach higher syntactic agreement. Web access does not improve their agreement consistently. Overall, LLMs can support experts in identifying correspondences, but they do not replace expert judgment.
No takes yet. Share an insight, caveat, or question.
Benjamin Arp (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: