Agentic Generative AI for Methodology-Grounded Modelling from Unstructured Documents: Design and Evaluation of a Multi-Agent Ecosystem Mapping Pipeline

Modelling constitutes a disciplined transformation process through which heterogeneous, unstructured evidence is translated into structured representations that support reasoning and decision-making. The integration of generative artificial intelligence into such processes introduces new possibilities for automation, yet risks undermining methodological rigour, traceability, and human accountability. This paper proposes a methodology-grounded multi-agent architecture for constructing structured business ecosystem maps from unstructured document collections. The architecture decomposes the modelling lifecycle into specialised agent functions covering boundary specification, source discovery, document analysis, semantic extraction, and controlled model editing, addressing four of the five methodology stages while leaving automated completeness verification outside the current scope. A central orchestrator coordinates agents while enforcing ontological constraints derived from a formal modelling methodology. All proposed modifications are staged for human review before execution, and each map element maintains explicit provenance links to source material. To evaluate the reliability and correctness of generative modelling pipelines, a hybrid evaluation framework integrates operational metrics, semantic assessment using an LLM-based judge, and human agreement validation. Empirical evaluation across 34 generative models and 4382 experimental runs characterises capabilities across modelling tasks. In a controlled single-document extraction task, text-based extraction achieves a mean semantic match score of 0.947, whereas interaction extraction scores 0.431 and visual diagram interpretation scores 0.470, identifying relational reasoning and multimodal interpretation as principal bottlenecks. Model performance varies across agent roles, with task-aligned model selection associated with larger performance changes than hyperparameter tuning; the architecture’s causal contribution is not isolated, and comparison against monolithic or ablated baselines remains future work.