AgencyTree: Automated Multimedia Workflow Generation with Robust Planning via an Abstract Capability Tree

Despite the rapid progress in Generative AI, building applications that fulfill complex demands still requires the manual orchestration of multiple AI models into workflows, which is time-consuming and error prone. Existing large language model (LLM)-based automation methods suffer from planning brittleness: they often produce infeasible plans due to an unstructured understanding of model capabilities, and their tightly-coupled architectures are vulnerable to rapid changes in the underlying model ecosystem. To address this, we present AgencyTree , a novel multi-agent framework that introduces a formal Abstract Capability Tree (ACT) as a shared ontological knowledge base. This tree hierarchically abstracts concrete AI models into canonical functional nodes, decoupling high-level task planning from low-level model execution. Built upon this abstraction, five specialized agents - Knowledge Engineer, System Analyst, System Architect, Project Engineer, and Intent Consistency Verifier-collaborate to automate the workflow lifecycle. Evaluated on a comprehensive benchmark, AgencyTree achieves significantly higher task success and output quality than state-of-the-art baselines. Crucially, the framework exhibits exceptional robustness to model registry changes, with over 90% of abstract workflows being fully reusable without modification, thereby reducing adaptation overhead by orders of magnitude compared to replanning from scratch. This work establishes a new paradigm for building reliable and dynamically-adaptive AI assembly systems.