What question did this study set out to answer?

To develop an open-source framework that orchestrates multiple large language models for optimal task performance.

June 13, 2026Open Access

Multi-LLM Prototype: Open-source software for adaptive orchestration, conflict resolution, and cost-aware routing across heterogeneous large language models

MTMohamedou Cheikh TouradUniversity of Nouakchott Al Aasriya MLMohamed LachgarCadi Ayyad University

Key Points

To develop an open-source framework that orchestrates multiple large language models for optimal task performance.
Developed Multi-LLM Prototype using FastAPI and Next.js for routing and conflict resolution.
Implemented a transparent routing score and a four-stage conflict-resolution pipeline.
Created a confidence-weighted synthesis step using evidence theory for task completion.
Achieved improved task performance by directing subtasks to the most suitable model based on their strengths.
Introduced a traceable and inspectable feedback loop for adaptive model selection.
Validated the system's effectiveness in handling task disagreements through defined metrics.

Abstract

We present Multi-LLM Prototype, an open-source research framework that puts several large language models (LLMs) to work on the same prompt, under a single decision process. The starting observation is simple: in practice, no single model is consistently best on quality, latency, cost and domain at the same time, so picking one model per request leaves value on the table. Our system breaks an incoming prompt into smaller subtasks, scores each candidate model on every subtask through an interpretable multi-criteria function, sends each subtask to whichever model fits it best, and then handles disagreement explicitly through a staged conflict-resolution pipeline rather than papering over it. Compared to existing orchestration frameworks, our specific contributions are a transparent linear routing score with documented default weights and a published sensitivity sweep, a four-stage conflict-resolution pipeline grounded in an explicit cosine-distance conflict metric, a confidence-weighted synthesis step inspired by evidence theory, and an inspectable adaptive feedback loop whose update rule is given in closed form. We have built it as both an experimentation framework and an operational prototype, on top of a FastAPI backend, a Next.js frontend, pluggable provider adapters, WebSocket streaming, explainability dashboards and a Docker-based deployment. The aim is to give researchers and practitioners a reproducible environment for testing multi-model strategies under explicit budget and performance constraints, with every routing decision traceable end to end.

Ask AI

Helpful

Bookmark

View Full Paper