Pipeline or Agentic RAG on a Single Model GPU: Four Times as Many Concurrent Users at Equal Latency and Predictable Per-Ticket Cost

When documentation may not leave the organisation, a question-answering system over it must run on open-weight models on the organisation's own GPUs. We compared two designs of such a system — a fixed retrieval pipeline and an agent that decides for itself what to search and read — on real support tickets over a private Russian-language knowledge base. At equal latency, the pipeline serves four times as many concurrent users on one model GPU. The pipeline sustains a median response time of about 100 s at 40 concurrent tickets; the agent does so only at 10. When the load grew from 20 to 40 tickets, the agent's response time almost doubled (141 → 270 s) and its throughput barely grew, while the pipeline continued to scale. At 40 tickets the agent generated tokens 3.7 times more slowly: its context grows with every page it reads. The measurement used the smaller open-weight model gpt-oss-20b under the default vLLM configuration; a memory-capacity estimate for a 27B-class model gives a 5–6× ratio of contexts that fit in GPU memory at once. The pipeline's cost is predictable; the agent's is not. Without a step limit the agent made from 5 to 162 tool calls per ticket. With a frontier model accessed through an API, the agent's most expensive ticket cost 2.5 times the median, while the pipeline's price per ticket is nearly constant. Neither design has a robust quality advantage. By the pre-recorded rule two of the three judges, both lenient, scored the agent higher; the third preferred the pipeline, but within its own run-to-run variation; and under a different source protocol the agent's advantage disappeared. In the pipeline, the open-weight DeepSeek-V4.1-Flash is within half a point (on a five-point scale) of a frontier model according to two of three judges. For self-hosted deployments the fixed pipeline is the practical choice.