Answers are grounded in this paper and cite their sources.
Agentic large language models (LLMs) are proposed for cyber-physical control, yet evidence on how reliably small open-weight models turn sensor context into actuator commands is scarce. We present a reproducible benchmark grounded in an operational ESP32-based greenhouse controller: ninety-six generated test cases run against the production tool schema and prompts, scored deterministically, with the controller’s rule-based fallback ported verbatim as baseline. Four quantized models (3–14 B) ran on a pinned llama.cpp stack in two locked environments two months apart (12,000 scored runs). Three findings dominate. Autonomous self-initiation is far less reliable than command execution: models executing 65–100% of chat commands fall to 0.4–64.6% when acting unprompted. Model rankings are environment-conditional: one environment favoured Qwen2.5-7B over the 14B (McNemar p = 0.016; cluster-robust GEE p = 0.084), the other decisively reversed it at identical seeds ( p < 0.001; 77.5% cross-environment agreement), requiring re-benchmarking on the deployment stack. The rule baseline matches the second-best model with the best refusal behaviour: the LLM’s measured value is language-mediated actuation, knowledge capture, and auditability, not superior regulation. Field telemetry and thermal replay show the hybrid controller running unattended through the growing season, rules holding the safety-critical floor. All instruments and both environments’ traces are released.
No takes yet. Share an insight, caveat, or question.
Penica et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: