From Generation to Verification: A Reproducible Empirical Study of Verification-Guided LLM Workflows for Software Engineering

Large language models (LLMs) have demonstrated considerable capabilities in automated code generation; however, producing syntactically valid and plausible code does not necessarily guarantee functional correctness. Verification-guided workflows seek to improve generated software through iterative refinement informed by execution results and software-quality diagnostics. Nevertheless, whether increasingly sophisticated verification mechanisms yield measurable improvements in independently evaluated correctness sufficient to justify their computational costs remains an open empirical question. This study introduces Guided Verification for Software Engineering (GV-SE), a reproducible experimental framework for systematically evaluating five LLM-assisted software development workflows: direct generation (C0), structured generation (C1), self-reflective refinement (C2), test-guided refinement (C3), and multi-channel verification-guided refinement (C4). The experiment involved 20 benchmark tasks (10 HumanEval+ and 10 MBPP+), three frozen LLM configurations, and 300 prespecified experimental runs, of which 292 (97.3%) were completed successfully. Model-visible verification evidence was strictly separated from an independent held-out evaluation oracle to prevent feedback leakage. A binomial generalized linear mixed-effects model (GLMM) was employed to evaluate five prespecified confirmatory contrasts, with Holm correction for multiple comparisons. The estimated marginal probabilities of held-out functional correctness were 0.488 for C0, 0.756 for C1, 0.689 for C2, 0.756 for C3, and 0.514 for C4. Although C1 and C3 exhibited the highest estimated probabilities, none of the five confirmatory contrasts reached statistical significance after Holm adjustment. In particular, C4 did not demonstrate a statistically significant improvement over C0 (OR = 1.110, 95% CI [0.325, 3.789], Holm-adjusted p = 0.868) or C3 (OR = 0.341, 95% CI [0.085, 1.363], Holm-adjusted p = 0.462). These conclusions remained unchanged across four extreme missing-outcome sensitivity scenarios. Analysis of refinement behavior revealed that C3 initiated repair in 18 of its 60 completed runs, whereas C4 performed two verification-guided repair iterations in each of its 60 completed runs. Despite this additional verification activity, exploratory adjusted cost analyses indicated that C4 consumed approximately 6.05 times as many tokens and incurred 3.88 times the model inference latency of C0. Overall, the findings indicate that greater verification intensity does not necessarily translate into statistically demonstrable improvements in independently evaluated functional correctness. GV-SE provides a reproducible empirical foundation for distinguishing verification activity from demonstrated repair effectiveness and highlights the importance of jointly evaluating functional correctness, refinement behavior, diagnostic evidence, and computational costs when designing and assessing LLM-assisted software engineering workflows.