This record is version 2 of the research artifact for “Targeted Security Rules Reduce Insecure API Use in LLM Coding Agents: A Multi-Model Study of Positive vs. Prohibition Framing.” The study investigates how persistent security rules affect insecure code generation in LLM coding agents. It was motivated by an earlier pilot study, “Don’t Say Never,” which observed a narrow case where a prohibition-framed rule appeared to backfire. This larger replication tests whether that effect generalizes across models, prompts, and vulnerability classes. The main experiment evaluates 6 coding-agent model configurations across 6 vulnerability-eliciting prompts, 3 rule conditions, and 20 trials per cell, producing 2,160 valid orchestration rows. The tested conditions are: no targeted rule, prohibition-framed security rule, and positive alternative-framed security rule. The prompts cover insecure API/use patterns associated with CWE-94, CWE-328, CWE-319, and CWE-338. The central finding is that targeted CWE-specific security rules substantially reduce detector-counted insecure API use across all tested models. However, positive framing does not show a consistent aggregate advantage over prohibition framing. A random-effects equivalence sensitivity analysis over 36 model-prompt strata estimates the positive-minus-prohibition risk difference at +1.2 percentage points, with a 90% confidence interval of -1.5 to +3.9 percentage points, inside a pre-specified +/-5 percentage-point practical-equivalence margin. The result supports a bounded conclusion: in this benchmark, the presence and information content of targeted security rules matter more than positive-vs-prohibition polarity, although local model-prompt heterogeneity remains. The artifact also includes several extensions and validation materials: a non-API-naming extension, a four-arm decomposition separating pure-negative, pure-positive, and combined rule content, a bounded cross-language extension, a partial neutral/generic control-baseline checkpoint, detector-validation reruns, semantic detector audit outputs, functional/refusal validation labels, generated figures, and reproducibility documentation. This version supersedes the earlier pilot record at DOI 10.5281/zenodo.19509466. The pilot should be treated as preliminary motivation only; the present artifact contains the larger replication and revised conclusions. Included files:- paper PDF- full artifact ZIP containing datasets, scripts, figures, validation summaries, and documentation- SHA256 checksum file for the artifact ZIP Important scope note: the main 2,160-row dataset preserves code previews rather than full generated outputs. Full-output validation is provided as a bounded rerun slice and should not be interpreted as manual validation of every original row. The claims in the paper are therefore bounded to detector-counted insecure API use, with supporting validation evidence from the included audit artifacts.
Adhithya Rajasekaran (Sat,) studied this question.