Preregistration package for Self-Improving SDLC Through Retrospective and Notary

This deposit pre-registers an empirical study of an agentic software development pipeline that revises its own instructions. Each issue goes through four stages: a Planning, an Implementation, a Review and a Fix Agent. The Review Agent's prompt is held fixed and serves as the measurement instrument. After every ten pull requests, a Retrospective Agent proposes edits to the learned rules of the Planning, Implementation and Fix Agents. A Retrospective Notary checks each proposal against the review comments it cites, then accepts, merges or rejects it. It keeps the rules within a token budget and records the provenance of each rule. The study runs four trajectories: two replicates on each of two Python repositories, sqlglot and dvc. The issues come from SWE-rebench and are processed in chronological order. Each trajectory runs ten windows of ten pull requests. All agents use Qwen3.6, served locally with vLLM on NVIDIA DGX Spark machines. The study has three research questions: - RQ1: how the rules evolve. - RQ2: review comments per pull request, their severity and their depth. These are measured on 60 held-out issues per repository, comparing the initial rules (v0) with the final rules (v10) on the same issues. - RQ3: external quality, measured by hidden-test resolve rate, new visible-test failures, patch size and cost. The deposit contains: - the study design and the analysis plan (v2) - the experiment configuration - the seed rules - the codebook and policy for classifying comment depth It corresponds to git tag prereg-2026-11-05. Prompt hashes: { "base_files": { "fix": "963a1c4d714a6160700eb4eb9b8e5f8edd58934365259630875f14d2033c700f", "implementation": "ec91e461b84534e7d304752a443ed85900b0a0b1abce72a9159a12c2c35eb1ef", "notary": "9a39c3b03834ee07613456d1e51c02ffb0e1674c28b77d3c9866e43216e24f09", "planning": "e2c70bd96ec395ce9a79304a82ebad9b85514b46f1d4eb52fc9e6e47058e6fa6", "retrospective": "3df3e5042939318e17d87ae951e9ca585b99f0ac372da7eecb656395cfe8dea2", "review": "0c7be775a9151cc3f3d0363df0d2bf58eeb9716a4f0098e328b4c64d8718920d" }, "system_prompts": { "notary": "cc3f8ffe7f065894f97a3c2d3125d6d0a41d54c2b1821161bd4099f167d3a40d", "retrospective": "6d7809751bdf58d5ae5a92d564d3746d173d4d31cd78ff2bc9f98f7eb1cdc538", "review": "dde62f0b9fda76a0ad3f11a5c08751c3fc7d69b2ecff20521ba22fe1444244bf" }}