Frozen Actors: A Policy-Gradient View of Backprop-Free Training, with an Application to Binary Networks

Site-perturbation training estimates activation gradients by injecting Gaussian noise at each layer and rewarding draws by the loss decrease; the recent Dust method uses it to pretrain transformers without backpropagation. We analyze this estimator as a score-function estimate for a frozen Gaussian policy (node perturbation) and draw three consequences. (i) Per-token rewards yield partial (diagonal) derivatives; the O(1) cross-token term is missing, survives the small-noise and infinite-population limits, and is invisible to the method's own fidelity metric, while a causal suffix sum fixes it at O(T) cost. (ii) For loss-difference rewards the predictable part of the reward is O(sigma^2), so any state-dependent critic can remove at most O(sigma^2) of the reward variance; we predict and confirm that critic+GAE gives little on Dust-style rewards (critic R^2 = 0.12, 1.4x variance reduction, no accuracy gain) while it helps when rewards have an O(1) mean (gradient cosine 0.902 -> 0.949 on a black-box control task). (iii) Replacing the straight-through estimator's backward pass with a forward site measurement trains a 1-bit MLP to 95.40 +/- 0.05% versus 92.24 +/- 0.23% for a tuned STE under a matched budget and zero backward calls, and a diagnostic shows the two are different relaxations (cosine 0.97 on smooth sites, 0.25 on binarized ones). This is a preprint of work in progress. Code, experiment logs and the LaTeX source are archived with this record. A matched comparison with FOGZO (NeurIPS 2025) and CIFAR-scale binary experiments are planned for a future version. Use of generative AI. The research idea, the research questions, the choice of experiments, the interpretation of the results, and the conclusions are the author's. The manuscript text was written by a large language model, DeepSeek V4.1 Flash 1017, under the author's direction and supervision; the same model was used as a coding assistant to implement the training scripts, run the experiments, and generate the tables and figures from the archived result files. No result was invented or altered by the model: every number in the paper is produced by the committed scripts from the JSON result files archived with this record, and every reference was checked against its arXiv, DOI, or proceedings record. The author is responsible for the correctness and originality of the entire content; the model is not an author.