Large language models can exhibit behavioral drift under changes in sampling temperature, exposure to safety-sensitive prompts, fine-tuning, and adversarial prompt injection. Many recent related methods either require access to model internals such as hidden states, attention weights, or token probability distributions, or operate on input prompt embeddings rather than generated outputs. This paper describes a training-free stability metric computed from generated outputs alone and evaluates it across eight validation tests centered on TinyLlama 1.1B-Chat, with additional validation on Phi-2 2.7B, GPT-2, and a base TinyLlama checkpoint. The metric is defined as Phi = I × rho - alpha × S, where I is an identity component computed from cosine similarity between baseline and comparison output embeddings, rho is a temporal coherence component measured across sequential evaluation windows, S is an output variation component derived from multiple sampled outputs, and alpha is a coupling constant fixed at 0.1. The tested settings produced measurable drops in Phi across four tested drift conditions: a 0.282 drop under temperature induced quality degradation on TinyLlama, a 0.072 drop accompanied by a refusal rate collapse from 14% to 3% on safety sensitive prompts, a 0.254 drop between a base and instruction tuned checkpoint pair, and a 0.207 drop under an adversarial prompt injection wrapper with the same seeds in both conditions. The same safety-regime evaluation framework reproduced on Phi-2 with a 0.044 drop using the same temperature pair and prompt suite with a reduced sample count for runtime reasons, and Phi tracked sampling temperature continuously with a correlation of -0.97 across eight temperature points. The method operates in black-box settings using only generated text and an external embedding model. Results are bounded to the tested models, prompt suites, and configurations; broader generalization is not established. The methods described in this paper are the subject of U.S. Provisional Patent Application No. 63/973,673 (filed February 2, 2026). No license to implement or commercialize the described methods is granted by this publication. All rights reserved.
Shawn Barnicle (Sat,) studied this question.