Working paper tests context severance effects on LLM output in business planning critiques, suggesting new methodologies.
[Working paper — not peer-reviewed. This record contains full texts in both English and Japanese, plus a complete replication kit and raw data.] Self-refinement by large language models (LLMs) is known to be lenient due to self-preference bias. This paper formalizes Context Cut, a design operation that deliberately severs context and provenance information between the stages of an LLM pipeline, and tests its effect in a controlled experiment. Severance comes in two grades: a soft cut (instructing the model to disregard the history) and a hard cut (passing only the artifact into a fresh context). Using 30 business-planning briefs, we varied only the critique-stage configuration (no cut / soft cut / hard cut / switch to a different model) and measured both critique properties and the quality of revised final drafts via blind pairwise comparison. The hard cut made critiques significantly longer (+38%, p=1.7e-6) but did not change the number of issues raised, and produced no difference in final-draft quality (win rate 55.0%, p=0.52; inter-rater agreement at chance). By contrast, the composite of context cut and model switching—presenting the artifact alone to a different model in a fresh context—significantly improved final-draft quality (win rate 68.3%, p=0.006). This composite was the only configuration whose effectiveness was confirmed; model switching alone (with inherited context) remains untested. The soft cut reduced both the number and volume of critique points. These results suggest that provenance cues reside not in the conversational context but in the artifact itself: context-level provenance blinding cannot erase self-hood, and the effective unit of otherness in a critique stage is the model, not the context. The method and its test bed are implemented as the "Auto Factory" feature of the commercial application ProAI Studio. This is the third report in the research program Vacancy-Based Creativity Methods.
No takes yet. Share an insight, caveat, or question.
Shoichi Furuta (2026) studied this question.