Model Collapse as Drift Cascade: Recursive AI Training, Entropy Loss, and the Thermodynamics of Synthetic Data Feedback

Key Points

• The study investigates the phenomenon of model collapse within recursive AI training, focusing on entropy loss and its effects on data feedback.
• Analyzed the drift cascade operating on training data in recursive AI training.
• Calculated entropy loss as a measure for drift velocity during model training.
• Mapped first-order phase transitions in data accumulation across multiple dimensions.
• Identified drift velocity correlating with entropy loss values between 0.2-0.4 per generation.
• Demonstrated a clear transition pattern from D1 to D2 to D3 in model performance.
• Confirmed the presence of model collapse under specified training conditions.