Information-Theoretic Limits of Safety Verification for Self-Improving Systems

Key Points

• This research investigates the feasibility of allowing self-modification in systems while controlling risk levels.
• Formalizing criteria for bounded risk and unbounded utility
• Proving classification impossibility using Hölder's inequality
• Deriving tighter bounds through NP counting methods
• Validating theories using empirical data from GPT-2
• Proven impossibility of achieving both bounded risk and unbounded utility with conventional classifiers
• Hölder's inequality shows maximum classifier utility grows slower than linear growth
• A Lipschitz ball verifier can maintain zero risk while allowing positive utility