What question did this study set out to answer?

The research aims to assess the effectiveness of AI models in critiquing academic papers through an adversarial framework.

January 14, 20260 citationsOpen Access

Adversarial AI Peer Review Transcripts: Attack-Defense Protocol Testing

View Full Paper

BBÉO

Key Points

The research aims to assess the effectiveness of AI models in critiquing academic papers through an adversarial framework.
Transcripts from peer review sessions using Claude Sonnet 4.5 and Grok 4.1 AI models.
Multi-round attack-defense protocol where AI critiques and authors respond based solely on paper content.
Evaluation of models in the domains of foundational physics and meta-science.
High consistency in performance across four sessions.
Initial AI understanding ranged from 30-50%, improved to 90-95% after revisions.
Demonstrates AI's reasoning capabilities and meta-cognitive abilities.

Abstract

This dataset contains complete transcripts of adversarial peer review sessions between frontier AI models (Claude Sonnet 4. 5 and Grok 4. 1 Thinking) and research papers, using a novel Attack-Defense protocol. CONTENTS: • Transcript 1: Claude 4. 5 vs Foundational Physics Paper (~44 pages, 3 major correction rounds) • Transcript 2: Grok 4. 1 vs Foundational Physics Paper (~16 pages, 4 major correction rounds) • Transcript 3: Claude 4. 5 vs Meta-Science Case Study (~40 pages, 4 major correction rounds) • Transcript 4: Grok 4. 1 vs Meta-Science Case Study (~50 pages, 7 major correction rounds) METHODOLOGY: Attack-Defense protocol where AI critiques research papers and authors defend using only paper content (no personal opinions). Multi-round iterative process until resolution. KEY FINDINGS: • Pattern 100% consistent across 4 sessions • Initial: Misread/wrong criteria (30-50% understanding) • Final: High scores after correction (90-95% range) • Demonstrates: AI reasoning capability, failure modes, recovery patterns, meta-cognitive abilities LANGUAGE: Vietnamese (demonstrates cross-lingual deep reasoning in academic context) PURPOSE: • Transparent AI evaluation (full process visible) • Research data for verification and replication • Benchmark for adversarial peer review capability • Educational resource for AI-assisted research EXPERIMENTAL CONDITIONS: • Date: January 2026 • Models: Claude Sonnet 4. 5, Grok 4. 1 (Jan 2026 versions) • Domain: Foundational physics + Meta-science • Test type: Out-of-pattern adversarial cases • No editing, no cherry-picking RELATED MATERIALS: • Main paper: "Peer Review with AI & Author Stress Test" BÉO. (2026). Peer Review with AI & Author Stress Test: A Two-Way Protocol for Research Quality Pre-Verification. Zenodo. https: //doi. org/10. 5281/zenodo. 18224556 • Video recordings: https: //www. youtube. com/playlist? list=PLJ3dkTh-U9b8OLdws0M6N₅bjGNsaAGzf • Framework: Section Zero DOI: 10. 5281/zenodo. 18091473 LICENSE TERMS: This dataset is available under dual licensing: 1. ACADEMIC & NON-COMMERCIAL USE: License: CC BY 4. 0 Free for: Research, education, personal study 2. COMMERCIAL USE: (AI model training, product development, commercial platforms) Requires separate commercial agreement. Contact: beo@beolabs. org DEFINITIONS: • Commercial use: AI training by for-profit entities, integration into commercial products/services• Non-commercial: Academic research, education, personal study, non-profit open-source projects Questions? Contact: beo@beolabs. org CITATION: BÉO. (2026). Adversarial AI Peer Review Transcripts: Attack-Defense Protocol Testing. Zenodo. https: //doi. org/10. 5281/zenodo. 18211530 CONTACT: beo@beolabs. org

Demander à l'IA

Bookmark

View Full Paper

Demander à l'IA

Bookmark

View Full Paper

Adversarial AI Peer Review Transcripts: Attack-Defense Protocol Testing

Key Points

Abstract

Cite This Study