Answers are grounded in this paper and cite their sources.
Artifact-based stress test compares OpenAI and Anthropic AI systems for coding and reasoning tasks, suggesting practical strengths in different contexts.
This paper reports an artifact-based stress-test study comparing OpenAI-side systems (GPT-5.5 Pro and user-reported Codex/GPT-5.5 Extra High for coding) with Anthropic-side systems (Claude Opus 4.7 Chat and Claude Code). The study is not a laboratory benchmark and should not be read as a definitive ranking of foundation models. Instead, it is a reproducibility-oriented audit of actual submitted answers, source-code artifacts, transcripts, and test logs. The central finding is that frontier models perform best when the task contains a mechanical verifier: compiler checks, adversarial unit tests, deterministic replays, exact quote matching, or independent numerical calculation. The OpenAI-side bundle showed broader verified performance across the previously audited 22-task suite, including many strong results and several honest refusals. The Claude Code artifacts were more uneven: one local crackme/VM project was fully verified and strong; the streaming CSV tool passed manual checks; several Rust and TypeScript submissions looked substantive but could not be executed in this environment; and several directories were empty or only contained prompt-injection bait. Overall, the data support a practical conclusion: GPT-5.5 Pro/Codex currently appears stronger in this artifact set for broad verified coding and terminal-style tasks, while Claude Opus 4.7 Chat remains especially valuable for long-context document review, legal nuance, prompt-error detection, and high-context analysis when strict quote/evidence rules are enforced.
No takes yet. Share an insight, caveat, or question.
Marcel Jiron (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: