Optimization and Discovery in AI Scientist Systems: Tracing Evidence, Claims, and Decisions

Scientific discovery depends not only on obtaining a better result but also on deciding what the result establishes and what to investigate next. We trace those decisions across twenty-one reported AI Scientist configurations using Servo's functional roles and the decision vocabulary of partially observable Markov decision processes and Bayesian experimental design. Five source-checked accounts anchor the analysis. ScientistTwo reports improvement on five of six metrics but withholds credit from the proposed multicore mechanism after ablation: the performance finding stands while the contribution claim does not. Adam's growth-curve evidence updates a logical model, while the cited twenty-hypothesis account does not specify an information-based test ordering; PiEvo states a distinct rule for selecting surrogate queries about candidate principles. Coscientist's separate episodes resolve an implementation error and optimize reaction conditions against precollected condition-yield data. Robin's drug ranking informs human selection for an initial assay, while its subsequent RNA-seq proposal seeks evidence about the remaining transcription question. These are differences in what an observation licenses and what the next action is for, not stages of scientific maturity. Our contribution is a source-grounded determination of those links, their actors, and their unresolved warrants within the reported accounts. It neither estimates discovery performance nor treats information gain as a universal objective; an automated catalogue map supplies context rather than validation. This manuscript is a revised and expanded version of "Formalizing AI Scientist Systems: A Component Framework and Theoretical Analysis," published at CAISc 2026. The title and content have been revised. The present version has not undergone a separate peer review. Conference version: https://openreview.net/forum?id=ItSw72nR4c