The development of AI reasoning models requires training data that captures not only correct answers but correct reasoning processes—the chain of evidence evaluation, hypothesis formation, adversarial challenge, and confidence calibration that produces a defensible conclusion. We describe a system and method for generating a ground-truth-mapped AI reasoning corpus by pairing multi-agent deliberation sessions with prediction market contracts as external resolution sources. Each session is linked to a contract at initiation time, establishing a binary ground truth anchor before deliberation begins. A provenance-complete transcript logging layer records model identifier, provider, and temperature at the individual message level for every agent contribution. An outcome tracking pipeline monitors linked contracts and automatically generates calibration records when contracts resolve, computing Brier scores from ensemble confidence grades against binary resolution outcomes. A corpus quality tier classification—High, Medium, or Flagged—controls for hindsight contamination based on whether the contract was live or already resolved at session initiation. The accumulated corpus constitutes a proprietary data asset in which every reasoning record is linked to a verifiable external outcome, every agent contribution is provenance-attributed to a specific model and configuration, and every calibration record includes controls for hindsight contamination and session origin.
Kelly et al. (Mon,) studied this question.