Study protocol outlines a benchmarking framework for factual accuracy in AI assistant outputs across European SaaS firms, indicating a scalable model for auditing generative claims.
Pre-registered study protocol. This record contains no results. Commercial measurement of brand presence in AI assistants reports whether an engine mentions, cites or recommends a brand. This protocol measures a different property: whether the factual statements an engine makes about a brand match what the brand itself publishes. We are not aware of a public dataset that measures this at scale. The protocol fixes, before data collection: ground truth taken exclusively from each brand's own published pages, snapshotted with URL and date before querying; nine machine-checkable fact types (entry plan price, currency, tier count, free tier, HQ country, founding year, founder names, a named integration, a named market or language) plus one reserve; a sample of 150 to 200 European B2B SaaS companies with public pricing, stratified by stage across at least five countries; three runs per engine, brand and fact on non-consecutive days, with a stability band reported alongside every headline figure; a five-bucket scoring rubric (correct, stale, fabricated, hedged, absent); and reporting rules that aggregate results and never present a named brand unfavourably. Limitations: this is a protocol, not a result. Sample size, engine list and country coverage are targets. The results will be published as a separate record and linked from this one, together with the prompt templates, the scoring rubric and the brand selection criteria. The method is intended to repeat quarterly.
No takes yet. Share an insight, caveat, or question.
Daniel Constantin (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: