A chain of evidence with mixed comparisons: models for multi‐parameter synthesis and consistency of evidence

Multi-parameter evidence synthesis is a generalization of meta-analysis in which several parameters are estimated jointly. Here this approach is applied to a common form of data structure in which mixed pairwise comparisons are made between treatments in a 'chain of evidence' structure. The data set investigated here, which first appeared in the confidence profile method literature, features three types of data relating to thrombolytic treatment following acute myocardial infarction. There is information on reperfusion (coronary patency) following treatment, information on survival in reperfused and non-reperfused patients (conditional survival), and independent information on the effect of treatment on survival without specifying reperfusion state (overall survival). The objective of this study is to explore models for combining these three types of evidence within a single model, and some of the evidence consistency issues that arise. Bayesian Markov chain Monte Carlo methods are used to fit two models. The first, proposed in the confidence profile literature, assumes there are no differences between studies in baseline effects (equal study effects). The second model assumes fixed treatment effects but allows for study differences in baseline reperfusion and conditional survival rates by assuming random study effects. The equal study effects model fits the data poorly, and cross-validation shows that the overall survival evidence is not consistent with the reperfusion and conditional survival evidence under this model. The evidence appears to be consistent within the more loosely structured random study effects model, but results are sensitive to prior assumptions about the between-study variance. The inconsistency between the three evidence types relates to overall survival, rather than to the effect of treatment on survival. Multi-parameter evidence synthesis, with suitable checks for evidence consistency, can be used to help determine whether chains of evidence 'add up' epidemiologically within an overall model, and to construct parameter distributions for an entire model simultaneously, consistent with all the available data.