Flawed meta-analysis of a flawed literature: commentary on Versteeg et al.

Should authors review and synthesize results of their own studies in meta-analyses? Properly done, meta-analyses provide independent, quantitative assessments not only of overall effect sizes found in a literature, but of the quality and limitations of contributing studies, and the nature and sources of variations in effect size. Avoidance of bias in this process is considered paramount. The Cochrane Collaboration attempts to minimize bias in its meta-analyses by routinely excluding authors of studies from evaluation of their own studies' eligibility for a review, the extraction of data from their studies, and assessment of the risk of bias posed by the quality of their studies. Even when not expressly forbidden, published meta-analyses conducted by an investigator group that dominates a particular literature are unusual because of the appearance of the conflict of interest and therefore risk of bias. It is, therefore, rather odd that Johan Denollet and colleagues at Tilburg University provided a meta-analysis1 for which selection criteria yielded eight studies from their own group and only two from outside their investigator group, and that Denollet and his immediate colleagues also performed quality ratings on what are mainly their own studies. For a study to be included in the meta-analysis, calculation of Type D had to be by what Denollet and colleagues deemed the “standardized method”. Negative effects of Type D personality on mental and physical health are hypothesized to be the result of the co-occurrence of high social inhibition and high negative affectivity. The “standard method” adopted by Denollet and his colleagues is to construct a cross-tabulation of dichotomized social inhibition and negative affectivity and to select out the high/high quadrant for comparison with the other three quadrants. However, the generally accepted practice for determining the joint effects of two personality variables is to enter them as continuous variables in regression equations with appropriate adjustment for confounds and then to examine their interaction effect.2 There is a longstanding consensus that dichotomization of continuous variables loses valuable information and that construction of a 2 × 2 cross-tabulation introduces a high probability of spurious results.3,4 We are unaware of this assessment being contested anywhere in the contemporary literature. We have difficulty believing that Denollet and his colleagues have never encountered corresponding methodological and statistical criticisms in the review of their many papers, yet nowhere can we find acknowledgment or rebuttal. This undoubtedly has contributed to the dismissal by the larger field of the Type D personality literature that Denollet has lamented.5 Thus, the requirement that studies of Type D personality adopt the “standard method” in order to be included in this meta-analysis effectively excludes methodologically and statistically adequate studies, but includes studies that would be judged seriously flawed by the rest of the research community. Exploration of correlates of Type D personality with mental health outcomes are not particularly informative because one component of Type D – negative affectivity – is as highly correlated with measures of depression and other negative mental health outcomes, as their internal consistency allows.6 Apparent advantages of Type D personality that are seen when it is compared with measures of depression should also be anticipated, but can be expected to be spurious – the result of the construction of the Type D variable. We are confident that if a measure of depression were substituted for the measure of negative affectivity used to construct the measure of Type D, it too would be shown to be superior to alternatives – again, the result of statistical tricks. Multivariate analyses of the relationship of Type D personality to physical health status are similarly plagued by spurious results and, here too, any apparent advantage of Type D personality over other predictors is to be anticipated and would most likely disappear if more appropriate means of data reduction and analyses were employed. The meta-analysis article ends with a statement familiar from the conclusions of other articles by the Tilburg group: “timely identification and treatment of high-risk patients with Type D personality is warranted to improve individualized care for the physical and mental health status”. Yet there is no empirically based treatment for Type D personality, no empirical demonstration that such treatment will improve physical and mental health status, and nothing further added by this flawed meta-analysis to change that assessment.