Co-authorship group significantly impacts reported arterial blood pressure variations in rodents exposed to intermittent hypoxia: a meta-research study

Laboratory conditions are known confounding factors in pre-clinical research and typically have been shown to affect mouse behaviour despite extensive protocol standardization.1 This phenomenon, called phenotypic plasticity is due to genetic–environmental interactions and leads to variability in results across laboratories using the same protocol and methods.2,3 To address this variability and the reproducibility of pre-clinical research, our group recently performed an extensive literature review with two meta-analyses aiming at elucidating the cardio-vascular impact of intermittent hypoxia (IH) exposure in rodents.4,5 In these meta-analyses, the heterogeneity in the results was considerable, and although strain and severity of hypoxic burden (FiO2) were significantly associated with mean arterial pressure (MAP),4 the available information on rodents characteristics or the severity and duration of IH only partially explained this heterogeneity even in multi-variate meta-regressions. The goal of the present work is to verify whether, beyond hypoxic burden and strain, significant heterogeneity is explained by methodological issues related to the non-standardization of the experiment between the different teams in the field. To explore the impact of the laboratory environment on rodents exposed to IH we identified collaboration networks through authorship affiliation.6 We then assessed the impact of the author group on MAP, the most frequently used cardio-vascular parameter reported in meta-analyses. This article is reported according to PRISMA recommendations. Details of our literature search strategy are given in the PROSPERO protocols (CRD42020169940 and CRD42020170266) and the published articles.4,5 We identified the author networks through the VOSviewer software.7 An author group was defined when it included three or more separate published studies from a given group of authors. We appraised the impact of the author group through a meta-analysis of mean arterial pressures (MAP) in normoxic rodent arms and then the mean differences in MAP between normoxic and hypoxic arms. The meta-analyses were performed using hierarchical random effect models allowing for correlation between arms and differences in MAP measurement methods (arterial catheterization or plethysmography). Then, we sub-grouped the results by author group to assess laboratory estimates and we tested the differences between subgroups. Finally, we performed uni- and multi-variate meta-regressions for rodents and IH protocol characteristics, and we assessed the impact of adding the author group as a new hierarchical level (in addition to studies) on the residual heterogeneity in the meta-analysis. The heterogeneity was evaluated by I2 statistics and the subgroup difference was deemed significant if P < 0.05. Statistical analyses were performed with R (version 4.1.2). Supporting information can be found on Open Science Framework (https://osf.io/86sk9/). Overall, 88 studies and 274 arms were included in this meta-analysis. The authorship dependency analysis identified 12 networks of authors who published at least 3 separate pre-clinical studies on MAP, accounting for 58 (65.9%) of the included studies. These networks were mostly groups of authors belonging to the same institution, and we could not find any study conducting the same experiment in multiple laboratories. The remaining studies were included in a large heterogeneous group (group X). The network analysis can be visualized through the following link: https://tinyurl.com/y7rzzve3. Mean FiO2 used in IH models are reported for each author group. In normoxic control groups, the MAP was 89.9 mmHg (95% confidence interval [CI 82.8, 97.1), n = 25, I2 = 96%] and 105.6 mmHg (95% CI 102.7, 108.5), n = 101, I2 = 98% in studies measuring arterial blood pressure by plethysmography and catheterization, respectively. In both meta-analyses subgroup analyses by author group was significant (P < 0.01). The average MAP in normoxic groups is presented in Figure 1A. After adjusting the meta-analysis on rodent characteristics (species, strain, sex, and bodyweight) through multi-variate meta-regressions, the author network still significantly affected the results (P < 0.01). (A) Forest plot presenting the results of the meta-analysis using the mean arterial pressures (MAP) (mmHg) in normoxic rodent groups only (left panel) and the mean differences between hypoxic and normoxic groups (right panel) by author group. (B) Meta-regression between mean arterial pressure (MAP) in normoxic rodent groups and mean difference between hypoxic and normoxic groups. The results of the meta-analysis assessing the mean difference between hypoxic and normoxic groups, sub-grouped by author group are presented in Figure 1A. Overall, IH exposure increased the MAP by 13.00 mmHg (95% CI 10.9, 15.1), n = 137 with a significant heterogeneity (I2 = 91%) and author group subgroup difference (P < 0.01). The impact of the author group was still significant despite adjusting for IH protocol characteristics, rodent characteristics, and MAP in normoxic groups through multi-variate meta-regressions. In the hierarchical meta-analysis adding a level for the author group in addition to the study decreased the residual heterogeneity from 39.6% to 23.7%. In the multi-variate meta-regressions and in the hierarchical meta-analysis including a random effect for the author group and study, the MAP in the normoxic groups (as a proxy of baseline characteristics of the model), but not the method of measurement, strongly impacted the results with a significant meta-regression slope −0.24 (−0.36, −0.12), P < 0.001, despite adjustment for all hypoxic protocol parameters (Figure 1B). Our results show that the impact of IH in rodent models is highly variable according to the author group and thus laboratory conditions. These large differences are not fully explained by the rodent’s characteristics or by the IH exposure protocol described in the articles, or the arterial pressure measurement method. A significant part of this heterogeneity could be related to the unreported environment: for example, diet and housing conditions could affect the phenotype, in particular by controlling the microbiome, known to be able to affect blood pressure and vascular function in IH.8,9 Bias in measuring the outcome and protocol conditions, or phenotypic plasticity, could also account for some heterogeneity. This variability in the results limits the generalizability of studies conducted in a single laboratory despite homogenous protocols and models. We, therefore, encourage researchers in the field to conduct studies across multiple laboratories to deliberately introduce possible differences in conditions, thus introducing variability to enhance the external validity of the findings.10,11 The choice of the model seems also particularly important since the impact of IH on arterial pressure is correlated with the baseline MAP under normoxic conditions, whatever the IH protocol applied. Finally, better standardization appears to be mandatory in view of the relatively limited number of research groups working in the field of experimental IH exposure. From an analytical perspective, these results show that including a hierarchical level for the author group in pre-clinical meta-analyses could be a way to decrease residual heterogeneity and understand inter-laboratory variability. Moreover, meta-analyses sub-grouped by author network could be a tool to highlight phenotypic plasticity in the field and a first step to designing collaborative projects. Collaborative studies applying the same protocol in multiple laboratories would be important to conduct to distinguish the impact of the laboratory environment from other confounding factors. The results of this meta-analysis are limited by the selective outcomes reporting and the risk of bias of included studies.4,5 Here we only analysed MAP and the impact of author groups on other cardio-vascular parameters remains to be studied. Similar studies are clearly needed in other pre-clinical models of hypertension, in order to confirm our results on the whole spectrum of rodent models of hypertension. Conceptualization: C.K.; Methodology: C.K.; Investigation: C.K., B.E.L., C.A., E.B., Q.B., G.F., O.H., A.B.M.; Visualization: C.K.; Funding acquisition: J.L.P.; Supervision: J.L.P., A.B., J.L.C.; Writing—original draft: C.K.; Writing—review & editing: J.L.C., A.B.M., J.L.P., D.G.R., P.L. We thank Alison Foote (Grenoble, France) for editing the manuscript. This work has been partially supported by MIAI @ university Grenoble Alpes (ANR-19-P3IA-0003).