Blood flow distribution to the respiratory muscles – physiology versus statistics

Ramsook and colleagues in a recent article in Experimental Physiology investigated how blood flow is distributed to the sternocleidomastoid (SCM) – an accessory respiratory muscle – during cycling exercise of different intensities and also when mimicking the ventilatory response of exercise (hyperpnoea alone) (Ramsook et al., 2020). The authors compared blood flow index (BFI, a near-infrared spectroscopy (NIRS)-based technique used as a surrogate for blood flow itself) and electromyographical activity. It was concluded that blood flow to the SCM is influenced by whole-body exercise, that is, it received less blood during cycling than during hyperpnoea alone even at low intensities, in contrast to what was seen at the seventh intercostal space, another muscle involved with inspiration. While the discussion and conclusion derived from the findings are reasonable from a physiological point of view, a serious issue with regards to the statistical tests performed apparently went unnoticed and can impact the certainty around the interpretation and conclusion from the dataset. The realization of such issues was only possible due to the authors’ unusual diligence in reporting the statistical tests performed, for which they should be commended. Far from being an exception to be called out, some aspects that will be raised are common in the literature (including some of my own previous work, unfortunately). Given the recent efforts by The Physiological Society to improve statistical reporting and adequacy, this could be an opportunity to raise awareness to these important aspects, especially as they still go unnoticed by peer-reviewers, editors and authors. Power calculation for determination of the required sample size was performed a priori, as recommended, but the authors inadvertently used wrong inputs in their calculations: ‘assuming a medium effect size (0.5) based on the close association with respiratory muscle blood flow and minute ventilation … the analysis suggested a sample size of n = 4’ (Ramsook et al., 2020). The problem is twofold: first, and more importantly, the authors mistook an effect size F (as the input is called in G*Power under a repeated measures this design) for Cohen's D (our assumption, as Cohen's D is defined as 0.5 for a medium effect). To put this in perspective, a much more simple paired Student's t-test would require an effect size of 2.2 (more than twice a ‘large’ effect of 1.0) to require only n = 4. The issue is further compounded by the assumed correlation between respiratory muscle blood flow and minute ventilation: the authors most likely refer to the required correlation needed for the sample size calculation in G*Power, but this refers to test–retest correlation. Given the high variability of NIRS data, the 0.9 chosen by the authors is likely too high (our own calculation from back-extrapolation of the sample size and other parameters into G*Power). For an illustrative example, a dependent t-test for matched pairs requires n = 34 to detect an effect size D = 0.5 (or more precisely DZ) with 80% power. While underpowered analyses are by no means uncommon in the scientific literature, this case serves as a cautionary tale of the need of properly assessing sample sizes with the correct equation for a given study design and also of the value of properly reporting the procedures performed. Realizing that actual power was lower than anticipated – likely between 20% and 35% instead of the targeted 80% – is crucial to interpreting the findings. Low powered tests will not only increase the false-discovery rate (i.e. the proportion of false-positive tests in the universe of all positive tests), it will also bias the results of tests with significant P-values towards more extreme effect-sizes than any real differences between the groups being investigated, which is known as the inflation effect (Colquhoun, 2014). These issues have important ramifications to the interpretation of the results: EMG data concerning the vastus lateralis did not show significant increases when intensity was increased from 30% to 60% or even 90% of peak power output, and the same was the case with BFI, in contrast to other studies (Habazettl et al., 2010). If the techniques are unable to detect differences (at the given sample size) that would be widely expected (at the very least between 30% and 90% peak power output), there is much less chance that findings in the more unpredictable SCM will reflect the true physiological phenomenon, in terms of both ‘significance’ and ‘magnitude’. With regards to the magnitude of differences, the choice of axis for the different muscles when reporting BFI might further convey an inadequate sense of magnitude. While these can result from automatic suggestion from graphing software, an active effort is needed to prevent it. For instance, the BFI data for the vastus lateralis (VL, where no difference was found for factor intensity) is given on a 0–6000 range, while BFI for the SCM (where a difference was found) is shown side-by-side on a zoomed-in 0–1000 scale, amplifying the impression of magnitude. A different perspective from this magnitude could be interpreted from the data in Table 3: while BFI for the SCM increased ∼1.5-fold when the same level of ventilation was performed without exercise at the highest investigated level (i.e. hyperpnoea vs. exercise comparison), reaching 493% rest, the VL BFI reached 411% rest during the 90% hyperpnoea alone, where presumably little or no muscle activity was present (see EMG data). Evidently, comparing NIRS data from different muscles is not a perfect exercise, but it nonetheless offers some insights into the magnitude of changes that can be seen from the technique with minimal electrical activity in the muscles. A final important point with regards to the statistical analysis in the article is that it is stated that: ‘when [my emphasis] significant main effects were observed, data were tested for significant interaction effects’. There is no statistical requirement to wait for a significant main effect in order to explore whether there are or not significant interactions. While this was likely a typo, it is telling when this wording (along with a power calculation requiring four subjects to detect moderate differences with a complex study design) goes unnoticed from design conception to publication. None declared. Sole author.