Critical appraisal of a meta-analysis on pelvic lymph node dissection in muscle-invasive bladder cancer

Dear Editor(s), I appreciate the comprehensive nature of the meta-analysis by Madhavan et al.[1] regarding the extent of pelvic lymph node dissection in muscle-invasive bladder cancer. Upon reviewing the manuscript, I would like to offer a refined, specific, and detailed set of observations. The following points seek to clarify the limitations or nuances that, although partly acknowledged in the discussion, warrant further elaboration or a more explicit critical appraisal. DEGREE AND CLINICAL IMPACT OF METHODOLOGICAL HETEROGENEITY While the manuscript notes the differences in chemotherapy use, inclusion criteria (cT1 in Gschwend et al.[2] vs. T2+ in Lerner et al.[3]), and geographic context, the concrete implications of these variations on patient outcomes are not fully examined. For instance: Chemotherapy use: In the trial by Lerner et al., >55% of the patients received neoadjuvant chemotherapy (NACT), whereas in the trial by Gschwend et al. these patients were excluded. This fundamentally changes not just the tumor staging at the surgery (e.g., more pT0 in the NACT group) but possibly the very effect size being measured, since the lymph node status post-NACT may be biologically and prognostically distinct Stage inclusion: Combining patients with cT1 disease (better prognosis and less nodal involvement) with those who have cT2-T4 brings in clinical heterogeneity that can dilute the specific signals relating to the value of extensive lymphadenectomy in the higher stage disease The manuscript references these differences but does not provide sensitivity analyses, subgroup comparisons, or a quantified estimate of their influence on the meta-analytic results. STATISTICAL HETEROGENEITY AND ITS HANDLING The reported I² for overall survival (OS) and early mortality is high (OS: 69%, 30-day mortality: 71%), indicating substantial statistical heterogeneity. The current text notes heterogeneity and the use of random-effects models but does not dissect the likely contributors to this variability or how it might affect the overall confidence in the pooled results No further action is reported, such as leave-one-out analysis, subgroup meta-analysis (e.g., by chemotherapy status or node yield), or meta-regression to reduce or explain heterogeneity Such analyses are crucial because a high I² in a meta-analysis of just two studies can indicate real differences whose clinical impact ought not be dismissed as "random variation." CERTAINTY OF EVIDENCE AND ITS CLINICAL RELEVANCE The authors downgraded their certainty estimates, noting the small number of studies and wide confidence intervals crossing unity (no effect). However, the practical implication – whether clinicians should or should not change the practice based on these findings – remains somewhat ambiguous The manuscript could further emphasize the very low certainty for RFS and only low certainty for OS, as per GRADE, which should signal to clinicians and regulatory bodies (including those, like the Food and Drug Administration, using such evidence) to retain equipoise and continue research Moreover, while the post hoc power is reported for a 10% difference, actual event rates may be lower due to contemporary therapy, and small but important effects may not be detectable, underscoring the risk of type II error. GENERALIZABILITY AND SUBGROUP CONSIDERATIONS Though the manuscript briefly suggests that "select patients" may benefit from extended dissection, it would be valuable to specifically call out: The absence of subgroup or individual patient data analyses means the results might not apply to those with high-risk features, bulky nodes, variant histology, or younger/older age groups The trials' populations were relatively homogeneous in terms of performance status and comorbidity, unlike many real-world patients who may differ in surgical tolerance and risk profiles. PRISMA TRANSPARENCY AND LITERATURE HANDLING The review references PRISMA compliance, but: The selection process, precise yield of the database search, and reasons for exclusion of studies (including potentially eligible nonrandomized data) are not summarized in the main text A more thorough explanation, including a brief description in the main narrative, increases transparency and replicability, a cornerstone of systematic review methodology. In summary, while many limitations are acknowledged in the discussion, a more explicit treatment of clinical and statistical heterogeneity, GRADE certainty implications, and subgroup limitations would be valuable. Given your background in regulatory science, these aspects are crucial for those assessing not only the clinical practice but also the evidence standards for health policy and potential regulatory actions. Thank you for the opportunity to provide these clarifications. Financial support and sponsorship: Nil. Conflicts of interest: There are no conflicts of interest.