Analysis of Survival by Tumor Response and Other Comparisons of Time-to-Event by Outcome Variables

Twenty-five years ago, in its first volume, the Journal of Clinical Oncology published our methodologic article, which highlighted an inappropriate analytic approach then in common use and suggested alternative unbiased techniques. Cited more than 380 times, this article and similar ones by other authors have had a large impact on how results from early-phase clinical trials are reported, almost completely eliminating so-called survival by tumor response analyses from the oncology literature. The success of treatments for patients with advanced cancers at diagnosis or recurrence is often assessed by computing overall response rates and survival from the start of treatment. Before 1983, analysis of survival by tumor response category was commonly included as part of the reports of these treatments. Patients were characterized as responders or nonresponders, estimates of survival from the start of treatment were calculated for each responder category, and differences in survival by these tumor response categories were compared using a significance test appropriate for time-to-event data. These analyses had been used to bolster the claim of benefit for the treatment, often as a surrogate for a randomized trial. Responders benefited from the therapy and thus constituted the treatment group. Nonresponders did not benefit, and thus their survival could be considered similar to that for untreated patients. Similarly, these analyses had also been used to argue that therapies that increase the response rate should necessarily result in increased survival. In March 1983, Weiss et al reported the results of a review of articles published in Cancer or Cancer Treatment Reports and found 228 articles presenting data on responders and nonresponders, with 61% containing formal statistical comparisons of survival by tumor response category. They identified problems in the interpretation of these comparisons. In November 1983, our article in the Journal of Clinical Oncology, “Analysis of Survival by Tumor Response,” was published. We showed that the usual methods of comparing responders with nonresponders were wrong, leading to biased estimates of the survival distributions, invalid statistical tests, and misleading conclusions. This bias results in part from the fact that responders must live long enough for response to be observed; there is no such requirement for nonresponders. We described several valid approaches to comparing survival by response category. One approach, the landmark method, determines each patient’s response at some fixed time point, with survival estimates calculated from that time point and associated statistical tests being conditional on patients’ landmark responses. Note that in this method, patients who die before the landmark time point are excluded from the analysis. An alternative approach treats response status as a timedependent covariate, where all patients begin in the nonresponse state and patients move to the response state at the time of their response. Shortly thereafter, Simon and Makuch proposed a method of obtaining estimates of survival probabilities for responders and nonresponders, combining ideas from the landmark and time-dependent covariate approaches. Even when these analyses were performed appropriately, we argued that longer survival for responders, as compared with nonresponders, could not be used to conclude that response caused longer survival. Response might act as a surrogate marker for prognostically favorable patients. Thus responders may survive longer than nonresponders, not because of an effect of response on survival, but because response identifies patients with pretreatment characteristics that favor longer survival. It is generally difficult to distinguish between cases where response prolongs survival and cases where it simply acts as a marker for favorable-prognosis patients. In 1985, Cancer Treatment Reports indicated it would not publish comparisons of survival by tumor response. Notwithstanding the reports by Weiss et al and us, Anderson and Davis showed that between July 1984 and June 1985, the Journal of Clinical Oncology published 18 articles that included analyses of survival by tumor response, 10 of which provided inappropriate statistical comparisons of survival of responders and nonresponders. They suggested that the Journal of Clinical Oncology follow the lead of Cancer Treatment Reports and no longer publish articles that include survival by tumor response. An editorial accompanying the letter by the then-editor of the Journal of Clinical Oncology, Joseph Bertino, MD, indicated that “authors should not compare survival of responders and nonresponders without discussing the limitations of such a comparison”. JOURNAL OF CLINICAL ONCOLOGY C E L E B R A T I N G 2 5 Y E A R S O F J C O VOLUME 26 NUMBER 24 AUGUST 2