On 2020-09-18 20:29:54, user David C. Norris, MD wrote:
This paper is fundamentally misconceived:
Biostatistically
This paper apparently arises out of the biostatistical perspective which presently dominates the design and analysis of dose-finding trials in oncology. Yet even by purely statistical standards, it suffers serious shortcomings. Most notably, it looks for an interaction (viz., dose-response) without first demonstrating or ensuring the existence of a main effect. Reference #153 in this paper (Hazim et al. 2020) reported a 5% median response rate in a systematic review of recent dose-finding trials. Would the authors venture to estimate what fraction of their 93 ‘analysis series’ employed a drug with a substantial therapeutic effect? Some indication might be found in what fraction of the treatments unequivocally demonstrated a therapeutic effect in subsequent phase 2 or 3 trials. Adashek et al. (2019) document a secular trend in overall response rate (ORR) observed in phase 1 trials which is “now almost 20%, or even higher (~42%) when a genomic biomarker is used for patient selection.”
Also arguably well within the purview of biostatistics would have been a decision-theoretic framing of phase 1 cancer trials. These trials may be understood as the earliest clinical steps in a learn-as-you-go (adaptive) drug-development process (Palmer 2002; Berry 2004). On such an understanding, aiming to treat early-phase participants at maximum tolerated doses (MTDs) in no way “dictates that an assumption is made … that higher doses are always more efficacious” (p. 4; italics in original). The authors’ use of “dictates” suggests they see something of logical necessity in this, and their further insertion of the logical quantifier “always” only exacerbates their overreach in formulating this central tenet of their study. Even the distinction between a logical assumption and a statistical prior gets lost in the shuffle. To remedy all this, the authors might consider attempting to state formally their understanding of the individual phase 1 trial participant’s decision-problem, complete with its essential uncertainties and some plausible utilities. (Within the community of investigators whom they address in the final paragraph of their Discussion, there is, I believe, broad agreement on the doctrine that these trials have therapeutic intent (Weber et al. 2016; Burris 2019). The authors would do well to take this patient-centered view as their starting point, as opposed to the dose-centered and unitary goal they proclaim at the end of their current Discussion.)
Furthermore, statistics is nothing if not a discipline for “mastering variation” (Senn 2016), and a paper that sets out to question the strict monotonicity of dose-efficacy ought also enquire as to the presence of inter-individual heterogeneity in dose-response. Note that such heterogeneity would tend to attenuate the maximum slope of a convex dose-response in aggregate.
Finally, the absence-of-evidence fallacy is widely appreciated among professional statisticians, yet seems to have been indulged liberally here without any safeguards such as are usually provided by power calculations.
Pharmacologically
Within statistics, there is a doctrine that statistical analysts should always engage ‘subject-matter experts’. But one sees in this paper no sign that any pharmacological concepts—let alone expertise—have been brought to bear on what would seem to be a pharmacological question. At a minimum, in any serious challenge to the ‘MTD heuristic’—as I have called it—one expects to find distinctions between on-target and off-target toxicities. In an analysis that invokes dose-response plateaus (whether these are conceived as approximate or absolute in this paper remains unclear), we ought to find discussion of receptor occupancy and saturation as underlying realistic mechanisms.
To some extent, a neglect of subject-matter knowledge may be embedded in the very form of the present analysis, which tries to deal with its question in aggregate (through statistical techniques such as standardization) rather than in its particulars.
Clinically
In the final paragraph of their Discussion, the authors proffer advice to clinical investigators. In light of the limitations—statistical, logical, subject-matter—catalogued above, this is premature and should be omitted. Any given phase 1 clinical investigator will be considering a candidate drug in its particulars, conditional on a great deal of preclinical data and perhaps even nontrivial PKPD and systems-pharmacology modeling. The authors acknowledge as much (p. 16), seeming to appreciate that they have conducted an unconditional analysis of highly conditioned decision-making. To investigators thus intimately engaged with pharmacologic particulars, the null conclusions from a marginal analysis such as this one can contribute little useful guidance. If it were proposed to submit this work for peer review in substantially its present form, only a statistical audience should be addressed—and then solely with a cautionary note that the finding of a dose-response interaction will not leap out at a statistician from a convenience sample of phase 1 studies in which a therapeutic main effect remains dubious and unexamined. The main lesson of this work is that statisticians ought to investigate questions of pharmacology in their particulars, and with recourse to subject-matter concepts and expertise.
References
Adashek, Jacob J., Patricia M. LoRusso, David S. Hong, and Razelle Kurzrock. 2019. “Phase I Trials as Valid Therapeutic Options for Patients with Cancer.” Nature Reviews Clinical Oncology, September. https://doi.org/10.1038/s41....
Berry, Donald A. 2004. “Bayesian Statistics and the Efficiency and Ethics of Clinical Trials.” Statistical Science 19 (1): 175–87. https://doi.org/10.1214/088....
Burris, Howard A. 2019. “Correcting the ASCO Position on Phase I Clinical Trials in Cancer.” Nature Reviews Clinical Oncology, December. https://doi.org/10.1038/s41....
Hazim, Antonious, Gordon Mills, Vinay Prasad, Alyson Haslam, and Emerson Y. Chen. 2020. “Relationship Between Response and Dose in Published, Contemporary Phase I Oncology Trials.” Journal of the National Comprehensive Cancer Network 18 (4): 428–33. https://doi.org/10.6004/jnc....
Palmer, C. R. 2002. “Ethics, Data-Dependent Designs, and the Strategy of Clinical Trials: Time to Start Learning-as-We-Go?” Statistical Methods in Medical Research 11 (5): 381–402. https://doi.org/10.1191/096....
Senn, Stephen. 2016. “Mastering Variation: Variance Components and Personalised Medicine.” Statistics in Medicine 35 (7): 966–77. https://doi.org/10.1002/sim....
Weber, Jeffrey S., Laura A. Levit, Peter C. Adamson, Suanna S. Bruinooge, Howard A. Burris, Michael A. Carducci, Adam P. Dicker, et al. 2016. “Reaffirming and Clarifying the American Society of Clinical Oncology’s Policy Statement on the Critical Role of Phase I Trials in Cancer Research and Treatment.” Journal of Clinical Oncology 35 (2): 139–40. https://doi.org/10.1200/JCO....