Article Tools
Clinical Trials
Article Tools
Translation of Innovative Designs Into Phase I Trials
Abstract
Phase I clinical trials of new anticancer therapies determine suitable doses for further testing. Optimization of their design is vital in that they enroll cancer patients whose well-being is distinctly at risk. This study examines the effectiveness of knowledge transfer about more effective statistical designs to clinical practice.
We examined abstract records of cancer phase I trials from the Science Citation Index database between 1991 and 2006 and classified them into clinical (dose-finding trials) and statistical trials (methodologic studies of dose-escalation designs). We then mapped these two sets by tracking which trials adopted new statistical designs.
One thousand two hundred thirty-five clinical and 90 statistical studies were identified. Only 1.6% of the phase I cancer trials (20 of 1,235 trials) followed a design proposed in one of the statistical studies. These 20 clinical studies showed extensive lags between publication of the statistical paper and its translation into a clinical paper. These 20 clinical trials followed Bayesian adaptive designs. The remainder used variations of the standard up-and-down method.
A consequence of using less effective designs is that more patients are treated with doses outside the therapeutic window. Simulation studies have shown that up-and-down designs treated only 35% of patients at optimal dose levels versus 55% for Bayesian adaptive designs. This implies needless loss of treatment efficacy and, possibly, lives. We suggest that regulatory agencies (eg, US Food and Drug Administration) should proactively encourage the adoption of statistical designs that would allow more patients to be treated at near-optimal doses while controlling for excessive toxicity.
This year, some 1,440,000 new cancer patients are expected to be diagnosed and approximately 560,000 Americans are expected to die of cancer—more than 1,500 people per day.1 Commensurate with the burden cancer places on society, efforts to develop new therapies have never been greater; there are some 399 medicines to treat cancer now actively in development.2
Clinical trials of new anticancer therapies are widespread, critically important tools in the search for more effective cancer treatments. Cancer trials typically proceed through several distinct phases. The major objective in phase I trials is to identify a working dose for subsequent studies, whereas the primary end point in phase II and III trials is treatment efficacy. Phase I trials represent the first testing of an investigational agent in humans and act as a point of translation of years of laboratory research into the clinic. Whereas phase I trials in other areas of medicine enroll healthy participants, phase I oncology trials typically enroll patients who have cancer and who have exhausted standard treatment options.3 As we write, there are 1,384 phase I trials open to cancer patients in the United States.4
Dose finding is the critical first step in the process to scrutinize whether a new agent will help cancer patients. The fundamental conflict underlying the design of cancer phase I clinical trials is that increasing the dose slowly to avoid unacceptable toxic events must be balanced against treating many patients at suboptimal or nontherapeutic doses.5 Ideally, from a therapeutic perspective, clinical trials should be designed to maximize the number of patients receiving an optimal dose. Consequently, more patients would be treated with therapeutic doses of promising new agents, and fewer patients would have to suffer the deleterious effects of toxic doses.
There has been a substantial effort from the cancer research and treatment community to expedite translation of basic science and laboratory findings into clinical trials and to accelerate improvement in cancer patients' survival and quality of life. Research biostatisticians dedicated to the discovery and improvement of clinical trial design play a fundamental role in the process of drug discovery and treatment. As in other domains, they generate new methods that are published in specialized journals, using a language typical to the profession. The main objective of this study is to quantify the transfer of knowledge from statistical research about the design of cancer phase I trials to clinical researchers actually performing phase I trials.
The following steps were taken to quantify the transfer of knowledge from statistical research about the design of cancer phase I trials to clinical researchers actually performing such trials. First, we sought and examined all abstract records of cancer phase I trials from the Science Citation Index database published between 1991 and 2006. Second, we classified them into clinical (phase I trials in humans designed to find a maximum-tolerated dose [MTD] of a cancer regimen) and statistical trials (methodologic studies of dose-escalation designs of cancer regimens published in statistical journals). Third, we then mapped these two sets by tracking which clinical phase I trials cited and adopted any of the new statistical designs.
Because our analysis required access to the cited reference list for the published phase I clinical trials, the primary database for biomedical research publication, PubMed (MEDLINE),6 was not suitable. Instead, we used the Science Citation Index Expanded,7 which compiles fundamental research from 22,000 journals and provides ample coverage of the biomedical literature.
We determined that the best results would have variants of the strings “phase one” and “clinical trial” in the title, abstract, or keywords sections of the record. We used the “TS” field tag in our search strategies to restrict our search to these content fields. The Boolean SAME operator is used to add an additional constraint that the two strings occur within the same sentence when found in an abstract. Because phase I type trials exist for other diseases, “cancer” is included as a search term. Additional limits were set such that we retrieved journal articles from 1991 to 2006 so we could profile trends over the past 15 years. The resulting Boolean search algorithm was as follows:
TS = ((((phase I) OR (phase 1) OR (phase one)) SAME (study or studies OR trial*)) AND cancer AND (patients OR subjects)))
As of February 13, 2006, this search strategy matched 3,527 records. We downloaded the resulting sets of abstract records, including the references cited by each of those papers. We then used text mining software, VantagePoint (Search Technology Inc, Norcross, GA),8 to remove duplicates (10 duplicates removed), consolidate name variations, and perform other data cleaning. We then removed 140 records for which there was no abstract text. The remaining 3,377 Science Citation Index records were then classified as discussed in the next section.
Abstracts were assigned to one of the following four outcomes.
These were methodologic studies of dose-escalation designs of cancer regimens published in statistical journals.
These were phase I trials in humans designed to find the MTD of a cancer regimen.
These trials were often designed to demonstrate successful administration of a new compound to humans without attempting to escalate a dose, often seen with biologic compounds. Other examples were the use of established single or combination chemotherapeutic or targeted anticancer agents with predetermined doses in other cancers being applied for the first time to a different cancer.
Abstracts were excluded most often because they were reviews and had been included in the initial database because the phrase “phase I” was in the abstract. Other frequent reasons for exclusion related to the use of the phrase “phase I” without the study truly being a phase I trial, as in the following situations: data presented supporting a future phase I trial, the use of patients' samples from a phase I trial in other trials, and the validation of an assay using patients in phase I trials.
Figure 1 shows the parsing of the search results. We located 3,377 records. Ninety abstracts reflected methodologic studies of dose-escalation designs of cancer regimens published in statistical journals. From the remaining 3,287 nonmethodologic abstracts, we set aside 177 as proof-of-principle trials and 1,875 as not clinical phase I trials.
The remaining 1,235 abstracts described a dose-escalation design to find the MTD of a new agent or combination. They were published in 116 journals. Two journals had more than 100 papers (Clinical Cancer Research and Journal of Clinical Oncology). Another 14 journals published 20 or more papers; 53 journals published two or more; and the remainder (47 journals) each had one paper. Regarding the 90 statistical papers, they were published in 13 statistical journals. The distribution of occurrences was also highly skewed: 38 were in Statistics in Medicine; 27 were in Biometrics; and the other 11 journals published one to four papers each.
Figure 2 breaks out the 1,235 clinical and 90 statistical papers by year of publication. Connecting lines correspond to citations of statistical papers by clinical papers. Only 1.6% of the phase I cancer trials (20 of 1,235 trials) followed a design proposed in one of the statistical papers. These 20 clinical studies showed extensive time lags since publication of the statistical paper or shared coauthorship between the clinical and statistical paper.
An overwhelming 98.4% of the clinical trials (1,215 of 1,235 trials) followed variations of the standard up-and-down method.9,10 Only 20 trials used Bayesian adaptive designs (17 used continual reassessment method [CRM]5,11-13 and three used escalation with overdose control [EWOC]).14-16
The standard modified Fibonacci up-and-down method of dose escalation has been by far the favored design for phase I trials. Despite being the most used design in dose-finding cancer phase I trials, there is an unusual degree of confusion regarding its description. The standard method of dose escalation consists of the following two parts: a collection of doses to be ascertained and an algorithm to select the best dose or MTD from this set of doses. The dose sequence is commonly referred to as a modified Fibonacci sequence. The algorithm used to search for the MTD is referred to as up-and-down.
The Fibonacci numbers are the sequence of numbers defined by the linear recurrence equation Fn = Fn − 1 + Fn − 2 with F1 = F2 = 1. Thus, the Fibonacci numbers for n = 1, 2, … are 1, 1, 2, 3, 5, 8, 13, 21, … The ratio of successive Fibonacci numbers Fn/Fn − 1 approaches the golden ratio as n approaches infinity.17 Whereas there is a single definition of a Fibonacci sequence, what is a modified Fibonacci sequence? Unsatisfactorily, in the context of a phase I trial, the answer is any sequence of increasing numbers constructed by the investigators of a clinical protocol before the trial begins! The label of modified Fibonacci adds gravitas to a set of numbers that lacks any other meaningful attribute.
Another source of confusion is to relate the dose-finding procedure in a phase I trial to a Fibonaccian search.18 The Fibonaccian search algorithm19 finds the maximum of a unimodal function on an interval by evaluating points placed according to a Fibonacci sequence. The Fibonaccian search method minimizes the maximum number of evaluations needed to reduce the interval of uncertainty to within the prescribed length. The Fibonaccian search has not been used in cancer phase I trials because it is unethical to ascribe a higher than necessary dose to a cancer patient. The most common method, up-and-down, is described next.
The up-and-down approach was first described by Dixon and Mood9 on testing the sensitivity of explosives to shock. It was assumed that the variable under analysis was normally distributed and that the sample size must be large if the analysis to be described is to be applicable. They concluded that the up-and-down method was particularly effective for estimating the mean. Brownlee et al20 studied the properties of the up-and-down method with small samples and demonstrated that the Dixon-Mood formula for the asymptotic variance was reasonably reliable even in samples as small as five to 10, but they did not consider the problem of estimating the scale parameter σ. Statistical properties of the up-and-down method are presented in Ivanova and Flournoy.21 Edler22 provided some historical details on dose finding in phase I cancer trials and concluded, “The most popular and most cited scheme is the so-called modified Fibonacci dose escalation scheme. A review of the relevant literature does not provide an original justification of this method; it seems to be more often quoted and copied than truly understood.”
The greatest advantage of the up-and-down design is its simplicity and ease of implementation. It does not require either biostatistical or computational support. Regarding its disadvantages, there are many.
First, it has poor small-sample properties (eg, estimators of the MTD tend to be biased or inconsistent).20,23,24 Second, it is anticonservative, assigning toxic doses to a relatively large number of patients.23-25 Several alternative methods perform better than the traditional method by minimizing both under- and overdosing. For example, in simulation studies, up-and-down methods treated, on average, 35% of the patients at optimal dose levels, whereas EWOC treated 55% of the patients at optimal levels. Furthermore, EWOC assigned fewer patients to either subtherapeutic or severely toxic dose levels and estimated the MTD with smaller average bias and mean squared error than the up-and-down methods.26
Third, the choice of the targeted probability of dose-limiting toxicity is severely restricted. Fourth, the choice of dose at any stage of the trial is based only on the responses observed in the last group of patients. Therefore, all of the information from any other cohort is not used.
Finally, the succession of dose levels is completely determined by choices made before the onset of the trial. Adaptive methods are preferable because deciding on the magnitude of the jump from one dose to another can incorporate the results obtained so far. We showed that, as a trial progresses, the change between consecutive EWOC dose assignments becomes smaller and smaller. Eventually, all patients beyond a certain time would be treated at doses sufficiently close to the MTD.15
The cancer research and treatment community is working hard to translate basic science and laboratory findings faster into clinical trials. This reflects the commitment to accelerate improvement in cancer patients' survival and quality of life. Research biostatisticians dedicated to the discovery and improvement of clinical trial design play a fundamental role in the process of drug discovery and treatment. As in any science, biostatisticians generate results (in their case, improved methods) that are published in specialized journals using a language typical to the profession. Sadly, most of these new methods are rarely used, although they may offer advantages over the currently adopted standard. Granted, the original CRM suffered some criticism in that it tends to assign more patients to toxic doses compared with traditional 3 + 3 designs, it takes a longer time to complete the trial compared with traditional designs, and there are statistical model assumptions. However, Goodman et al12 proposed some modifications to the CRM by reducing the length of the trial and incidence of toxicities. Moreover, O'Quigley and Shen13 showed that the method is robust under a reasonable class of model misspecifications. Despite a compelling rationale for their use, EWOC and other Bayesian methods are not commonly used in phase I trials. Some of the possible reasons have been discussed elsewhere.27,28
As biostatisticians, it is our responsibility not only to develop new and better designs, but also to shepherd new approaches into clinical practice. Are such difficulties in research knowledge utilization typical or unusual? Regrettably, sluggish transfer of data is not so unusual29; information does not generally flow effectively and rapidly30 far from its research domain. Decades of experience regarding technologic innovation indicate that direct contact is the best mechanism for knowledge transfer.31 In our phase I trials, the two instances of relatively rapid uptake (1998 and 2002 statistics papers) both involved direct personal knowledge; each had a shared coauthor on the clinical paper.
Our data suggest a startling paucity of translation of modern statistical methodology into the design of phase I cancer clinical trials. As indicated in Figure 2, when transfer of phase I design improvements into clinical practice does occur, it tends to be quite slow; the time lags are notable. How strong are impediments to change current clinical research practices, and what can be done to reduce these? Improved uptake of alternative experimental designs in clinical trials will require effective, two-way communication between biostatisticians and clinical colleagues. However, the most effective and fastest way to improve the present situation is a change in the attitude of the gatekeeper—regulatory agencies. The US Food and Drug Administration should proactively encourage the adoption of statistical designs that would allow more patients to be treated at near-optimal doses while controlling for excessive toxicity. After all, the real justification for using the standard modified Fibonacci up-and-down design is tradition (ie, “This is how we always have done it”) and comfort level, not scientific reasoning or clinical evidence.
The author(s) indicated no potential conflicts of interest.
Conception and design: André Rogatko, Alan Porter
Financial support: André Rogatko
Administrative support: André Rogatko
Provision of study materials or patients: André Rogatko
Collection and assembly of data: André Rogatko, William Jonas
Data analysis and interpretation: André Rogatko, David Schoeneck, William Jonas, Mourad Tighiouart, Alan Porter
Manuscript writing: André Rogatko, Fadlo R. Khuri, Alan Porter
Final approval of manuscript: André Rogatko
Fig 1. Breakdown of cancer phase I articles identified. SCI, Science Citation Index; MF-UD, modified Fibonacci up-and-down; CRM, continual reassessment method; EWOC, escalation with overdose control.
Fig 2. Clinical cancer phase I articles citing statistical articles. Each dot represents an article. Connecting lines correspond to citations of statistical papers by the clinical articles.
Supported in part by grants from Georgia Cancer Coalition, Distinguished Cancer Clinicians and Scientists Program (A.R., F.R.K.), Robert W. Woodruff Health Sciences Center Fund (A.R.), and NIH/NCI Grants No. 1 P01 CA116676 (F.R.K., A.R., M.T.) and P20 CA103735 (A.R., F.R.K.). A.R. is a Georgia Cancer Coalition Distinguished Scholar; F.R.K. is a Georgia Cancer Coalition Distinguished Clinician.
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
| 1. | American Cancer Society: Cancer Facts and Figures 2007. http://www.cancer.org/docroot/STT/content/STT_1x_Cancer_Facts__Figures_2007.asp Google Scholar |
| 2. | Pharmaceutical Research and Manufacturers of America: Medicines in development. http://www.phrma.org/newmedicines/resources Google Scholar |
| 3. | Roberts TG Jr, Goulart BH, Squitieri L, et al: Trends in the risks and benefits to patients with cancer participating in phase 1 clinical trials. JAMA 292: : 2130,2004 -2140, Crossref, Medline, Google Scholar |
| 4. | National Institutes of Health: ClinicalTrials.gov. http://www.clinicaltrials.gov Google Scholar |
| 5. | O'Quigley J, Pepe M, Fisher L: Continual reassessment method: A practical design for phase 1 clinical trials in cancer. Biometrics 46: : 33,1990 -48, Crossref, Medline, Google Scholar |
| 6. | National Center for Biotechnology Information: PubMed. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi Google Scholar |
| 7. | Thomson Scientific: Science Citation Index Expanded. http://scientific.thomson.com/products/scie/ Google Scholar |
| 8. | Search Technology Inc: VantagePoint. http://www.thevantagepoint.com/ Google Scholar |
| 9. | Dixon WJ, Mood AM: A method for obtaining and analyzing sensitivity data. J Am Stat Assoc 43: : 109,1948 -126, Crossref, Google Scholar |
| 10. | Rosenberger WF, Haines LM: Competing designs for phase I clinical trials: A review. Stat Med 21: : 2757,2002 -2770, Crossref, Medline, Google Scholar |
| 11. | Potter DM: Adaptive dose finding for phase I clinical trials of drugs used for chemotherapy of cancer. Stat Med 21: : 1805,2002 -1823, Crossref, Medline, Google Scholar |
| 12. | Goodman SN, Zahurak ML, Piantadosi S: Some practical improvements in the continual reassessment method for phase I studies. Stat Med 14: : 1149,1995 -1161, Crossref, Medline, Google Scholar |
| 13. | O'Quigley J, Shen LZ: Continual reassessment method: A likelihood approach. Biometrics 52: : 673,1996 -684, Crossref, Medline, Google Scholar |
| 14. | Babb JS, Rogatko A: Patient specific dosing in a cancer phase I clinical trial. Stat Med 20: : 2079,2001 -2090, Crossref, Medline, Google Scholar |
| 15. | Zacks S, Rogatko A, Babb J: Optimal Bayesian-feasible dose escalation for cancer phase I trials. Stat Prob Lett 38: : 215,1998 -220, Crossref, Google Scholar |
| 16. | Tighiouart M, Rogatko A, Babb JS: Flexible Bayesian methods for cancer phase I clinical trials: Dose escalation with overdose control. Stat Med 24: : 2183,2005 -2196, Crossref, Medline, Google Scholar |
| 17. | Chandra P, Weisstein EW: Fibonacci number. http://mathworld.wolfram.com/FibonacciNumber.html Google Scholar |
| 18. | Omura GA: Modified Fibonacci search. J Clin Oncol 21: : 3177,2003 , Link, Google Scholar |
| 19. | Black PE: Dictionary of Algorithms and Data Structures. http://www.nist.gov/dads/HTML/fibonaccianSearch.html Google Scholar |
| 20. | Brownlee KA, Hodges JL, Rosenblatt M: The up-and-down method with small samples. Am Stat Assoc J 48: : 262,1953 -277, Crossref, Google Scholar |
| 21. | Ivanova A, Flournoy N: Group up-and-down designs in toxicity studies, in Chevret S (ed): Statistical Methods for Dose-Finding Experiments (ed 1) . West Sussex, United Kingdom, John Wiley and Sons, Ltd, , pp 115,2006 -128 Google Scholar |
| 22. | Edler L: Statistical requirements of phase I studies. Onkologie 13: : 90,1990 -95, Medline, Google Scholar |
| 23. | Storer BE: Design and analysis of phase I clinical trials. Biometrics 45: : 925,1989 -937, Crossref, Medline, Google Scholar |
| 24. | O'Quigley J: Theoretical study of the continual reassessment method. J Stat Planning Inference 136: : 1765,2006 -1780, Crossref, Google Scholar |
| 25. | O'Quigley J, Chevret S: Methods for dose finding studies in cancer clinical trials: A review and results of a Monte Carlo study. Stat Med 10: : 1647,1991 -1664, Crossref, Medline, Google Scholar |
| 26. | Babb J, Rogatko A, Zacks S: Cancer phase I clinical trials: Efficient dose escalation with overdose control. Stat Med 17: : 1103,1998 -1120, Crossref, Medline, Google Scholar |
| 27. | Rogatko A, Babb JS, Tighiouart M, et al: New paradigm in dose-finding trials: Patient-specific dosing and beyond phase I. Clin Cancer Res 11: : 5342,2005 -5346, Crossref, Medline, Google Scholar |
| 28. | Dent SF, Eisenhauer EA: Phase I trial design: Are new methodologies being put into practice? Ann Oncol 7: : 561,1996 -566, Crossref, Medline, Google Scholar |
| 29. | Roessner D, Bozeman B, Feller I, et al: The role of NSF's Support of Engineering in Enabling Technological Innovation, First Year Final Report, Washington, DC, SRI International, The Science and Technology Policy Program, 1997 Google Scholar |
| 30. | Illinois Institute of Technology Research: Technology in Retrospect and Critical Events in Science. Washington, DC, National Science Foundation, Illinois Institute of Technology Research, 1968 Google Scholar |
| 31. | Tornatzky L, Fleischer M: The Process of Technological Innovation . Lexington, MA, Lexington Books, 1990 Google Scholar |