Skip to main content


Front. Oncol., 10 October 2019
Sec. Radiation Oncology
Volume 9 - 2019 |

Clinical Outcome Assessments Toolbox for Radiopharmaceuticals

Charles A. Kunos1*, Jacek Capala2, Adam P. Dicker3, Benjamin Movsas4, Susan Percy Ivy1 and Lori M. Minasian5
  • 1Cancer Therapy Evaluation Program, National Cancer Institute, Bethesda, MD, United States
  • 2Radiation Research Program, National Cancer Institute, Bethesda, MD, United States
  • 3Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA, United States
  • 4Henry Ford Hospital, Detroit, MI, United States
  • 5Division of Cancer Prevention, National Cancer Institute, Bethesda, MD, United States

For nearly 40 years, the U.S. National Cancer Institute (NCI) has funded health-related quality-of-life (HRQOL) and symptom management in oncology clinical trials as a method for including a cancer patient's experience during and after treatment. The NCI's planned scope for HRQOL, symptom and patient-reported outcomes management research is explained as it pertains to radiopharmaceutical clinical development. An effort already underway to support protocol authoring via an NCI Cancer Therapy Evaluation Program (CTEP) Centralized Protocol Writing Service (CPWS) is described as this service aids incorporation of HRQOL, symptom and patient-reported outcomes management research into sponsored protocols.


For nearly four decades, the National Cancer Institute (NCI) sponsored clinical trials have provided resources for research in health-related quality of life (HRQOL) and in symptom management for cancer patients (1). These resources have included infrastructure for cancer patient clinical trials that have symptoms as a primary end point, funding for investigator-initiated correlative studies involving HRQOL end points in late phase clinical trials, and grants studying the key issues and challenges facing investigators for implementing HRQOL and symptom management into its early phase clinical trials (1, 2).

Late phase clinical trials seek to improve cancer patient survival and more consideration has been given in these trials to the way in which cancer patients live during and after their treatments. A desire to meet HRQOL needs of cancer patients has incentivized NCI sponsored clinical trials to consider piloting the collection of HRQOL and patient-reported outcomes (PROs) by wearable digital technology like mobile phone applications or wristband sensors in parallel with its early phase clinical trials of radiopharmaceuticals. NCI sponsored clinical trials offer this strategic vision because radiopharmaceuticals have drug-like pharmacology in that these radioactive drugs have quantifiable pharmacokinetics, body weight-driven prescriptions, and predictable organ toxicities. Radiopharmaceuticals fit well into the programmatic mission of patient safety and symptom management for NCI sponsored clinical trials. Thus, integrating pilot HRQOL tools into early phase safety trials that are eventually intended to be used in late phase efficacy trials makes sense (3). Wearable digital technology in the form of mobile phone applications or wristband sensors captures in near-time the HRQOL and PRO data linked to acute toxicity, prompt and iterative symptom management, as well as reasons for treatment-related drug holiday or drug discontinuation (4).

The challenges and opportunities for integrating PRO and biometric endpoints into the roll-out of NCI sponsored radiopharmaceutical trials are discussed as the primary emphasis of this article. Opportunistic examples related to the Cancer Therapy Evaluation Program (CTEP) Centralized Protocol Writing Service (CPWS) and its incorporation of HRQOL, symptom and patient-reported outcomes management research into early-phase patient safety trials of radium-223 (Xofigo) or lutetium-177 dotatate (Lutathera) provide context for the discussion.

Challenges and Opportunities

From the time of cancer diagnosis to the end of life, cancer patients encounter a variety of functional and physical challenges (1). Undesirable outcomes from cancer or its treatment may range in scope from transient and reversible (for example, nausea or low white blood cell count), to cumulative (fatigue or abdominal pain), to subacute (3-month post-therapy cough from pneumonitis), or to late persistent and unremitting (dry mouth or vaginal dryness) (5). Pain, fatigue, and nausea are the most commonly encountered symptoms that occur along the trajectory of modern radiopharmaceutical treatment experience (6, 7). Cancer patients given radiopharmaceuticals may also have decreased appetite, vomiting, bruising easily, diarrhea, aching joints or muscles, or headache at various stages of their illness (6, 7). If not managed prospectively, a radiopharmaceutical-treated patient's physical, mental, or emotional well-being might be disrupted, thus impacting routine activities of daily living (Figure 1). Despite the long existence of these concepts (8), only now are “wearable” opportunities for symptom data collection becoming a reality (9). Wearable digital technology has evolved biometrics, or a capacity to observe, detect, and quantify, or in appropriate instances to intervene in, health parameters of the human body. Digital devices like wristband sensors now compute fitness and hydration level or out-of-bed activity and duration (Figure 1).


Figure 1. Development of digital patient-reported outcome measures for radiopharmaceuticals. Wearable digital technology is now commonplace among cancer patients. Patient-reported outcomes (PRO) or common terminology criteria for adverse events (CTCAE) items can be digitized for capture in the clinical development of radiopharmaceuticals. (A) Depicts an example of a bone-homing radiopharmaceutical intending to irradiate overlying cancer cells in the nearby bone marrow. It also shows that the radiopharmaceutical can irradiate osteocytes in a bone's mineralized matrix or bone marrow progenitor cells. Alpha particle-, beta particle-, or conversion electron-emitting radiopharmaceuticals would all have differential effects on normal cells depending on the range of irradiation. (B) Indicates what symptoms or clinical signs might manifest after multiple cycles of radiopharmaceutical administration. For example, PRO events like fatigue or CTCAE events like anemia that lessen overall performance status could become apparent. (C) Depicts some available commercial devices that quantifiably track activity or hydration status that might reflect some PRO or CTCAE items, or, digitally capture patient experiences in near real-time through applications or chatbots. Further research with wearable digital technology is needed.

For many patient-reported symptoms, meaningful interventions have not been well-studied due to a scarcity of data on the incidence, prevalence, trajectory, and severity of symptoms (1). There is an imperfect knowledge of the physiologic mechanisms underlying symptoms altered by cancer treatments. NCI sponsored clinical trials offer a mechanism for scientifically and intellectually interesting radiopharmaceutical studies that incorporate HRQOL and PRO end points because they provide an opportunity safely and efficiently to study toxicity from the viewpoint of the patient in a near-time digital format. Currently, a number of projects to address HRQOL and PRO research gaps using digital technology are considered in NCI sponsored clinical trials. Indeed, digital technology might improve near real-time collection of HRQOL and PRO end points (10), but might also impact patient outcomes (11). Digital devices like a mobile phone application could capture near-time toxicity on patient-reported pain, fatigue, and nausea.

Perspectives on Radiopharmaceutical Patient Reported Outcomes

From the outset, NCI sponsored clinical trials use an existing five-point scale Common Terminology Criteria for Adverse Events (CTCAE, version 5) toolbox for safety data and adverse event evaluation on radiopharmaceutical trials. While this method has limitations (Table 1), this approach builds upon prior notions that radiopharmaceutical-attributed toxicity falls into discrete toxicity categories that require medical instruments, technical training, or observable or subjective components (5, 12). For now, NCI investigators consider adverse events detected by instruments or those providers with technical training to follow CTCAE terminology and grading of severity. Adverse events that are subjective in nature with observable aspects (like radiation-induced diarrhea) or without observable qualities (like radiation-induced nausea) are amenable to patient reporting. Take for instance a trial participant's pretreatment grade 1 severity of frequent loose stools. On a trial evaluating the radiopharmaceutical radium-223 [a calcium mimetic eliminated via the relatively radiosensitive large intestine (13)], a participant's post-treatment severity of frequent loose stools might rise to grade 2, require antidiarrheal medication, and interfere with grocery shopping. CTCAE reports would capture the objective severity of loose stools requiring a physician-directed intervention in this case, but not necessarily the specific disruption of an instrumental activity of daily living. A PRO-CTCAE (v1.0) toolbox (14) incorporated into a radiopharmaceutical trial might improve the evaluation of this adverse event and provide the patient experience (Table 2). In this case scenario, capturing the patient's perspective on diarrheal frequency offers better qualified information on how an individual participant lives during and after their radiopharmaceutical treatment. For this reason, NCI investigators plan to list select toxicities like diarrhea as an adverse event of special interest when studying radium-223. As iterated elsewhere, an adverse event of special interest is a toxicity for which an expedited adverse event report must be filed to the NCI in its sponsored trials (5). PRO-CTCAE data have not been collected on radiopharmaceutical trials before, in part, because collection of such data is not common in early phase trials. Biometric data for trial endpoints (e.g., fasting glucose or specific changes in systolic and diastolic blood pressure parameters) are integrated in some NCI sponsored clinical trials.


Table 1. Complementary use of CTCAE and patient-reported outcome item formats for radiopharmaceuticals*.


Table 2. Radiopharmaceutical patient-reported outcomes version of the CTCAE item formats*.

Collection of biometric data or patient-reported outcomes in radiopharmaceutical trials is recommended. Investigators should consider employing the HRQOL instruments that measure, as optimally as possible, the relevant toxicity domains particularly relevant to the agent's mechanism of action (e.g., such as a radiopharmaceutical acting as a calcium mimetic and causing diarrhea), residence time (i.e., how long does a radiopharmaceutical “stick” to a target), and elimination from the body (like bowel or renal excretion inducing radiation-related enteritis or cystitis). A trial can incorporate the PRO instrument to provide information on specific symptoms or functional status, and any impact of the cancer and its treatment on HRQOL. Studies indicate that well-designed and well-conducted HRQOL research might guide future clinical trial design and morbidity end points by identifying certain patient conditions that variably confound HRQOL (1416). For the best return on research investment, HRQOL research should detect HRQOL items both important to patients and likely to be impacted by the radiopharmaceutical intervention or the underlying cancer (1). As more trials find effective treatments, both patients and their physicians will want data on HRQOL and the influence radiopharmaceuticals will have on their physical health and functional performance.

Because of the ongoing discussions to incorporate HRQOL and symptom management in its randomized trials, NCI stakeholders have adapted CONSORT (consolidated standards of reporting trials) guidelines (17) for the reporting of radiopharmaceutical clinical trials that might incorporate such end points (Table 3). To date, there are no formal examples in which radiopharmaceutical trials have included HRQOL instruments. NCI stakeholders share their thoughts on this topic here as this sort of data in its trials should provide, to future patients and to their physicians, information regarding an expected course of radiopharmaceutical therapy alone or in combination. Such data should also define potential for recovery from radiopharmaceutical-related toxicity.


Table 3. Reporting radiopharmaceutical trials with patient-reported outcomes.

Digital therapeutics provide another opportunity for advancements (18). These interventions are often pushed onward to the health consumer by high-quality software programs that integrate protocol-defined management steps to prevent, manage, or treat a medical disorder or a disease like cancer (18). Independently or together with medications, devices, or other therapies, digital therapeutics raise the “quality” level of patient care for enhanced health outcomes (18). At present, digital therapeutic devices are expected to incorporate best health industry practices relating to design, clinical testing, usability, and personal data security (18). Regulatory bodies now recognize digital therapeutics as a means to support drug product claims for risk, efficacy, and intended clinical indication (18). Digital therapeutics allow patients, healthcare providers, and payers to have smart and handy tools to address health conditions through high-quality, safe, and effective data-driven interventions (18). One mechanism that the NCI might use to write-in digital therapeutics in its trials is a centralized protocol writing service.

Perspectives on a CTEP Centralized Protocol Writing Service

NCI CTEP launched a Centralized Protocol Writing Service (CPWS) to aid its Experimental Therapeutics Clinical Trials Network (ETCTN) investigators for streamlined development of clinical trial protocols (Figure 2). The CPWS offers this service for the initial clinical trial document development to support rapid protocol activation; it does not provide service for post-activation protocol amendments. NCI CTEP considers the principal investigator as the accountable leader of a clinical team, meaning they are the individual who interacts with the CPWS and who conducts the clinical investigation. NCI CTEP presumes of the principal investigator the role for protocol document oversight, the responsibility for delegation of written tasks, and the provision of responses to feedback from NCI CTEP, CPWS, or other regulatory agencies. After a CPWS kick-off teleconference, there are iterative and interactive feedback loops that are intended to incorporate scientific, clinical, procedural, logistical, or regulatory items in a clinical trial protocol document (Figure 2). Once reviewed and approved by the principal investigator and the CPWS team, NCI CTEP provides final review and obtains any need additional reviews prior to actual protocol activation. As of March 2019, two radiopharmaceutical clinical trial protocols for radium-223 (Xofigo) were written by ETCTN principal investigators and the CPWS. Protocol authoring by the CPWS took an average 33 days, compared to a 60-day target. The CPWS will be engaged in writing lutetium-177 dotatate (Lutathera) radiopharmaceutical clinical trial protocols in the near-term future.


Figure 2. NCI Cancer Therapy Evaluation Program (CTEP) centralized protocol writing service. Charted is the workflow for the U.S. National Cancer Institute's CTEP Centralized Protocol Writing Service (CPWS) Program. From left to right, the chart is organized by the main protocol authoring entity involved in document writing inclusive of the Lead Protocol Organization, CTEP contract support (or CPWS), CTEP branches, or additional reviewers or other Federal agencies. Steps 1 and 2 initially activate CPWS protocol authoring. Steps 3 through 5 represent iterative and interactive feedback loops between CPWS and the principal investigator charged with protocol authoring. Step 6 represents a joint principal investigator and CPWS approval of the draft protocol. Step 7 and Step 8 involve scientific and logistical CTEP and non-CTEP reviews.


NCI sponsored clinical trials have supported the growth and execution of HRQOL and symptom management studies into clinical trials through a variety of pilot opportunities as part of protocol development. NCI investigators and stakeholders appreciate that early phase clinical trials evaluate the safety, and perhaps efficacy, of cancer treatment interventions among a diverse spectrum of cancer disease stages. In some instances, like the clinical development of radiopharmaceuticals, it makes sense to incorporate HRQOL and/or PRO tools in the early evaluation of agent safety when there is an anticipated impact collectively on patients, their caregivers, and their family members. This type of research can provide valuable data to patients, investigators, and regulators in early phases of clinical development before launching late phase clinical trials. The new NCI CTEP CPWS provides early phase trial investigators a means for iterative and interactive protocol writing, which may include HRQOL or PRO assessments in NCI sponsored clinical trials.

Ethics Statement

The research presented in this article involved the collection or study of existing data, documents, and records that were publicly available. The research is regarded exempt from Institutional Review Board oversight.

Author Contributions

CK, JC, AD, BM, LM, and SI contributed to the collection and review of any perspective data, analysis, and authentication, and the writing and approval of this manuscript. The views expressed are those of the authors and not those of the U.S. Federal government. Links or discussion of specific radiopharmaceutical drug products do not constitute endorsement.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


CK, JC, AD, BM, LM, and SI would like to acknowledge the Cancer Therapy Evaluation Program and Radiation Research Program of the Division of Cancer Treatment and Diagnosis and the Division of Cancer Prevention, National Cancer Institute for supporting this work. AD is supported by the Transdisciplinary Integrated Population Science Program of the Sidney Kimmel Cancer Center and a Challenge Grant from the Prostate Cancer Foundation.


1. Minasian LM, O'Mara AM, Reeve BB, Denicoff AM, Kelaghan J, Rowland JH, et al., Health-related quality of life and symptom management research sponsored by the National Cancer Institute. J Clin Oncol. (2007) 25:5128–32. doi: 10.1200/JCO.2007.12.6672

PubMed Abstract | CrossRef Full Text | Google Scholar

2. O'Mara AM, Denicoff AM. Health related quality of life in NCI-sponsored cancer treatment trials. Semin Oncol Nurs. (2010)26:68–78. doi: 10.1016/j.soncn.2009.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Chen A, Mitchell S, Minasian St L, Germain D. Incorporating patient-reported outcomes into early phase trials. In: Kummar S, Takimoto C, editors. Novel Designs of Early Phase Trials for Cancer Therapeutics. 1st ed. Cambride, MA: Academic Press (2018). p. 193–205.

Google Scholar

4. Dicker AP, Jim HSL. Intersection of digital health and oncology. JCO Clin Cancer Inform. (2018) 2:1–4. doi: 10.1200/CCI.18.00070

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kunos C, Capala J, Finnigan S, Smith G, Ivy S. Radiopharmaceuticals for relapsed or refractory ovarian cancer. Front Oncol. (2019) 9:180. doi: 10.3389/fonc.2019.00180

CrossRef Full Text | Google Scholar

6. Parker C, Nilsson S, Heinrich D, Helle SI, O'Sullivan JM, Fossa SD, et al., Alpha emitter radium-223 and survival in metastatic prostate cancer. N Engl J Med. (2013) 369:213–23. doi: 10.1056/NEJMoa1213755

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Strosberg J, El-Haddad G, Wolin E, Hendifar A, Yao J, Chasen B, et al., Phase 3 trial of 177Lu-dotatate for midgut neuroendocrine tumors. N Engl J Med. (2017) 376:125–35. doi: 10.1056/NEJMoa1607427

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Food and Drug Administration Center for Drug Evaluation and Research (CDER). Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Available online at: (accessed March 26, 2019).

Google Scholar

9. Kim J, Campbell AS, de Avila BE, Wang J. Wearable biosensors for healthcare monitoring. Nat Biotechnol. (2019) 37:389–406. doi: 10.1038/s41587-019-0045-y

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Movsas B, Hunt D, Watkins-Bruner D, Lee WR, Tharpe H, Goldstein D, et al. Can electronic web-based technology improve quality of life data collection? Analysis of Radiation Therapy Oncology Group 0828. Pract Radiat Oncol. (2014) 4:187–91. doi: 10.1016/j.prro.2013.07.014

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. (2016) 34:557–65. doi: 10.1200/JCO.2015.63.0830

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bruner DW, Movsas B, Basch E. Capturing the patient perspective: patient-reported outcomes as clinical trial endpoints. Am Soc Clin Oncol Educ Book. (2012) 139–44. doi: 10.14694/EdBook_AM.2012.32.139

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Carrasquillo JA, O'Donoghue JA, Pandit-Taskar N, Humm JL, Rathkopf DE, Slovin SF, et al. Phase I pharmacokinetic and biodistribution study with escalating doses of 223Ra-dichloride in men with castration-resistant metastatic prostate cancer. Eur J Nucl Med Mol Imaging. (2013) 40:1384–93. doi: 10.1007/s00259-013-2427-6

CrossRef Full Text | Google Scholar

14. Dueck AC, Mendoza TR, Mitchell SA, Reeve BB, Castro KM, Rogak LJ, et al. Validity and reliability of the US National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). JAMA Oncol. (2015) 1:1051–9. doi: 10.1001/jamaoncol.2015.2639

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC, et al. Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. (2014) 106:dju244. doi: 10.1093/jnci/dju244

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Thanarajasingam G, Minasian LM, Baron F, Cavalli F, De Claro RA, Dueck AC, et al. Beyond maximum grade: modernising the assessment and reporting of adverse events in haematological malignancies. Lancet Haematol. (2018) 5:e563–98. doi: 10.1016/S2352-3026(18)30051-6

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. (2013) 309:814–22. doi: 10.1001/jama.2013.879

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Digital Therapeutics Alliance. Combining technology and evidence-based medicine to transform personalized patient care. In: Digital Therapeutics Alliance, editors. Digital Therapeutics. 1st ed. (2018). p. 1–16. Available online at:

Google Scholar

Keywords: radiopharmaceutical, cancer, patient reported outcome (PRO), digital device usage, clinical outcome assessment, radiotherapy, radiotherapy adverse effects

Citation: Kunos CA, Capala J, Dicker AP, Movsas B, Ivy SP and Minasian LM (2019) Clinical Outcome Assessments Toolbox for Radiopharmaceuticals. Front. Oncol. 9:1028. doi: 10.3389/fonc.2019.01028

Received: 29 June 2019; Accepted: 23 September 2019;
Published: 10 October 2019.

Edited by:

Thomas FitzGerald, University of Massachusetts Boston, United States

Reviewed by:

Ying Xiao, University of Pennsylvania, United States
Alan Charles Hartford, Dartmouth–Hitchcock Medical Center, United States

Copyright © 2019 Kunos, Capala, Dicker, Movsas, Ivy and Minasian. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Charles A. Kunos,