Electronic Patient Reporting of Adverse Events and Quality of Life: A Prospective Feasibility Study in General Oncology

PURPOSE: Adverse event (AE) reporting is essential in clinical trials. Clinician interpretation can result in under-reporting; therefore, the value of patient self-reporting has been recognized. The National Cancer Institute has developed a Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) for direct patient AE reporting. A nonrandomized prospective cohort feasibility study aimed to explore the compliance and acceptability of an electronic (Internet or telephone) system for collecting patient self-reported AEs and quality of life (QOL). METHODS: Oncology patients undergoing treatment (chemotherapy, targeted agents, hormone therapy, radiotherapy, and/or surgery) at 2 hospitals were sent automated weekly reminders to complete PRO-CTCAE once a week and QOL (for a maximum of 12 weeks). Patients had to speak/understand English and have access to the Internet or a touch-tone telephone. Primary outcome was compliance (proportion of expected questionnaires), and recruitment rate, attrition, and patient/staff feedback were also explored. RESULTS: Of 520 patients, 249 consented (47.9%)—mean age was 62 years, 51% were male, and 70% were married—and 230 remained on the study at week 12. PRO-CTCAE was completed at 2,301 (74.9%) of 3,074 timepoints and QOL at 749 (79.1%) of 947 timepoints. Individual weekly/once every 4 weeks compliance reduced over time but was more than 60% throughout. Of 230 patients, 106 (46.1%) completed 13 or more PRO-CTCAE, and 136 (59.1%) of 230 patients completed 4 QOL questionnaires. Most were completed on the Internet (82.3%; mean age, 60.8 years), which was quicker, but older patients preferred the telephone option (mean age, 70.0 years). Positive feedback was received from patients and staff. CONCLUSION: Self-reporting of AEs and QOL using an electronic home-based system is feasible and acceptable. Implementation of this approach in cancer clinical trials may improve the precision and accuracy of AE reporting.

QUESTION ASKED: Is it feasible and acceptable, and what level of compliance is achieved when asking oncology patients to regularly self-report both adverse events (Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events [PRO-CTCAE]) and health-related quality of life (HRQOL) at home on an electronic system (Internet/ telephone option)?
SUMMARY ANSWER: High compliance was observed for weekly patient self-reported PRO-CTCAE plus HRQOL completions every 4 weeks. Patient feedback was positive, highlighting a willingness to complete questionnaires, with the system being easy to use and problem free.
WHAT WE DID: Two hundred forty-nine oncology patients receiving a variety of treatments from 2 United Kingdom hospitals were sent weekly auto-reminders to complete PRO-CTCAE and every 4 weeks, the HRQOL was also available. These self-reported questionnaires were completed online or via a telephone-based interactive voice response system (IVRS) for a 12-week period.
WHAT WE FOUND: Overall compliance was more than 60% at each individual timepoint and the cumulative compliance was 75% to 80% for each questionnaire. Some differences were observed across the treatment groups (chemotherapy higher compliance), patient age (older patients completed the PRO-CTCAE more often), and choice of system (greater HRQOL completion with IVRS). The Internet option was more popular (82.3%) and quicker, but some older patients (16.9%) preferred the IVRS. Most participants remained on the study throughout the 12-week period, with only 13 active withdrawals, 6 deaths, and 18 passive withdrawals (not completing any electronic questionnaires postbaseline).
BIAS, CONFOUNDING FACTORS: This feasibility study was carried out in a general oncology setting in a hypothetical trial scenario, rather than among active clinical trial participants. This influenced how the study was presented and some of the practicalities (baseline completion was defined flexibly rather than stringently as per clinical trial protocols), and potentially affected uptake and compliance. The recruitment rate was relatively low; therefore, there may be biases as a result of self-selection.

REAL-LIFE IMPLICATIONS:
The value of routine PRO collection is increasingly recognized in cancer clinical trials, and the need for innovative methods to achieve this remotely have been heightened further during and after the coronavirus disease 2019 pandemic. Our findings highlight the willingness of patients to selfreport from home and the acceptability of the electronic system. Additional exploration of this system within active clinical trials is warranted to explore whether implementation can improve the precision and accuracy of adverse event reporting, leading to improved data quality and patient safety.

INTRODUCTION
As more cancer treatments with innovative modes of action are evaluated via clinical trials, the need to monitor and record adverse events (AEs) 1 increases to ensure safety and guide prescribing and patient information. 2 The Common Terminology Criteria for Adverse Events (CTCAE) 3 is the standard method of reporting AEs, allowing clinicians to uniformly classify AEs and their severity, but its focus is on safety rather than exploring patient experiences. 4 Clinician interpretation can result in under-reporting of lowergrade morbidity and severity downgrading 2,5,6 ; therefore, patient self-reporting is a complementary alternative. 7,8 The CTCAE system is suitable for acute AEs when patients receive regular oncologist review; however, many new treatments are outpatient-based oral therapies that require less frequent reviews. Although these treatments do not result in the acute severe AEs observed with chemotherapy, they often result in low-to moderate-grade prolonged symptoms that significantly impact patients' lives and treatment adherence. 1,9 CTCAE is less appropriate for monitoring AEs during these treatments. 7 Patient-reported outcomes (PROs) enable systematic, subjective measurement of symptoms and experiences. Health-Related Quality of Life (HRQOL) Author affiliations and support information (if applicable) appear at the end of this article.
questionnaires are well-established PRO measures used in phase II and III randomized clinical trials. 10 Implementing PROs to routinely monitor individual patients' AEs is relatively uncommon, despite accumulating evidence demonstrating improvement in communication, patient wellbeing, symptom control, and survival when clinicians receive these data. [11][12][13][14] The National Cancer Institute developed the Patient-Reported Outcome version of the CTCAE item bank (PRO-CTCAE) to obtain direct patient AEs in trials. 15 Technology can help PRO-CTCAE to become a standardized trial approach, 15,16 and Web-based PRO efficiency and patient acceptability has been demonstrated. 17,18 Of the United Kingdom population, 90% are regular Internet users; however, 24% of those age 65 years and older reported no Internet use in the past 3 months. 19 Therefore, a telephone interactive voice response system (IVRS) widens eligibility.
Oncology clinicians, trialists, the pharmaceutical industry, and regulatory authorities recognize the importance of collecting PRO data in drug development and evaluation. 1,10,20,21 Concerns still exist about potential patient burden and data collection frequency. 22 There is a pressing need to develop robust systems to collect PRO-CTCAE alongside clinicianreported CTCAE, as well as explore the feasibility of collecting frequent PRO-CTCAE in addition to HRQOL at prespecified trial timepoints where QOL is an important primary or secondary outcome. This feasibility study evaluated a self-reported electronic system-Internet/IVRS-to collect PRO-CTCAE and HRQOL in a mixed group of oncology patients and to combine the data with a standard clinical trial database. Primary outcome was compliance and secondary outcomes included recruitment rates, Internet/IVRS uptake, attrition, patient and staff feedback, and system testing.

Patient Population and Study Design
This proof-of-principle, nonrandomized, prospective, cohort feasibility study was conducted in 2 United Kingdom hospitals between August 19, 2014, and October 7, 2015, with 12-week follow-up. The study design modeled a hypothetical clinical trial scenario in which PRO-CTCAE data are collected for safety (real time, as frequently as needed) and HRQOL at predefined timepoints to assess treatment outcomes.
Our aim when defining the study population was to capture a range of patients across the main cancer sites and treatments. We anticipated that this would increase the generalizability of the findings. Consecutive eligible patients were approached. Therefore, the sample target was 210 participants with early or metastatic cancer receiving either chemotherapy, targeted agents, hormone therapy, radiotherapy, or surgery (42 patients per group, allowing for 30% attrition), 23 plus 42 patients with Eastern Cooperative Oncology Group (ECOG) performance status score of 2 to 3 receiving any treatment. Eligibility criteria included speaking/ understanding English and access to a touch-tone telephone and/or Internet. Patients on clinical trials with regular HRQOL assessments were excluded. Written informed consent was obtained from study participants. Ethical approval was granted from the National Health Service Leeds-East Research Ethics Committee (14.YH.0181).

Study Outcomes
• Compliance was defined as the proportion of completed questionnaires out of expected completions. This was calculated cumulatively for each questionnaire-PRO-CTCAE and HRQOL-and at each expected timepoint weekly (7-day window)/every 4 weeks (28-day window) across gender, system, study group, and age. Multiple completions per week were discounted from compliance calculations. The number of completions per participant was also calculated, including multiple completions. • Recruitment rate was defined as the proportion of patients who consented to the study relative to eligible patients approached. Demographics-age, gender, diagnosis, and treatment-were compared between consenting patients and declining patients. • Internet/IVRS uptake was calculated in 2 ways: the proportion who preferred each system at baseline and the proportion who used each system most often during the study. • Active withdrawal was the predominant attrition outcome, defined as the proportion of participants who actively requested withdrawal from the study out of participants who consented. Any withdrawal reasons were recorded where provided. In addition, passive withdrawal was defined as the proportion of participants who did not complete any electronic questionnaires postbaseline but did not request withdrawal. Passive withdrawals are included in the compliance calculations, but active withdrawals were excluded after the withdrawal date. • Patient and staff feedback was captured on a paper end-of-study (EOS) questionnaire that explored experiences of the system(s) and staff views. • System testing was assessed through the documentation of system issues and phone calls and comments from patients and staff. Call and online session time logs were also recorded.

Electronic System and Questionnaires
Participant registration via the clinical trials research unit (CTRU) registration system provided the trial number and username and password for accessing the electronic system to complete the 2 self-reported questionnaires.
First, 21 PRO-CTCAE items 15 covering 11 common symptoms (pain and/or discomfort, nausea, vomiting, diarrhea, constipation, headache, mouth/throat sores, fatigue, skin rash, flu-like symptoms, and muscle ache) were available at any time. Second, the European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire C30 (EORTC-QLQ-C30) 30-item cancer-specific measure exploring HRQOL 24 was available once every 4 weeks.
Holch et al 25 provide a detailed overview of the Internet (QTool) system. Internet system users were presented with 1 (PRO-CTCAE) or 2 questionnaires (if EORTC-QLQ-C30 was due). The IVRS that was developed specifically for this study by CTRU technicians was a freephone number. To ease the log-in process on the IVRS, all usernames were a 5-digit numerical code, and passwords were a 4-digit numerical code. The same format was used on the Internet system to enable participants to switch between systems if required. Once logged in, IVRS users heard a welcome message, then heard the EORTC-QLQ-C30 questionnaire (if due, every 4 weeks) and/or the PRO-CTCAE. Each question was read out, followed by the response options (eg, "Question number 13. In the last 7 days, what was the SEVERITY of your fatigue, tiredness, or lack of energy at its worst? Press 1 then hash for none, 2 then hash for mild, 3 then hash for moderate.…") Patients could interrupt the responses with their selected response and could take a break-hang up and re-ringafter the first questionnaire if needed.
A specific application pulled the data from a separate server that held the Internet/QTool data, which was then stored in a specific PRO database on CTRU servers. A program was created to generate/send automated reports to the local research team detailing completions. The PRO database was combined with the standard trial database (Elsevier's MACRO Data Capture System) that contained all the participant clinical record forms for statistical analysis.

Procedure
Eligible patients were initially approached by clinical/research staff with the study information sheet. Recruitment was undertaken sequentially across the treatment subgroups and stopped once each prespecified target number was reached.
Demographic details and computer use (frequency and perceived confidence) were self-reported at baseline. Clinical data captured from medical records included diagnosis, treatment, ECOG performance status, and treatment changes (discontinuation/new treatments). Baseline electronic questionnaires were completed in clinic, or a brief demonstration was provided and the full questionnaires completed at home on the preferred system. All participants received step-by-step instructions covering both systems. Participants were reminded (by automated e-mail, text, or both) weekly-every 7 days from the registration date-for 12 weeks to complete the electronic questionnaires. EOS paper questionnaires were sent/given once each participant had completed the 12 weeks.
Any deaths or withdrawal requests were actioned promptly (reminders stopped). All participants were reminded that the system was not intended to replace usual care or methods of reporting any symptoms or concerns to their clinical team. However, researchers endeavored to attach a paper report summarizing any recent AEs to participants' medical notes at any scheduled outpatient visits.

Analysis
Data analysis was performed using STATA between August 2016 and February 2017. Descriptive analysis of the study outcomes was conducted. Missing data were not imputed. System issues were coded thematically into generic and patient issues.

Study Population
Of 888 patients who were assessed for eligibility during the study period, 368 were ineligible, and of 520 approached, 249 consented to the study (recruitment rate 47.9%; Fig 1).
Overall combined compliance for both systems was 74.9% for PRO-CTCAE (completed at 2,301 of 3,074 timepoints) and 79.1% for EORTC (completed at 749 of 947 timepoints).     The system uptake reported is the system used most often during the study period. The most commonly used system was missing from 9 participants who did not complete any questionnaires and 1 patient who had an even split between IVRS and Internet usage. b Advanced level qualification in the United Kingdom undertaken in school or college. benefit for auditing symptoms and directing consultations and in clinical trials was emphasized.

System Testing
During the 17-month study, 3,170 questionnaires were completed (2,413 PRO-CTCAE questionnaires and 757 EORTC questionnaires) and proportionately few issues were recorded-38 patient-related and 69 system-related issues. Notable system issues included reminder problems (n 5 20), information technology/server issues (n 5 7) affecting access or auto-emails, completion report issues (n 5 12), and some technical difficulties (Internet, n 5 11; IVRS, n 5 8). Overall downtime was not systematically recorded, but isolated incidences were experienced. Patient issues included initial access problems (n 5 19), completion queries (n 5 8), and lost/forgotten username and/ or password (n 5 6). The PRO database was combined successfully with the standard clinical trial database with only a single issue during the data retrieval/combining process.

DISCUSSION
To our knowledge, this is the first study illustrating the feasibility and acceptability of an electronic system to collect both self-reported PRO-CTAE and HRQOL data from a diverse sample of United Kingdom patients with cancer. The results demonstrate patient willingness, low attrition (5.2%; 13 of 249 active withdrawals), and high compliance (approximately 60% to 70% week 12; cumulative, approximately 75% to 80%). Most participants preferred the Internet, but approximately 17% of those who were older used IVRS, indicating an that alternative mode was desirable. Combining PRO data with the clinical trial database worked effectively with few system issues recorded. This approach is therefore recommended and a comparative evaluation is now required.
The 47.9% recruitment rate is lower than the 70% uptake rate routinely observed in PRO studies, 11 but is encouraging compared with the 31% to 64% uptake rate in mailed cancer surveys. 26 Target participants included various diagnoses, which made robust interpretation of any differences between cancer groups challenging as a result of small frequencies. Disease site was not associated with the rate of consent for the study; however, uptake was specifically low in some groups (eg, prostate and hormone therapy). Therefore, the study findings cannot be generalized to all cancer populations and may not reflect Compliance overall was high (approximately 60% to 70% at week 12; cumulative, approximately, 75% to 80%) and higher still in chemotherapy patients regularly going to the hospital for treatment reviews. PRO-CTCAE items chosen were particularly applicable to chemotherapyrelated symptoms. Lower weekly compliance for surgical patients may be a result of the intense recovery period immediately postsurgery and less frequent review postsurgery. Some clinical trials that included paper-based QOL measures have observed higher compliance (75% at 6 months 28 ), but the current study had more intense frequency (weekly/every 4 weeks). In an active clinical trial, compliance may be higher because of patient selection, trial-related incentives, and item relevance. 29 Basch et al 27 reported 92% cumulative compliance for home PRO-CTCAE reporting in trial patients, but 14.7% of questionnaires were collected via a coordinator directly ringing patients.
Despite most participants choosing to use the Internet, some older patients preferred the IVRS. 27 An alternative real-time method is desirable if the purpose of collection is tracking symptoms/AEs, whereas if the aim is obtaining HRQOL outcomes, paper-based alternatives may suffice. Overall compliance did not differ across systems, but different patterns were observed, potentially influenced by the order of presentation: EORTC was presented first on the IVRS at the once every 4 weeks intervals (explaining higher EORTC IVRS compliance), whereas Internet users had to proactively choose both questionnaires. Furthermore, completions on the Internet were quicker, which may be a result of the systems themselves (ie, IVRS required the questions/answers to be read out at a comprehensible speed), but it may also reflect the younger, more educated Internet users. Patient feedback was positive, finding the systems acceptable and problem free. Most issues were related to initial queries or access problems and were quickly resolved. Reasons for noncompletion were a result of being unwell or forgetting, rather than negativity about the systems.
The systems were only available in English, which is a limitation that should be addressed in future work. Furthermore, the general oncology setting influenced how the study was presented (focusing on system testing rather than on the value of collecting PROs in trials) and potentially affected uptake and compliance. The definition of baseline differed to what would be done in a formal clinical trial setup: A lenient baseline PRO-CTCAE/HRQOL completion (up to 3 days postregistration) was used, rather than prerandomization, as in many clinical trials.
The study highlights implementation considerations for future actual clinical trial use, including the need for a serious AE (or suspected unexpected serious adverse reaction) alert system, clinicians directly receiving symptom summary reports or alerts, and a function for individual symptom reporting or adding other free-text symptoms. Some of the considerations also relate to concerns voiced regarding the ethics of PRO data in clinical trials and the timescale of data review. 30 Using an electronic PRO system with automated alerts helps with the timely management of these issues compared with paper-based methods, but creates new training needs for trial staff. 31 Furthermore, electronic data collection requires continuous assessment to ensure that information technology systems and infrastructure meet clinical trial regulation and Medicines and Healthcare products Regulatory Agency and European Medicines Agency requirements. 32 In conclusion, routine collection of PRO data in cancer clinical trials is increasingly important. This study illustrates the feasibility and patient acceptance of electronic PROs during treatment and incorporating PRO data into clinical trials. Additional exploration of this system is warranted where PROs are combined with other trial data-alongside traditional approaches-to improve data quality and patient safety.