Patients’ daily reporting of symptoms via mobile application reveals a significant difference between patients’ perceptions and doctors’ interpretations

Purpose: Electronic patient-reported outcomes (ePROs) are gaining importance. The aim of this study was to investigate the difference in the reporting of symptoms between patients via mobile application (m-app) and doctor assessments. Additionally, usability and satisfaction with the use of the m-app were assessed.

Methods: In this single-center prospective cohort study, we analyzed ePROs in 46 patients receiving (neo)adjuvant chemotherapy for early breast cancer. Patients recorded the occurrence and intensity of symptoms via the Android-based m-app daily. Three-monthly, patients completed validated quality of life questionnaires (EORTC C30 and BR23). For the 10 most common symptoms reported by patients, we compared the frequencies and grades between patients and doctors. Additionally, we compared daily symptom reports with questionnaire results. Finally, the usefulness of and level of satisfaction with the m-app by patients and doctors were evaluated.

Results: During the study, patients recorded almost twice as many different symptoms through the m-app as doctors did in the electronic health records (75 vs 49). Symptoms were described by patients as mild (67%), moderate (30%), or severe (3%). The frequency and intensity of symptoms reported by patients were significantly higher than those reported by doctors (p<0.001). Fatigue, insomnia and dry mouth were the three main symptoms reported in more than 75% of patients, but insomnia and dry mouth were also the symptoms most often underreported by doctors. After three months of chemotherapy, QoL assessments revealed worsening of physical, social, cognitive and role functioning, increased fatigue, systemic therapy side effects, and dyspnea but a reduction in arm and shoulder problems. Patients and doctors rated the m-app with an overall score of 4.5 out of 5 (IQR 1.0). Patients expressed high levels of satisfaction with and usability of the m-app. In contrast, doctors were somewhat reluctant to perceive ePROs as an additional burden with patient management.

Conclusion: This study revealed a significant discrepancy between patients’ daily symptom reports via the m-app and doctors’ assessments, indicating that healthcare professionals may not fully capture patients’ experiences. This underscores the importance of integrating PROs to accurately evaluate patients’ conditions. Our m-app serves as a promising tool for reporting and managing side effects during systemic treatment of breast cancer.

1 Introduction

Breast cancer is the most common cancer among women in Slovenia. In 2019, the age-standardized incidence rate was 108.4 per 100,000 women according to the European standard population (1). Over time, the net survival of breast cancer patients in Slovenia has improved significantly, from 77.5% (95% CI 76–79) during 1997–2001 to 87.6% (95% CI 86.3–88.9) during 2011–2015 (2). While survival rates have improved, maintaining quality of life (QoL) during cancer treatment is increasingly prioritized in clinical trials and real-world practice. Enhancing overall survival (OS) or QoL is a key endpoint for the regulatory approval of new therapies (3).

The evaluation of QoL covers different aspects of an individual’s daily life, including physical and emotional well-being, psychosocial functioning and treatment-related toxicity. Information about a patient’s health that comes directly from the patient without the intervention or interpretation of a healthcare professional are patient-reported outcomes (PROs) (4). PROs are assessed via patient-reported outcome measures (PROMs), usually with the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) (5) or validated questionnaires. The European Organization for Research and Treatment of Cancer (EORTC) has developed three primary types of questionnaires to evaluate patient well-being: core questionnaires, module questionnaires, and standalone questionnaires. Core questionnaires serve as a basis for assessing quality of life in all cancer patients, whereas module questionnaires aim to increase sensitivity and specificity in specific types of cancer (6). The most commonly utilized PROMs in breast cancer include the EORTC Quality of Life Core Questionnaire 30 (EORTC QLQ C-30) and EORTC Breast Module 23 (EORTC QLQ BR-23) (7, 8). The impact of using PROMs to improve QoL has been demonstrated in several clinical trials, which revealed that early detection of adverse events not only increased QoL but also reduced the hospitalization rate and even prolonged OS (9–14). However, the main limitation of most existing methods of data collection is the recording of data on predefined dates (e.g., at clinic visits, every 12 weeks), which leads to information gaps between assessments and the use of limited sets of preselected symptoms that cannot capture all that actually occurred during therapy. Advances in informatics and mobile technology have enabled electronic patient-reported outcomes (ePROs) in real time via mobile applications (m-apps), offering a potential solution to overcome these limitations. Because the use of PROs even prolongs OS, ePROs are a way to become digital therapeutics (15). The use of the m-app with the PRO-CTCAE as a tool for self-reporting significantly empowers patients to adequately report adverse events of cancer chemotherapy and increases their self-management of treatment-related side effects (16–18). Despite their technological feasibility, the widespread adoption of such m-apps remains limited, probably due to the challenges in managing severe symptoms and physicians’ reluctance to accept new technologies (19).

To investigate the feasibility and utility of ePRO collection with our custom-made m-app OnkoVed (20), we conducted a prospective cohort study in patients receiving chemotherapy. Our primary objective was to assess the frequency and severity of symptoms reported by patients on a daily basis. We also assessed QoL via validated QLQ questionnaires within the m-app. In addition, we investigated the concordance between patient-reported and doctor-reported symptoms. Finally, we evaluated the usability and satisfaction of the m-app by patients and doctors.

2 Patients and methods

2.1 Study design

A single-center prospective study was conducted at the Institute of Oncology Ljubljana to assess the frequency and severity of symptoms occurring daily during chemotherapy for solid cancer. Patients received access to m-app OnkoVed, which includes a symptom collection module and the EORTC QLQ C30 and QLQ BR23 electronic questionnaires. In the present study, we report the results for patients with early breast cancer. The duration of the study was limited to the duration of (neo)adjuvant treatment (4–6 months). Patient accrual took place from April 2021 to July 2022. The study protocol was approved by the institutional review board and the ethics committee (ERIDNPVO-0003/2021 and ERIDNPVO-0011/2023). All patients signed an informed consent form before inclusion in the study. The study was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice.

2.2 Patients

The inclusion criteria were as follows: age ≥ 18 years, early breast cancer (stage I-III), planned treatment with neoadjuvant or adjuvant chemotherapy (and anti-HER2 therapy in case of HER2 positivity), owner of an Android smartphone, and experience in smartphone use. The exclusion criteria were metastatic disease, owner of a non-Android smartphone, not proficient in smartphone usage or not understanding the Slovenian language.

2.3 Development of the mobile application OnkoVed

On the basis of our previous experience with m-app mPRO Mamma (21), which was designed only for patients with breast cancer, we made an upgrade to m-app OnkoVed to be suitable for all cancer patients receiving systemic therapy (20). This application contains a collection of all symptoms described in the PRO-CTCAE (5). If any symptoms occurred, the patient was required to record them and indicate their severity on a three-point scale as mild, moderate or severe. If there were no symptoms, no action was needed. In addition, electronic versions of the QLQ C30 and BR23 were added. At predetermined timepoints, patients sent data to the Institute of Oncology.

2.4 Study procedures

Upon enrollment in the study, patients received written instructions for using the m-app OnkoVed. They proceeded by downloading the application to their smartphones via the Google Play Store. As the application is publicly available and certain features are meant to be used by other interested parties, an activation of study-specific features (i.e. symptom sharing) was needed, which could be performed via the credentials provided in the instructions. The study nurse demonstrated to each participant how to use the application and send the reports. The m-app contains modules on patients’ demographic and cancer data, a symptom module (Figure 1A) with relevant tasks and useful tips according to the severity of a symptom (symptom tool-kit) (Figure 1B) and their cumulative frequency (Figure 1C), a module with the QLQ C30 and QLQ BR23 questionnaires, and a module with useful links (visiting planners, booklets, and cancer patients’ associations). After the account was activated, patients independently configured daily in-app notifications for symptom logging at their preferred times and could retrospectively enter symptom data for the preceding five days, with the primary impetus for daily app usage resting with the patient. Additionally, three-monthly notifications for answering QLQ C30 and QLQ BR23 were created. At the end of each chemotherapy cycle, a day before the next scheduled visit, the patient was required to send an e-PRO report. All collected data were transmitted in a secure encrypted format to a dedicated e-mail address created and maintained by the Institute of Oncology. With respect to patient engagement, the study nurses provided a phone reminder for daily symptom reporting only if a patient had not submitted their report the day prior to a scheduled check-up, which occurred every two to three weeks; the exact frequency of these calls was not recorded. Patients were not contacted if there was no interaction with the app.

Figure 1

Figure 1. Screenshot of the mobile application showing patient-recorded symptoms with their grades (A). Screenshots of the description of symptom fatigue and helpful tips for self-managing this symptom (B) and graphical presentation of symptoms (their grades and duration) (C).

At the scheduled appointment with the doctor, the study nurse provided the printed ePRO report received from the patient to the doctor. The application generated reports in three formats: free text, graphical presentation and an Excel table. In the free text, data concerning all symptoms were reported as follows: day/cycle/symptom type/grade (Supplementary Figure 1). In the graphical presentation, time course display of the symptoms with their grades were highlighted in different colors (Supplementary Figure 2). Finally, all symptoms were presented on an Excel table, which was suitable for statistical analysis (Supplementary Figure 3).

Doctors were informed that the ePRO app featured a list of 80 patient-adapted symptoms based on the PRO-CTCAE framework and that the system generated comprehensive reports in text, graphical, and spreadsheet formats. While the intention was for doctors to review these reports, discuss them with patients, and report them in the patient’s electronic health record (EHR), most continued with their standard consultation practices, prompting a retrospective analysis comparing patient-reported symptoms with doctor documentation.

One of the doctors performed retrospective grading of all doctors’ reporting of symptoms according to the Common Terminology Criteria for Adverse Events v5 (22) at the end of the study. Doctors’ reported symptoms (at check-ups) were compared with the daily reports of patients. We were aware of the possibility of assessment bias in the doctor’s assessment of symptom grade due to the retrospective nature of the assessment.

2.5 Study endpoints

The primary outcome was the frequency and severity of patient-reported symptoms during chemotherapy on the basis of daily m-app records and doctor reports in EHRs during outpatient visits. We specifically focused on the ten most frequently reported symptoms and compared the concordance of symptom frequency and grades reported by patients and doctors. The duration of symptoms was also assessed. The secondary outcome was the change in QoL as assessed by the EORTC QLQ-C30 and the QLQ-BR23. Finally, we assessed the usability and satisfaction of patients and doctors with the m-app.

2.6 Statistical analysis

Data on patient demographics and cancer treatment are presented as numbers and percentages. Symptom frequency and rates are presented as the means and standard deviations. Symptom duration and patient age are presented as medians and interquartile ranges. Comparisons between groups (patient reports versus doctor reports) were performed using the chi-square test. P<0.05 was considered statistically significant. For the EORTC QLQ C30 and the QLQ BR23, we followed the EORTC guidelines (23). In brief, for the functioning scales (range 0–100), higher values indicate better functioning, and for the symptom scales (range 0–100), higher values indicate worse symptoms. To compare the differences in functioning and symptoms at 3 months from baseline, we used paired Student’s t-test with 1000 bootstrap tests. Calculations and analyses were performed in Excel and SPSS v.22.

3 Results

3.1 Study patients

We approached 160 patients with early breast cancer. Sixty-four percent of patients were ineligible or did not consent. In the end, 58 patients signed an informed consent. It was later revealed that three patients did not meet the inclusion criteria (two were metastatic and one was not treated with chemotherapy). Nine of 55 (16%) eligible patients did not send any report and were also excluded from the study. Here, we present data on 46 female patients with early breast cancer who were treated with neoadjuvant or adjuvant systemic therapy. This represents 29% of the assessed population. Figure 2 shows the consort diagram of the patients approached and included in the study and their study procedures.

Figure 2

Figure 2. Study design and workflow diagram.

The median age of the patients was 53.4 years (range 35–69 years). The demographic and tumor characteristics of the patients are presented in Table 1. Briefly, at inclusion, 58.7% of the patients had tumors > 20 mm in size, 32.6% had positive lymph nodes, and 63% had Grade 3 tumors. On the basis of immunohistochemistry, 60.9% of the tumors were luminal type A/B, 28.3% were HER2+ and 10.8% were triple-negative. Adjuvant treatment was delivered in 60.9% of patients, and 39.1% started neoadjuvant treatment. The most commonly used chemotherapy regimen was sequential treatment with anthracyclines and taxanes (65.2% of all patients). More than half of the patients (51.2%) underwent breast-conserving surgery, and 48.8% underwent mastectomy. In the mastectomy group, 19 of 22 patients underwent immediate reconstruction with a tissue expander or abdominal flap. Only 19.6% of patients had axillary dissection, and all others had sentinel node biopsy.

Table 1

Table 1. Patients demographic and tumor characteristics.

3.2 Frequency and grades of symptoms

Over the entire study period, patients reported 75 different symptoms (minimum 4 and maximum 45). The median number of symptoms per patient was 25 (IQR 18–31.8). Among these, 16.2 (67%) had a maximum grade of 1, 7.3 (30%) had a maximum grade of 2, and 0.7 (3%) had a maximum grade of 3. Doctors observed 49 (minimum of 1 and maximum of 21) different symptoms; the median number of symptoms per patient was 7 (IQR 6–10). Among these symptoms, 6.3 (80%) were Grade 1, 1.4 (17%) were Grade 2, and 0.2 (3%) were Grade 3 (Figure 3A). There was a statistically significant difference in the reporting frequency of symptoms between patients’ and doctors’ reports (p ≤ 0.0001 for grades 1 and 2 and p=0.004 for grade 3), as shown in Figure 3B.

Figure 3

Figure 3. Number and grades of symptoms per patient according to maximal severity. (A) Comparison of Grade 1–3 symptoms reported by patients and doctors, presented as boxplots. Boxes represent quartiles 25 and 75 with the means (indicated by crosses) and medians (indicated by lines) inside the box. (B) Summarized mean number of symptoms by maximal grade assessed by patients and doctors. G1, Grade 1; G2, Grade; G3, Grade 2.

On the basis of the m-app reports, the ten most frequently reported symptoms were fatigue, insomnia, dry mouth, ulcers of the oral mucosa, muscle pain, changes in smell and taste, headache, hair loss, arthralgia and diarrhea. These symptoms were experienced by 65–87% of all patients. The doctor’s reports of the frequencies and grades of symptoms were the closest to the patients’ reports with oral ulcers and hair loss. With respect to the other 8 symptoms, the doctor’s estimate was significantly lower in frequency and grade than the patient’s estimate was. The symptoms most underreported by doctors were insomnia and dry mouth (Figure 4). Six Grade 3 symptoms reported by patients are presented in Figure 4; in addition, abdominal pain, obstipation, dysuria, fever, skin rash and nail changes were reported as G3. The doctors reported 4 symptoms of G3, as presented in Figure 4, as well as polyneuropathy and vertigo.

Figure 4

Figure 4. Comparison of the 10 symptoms most frequently reported by patients (daily via mobile application) and by doctors in electronic health records. Symptoms are presented as the percentage of patients experiencing these symptoms, with the corresponding severity expressed in grades 1–3.

3.3 Duration of symptoms

The mean time of recording symptoms in m-app was 115 days, ranging from 18–205 days. The cumulative duration of each of the 10 most often reported symptoms is presented in Figure 5. Fatigue, dry mouth and loss of taste had higher interquartile ranges than did the other factors. Except for fatigue and dry mouth, the median duration of each symptom was mostly short (less than 20 days). Furthermore, the period without symptoms was relatively long, accounting for 42% of the total duration of systemic therapy.

Figure 5

Figure 5. Duration of the 10 most frequent symptoms in patients on systemic therapy, presented in boxplots. Boxes represent the interquartile range with the mean (cross) and median (line) in the box, whiskers are to a maximum of 1.5 times the interquartile range, and points outside the whiskers represent outliers. The last boxplot represents the time without symptoms.

3.4 Changes in quality of life based on the EORTC C30 and BR23 questionnaires

At baseline, patients reported high scores (>75) on all five functional scales of the EORTC QLQ C30 and the EORTC QLQ BR23 for body image. However, their perception of global health status (GHS/QoL) was not high (53.1), and their future perspective was low (36.2). Symptom scores ≥ 20 were reported for insomnia (40.6), fatigue (34.3), pain (25.4) and breast symptoms (23.3). A detailed description of the functional and symptom ratings is provided in Table 2.

Table 2

Table 2. Baseline patient-reported outcomes based on the European Organization for Research and Treatment of Cancer Quality of Life Core Questionnaire 30 (EORTC QLQ C30) and Breast Module 23 (EORTC QLQ BR23) and reports after 3 months of systemic treatment and changes from baseline (differences).

After 3 months of systemic treatment, patients reported significant deterioration in body image and physical, cognitive and social functioning; however, sexual functioning increased. Symptoms scales revealed significantly increased severity of some symptoms: hair loss for 73.5 points, fatigue for 19.9 points, systemic therapy side effects for 19.6 points and dyspnea for 13.7 points. On the other hand, their arm and breast symptoms significantly decreased. The GHS/QoL score remained stable (Table 2).

3.5 Satisfaction with the mobile application

Six months after the end of the study, we collected anonymous feedback from both patients and doctors regarding the usefulness of and level of satisfaction with the m-app. Results are presented in Figure 6. A total of 28 patients (61%) and 4 doctors (44%) provided feedback. Among the 10 doctors who recruited patients, 4 doctors included only two patients, other the 6 doctors included 4–13 patients for participation in this study with ePROs. Patients and doctors rated the application with an overall score of 4.5 out of 5 (IQR 1.0). Patients generally remarked on applications’ ease of use and the included useful information and that they would recommend it to others. The IQRs for all these questions ranged from 4–5. The largest variance was in the answers to the questions of whether the notifications were useful and whether they used the application all the time. However, they were slightly less satisfied with the feedback from doctors. The doctors did not always have a printed report of the patients’ reported symptoms at the next check-up 4 (IQR 2.8–5). Patients rated feedback from doctors discussing e-reports and encouraging continued m-app usage even less favorably 3 (IQR 2.0–4.3).

Figure 6

Figure 6. Evaluation of the mobile application (m-app) by patients (A) and by doctors (B). The results are presented as boxplots and whiskers. The evaluation was performed on a 5-point scale, where 1 represents the lowest value or level of satisfaction and 5 represents the highest value or level of satisfaction. The orange part of the box represents quartile 25 to the median, and the gray part of the box represents quartile 75 to the median.

Doctors were slightly less satisfied with the application, agreeing that it had useful tips for patients. However, they reported that it did not make it easier for them to take a medical history. Furthermore, they reported that it took them even more time and that patients were more demanding, so they would be less likely to recommend it to other doctors. Among the four that returned questionnaires, one expressed dissatisfaction, claiming the increased time required for patient examination and the impracticality of the application’s data for outpatient care. However, the remaining 3 doctors (75%) were satisfied with the application and did not consider it an additional burden during patient treatment. Notably, they mostly agreed that e-PRO information should be part of a patient’s EHR, but there should be warranted alerts on serious reports.

4 Discussion

This prospective cohort study focused on daily symptom reporting in patients undergoing chemotherapy for early breast cancer. Patients using ePROs reported a significantly higher number of symptoms, and these symptoms were more severe than those reported in the subsequent assessment and interpretation by doctors during the same patients’ next outpatient visit. The symptoms most underreported by doctors were insomnia and dry mouth. We also used two validated questionnaires, the EORTC QLQ C30 and BR23, and compared periodic and daily symptom reporting. Patient and doctor satisfaction with and usability of the m-app were evaluated by patients and doctors. Both expressed high levels of satisfaction, although doctors expressed a need for a plan to manage severe symptoms when they were reported and a concern that they would be able to take over the additional workload to react to severe symptoms.

In the last decade, ePROs have become a hot topic due to advancements in digital technology and the widespread availability of smartphones, making these services globally accessible. The logical next step is the use of ePROs as a communication tool between patients and doctors, particularly in medical treatments such as cancer treatment, where prompt action at the onset of symptoms is crucial. The goal is to ensure the maintenance of patients’ QoL and adherence to their treatment regimens. Several small studies have been performed worldwide; however, the routine use of ePROMs and incorporation of these data into EHRs were still very rare until recently (11, 24, 25). The experience of The Christie NHS Foundation Trust, launched in 2019, stands out as a key example of large-scale ePROM integration into routine oncology care (26, 27). Real-time data entry into the EHR—without relying on separate platforms—enhanced the clinical usefulness for treating clinicians. Reported benefits include fewer hospital admissions, better quality of life, potential survival benefits, and more efficient consultations (26, 28). Later evaluations identified barriers among older patients with limited access to technology and those receiving radical treatment. Clinicians have also requested clearer data displays within EHR (27). Another UK oncology center that uses ePROMs for patients receiving targeted and immunotherapies has shown a reduced need for face-to-face visits (29). A program using the PRO-CTCAE and pain scales during palliative radiotherapy offered automated self-care guidance and guidance for contacting healthcare providers (30). Studies with ePROM implementation highlight benefits such as improved symptom control, patient-provider communication and quality of life (30–32). The European Society for Medical Oncology (ESMO) now recommends PROM-based monitoring after cancer treatment, including near end-of-life care (33).

In this trial, we prospectively assessed daily symptom reports. Over 4–6 months of chemotherapy, patients reported 75 symptoms (median 25 per patient): 67% mild, 30% moderate, and 3% severe, with fatigue, insomnia, stomatitis, myalgias, arthralgias, and diarrhea among the severe symptoms. Our findings are consistent with an Italian study that also identified fatigue as a highly prevalent but often overlooked severe side effect of adjuvant systemic therapy (12). We then compared patient-reported symptoms with doctor assessments at regular check-ups. Doctors reported fewer symptoms (49 vs. 75, median 7 per patient, IQR 6–10) and generally graded them lower (Figure 3). However, Grade 3 symptoms accounted for 3% of the symptoms in both reports. Patients most frequently reported fatigue, insomnia, and dry mouth (Figure 4), with insomnia and dry mouth being symptoms the most underreported by doctors. Our findings align with those of previous studies (9, 34) showing that doctors tend to underestimate symptoms, particularly those where specific treatments are unavailable. Notably, Grade 3 muscle/joint pain, fatigue, and nausea were underestimated (35). Overall, our results suggest that patients report numerous adverse events affecting their QoL. In contrast, doctors prioritize reporting those that influence treatment decisions, such as not prescribing (postponing) therapy, dose adjustments, or supportive therapy adjustments. We acknowledge the potential for assessment bias in the doctor’s grading of symptom, given the retrospective nature of the assessment. The risk was to some extent alleviated by the use of single assessor. While it may enhance consistency, it does not eliminate bias and could introduce a single-assessor bias, where that an individual’s interpretation systematically influences the data.

In addition, our study highlights the effectiveness of the m-app in managing adverse symptoms. The low percentage of Grade 3 symptoms is likely due to the supportive measures provided by the m-app toolkit, which helps patients self-manage mild to moderate symptoms, preventing escalation to Grade 3. This aligns with previous studies showing that PROs enable early recognition and management of adverse events, reducing symptom duration (12, 36). Notably, our patients were symptom-free for 42% of the treatment duration, whereas Antonuzzo et al. (12) reported a symptom-free period of only 15%, likely due to differences in patient characteristics, chemotherapy regimens, or study methodologies. In our previous prospective cohort study, we demonstrated that early breast cancer patients using the m-app toolkit experienced improved quality of life (QoL) both in the first week after chemotherapy and at the end of treatment compared with the control group without application support (21). Similarly, Fjell et al. reported that an interactive application reduced symptom burden and improved emotional well-being during neoadjuvant chemotherapy for patients with breast cancer (37).

There is a worldwide debate regarding the optimal frequency of ePRO assessment during cancer therapy: once-weekly or daily assessment. Both approaches have advantages and limitations, with more precise symptom reporting in daily monitoring and less burdensome reporting weekly. Symptom monitoring empowers patients to actively contribute to problem-solving measures. On the other hand, experiences revealed survey fatigue in patients, which means that they responded less regularly when they became familiar with symptom management. Therefore, strategies on the frequency of symptom recording should be adjusted to a particular setting (active chemotherapy treatment, maintenance treatment, follow-up on chronic therapy) (38). For example, during adjuvant endocrine therapy, 73.2% of patients completed the baseline survey, whereas 69.6% participated in at least one 3-monthly ePRO follow-up survey via a smartphone within the first six months (39). Like our study, Daly et al. utilized daily symptom reporting in adults receiving anticancer therapy. They analyzed the impact of remote monitoring of red alerts (severe symptoms) and reported that 8.7% of patients who triggered a red alert required an acute care visit within seven days, whereas 2.9% of those without a red alert required an acute care visit. However, after six months, only a quarter of the patients continued daily reporting. The most common triggers for red alerts are pain, dyspnea, and functional decline (40).

A key consideration is whether real-time symptom reporting, despite its complexity for both patients and healthcare professionals, has advantages over cross-sectional validated questionnaires such as the QLQ-C30 and BR23. In our study, real-time reporting identified mouth ulcers as the 4th most prevalent symptom (Figure 4), and questions regarding this particular symptom are not available in either C30 or BR23. However, the updated version of BR23, BR42, now includes questions addressing mouth ulcers under items 57 and 58 (soreness and redness in the mouth). Among the 10 most common symptoms reported by our patients, three (dry mouth, taste changes, and hair loss) are covered in BR23, whereas the remaining six are included in C30. Notably, BR23 does not score symptoms individually but rather as part of a composite “systemic therapy side effects” score. Additionally, the timeframe of assessment differs: C30 evaluates symptoms over the past week, whereas BR23 covers the past four weeks. In our view, daily symptom reporting has advantages at the start of a new treatment or for treatments where several side effects are expected, allowing for immediate intervention, whereas C30 and BR23 provide a more comprehensive overview of baseline status, long-term symptom trends over milestones and monitoring adherence to treatment.

Despite the proven benefits, the widespread integration of e-PROs into routine clinical practice faces numerous challenges, including time consumption, logistical issues regarding severe symptom management and physician reluctance to adopt new technologies. With respect to adoption by doctors, four of our doctors included only two patients in the study, other the six included 4–13 patients. In our study, 56% of the doctors chose not to provide feedback on their evaluation and satisfaction with the application. Among the 44% who did share their opinions (4 doctors in total), the majority (75%) expressed satisfaction and supported its integration into daily practice. However, one doctor (25%) provided a strongly negative opinion, viewing the application as a burden that detracts from the valuable time spent with patients without offering significant additional benefits. There was some additional time spent on patients’ check-ups. The initial patient enrolment took an additional half hour from the physician, and the first follow-up visit was extended by approximately 5–10 minutes for report review and app discussion. Subsequent consultation times were not prolonged, as symptom history taking was potentially made more efficient.

In contrast, patients expressed a very positive opinion on the application. They saw it as a tool that provides a sense of security and helps them manage side effects, which is in line with the observations previously reported in different studies and meta-analyses (11, 41–43). This positive perception is reflected in high initial adherence to adverse effect reporting, which gradually declines as patients become more proficient in managing side effects. Similarly, Handa et al. reported a 25% decrease in adherence to m-app usage from the first chemotherapy cycle to the fourth cycle (35). Lee et al. reported that patients with greater expectations of the application’s usefulness and ease of use were more likely to adopt it, although ease of use did not directly impact compliance (39). We are aware, that usability and satisfaction survey feedback was received from 28 of 46 patients (61%), which represents attrition bias. Owing to survey anonymity and postal distribution, we could not increase the percentage of respondents using reminders, causing attrition bias. The delay in performing the survey (6 months poststudy) may have contributed to nonresponse, possibly introducing selection bias—more satisfied users may have been more likely to respond. Nevertheless, frequent users likely provided the most informed feedback. When considering the usability and satisfaction of m-app users, we need to be aware of the critical limitations of our study in terms of lack of representation from older, digital or health-literacy marginalized populations. On the basis of our sample of young and digitally literate women with breast cancer, it is difficult to draw conclusions about the generalizability of the findings to the general population. Among the physicians, 4 out of 10 responded. Nonrespondents may reflect resistance to use ePROMs, as they had not adopted them in routine practice. This may partly explain also the observed difference in symptom reporting grades.

Considering all the advantages and barriers, we believe that ePROs are promising to become an integral part of routine patient assessment in the near future. Only a quarter of the surveyed practitioners reported capturing PROs in routine clinical practice (44). Many countries have experience with various cancer applications, but only a few enable real-time recording and sharing of ePROs with clinicians (26, 29, 30, 45, 46). Despite the known advantages, the acceptance of ePROs in routine clinical care has been hindered by work processes and technology-based challenges (18). Clinic staff reported frustration and dissatisfaction, citing increased workload without perceived benefits for oncological treatment (11). Further investigation is needed to integrate these applications with EHR, emphasizing the importance of implementation standards, certification and user feedback. The inclusion of stakeholders, health professionals, IT specialists, and patient representatives is crucial in the process (47–50). This multidisciplinary approach could improve patients’ perceived health, outcomes, and hospital resource optimization for ePROs. The benefits of remote control of symptoms include proactive early intervention to reduce symptoms, symptom self-management and improved patient–provider relationships (38).

Our study has several strengths and limitations. One of the major strengths, in our opinion, is daily symptom reporting via the m-app, as only a few studies have enabled real-time symptom monitoring thus far. Additionally, our study empowers patients with appropriate tips for self-management of mild to moderate symptoms. Furthermore, it also offers valuable insights into the different perceptions of adverse effects between patients and doctors, as well as their opinions regarding PRO usage. The main limitations, on the other hand, are the small number of participants, the single-center nature of the study and the requirement for smartphone access, which excluded a significant portion of the elderly population. We acknowledge the temporal mismatch between patients’ daily reporting and retrospective grading of doctors’ symptom assessment as a potential confounder. Next, we did not anticipate creating alerts for Grade 3 symptoms; instead, following our usual practice, we instructed patients to visit the nearest emergency department. There were also some increased costs regarding staff engagement (patient education and preparing reports for doctors). Finally, we used our own questionnaire to evaluate the level of satisfaction and usability of the m-app; however, we later discovered that a validated usability questionnaire, the mHealth App Usability Questionnaire (MAUQ), already exists (51). Therefore, we have validated this questionnaire for the Slovenian language for future studies with the m-app.

5 Conclusion

Our study demonstrates that real-time symptom tracking via an m-app improves symptom detection, with patients reporting nearly twice as many symptoms as doctors do. While patients found the application highly useful (4.5/5 rating), doctors were more hesitant, perceiving ePROs as an additional burden. Despite some reluctance, mobile-based ePRO follow-up is promising, but requires more systematic, inclusive, and scalable implementation studies to establish feasibility and sustainability in routine practice.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The study involving humans was approved by Commission for the Peer Review of Research Protocols and Ethics Committee at the Institute of Oncology Ljubljana. The study was conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

CGK: Visualization, Resources, Formal analysis, Funding acquisition, Writing – original draft, Supervision, Data curation, Investigation, Writing – review & editing, Methodology, Conceptualization. NP: Writing – original draft, Writing – review & editing, Formal analysis, Data curation, Visualization, Investigation. MS: Project administration, Data curation, Writing – review & editing, Investigation. EG: Writing – review & editing, Investigation, Data curation, Project administration, Visualization. TK: Software, Resources, Funding acquisition, Writing – review & editing, Validation, Methodology, Visualization, Supervision, Conceptualization. MK: Software, Resources, Writing – review & editing, Visualization, Funding acquisition, Methodology, Validation, Conceptualization, Supervision.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. The authors acknowledge the financial support from the Slovenian Research Agency (research core funding No. P2-209 and P3-0321).

Acknowledgments

The authors would like to express their immense gratitude to medical oncologists, study nurses and patients.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2025.1595322/full#supplementary-material

Supplementary Figure 1 | Free text presentation of patient reported symptoms.

Supplementary Figure 2 | Graphical presentations of time-course of patient reported symptoms and their grades.

Supplementary Figure 3 | Patient reported symptoms in Excel table with their grades.

References

1. Cancer Registry Slovenia Slora.si. Incidenčne mere (2022). Available online at: http://www.slora.si/incidenca (Accessed March 5, 2025).

Google Scholar

2. Zadnik V, Zagar T, Lokar K, Tomsic S, Konjevic AD, and Zakotnik B. Trends in population-based cancer survival in Slovenia. Radiol Oncol. (2021) 55:42–9. doi: 10.2478/raon-2021-0003

PubMed Abstract | Crossref Full Text | Google Scholar

3. Mittal A, Kim MS, Dunn S, Wright K, and Gyawali B. Frequently asked questions on surrogate endpoints in oncology-opportunities, pitfalls, and the way forward. EClinicalMedicine. (2024) 76:102824. doi: 10.1016/j.eclinm.2024.102824

PubMed Abstract | Crossref Full Text | Google Scholar

4. Weldring T and Smith SM. Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights. (2013) 6:61–8. doi: 10.4137/HSI.S11093

PubMed Abstract | Crossref Full Text | Google Scholar

5. Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC, et al. Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. (2014) 106:dju244. doi: 10.1093/jnci/dju244

PubMed Abstract | Crossref Full Text | Google Scholar

6. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, NJ D, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. (1993) 85:365–76. doi: 10.1093/jnci/85.5.365

PubMed Abstract | Crossref Full Text | Google Scholar

7. Michels FAS, Latorre MDRDDO, and Maciel MDS. Validity, reliability and understanding of the EORTC-C30 and EORTC-BR23, quality of life questionnaires specific for breast cancer. Rev Bras Epidemiol. (2013) 16:352–63. doi: 10.1590/S1415-790X2013000200011

PubMed Abstract | Crossref Full Text | Google Scholar

8. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. (2016) 34:2198. doi: 10.1200/JCO.2015.63.0830

PubMed Abstract | Crossref Full Text | Google Scholar

9. Fromme EK, Eilers KM, Mori M, Hsieh YC, and Beer TM. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol. (2004) 22:3485–90. doi: 10.1200/JCO.2004.03.025

PubMed Abstract | Crossref Full Text | Google Scholar

10. Denis F, Basch E, Septans AL, Bennouna J, Urban T, Dueck AC, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA. (2019) 321:306–7. doi: 10.1001/jama.2018.18085

PubMed Abstract | Crossref Full Text | Google Scholar

11. Kiderlen TR, Schnack A, and de Wit M. Essential barriers and considerations for the implementation of electronic patient-reported outcome (ePRO) measures in oncological practice: contextualizing the results of a feasibility study with existing literature. Z Gesundh Wiss. (2022) 31:2071–88. doi: 10.1007/s10389-022-01767-3

PubMed Abstract | Crossref Full Text | Google Scholar

12. Antonuzzo A, Ripamonti CI, Roila F, Sbrana A, Galli L, Miccinesi G, et al. Effectiveness of a phone-based nurse monitoring assessment and intervention for chemotherapy-related toxicity: A randomized multicenter trial. Front Oncol. (2022) 12:925366. doi: 10.3389/fonc.2022.925366

PubMed Abstract | Crossref Full Text | Google Scholar

13. Basch E, Schrag D, Henson S, Jansen J, Ginos B, AM S, et al. Outcomes among patients with metastatic cancer: A randomized clinical trial. JAMA. (2022) 327:2413–22. doi: 10.1001/jama.2022.9265

PubMed Abstract | Crossref Full Text | Google Scholar

14. Basch E, GN M, and Dueck AC. Electronic patient-reported outcomes as digital therapeutics to improve cancer outcomes. JCO Oncol Pract. (2020) 16:541–2. doi: 10.1200/OP.20.00264

PubMed Abstract | Crossref Full Text | Google Scholar

15. Kluetz PG, Chingos DT, Basc hEM, and Mitchell SA. Patient-reported outcomes in cancer clinical trials: measuring symptomatic adverse events with the national cancer institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Am Soc Clin Oncol Educ Book. (2016) 35:67–73. doi: 10.1200/EDBK_159514

PubMed Abstract | Crossref Full Text | Google Scholar

16. Pradhan A, Dwivedi P, Pareek P, Kalra S, Singh S, SB V, et al. Electronic-PRO Measures for adverse events Of Treatment In ONcology (E-PROMOTION). Int J Med Inform. (2024) 182:105305. doi: 10.1016/j.ijmedinf.2023.105305

PubMed Abstract | Crossref Full Text | Google Scholar

17. Lee M, Kang D, Kang E, Kim S, Kim Y, JS A, et al. Efficacy of the PRO-CTCAE mobile application for improving patient participation in symptom management during cancer treatment: a randomized controlled trial. Support Care Cancer. (2023) 31:321. doi: 10.1007/s00520-023-07779-3

PubMed Abstract | Crossref Full Text | Google Scholar

18. Sandhu S, King Z, Wong M, Bissell S, Sperling J, Gray M, et al. Implementation of electronic patient-reported outcomes in routine cancer care at an academic center: identifying opportunities and challenges. JCO Oncol Pract. (2020) 16:e1255–e63. doi: 10.1200/OP.20.00357

PubMed Abstract | Crossref Full Text | Google Scholar

19. Borges do Nascimento IJ, Abdulazeem H, Vasanthan LT, Martinez EZ, Zucoloto ML, Østengaard L, et al. Barriers and facilitators to utilizing digital health technologies by healthcare professionals. NPJ Digit Med. (2023) 6:161. doi: 10.1038/s41746-023-00899-4

PubMed Abstract | Crossref Full Text | Google Scholar

20. Google Play Store. OnkoVed-app (2021). Available online at: https://play.google.com/store/apps/details?id=com.kovac.timotej.breastcare&hl=en (Accessed March 15, 2025).

Google Scholar

21. Grašič Kuhar C, Gortnar Cepeda T, Kovač T, Kukar M, and Ružić Gorenjec N. Mobile app for symptom management and associated quality of life during systemic treatment in early stage breast cancer: nonrandomized controlled prospective cohort study. JMIR Mhealth Uhealth. (2020) 8:e17408. doi: 10.2196/17408

PubMed Abstract | Crossref Full Text | Google Scholar

22. U. S. Department of Health and Human Services. Common Terminology Criteria for Adverse Events (CTCAE) v5.0 (2017). Available online at: https://ctep.cancer.gov/protocoldevelopment/electronic_applications/docs/ctcae_v5_quick_reference_5x7.pdf (Accessed March 5, 2025).

Google Scholar

23. Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D, Bottomley A, et al. The EORTC QLQ-C30 scoring manual. 3rd Edition. Brussels: European Organisation for Research and Treatment of Cancer (2001).

Google Scholar

24. Basch E, Barbera L, Kerrigan CL, and Velikova G. Implementation of patient-reported outcomes in routine medical care. Am Soc Clin Oncol Educ Book. (2018) 38:122–34. doi: 10.1200/EDBK_200383

PubMed Abstract | Crossref Full Text | Google Scholar

25. Cheung YT, Chan A, Charalambous A, Darling HS, Eng L, Grech L, et al. The use of patient-reported outcomes in routine cancer care: preliminary insights from a multinational scoping survey of oncology practitioners. Support Care Cancer. (2022) 30:1427–39. doi: 10.1007/s00520-021-06545-7

PubMed Abstract | Crossref Full Text | Google Scholar

26. Crockett C, Gomes F, Faivre-Finn C, Howell S, Kasipandian V, Smith E, et al. The routine clinical implementation of electronic patient-reported outcome measures (ePROMs) at the christie NHS foundation trust. Clin Oncol (R Coll Radiol). (2021) 33:761–4. doi: 10.1016/j.clon.2021.06.004

PubMed Abstract | Crossref Full Text | Google Scholar

27. Shipman LA, Price J, Abdulwahid D, Bayman N, Blackhall F, Califano R, et al. Service evaluation of myChristie-myHealth, an electronic patient-reported outcome measure integrated into clinical cancer care. JCO Clin Cancer Inform. (2024) 8:e2300162. doi: 10.1200/CCI.23.00162

PubMed Abstract | Crossref Full Text | Google Scholar

28. Crockett C, Faivre-Finn C, Yorke J, and Price J. Initial clinical experience of routine electronic patient-reported outcome measures at the christie NHS foundation trust. Clin Oncol (R Coll Radiol). (2022) 34:e168. doi: 10.1016/j.clon.2021.12.011

PubMed Abstract | Crossref Full Text | Google Scholar

29. Sivanandan MA, O’Cathail M, and Christian J. Electronic patient-reported outcome measures for monitoring of patients on cancer treatment can reduce the need for face-to-face hospital visits: A tertiary UK oncology centre experience. Clin Oncol (R Coll Radiol). (2022) 34:e79. doi: 10.1016/j.clon.2021.10.003

PubMed Abstract | Crossref Full Text | Google Scholar

30. Oldenburger E, Isebaert S, Coolbrandt A, Van Audenhove C, and Haustermans K. Electronic Patient Reported Outcome Measures after palliative radiotherapy: evaluation of implementation. Ann Palliat Med. (2024) 13:1317–31. doi: 10.21037/apm-24-74

PubMed Abstract | Crossref Full Text | Google Scholar

31. Payne A, Horne A, Bayman N, Blackhall F, Bostock L, Chan C, et al. Patient and clinician-reported experiences of using electronic patient reported outcome measures (ePROMs) as part of routine cancer care. J Patient Rep Outcomes. (2023) 7:42. doi: 10.1186/s41687-023-00544-4

PubMed Abstract | Crossref Full Text | Google Scholar

32. Hubel NJ, Vorbach V, de Ligt KM, Rathgeber IS, Beyer K, Wintner LM, et al. Sustainability and time trends in electronic patient-reported outcome assessment in routine cancer care: systematic scoping review and follow-up survey. J Med Internet Res. (2025) 27:e69398. doi: 10.2196/69398

PubMed Abstract | Crossref Full Text | Google Scholar

33. Di Maio M, Basch E, Denis F, Fallowfield LJ, PA G, Howell D, ESMO Guidelines Committee, et al. The role of patient-reported outcome measures in the continuum of cancer clinical care: ESMO Clinical Practice Guideline. Ann Oncol. (2022) 33:878–92. doi: 10.1016/j.annonc.2022.04.007

PubMed Abstract | Crossref Full Text | Google Scholar

34. Detmar SB, Muller MJ, Schornagel JH, Wever LD, and Aaronson NK. Health-related quality-of-life assessments and patient–physician communication: a randomized controlled trial [published correction appears in. JAMA. 2003 Feb 26;289(8):987.]. JAMA. (2002) 288:3027–34. doi: 10.1001/jama.288.23.3027

PubMed Abstract | Crossref Full Text | Google Scholar

35. Handa S, Okuyama H, Yamamoto H, Nakamura S, and Kato Y. Effectiveness of a smartphone application as a support tool for patients undergoing breast cancer chemotherapy: A randomized controlled trial. Clin Breast Cancer. (2020) 20:201–8. doi: 10.1016/j.clbc.2020.01.004

PubMed Abstract | Crossref Full Text | Google Scholar

36. Molassiotis A, Brearley S, Saunders M, Craven O, Wardley A, Farrell C, et al. Effectiveness of a home care nursing program in the symptom management of patients with colorectal and breast cancer receiving oral chemotherapy: a randomized, controlled trial. J Clin Oncol. (2009) 27:6191–8. doi: 10.1200/JCO.2008.20.6755

PubMed Abstract | Crossref Full Text | Google Scholar

37. Fjell M, Langius-Eklöf A, Nilsson M, Wengström Y, and Sundberg K. Reduced symptom burden with the support of an interactive app during neoadjuvant chemotherapy for breast cancer - A randomized controlled trial. Breast. (2020) 51:85–93. doi: 10.1016/j.breast.2020.03.004

PubMed Abstract | Crossref Full Text | Google Scholar

38. Rocque GB. Learning from real-world implementation of daily home-based symptom monitoring in patients with cancer. JAMA Netw Open. (2022) 5:e221090. doi: 10.1001/jamanetworkopen.2022.1090

PubMed Abstract | Crossref Full Text | Google Scholar

39. Lee M, Kang D, Kang E, Kim S, Kim Y, Ahn JS, et al. Efficacy of the PRO-CTCAE mobile application for improving patient participation in symptom management during cancer treatment: a randomized controlled trial. Support Care Cancer. (2023) 31:321. doi: 10.1007/s00520-023-07779-3

PubMed Abstract | Crossref Full Text | Google Scholar

40. Daly B, Nicholas K, Flynn J, Silva N, Panageas K, Mao JJ, et al. Analysis of a remote monitoring program for symptoms among adults with cancer receiving antineoplastic therapy. JAMA Netw Open. (2022) 5:e221078. doi: 10.1001/jamanetworkopen.2022.1078

PubMed Abstract | Crossref Full Text | Google Scholar

41. Kennedy F, Absolom K, Clayton B, Rogers Z, Gordon K, O’Connell Francischetto E, et al. Electronic patient reporting of adverse events and quality of life: A prospective feasibility study in general oncology. JCO Oncol Pract. (2021) 17:e386–96. doi: 10.1200/OP.20.00118

PubMed Abstract | Crossref Full Text | Google Scholar

42. Nguyen H, Butow P, Dhillon H, and Sundaresan P. A review of the barriers to using Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs) in routine cancer care. J Med Radiat Sci. (2021) 68:186–95. doi: 10.1002/jmrs.421

PubMed Abstract | Crossref Full Text | Google Scholar

43. Richards HS, Blazeby JM, Portal A, Harding R, Reed T, Lander T, et al. A real-time electronic symptom monitoring system for patients after discharge following surgery: a pilot study in cancer-related surgery. BMC Cancer. (2020) 20:543. doi: 10.1186/s12885-020-07027-5

PubMed Abstract | Crossref Full Text | Google Scholar

44. Aanes SG, Wiig S, Nieder C, and Haukland EC. ESMO real world data and digital oncology (Elsevier), Vol. 6. (2024). p. 100088. p. 100088. doi: 10.1016/j.esmorw.2024.100088.

Crossref Full Text | Google Scholar

45. Hage AM, Gebert P, Kühn F, Pross T, Grittner U, and Karsten MM. Examining the feasibility of an application-based patient-reported outcome monitoring for breast cancer patients: A pretest for the PRO B study. Int J Environ Res Public Health. (2022) 19:8284. doi: 10.3390/ijerph19148284

PubMed Abstract | Crossref Full Text | Google Scholar

46. Vercell A, Gasteiger N, Yorke J, and Dowding D. Patient-facing cancer mobile apps that enable patient reported outcome data to be collected: A systematic review of content, functionality, quality, and ability to integrate with electronic health records. Int J Med Inform. (2023) 170:104931. doi: 10.1016/j.ijmedinf.2022.104931

PubMed Abstract | Crossref Full Text | Google Scholar

47. Tsangaris E, Hyland C, Liang G, O’Gorman J, Huerta DT, Kim E, et al. Feasibility of implementing patient-reported outcome measures into routine breast cancer care delivery using a novel collection and reporting platform. JAMIA Open. (2023) 6:ooad108. doi: 10.1093/jamiaopen/ooad108

PubMed Abstract | Crossref Full Text | Google Scholar

48. da Silva Lopes AM, Colomer-Lahiguera S, Darnac C, Giacomini S, Bugeia S, Gutknecht G, et al. Development of an eHealth-enhanced model of care for the monitoring and management of immune-related adverse events in patients treated with immune checkpoint inhibitors. Support Care Cancer. (2023) 31:484. doi: 10.1007/s00520-023-07934-w

PubMed Abstract | Crossref Full Text | Google Scholar

49. Taramasco C, Rimassa C, Noël R, ML BS, and Sánchez C. Co-design of a mobile app for engaging breast cancer patients in reporting health experiences: qualitative case study. J Med Internet Res. (2023) 25:e45968. doi: 10.2196/45968

PubMed Abstract | Crossref Full Text | Google Scholar

50. Tsangaris E, Edelen M, Means J, Gregorowitsch M, O’Gorman J, Pattanaik R, et al. User-centered design and agile development of a novel mobile health application and clinician dashboard to support the collection and reporting of patient-reported outcomes for breast cancer care. BMJ Surg Interv Health Technol. (2022) 4:e000119. doi: 10.1136/bmjsit-2021-000119

PubMed Abstract | Crossref Full Text | Google Scholar

51. Zhou L, Bao J, Setiawan IMA, Saptono A, and Parmanto B. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth. (2019) 7:e11500. doi: 10.2196/11500

PubMed Abstract | Crossref Full Text | Google Scholar