This is an open-access article distributed under the terms of the
A stratified approach to cancer follow-up care, including remote monitoring and supported self-management for suitable patients, is increasingly recommended. Patient portals can facilitate such an approach, allowing access to results and information. There is limited understanding of the use of portals within this context.
This paper reports the acceptability and usability of a patient portal developed to facilitate a remote monitoring and supported self-management follow-up care programme for men with prostate cancer.
A mixed methods evaluation, including analysis of service utilisation data, a survey of users' experiences and satisfaction, and telephone interviews of non-users' views and experiences.
Sixty percent of eligible patients registered to use the portal. Of these, 37% logged in at least once over a 6-month period and 52% over 12 months. Satisfaction among these users was reasonably high. Use of the portal in general was rated as very easy or easy by over 70% of respondents, and the majority felt the portal had helped them manage their condition in various ways. However, a large minority (40%) did not use the portal, with reasons for non-use including lack of access to computing facilities and lack of computer skills. Those who were older were less likely to register to use the portal.
A large proportion of participants found the patient portal acceptable and easy to use. Reasons for non-use should be addressed in order to maximise system efficiencies and minimise inequalities in care, and the needs of specific groups should be taken into account.
The use of digital solutions for patient care management is undergoing rapid expansion internationally. Developments include electronic medical/health records, telemonitoring systems, online patient portals, web-based interventions and mobile apps. Such solutions are key to addressing increasing demand for health care (
Patient portals are secure online websites (
With improving survival rates, cancer is now considered a long-term condition. Post-treatment follow-up care to monitor for disease recurrence and to address treatment related side-effects continues for a number of years, but services are increasingly challenged to manage this activity for a rapidly growing population of patients. In the UK, a stratified approach to follow-up care is recommended, replacing routine face-to-face follow-up care appointments with remote monitoring and supported self-management for suitable patients (
There has, to date, been limited evaluation of the use of patient portals in cancer populations (
This paper reports the acceptability and usability of a patient portal developed to facilitate a remote monitoring and supported self-management follow-up care programme for men with prostate cancer.
As part of a global initiative to improve outcomes for men treated for prostate cancer, the Movember Foundation funded a service improvement initiative, the TrueNTH Supported Self-Management and Follow Up Programme, in five NHS Trusts in England from 2014 to 2017. The aim of the Programme was to redesign the post-treatment follow up care pathway, moving from a traditional model of clinic/office based follow up appointments to one of remote monitoring and supported self-management for men at low risk of recurrence and able to manage their own health needs. Evaluation of the impact of the Programme as a whole on patient reported outcomes and costs is reported elsewhere (
The Programme is enabled through an integrated online system, which encompasses the patient portal alongside a healthcare professional accessed clinical monitoring system. The online system was designed by a working group involving clinicians, patients, academics, IT specialists and third sector representatives. The system uses a platform developed by Get Real Health™. It was piloted with two NHS organisations in 2014 and, following minor modifications, was rolled out to another three NHS organisations. Version 1 of the System was launched on 02/07/2014. Patients experienced difficulties with a complex registration process, so a second version with a modified process was launched on 20/10/2014. The system is managed by University Hospital Southampton NHS Foundation Trust through the Microsoft Azure cloud-based platform. A screen shot of the home page is shown in
Screenshot of patient portal home page*. *Figure provided by Informatics Department, University Hospital Southampton NHS Foundation Trust.
Men are assessed for suitability for the Programme at their end of treatment appointment. If they meet the Programme criteria (which include assessment against defined clinical criteria, as well as symptom resolution and patient preference) they are introduced to the Programme by a Cancer Support Worker who, under supervision of a Clinical Nurse Specialist, manages the Programme on a day-to-day basis. Henceforth, men do not have scheduled clinic/office-based appointments, with their care being managed remotely. All men entering the Programme are registered on the system, since their care is managed through the clinical monitoring aspect of the system. This allows the clinical team to track patients’ follow-up care and test results and to run on-line “virtual clinics”, whereby they view details of all patients due for review, record their decisions about follow up care and communicate these to the patient. Blood tests for the marker of prostate cancer recurrence (Prostate Specific Antigen or PSA) are taken by the man's General Practice or hospital phlebotomy service and results transferred directly from the pathology laboratory to the online system. PSA values are monitored by the Support Worker and Clinical Nurse Specialist through the clinical monitoring system and men are only recalled to clinic if their results prompt further investigation. Men choose whether or not to register with the patient portal. They attend a four-hour workshop to introduce them to the Programme and during this session can view the patient portal and ask questions. All men also receive letters detailing their test result and have easy access to the clinical team by telephone, ensuring that men who choose not to register on the portal still have access to information and support.
The patient portal has four broad functions:
The PSA results and tracker functionality presents results to patients as soon as available, often within a few hours of the blood test being taken. Previous PSA results are held on the system and presented on a line graph to allow comparison over time.
An e-mail style messaging service allows for secure messaging between patient and clinician. Messages are internal to the system and can only be viewed from within it. Notifications are sent to patient and clinicians' external e-mail accounts prompting them to log in and view the message. The messaging service allows patients to contact their clinical team with any questions or concerns without having to telephone or visit the hospital, while allowing the clinicians to respond to messages when appropriate and convenient.
A Health MOT (holistic needs assessment or HNA) comprises a questionnaire completed by the patient within the portal. HNAs ask about emotional, practical, financial and clinical concerns in order to facilitate personalised care (
A patient information function provides an online space for validated information about prostate cancer, in the form of documents, links to other websites and film clips. This includes information on managing side effects and consequences of cancer, healthy lifestyles and support for computer/IT skills.
A mixed methods design was used to assess the experiences of both users and non-users of the portal. This included analysis of utilisation data, an online survey of the experience and satisfaction of those who chose to have access to the patient portal, and structured telephone interviews of non-users' views and experiences. The evaluation took place between July and September 2017, three years after introduction of the portal into the prostate cancer follow up care pathway. All patients at the five NHS Trusts registered on the Programme on 31 August 2017 were included.
Portal utilisation data were extracted using bespoke data extracts. The data items for each patient included date registered on the Programme, NHS Trust, whether they were registered to use the portal, and how many times they had logged into the portal during the 6 months and 12 months prior to 31 August 2017. Data relating to website traffic and use of the key functions of the portal (patient information, clinician messaging, PSA results, and Health MOT) were collected anonymously using Google Analytics.
An online survey to assess users' experiences of the portal was sent to all registered patients who had logged onto the portal more than once over a 28-month period from 1 January 2015 to 30 April 2017. A link to the survey was e-mailed to these users on 17 August 2017 (allowing time for those who registered during the latter months to form an opinion of the portal). The survey was open for a period of 29 days. Questions were asked about views and use of the portal; support for use; and impact on care management. The survey included various question types including list, category, ranking, quantity and grid, along with free text boxes to allow respondents to expand on their responses (see
Structured telephone interviews were conducted with two groups of patients at the host Trust who were registered on the Programme on 31/12/2016 but not actively using the portal: those who had chosen not to register to use the patient portal; and those who had registered to use the portal but had never logged in or had logged in only once. Questions to these groups included reasons for non-use, awareness of support and general computer usage. Patients considered too unwell by the clinical team to be approached were excluded. Trained interviewers conducted the interviews, which took approximately five minutes each and followed a structure template. The interviews were conducted between 24 August 2017 and 19 September 2017.
The study met the UK NHS Health Research Authority and University Hospital Southampton NHS Foundation Trust's (UHSFT) Service Evaluation Guideline criteria for classification as service evaluation. University Hospital Southampton NHS Foundation Trust's (UHSFT) Service Evaluation Guideline does not require approval from a research ethics committee. Survey participants were informed by email and at the start of the survey about the purpose of the study and that responses would be anonymous. Consent to participate was assumed by survey completion. Interview participants were given information about the purpose and voluntary nature of the interviews by text, with an option to opt out. At the start of an interview, the interviewers explained the same information, and asked for verbal consent to take part. Approval was sought from each NHS Trust via the lead clinician, using processes required by each site where the evaluation took place. Data were managed in line with information governance and data collection policies.
Utilisation data and web page views were summarised descriptively, using Microsoft Excel.
The results of the survey were summarised descriptively using Microsoft Excel. Free text responses were coded by a researcher experienced in qualitative coding, using the NVivo software (
Results of the structured telephone interviews with non-users were summarised descriptively in Microsoft Excel.
Sixty percent (1,556/2,599) of those who were eligible chose to register to use the portal (see
Portal registration and use.
Site | Total men registered on Programme |
Registered to use the portal | Not registered to use the portal |
||
---|---|---|---|---|---|
Total |
Logged in during 6 months to 31/08/17 |
Logged in during 12 months to 31/08/17 |
|||
481 (18.5) | 332 (69.0) | 133 (40.1) | 194 (58.4) | 149 (31.0) | |
905 (34.8) | 421 (46.5) | 182 (43.2) | 225 (53.4) | 484 (53.5) | |
257 (9.9) | 187 (72.8) | 85 (45.5) | 117 (62.6) | 70 (27.2) | |
691 (26.6) | 505 (73.1) | 126 (25.0) | 199 (39.4) | 186 (26.9) | |
265 (10.2) | 111 (41.9) | 49 (44.1) | 75 (67.6) | 154 (58.1) | |
Frequency of Portal use.
Responses |
|
---|---|
Less than once every three months | 88 (17.0) |
Once every three months | 135 (26.1) |
Once a month | 41 (7.9) |
Two to three times a month | 15 (2.9) |
Once a week or more | 9 (1.7) |
Only when I am expecting a test result | 213 (41.1) |
Missing | 17 (3.3) |
Portal registration by age.
There was a weaker, but still significant, correlation (
User log in over prior 6- and 12-month period.
Portal web page views over a 6 month period.
Portal Main Patient Functions Page Views (1 March to 31 August 2017) | ||
---|---|---|
PSA Results Tracker | 4,275 | 46.7 |
Messaging | 2,245 | 24.5 |
Patient Information | 1,329 | 14.5 |
Health MOT | 1,305 | 14.3 |
A response rate of 49% (
The portal's key functions (finding patient information, messaging the clinical team, checking PSA results and completing a Health MOT) were each rated as very easy or easy to use by over 70% of users (
Portal ease of use.
Very difficult |
Difficult |
Neither difficult or easy |
Easy |
Very easy |
|
---|---|---|---|---|---|
Use of the portal in general | 19 (3.9) | 23 (4.8) | 94 (19.5) | 166 (34.4) | 181 (37.5) |
Register to use the IT Service | 20 (4.3) | 48 (10.4) | 86 (18.6) | 168 (36.3) | 141 (30.5) |
Log in | 28 (6.1) | 36 (7.8) | 76 (16.5) | 145 (31.5) | 176 (38.2) |
Change your password | 18 (7.1) | 19 (7.5) | 60 (23.8) | 75 (29.8) | 80 (31.7) |
Find patient information | 17 (3.7) | 32 (6.9) | 85 (18.4) | 162 (35.0) | 167 (36.1) |
Message your clinical team | 18 (4.5) | 16 (4.0) | 60 (15.1) | 140 (35.3) | 163 (41.1) |
Check your PSA results | 25 (5.2) | 11 (2.3) | 35 (7.3) | 162 (33.8) | 247 (51.5) |
Complete a Health MOT | 21 (5.6) | 20 (5.3) | 58 (15.5) | 135 (36.0) | 141 (37.6) |
The portal functions were rated by 78%–91% of respondents as helpful or very helpful in managing their condition (
Ways the portal helped users their condition.
Aspects of portal that helped users manage their condition.
Not helpful |
Somewhat helpful |
Helpful |
Very helpful |
|
---|---|---|---|---|
Patient information | 30 (7.7) | 57 (14.6) | 163 (41.7) | 141 (36.1) |
Messages to or from your clinical team | 25 (7.2) | 21 (6.1) | 129 (37.3) | 171 (49.4) |
PSA results | 19 (4.3) | 19 (4.3) | 116 (26.2) | 288 (65.2) |
Health MOT | 31 (9.1) | 35 (10.3) | 120 (35.2) | 155 (45.5) |
Open ended responses to the survey gave additional detail about the value of the portal for users. A total of 963 responses were received from 518 participants. Respondents reported the PSA result and tracking function to be quick, convenient, saving on time travelling to the hospital, providing peace of mind and reducing stress and anxiety, and the clinician messaging function to be quick, easy and providing peace of mind.
“
Open-ended responses also shed light on problems men were having with the portal, including difficulties with the log-in process, problems with passwords and access, difficulties of compatibility with some operating systems and lack of support for use. A number of men indicated that their infrequent use of the portal compounded issues of ease of use and navigation.
A small number of respondents mentioned the impersonal nature of the portal, a preference for face-to-face clinical contact, and a lack of appropriateness of an IT solution for patients with low IT literacy. Conversely, a few respondents mentioned a desire for the portal to expand to include comprehensive medical reports and links/incorporation of other clinical issues.
Telephone interviews were conducted with 38 men who chose not to register to use the portal, and with 10 men who had registered but then did not used the portal (
The reasons given by men who had registered but not used the portal (
Among those who had chosen not to register with the portal at all (
IT and digital solutions have a major part to play in the remote medical management of patients and in the delivery of personalised care (
This study considered the acceptability and usability of a patient portal which is part of a cancer remote monitoring and supported self-management follow-up care pathway. The portal was a feasible option and was acceptable to the majority of men registered on the Programme. Most of those who opted to use it found the functions easy to use and valuable in management of their prostate cancer follow-up care, through access to test results, easy contact with their clinical team and access to information. There was a number, however, who found practicalities such as registering, logging in and changing passwords to be difficult, which highlights a danger that people can be lost to the system early on without support for these issues. These findings add to a growing body of evidence regarding the feasibility, acceptability and usefulness of patient portals among people living with and beyond cancer (
There was a low frequency of use, with users often only logging in to view test results, and low use of other functions such as information pages. Low use in this study is likely to be partly related to these patients being in their post treatment follow up phase and thus moving on from cancer. However, this type of usage has been found across other studies (
There was a large minority of men offered access who chose not to use the portal. The study gave some insights into reasons for non-use, with lack of IT experience being commonly cited. Other studies have similarly reported low uptake (
There are concerns that digital solutions may contribute to health inequalities (
Conceived as a pragmatic evaluation, the focus of the work reported here was on satisfaction, reach and acceptability of the portal in order to inform its development. While this evaluation did not relate patient outcomes to portal use, an evaluation of the impact on patient reported outcomes of the whole follow-up care pathway, of which the portal is a part, has shown this type of follow-up care to be broadly comparable to traditional clinic based care in terms of patient reported outcomes and costs (
The lessons learned from this study make an important contribution to our understanding of models of virtual cancer follow-up care, indicating acceptability and usability of a digital portal as part of this model. It also contributes to the wider literature on portal use for the management of long-term conditions generally. It has contributed to understanding reasons for non-use which, if addressed, could help to minimise potential inequalities in care. The study indicates a need for services to be flexible in their expectations of patient uptake and use of a portal and its varied functions. While many of the lessons learned are likely to be relevant to such services for other types of cancer patients, and for other long-term conditions, there is a need for clinician and user input into design for each specific group of patients.
The datasets presented in this article are not readily available because
The study met the UK NHS Health Research Authority and University Hospital Southampton NHS Foundation Trust's (UHSFT) Service Evaluation Guideline criteria for classification as service evaluation. University Hospital Southampton NHS Foundation Trust's (UHSFT) Service Evaluation Guideline does not require approval from a research ethics committee. Survey participants were informed by email and at the start of the survey about the purpose of the study and that responses would be anonymous. Consent to participate was assumed by survey completion. Interview participants were given information about the purpose and voluntary nature of the interviews by text, with an option to opt out. At the start of an interview, the interviewers explained the same information, and asked for verbal consent to take part. Approval was sought from each NHS Trust via the lead clinician, using processes required by each site where the evaluation took place. Data were managed in line with information governance and data collection policies.
DO’C refined the evaluation design, and acquired, analysed and interpreted data, and wrote the evaluation and journal manuscript drafts. JF contributed to study design, interpretation of data and critically reviewed, revised and wrote the evaluation and journal manuscript drafts. HB conceptualised and designed the evaluation, interpreted the data and critically reviewed and revised the journal manuscript drafts. KH conceptualised and designed the evaluation, acquired data and critically reviewed and revised the journal manuscript drafts. JW conceptualised and designed the evaluation, and critically reviewed and revised the journal manuscript drafts. CF contributed to study design, critically reviewed and revised the journal manuscript drafts. AR conceptualised and designed the evaluation, interpreted the data and critically reviewed and revised the journal manuscript drafts. All authors contributed to the article and approved the submitted version.
This work was funded by the Movember Foundation, in partnership with Prostate Cancer UK, as part of the TrueNTH programme, grant number 250–25/30/40. The views expressed in this article are those of the author(s) and not necessarily those of the Movember Foundation, Prostate Cancer UK, or the TrueNTH programme. Alison Richardson is the Director of NIHR ARC Wessex and this study is supported by the National Institute for Health and Care Research ARC Wessex. The views expressed in this publication are those of the author(s) and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.
The authors would like to thank staff at the Trusts involved in the development and implementation of the Programme, in particular Jon McFarlane and Miranda Benney who acted as clinical advisers. In addition, the University Hospital Southampton NHS Foundation Trust (UHS NHSFT) Informatics team for their work in building and supporting the online survey, and switchboard staff for support with telephone surveys. The authors are grateful to the two patients who piloted the online survey and all those who participated in the evaluation.
At the time the research was conducted, DO’C worked as an independent consultant and received payment from University Hospital Southampton NHS Foundation Trust to undertake the evaluation. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
The Supplementary Material for this article can be found online at: