The feasibility, acceptability, and usability of telehealth visits

Background Telemedicine is now common practice for many fields of medicine, but questions remain as to whether telemedicine will continue as an important patient care modality once COVID-19 becomes endemic. We explored provider and patients’ perspectives on telemedicine implementation. Methods Physicians from three specialties within the Department of Medicine of a single institution were electronically surveyed regarding their perceptions of satisfaction, benefits, and challenges of video visits, as well as the quality of interactions with patients. Patients were surveyed via telephone by the Survey Research Group at Cornell about participation in video visits, challenges encountered, perceived benefits, preferences for care, and overall satisfaction. Results Providers reported an overwhelmingly positive experience with video visits, with the vast majority agreeing that they were comfortable with the modality (98%) and that it was easy to interact with patients (92%). Most providers (72%) wanted to have more telemedicine encounters in the future. Key factors interfering with successful telemedicine encounters were technical challenges and insufficient technical support. Overall, patients also perceived video visits very positively regarding ease of communication and care received and had few privacy concerns. Some (10%–15%) patients expressed interest in receiving more technical support and training. There was a gradient of satisfaction with telemedicine across specialties with patients receiving weight management reporting more favorable responses while patients with lymphoma expressed more mixed responses. Conclusion Both providers and patients found telemedicine to be an acceptable and useful modality to provide or receive medical care. The principal barrier to successful encounters was technical challenges.


Introduction
The COVID-19 pandemic dramatically changed the administration of patient care as seen in the rising use of telemedicine and other online health services (1,2). The virtual modality reduced face-to-face patient interactions to help mitigate the spread of the SARS-CoV-2 virus. Telemedicine is currently common practice for many fields of medicine, but questions remain as to whether telemedicine will continue as an important patient care modality once COVID-19 becomes endemic (3,4). Important issues to consider include understanding barriers and facilitators to the continued successful implementation of telemedicine (5)(6)(7); and whether providers' and patients' attitudes support continued use of telemedicine, and in which contexts (3,4,8,9).
Differences in medical care paradigms suggest that some specialties may be better suited to the use of telemedicine than others (1,4,(8)(9)(10)(11). For example, chiropractic therapy is less suited for telemedicine due to the hands-on nature of the practice, but psychiatric evaluations can be readily performed through a video visit. Even within some fields, certain visits are more amenable to telemedicine; for example, surgery must happen in person, but surgical follow up can be successfully achieved by a video visit. Some research studies have assessed providers' attitudes toward telemedicine and perceptions of efficacy of telemedicine patient visits (2,12,13) or patients (2,12,13), but there are still existing gaps in knowledge in this area, including information regarding which types of visits are best suited to the video modality.
We performed a survey-based study from October 2020 to April 2021 of three internal medicine subspecialties (Endocrinology, Gastroenterology, and Hematology/Medical Oncology) at an academic medical center in New York to explore the perceptions of video visits and challenges to implementation. A unique aspect of our study is that we obtained the perspective of both providers and patients. We also collected an in-depth exploration of the specific challenges experienced during the use of telemedicine. Additionally, we assessed satisfaction with telemedicine visits for both the providers and the patients, gathering data across various types of visits and specialties. Taken together, our data inform future uses of telemedicine as well as areas of improvement to optimize the use of this modality for healthcare delivery.

Methods
An anonymous Redcap survey was emailed to physicians from three specialties within the Department of Medicine at Weill Cornell Medicine: Gastroenterology, Endocrinology, and Hematology/ Oncology. The survey contained 21 items related to perceptions of satisfaction with video/telehealth visits, benefits of video visits, challenges with video visits, and quality of interactions with patients. The responses for 13 of the items were rated on a 5-point Likert scale ranging from strongly agree to strongly disagree. Four of the items were open-ended including perceptions of benefits and challenges to video visits; patient populations experiencing challenges engaging in video visits; and types of patient interactions for which video visits were appropriate.
A patient survey was also administered to patients of the physicians. The patient survey was administered via telephone by the Survey Research Group at Cornell University. Patients were identified by the providers in each of the subspecialties. Initially they were sent a letter from their provider informing them about the study and that they were going to be contacted by the Survey Research Institute. The patients were contacted up to three times via telephone. All patient responses were de-identified. The patient survey contained 25 items that were related to participation in video visits with their provider and addressed challenges encountered during the visits, perceived benefits of the video visits, preferences for care interactions, and overall satisfaction with the video visits. Responses to the items were rated on a 5-point Likert scale ranging from strongly agree to strongly disagree.

Analysis
Responses to the provider and patient survey data were initially summarized using descriptive statistics such as percentages, means and standard deviations (where appropriate, e.g., age). Group differences (e.g., provider specialty, patient age groups) were analyzed using a between subjects ANOVA. For these analyses we used the entire scale range from strongly agree to strongly disagree but for clarity of presentation, we collapsed these categories into agree, neither agree nor disagree, and disagree and presented % agree in the tables, with the exception of Table 1.
The provider qualitative data for each of the four opened queries was analyzed by independent raters. Initially the data for each question were coded for themes. Then the two independent raters coded the data into the thematic categories. The inter-rater reliability was high (Κ = 0.7). Cases of disagreement among the raters was adjudicated by a third party.
The majority of providers agreed that they were comfortable with the video visit modality (98%) and that it was easy to interact with patients (92%; see Table 2). More than 85% believed that they were receiving accurate information from patients and that the patients understood the information that was being conveyed. Most providers (72%) wanted to have more telemedicine encounters in the future and 88% found the overall experience positive. However only 67% of the providers felt confident that most patients were able to use the videoconferencing devices. Examination of the qualitative data regarding the benefits of video visits indicated that the 55% of the providers felt that cost savings and convenience were major benefits associated with video visits and 43% felt that video visits provided broader access to patients as well as enhanced the patient's ability to access care. In addition, 25% of the providers indicated that video visits allowed for social distancing and kept patients safe (see Table 3). The following are examples of noted benefits gleaned from the qualitative data.
"The video visits keep patients safe during the pandemic and still engaged in care, we can also reach patients from a further radius. " "Video visits are less time consuming and expensive for patients. "

"Video visits allow patients to be seen in between yearly in person visits. "
When exploring the issues perceived as challenges to successful video visits 80% of the providers indicated inadequate technical support and close to 2/3 indicated that they frequently encountered technical challenges (see Table 4). The most common types of technical challenges included the inefficient or challenging design of the video visit system (49%), poor internet connectivity (32%) and poor audio/visual quality (30%). The following quotes from responses to the qualitative questions demonstrate examples of perceptions of challenges associated with the video visits. With respect to provider perceived types of patients for whom video visits were most challenging, 86% of the providers indicated older patients and 37% indicated patients that lacked technology skills or internet/system access. Eighteen percent of the providers found that limitations in lab testing and the ability to perform a physical exam constituted a major limitation to the value of video visits. For example, one of the providers indicated that "labs and physical exams are needed for sicker patients and that is a limitation to telehealth visits. " A similar percentage felt that video visits took more time than an The limitations in laboratory testing resulted in a major limitation to the value of my video visits.
The limitations in conducting a physical examination resulted in a major limitation in the value of my video visits. 9% (2) 20% (4) 50% (2) I think video visits take more time than in person visits.** 9% (2) 30% (6)  When the gastroenterologists were queried about the type of patients that would be good candidates for telemedicine visits 92% thought they could perform adequate health maintenance for IBD and chronic liver patients, 86% thought they would be able to monitor for worsening or progressive disease and 67% felt they could identify patients who needed to schedule infusions and improve adherence to scheduled therapy.
Some of the endocrinologists identified the type of patient interactions most suitable for telehealth with the following responses.

Results of patient survey
A total of 240 patients including 68% white, 12% African American, 9% Hispanic, and 6% that identified as mixed race or other were surveyed. The sample ranged in age from 19 to 94 years (M = 57.5, SD = 17.4). Forty-two percent of the sample were 65 years of age or older. The majority (63%) of the sample was female and fairly well-educated, 78% had a college degree or beyond.
Given that the providers indicated that older patients experienced the most challenges with video visits, we categorized the sample into three age groups (<46, 46-64, and >65 years) to examine potential age differences in technology experience. Thirty percent of the sample was <46 years. of age, 28% were between the ages of 46-64, and 42% were aged 65 and older. Overall, approximately two-thirds had experience using all modalities of technology including computer, mobile devices, and the internet. Patients in the older age group reported that they had significantly less computer experience [F(2, 226) = 6.5, p < 0.002] and mobile device experience [F(2,226) = 7.2, p < 0.001] than the other age groups. Not surprisingly, patients under the age of 45 reported significantly more internet experience than patients in the other age groups [F(2,226) = 3.9, p < 0.02].
Of the 240 patients surveyed, 90% had participated in video visits. The remaining 10% may have had either audio calls or failed televisits. Lack of internet access and preference for face-to-face visits were main reasons cited for not participating in a video visit. Overall, the patients perceived video visits very positively and indicated that it was easy to communicate with their providers, were satisfied with the care received, and most had no privacy concerns (see Table 1). Approximately 10%-15% of patients expressed interest in receiving more technical support and training. There was difference in the patient's age among the 3 specialties [F(2,232) = 6.8, p = 0.001]. The endocrine specialty had younger patients than either gastroenterology (p = 0.001) or lymphoma (p = 0.002).  Patients from the different specialties were asked their views on the role of video visits in their medical care. Approximately 76%-88% of gastroenterology patients felt that telemedicine was good for follow-up visits, discussing lab results and monitoring, but only 46% thought that it was sufficient to accomplish appointment goals and needs. The majority of the endocrine patients found video visits to be a good way to discuss blood glucose control, lab results, and thyroid, osteoporosis, and other endocrine disorders. Over 75% of endocrine patients were satisfied that their diabetes related questions were answered during their video visits and felt that this option would help them to better maintain more regular follow-ups. Twenty-two percent of the endocrine patients, however, did not perceive that telemedicine is a good modality to start new medication. The majority (85%) of weight management patients endorsed the utility of video visits for lifestyle counseling (including diet and exercise), discussing weight management, the adjustment of medications, and answering questions related to weight management. Approximately 50% of hematology patients agreed that a video visit was good for monitoring treatment and for having a follow-up after completion of treatment but only 25% felt that it was sufficient to accomplish their appointment treatment goals.

Discussion
This study assessed perceptions of telemedicine visits from both provider and patient perspectives in three internal medicine subspecialities. The results show that providers and patients that participated in videoconferencing health services had an overall positive experience. Providers and patients rated video visits very highly for their ease of communication, and ability to fulfill patient needs and appointment goals especially for health management tasks such as reviewing labs, monitoring, and follow-up visits. These types of activities appear to be the most suited for via telemedicine visits and the ones that will contribute to the sustainability of telehealth services. Overall, most patients reported no concerns related to privacy. Furthermore, the most providers surveyed indicated that they would like telemedicine integrated into their practice in the future, despite the return of safe in person, face to face care options.
Optimal application of the telemedicine modality may depend on factors intrinsic to the patient and their current needs. These results agree with previous studies investigating the feasibility and acceptability of telemedicine across subspecialities (1,2,4,7,8,10,(12)(13)(14)(15)(16). Previous studies have investigated the variability of acceptability of telemedicine between subspecialties of medicine with one reporting that 84.4% (n = 129) and 82.9% (n = 94) of internal medicine patients and provider, respectively, said they enjoyed televisits as compared to 94% (n = 94) and 64% (n = 25) of patients and providers, respectively, in family medicine (13). An analysis of telehealth satisfaction among patients undergoing cancer rehabilitation found that 94.8% (n = 76) agreed (with responses such as "quite a bit" or "very much") that tele-visits were a positive experience as did 63.1% (n = 155) of providers. About 84% of the of providers reported that the main patient issues of the visit were adequately addressed during the visit (12). On the other hand, patients undergoing cancer diagnosis and treatment expressed that they did not want to be told bad news over a video/phone visit, despite having an otherwise positive response to telehealth (15). Additionally, a study examining clinician satisfaction found that at least 25% (n = 112) of clinicians would like to see tele-visits implemented into their practice in the future (14).
We observed a gradient of patient satisfaction among the internal medicine subspecialities, with weight management patients indicating more positive responses while lymphoma patients relaying more mixed experiences with telehealth services. Our findings are consistent with other centers that have reported high levels of patient satisfaction with the use of telemedicine for weight management (17,18). GI providers expressed greater satisfaction than Endocrine providers. During the study period GI providers were still offering in-person visits for those whose conditions/treatments deemed the service necessary for in person care. In contrast, Endocrine was completely online at this time. These signals suggest that the severity and acuity of the disease, and the reason for the appointment should be taken into consideration when offering telehealth services. Additionally, these results favor a hybrid treatment model.
Our study found that tele-visits are a generally positive experience for both providers and patients. However, importantly the findings also indicated challenges associated with telehealth. The higher rate of these issues may have been intrinsic to the initial platform. In September 2020, the zoom platform was adopted and throughout the course of this study the platform was continually optimized and refined. The introduction of the technology in a rapid manner due to urgency associated with COVID-19 and the switching between platforms may have contributed to confusion and technical challenges. However, it should be noted that the providers reported that the older adult patients encountered the most challenges and as noted these patients had significantly less technology experience. Technology issues, such as lack of internet access or poor-quality video/audio (7), were the most prominent challenges for both providers and patients. This is in-line with the findings of Chang and colleagues (12) who found that a common patient barrier to use of telehealth, especially among underserved populations was lack of reliable internet as well as comfort with technology. Our findings also indicated that patients with low technical skills also experienced challenges with the telehealth visits. The providers in our sample also indicated that the video visits were particularly challenging for older patients. The older patients in our sample had significantly lower technology experience than the younger and middle-aged patients. These findings are consistent with recent data from the Pew Research Center (19) that although rates of technology adoption are increasing among older adults an age-related digit divide still exists (7). Given the increased reliance on telehealth for health management activities and that older adults tend to require more health care services, strategies need to be in place to ensure that people in all age groups have "meaningful" access to telehealth technology.
Successful implementation of telehealth for all patient populations requires telehealth systems to be designed for ease of use for patients with low technology skills and low technology efficacy. Success also requires that patients have access to the internet, appropriate technology, adequate training, and technical support. Unaddressed patient challenges with telehealth may exacerbate health inequities. Healthcare providers must also have access to easy-to-use systems, training, and technical support. In essence, telehealth systems should be designed using a user-centered design approach that involves participation in the design process from all user groups including both patients and providers (5). This study delineates both provider and patient perspectives regarding feasibility, acceptability, and utility of telemedicine across three medical specialties. The findings also provide information for the types of healthcare tasks for which telehealth visits are most suited and the patient populations for whom these visits are challenging. In addition, the findings suggest common types of challenges associated with telehealth. The study also has some limitations. The sample was primarily white with high levels of technology experience. It was also restricted to a few specialties at one academic medical center. Patient data on participation in visits was based on self-report. Further, this study was performed over the course of one time period, so it is not possible to examine how perceptions of telehealth changed over time as both providers and patients had more experience with tele-visits and perhaps the technology being used has improved. This would be an interesting follow-up study. Despite these weaknesses, the findings expand our understanding of the benefits and challenges associated with telehealth from both the patient and provider prospective. The COVID-19 pandemic sparked a rapid adoption of telehealth services. As we adjust to a new post pandemic normal, our findings suggest that telehealth will most likely remain an important and valued modality for ambulatory patient care.

Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement
The studies involving human participants were reviewed and approved by Weill Cornell IRB. The patients/participants provided their written informed consent to participate in this study.