# TELEMEDICINE IN LOW-RESOURCE SETTINGS, VOLUME 2

EDITED BY : Richard Wootton, Laurent Bonnardot and Sophie Delaigue PUBLISHED IN : Frontiers in Public Health

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-689-1 DOI 10.3389/978-2-88963-689-1

### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# TELEMEDICINE IN LOW-RESOURCE SETTINGS, VOLUME 2

Topic Editors:

Richard Wootton, University Hospital of North Norway, Norway Laurent Bonnardot, Université Paris Descartes, France Sophie Delaigue, Pan American Health Organization, United States

Telemedicine networks to support healthcare workers in resource-limited settings (often for humanitarian purposes) have evolved in a largely autonomous way. Communication between the networks has been informal and relatively limited in scope. This situation could be improved by developing a comprehensive approach to the collection and dissemination of information.

While many telemedicine networks remain experimental, some have matured to become routine services which assist health care delivery in challenging environments. For example, there are a few networks which have been in operation for over a decade, and provide store-and-forward telemedicine services to doctors in lowand middle-income countries. These networks deliver clinically useful services and improved healthcare access. However, like much of telemedicine, the formal evidence for their cost-effectiveness remains weak.

Topics of current research interest therefore include the cost-effectiveness of telemedicine in resource-limited settings. Other topics of interest concern outcomes data (and methods for gathering it) such as patient quality of life following a telemedicine episode, the knowledge-gain of healthcare staff involved in telemedicine, and staff recruitment and retention in rural areas. Finally, there is little published information about the performance of these telemedicine networks (and methods for measuring it), about how best to manage them, and about how to share resources between them.

Following the previous volume, this Research Topic will document current evidence supporting the use of telemedicine in resource-limited settings.

Citation: Wootton, R., Bonnardot, L., Delaigue, S., eds. (2020). Telemedicine in Low-Resource Settings, Volume 2. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-689-1

# Table of Contents


G. Khai Lin Huang, Gibson Pawape, Magdalene Taune, Stenard Hiasihri, Pilar Ustero, Daniel P. O'Brien, Philipp du Cros, Steve Graham, Richard Wootton and Suman S. Majumdar

*19 Experience of Supporting Telemedicine Networks With the Collegium System: First 6 Years*

Richard Wootton and Laurent Bonnardot


Richard Wootton and Barry O'Kane


Anne-Laure Messagier, Romain Blaizot, Pierre Couppié and Sophie Delaigue

*76 Improving Mental Health Care in Developing Countries Through Digital Technologies: A Mini Narrative Review of the Chilean Case*

Graciela Rojas, Vania Martínez, Pablo Martínez, Pamela Franco and Álvaro Jiménez-Molina

# Motivations and Barriers Associated With Physician Volunteerism for an International Telemedicine Organization

Erin J. Kim<sup>1</sup> , Steven Fox <sup>2</sup> , Meghan E. Moretti <sup>1</sup> , Michelle Turner <sup>1</sup> , Timothy D. Girard<sup>3</sup> and Stephen Y. Chan<sup>1</sup> \*

*<sup>1</sup> The Addis Clinic, Inc., Nashville, TN, United States, <sup>2</sup> Department of Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA, United States, <sup>3</sup> Department of Critical Care Medicine, Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States*

Introduction: The Addis Clinic uses volunteer physicians to implement an international humanitarian telemedicine program. We sought to identify motivations and barriers that may contribute to physician volunteerism in international telemedicine.

#### Edited by:

*Sophie Delaigue, Pan American Health Organization, United States*

#### Reviewed by:

*Deborah Elaine Seale, Des Moines University, United States Larry Kenith Olsen, Logan University, United States*

> \*Correspondence: *Stephen Y. Chan stephen.y.chan@gmail.com*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *02 April 2019* Accepted: *25 July 2019* Published: *13 August 2019*

#### Citation:

*Kim EJ, Fox S, Moretti ME, Turner M, Girard TD and Chan SY (2019) Motivations and Barriers Associated With Physician Volunteerism for an International Telemedicine Organization. Front. Public Health 7:224. doi: 10.3389/fpubh.2019.00224* Methods: We surveyed active and inactive volunteers working with The Addis Clinic. Descriptive statistics were used to examine closed-ended questions, while a qualitative approach identified overarching themes for open-ended questions. The Volunteer Functions Inventory framework was also applied.

Results: Among 69 active and 25 inactive volunteers, survey response rates of 74 and 72%, respectively, were attained. Volunteer cohorts exhibited comparable distributions across sex, marital status, and children. Active, as compared with inactive, participants were significantly more likely to be <40 years old (51 vs. 39%, *p* = 0.01), have prior experience with international/global health (67 vs. 39%, *p* = 0.04), and express an interest in international/global health work (82 vs. 50%, *p* = 0.008). Active volunteers were predominantly concerned with challenges regarding patient care: they more often reported the asynchronous nature of communication with frontline health workers as a significant barrier (37 vs. 6%, *p* = 0.047), and increased patient follow-up significantly drove their enthusiasm (64 vs. 35%, p = 0.05). Conversely, active volunteers were less likely to cite commitment/availability as a significant barrier for participation (33 vs. 72%, *p* = 0.002), less likely to be incentivized by opportunities to fulfill professional obligations (14 vs. 59%, *p* = 0.001), and more likely to be satisfied with the telemedicine experience (86 vs. 0%, *p* < 0.0001). Opportunities to receive remuneration or recognition did not increase the likelihood of volunteering for either cohort. Malpractice concerns were cited in a comparable minority across cohorts (20 vs. 17%).

Conclusions: Age and global health experience/interest were significant predictors of physician volunteerism. While inactive volunteers reported time commitment as a barrier, active participants were concerned with challenges regarding patient care and motivated by increased methods to connect with patients. Financial considerations and recognition

**4**

were infrequently reported as a barrier. With advances in telemedicine globally, results from this study can be used by organizations involved in international telemedicine to develop effective volunteer recruitment and retention strategies.

Keywords: telemedicine, global health, physician volunteers, international volunteerism, frontline health workers, motivations, barriers, humanitarian

### INTRODUCTION

Many remote, hard-to-reach communities in low- and middleincome countries (LMICs) suffer from a shortage of skilled health workers (1). These communities struggle with retaining and attracting a qualified health workforce, and at times, are served by a single frontline health worker (FHW) with limited clinical training (2). Patients with complex conditions and other chronic illnesses may need to travel long distances across difficult terrain to receive specialized medical care (3). Many of the FHWs in these communities are isolated and lack the type of professional support found in urban areas. Addressing workforce challenges has the potential to improve patient outcomes and build stronger health systems (4). Consequently, demand for volunteers with medical expertise has risen significantly as communities and governments lack the financial resources to recruit, train, and maintain local health workers (5, 6).

Many of the volunteers serving in these communities are healthcare professionals from developed countries seeking global health experiences for humanitarian and/or careerrelated reasons. Volunteer assignments are typically shortterm placements lasting anywhere between 2 and 3 weeks to <2 months. This form of volunteerism, while popular, raises important ethical considerations. For instance, local organizations that host international volunteers have expressed concerns about temporary volunteers competing with, or even substituting, locally trained health workers (5, 7). Additionally, the sustainability and effect of using short-term volunteers to supplement the local health workforce have been questioned (6).

With the advances in mobile technology and the number of internet users on the rise globally, there is an emerging consensus among key stakeholders that ICT4D (Information and Communications Technologies for Development) interventions have the potential to strengthen FHWs' clinical knowledge and improve patient outcomes (8–10). Organizations supporting FHWs in rural communities have used technology to train, motivate, and share current medical guidelines, as well as create a virtual community of health workers (11, 12).

Telemedicine technology is one example of an ICT4D intervention that can be used to support local health systems by giving FHWs continuous access to medical experts located remotely. Evidence suggests that digital telemedicine tools have the potential to strengthen health systems by supporting and training FHWs in geographically separate communities (8, 13). Additionally, international telemedicine organizations provide timely access to specialists, reducing the need for patients to travel long distances for specialized medical care (14–16). Brandling-Bennett et al., found that consultations via e-mail between physicians in Boston, Massachusetts with a mobile nurse in Cambodia reduced the number of patients requiring outside referrals (17).

The Addis Clinic is one such nonprofit organization that uses telemedicine to address many of the concerns highlighted by local organizations in LMICs. The Addis Clinic connects FHWs working in low resource areas with a network of specialty physician volunteers, offering a mechanism for communication of medical information and recommendations through its internet-based platform<sup>1</sup> . Namely, The Addis Clinic facilitates the exchange of medical knowledge through teleconsultations submitted by FHWs that are then triaged to the appropriate physician volunteers. Based on a "store-andforward" technology, this platform is ideal in low resource settings with limited connectivity<sup>2</sup> . It allows FHWs to initiate and store teleconsultations on their devices without internet or satellite connectivity. As soon as FHWs are within range, they can submit their saved cases to physician volunteers located in geographically separate areas of the world.

While international telemedicine organizations such as The Addis Clinic have the potential to strengthen the health workforce in remote, resource constrained communities, they rely heavily on physician volunteerism (16, 18). In many ways, telemedicine can facilitate participation of physicians who otherwise would not volunteer to serve internationally. Namely, it eliminates the barrier of traveling abroad for physician volunteers and allows them to volunteer from their location of choice. This encourages long-term volunteerism and can offer sustainable support for communities in resource constrained areas. Furthermore, it addresses the challenge of partner organizations being susceptible to gaps in the provision of health services with short-term volunteers.

Yet, the motivations and barriers of international volunteers in organizations such as The Addis Clinic are not obvious. Researchers examining healthcare professionals returning from short-term medical trips have found that most international volunteers are motivated for altruistic reasons, but self-interested motives such as the opportunity to travel also played a role (19). Furthermore, prior researchers have assessed the effect of international volunteers on local health systems, as well as perspectives from partner organizations (5, 6). While many researchers have evaluated the various experiences of international volunteerism, few have examined the motivations and barriers of healthcare professionals volunteering virtually. In this study, we sought to identify motivations and barriers that may contribute to physician volunteerism in international telemedicine operations, thus providing quantifiable data

<sup>1</sup>https://www.addisclinic.org/

<sup>2</sup>https://www.collegiumtelemedicus.org/ct/index.php

to develop effective recruitment and retention strategies in the future.

### MATERIALS AND METHODS

### Survey Instrument and Participants

An initial email invitation with a personal link to the survey was sent to active (69) and inactive (25) physician volunteers for The Addis Clinic in November of 2018. Survey responses were collected for 1 month with targeted follow-up emails sent 2 and 3 weeks after the initial invitation. Reminders about the survey were included in the physician newsletter and social media. Active volunteers were defined as physicians currently on the roster for teleconsultations and have consulted on a case within the past 12 months. Inactive volunteers were defined as physicians who have expressed interest in volunteering with The Addis Clinic, but never onboarded or have not consulted on a case within the past 12 months. More than 70% of the physician volunteers started volunteering in 2017, while fewer than 10% had been involved with the organization since its inception in 2011. The remaining 20% of volunteers onboarded between 2012 and 2017. Volunteer turnover has been low in recent years with nearly 90% of volunteers leaving the organization before 2014.

The University of Pittsburgh online survey system (Qualtrics) was used to distribute the survey. The survey consisted of 30 close-ended (e.g., multiple choice, Likert-type scale, dichotomous) and four open-ended questions. Questions were designed to assess volunteers' level of satisfaction with and perceptions regarding motivations and barriers to working with The Addis Clinic. The survey also included a demographic section to better understand background characteristics of the respondents. Survey results were organized into Demographics, Volunteer Experience/Background, Barriers to Volunteer Work, and Motivations to Volunteer Work. Findings under Volunteer Experience/Background include survey questions designed to assess participants' experience with global health and telemedicine, in general. The questions were developed in collaboration with several Addis Clinic staff members, to include points that have been identified as potential factors influencing volunteers' perceptions of their experience with The Addis Clinic and their motivations to volunteer. Prior to dissemination, the survey was piloted with two physician volunteers, as well as reviewed by a survey expert to improve the accuracy of the instrument. The survey was approved by the University of Pittsburgh Medical Center (UPMC) Institutional Review Board (PRO18060522). All participants were given the opportunity to opt-out of the survey. Participants were not required to complete the survey in its entirety.

### DATA ANALYSIS

A mixed methods approach was used for survey analysis. Descriptive statistics such as frequency analysis were used to examine closed-ended questions, while a qualitative approach was used to identify overarching themes for open-ended questions. Survey results were imported into Microsoft Excel and STATA for analysis.

Four open-ended questions gave volunteers the opportunity to enter free text. One of these questions centered around understanding motivations for volunteering for The Addis Clinic. Specifically, respondents were asked what they enjoyed most about volunteering. Only responses from active volunteers were analyzed. Survey responses were thematically analyzed using the Volunteer Functions Inventory (VFI). This approach examines six motivational functions or motives for volunteering (20). It assumes that individuals volunteer their time and skills for the same organization for different reasons. The six motivational functions include:


Three members of the research team (EK, SF, MM) reviewed the responses independently. Each reviewer assigned the most appropriate function based on his/her impression of the data. If more than one motivational function applied, reviewers had the option to identify up to three functions by prioritizing the most applicable as primary and least applicable as tertiary. Responses that did not fit any of the functions were identified as "Other." Results were validated by triangulating the reviewers' assessments and the frequency for each motivational function was tabulated.

Fisher's exact tests or chi-squared tests were used to compare survey responses between active and inactive participants. Two-sided p-values of 0.05 or less were deemed to meet statistical significance.

### RESULTS

### Demographic Characteristics

Among 69 active and 25 inactive volunteers, survey response rates of 74 and 72% were attained, respectively. More than 95% of the physician volunteers resided in the United States (U.S.), while the remaining 5% resided in Canada or the United Kingdom. Demographic characteristics for both groups are shown in **Table 1**.

Notably, in this cohort, active, as compared with inactive, volunteers were more likely to be <40 years old (51 vs. 39%, p = 0.01). Of trends that did not reach statistical significance, active participants were less likely to have practiced medicine for >10 years (34 vs. 61%), work full-time (78 vs. 94%), and be employed at an academic institution (35 vs. 50%). Additionally, other non-significant trends showed that active volunteers were more likely to view religion/faith as personally important (64 vs. 39%) and earn <\$200,000 per year (52 vs. 44%). Volunteer cohorts exhibited comparable distributions across sex, marital status, children, and weeks of vacation per year. More than half of respondents for both active and inactive volunteers had children under the age of 18 (59 vs. 67%), were married (76 vs. 78%), and



#### TABLE 1 | Continued


had 3–4 weeks of vacation per year (63 vs. 65%). Both cohorts were relatively equivalent in terms of gender distribution.

### Volunteer Experience/Background

There was a wide range of medical specialties represented among survey participants. Active volunteers listed more than 20 specialties, however, nearly half specialized in primary care specialties such as Pediatrics (12%), Internal Medicine (20%), and Family Medicine (14%). Inactive volunteers represented 12 specialties (39% of which were in primary care specialties), including Internal Medicine (22%), Pulmonology (17%), and Radiology (17%). Importantly, we found that 46% of active participants (n = 51) were in primary care specialties, which was not significantly more than 39% of the inactive (n = 18) participants (p = 0.65).

A vast majority of volunteers, both active (60%) and inactive (100%), were first introduced to The Addis Clinic through wordof-mouth from friends, current and past volunteers, colleagues, or members of the Board of Directors/staff. Active volunteers were also introduced to The Addis Clinic via internet search (17%), conference proceedings (13%), and social media (10%).

Active, as compared with inactive, participants were significantly more likely to have had prior experience with international/global health (67 vs. 39%, p = 0.04), as well as express an interest in international/global health (82 vs. 50%, p = 0.008), with several active volunteers having prior experience on short-term medical trips (**Figure 1**). Though not a significant finding, active respondents tended to have greater experience with providing telemedicine services both domestically and/or internationally (28%) (**Table 1**). The small number of inactive

volunteers that reported telemedicine experience had done so in a domestic setting only (6%).

### Barriers to Volunteer Work

Responses regarding barriers to volunteering are summarized in **Figure 2**. A comparable minority for both active and inactive volunteers cited malpractice concerns, particularly in the international setting (20 and 17%, respectively). Time commitment/availability were less often a significant barrier for active, as compared with inactive, volunteers (33 vs. 72%, p = 0.002). On the other hand, the asynchronous nature of communication with FHWs was reported as a significant barrier for active, as compared with inactive, volunteers (37 vs. 6%, p = 0.047). A consistent minority among active volunteers also cited other non-significant barriers, including questions about the quality of care provided by telemedicine (31%). A trend toward a lower percentage of inactive participants voiced the same concerns (11%). Additional concerns (37%) that were not significant, shared mainly among active volunteers ("Other"), included an inability to follow-up with patients and the uncertainties of treating lesser known diseases more commonly seen in LMICs.

Aspects of the teleconsultation process participants found challenging are summarized in **Figure 3**. Active, compared with inactive volunteers, were less likely to report language/cultural considerations (8 vs. 50%, p = 0.01), and the technology interface (4 vs. 43%, p = 0.01) as significant barriers to the teleconsultation process. Though not a significant difference, active volunteers were more concerned with aspects of patient care, with 46% citing the availability of follow-up information from FHWs and 29% questioning the availability of relevant medical/patient information from FHWs. Features of work-flow were not seen as an obstacle by a majority of either active or inactive volunteers. Less than 20% of active and inactive respondents found the frequency of back-and-forth communication with FHWs as problematic, while 4% active and 17% inactive claimed the time frame (24–48 h) required to respond to teleconsultations as difficult.

### Motivations for Volunteer Work

Motivations toward volunteering varied among active volunteers. A frequency distribution table (**Table 2**) summarizes the results of the VFI analysis.

Notably, common themes emerged throughout the responses. The ability to help patients in resource-constrained environments was most important to active volunteers, while the motivation to better understand a different culture and gain new skills was the second most common motivation:

"Sense of providing service to a patient that otherwise would not have access to a specialist."

"Helping to provide a service otherwise not available to those who need it."

"Interesting to see the cases detailed in the newsletter and learning how to care for patients in settings with limited resources."

In some cases, the reasons for volunteering were more complex with more than one motivational function attributed to a participant's response. Underlying secondary and tertiary themes emerged for a handful of respondents where, for example, both the "values" and "understanding" functions were applicable:

"Learning about patients who live in the country I once lived and being able to get the chance to contribute."

Active and inactive volunteers differed in their views of incentives for volunteering (**Figure 4**). A higher percentage

17%, respectively).

of active, compared to inactive, volunteers felt that more patient follow-up would significantly increase their enthusiasm for volunteering (64 vs. 35%, p = 0.05). Though not a significant finding, the use of real-time technology for direct encounters was viewed as a potential incentive among both active and inactive volunteers (45 vs. 33%). Furthermore, active, as compared with inactive volunteers, were significantly less likely to be incentivized if given opportunities to fulfill professional obligations and receive continuing education credit (14 vs. 59%, p = 0.001). Interestingly, opportunities to receive remuneration

### TABLE 2 | Motivational functions for volunteering.


or recognition were unlikely to incentivize volunteering for either cohort.

A large majority of active volunteers were satisfied with The Addis Clinic teleconsultation process (p < 0.0001), with 86% citing satisfaction in general and 78% citing satisfaction with the cases submitted by FHWs (**Table 3**). Inactive volunteers; however, were indifferent to the teleconsultation process, with 86% claiming neither satisfied nor dissatisfied and 14% reporting dissatisfied. Similarly, 83% of inactive volunteers were neither satisfied nor dissatisfied with the cases submitted by FHWs.

### DISCUSSION

### Summary of Findings

By surveying active and inactive physician volunteers at a single humanitarian telemedicine organization (The Addis Clinic), our results reveal several critical factors for motivating and maintaining physician volunteerism. Younger age and having global health experience/interest were significant predictors of consistent physician volunteerism. While inactive volunteers reported time commitment/availability as a significant barrier to volunteer work, active participants were concerned with challenges regarding patient care and information exchange. Additionally, active volunteers were motivated by increased methods to connect with patients. Surprisingly, financial considerations and recognition were infrequently reported as a barrier to volunteer work. Together, these results serve as a valuable guide toward optimizing the strategies for recruiting and maintaining engaged physician volunteers in international telemedicine.

### Barriers

Our data indicated that specific demographic features were associated with a greater willingness to volunteer. These demographics, at least in part, may help to explain the fact that time commitment/availability was ranked more frequently as a predominant barrier to volunteering by inactive respondents. That is, more than half of the inactive volunteers were >40 years old and at the peak of their professional careers. Additionally, 67% of inactive respondents reported having school-aged children. Thus, it is possible that challenges of work-life balance dissuaded inactive participants from committing to volunteering further or at all. Furthermore, it is reasonable to extrapolate that coupling their limited time availability with minimal experience and interest in global health and telemedicine would create challenges in maintaining longitudinal interest in volunteering. Our study also found that language and cultural considerations created a significant barrier for inactive volunteers, similar to reports from other international volunteers participating on short-term assignments (6, 19). These insights suggest that humanitarian telemedicine organizations such as The Addis Clinic could preferentially recruit physicians at specific life stages and fitting certain demographic profiles that are more amenable to volunteerism. Future surveys of a larger cohort of inactive volunteers over a broader time window will also be valuable to ensure consistency of perspectives across multiple cycles of participants.

Active participants; however, described a broader scope of barriers with telemedicine that thematically centered on challenges of patient care and information exchange (**Figure 2**). Furthermore, over one-third of all active volunteers found the asynchronous communication with FHWs as a significant barrier. This may be linked to the result that nearly half of active respondents were more likely to volunteer if they were able to directly interact with patients via real-time technology. Additionally, greater follow-up information about the patient significantly increased enthusiasm among active volunteers, especially for complex cases. Active participants desired more information on the local resources available to ensure the correct diagnosis and management plan was in place. These principles may also reflect a difference in motivation for physicians volunteering longitudinally as compared with more short-term programs where retention of physician motivation may be less important.

Furthermore, shared themes may underlie the breadth of barriers described by active participants, including notions that physician volunteers from developed countries often are unaccustomed to the pace and workflow processes common in LMICs (6). Physician volunteers may find the store-and-forward nature of asynchronous telemedicine cumbersome because of the inability to receive immediate feedback. Moreover, managing the continuum of care, including a slower process of gathering follow-up information, may be an unfamiliar challenge in resource-rich health systems.

In the future, as telemedicine programs grow in developed countries, it is likely that some of these challenges will become less foreign to physicians who volunteer virtually<sup>3</sup> . Additionally, efforts from The Addis Clinic are underway to initiate a more structured physician education platform unique to practicing telemedicine in resource-poor environments. Such a program could be effective in bridging the cultural and medical disconnect among active volunteers and FHWs working in remote, hard-to-reach areas.

### Motivations

It is notable that the potential for greater patient contact in general increased enthusiasm among active volunteers. This may

<sup>3</sup>https://www.mordorintelligence.com/industry-reports/global-telemedicinemarket-industry

#### TABLE 3 | Satisfaction with the teleconsultation process.


stem from the fact that contemporary U.S.-trained physicians in general have been vocal about a desire to establish direct patient-physician relationships, despite geographical or other logistical barriers (21). This sentiment may also encapsulate specific root causes of U.S. physician dissatisfaction and burnout, where time to interact with patients continues to diminish in lieu of increased demands for paperwork, and insurance documentation by physicians on electronic health record systems (21). These notions also correspond with the results from the VFI analysis (**Table 2**) that demonstrated the "values" function as the primary motivation for volunteering among active respondents. Additionally, the desire to help and concern for others could be augmented by the fact that the majority of active volunteers are foreign-born physicians. With the rise in healthcare professionals seeking global health experiences, it may be advantageous for organizations to target foreign-born physicians as volunteers. Certainly, video and real-time technologies as methods for more direct patient contact are already available for international telemedicine endeavors (22). However, real-time video-based appointments across global time zones would require a much more structured and often inflexible demand on physician volunteers' schedules—a notion that is at odds with the barrier of time commitment/availability. Based on these results, innovative ideas to reconcile these seemingly opposing motivations and barriers to volunteering are worth exploring.

Interestingly, a number of inactive participants stated that they were more likely to volunteer if receiving professional incentives such as continuing education credit. Since most of the inactive volunteers were at the peak of their careers and dealing with limited time availability, it could be reasonable to infer that inactive respondents may be more amenable to volunteer if it would also maximize their opportunities to fulfill additional personal and/or career-related obligations. However, our results also revealed that such fulfillment would not necessarily be embodied by personal financial gain. In fact, monetary compensation did not serve as a key motivation for volunteering in a majority of both active and inactive volunteers, consistent with prior researchers finding humanitarian and altruistic rationale as a central driver for international volunteers (23, 24). Thus, guided by these findings, it may behoove The Addis Clinic and other like-minded organizations to partner with physician employers to provide innovative mechanisms facilitating physician promotion and/or offering time-saving measures for physicians in other aspects of the work day, if choosing to volunteer in international telemedicine service. Offering malpractice coverage

for international telemedicine may also be effective in mitigating the concerns of the ∼20% of participants in both active and inactive cohorts.

### Study Limitations

All participants were volunteers for a single telemedicine organization, which incurs bias and may limit generalizability to other organizations. Additionally, there was a relatively small pool of inactive physician volunteers, as volunteer retention for The Addis Clinic remains high, with fewer than 20% of volunteers leaving the organization since its founding. Furthermore, many of the inactive volunteers have not been recently involved with The Addis Clinic. Consequently, some of their perceived barriers to the teleconsultation process may not be as relevant in the current era of the program, given the significant organizational changes since 2016. For instance, nearly half of the inactive respondents noted the telemedicine technology as a major barrier to volunteering; however, only 4% of active respondents agreed. It is important to note that The Addis Clinic switched to a separate telemedicine platform in 2017. To address this limitation, future studies could be structured to solicit feedback from volunteers prospectively as they leave the organization. Additionally, given the small sample size for both the active and inactive cohorts, more detailed analyses regarding the specific roles of various demographic characteristics on volunteerism were challenging. To better understand whether demographics were factors in the responses of volunteers, larger sample sizes will be needed to maximize statistical power. Furthermore, the VFI analysis found that the motivations for volunteering are complex, with more than one motivational function attributed to a participant's response. The scope of this study provided an initial interpretation of the qualitative data, but additional studies are warranted to expand on the findings. Finally, this study was not designed to compare accuracy and efficacy of physician volunteer diagnoses and recommendations, which could play a central role in reinforcing vs. reducing longterm physician motivations. Given these emerging concepts revealed in this initial study, a more in-depth analysis across multiple telemedicine organizations would be valuable to better understand the underlying thought processes for physicians choosing to volunteer virtually.

### REFERENCES


### CONCLUSION

By harnessing the power of telemedicine technology coupled with physician volunteerism, international telemedicine programs have the potential to strengthen the local health workforce and improve health outcomes in remote communities. Guided by the principles revealed in this proof-of-concept study, it will be important for organizations to develop volunteer programs that are mutually beneficial to physician volunteers and partner organizations to ensure long-term interest. More research is needed to better understand the sustainability and potential for large-scale implementation of organizations using virtual volunteers. Organizations may consider applying these findings to develop focused volunteer recruitment and retention strategies.

### DATA AVAILABILITY

The datasets generated for this study are available on request to the corresponding author.

### AUTHOR CONTRIBUTIONS

EK, SF, MT, TG, and SC conceived the study, designed the survey and analysis, and wrote the manuscript. EK and TG conducted the quantitative analysis. EK, SF, and MM performed the Volunteer Functions Inventory analysis. All authors participated in interpreting the results and revising the manuscript.

### FUNDING

This work was supported by private philanthropic support for The Addis Clinic.

### ACKNOWLEDGMENTS

We acknowledge The Addis Clinic Board of Directors for strategic advice, and Richard Wootton and Collegium Telemedicus for support with the telemedicine platform. Additionally, we acknowledge Rickie Brawer, PhD, MPH (Thomas Jefferson University Hospitals), as a consultant in the development of the survey.


**Conflict of Interest Statement:** SC currently is the Chair of the Board of Directors of The Addis Clinic, Inc., and holds a faculty position at the University of Pittsburgh School of Medicine and UPMC. SC has served as a consultant for Zogenix, Aerpio, Vivus, and United Therapeutics. SC holds research grants for Pfizer and Actelion. SC holds positions as director, officer, and shareholder of Numa Therapeutics. SC has served as inventor of patent applications regarding new therapies in pulmonary hypertension.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Kim, Fox, Moretti, Turner, Girard and Chan. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Telemedicine in Resource-Limited Settings to Optimize Care for Multidrug-Resistant Tuberculosis

G. Khai Lin Huang<sup>1</sup> \*, Gibson Pawape<sup>2</sup> , Magdalene Taune<sup>2</sup> , Stenard Hiasihri <sup>1</sup> , Pilar Ustero<sup>1</sup> , Daniel P. O'Brien1,3, Philipp du Cros <sup>1</sup> , Steve Graham<sup>1</sup> , Richard Wootton4,5 and Suman S. Majumdar <sup>1</sup>

*<sup>1</sup> Tuberculosis Elimination and Implementation Science Group, Burnet Institute, Melbourne, VIC, Australia, <sup>2</sup> Daru General Hospital, National Department of Health, Daru, Papua New Guinea, <sup>3</sup> Department of Infectious Diseases, Barwon Health, Geelong, VIC, Australia, <sup>4</sup> Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, Tromsø, Norway, <sup>5</sup> Faculty of Health Sciences, University of Tromsø, Tromsø, Norway*

The emergence and spread of multidrug-resistant tuberculosis (MDR-TB) poses a major threat to the global targets for TB control. In recent years, an evolving science and evidence base for MDR-TB has led to much needed changes in international guidelines promoting the use of newer TB drugs and regimens for MDR-TB, however, there remains a significant implementation gap. Due to the complexity of treating MDR-TB, management of cases is often supported by an expert multidisciplinary team, or clinical expert group. This service is often centralized, and may be delivered through a telemedicine platform. We have implemented a Web-based "store-and-forward" telemedicine service to optimize MDR-TB patient care in Daru, a remote and resource limited setting in Papua New Guinea (PNG). From April 2016 to February 2019, 237 cases were discussed using the service. This encompassed diagnostic (presumptive) and treatment cases, and more recently, support to the scale up of preventative therapy for latent TB infection. There were 75 cases in which the use of Bedaquiline was discussed or mentioned, with a high frequency of discussions occurring in the initial period (26 cases in the first 12 months), which has appeared to decrease as clinicians gained familiarity with use of the drug (15 cases in the last 12 months). This service has supported high quality clinical care and fostered collaboration between clinicians and technical experts in a shared learning environment.

Keywords: tuberculosis, telemedicine, multidrug-resistant, resource-limited, clinical expert group, consilium, digital health

### INTRODUCTION

Tuberculosis (TB) is one of the top 10 causes of death globally and is the leading cause of death from a single infectious agent. It is a disease that disproportionally affects low- and middle- income countries, where it remains a major public health issue (1). The ambitious goal of the World Health organization's (WHO) End TB Strategy (2015–2035) is to eliminate TB as a public health threat, targeting an incidence rate below 10 cases per 100,000 population per year. The three pillars of the strategy are: integrated patient-centered TB care and prevention, bold policies and supportive systems, and intensified research and innovation (2).

The emergence and spread of multidrug-resistant tuberculosis (MDR-TB, defined as resistance to the most effective first-line anti-TB medications: rifampicin and isoniazid)

#### Edited by:

*Harshad Thakur, Tata Institute of Social Sciences, India*

### Reviewed by:

*Charles Doarn, University of Cincinnati, United States Simon Tiberi, Queen Mary University of London, United Kingdom*

> \*Correspondence: *G. Khai Lin Huang*

*khai.huang@burnet.edu.au*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *23 April 2019* Accepted: *25 July 2019* Published: *13 August 2019*

#### Citation:

*Huang GKL, Pawape G, Taune M, Hiasihri S, Ustero P, O'Brien DP, du Cros P, Graham S, Wootton R and Majumdar SS (2019) Telemedicine in Resource-Limited Settings to Optimize Care for Multidrug-Resistant Tuberculosis. Front. Public Health 7:222. doi: 10.3389/fpubh.2019.00222*

**14**

poses a risk to achieving the End TB goals. MDR-TB is a major contributor to mortality and the financial burden of the antimicrobial-resistance threat (3). MDR-TB care is complex and challenging and the burden is highest in settings where health systems have significant challenges. Globally, patient outcomes remain poor, with 55% of those enrolled completing treatment successfully in 2016 (1), although success rates can be much higher in well-resourced settings (4), and optimized programs (1). The treatment duration for MDR-TB is lengthy (9 months to 2 years), with a regimen of potentially toxic medications, making it more complex and more costly to deliver than drug-susceptible TB (1). The WHO recommends a community-based model of care for MDR-TB, which facilitates decentralization and scale-up in high-burden and low resource settings.

In recent years, an evolving science and evidence base for MDR-TB has led to much needed changes in international guidelines promoting the use of newer TB drugs and regimens for MDR-TB, with further changes anticipated in coming years (5). However, there remains a major implementation gap, with significant barriers to the knowledge-delivery ("know-do") pathway in MDR-TB care for a number of reasons (6). First, knowledge and experience with managing complicated cases and using the newer drugs such as bedaquiline and delamanid is often limited to a few sites. These sites are typically centralized specialist referral hospitals where expertise, training, clinical advice and active drug safety monitoring can be provided. Second, health workers may have limited training or capacity in settings where MDR-TB care is needed most and is often provided at decentralized sites.

Technical assistance and expertise, including clinical support, either within a country or from international partners plays an important role in optimizing MDR-TB in high burden/resource limited settings (7). Telemedicine is one such modality of providing clinical and technical assistance and can potentially have a role in closing the "know-do" gap to improve patient outcomes and quality of care.

We describe the global utility and experience with telemedicine for MDR-TB care and our own experience with the implementation of a Web-based telemedicine platform to support MDR-TB care in a remote setting of Papua New Guinea (PNG). This model has facilitated the delivery of technical assistance to clinicians in the field through a store-and-forward text-based platform suitable for the local context.

### CURRENT ROLE AND GLOBAL EXPERIENCE WITH TELEMEDICINE IN MDR-TB CARE

Telemedicine is a broad term within the domain of digital health, that encompasses a wide scope of practices, all relating to the delivery of health care at a distance (8). Digital health interventions and innovations have the potential to build upon all three pillars of the End TB Strategy. The WHO has released a digital health strategic agenda (9) describing different approaches. The potential applications of digital health are broad and may include: utilizing innovative approaches such as videoobserved therapy to allow for a more patient-centered approach, applications for adherence support, remote patient consultations and remote technical assistance including consensus expert opinions for complex cases (referred to as TB "consilium" in some settings) (see **Figure 1**).

FIGURE 1 | "Digital health in tuberculosis program delivery." In many countries, TB programs are structured across a number of levels: from the most peripheral (pink) where patients may be located in village or community settings, to subnational levels (blue) where clinicians may be based at provincial or district health centers or hospitals, to the national or supranational level (yellow) where technical advisors and national TB programs are based. There are a number of digital health applications that have the potential to help bridge some of these "gaps".

Due to the complexity of treating MDR-TB, which involves long, complex and expensive regimens and close monitoring for side effects, management of these cases is often supported by an expert multidisciplinary team who provide advice and guidance to the treating clinician. This group has been termed a "TB consilium" in some settings (10), with origins in Eastern European and former Soviet Union TB control programs. TB consilia are typically centralized, with the ability to discuss the most difficult cases at a higher level (e.g., national or supranational level). In more recent times, as individual members may be remotely located, Web-based telemedicine platforms have been increasingly utilized. This format is typically not "real-time," but rather a "store-and-forward" arrangement whereby clinicians are able to upload case summaries to seek specific management advice or opinions.

A number of countries, including Belarus, Belgium, France, Mexico, Portugal, and the UK have implemented national level TB consilia (11, 12). These were compared in a recent review (11). Notably, a number of countries were using emailbased discussion, which has privacy and legal implications, particularly where identifiable data or clinical images are used (13). Alternately, other countries utilized a regular (monthly) meeting, with attendance either in person or remotely. However, this may not be ideal in terms of timely advice for patient care.

The European Respiratory Society (ERS)/WHO electronic consilium (14) was a supranational, free-to-access, bilingual, Web-based consilium service. Between 2013 and 2018, this system managed more than 400 TB cases from a number of different continents (10). Following the success of the ERS/WHO consilium, a newly promoted initiative: the Global TB Network, was launched in 2018, incorporating a Global TB consilium (15). An additional feature of this network is the provision of support for the management of latent TB infection, an area for which there is increasing focus.

In many medium and high TB incidence settings, TB consilia have acted as an approval or consensus decision system for accessing new drugs or regimens for MDR TB, specifically relating to the use of the new TB drugs: bedaquiline and delamanid (11, 16, 17). Initially, from 2013, where access to these drugs was only via compassionate access or pilot programs, this facilitated a "gatekeeper" function with restricted use for specific indications only. However, with the recent update to WHO treatment guidelines for MDR TB (5), bedaquiline is now recommended upfront for all patients with MDR TB, and hence should no longer require approval for procurement and administration unless supplies are limited and therefore prioritized. In some countries, Delamanid remains on special access mechanisms, such as compassionate use basis and hence consilia remain the main way of approval for accessing the drug.

Finally, consilium can serve an educational role: for clinicians to review how previous similar cases have been managed, and also to have direct access to seek guidance from experienced advisors and mentors. Submitted cases may be rare, unusual or the first experience with a particular regimen or combination of drugs (18), in which case sharing of experience can be beneficial.

## Case Study: Telemedicine to Support MDR-TB Care in Papua New Guinea

Papua New Guinea (PNG) is classified as a high burden country for MDR-TB, TB and TB/HIV by the WHO. It has the highest estimated TB incidence rate in the Western Pacific region at 432 per 100,000 people (1). An unprecedented outbreak of MDR-TB has been reported on Daru Island in the Western Province of PNG (19). This led to the establishment of the emergency response task force for MDR-TB in 2014, led by the National Department of Health in PNG with support from the Australian Government, across three identified "hotspot" provinces, including Daru, Western Province. The programmatic interventions during the response in Daru 2014-17 and clinical care model have been described elsewhere (20, 21).

The Burnet Institute, based in Australia, has been the technical assistance partner in the multi-stakeholder response in Western Province through the RID-TB project (Reducing the Impact of Drug-resistant TB in Western Province) since August 2014. The RID-TB project supports the provincial TB program in program design and evaluation, implementation, clinical care, capacity building and training, health systems strengthening and operational research. The RID-TB project staff include long term advisors (remote/visiting) and field-based TB specialists. Clinical care is provided by medical officers and health workers at Daru General Hospital. A clinical expert group (CEG) (or consilium) was formed to support MDR-TB care and provide advice, primarily on patients to whom the standardized care pathway did not apply e.g., pediatric cases, extra-pulmonary TB, TB-HIV, extensively drug-resistant TB (XDR-TB) and management of adverse effects or complex cases. In particular, there was a need to provide clinical consensus decision for patients initiated on newer TB drugs—bedaquiline and delamanid—which were initially obtained via compassionate access, and for which there was limited in-country experience.

Because of the remote location, the CEG initially communicated de-identified case discussions on email and a file server for records. After assessment of various options, the project implemented a telemedicine platform in April 2016: Collegium Telemedicus (22). The choice of platform was based on the considerations of data security and accountability, cost, bandwidth requirements, ease of use, and the capacity for twoway discussions between clinicians and the CEG. The platform allows users to submit case summaries, and also to attach relevant results, to assist in interpretation e.g., X ray images or electrocardiogram (ECG) records. Our CEG is comprised of a group of technical experts with a broad range of international experience, who are also familiar with the Daru context. Because of time limitations and varying availability, the group has a monthly roster, to ensure that there is always a timely response, although areas of expertise can be drawn on for specific cases.

The objectives of the consilium/CEG: are to provide clinical support in complex case management, with the additional functions of capacity building, quality assurance, and the formation of collaborations between physicians from different locations (e.g., Daru General Hospital, Port Moresby General Hospital and the National TB Program).


From April 2016 to February 2019, 237 cases were discussed using the platform. This encompassed diagnostic (presumptive) and treatment cases. More recently, support to the scale up of preventative therapy for latent TB infection has been included. Across this time period, there have been 44 different "user" accounts (either with the capacity to make referrals, or view only for educational purposes), and a current pool of five technical experts providing technical support and advice across a range of areas. The median response time from referral for cases was relatively prompt at 17.6 h, facilitating timely decision making. The median number of messages exchanged for each case was 10, reflecting the amount of user engagement and discussions occurring.

The program has seen a high turnover of field-based staff, due to the remote and challenging context. As a result, there can be some variation in practices with different clinicians and one key benefit of Collegium Telemedicus that we have noted is for continuity of patient care. With patients often encountering several different clinicians across their "journey," this allows transparency in the rationale for previous decision making. As a learning tool, the consilium has also served as a useful resource to explore how similar patients may have been approached in the past e.g., MDR-TB infections in pregnancy.

The telemedicine service facilitated the uptake of innovations where technical support was required. The programmatic use of the TB drug Bedaquiline was scaled up in PNG in 2016, procured by the National TB Program, and supported by the global donation program (21). In this context, the consilium proved useful in supporting local clinical decision making in the scale up of this drug where there was little previous in-country experience and a limitation on its supply. There were 75 cases in which the use of Bedaquiline was discussed or mentioned, with a high frequency of discussions occurring in the initial period (26 cases in the first 12 months), which has appeared to decrease as clinicians gained familiarity with use of the drug (15 cases in the last 12 months).

A number of progress/feedback reports were completed by program participants. Of the five reports that were completed across 2016–2018, all respondents felt that the responses they received on Collegium Telemedicus were timely and welladapted to the local environment. However, in response to whether there would be a benefit to the eventual outcome for the patient, two participants responded "yes," two responded "perhaps" and one responded "no". Three of the five respondents felt that there was educational benefit in the process.

Although there were a number of concurrent interventions across this time period, we believe that this service has contributed, in part, to the very good outcomes now seen for MDR-TB patients in Daru (20, 21). Treatment success rates for the 2015 MDR TB enrolment cohort were 81%, with only 4% of patients being lost to follow up and 4% failing treatment (20).

An analysis of the perceived benefits and challenges of the telemedicine platform according to the defined objectives, is provided in **Table 1**. There have been a number of challenges throughout the implementation process including high fieldstaff turnover and variable engagement depending on staff preferences. Internet access and connectivity has been a limiting issue at times, despite the low bandwidth requirements. While expansion to telemedicine services that incorporate real-time video linkages is an appealing option, it is currently still not possible for many rural sites in PNG.

### CONCLUSION

Implementation of the WHO End TB strategy will require ongoing innovation and the introduction of new treatments and care models for MDR-TB. The WHO guidelines will have a number of significant and ongoing updates as new evidence is generated, in particular for MDR-TB treatment. These will require technical assistance in resource-limited settings, where training and capacity gaps may exist.

The RID-TB technical assistance project in Daru, a remote and resource limited setting in PNG, implemented a telemedicine platform to optimize MDR-TB patient care. The telemedicine platform supported high quality clinical care and fostered collaboration between clinicians and technical experts in a shared learning environment. In particular, it bridged the knowledgedelivery gap in supporting the scale-up of innovations such as implementation of bedaquiline for MDR-TB and management of complex cases.

Telemedicine is a key intervention to optimize patient care and build local capacity and clinical collaborations in the management of MDR-TB in low resourced settings, particularly in the landscape of new and emerging evidence and evolving standards of care in MDR-TB anticipated in the coming decade.

### DATA AVAILABILITY

The datasets for this manuscript are not publicly available. Requests to access the datasets should be directed to khai.huang@burnet.edu.au.

### REFERENCES


### AUTHOR CONTRIBUTIONS

GH, GP, RW, PdC, and SM: concept. GH and RW: acquisition of data. GH, GP, DO'B, PdC, RW, and SM: drafting of manuscript. GH, GP, MT, SH, PU, DO'B, PdC, SG, RW, and SM: critical review and final approval of manuscript.

### FUNDING

The Reducing the Impact of Drug-resistant TB (RID-TB) project was supported by the Australian Government and implemented by the Burnet Institute. The Stronger Systems for Drug-resistant Tuberculosis and Malaria research grant was supported by the Australian Government and implemented by the Menzies School of Health Research and the Burnet Institute. The views expressed in this publication are the author's alone and are not necessarily the views of the Australian Government.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Huang, Pawape, Taune, Hiasihri, Ustero, O'Brien, du Cros, Graham, Wootton and Majumdar. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Experience of Supporting Telemedicine Networks With the Collegium System: First 6 Years

#### Richard Wootton<sup>1</sup> \* and Laurent Bonnardot 2,3

<sup>1</sup> Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, Tromsø, Norway, <sup>2</sup> Fondation Médecins Sans Frontières, Paris, France, <sup>3</sup> Department of Medical Ethics and Legal Medicine, Paris Descartes University, Paris, France

#### Edited by:

William Edson Aaronson, Temple University, United States

#### Reviewed by:

Pradeep Nair, Central University of Himachal Pradesh, India Deborah Elaine Seale, Des Moines University, United States

\*Correspondence:

Richard Wootton r\_wootton@pobox.com

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 14 April 2019 Accepted: 29 July 2019 Published: 21 August 2019

#### Citation:

Wootton R and Bonnardot L (2019) Experience of Supporting Telemedicine Networks With the Collegium System: First 6 Years. Front. Public Health 7:226. doi: 10.3389/fpubh.2019.00226 The Collegium system was first made available in 2012 to support organizations conducting humanitarian or non-commercial telemedicine work in low resource settings. It provides the technical infrastructure necessary to establish a store-and-forward telemedicine service. During the subsequent 6 years a total of 46 networks were established, based on the Collegium infrastructure. The majority of the networks were set up to provide a clinical service (33), with six designed for education and training, and the remainder for test or administrative purposes. Of the potentially operational networks which were set up (i.e., those established for clinical or educational purposes), 15 networks (38%) were stillborn and did not handle a single case after being established. In contrast, the two most active networks had handled almost 12,000 cases. The average case rate of the five most active clinical networks operating in low-resource settings (i.e., the total number of cases divided by the length of time for which the network had been established) ranged from 0.5 to 29.4 cases/week. Across the networks there was little evidence of sigmoidal growth in activity, which is consistent with reports of other telemedicine activity in North America. A brief survey was sent to 49 network coordinators, from 31 networks. Responses were received from 9 coordinators (18% of those invited to participate). The median satisfaction with the system was 8 (on a scale from 1 = not at all satisfied to 10 = very satisfied). The free text comments were mainly technical suggestions regarding image transfer, the mobile application, or other modes of communication. The results of operating the Collegium system demonstrate that supporting telemedicine work in low resource settings can be successful, since the networks handled a very wide range of clinical cases, and at activity levels up to several cases per day. However, approximately one-third of the networks that were established did not handle a single clinical case. Nonetheless, this might represent a form of success in the sense that it prevented the waste of resource involved in an organization purchasing a telemedicine infrastructure only to find that it was not used.

Keywords: telemedicine, low-resource settings, tele-expertise, education, humanitarian

### INTRODUCTION

Collegium Telemedicus is a not-for-profit organization which provides the technical infrastructure necessary to establish a store-and-forward telemedicine service. The aim is to support organizations conducting humanitarian or non-commercial work in low resource settings. There is no charge for using the Collegium system if the telemedicine service is humanitarian in nature (see https://collegiumtelemedicus.org for further details). The Collegium system is provided under a Software As A Service model, and is designed to be easy to use ("usability"), with few technical (hardware and software) requirements for users, and to be able to serve requests from increasing numbers of users ("scalability").

The Collegium system was first made available in November 2012 (1). Our hypothesis was that organizations delivering health care in low-resource settings would make use of it to establish trial telemedicine services. We expected that those which succeeded would then either convert to a bespoke service based on the Collegium model, or would transfer their operation to another telemedicine provider.

The aim of the present study was to verify or refute the hypothesis about the use of the system, and to examine the outcomes of the first 6 years of its use.

### METHODS

Information for potential users of the system was made available via the web<sup>1</sup> . Each organization establishing a network on the Collegium system provided a prospectus describing the purpose of the proposed work, and nominated an individual as a sponsor or guarantor. All network guarantors provided consent to the use of anonymized data from their network for research purposes. Anonymized information about the nature of each network was extracted from its prospectus. This included:


In order to measure network activity, the numbers of clinical or educational cases (i.e., non-test cases) managed on each network were examined for the epoch November 2012 to October 2018, inclusive. A clinical case corresponded to a patient, and an educational case corresponded to a case report. Mean case rates were calculated from the date that each network was established until 31 October 2018, i.e., the total number of cases received in this period was divided by the number of days of operation and expressed as a mean rate (cases/week). For convenience, network activity was also summarized as:

TABLE 1 | Main purpose of the networks operating in the Collegium domain, and in private domains\*.


\*Private domains represent Collegium networks that are accessed via the URL belonging to the sponsoring organization, rather than through the Collegium home page at https:// collegiumtelemedicus.org.


The types of case actually handled in the networks, as opposed to the types of cases that were specified in the network prospectus, was examined with reference to the types of specialist involved in answering each case. Thus, if a particular case had been sent to a pediatrician and to a radiologist for reply, the case was categorized as requiring both pediatric and radiologic expertise.

Finally, a short survey (Appendix 1 in **Supplementary Material**) was sent by email to all network coordinators for whom there was a valid email address.

Statistical analyses were conducted using the R Framework via the Wessa interface (2).

### RESULTS

At the end of the study period a total of 46 networks had been established, based on the Collegium infrastructure. The majority of the networks (33 or 72%) had been designed to provide a clinical service. Six networks (13%) had been designed to provide education or training. The other seven networks (15%) were used by Collegium for internal administrative purposes concerning the development of the system, e.g., software testing (see **Table 1**).

### Characteristics

There was wide variation in the characteristics of the networks and in the areas they were designed to serve (**Table 2**). Almost half of the clinical networks had been established in order to manage telemedicine cases of all specialties (surgical, medical, nursing, allied health). The other half of the networks had been established to manage single-specialty cases, such as radiology, dermatology, or psychiatry (**Figure 1**).

The networks were mainly set up by informal groups of clinicians (30 of the 39 networks). The sponsor of each network (i.e., the person requesting that it be established) was most

<sup>1</sup>Collegium Telemedicus home page. Available online at: https:// collegiumtelemedicus.org

#### TABLE 2 | Characteristics of the networks.


<sup>a</sup>Purpose: Admin, administrative or test purposes; Edu, educational or training; Clin, clinical.

<sup>b</sup>Organization: type of organization. Informal/Charity (humanitarian organization)/Other.

<sup>c</sup>Speciality: clinical areas managed.

<sup>d</sup>Languages: network languages.

<sup>e</sup>Sponsor: country of sponsor.

<sup>f</sup> Catchment: countries of referrals.

<sup>g</sup>Activity: 0 = no activity (fewer than 10 clinical cases); 1 = some cases handled but not active at the time of study; 2 = active at the time of study.

commonly based in France or in the US (12 and 11 networks, respectively, see **Figure 2**). All networks which stated the fact in their prospectus were using volunteer specialists to provide the necessary expertise.

Four networks were established by charitable organizations and five networks were established by other organizations (e.g., those investigating telemedicine for commercial or semicommercial reasons).

Approximately half of the networks were set up to use a single language, most commonly English. However, the Collegium system is available in four other languages (French, Spanish, Portuguese, and Arabic) and almost half of the networks elected to work in multiple languages (see **Table 3**).

### Activity

There was wide variation in the activity levels on the networks. Of the 39 potentially operational networks which were set up (i.e., those established for clinical or educational purposes), 15 networks (38%) were stillborn and had not handled a single case after being established. In contrast, the two most active networks, both clinical, had handled almost 12,000 cases.

A total of 33 networks had been set up to provide a clinical service. All but one had been established to provide a service to referrers in low-resource settings, mainly in developing countries; one network was operating in an industrialized country (albeit one with referrers based in remote regions). Six of the 33 clinical networks (18%) could be considered to have matured into routine services, having handled more than 100 cases each (see **Table 4**). Two of the six educational networks (33%) could be considered to have matured into routine services, having handled more than 100 cases each.

The average case rate of the five clinically-active networks operating in low-resource settings (i.e., the total number of cases divided by the length of time for which the network had been established) ranged from 0.5 to 29.4 cases/week. However, within the networks, the case rates fluctuated considerably. For example, approximately 2 years after it had been established network 25 demonstrated a large peak in its the weekly case rate (see **Figure 3**). This peak was due to cases being submitted from a hospital which suffered a sudden shortage of specialists. When new staff were appointed at that hospital, its use of telemedicine ended abruptly; the network reverted to a relatively constant level of referrals, as demonstrated across the remainder of the study period.

There was a wide range of types of case handled in the 39 nontest networks, including medical, surgical, nursing, and allied health (**Figure 4**).

### Survey

The survey was sent to 49 network coordinators, from 31 networks. Responses were received from 9 coordinators (18% of those invited to participate; **Table 5**). The median satisfaction score was 8. The comments made by the responders mainly concerned the technical aspects of the system. Overall, they stated that they were satisfied with it and grateful to be able to use a system that was reliable and efficient. The free-text comments are summarized in **Table 6**.

#### TABLE 3 | Languages used by the clinical and educational networks.


\*Primary network language shown first.

### DISCUSSION

Our aim in making the Collegium system available was to facilitate the introduction of telemedicine by organizations delivering health care in low-resource settings. There is reasonable evidence that telemedicine in this environment is clinically useful and provides valuable support for remote staff (3, 4). However, a barrier to any organization contemplating the introduction of a telemedicine service is the provision of TABLE 4 | Networks which had handled more than 100 cases (excluding administrative/test networks).


\*Excluding test cases.

the necessary technical infrastructure. Unless the organization is able to join an existing network operated by somebody else, then it will be necessary to start one from scratch, either using existing software or building the software required. While starting a telemedicine network from scratch is perfectly possible, it requires appropriate technical expertise. Also, an underlying IT infrastructure is required, e.g., a web server, whether existing software or bespoke software is employed. The point of the Collegium approach is that both the software and the IT infrastructure are made available as an integrated package, so that only minimal setting up is required before telemedicine work can begin.

However, the technical infrastructure is not the only thing required in order to establish a successful and sustainable telemedicine service. That is, the infrastructure is a necessary but not sufficient condition for successful telemedicine (as was

pointed out by one of the survey respondents), because there must also be engagement from referrers submitting cases, experts who provide responses, and some kind of clinical supervision to ensure that the whole process runs smoothly. These human factors are critical to the initial phases of a telemedicine network, and to its eventual adoption into routine health care (or not).

During the first 6 years of operation, 33 networks were set up to provide a clinical telemedicine service. This verifies our hypothesis that organizations delivering health care in lowresource settings would use the Collegium system to establish trial telemedicine services. Of these, six networks could be considered to have matured into routine clinical services, i.e., they could be considered as successful networks from the Collegium point of view. There are no published data from any similar system to compare this with, and we are not able to conclude that a "success" rate of 18% is either disappointing or encouraging. We are able to state however, that many of the "failed" networks did not process a single telemedicine case, i.e., failure occurred at the initial inception, rather than after the first few months when the workload began to build up. We can also speculate that the networks with zero activity may have prevented their parent organizations from wasting scarce resources on setting up expensive telemedicine networks that were then unused. In this sense, perhaps these networks can be counted as a success.

Wide variations of activity were observed in the 39 clinical and educational networks established over the 6-year study period. We do not know the reasons for this, but we can assume that it is related to the non-infrastructural factors necessary for successful telemedicine:


#### TABLE 5 | Summary of survey responses.

#### TABLE 6 | Free-text responses to survey.



access value)

Practical experience shows that the mechanism that matches demand for information to supply can only be partially automated; some human intervention is almost always required. In the Collegium system, this is provided by one or more network Coordinators whose job is to supervise the case flow, and when necessary to select suitable Specialists for the problem in hand. Although some of the successful networks have exhibited signs of strain in managing high case loads, we are not aware that any network has failed because of a lack of human Coordinators. Nor do there seem to have been problems in recruiting appropriate Specialists. We suspect that many of the networks which failed to establish themselves failed at point 1, i.e., they were unrealistic in expecting referring doctors to submit cases.

The underlying reasons for unrealistic expectations about initial referrals in a new network are not known. One possibility may be that local doctors practise in an environment where access to specialist opinions is always difficult, so they are used to managing complex cases alone, and there is no custom and practice of obtaining external advice. Another possibility is that they are working in a healthcare system that is so damaged or non-existent, that priorities are different and push local doctors to manage patients by themselves, without any expert advice. These are all factors identified in Labrique et al.'s review of best practice for scaling digital health initiatives in LMICs which were derived from practical experience in real-life case studies (5). They represent an important area for future research.

### Growth of Activity

The networks using the Collegium system had mainly come to it ab initio, i.e., they wished to start a new telemedicine network within their sphere of operations where none had existed before. What pattern of network activity would therefore be expected? Clearly activity would begin from zero, and would, if the network were successful, eventually settle at some reasonably steady rate which would reflect the routine use of telemedicine in their health care environment. It would not be unreasonable to expect a sigmoidal growth curve (**Figure 5**). Such growth curves are widely observed as technological innovations are adopted in health care, and into industry more generally. Growth curves of this shape result from the gradual adoption of a new technique within an organization during which early adopters accept it quickly, while late adopters ("laggards") take much longer (6).

It is not clear, however, whether sigmoidal growth curves can be expected in telemedicine. Grigsby et al. provided one of the first quantitative reports on the adoption of telemedicine in North America (7). However, the data they cited on the numbers of teleconsultations reported during consecutive annual surveys show little evidence of sigmoidal growth. Similarly, a later study by Darkins provided little evidence of sigmoidal growth in the number of teleconsultations in the VHA healthcare system (8). The Collegium data are consistent with these reports. Taking

### REFERENCES


the five small clinical networks and calculating their referral "trajectories" (i.e., adjusting their cumulative referrals so that they all start at month zero and end at 100% after 15 months) shows that there is only weak evidence of accelerating growth after start-up (**Figure 6**).

### CONCLUSION

During the first 6 years of the availability of the Collegium system, a total of 39 telemedicine networks were set up for clinical or educational purposes. Although a substantial proportion of the networks that were set up using the Collegium infrastructure approximately one-third—did not handle a single clinical case, this might represent a form of success in the sense that it prevented the waste of resource involved in an organization purchasing a telemedicine infrastructure only to find that it was not used. The reasons for unrealistic expectations about initial referrals in a new network are not presently understood.

The remaining 62% of networks handled a wide range of clinical cases, and at activity levels ranging from less than one case per week to several cases per day. The overall satisfaction of the network coordinators who responded to the survey was uniformly high. They identified many benefits in using the system and offered various constructive suggestions about improving its future development.

The present study suggests that the Collegium system has fulfilled its aims in providing useful support for a range of organizations conducting humanitarian or non-commercial work in low resource settings. However, it only solves part of the problem of setting up a successful telemedicine network, and organizations contemplating such a step should not underestimate the non-technical aspects.

### DATA AVAILABILITY

All datasets generated for this study are included in the manuscript and/or the **Supplementary Files**.

### AUTHOR CONTRIBUTIONS

RW and LB conceived and designed the study, contributed to writing the manuscript, and approved the submitted version.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh. 2019.00226/full#supplementary-material


pilot trial – a comprehensive analysis. Front Public Health. (2015) 3:257. doi: 10.3389/fpubh.2015.00257


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The authors are members of the Steering Group for Collegium Telemedicus, which is a not-for-profit organization that makes the Collegium telemedicine system freely available for humanitarian purposes (see https://collegiumtelemedicus.org).

Copyright © 2019 Wootton and Bonnardot. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Tele-Ultrasound in Resource-Limited Settings: A Systematic Review

Noel Britton<sup>1</sup> , Michael A. Miller 2†, Sami Safadi <sup>3</sup> , Ariel Siegel <sup>3</sup> , Andrea R. Levine1,3 and Michael T. McCurdy <sup>3</sup> \*

<sup>1</sup> University of Pittsburgh Graduate School of Public Health, Pittsburgh, PA, United States, <sup>2</sup> University of Pittsburgh Medical Center, Pittsburgh, PA, United States, <sup>3</sup> University of Maryland School of Medicine, Baltimore, MD, United States

Background: Telemedicine, or healthcare delivery from a distance, has evolved over the past 50 years and helped alter health care delivery to patients around the globe. Its integration into numerous domains has permitted high quality care that transcends obstacles of geographic distance, lack of access to health care providers, and cost. Ultrasound is an effective diagnostic tool and its application within telemedicine ("tele-ultrasound") has advanced substantially in recent years, particularly in high-income settings. However, the utility of tele-ultrasound in resource-limited settings is less firmly established.

#### Edited by:

Laurent Bonnardot, Université Paris Descartes, France

#### Reviewed by:

Margo Bergman, University of Washington Tacoma, United States Mario Ettore Giardini, University of Strathclyde, United Kingdom

#### \*Correspondence:

Michael T. McCurdy drmccurdy@gmail.com

†Co-first authorship

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 01 April 2019 Accepted: 13 August 2019 Published: 04 September 2019

#### Citation:

Britton N, Miller MA, Safadi S, Siegel A, Levine AR and McCurdy MT (2019) Tele-Ultrasound in Resource-Limited Settings: A Systematic Review. Front. Public Health 7:244. doi: 10.3389/fpubh.2019.00244 Objective: To determine whether remote tele-ultrasound is a feasible, accurate, and care-altering imaging tool in resource-limited settings.

Data Sources: PubMed, MEDLINE, and Embase.

Study Eligibility Criteria: Twelve original articles met the following eligibility criteria: full manuscript available, written in English, including a direct patient-care intervention, performed in a resource-limited setting, images sent to a remote expert reader for interpretation and feedback, contained objective data on the impact of tele-ultrasound.

Study Appraisal and Synthesis Methods: Abstracts were independently screened by two authors against inclusion criteria for full-text review. Any discrepancies were settled by a senior author. Data was extracted from each study using a modified Cochrane Consumers and Communication Review Group's data extraction template. Study bias was evaluated using the ROBINS-I tool.

Results: The study results reflect the diverse applications of tele-ultrasound in low-resource settings. Africa was the most common study location. The specialties of cardiology and obstetrics comprised most studies. Two studies primarily relied on smartphones for image recording and transmission. Real-time, rather than asynchronous, tele-ultrasound image interpretation occurred in five of the 12 studies. The most common outcome measures were image quality, telemedicine system requirements, diagnostic accuracy, and changes in clinical management.

Limitations: The studies included were of poor quality with a dearth of randomized control trials and with significant between study heterogeneity which resulted in incomplete data and made cross study comparison difficult.

**28**

Conclusions and Implications of Key Findings: Low-quality evidence suggests that ultrasound images acquired in resource-limited settings and transmitted using a telemedical platform to an expert interpreter are of satisfactory quality and value for clinical diagnosis and management.

Keywords: telemedicine, eHealth, ultrasound, resource-limited, tele-ultrasound, LMIC, tele-radiology, global health

### INTRODUCTION

### Background and Rationale

Global health encompasses both research and action aimed at promoting health for all persons, independent of national boundaries (1). Common barriers to global health initiatives include lack of healthcare access and lack of resources (2). Telemedicine (also called mHealth, telehealth, e-health), or literally "healing at a distance," is a tool well-suited to reduce these barriers (3). The term telemedicine specifically refers to care provided by a physician whereas telehealth is a more global term that encapsulates care provided by all healthcare professionals (e.g., pharmacists, nurses) (4). While its manifestation and implementation can vary across different medical specialties, telemedicine universally attempts to utilize technology to provide clinical support to patients across geographical barriers in an effort to improve patient health outcomes (2). Telemedicine, therefore, functionally expands patient access to care by mitigating geographic barrier to healthcare (5).

The history of telemedicine dates back more than a century. An article published in the Lancet in 1879 describes the use of the telephone to reduce patient office visits (6). In the 1900's there were there are reports about physicians using the radio to make a medical diagnosis. In 1906, a paper was published by Willem Einthoven, the inventor of the electrocardiogram, about the use of tele-cardiogram. Einthoven used the telephone cable to transmit a signal from the hospital to his laboratory, 1.5 km away. He subsequently utilized telecardiogram to remotely analyze clinical EKGs from patients in the hospital. By the 1920's, telemedicine provided medical consultation from medical centers in Italy, Norway and France to patients aboard ships and on remote islands (6, 7). By the 1950's, the transmission of radiographic images began in the United States and occurred shortly thereafter in Canada (8). The United States National Aeronautics and Space Administration's (NASA) adopted telemedicine in the 1960's in an effort to ensure safety in space flight. What began as remote monitoring of biometric data in the 1960s gradually escalated to ensure that astronauts could receive an accurate diagnosis by onboard crewmates in the event of a medical emergency. NASA ultimately developed a terrestrial parallel program called Space Technology Applied to Rural Papago Health Care (STARPAHC) (9). In collaboration with the Tohono O'odham tribe of Southern Arizona and the Indian Health Service, NASA used rudimentary telemedicine technology to successfully connect patients in resource-limited areas with physicians at hospitals elsewhere in the state via mobile support units (10). Since then, the field of telemedicine has evolved rapidly, propelled by major technological advances including email, mobile phones, the internet, ultrasound technologies, videoconferencing, and smartphones.

As telemedicine evolved, the field of ultrasonography matured in parallel. By the 1990s, ultrasound technology had developed into a bedside tool that physicians, particularly emergency physicians, were routinely utilizing (11). Ultrasound is a safe (non-ionizing) and portable tool capable of being used in a diagnostic or interventional capacity. Ultrasound has both 2D and 3D capabilities, can be analyzed in realtime, and is a comparatively low-cost imaging modality (12). Moreover, a growing body of evidence demonstrates that bedside ultrasound is more accurate than conventional physical exam for cardiovascular diagnoses (13). In low- and middle-income countries (LMICs), ultrasound and plain radiographs are often the only available imaging modalities (14). As ultrasound machines became increasingly portable and as technologies to support data transmission became commercially available, adequate infrastructure could support the emergence of teleultrasound. The tele-ultrasound paradigm involves performing bedside ultrasound at one location with images transmitted and interpreted by a provider located in a geographically distant location. This process can be conducted either in a synchronous, or real-time manner, or in an asynchronous manner. Asynchronous tele-ultrasound utilizes a store-andforward technique in which images are captured, stored, and later transmitted for image interpretation. Tele-ultrasound offered a seamless solution for skeptics of telemedicine who questioned the ability to ascertain a meaningful physical examination from afar.

Studies based in high-income countries suggest that teleultrasound is clinically valuable. Tele-ultrasound has been successfully used in diverse settings, including telecardiology consultation for neonatal units in Northern Ireland, airplanes in flight, Antarctic research stations, even at the International Space Station (15–18). Furthermore, studies have clearly demonstrated that images can be reliably transmitted between geographically distinct locations without loss of clinically important image quality via commercially available twoway audiovisual technology (19–21). Instrumental to the evolution and global utilization of tele-ultrasound was the finding that minimally trained sonographers can acquire high quality images using real-time guidance from experts afar, an infrastructure called remote tele-mentored ultrasound (RTMUS) (20, 22). RTMUS utilizes a single centrally-located physician trained in bedside ultrasound who guides a geographicallyremoved bedside provider in image acquisition and performs image interpretation from afar. Early work in high-income countries demonstrated that remote tele-mentored ultrasound was feasible and accurate in cardiac, trauma, and critical care applications (22–25).

### Objective

Tele-ultrasound is increasingly used to provide global health care. Even in high-income countries, patient care is frequently limited by a lack of access to trained clinicians. This supplydemand mismatch is further exaggerated in resource-limited settings where a dearth of subspecialty and procedurally-trained physicians often exists and the resources available to those physicians may be limited by economic constraints. The use of tele-ultrasound in resource-limited countries is, therefore, a rapidly burgeoning field. Due to the topic's clinical significance, a need exists to aggregate the various studies on the topic of tele-ultrasound in resource-limited settings. The goal of this paper is to systematically review the literature to determine whether remote tele-ultrasound is a feasible and accurate imaging modality that alters the care provided to patients in resourcelimited settings compared to the standard of care. To our knowledge, no prior systematic review has been conducted on this topic.

### METHODS

### Design and Study Selection

We performed a review of all published reports of teleultrasound in resource-limited settings. This review follows the PRISMA guidance for systematic reviews (26). We included full manuscripts written in the English language and we excluded non-human studies, studies using exclusively 1D ultrasound, review articles, abstracts, case reports, and editorials. Inclusion criteria required: (1) a direct patient-care intervention; (2) performance in a resource-limited setting; (3) patient ultrasound images sent to a remote, expert reader for interpretation and feedback; and (4) objective data on the clinical impact of teleultrasound. In this study, we defined resource-limited settings as low-resource areas in LMICs. We excluded studies conducted in remote areas of resource-abundant countries. Studies that involved images collected by robotic arm or under the aide of virtual reality technologies were excluded. Only studies published before January 1, 2019 were included.

### Search Strategy

The literature search was conducted under the direction of the University of Maryland Health Sciences and Human Services Library Systematic Review Consultation Service (Baltimore, MD, USA). Databases searched include PubMed, MEDLINE, and Embase. Search terms included "ultrasound" AND "telemedicine" AND "resource-limited" present in the title or abstract, as well as common synonyms, including sonography, eHealth, developing world, and more (detailed electronic search strategy including relevant MESH terms for PubMed and MEDLINE and Emtree/exploded terms for Embase in **Supplementary Materials 1.1 and 1.2**, respectively). Relevant MESH and Emtree/exploded terms were also included. The database searches were completed on February 1, 2019 with all manuscripts published prior to January 1, 2019 evaluated for eligibility for inclusion in this review. Next, we searched the references of included papers to identify additional studies meeting inclusion criteria.

Two authors independently screened abstracts and selected candidate articles for full text review. If either author wanted to include a study for full text review, the full text was reviewed in its entirety. Full text review of remaining studies based on inclusion and exclusion criteria identified the final group of studies. A senior author settled any discrepancy in article selection between the two initial authors.

### Data Extraction and Analysis

Data extracted from each study included: study type, study location, publication year, tele-ultrasound method (real-time vs. asynchronous), sample size, patient demographics, organ system assessed, available cost data, ultrasound performer training level, interpreter training level and location, ultrasound type, telemedicine platform, and clinical outcomes. Descriptive statistics were used to report trends in the performer training level, specialty, and outcome measured.

### Bias Assessment

We utilized a tool adapted from the ROBINS-I from the Cochrane collaboration in order to evaluate bias at an individual study level (**Table 1**) (39). The quality of the studies included in this systematic review was poor which precluded any further quantitative data analysis.

### Summary Measures and Synthesis of Results

No summary measures were utilized in this narrative systematic review. The heterogeneity and quality of the studies prevented data from being combined or any formal measures of data consistency to be performed. Furthermore, due to the data quality, no meta-analysis was performed and there is no plan for a follow up meta-analysis.

## RESULTS

### Search Results

A literature search conducted in PubMed, MEDLINE, and Embase resulted in 69 articles with filters for English language and human subjects applied. Ten duplicates were removed for a total of 59 unique articles. After title and abstract review, 29 articles were removed due to failure to meet inclusion criteria. Fifteen additional articles were identified from citations. Of the remaining 45 articles that underwent full text evaluation, 16 were removed for either not involving a patient-care intervention or not reporting clinical outcomes, 13 were eliminated for failing to meet criteria for a resource-limited setting, and two abstracts without accompanying manuscripts were removed (one had an English title and abstract but foreign language full text; one utilized only 1D ultrasound). A total of 12 studies were included in the final analysis (27–38). A schematic of the study search and selection process is shown in **Figure 1**.

#### TABLE 1 | Assessment of bias in individual papers.


(Continued)

Tele-Ultrasound in Resource-Limited Settings

Britton et al.

Frontiers in Public Health | www.frontiersin.org


(Continued)

Tele-Ultrasound in Resource-Limited Settings


## Risk of Bias

We utilized the ROBINS-I tool provided by the Cochrane Collaboration to assess for bias within each individual study. The majority of the studies evaluated were non-randomized control trials, making the ROBINS-I tool most appropriate. The results of the bias evaluation are shown in **Table 1**. Explanation of the risk judgement categories are shown in **Table 2**. The heterogeneity of the studies and the lack of principal summary measures in the majority of the studies made any evaluation of between study bias using a tool such as GRADE meaningless (40).

### Synthesized Findings

The results of the included studies are summarized in **Tables 3**–**5**. The studies were conducted over diverse geographic locations. Of the twelve studies, six were in Africa (two in Togo, one in Mali, two in Uganda, one in Kenya), two in South Asia (India), two in South America (Chile and Brazil), one in Europe (Serbia), and one in the Caribbean (Dominican Republic). The study size ranged from 22 subjects to 12,048. Four of the studies were pilot or feasibility studies and two were single-site randomized trials with patients enrolled into either an experimental arm (telemedicine) or control (standard care that did not include telemedicine) arm.

Hospitalized patients were enrolled in five studies, outpatients in clinics were enrolled in six studies, and patients in both schools and clinics were enrolled in one study. The medical scope of the studies varied widely. Four studies primarily involved obstetrics (**Table 3**), three studies focused on cardiology (**Table 4**), and five studies focused on general practice or other specialties (**Table 5**). Five studies were designed as screening programs accomplished by tele-ultrasound.

Pocket and other portable ultrasound machines were the most commonly used ultrasound devices. In five studies, video images were captured via the ultrasound machine. Still images were captured in two studies, but the type of image capture was not specified in four studies. All studies required internet access. Four studies were designed to operate using lowbandwidth connections. Four studies used cameras to record images from the screen of the ultrasound machine, while the remaining eight studies sent ultrasound images without the use of camera recordings. Two studies relied on smartphones for image recording and transmission, and two studies utilized satellite for internet connectivity.

Substantial study diversity existed regarding who functioned as the bedside ultrasonographer and the level of expertise of the image interpreter. Physicians and midwives scanned patients in nine of the twelve studies. Scans were also obtained by trained sonographers, technicians, and non-healthcare professionals. Remote interpreters were dispersed around the globe. When the ultrasonographer and interpreter were in the same country, the interpreter was generally found at a large academic or major referral center. All image interpreters were trained radiologists or cardiologists. Tele-ultrasound was performed synchronously in four studies and asynchronously in six studies. Both methods were used in one study, and it was not specified which method was used in a final study. Remote tele-mentored ultrasonography was used in four of the twelve studies.

TABLE 1 |

Continued

TABLE 2 | ROBINS-I risk categories.


The most commonly reported outcome in the twelve studies was diagnostic utility of tele-ultrasound (eight studies), followed by assessment of image quality (six studies). Other reported outcomes include management impact of tele-ultrasound (four studies), telemedicine system requirements (four studies), impact on follow-up (four studies), data transmission reliability and quality (three studies), effectiveness of education program for ultrasound examiner (two studies), patient cost savings (two studies), time to referral or follow-up (two studies), likelihood of hospitalization or adverse outcome (one study), and patient satisfaction with tele-ultrasound (one study). All the studies showed generally positive results for the primary outcome measures that were assessed.

Cost was a commonly addressed concern throughout the included studies, with an emphasis on the need for affordable tele-ultrasound platforms to make its use feasible in lowresource settings. Several of the studies built telemedicine platforms using open-source or low-cost, commercially available software, over-the-counter hardware, and low-cost portable ultrasound machines in an effort to minimize costs. Two studies explored cost from the patients' perspective and found that the introduction of tele-ultrasound was associated with lower out-ofpocket costs for the patient, generally due to minimizing need for travel to larger medical centers with formal imaging capacity.

Our formal analysis of bias using the ROBINS-I tool identified significant potential risks of bias in most of the studies. The majority of the studies had both a risk of bias in measurement of outcomes (ten studies) and in the selection of participants into the study (nine studies). Due to inconsistencies in study outcomes reported, the lack of principal summary measures, and the low quality of the studies included which primarily encompassed technical feasibility and observational studies with few randomized control trials, no meta-analysis was performed.

## DISCUSSION

### Summary of Main Findings

This systematic review suggests that tele-ultrasound performed in resource-limited settings can reliably produce satisfactory images with diagnostic utility that guide clinical management. According to the World Health Organization (WHO), imaging is needed for diagnosis in 20–30% of clinical cases and ultrasound and/or plain radiographs are sufficient for 80–90% of those cases. Yet, two-thirds of the world's population remains without access to medical imaging (14). Ultrasound, integrated into a telemedicine platform expands access to a safe, accessible, and affordable diagnostic imaging modality to populations in resource-limited settings.

Globally, ultrasound is a burgeoning diagnostic tool that often offers more insight into patient pathophysiology than the stethoscope. Thoracic ultrasound, as compared to chest radiography, has a high sensitivity and specificity for diagnosing cardiogenic pulmonary edema, pneumonia, COPD, pneumothorax, and pulmonary embolism in both the intensive care unit and the emergency department (41, 42). In fact, lung ultrasound is superior to chest radiograph in diagnosing pneumonia in the emergency department (43). In resourcelimited settings, lung ultrasound was more sensitive and specific than chest radiograph to diagnose pneumonia (44). Furthermore,

#### TABLE 3 | Study data for obstetrics-related papers.



#### TABLE 5 | Study data for non-obstetrics and non-cardiology-related papers.


Tele-Ultrasound in Resource-Limited Settings

point-of-care ultrasound can be incorporated into a telemedicine platform and performed with relatively little training by nonphysicians located at the bedside under the real-time guidance from ultrasound experts (20, 21, 45, 46). Thus, the use of RTMUS obviates the need for a bedside ultrasound expert to acquire images or a local expert to interpret them. RTMUS is particularly relevant in resource-limited settings in LMICs, where a scarcity of physicians often exists with expertise in ultrasound or with training in ultrasound-heavy subspecialties such as cardiology or obstetrics. Task-shifting ultrasound performance away from formally-trained sonographers and physicians to non-experts, while maintaining high quality imaging, helps establish a sustainable and cost-effective telemedicine program (47). This task-shifting also dramatically expands patient access to otherwise inaccessible subspecialists.

The studies included in this systematic review reinforce the concept that adequate ultrasound acquisition techniques can be taught in a remote tele-mentored manner. In cardiac ultrasound (**Table 4**), the high success rates for visualization of anatomic structures by non-experts allows for changes in medical management in the absence of a bedside physician. These changes include earlier treatment and appropriate escalation of care to tertiary centers (37). By utilizing non-experts as ultrasonographers, a larger population of patients gains access to ultrasonography as a diagnostic tool and to cardiology expertise. In this review, non-experts included physicians unfamiliar with a designated ultrasound approach, nurse research coordinators, a biomedical technician, and an imaging technician. Additional studies that did not meet the requirements for this review included custodians and medical interpreters as the non-experts performing the ultrasound (20). Collectively, these studies inform the conclusion that the quality of the ultrasound images obtained by non-experts are sufficient for interpretation by experts remotely.

Our literature review indicates that tele-ultrasound was frequently used in the field of cardiology (**Table 4**). Teleultrasound has demonstrated success in producing high quality, diagnostically significant images which alter management, decrease time to treatment, and provide more cost-effective care, especially when coupled with supporting data such as electrocardiogram, chest radiography, laboratory results, and clinical history (29, 30, 33, 35, 37, 48). In Aragonda, India, the use of remote tele-mentored echocardiography allowed for the diagnosis of pediatric cardiovascular pathology, resulting in a 29% referral for cardiac surgery based on those findings (35). In Bangladore, India, tele-ultrasound was used to assess times to treatment and long-term outcomes among children with structural heart disease. Images were collected in asynchronously and interpreted by a global consortium of cardiologists. Tele-ultrasound reduced the time to referral for valvular interventions and reduced the likelihood of both hospitalization and death (30). Though uncommon in high-income countries and likely underreported in low-income ones, rheumatic heart disease (RHD) is a major source of morbidity and mortality in LMICs (49). In the PROVAR study from Brazil, nonexpert ultrasonographers successfully screened schoolchildren for RHD and images were interpreted by geographically-removed experts (33). Collectively, cardiology-based tele-ultrasound studies demonstrate the transformative potential of utilizing this imaging modality in a resource-limited setting as a tool to better understand the epidemiological impact of a disease and to improve disease management and outcomes.

Obstetrics is an additional medical specialty in which ultrasound is heavily utilized around the globe (50). Unfortunately, supply of ultrasound machines, sonographers, and radiologists in LMICs is very low. For example, only two radiologists work in Liberia (51). In an attempt to overcome such challenges, ultrasound training programs have taught nonexperts either to independently perform obstetric ultrasounds to screen for high-risk pregnancies (52–55) or to utilize teleultrasound (29, 34, 37, 38). Of the multiple studies addressing the role of tele-ultrasound in resource-limited countries, the four included in this review focus on the obstetrics teleultrasound evaluation (**Table 3**). Ultrasonographers included physicians and midwives without prior obstetrics ultrasound training, but none of the obstetrics studies utilized RTMUS. Collectively, these studies concluded that ultrasound acquired accurate fetal structural views, allowed for the modification of perinatal care, and helped facilitate transfer to specialty centers when needed. Tele-ultrasound performed by a novice ultrasonographer prevented the need for additional re-imaging and yielded results available to the patient within 15 min. Image acquisition can be taught from a distance via the internet and a telemedicine platform is reliably able to transmit high quality images (29, 34, 37, 38).

Most of the studies included in this review implemented a brief training program for novice bedside ultrasonographers, regardless of the use of remote tele-mentored, real-time instruction. The training courses offered ranged from 3 days to 3 months. No correlation existed between the ultrasonographer's length of training and ability to adequately perform bedside ultrasound. Based on research not included in this systematic review, synchronous RTMUS can be successfully performed with <60 min of training (20, 21, 45, 46).

To date, we are unaware of any studies directly comparing synchronous to asynchronous telemedicine or tele-ultrasound. However, we believe an implicit benefit exists with using synchronous tele-ultrasound. Real-time image acquisition is well-suited to be combined with remote tele-mentoring to establish a hub-and-spoke paradigm whereby a single trained ultrasonographer can mentor numerous geographically removed ultrasound-naïve bedside providers to maximize the global reach of tele-ultrasound. By capitalizing on the concept of task-shifting inherent to RTMUS, any person located at the patient's bedside can function as the bedside ultrasonographer. Furthermore, realtime image acquisition and interpretation reduces delays in patient care and the need to return for follow up images, which may occur in an asynchronous point-and-store model of teleultrasound. Synchronous image acquisition also allows for realtime image quality control. As technology improves, wireless network and mobile phone access become more globally reliable, and commercially-available real-time audiovisual software (e.g., Skype, FaceTime) develop HIPAA-compliant platforms, the use of synchronous, RTMUS systems will be universally within reach.

The potential impacts of tele-ultrasound in LMICs are substantial with regard to the scope and breadth of both the numerous clinical areas (e.g., respiratory failure, hemodynamic compromise, procedural guidance) and the stakeholders (e.g., patients, providers, health systems) affected. The results of this systematic review, however, should be interpreted within the preestablished boundaries defined by the question we sought to answer using existing relevant studies. Specifically, this review addresses the feasibility of tele-ultrasound in LMICs and its clinical benefit to patients. Though certainly relevant to public health, this review was not intended to analyze the potential economic or workflow impacts of this technology on the health care providers or the health care system within each country. As public policy lies at the intersection of economic analysis and patient benefit, this systematic review cannot independently support changes to public policy but instead serves to further highlight the important clinical impact on patients.

### Limitations

Several limitations and biases impacted this review. It is possible that some articles were not assessed for eligibility due to the constraints of English language-only texts or articles not indexed on PubMed, MEDLINE, or Embase. Our goal was to capture those studies that utilized tele-ultrasound in resourcelimited settings that involved direct patient care investigations and reported those outcomes accordingly. Excluding remote areas of high-income countries from our definition of resourcelimited settings changed the available group of studies. While many important investigations have examined tele-ultrasound in remote settings of high-income countries (15, 16, 56), we chose to examine LMICs specifically in this review due to the fundamental differences in financial resources, healthcare personnel training and availability, health systems, and infrastructure that separate high-income countries from LMICs. Similar reasoning explains the exclusion of studies using robotic arm and virtual reality technologies. Several studies on the topic of tele-ultrasound in resource-limited settings were not included because they did not report a patient care intervention or meaningful clinical outcomes (20, 57–60). These studies were excluded because the goal of this review was to highlight those studies most germane to clinical practice, and studies in non-clinical environments that do not collect results relating to patient care are less easily clinically applicable.

The breadth of tele-ultrasound utilization was reflected in the marked heterogeneity of study designs. These varied designs led to different goals, outcomes, and reported data; moreover, their differences resulted in incomplete data when comparing studies. Many of the reported outcomes are related to technical feasibility or image quality interpretation and this evaluation is entirely subjective without the use of any standardized or validated measurement. This was compounded by the reality that the articles themselves were generally low quality and deemed as having a moderate to severe risk for bias, ranging from the selection of patients to be included in the study to bias regarding the selection of outcomes evaluated (**Table 1**). The high risk of bias in the majority of the included studies does limit the internal validity of the included studies. The missingness of reported data among the studies including the study design, patient selection, and participant selection limited the comparison of outcomes between studies. Control groups and randomization were rare. Nearly one-third of the studies were either pilot or feasibility studies. Collectively, this prevented substantial quantitative analysis of these studies and would certainly preclude any quantitative synthesis of the results into a meta-analysis. While all the articles reflected the promise of tele-ultrasound in resourcelimited settings, the need for higher quality evidence in the future is obvious.

### CONCLUSIONS

The global burden of disease in resource-limited countries often outpaces the access to diagnostic modalities needed to identify disease and the availability of trained clinicians to treat disease. This supply-demand mismatch makes ultrasound a precious tool in resource-limited countries. Ultrasound is a low-cost, reliable, diagnostic tool which can be performed by minimally-trained bedside providers. Over the last quarter century, numerous advances have precipitated the feasibility and success of remote teleultrasound in resource-limited settings. Technologically speaking, ultrasound machines have become smaller, more portable and durable, and the relative cost has decreased dramatically. Smartphones are becoming more commonplace and seamlessly operate numerous software options which are capable of functioning as affordable handheld telemedicine platforms. Lastly, global connectivity is increasing, particularly wireless cellular and internet access. These advances, in concert, have made tele-ultrasound feasible and invaluable in resource-limited settings.

## AUTHOR CONTRIBUTIONS

MAM performed initial systematic review and first draft of manuscript. SS helped with initial review and with manuscript edits. AS contributed to initial manuscript review of manuscripts included in systematic review and initial drafting of manuscript. AL edited manuscript and contributed to bias analysis. MTM oversaw project, involved in systematic review, and involved in drafting and revising the manuscript. NB performed bias analysis, revised paper to include bias analysis, and performed manuscript edits for second and third submission of paper.

### ACKNOWLEDGMENTS

We would like to thank the University of Maryland Health Sciences and Human Services Library Systematic Review Consultation Service (Baltimore, MD, USA) for their guidance with the literature search.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh. 2019.00244/full#supplementary-material

### REFERENCES


to images on the ultrasound machine. J Crit Care. (2016) 33:51–5. doi: 10.1016/j.jcrc.2016.02.019


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Britton, Miller, Safadi, Siegel, Levine and McCurdy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Telepathology in Low Resource African Settings

#### Nnamdi Orah<sup>1</sup> and Olorunda Rotimi 2,3 \*

<sup>1</sup> Specialist Laboratories Nigeria Limited (The Specialist Laboratories), Lagos, Nigeria, <sup>2</sup> Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom, <sup>3</sup> Department of Cellular and Molecular Pathology, St. James's University Hospital, Leeds, United Kingdom

#### Keywords: external quality, teaching, Africa, low resource, telepathology, digital slides

There is a critical shortage of pathology services in Africa. It is estimated that there are more than 500,000 people per pathologist in much of the continent, with this ratio exceeding 5 million to 1 in some countries (1). In Nigeria, for example, there are about 105 pathologists for an estimated population size of 200 million people, giving a ratio of one pathologist to every circa two million people.

Technological innovation has enabled "leap-frogging" for many developing countries in critical sectors like health. Efforts to bridge the human resource gap in pathology have led to the evolution and spread of diverse telepathology solutions across the continent.

#### Edited by:

Laurent Bonnardot, Université Paris Descartes, France

#### Reviewed by:

Aliyah Sohani, Massachusetts General Hospital, United States Camilo Adem, Institut de Pathologie de Paris (IIP), France

> \*Correspondence: Olorunda Rotimi olorunda.rotimi@nhs.net

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 23 April 2019 Accepted: 30 August 2019 Published: 13 September 2019

#### Citation:

Orah N and Rotimi O (2019) Telepathology in Low Resource African Settings. Front. Public Health 7:264. doi: 10.3389/fpubh.2019.00264

Telepathology is the diagnosis of surgical pathology cases at a distance using real-time video imaging or stored images (2). The definition of the American Telemedicine Association describes telepathology as the following: "A form of communication between medical professionals that includes the transmission of pathology images and associated clinical information for the purpose of various clinical applications including, but not limited to, primary diagnoses, rapid cytology interpretation, intraoperative and second opinion consultations, ancillary study review, archiving, and quality activities (3)".

Telepathology can be divided into four platforms: static images, whole slide imaging (WSI), dynamic nonrobotic telemicroscopy and dynamic robotic telemicroscopy (4).

Static telepathology involves the examination of precaptured still digital images (snapshots) that can be transmitted via e-mail or stored on a shared server. The benefits derive from its low cost, simple technology required and low maintenance. The images are also small and therefore are easier to manage and store. However, there are several drawbacks including sampling error, limited fields of view, lack of remote controls, and the person taking the images has to have some form of training to select appropriate diagnostic fields. Acquiring images can also be labor intensive.

WSI involves digitization (scanning) of glass slides to produce high-resolution digital slides allowing the consulting pathologist to see the entire specimen at a range of magnifications. It has been shown to be remarkably suitable for telepathology, because digital slides are of high resolution and there is user control of view and magnifications. Its drawbacks are high cost of acquisition and maintenance, the long duration it may take to scan the slides, requirement for large bandwidth internet service and storage difficulties following the large sizes of images produced.

Nonrobotic telemicroscopy involves real-time transmission of live images via a video calling platform, e.g., Zoom <sup>R</sup> , Facetime <sup>R</sup> , etc., to consulting pathologists who have no control over the display, while with robotic telemicroscopy the consulting pathologist has control over the live images. Advantages of robotic telepathology include access to the entire slide, user control of the microscope and image with respect to fields and magnification, good image quality, and fast driving speed. Its disadvantages include high cost of the requisite technology, high bandwidth requirements, as well as high costs of acquisition and maintenance (5).

These platforms have provided options for telepathology in developing and underdeveloped countries in sub-Saharan Africa. In this review, we explore the different telepathology solutions that have been adopted in sub-Saharan Africa, their successes and limitations, and potential solutions to these limitations.

### ROLE OF TELEPATHOLOGY IN LOW RESOURCE SETTINGS

### 1. **Teaching/Education**

In 2014 and 2015, the Anatomic and Molecular Pathology Department of Lagos University Teaching Hospital, Lagos, Nigeria in collaboration with St. James's Hospital, Leeds, United Kingdom initiated a telepathology postgraduate teaching program using whole-slide imaging (WSI) on the Aperio Digital Pathology System (Leica Biosystems, Buffalo Grove, IL, **Figure 1**) (6). This collaboration was strictly for teaching purposes only. The scanned slides were accessed via the Leeds virtual pathology website by the participants and real-time discussions were held over Skype <sup>R</sup> (Microsoft, Redmond, Washington) with the course facilitators in Leeds and Birmingham, United Kingdom. There were 12 sessions of online teaching with an average attendance of 15 participants per session. The participants were later surveyed on the value of this mode of teaching and over 85% of the 42 respondents assessed the lectures as excellent/good as reported by Rotimi et al. (6). **Figure 2** shows some pictures taken during these sessions. The advantages of this mode of teaching lie in its convenience and affordability. The participants and the facilitators are able to learn and teach, respectively, stationed in their natural environments. The cost of transportation and other logistics for on-site teaching can also be avoided or greatly minimized.

### 2. **External Quality Assurance**

Through the vision and dedicated efforts of a Nigerian pathologist practicing in the UK, Dr. U Ugbokwe, a national (latterly international) diagnostic external quality assurance scheme started in 2010 (see http://www.tslworkshopsng.com/ home). For many years, this scheme was slide-based, but in 2016, digital slides (WSI) were introduced. Currently, the selected histology slides for this bi-annual Nigerian and Ghanaian National External Quality Assurance (EQA) scheme are hosted on the Leeds Virtual Pathology server, which has enabled pathologists and pathology residents from Nigeria and Ghana to access the EQA slides at their own convenience. Twenty-five slides are used on each occasion for the EQA. This has eliminated the need to transport the glass slides across states and country borders, thus easing logistic issues. Moreover, the number of participants at the discussion workshops increased from 56 in 2010 to 115 in 2018 with many participants having viewed the digital slides prior to the meetings. It is hoped that the discussions which follow the submission of answers to the EQA will also be held online via teleconferencing facilities soon.

### 3. **Diagnostics**

This appears to be the most common use of telepathology in low resource settings. In Malawi, at the Kamuzu Central Hospital, WSI is used to transmit images to collaborating pathologists in the United States for diagnosis (7). The scanned slides are then viewed together with the remote pathologist at a set time. Both parties have equal control over the field of view and magnification, and the final diagnosis is made in agreement with the local pathologist in contrast to models where the foreign pathologist acts as a "stand-alone" consultant. This serves as a good model for teaching as well.

Similar collaboration between the Department of Pathology at Mulago Hospital in Uganda and Fuerth Hospital in Germany has been reported (8). A total of 96 cases were analyzed. The remote pathologist, using robotic telepatholgy, had control over the focus, magnification and field selection. The slides were viewed over an internet browser-based dynamic imaging system which provided clinical information, gross description, and a digitized microscopy platform. The remote operator had to be trained on the use of this equipment and also on selecting the significant areas for review. It took about 30 min for the pathologist to learn to use the telepathology system and between 4 and 25 min to read a slide remotely.

This use of telepathology has significantly reduced the turnaround time (TAT) in many underdeveloped countries. This is especially true in countries with a paucity of pathologists. A collaboration between Brigham and Women's Hospital (BWH) in Boston, Massachusetts, USA and Butaro Cancer Centre of Excellence in Rwanda explored the difference in TAT when blocks and slides were sent over to BWH as compared with uploading the slides to the iPath case sharing platform (a static image platform). A total of 3,514 samples were analyzed in total. The study carried out by Muvugabigwi et al. (9) discovered that TAT was significantly shortened for tissue biopsies uploaded to the iPath case sharing platform. Sending the blocks and slides to BWH resulted in a median TAT of 30 days while the iPath platform's median TAT was 14 days. For laboratories dealing with the absence of on-site pathologists, this is a viable and cost-effective alternative to sending blocks/slides abroad for diagnosis.

The presence of telepathology options for obtaining a second opinion can be a positive factor in the recruitment of pathologists. It has been reported that pathologists have challenges when they work alone (10), and this may affect their retention in remote locations where they are often isolated. However, if static or dynamic forms of telepathology are available, the pathologist can communicate and share diagnosis with other local/foreign pathologists. Pairing remote locations with more experienced tertiary centers is a viable option that can be explored in bridging the gap of pathology service in Africa.

Apart from diagnostic use, this may also be a form of informal EQA for remote pathologists who may not have the resources or logistics to participate in established local/foreign quality assurance schemes.

The low operating costs, bi-directional communication, locally responsive services as well as user acceptance have fueled the adoption of telepathology in many underdeveloped countries.

FIGURE 1 | Digital slide scanner.

### CHALLENGES

However, telepathology faces many limitations in scaling and widespread adoption. Some of these challenges and proffered solutions include:

### 1. **Dearth of Infrastructure**

Power supply and high-speed internet are two major pieces of infrastructure needed for telepathology which are desperately lacking in underdeveloped countries. Rotimi et al. (6) reported participants' difficulties in accessing the virtual pathology website as well as breaks in video communication during the on-line lectures due to poor internet connectivity. Montgomery et al. (7) reported insufficient bandwidth as a limitation to smooth communication and timely loading of slides leading to conference interruptions and rescheduling. Even though a lot of progress has been made in the area of

internet penetration across sub Saharan Africa, many remote laboratories remain without basic internet access.

The government and hospital managers in these countries have a huge role to play in the success of telepathology by making investments in the development of infrastructure and in creating an enabling business environment for telecommunication companies to function. Subsidies and incentives can also be given to telecom companies who invest in rural internet and/or who provide dedicated high-speed internet services for health facilities.

2. **High cost of Telepathology Equipment**

WSI telepathology is cost-intensive. An Aperio ScanScope system (Leica Biosystems, Buffalo Grove, IL) costs about \$85,000 USD exclusive of a \$4500 USD service contract (7). Significant capital outlay is also required for the purchase of the necessary bandwidth as well as cloud storage space. This is far beyond the entire budget of most public/private laboratories in underdeveloped countries.

Cheaper alternatives involving transfer of static slide images have evolved, and successes have been reported in the achievement of concordance between diagnoses made on slide images when compared with diagnoses made on glass slides (9). The drawback to static image transfer as earlier stated includes its lack of control of magnification and focus by the pathologist receiving the images. This may lead to a missed diagnosis with its attendant clinical and medicolegal implications.

The American Society for Clinical Pathology (ASCP) under its Partners for Cancer Diagnosis and Treatment in Africa has brought together a lot of partners to provide telepathology solutions (11) to underserved populations in Africa. They have donated slide scanners and other digital pathology equipment to laboratories located mainly in East Africa with plans to spread the same over sub-Saharan Africa.

3. **Medico-legal Considerations**

Legislation regarding the liability of physicians when delivering care across borders is quite unclear (12). African

### REFERENCES


and European legislation have not set forth specific provisions but in the United States, physician licenses are not portable. A draft bill is being considered by the US Congress to facilitate telemedicine endeavors by addressing such legal barriers. There are also legal issues arising from security concerns associated with the confidentiality of medical information on the internet. The solution to this lies in the encryption and anonymization of patient data stored on the internet.

In our opinion, static and non-robotic telepathology that requires little capital outlay, can make use of facilities already available and requires little maintenance costs will thrive in Africa. This will somewhat ensure the viability of these collaborations and requiring little or no support from donor agencies or international organizations. However, the limitations associated with the use of these particular models of telepathology and the scarcity of resources imply a need for establishment of regional centers of excellence equipped with WSI capabilities where possible.

In summary, despite the limitations and challenges, telepathology has wide-reaching benefits for healthcare in underdeveloped countries. They serve as avenues for diagnosis, teaching, quality assurance, and research for many pathologists, and they support improvement initiatives hitherto impossible in these countries. The development of collaborations offers significant scientific opportunities for hospitals and academic institutions in underdeveloped countries and those in advanced countries. However, there must be a will to overcome challenges in infrastructure as well as a certain degree of flexibility and resourcefulness in making use of the facilities available.

### AUTHOR CONTRIBUTIONS

OR and NO contributed equally to the writing and editing of the manuscript.


**Conflict of Interest Statement:** NO was employed by 'The Specialist Laboratories Nigeria Limited'.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Orah and Rotimi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Time Required to Create a Referral in Various Store-and-Forward Telemedicine Networks

Richard Wootton<sup>1</sup> \* and Barry O'Kane<sup>2</sup>

*<sup>1</sup> Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, Tromsø, Norway, <sup>2</sup> EndZone.IO Ltd., Glasgow, United Kingdom*

Store and forward telemedicine is used routinely in health care, but there is little published information about *how* such telemedicine systems are used. For example, an important aspect of the system's usability is the length of time it takes to submit a referral. Referral-submission times were measured in networks based on the Collegium Telemedicus system. In a 25-week period in 2018/2019, eight Collegium networks received a total of 1,649 clinical or educational cases submitted via the web interface. The time to prepare a referral was measured in 669 of these cases, in two different ways. An *indirect* measurement of the referral-preparation time was calculated as the interval between the user logging in, and the referral being submitted. A *direct* measurement of the referral-preparation time was calculated as the interval between the user opening the referral page and the referral being submitted to the server. The difference between the two measurements represents time spent by the user on other activities after logging in, before beginning the referral. The median referral-preparation time, measured directly, was 888 s (IQR 512-1765). The median of the differences between the two preparation times was 27s (IQR 8-146). The referral-preparation times in the eight networks were broadly similar, despite the differences in the nature of their operation (clinical or educational), and the types of case handled (single specialty or multi-specialty). Quantitative information about aspects of the user interface, such as the referral-preparation time, is important not only in the initial system design, but also in its subsequent development.

#### Edited by:

*Connie J. Evashwick, George Washington University, United States*

#### Reviewed by:

*Elizabeth Krupinski, Emory University, United States Sabe S. Sabesan, James Cook University, Australia*

> \*Correspondence: *Richard Wootton r\_wootton@pobox.com*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *31 March 2019* Accepted: *27 August 2019* Published: *18 September 2019*

#### Citation:

*Wootton R and O'Kane B (2019) Time Required to Create a Referral in Various Store-and-Forward Telemedicine Networks. Front. Public Health 7:260. doi: 10.3389/fpubh.2019.00260* Keywords: teleconsulting, low resource setting, referral preparation, user interface, store and forward

### INTRODUCTION

Store and forward telemedicine has been used routinely in health care for several decades. Yet there is little published information about how such telemedicine systems are used. For example, an important aspect of the system's usability is the length of time it takes to submit a referral: if referrals take too long to prepare, users will find other ways of answering their clinical questions (e.g., ordinary email) even though these are likely to be inferior to the use of store and forward telemedicine (1, 2).

Designers of telemedicine systems will usually have intuitive ideas about referral preparation and other matters, but these ideas are not likely to be founded on fact. Quantitative information about these aspects of the user interface are important not only in the initial system design, but also in its subsequent development.

Collegium Telemedicus offers a store-and-forward telemedicine system to support organizations conducting humanitarian or non-commercial work in low resource settings (3, 4). The objective of the present study was to measure the time users were taking to prepare cases for submission, with the aim of informing the future development of the system.

### METHODS

Referral-preparation times were examined in the Collegium Telemedicus system, which at the time of the study comprised 33 clinical networks and 6 educational networks. A clinical case corresponded to a patient, and an educational case corresponded to a case report or a case-management discussion (5). The epoch examined was 8 September 2018 to 1 March 2019, a total of 25 weeks. Cases submitted via the Collegium mobile app were excluded, i.e., only cases submitted via the web interface were included in the present study.

The time taken to prepare a referral was calculated in two different ways:

An indirect measurement of the referral-preparation time was calculated as the interval between the user logging in, and the referral being submitted to the server; if subsequent referrals were submitted in the same session, they were ignored.

A direct measurement of the referral-preparation time was calculated as the interval between the user opening the referral page and the referral being submitted to the server.

Complex scenarios (see **Appendix**) during which a user allowed the session to time out, for example, were omitted. Thus, the present analysis was restricted to the simple situation where the user logged in, and after carrying out other operations such as reading messages concerning other cases, prepared the referral and submitted it. Note that whether a referral is to be transmitted via a store-and-forward telemedicine system, by plain email, or by instant messaging, the sender will probably need to summarize information from the patient record, obtain copies of clinical images and so forth, prior to initiating the process of electronic referral. Any prior (offline) time of this nature is not the subject of the present work.

A random sample of 1% of referrals were checked manually to confirm the classification into Simple or Complex.

The total size of any files uploaded with each referral was calculated in kByte.

### RESULTS

During the study period, a total of 2,161 clinical cases were submitted. After excluding cases submitted via the Collegium mobile app, there were 1,649 cases which had been submitted via eight different Collegium networks (**Table 1**). Of these, 669 were classified as Simple cases and the referral-preparation times were calculated by both methods.

Indirect measurement. The median referral-preparation time was 1074 s (IQR 584-2035).

Direct measurement. The median referral-preparation time was 888 s (IQR 512-1765).

The median of the differences between the two preparation times was 27 s (IQR 8-146).

Of the 669 cases which were analyzed, 505 cases were submitted with one or more files. In these 505 cases, the median total size of the files uploaded was 1248 kByte (IQR 534-7016). However, there was no association between the size of the files uploaded and the referral preparation time (r = 0.11).

The 669 cases were submitted by a total of 129 different referrers. The referrers submitted from one to 75 cases each (median 2).

### DISCUSSION

In a store-and-forward telemedicine system, referral-preparation time is important. A focus-group study of the opinions of potential users of a Canadian store-and-forward system for use in primary care, found that an important barrier to its use was the length of time required for primary-care practitioners to complete the e-referral (6). Designers therefore need to pay attention to the ease with which referral forms can be completed, especially in low-resource settings where there is unlikely to be an existing electronic medical record system, so that referral data will need to be entered manually. We have observed a tendency for specialists to request more and more detailed referral forms as they seek to deal with new requests in a single interaction. However, as referral forms become more complex, the time required to complete them increases. Clearly there is a tradeoff between very simple forms which are quick to complete but which require several interactions between referrer and specialist, and very detailed forms which are slow and laborious to complete, but which permit a specialist to provide an answer without further requests for information from the referrer.

The present study shows that the median time taken by users to prepare referrals in the Collegium system was 14.8 min, and the median time spent following login until the referral form was opened was 0.5 min. As far as we are aware, there are no published data concerning the operational use of store and forward telemedicine systems to compare this with. In a teledermatology research study, Nami et al. found that the average time required for writing the letter of referral to the GP, taking photographs and uploading the referral was 14.5 min (7). In an earlier teledermatology research study, Berghout et al. found that the average time from the start of a GP consultation to the submission of the teledermatology referral was 11.5 min (8).

The present analysis was restricted to the simple situation where the user logged in, and – after carrying out other operations such as reading messages concerning other cases – prepared the referral and submitted it. Such cases formed about 40% of all referrals submitted during the study period, i.e., users commonly behaved in a more complex manner, allowing their session to timeout for example, and then logging in again to resume it. Thus, the observation that the median time required to prepare a referral was 14.8 min represents a lower bound on submissions generally.


\**Cases submitted via the Collegium mobile app.*

The present study describes two methods of measuring the time required to prepare and submit a referral. The most accurate method is the direct measurement of referral-preparation time, although this requires the telemedicine system to be able to record the time that the user opens the referral form, and to deal with various complexities in user behavior, such as session timeouts and continuations. Use of the direct method in other, non-Collegium telemedicine networks is therefore likely to require system software changes.

In contrast, the indirect method requires only knowledge of when the user logged in and when the referral was submitted. Thus, indirect measurements of referral-preparation time could in principle be made in most telemedicine networks, since login times and referral submission times are normally recorded as a matter of course. As an example, the indirect referral times were measured in the Collegium system before and after a significant software upgrade had been carried out, on 19 July 2018. This upgrade improved the uploading of user files.

Two 24-week periods were compared, the first ending on 30 June 2018 and the second starting on 1 August 2018. During the first period, 1,402 simple cases were submitted and the median (indirect) referral preparation time was 1,144 s. During the second period, following the system upgrade, 1,323 simple cases were submitted and the median time was 1,182 s, see **Figure 1**. There was no significant difference between them (Wilcoxon p = 0.30). Thus, it seems likely that despite an improvement in file uploading, the effect on the overall time required to submit a referral was too small to be observed.

Part of the process of submitting a referral is that data files containing relevant information (e.g., clinical photographs) must be uploaded. In 75% of the cases studied, one or more files were uploaded – these files ranged in size from 8 kByte to 799.8 Mbyte. Uploading very large files, such as DICOM scan datasets, will take several seconds or even minutes, depending on the speed of the user's connection. However, this upload time does not represent a substantial proportion of the total time required to submit a referral, and in the networks studied there was no correlation between the size of the data files uploaded and the time taken to prepare the referrals. That is, the time spent by the user in composing the text of the referral and typing it in dominated the overall preparation time required. This also suggests that improving the referrer's connectivity, with the aim of reducing file upload times, may not necessarily be a good use of scarce resources.

The main study in the present work concerned 669 simple referrals from a total of eight networks. These networks covered both educational (n = 1) and clinical work (n = 7). Three of the clinical networks handled cases of all specialties, and four networks were restricted to specific specialties (radiology,

FIGURE 1 | Median referral times in the Collegium system during two 24-week periods immediately before and after a significant system upgrade. The ordinate is shown on a log scale. The lower boundary of the box indicates the 25th percentile, the line within the box marks the median, and the upper boundary of the box indicates the 75th percentile. The whiskers (error bars) above and below the box indicate the 90 and 10th percentiles.

and one educational). Three of the clinical networks handled cases of all specialties, and four networks were restricted to specific specialties (radiology, dermatology, and leprosy). The error bars above and below the median indicate the 75 and 25th percentiles.

dermatology and leprosy). The median referral-preparation times in these networks were broadly similar, see **Figure 2**. A one-way analysis of variance suggested that the between-network effect was marginally significant (F = 2.4, P < 0.02). Further work is required, using more extensive datasets.

Given a sufficiently large dataset from which referralpreparation times can be extracted, it would be possible to answer a number of subsidiary questions about how the telemedicine system was being used. For example, it is likely to be of interest to understand how the referral preparation time depends on each of the following factors:

• The source network and the type of cases handled in the network


### REFERENCES


With an understanding of these factors, and their effect on the referral-preparation time, suitable measures can be employed to improve the efficiency of referrals.

Referral-preparation time is an important, but little understood, factor in the use of a telemedicine system. Quantitative information about this, and other aspects of the user interface, is important not only in the initial system design, but also in its subsequent development.

### DATA AVAILABILITY

All datasets generated for this study are included in the manuscript.

### AUTHOR CONTRIBUTIONS

RW and BO'K conceived and designed the study, contributed to writing the manuscript, and read and approved the submitted version.


**Conflict of Interest Statement:** RW is a director of Collegium Telemedicus, a notfor-profit organization that provides (free) telemedicine services for humanitarian work in low-resource settings; he receives no salary from Collegium Telemedicus. BO'K was employed by company EndZone.IO Ltd.

Copyright © 2019 Wootton and O'Kane. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## APPENDIX Simple Submission


Indirect preparation time = t3 - t1 Direct preparation time = t3 - t2

### Excluded (Complex) Scenarios


# Experience With Store-and-Forward Consultations in Providing Access to Pediatric Endocrine Consultations in Low- and Middle-Income Countries

Julia E. von Oettingen<sup>1</sup> \*, Meghan Craven<sup>2</sup> , Regina Duperval <sup>3</sup> , Florence Sine St. Surin<sup>4</sup> , Ronald Eveillard<sup>5</sup> , Rony Saint Fleur <sup>6</sup> , Guy Van Vliet <sup>7</sup> , Jean-Pierre Chanoine<sup>8</sup> and Renault Louis <sup>9</sup>

<sup>1</sup> McGill University Health Centre Research Institute, Montreal, QC, Canada, <sup>2</sup> Division of Global Health, Boston Children's Hospital, Boston, MA, United States, <sup>3</sup> Department of Pediatrics, Hôpital St. Damien, Port-au-Prince, Haiti, <sup>4</sup> Department of Pediatrics, Hôpital Universitaire La Paix, Port-au-Prince, Haiti, <sup>5</sup> Department of Pediatrics, Hôpital de l'Université d'État d'Haiti, Port-au-Prince, Haiti, <sup>6</sup> Department of Pediatrics, Hôpital Universitaire Justinien, Cap-Haitien, Haiti, <sup>7</sup> Division of Endocrinology, Centre Hospitalier Universitaire Sainte Justine, Montreal, QC, Canada, <sup>8</sup> Department of Pediatrics, British Columbia Children's Hospital, Vancouver, BC, Canada, <sup>9</sup> Department of Pediatrics, Mirebalais University Hospital, Mirebalais,

#### Edited by:

Richard Wootton, University Hospital of North Norway, Norway

#### Reviewed by:

Farhad Fatehi, University of Queensland, Australia Malcolm David Cairns Donaldson, University of Glasgow, United Kingdom

> \*Correspondence: Julia E. von Oettingen julia.vonoettingen@mcgill.ca

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 25 May 2019 Accepted: 10 September 2019 Published: 25 September 2019

#### Citation:

von Oettingen JE, Craven M, Duperval R, Sine St. Surin F, Eveillard R, Saint Fleur R, Van Vliet G, Chanoine J-P and Louis R (2019) Experience With Store-and-Forward Consultations in Providing Access to Pediatric Endocrine Consultations in Low- and Middle-Income Countries. Front. Public Health 7:272. doi: 10.3389/fpubh.2019.00272 Haiti Pediatric specialists are often unavailable in low- and middle-income countries. As

part of multiple professional associations' efforts to improve access to endocrine expertise globally, a pediatric endocrine teleconsultation network was established on a store-and-forward teleconsultation platform to facilitate focused, language-appropriate advice that can be kept for future reference while bypassing real-time video-conferencing, and obviating the need for a scheduled appointment. User information was recorded, and quality statistics on network performance and qualitative evaluation by referring physicians were analyzed. Over a 3-year period, 81 referrers (88% from Haiti) and 13 pediatric endocrinologists registered onto the network and discussed 47 pediatric endocrine cases, exchanging a total of 412 messages for a median of 7 messages (IQR 5, 11) per case. Diagnoses spanned the spectrum of pediatric endocrine disorders. According to referrers, an appropriate expert was consulted and an answer provided sufficiently quickly in 100% of cases. The answer was well-adapted to their environment in 86%, and referrers were able to follow the advice given in 72%. All but one referrer found the advice helpful, it clarified the diagnosis in 88%, assisted with management in 93%, improved patient's symptoms in 77%, improved function in 77%, and was considered cost-saving in 50%. Perceived benefits of the consultations were academic instruction, setting-adapted advice beyond the scope of guidelines or textbooks, and advancement in the diagnostic process. Pediatric endocrine remote store-and-forward consultations in low- and middle-income countries may provide a reasonable alternative to face-to-face visits, providing clinical and educational benefit, and a potential for cost-saving.

Keywords: pediatric endocrinology, childhood diabetes, low-resource setting, teleconsultation, store-and-forward networks

## INTRODUCTION

In most low- and middle-income countries, access to pediatric subspecialty care is severely limited or entirely unavailable. Only a small number of trained pediatric subspecialists, if any, may be residing in-country. Even the most basic subspecialty education for health professionals including nurses, medical students, physicians in training, and in practice may be lacking when no teacher has been available for extended periods of time. As a result, medical conditions which are beyond the scope of most medical practitioners frequently go unrecognized or are mis-diagnosed, leading to potentially preventable morbidity, and mortality.

Telemedicine consultations can offer access access to specialty consultation in high-income countries (1, 2), and in low- and middle-income countries can fill a significant access gap, enabling physicians to care for patients when local subspecialty expertise is unavailable (3). However, reports of the successful use of teleconsultation services for pediatric endocrinology are limited (4, 5) and reports from low- and middle-income countries are not available.

Collegium Telemedicus (www.collegiumtelemedicus.org) is a store-and-forward consultation platform that is available on the web and on mobile devices, offering the opportunity to confidentially discuss clinical cases in a secure manner (6) while avoiding the downsides of informal, undocumented "curbside" consultations. As opposed to real-time consultation, "store-andforward" consultations are asynchronous and imply that the referring physician creates a consultation request that is stored on the platform until forwarded to a consultant (**Figure 1**). This approach allows the practitioner to communicate without the need for a scheduled appointment, circumvents the problem of low-speed internet connection not supporting real-time video-conferencing, allows the practitioner to return to the consultation document as and when required, and facilitates follow-up communication with the specialist as needed for patients requiring ongoing care.

In Haiti, a low-income country with a population of 11 million in Central America where one third of the population is under 15 years old, no pediatric endocrinologist is available. Endocrine disorders such as type 1 diabetes, hypoglycemia and adrenal disorders—all manageable conditions when adequately taken care of—are frequently fatal. Other conditions such as thyroid disease or disorders of sexual development are left untreated, causing significant morbidity. In 2015, the Pediatric Endocrine Society's International Relations Council (PES IRC) began working with Haitian medical schools, pediatric residency programs, local professional associations, the non-governmental organization Zanmi Lasante (Partners in Health), and the Ministry of Public and Population Health to develop the Pediatric Endocrinology Education Program for Haiti (PEEP-H, www.peephaiti.org). In partnership with the European Society for Pediatric Endocrinology (ESPE), a comprehensive training program was developed to establish pediatric endocrinology as a specialty in Haiti (7). The 4 year program consists of an onsite and remote training curriculum provided by a body of francophone pediatric endocrinologists to Haitian health professionals at all levels of training.

From the time of program initiation in 2016, PEEP-H has collaborated with CollegiumTelemedicus to use its platform to offer remote clinical consultation services and support in the diagnosis and management of pediatric patients affected by endocrine conditions in Haiti, while providing case-based teaching and education to the referring physicians. By 2017, access to the PEEP-H telemedicine network was opened up to any referring physician from a low- or middle-income country via a collaboration with the non-governmental organization Global Pediatric Endocrinology and Diabetes (GPED, www. globalpedendo.org).

The objective of this study was to critically assess the use of a pediatric endocrinology telemedicine network and evaluate its impact on clinical care and pediatric endocrine education in Haiti and beyond.

### MATERIALS AND METHODS

### Ethical Considerations

Given the quality improvement nature of the study, ethics review was waived by the Montreal Children's Hospital ethics review board. Referring physicians obtained verbal consent from all families to discuss their child's case with specialist consultants through teleconsultation within the PEEP-H/GPED network.

### Description of the Store-And-Forward Teleconsultation Process

The Collegium Telemedicus platform was used to create a network space for PEEP-H and GPED for the purpose of pediatric endocrine consultation services. Invited referring and specialist physicians were asked to fill in a registration form and their accounts were then authorized by the network coordinator (JO). During this process, users are able to choose their language of communication (English, French, Spanish, or Arabic). **Figure 1** provides an overview of the teleconsultation process: Upon successful login to the platform, referrers first specify the type of consultation (clinical advice for a specific case vs. concerning a group of patients), and the reason for consultation (advice on differential diagnosis, current management, need for transfer to a different level of care, general information on the pathology discussed, or other). They then use a pre-defined generic template (**Table 1**) to create new consult requests. Pictures, video files and documents can be uploaded as attachments as needed. Once their consult form is submitted, an email is automatically generated to alert the coordinator to a new case. The coordinator can then review the consult form and assign it to an appropriate specialist consultant. This allocation generates an email to the consultant prompting them to view and respond to the case in their account. Following this, referring physicians and consultants can communicate via messages on the platform, each message automatically generating an email alerting the receiving physician to a new message. Images, video clips, documents, and teaching materials can be attached to messages.

### Data Collection

Demographic data on participants who sign on to the Collegium Telemedicus system and quality and performance measures are collected as part of the network use (8), are securely stored on the online platform and can be viewed by the network coordinator. For the purpose of this study, we collected Collegium Telemedicus-generated data on project participants, case statistics, and consultation evaluation by the referring physicians.

### Participants

Participants included in the study were referring physicians (pediatricians, generalists, resident physicians) from low- and middle-income countries and licensed pediatric endocrinologists from any country who registered for the PEEP-H/GPED network on the Collegium Telemedicus platform between 03/2016 and 03/2019. Platform specialists were recruited among PEEP-H teaching professors, and by contacting pediatric endocrinologists among GPED members. The platform was advertised to referring users in several ways. In Haiti, PEEP-H program participants were encouraged to register during onsite and remote teaching sessions, and platform information was disseminated to general pediatricians and pediatric residents via email and at conferences offered by the Haitian Medical Association and the Haitian Pediatric Society. GPED has promoted the platform in its newsletters, on its website, and at international pediatric endocrine conferences. Lastly, in 2018 a link to the platform was created on the ESPE e-learning site that offers interactive chapters and case discussions in pediatric endocrinology to users worldwide (www.espe-elearning.org). Registration and platform account activation was overseen by the PEEP-H coordinator. Information was collected on participant's country and city of professional practice, institutional affiliation, level of training, languages comfortable to communicate in for the purpose of medical consultation, and number of cases referred or responded to.

### Case Statistics

For each consult case received through the PEEP-H/GPED network, we collected information on the age and sex of the child, presumed diagnosis, and whether the diagnosis was confirmed. We also obtained information on the country of origin of both referring physician and consultant, on the language the consultation was written in, the number of messages exchanged, and the duration of communication between the referring physician—consultant dyad as defined by the time between the consultation was received and the last message exchanged. We registered each case as coming from a PEEP-H vs. a GPED participant.

### Consultation Evaluation

We recorded monthly statistics on the number of cases received and on time delay (in hours) to allocation, time to consultant response, and referring physician wait time to answer. Allocation delay was defined as the time in hours from reception of a consult from a referring physician to assignment to a pediatric endocrinologist. This reflected the coordinator's performance. Time to consultant response was the time from allocation of

#### TABLE 1 | Consult template.


the consult to first response by the consultant and reflected the consultant's performance. The total wait time to answer was defined as the total time from referring physicians consult request to consultant response to the referring physician and reflected both coordinator and consultant physician performance.

Three weeks after the last message was exchanged between the referring physician and the consultant, an automated evaluation questionnaire containing both quantitative and qualitative questions was sent to each referring physician (**Table 2**). The questionnaire targets domains of sufficiently quick response, well-adapted answer, helpful advice and ability to follow advice, and assesses whether it benefited patient outcome, whether costs for patient and/or the hospital were saved, and whether there was an educational benefit.

### Urgent Advice

Although Collegium Telemedicus offers new consultation alerts via text message, a synchronous text message exchange between users is not currently possible. However, we anticipated that urgent questions for cases such as diabetic ketoacidosis, hypoglycemia, and adrenal crisis would require fast response times and a direct exchange via text message or phone call. We thus established a less formal platform for urgent questions using an independent encrypted text messaging group on WhatsApp in August 2017. Concurrent with their Collegium Telemedicus registration, participants could provide their phone number to be added to the text messaging group. Participants were reminded to send urgent questions without patient identifiers or images, and were encouraged to subsequently submit their full consult via the Collegium Telemedicus platform. One pediatric endocrinologist per week was assigned as coordinator of the text messaging group.

### Statistical Analysis

We used standard descriptive statistics for participant demographic information. We calculated means and standard deviations, median and interquartile ranges, and percentages, as appropriate, for case statistics and the quantitative data of the consultation evaluation. We structured the qualitative evaluation data by reviewing referring physician comments and assigning codes to broad concepts that appeared in the answers. We then identified themes to describe the most common responses to questions, patterns of codes in the answers provided, and areas that merit further exploration.

### RESULTS

### Participants

A total of 81 referrers and 18 pediatric endocrinologists were signed on to the PEEP-H/GPED network, of whom 71 (88%) and 13 (72%), respectively, were part of the PEEP-H network and the remainder were part of the GPED network from countries including Myanmar, Pakistan, Algeria, Egypt, Liberia, Mexico, Greece, Syria, and India (**Table 3**). The majority of referring physicians (n = 72, 88%) were from Haiti, of whom 31 (38%) were resident physicians and 49 (60%) were pediatricians and the most commonly used language of communication was French. Consultant pediatric endocrinologists were mostly from North America, and 16 (89%) were practicing at a University Hospital.

### Cases Statistics

Over a period of 3 years, 47 cases or on average 1.2 cases per month were referred. The rate of referrals has remained constant over time. Thirty-six cases (77%) contained an attachment including patient photographs, results of laboratory investigations or imaging. The majority of cases (38 or 81%) were received from Haiti through the PEEP-H network and were discussed in French (37 or 78%). The remainder 9 cases came from Myanmar (n = 6), Greece (n = 1), and Syria (n = 1) through the GPED network, and they were all discussed in English. Of the referrers who sent consult requests, each sent a median of 1 (IQR 1, 2) case, and consultants responded on average to a median of 3 (IQR 1, 7) cases. The median duration of communication between referring physician and consultant was 20.9 (IQR 20.2, 24.0) days, and a total of 412 messages were exchanged, for a

#### TABLE 2 | CollegiumTelemedicus automatic evaluation questionnaire.


median of 7 messages (IQR 5, 11) per case. The median allocation delay for each case was 13.6 h (IQR 2.5, 37.5), the median time to first response was 11.3 h (IQR 2.1, 40.0), for a median total answer delay of 43.1 h (IQR 15.1, 82.8).

Patient characteristics with reasons for consultation and presumed diagnoses are shown in **Table 4**. The median age of referred children was 2.0 (IQR 0.2, 7.0, range 0–16) years. Twenty-three (49%) were boys, 18 (38%) girls, 5 (11%) had ambiguous genitalia, and in 1 (2%) the sex was unknown.

### Consultation Evaluation

Detailed quality evaluations were available for 18 (38%) of cases.

### Referring Physician's Quality Evaluation

All referring physicians who provided a qualitative response thought that their case was sent to an appropriate expert and that they received an answer sufficiently quickly. Sixteen (89%) thought the answer was well-adapted to their environment and 13 (72%) felt they were able to follow the advice given. Five (28%) felt they were not able to follow the advice and cited as reasons patient loss to follow-up (n = 2), no urgency to treat (n = 1), inability to find suggested treatment modality locally (n = 1), and no reason provided (n = 1). All but one referrer found the advice helpful, and out of the 18 cases that were qualitatively evaluated, the consultation helped to clarify the diagnosis in 16, assisted with management in 17%, improved patient's symptoms in 14, and improved function in 14.

### **Perceived benefits**

Themes that emerged regarding benefits of the consultations were academic instruction, setting-adapted advice beyond the scope of guidelines or textbooks, and advancement in the

#### TABLE 3 | Participant characteristics.


diagnostic process. For example, one pediatrician from Myanmar commented: "We can't do most of the investigation here, as we have to send [them to] Thailand, it's very expensive. [. . . ] I did just urine ketone level and serum insulin in a child with persistent hypoglycemia and high glucose infusion rate. We can't do other investigations. So, this consultation is very useful for me to have a diagnosis confidently." Expressed frustrations were loss to followup and lack of progress in finding a diagnosis.

Fifteen referrers thought the eventual outcome for the patient will be beneficial for the patient. When asked about educational benefit, 14 of 17 respondents thought the reply received was beneficial to their education. Of 18 respondents, only 9 thought that the consultation was cost-saving for the patients or families, and only 7 of 18 respondents thought the consultation was cost-saving for the hospital.

In the qualitative comments, the ability to access specialist care and to make a diagnosis, and parental reassurance surfaced as themes that referrers identified as offsetting any cost. Pediatricians in Haiti commented:". . . it benefits everyone: patients, parents, residents, staff, hospital and, in short: the country."; "Because if we had not had the opinion of a specialist, in the context of our limitations, his case could have been misdiagnosed."

The perceived cost savings that surfaced in the comments were the ability to obtain specific exams, the context-adapted advice, and improved knowledge among health professionals: "We were able to find a quick and accurate interpretation of the hand radiograph used to evaluate bone age. We have the opinion of pediatric endocrinologists, a specialty currently absent in Haiti."

Themes that emerged from the referring physician's additional comments on their experience with the consultation platform were gratitude for the help received, a perceived increase in knowledge, and the ability to make a diagnosis. Pediatricians from Haiti and Myanmar commented: "The case was very interesting: it allowed me to increase my knowledge through the discussions I had with the specialist. Thank you so much.";"It's very useful for both our children and me. It's my first experience in such a case [. . . ] so it's difficult to recheck the lipid level and CK [Creatine Kinase]. [. . . ] The dietary advice is very beneficial."; "That's the first time I manage a case like that. It's very interesting, and I learned a lot."; "I deeply appreciate the time spent on my case and the prompt response I received."

### Urgent Advice

Between August 2017 and April 2019, a total of 129 physicians registered onto the text messaging group and responded to questions concerning 13 cases (**Table 5**), exchanging 325 messages. Ten cases were deemed appropriate for urgent advice, while three were not. The most common reason for seeking advice were disorders of glucose metabolism (diabetes and hypoglycemia), and ambiguous genitalia. The median time to response by a consultant physician was 13.5 min (IQR 5, 56). Six of the thirteen cases were transferred to the store-and-forward consultation platform for further consultation and discussion.

### DISCUSSION

To our knowledge, this is the first study to demonstrate that a store-and-forward telemedicine network can facilitate access to high-quality specialized pediatric endocrine care in low- and middle-income countries for non-urgent disorders. The results of our evaluation show that close to fifty children with often complex endocrine disorders benefited from effective and timely endocrine consultations. Over 80 referring physicians from lowincome countries who were predominantly based at tertiary care centers are now able to efficiently access the opinion and advice from a pediatric endocrinologist in four different languages. In the referring physician's perspective these remote consultations not only improve patient care most of the time, but also provide an important learning opportunity. Analysis of our text messaging network demonstrates that quick, more informal advice can be obtained in urgent situations.

### Global Pediatric Endocrine Teleconsultation

Reports on experience with teleconsultation for pediatric endocrine conditions are scarce and are unavailable from lowand middle-income countries. One Canadian study investigating TABLE 4 | Provisional diagnoses of referred patients by category of suspected endocrine disorder.


(Continued)

#### TABLE 4 | Continued


access to an adult teleconsultation network for patients located in remote areas of Northern Canada showed that 7% of consults were for endocrinology, although details were not available (10). A telepediatric service in Queensland, Australia, reported 13% pediatric diabetes and 3% endocrine consultations, but the report focused on cost assessment and did not provide detail on the pediatric endocrine consultations (4). While there is a clear need and demand for better access to specialty pediatric endocrine consultation even in high-income settings (5), this is even more relevant to low- and middleincome countries who need advice from specialists familiar with the constraints in resources, diagnostics and therapeutics, and who can adapt their advice to the local setting, taking language, and cultural aspects into account. More studies are needed to evaluate the impact of teleconsultation on health care access, risks and benefits to patients and referring providers, quality of consultations, and cost-effectiveness in lowresource settings.

### User Profile and Platform Use

Most referring and consulting physicians on our network are affiliated with a tertiary care university hospital center. The percentage (89%) was almost double as compared to the 45% reported from the Médecins Sans Frontières (MSF) network on the same platform (11). This most likely reflects the subspecialized and academic nature of pediatric endocrinology but may also suggest a selective uptake of platform use by the physician population targeted. Further dissemination of our platform to primary care physicians and pediatricians in offices, clinics, and community hospitals could sway the proportion of consults received toward primary care-based referrals. Taking the higher uptake in Haiti through the PEEP-H activities as an example, our group has liaised with other international on-site teaching events including the ESPE-led schools in Eastern Europe (Winter School), North Africa (Maghreb School), and Central Asia (Caucasus and Central Asia School) so these can act as channels of communication for the recruitment of potential referrers.

Despite a clear need for subspecialty expertise in low-income settings, our teleconsultation services have remained underutilized. Uptake in Haiti was likely higher due to promotion of the platform during onsite teaching activities by PEEP-H professors and the PEEP-H coordinator's dissemination activities. Of ∼300 pediatricians who practice in Haiti 38 (13%) have registered. Most were repeatedly invited, had several opportunities to register, or were provided with inperson or remote help to register online. The registration process per se may be a barrier due to low comfort level with electronic device and online platform use. In contrast, only a small number of international referring physicians have registered, all of whom seemed to be highly motivated pediatricians who were proactively searching for a way to obtain input on their challenging case(s). Our global uptake through GPED may have been less rapid because promotion strategies have not reached our target population. In an attempt to improve our reach we assigned regional point-persons across five continents in 2019 who will work with their local and regional referral network and with pediatric professional organizations to improve platform visibility, access, and use. Work is ongoing to seek information on barriers and facilitators from referring physicians. Observations from two recent reports from Brazil suggest adequate training and encouragement of providers as well as integration of consultations into regular work hours are opportunities to improve teleconsultation use and dissemination (12, 13), which may well apply to our network as well.

Further, while registration to the consultation platform was high in Haiti, the number of cases referred is likely to be below the number of cases that may benefit from consultation. As in many low-income countries, access to specialty consultation in Haiti has previously been severely limited and the culture of seeking advice from a subspecialist has not been established as a routine. Anecdotally, a referral only felt justified to many pediatric providers in Haiti if their own attempts at diagnosis and treatment were unrevealing. They only proceeded with teleconsultation if referral information (history, physical exam, initial diagnostic work-up) was felt to be high-quality and worthy of sending to a specialist. Despite ease of platform use and despite its availability on mobile devices and full function even with slow internet connections (11), referrers may lack consistent computer, mobile


TABLE 5 | Reasons for referral and consultant diagnoses of urgent consultations by endocrine category.

phone or internet access, may not be comfortable typing, and may perceive the time it takes to send a consultation as a barrier. These factors may increase the threshold to seek remote advice.

### Quality Assessment and Cost

Our quality statistics overall reflected a low-frequency but highquality and efficient use of our network. Our answer delay of 43.1 h largely resulted from a higher allocation delay and was longer than that reported in a previous study that used the Collegium Telemedicus platform (14), but similar to the 2-day response time reported from a Canadian network (1). However, most cases in pediatric endocrinology tend to be subacute or chronic, such that a delay of 48 h or less seems quite appropriate. In addition, we offered urgent case discussion via a separate urgent-response but less formal consultation forum where time to response was within less than an hour for all but three cases. Our response rate to detailed feedback requested from referrers after each case was 38%, which is higher than the 12% reported by the largest Collegium Telemedicus supported network and similar to their second largest network (42%) (8). The overall high satisfaction by referrers with impact on clinical care and education mimics evaluations from previous reports (11) but needs to be interpreted with some caution as there is a possibility these evaluations are biased by self-selection of referrers who were particularly satisfied with the consultation process. While we did not formally evaluate consultant experiences in this study, earlier reports and informal follow-up with our consultant specialists revealed insufficient follow-up on consultations (11). Whether the network manager or coordinator could encourage and enhance referrer-consultant interaction to facilitate shorter answer delays and ensure followup needs to be studied. Finally, the up to 50% cost savings reported in our network are closer to 20% in other networks (8). In both cases, however, cost-saving was not measured objectively but rather based on referrer's subjective impression. In addition, as all specialist services and platform use were provided free of charge, reimbursement of specialists, and platform fees were not taken into account. A thorough cost analysis from Australia demonstrated significant cost savings of their teleconsultation system above a certain number of referrals (4).

### Conclusion

In summary, pediatric endocrine remote store-and-forward consultations in low-resource settings result in a productive referrer-consultant communication enhancing and frequently replacing the need for a face-to-face visit. Referring physicians perceive a clinical and educational benefit from these interactions and see a potential for cost-saving. Despite this, global scale-up of network registration and use has been slow and merits investigation into barriers and opportunities for improvement. Further study is needed to formally evaluate the cost-effectiveness, long-term sustainability, and feasibility of integrating teleconsultation into the locally existing healthcare infrastructure.

### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

### ETHICS STATEMENT

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

### AUTHOR CONTRIBUTIONS

JO conceptualized the study, extracted and analyzed the data, and wrote the first draft of the manuscript. MC performed a literature review and revised the manuscript. RL contributed to data collection, analysis and interpretation, and revised the manuscript. The PEEP-H working group (FS, RE, RS GV, JC, RL) critically reviewed the manuscript drafts. All authors approved of the final version of the manuscript.

### REFERENCES


### FUNDING

The PEEP-H program was funded by the Pediatric Endocrine Society and by the Else-Kroener-Fresenius Stiftung (2016- HA160). JO is supported by a Fonds de Recherche Québec Santé clinician-scientist award.

### ACKNOWLEDGMENTS

We would like to thank all PEEP-H participants, professors, and administrators who help make this project happen, and all patients and families who are part of the PEEP-H project. We would also like to thank all PES International Relations Council members and the GPED executive committee members. The PEEP-H program is supported by the Pediatric Endocrine Society, the European Society for Pediatric Endocrinology, Zanmi Lasante, Haiti's participating university hospitals, medical schools, and medical professional organizations.

cross-sectional study and cost analysis of the use of eConsult in Nunavut. Int J Circumpolar Health. (2017) 76:1323493. doi: 10.1080/22423982.2017. 1323493


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MD declared a past co-authorship with one of the authors GV to the handling editor.

Copyright © 2019 von Oettingen, Craven, Duperval, Sine St. Surin, Eveillard, Saint Fleur, Van Vliet, Chanoine and Louis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Managing Epilepsy by Telemedicine in Resource-Poor Settings

#### Victor Patterson\*

*Visiting Neurologist, Nepal Epilepsy Association, Kathmandu, Nepal*

Epilepsy is a common and treatable disease; in rich countries the expectation is that two-thirds of people will have their seizure episodes controlled on medication. In lowand middle-income countries (LMICs) however most people are not on treatment either because no doctors live near them or the logistics of affordable drug supply is absent. People with epilepsy then are prone to the bad effects of this disease—death, disfigurement from accidents and burns, and social problems due to the stigma with which the disease is associated. So this represents a failure of conventional face-to-face medicine. Might a telemedicine approach do better? The World Health Organization has suggested that non-physician health workers are empowered to diagnose and manage epilepsy; to do this they will need considerable medical support, which might be provided by telemedicine through the telephone, smartphone applications or a combination. This paper sets out what telemedicine does at present for people with epilepsy in LMICs and suggests how it might be developed in the future.

#### Edited by:

*Sophie Delaigue, Pan American Health Organization, United States*

#### Reviewed by:

*Larry Kenith Olsen, Logan University, United States Mauricio F. Villamar, Harvard Medical School, United States*

### \*Correspondence:

*Victor Patterson vhp498@gmail.com*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *05 April 2019* Accepted: *18 October 2019* Published: *12 November 2019*

#### Citation:

*Patterson V (2019) Managing Epilepsy by Telemedicine in Resource-Poor Settings. Front. Public Health 7:321. doi: 10.3389/fpubh.2019.00321* Keywords: epilepsy, untreated epilepsy, epilepsy treatment gap, telemedicine, teleneurology, smartphone application, APP, LMICs

## EPILEPSY IN HIGH-INCOME COUNTRIES

Epilepsy affects about one person in every 200 throughout the world so is one of the commonest neurological diseases. It is due to intermittent paroxysms of disordered electrical activity in the brain causing loss or alteration of consciousness and usually a convulsion (these are called epileptic seizures). Although epilepsy can be associated with other structural brain disease most people with it have only epilepsy.

Epilepsy is treatable with medication; about two thirds of people with it (PWE) have no seizures on treatment but the treatment usually needs to be continued even when seizures have stopped. In this respect it is no different from other long-term conditions like diabetes mellitus. In high-income countries doctors are plentiful and accessible and there are usually insurance schemes which cover the costs of the medication. Sadly this isn't the case in low- and middle-income countries (LMICs).

### EPILEPSY IN LMICs

This is where most PWE in the world live−40 million of the total of about 50 million<sup>1</sup> . Here many PWE are not on any treatment for their disease and this gap in treatment ranges from about 30% to over 90% (1–3). The reasons for this are shown in **Table 1**.

Lack of access to doctors is an important cause (2, 3). Most patients in many LMICs live rurally whereas the vast majority of doctors live in towns and cities. Even then epilepsy is regarded as

<sup>1</sup>Available from: https://www.who.int/mental\_health/neurology/epilepsy/en/

Suicide


complicated by most doctors and many doctors will not treat it so PWE will have to go even farther from home for treatment.

For example, in the district of Myagdi in Nepal the principal town, Beni, is 2 days travel from the northern part of the district. But there may be no doctors to treat epilepsy there so that means another half-day travel to Pokhara, the nearest large town. If a neurologist is to be consulted then there will be need for another day's travel to Kathmandu. So this is a return journey of 1 week simply for a consultation. Most people are subsistence farmers and can afford neither the travel costs nor the time away from their land. Instead they will consult the traditional healer who at least will live nearby.

### EFFECTS OF EPILEPSY

Sudden Unexplained Death in Epilepsy (SUDEP)

The effects of epilepsy on individuals and families can be considerable. First epilepsy is a killer disease. PWE are much more likely to die prematurely than unaffected people and death is more likely if people are untreated (4, 5). Extrapolating these figures probably about 250 000 people die each year of epilepsy in the world which is a fairly staggering number. It attracts almost no attention even though it is a much greater number than the death toll from the 2014-16 Ebola outbreak in West Africa which attracted global headlines. This is because epilepsy deaths are diffused both in time and throughout the world. Unlike Ebola they don't occur in clusters and they are not much talked about. The commonest causes of death in which epilepsy is a factor are shown in **Table 2**.

As well as deaths there are two other important consequences of having epilepsy, injuries during a seizure, and stigma. Burns are the commonest serious injury and can have devastating effects especially where burns units are few. Stigma is the reaction which other community members have toward someone with epilepsy which is predicated by their beliefs about the condition. Often this results in PWE and their families being shunned or excluded from school often because the condition is thought to be contagious. This is wellshown in the docu-drama "Juneli" produced by the Nepal Epilepsy Association<sup>2</sup> .

### TIME FOR A RETHINK—A PUBLIC HEALTH APPROACH

Although epilepsy in general is regarded as a neurological problem it might be more useful to regard epilepsy in LMICs as a public health problem, and involve not just the specialists who treat individual cases, be they neurologists, psychiatrists, pediatricians or physicians, but public health doctors as well. This approach would deal with epilepsy in the same way as malaria and HIV/AIDS.

To do this the individual steps for both community and individual management need to be set out, and then ways of dealing with each of them determined. Once these steps are identified then solutions to them can be devised, tested, and funded. This is essentially the approach that has been used in HIV/AIDS. One such scheme, under the acronym AIDARP, is shown in **Table 3**.

Diagnosis, treatment and review are conventionally medical issues and determining how they should be done in the absence of doctors needs an innovative solution. This will have an immediate effect on the burden of epilepsy in a community unlike prevention of obvious causes of epilepsy—birth injury, brain injuries, and infections such as neurocysticercosis—which may take many years to have an effect.

### EMPOWERING NON-PHYSICIAN HEALTH WORKERS

This has been put forward as a solution by the World Health Organization (WHO) in a series of publications culminating in a declaration by the World Health Assembly in 2015<sup>3</sup> . They said ". . . by training non-specialist health care providers in order to provide them with basic knowledge for the management of epilepsy so that epilepsy can be diagnosed, treated, and followed up as much as possible in primary health care settings,. . . "

There are about 10 times more non-physician health workers (NPHWs) than doctors and they live much nearer PWE than doctors so this seems a good idea. But it is a very disruptive approach and one at odds with guidelines in richer countries where even specialists in fields other than epilepsy are discouraged from diagnosing and managing PWE. For example in the UK the influential Scottish Intercollegiate Guidelines Network (SIGN) guideline on

<sup>2</sup>Available from: https://www.youtube.com/watch?v=oLa0rlJGAck

<sup>3</sup>Available from: https://apps.who.int/iris/bitstream/handle/10665/251923/ B136\_R8-en.pdf?sequence=1&isAllowed=y

TABLE 4 | Questions to be answered in the diagnosis and treatment of epilepsy.

Is the episode epileptic or not? If so, is it provoked or symptomatic? What seizure types are present? What is the epilepsy type? What investigations have been done? What treatment is being taken? What is the best treatment?

epilepsy states that "The diagnosis of epilepsy should be made by an epilepsy specialist" 4 . This is completely impractical in LMICs if any progress is to be made to closing the treatment gap.

If NPHWs are going to take on this role they need tools which will makes their diagnosis and management as robust as possible. But first it is important to outline the steps required to diagnose epilepsy.

### STEPS IN DIAGNOSIS AND MANAGEMENT OF EPILEPSY

A number of questions need to be answered when a doctor encounters a patient with possible epilepsy in order to guide management. Those which I ask are shown in **Table 4**.

Of these, the first question about episode diagnose is much the most important. The distinction between epileptic seizures and the conditions which may mimic them is made entirely from the person's story and a description from an eye-witness of the episodes. There is no investigation here which helps and the diagnosis of episodes, even in the best hands, has a definite error rate of up to 18% (6).

This diagnosis is made with a series of questions about what happens before, during and after the episode. Experienced doctors use a Bayesian approach to diagnosis (although usually not consciously)—they start off with the likelihood of someone having epilepsy and then ask a series of questions which increase or decrease that likelihood. Some of these questions are likely to be more useful than others.

Determining whether episodes are epileptic or not usually takes the most time in the history; the other questions are more straightforwardly answered but again by history from the PWE and an eye-witness.

Videoclips of events can be recorded at home by patients' families and taken to a specialist for viewing, particularly in high-income countries where most patients have a smartphone. In my practice in LMICs this does not happen often, usually because patients' phones do not have the facility to capture video.

### A SMARTPHONE APPLICATION FOR EPISODE DIAGNOSIS

LMICs may be poor in many aspects of everyday life but they are generally not poor in access to mobile phone networks, often with data facilities. Most families possess a regular mobile phone and smartphone use is increasing rapidly. Smartphone applications (apps) therefore might be potentially useful in epilepsy care. We have developed an app for NPHWs to use to answer the question of whether an episode is epileptic or not<sup>5</sup> . The algorithm underlying this is based on a study from Nepal which analyzed the responses to 50 routinely-asked questions in a consecutive cohort of 67 patients presenting to an epilepsy clinic, some with epilepsy and some without it (7). It was possible for each question to calculate a Likelihood Ratio (8) of a positive answer indicating epilepsy and therefore to find the questions with the most discriminating likelihood ratios for or against epilepsy. These could then be applied sequentially to the pretest odds of epilepsy in a naïve-Bayesian way to end up with a post-test probability score of the person having an epileptic episode.

This algorithm was then converted into an app—Epilepsy Diagnosis Aid—by a software company (NetProphets Cyberworks Pvt., Noida, India). This presentation had considerable advantages over other presentation methods such as paper, an electronic calculator or a web-platform in that the records could be stored and viewed on an existing personal device which could be used offline, uploaded to an internet server when a connection was available, and downloaded for batch analysis at a later stage.

This app was then validated by NPHWs and inexperienced doctors in 132 patients with the results compared to the gold standard of a face-to-face consultation by an epilepsy specialist (9). Sensitivity was 88% and specificity 100%. The app was shown to be easily-used by 15 computer-naïve village health workers (10). In a further study these health workers used the app to diagnose episodes in 96 patients, both established and newlypresenting; their diagnostic accuracy compared to an epilepsy specialist was 92% compared to 93% obtained by non-specialist doctors (11). Thus, NPHWs can be trained to answer the first question in **Table 4** using this app.

Compared to established paper-based ways of episode diagnosis this app results in far fewer misdiagnoses-−8% as opposed to 25% in the study of Anand (12) which used a pragmatically-derived algorithm. Comparative data from the WHO mhGap algorithm<sup>6</sup> (also available as an app) have not been published. At present the basic version of the Epilepsy Diagnosis Aid app is available free from the Google Playstore and Apple app stores.

A similar tool for children has been described and validated (13, 14). This uses a pragmatic rather than a Bayesian approach but has not been presented as an app.

<sup>4</sup>Available from: https://www.sign.ac.uk/assets/sign143\_2018.pdf

<sup>5</sup>Available from: www.epilepsyapp.org

<sup>6</sup>Available from: https://www.paho.org/mhgap/en/Epilepsy\_flowchart.html? reload

### A COMBINED TELEMEDICINE APPROACH IN NEPAL

But this app on its own does not empower NPHWs to diagnose and manage epilepsy as envisaged by the WHO. It needs something added. In a study from Myagdi, the rural district in Nepal referred to earlier we have done this (15). Here we trained some local villagers without any health background in epilepsy and in using the app. We provided them with some educational materials about epilepsy and sent them back to their villages to educate their communities about epilepsy and offer treatment. When these epilepsy field workers (EFWs -they were not even NPHWs) identified someone with possible epilepsy they were able to use the app and derive a probability score. They then telephoned an epilepsy specialist in Kathmandu who was able to talk to the patient, with the EFW present, and prescribe treatment which the EFW then arranged. The crux of this method is that knowing the app score, and having confidence in it, reduced substantially the time required for history taking and diagnosis by the specialist.

This combined telemedicine approach was judged for the dimensions of quality as defined by the US Institute of Medicine (16)—safety, effectiveness, efficiency, equity, patientcentredness, and timeliness. For safety, PWE were assessed where possible face-to-face by a different epilepsy specialist to look for misdiagnosis and none was found. There was no excess mortality and reported AED side-effects were only 5% (15). Most people had significant reductions in their seizures and most patients were highly satisfied with the service which did not involve them in any significant travel. The epilepsy treatment gap was reduced from 43 to 9%, some patients opting not to take AEDs. It provided much better use of the epilepsy specialist's time.

The advantages of this approach is that the EFWs can contribute to the other public health aspects of epilepsy referred to earlier—awareness, identification, and prevention.

### REVIEW BY TELEPHONE

Of all telemedicine techniques the telephone is perhaps the most potent especially with the advent of the mobile phone which make it both ubiquitous and portable. Its commonest use in epilepsy in both high- and low-income countries is to obtain an eye-witness account of an episode where the patient is in the clinic but the eye-witness is somewhere else. There is little published on this probably because it is so obviously beneficial. The telephone is also used extensively by epilepsy specialists in high-income countries, both nurses and doctors.

The conventional method for reviewing PWE in both parts of the world is to have them come to see the specialist at a clinic, the location of this being at the specialist's convenience rather than that of the PWE. In a recent double-blind study from a large specialist epilepsy clinic in India (17), this approach was compared with telephone review in PWE whose epilepsy was judged stable. The authors found no difference in seizure control or adherence between the two groups but found that the PWE in the telephone group had considerably lower personal costs and were less likely to default from follow-up than the conventionally-managed group.

### EMAIL AND WEBSERVERS

It is possible to exchange information between patients and doctors using email or text messaging and this is a widespread and informal method. But it is not likely to be generally applicable either because many PWE don't feel comfortable with email or their doctors don't want to share their email address.

A more beneficial method, however, is email communication between doctors or NPHWs in LMICs with specialists elsewhere. Again, this is often done informally but there are more formal systems for doing this such as the system run by Swinfen Telemedicine (18),<sup>7</sup> . This uses a webserver rather than email both for reasons of security and ease of record-keeping. This system, and the one run by the charity Médecins Sans Frontières (19), have treated PWE. These systems allow epilepsy specialists in highincome countries to contribute to the care of PWE in LMICs without leaving their offices.

### EEG

The electroencephalogram (EEG) is a system whereby the brain's electrical activity is recorded through the skull. In between episodes this is surprisingly insensitive at diagnosing epilepsy. In a study from Bhutan the sensitivity of standard EEG was only 25% and that of a smartphone-based EEG system only 17% (20). The proper use of EEG is in the evaluation of those PWE whose epilepsy is not responding to AEDs and in whom surgery is being considered. This is not the most pressing problem for epilepsy management in resource-poor countries where up to 90% of PWE are not on any treatment. Another issue with EEGs is the reporting of the record which requires both experience and expertise, qualities not always available in resource-poor countries.

EEGs are now produced in digital format which generates a file which can be uploaded to a server over an internet connection and be reported remotely by an expert. This system has been used in the UK where there is a shortage of doctors qualified to report EEGs (21). The lead author has founded a charity to deliver this service throughout the resource-poor world<sup>8</sup> .

### SMS MESSAGING

Text messaging using short messaging service (SMS) on mobile phones has been used as a way of continuing with epilepsy education in epilepsy patients under review (22). The authors of this study from Malaysia found that knowledge of epilepsy, medication adherence, and review attendance were all better in the group receiving SMS messages compared with a control group, which received conventional written information only.

### VIDEOCONFERENCING

There is a common delusion in many medical circles that telemedicine equals videoconferencing. The problem with

<sup>7</sup>Available at: http://www.swinfencharitabletrust.org/

<sup>8</sup>Available at: https://www.teleeeg.org/

holding this view is that all the modalities mentioned above are disregarded and their potential ignored. Videoconferencing is included here because there is essentially no published work on its use in epilepsy in LMICs simply because organizing videoconferencing at the best of times is a complicated business requiring two doctors and a patient to be in the same place at the same time and the communication between them in terms of internet bandwidth to be high-quality and consistent. The former is difficult to arrange, the latter usually impossible. This is why it doesn't happen very often. In any event, for epilepsy, where the diagnosis is obtained by the history (compared to stroke where it is obtained by examination), a video adds very little.

### FUTURE

It is increasingly clear that, on its own, conventional face-toface medicine delivered by doctors is going to do little for epilepsy care in poorer countries so alternative ways of practice

### REFERENCES


must be found. The use of the plural is important: there is no single way which will improve care everywhere and each local circumstance, in terms of location and of available human input, will require a slightly different approach. But telemedicine, with apps, webservers and the telephone, has the wherewithal to provide this necessary variety of approach; and its full potential is yet to be reached.

### AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and has approved it for publication.

### ACKNOWLEDGMENTS

I would like to thank my co-members of Action Against Untreated Epilepsy—Mamta Singh, Janita Keating, Hemav Rajbhandari, Priya Jain—for many helpful discussions and conversations.


epilepsy: a prospective study. Sci Rep. (2017) 7:45567 doi: 10.1038/srep 45567


**Conflict of Interest:** The author holds the intellectual property rights of an epilepsy diagnosis app.

Copyright © 2019 Patterson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Teledermatology Use in Remote Areas of French Guiana: Experience From a Long-Running System

Anne-Laure Messagier <sup>1</sup> \*, Romain Blaizot 1,2, Pierre Couppié1,2 and Sophie Delaigue<sup>1</sup>

<sup>1</sup> Dermatology Department, Centre Hospitalier Andrée-Rosemon, Cayenne, French Guiana, <sup>2</sup> EA 3593 Ecosystèmes Amazoniens et Pathologies Tropicales, University of French Guiana, Cayenne, French Guiana

Introduction: French Guiana is an overseas region of France on the north coast of South America and is mostly covered by tropical rainforest. Most human settlements are located along the coast while some settlements are scattered across the hinterland. In 2001, the French public health service launched a telemedicine pilot project between the main hospital in Cayenne and remote health centers in French Guiana to tackle healthcare access inequalities. The aim of the present study was to review dermatology cases of the French Guiana telemedicine network to assess the use of telemedicine in dermatology, in order to evaluate its usefulness and propose ways to improve the system.

#### Edited by:

Sunjoo Kang, Yonsei University, South Korea

#### Reviewed by:

Pradeep Nair, Central University of Himachal Pradesh, India Kwuy Im Jung, Kosin University, South Korea

\*Correspondence: Anne-Laure Messagier annelaure.messagier@hotmail.fr

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 23 April 2019 Accepted: 02 December 2019 Published: 19 December 2019

#### Citation:

Messagier A-L, Blaizot R, Couppié P and Delaigue S (2019) Teledermatology Use in Remote Areas of French Guiana: Experience From a Long-Running System. Front. Public Health 7:387. doi: 10.3389/fpubh.2019.00387

Methods: A retrospective study was conducted on all dermatology cases referred between July 2015 and December 2016 through the French Guiana platform. The Model for Assessment of Telemedicine (MAST) methodology was used as recommended by the European Union.

Results: A total of 254 cases were reviewed by dermatologists at Cayenne hospital over the 18-month study period, with a mean of 14 cases per month. All the 16 peripheral health centers used the telemedicine service during the study. In most cases (202/254, 80%), specialists provided a single diagnosis to the referrers. Infectious diseases represented the main reasons for requests (92/202, 46%) including 32% (29/92) of neglected tropical diseases like leprosy and cutaneous leishmaniasis. A total of 39% (100/258) peripheral centers answered the end-users' survey, and more than 85% found the answer delay was fast, the service useful and with an educational benefit. Overall, the accuracy of the diagnosis increased with the quality of the pictures provided, though the latter was good in only 60% (75/125) of the cases. Most patients for whom a teleconsultations has been required (234/254, 92%) have been managed in the peripheral health centers, while referring the patient to Cayenne was necessary for only 20/254 (8%).

Conclusion: The telemedicine system in French Guiana appears to be an interesting solution to the lack of specialists and allowed a better access to specialized dermatology care for people living in the remote areas of this region.

Keywords: telemedicine, teledermatology, French Guiana, remote area, neglected tropical diseases, long-running system

## INTRODUCTION

French Guiana is a department of 83,534 km² located on the north coast of South America whose area is close to Portugal. Ninety-five percent of the area is covered by rainforest. Most of its 280,000-inhabitants population is settled on the coast. The road network is formed by a main axis linking Brazil to Suriname via coastal cities, the interior of the land is only accessible by plane or by boat. For a substantial part of the population, health care access is a real struggle. In 2001, to alleviate this inequality and ease the access of isolated populations to specialists, the introduction of telemedicine was contemplated, through a partnership between the Cayenne Hospital, the National Center for Space Studies (CNES) and Institute of Medicine and Spatial Physiology (MEDES). The first medical specialties of the project were dermatology, parasitology, and cardiology. Gradually, every single medical specialty of the Cayenne Hospital was represented to make available specialized advice to the 16 Delocalized Centers for Prevention and Care (CDPS) spread all over the remote areas of the territory. In dermatology, since diagnosis is mostly visual, telemedicine use was a good approach. In tropical areas, skin diseases are a common reason for consultation in general practice. Among the most frequent medical conditions are cosmopolitan infectious skin disease but also less known tropical pathologies such as leprosy and cutaneous leishmaniasis for whom early diagnosis and treatment can critically help in limiting disease-related complications. In remote areas, these diagnosis represent a real challenge for non-specialists healthcare practitioners.

The main objective of this study was as follows. First, to assessing the quality of service for dermatology telemedicine service performed at the Delocalized Centers for Prevention

and Care (CDPS) in French Guiana from July 1, 2015 to December 31, 2016. Second, to evaluate the usefulness of the telemedicine service system for referrer who worked at the Delocalized Centers for Prevention and Care (CDPS) in 2016 and 2017. Third, to propose ways to improve the telemedicine system.

### MATERIALS AND METHODS

### General Principles

All dermatology cases received between July 1, 2015 and December 31, 2016 were retrospectively studied. Data were collected manually by a resident of the dermatology department in an Excel file from the dermatology requests on the Lotus Notes software (IBM, Armonk, USA). We also conducted a short satisfaction survey of referrers who worked in CDPS between 2016 and 2017. The data collected was then subjected to a descriptive statistical analysis. The following functions were mainly used: average, median, percentage.

### The Telemedicine System in French Guiana

French Guiana owns three hospitals, all located on the coastal cities of Cayenne, Saint-Laurent du Maroni and Kourou. The rest of the territory is covered by 16 health centers (CDPS) (**Figure 1**). These centers are administrative subdivisions of the Cayenne Hospital, only eight of them have physicians and medical examination possibility is very limited locally. Dermatologists in French Guiana aren't numerous: four private dermatologists, one dermatologist at the penitentiary center and Cayenne hospital is the only center benefiting from a dermatology department with three dermatologists currently working there. On-field missions to Maripasoula and Saint-Georges are performed by dermatologists of the Cayenne hospital on a monthly basis. They also provide telemedicine consultations for all CDPS in the territory as part of their usual function. Each CDPS is equipped with a computer with access to the telemedicine computer software, they also have a digital camera and internet connection. The request form includes a specific medical observation according to the specialists they reach out to and they can attach additional documents (such as photographs). It is then automatically sent to the specialist according to the request form chosen. The dermatologist based in Cayenne does not receive an alert when a new dermatology request form is sent. They would regularly check the telemedicine platform from the dermatology department, and they answer directly to referrer. Telemedicine is also available in partnership with Martinique for some medical specialty that are not represented in French Guiana, such as neuroimaging and neurosurgery.

### The MAST Evaluation System (Model for Assessment of Telemedicine Applications)

The different evaluation indicators of the French Guiana telemedicine system for dermatology were organized according to the MAST evaluation grid. The MAST methodology is a multidisciplinary approach to evaluating telemedicine projects recommended at European level.


Legal, socio-cultural, and ethical aspects There is no system for collecting patient consent on the platform

<sup>∗</sup>Corresponds to the dermatology residential student in charge of the study.

∗∗The list of professionals who worked in the health centers in 2017 was provided by the Cayenne Hospital. The survey was accessible via software hosted on the web: https://www.surveymonkey.com.

∗∗∗No radiology available on site, only a "descrambling" ultrasound system is available, no delocalized biology only rapid tests (CRP, TROD). Material for microbiological specimens and skin biopsies is available.

## RESULTS


### Health Problems and Characteristics of the Demand for Care

During the 18 months of study, 254 dermatology cases were analyzed into detail. The Maripasoula CDPS was the lead referrer for dermatology with 26% (65/254) of requests (**Figure 2**). In their answers, specialists proposed a single diagnosis in 80% (202/254) of cases. In decreasing order of frequency, infectious (92/202, 46%) and then inflammatory (85/202, 42%) dermatoses occurred. Neglected tropical disease accounted for 11% (29/254) of all cases including 17 cases of leishmaniasis and 7 cases of leprosy (three paucibacillary, three multibacillary, one type 1 reversal reaction) diagnosed clinically. Examples of treated cases are illustrated in **Figures 3**–**8**. Most cases were acute cases (148/254, 58%), we haven't notice any chronic patients with a follow up of their condition through the platform. On average, requests had more than three photos, and 94% of cases contained at least one image.

### Quality and Safety

When filled by specialists, the 95% certainty concerned 46% (42/92) of the answered. The diagnostic certainty was higher when the quality of the photograph was judged to be good by the specialist (**Figure 9**). There was no difference in diagnostic certainty depending on whether the suspected pathology was inflammatory or infectious. The median time to answer dermatology cases was 1 day and 12 min. Health care practitioners from peripheral centers reported having used telemedicine for 71% of them, and more than 80% are satisfied with the response time.

### Clinical Effectiveness

When the referrer and specialists had proposed a single diagnosis, the concordance rate was 55% corresponding to a similar diagnosis by the two correspondents. Following the diagnosis suspected, the specialist mostly recommended no further examination (168/254, 66%), then a cutaneous biopsy (44/254, 17%). Also, most of cases required a topical treatment (87/254, 34%). According to the satisfaction survey, more than


Standardized diagnosis (number of cases and proportion) Total: 254 (100%)


\*Neglected tropical diseases.

\*\*Also called Caripito itch or "papillonite" in French Guiana.

90% considered teleexpertise as very useful with an improvement in their personal knowledge.

### Economic Aspect

Among the 254 dermatology cases, only 8% (20/254) required face-to-face specialized advice at the expertise center in Cayenne with varying degrees of urgency. Two required emergency medical evacuation (one machete surinfected wound and one deep wound with myiasis). An evacuation involved one of the three patients assessed as very urgent by the referrer. The others required a planned hospitalization or specialized hospital consultation. More than half of the telemedicine users mentioned a decrease in the number of trips for the patient as well as a decrease in health care system expenses thanks to the telemedicine use.

### Organizational Aspect

All the CDPS used the system during the study period. One specialist was mostly involved into the dermatology telemedicine program, as he answered 73% (186/254) of all cases. Referrers

FIGURE 3 | Suspected bacterial infection in a young adult, resistant to antibiotic therapy, considered very urgent by the referral. After expertise: probable leishmaniasis with need to carry out parasitological samples (PCR, cultures) for species identification, then start treatment with Pentamidine.

from peripheral health centers were mostly physicians (177/254, 70%). The majority of cases were considered non-urgent by the referrer (141/190, 74%). Most patients required a single consultation with one question from the referrer and a single answer from the specialist. Few patients required a follow-up of the case with more questions or a second request at a later stage (17/254, 7%). Generally, the referrers rarely gave (56%) follow-up patient informations to the specialist.

### DISCUSSION

The teledermatology system worked regularly during the 18 months of the study allowing permanent support by the center of expertise in dermatology, to non-specialist health workers from peripheral centers, as part of the activities of the service. With an average of 14 cases per month, all CDPS used the telemedicine

FIGURE 5 | Suspicion of borderline leprosy with type 1 reversal reaction, requiring hospitalization.

FIGURE 6 | Tungiasis in a febrile patient. Confirmation of diagnosis and introduction of general antibiotic therapy in the peripheral centers (in case of clinical arguments for a dermohypodermitis complication) associated with a manual scalpel extraction of the fleas.

system for dermatology, some more than others. The system was reliable and effective for managing the vast majority of CDPS patients. This system is probably under-used by paramedical staff. Indeed the majority of requests come from physicians in peripheral centers, whereas the permanence of care is mainly represented by nurses in certain areas. The main weak points were the lack of information on patient's treatment follow-up and on clinical observations as well as the large number of photographs of insufficient quality.

FIGURE 7 | Bullous impetigo in a child, treated in the peripheral center by a general and local antibiotic because of the diffuse impairment.

FIGURE 8 | Suspicion of "papillonite" in a patient with a very itchy popular rash of the trunc and limbs.

### The Limits of the Study

Dermatology has been available since the launching of the system in 2001. Between 2001 and 2016, there has been an overall trend toward an increase in the use of teleservice in dermatology with a total of 1,790 applications and 112 requests per year in average. Dermatology is the most requested specialty since 2003 (radiology requests excluded). However, the study focused on a 18 months period. One reason was that the system has not been designed to extract data easily and it could have been challenging to study in details all the indicators of the MAST. Also, the retrospective nature generates biases imposing that lead us to be cautious about the interpretation of the results of this study. In fact, the feedback survey was not filled out prospectively after each request. Physicians and nurses working in CDPS in 2017 were contacted to answer the evaluation survey in order to limit the memory bias.

An objective method of evaluating changes in practice after regular use of the system would have made it possible to evaluate the evolution of referrers' knowledge of dermatology (diagnostic concordance) and the quality of the requests. Such a study was hardly feasible in the face of the frequent turn over in teams working in CDPS. The feedback survey response rate of 38.8% remained low. The Médecins Sans Frontières telemedicine system, which has been operating for 7 years, has been regularly evaluated by feedback surveys from users with comparable rate around 30% (1, 2).

### A Reliable, Efficient, Accessible, and Secure System Responding to the Needs of the Territory Marked by a Lack of Doctors Needs

#### With two times less generalists and four times less specialists the medical density in French Guiana is lower than the metropolis and is concentrated on the coast. General practitioners would need the advice of a dermatologist in 25% of consultations for a dermatological reason<sup>1</sup> . Skin diseases are a very common reason for consultation in tropical settings and the infectious dermatoses play an important role as underlined by our study (46% of the cases). These often involve common pyodermalike infectious diseases. However, neglected tropical diseases with specific management must also be detected without delay, such as leishmaniasis, leprosy, and Buruli ulcer. Neglected tropical diseases account for 11% of diagnoses in our study. The diagnosis and the fast management of some of these pathologies is an issue for the patient for whom an early diagnosis will limit complications and disabilities and a public health issue to reduce morbidity and limit transmission. There is little telemedicine experience for neglected tropical diseases in the literature. Brazilian study assesses the relevance of telemedicine for confirming a diagnosis of distant leprosy compared to a faceto-face examination. The specificity was 78% suggesting that telemedicine could be a useful method of diagnostic assistance to control this pathology, which remains a public health problem in some countries (3).

<sup>1</sup> Stéphanie Avogadro-Leroy, "Pathologies cutanées en médecine générale : une étude quantitative en Haute-Normandie," December 2012. https://dumas.ccsd. cnrs.fr/dumas-00768337/document.

### A Reliable, Efficient, Accessible, and Secure System

All requests received a response from specialists attesting to the reliability of the system. The median response time was 1 day and 12 min. In France the average time for appointments with a dermatologist is estimated at 64 days<sup>2</sup> . A telemedicine experiment in Hauts-de-France for the detection of cutaneous tumors linking 11 liberal dermatologists and 91 general practitioners had a response time of 3.5 days<sup>3</sup> . The Médecins Sans Frontières telemedicine system with 11 dermatologists around the world had a median time of 10.2 h (1). In Brazil the state of Minas Gerais telemedicine system (Telehealth Network of Minas Gerais TNMG) providing support to small municipalities had a response time for all specialties from 12 to 48 h (4). The French Guiana telemedicine system for dermatology therefore has a response time that seems acceptable. This system is easily accessible in the CPDS, all equipped with a computer with digital camera. On the other hand, this network is secure and facilitates the sharing of medical information.

### Get Expert Center Membership to Support Front-Line Health Workers

The current system relies on the participation of dermatologists from the Cayenne Hospital Department. Its interest based on the territorial expertise is to make sure of the good knowledge of local pathologies (with a black skin experience), the therapeutic possibilities in the field and thus to bring an adapted answer. Dermatologists have integrated telemedicine into their practice without financial incentive or strengthening of teams. The use of the regional network of specialists has might has helped to get appropriate answers to the resources as 60% were very welladapted and 76% of cases didn't require any additional exams and 57% could be treated with local treatment.

### The Majority of Telexpertise Cases Are Managed Locally at CDPS, Thus Limiting Transfers to the Specialized Center

Our study showed a strong participation of the Maripasoula site (26%), this center was also the most demanding center for all specialties since 2014. This is probably related to the fact that it is the most important center in terms of in-patient consultations. The number of teleexpertise represents ∼2.5% compared to the total number of consultations of the center including every medical specialty in 2016.

The small number of patients (8%) who are urgently evacuated or referred for specialized consultation or hospitalization at the Cayenne Hospital Center, suggests that the system has prevented a certain number of face to face consultation (92%) and therefore reduced costs related to these and associated transportation. A Brazilian study estimated at 81% the number of specialized consultations avoided thanks to telemedicine (5). In a literature review of 12 studies using teledermatology, the percentage of avoided travel was 43% (6). A medico-economic study conducted between 2001 and 2010 showed that French Guiana telemedicine network was cost-effective. Technological investments and operating costs were amortized by the savings made on lower transport costs, medical evacuations, hospitalizations, and consultations. Thus, out of 2,121 requests made between 2001 and 2010 it was estimated that the total cost in the absence of telemedicine would have been 1,538,930 Euros in travel expenses, consultations, and hospitalizations. Requests for radiology between 2006 and 2010 resulted in a total saving estimated at 354,000 euros by avoiding 59 medical evacuations to Fort-de-France (Martinique). The total economy was therefore estimated at EUR 1,892,930, while the total budget allocated to the system since 2001 was EUR 1,696,000. The new funding

<sup>2</sup>Médecins : Le Délai d'attente Pour Obtenir Un Rendez-Vous Région Par Région," http://www.lefigaro.fr/conjoncture/2017/03/23/20002-20170323ARTFIG00004 dentiste-orl-opthalmologue-decouvrez-le-delai-d-attente-par-region.php. 3 "Télédermatologie," URPS Médecin libéraux Hauts de France http://www. urpsml-hdf.fr/teledermatologie/.

prospects for telemedicine could promote its development. In fact, in 2018, it entered the common law of medical practices and is now reimbursed by the health insurance<sup>4</sup> .

### Satisfied and Better Trained Users to Manage Dermatoses in Tropical Areas

The feedback survey sent to doctors and nurses working in the CDPS centers in 2017 evaluated the satisfaction of the Guyanese teledermatology system at 7/10. The survey also highlighted an educational benefit for 93% of respondents. The interest of teledermatology in the continuing education of non-specialist doctors has been shown in several studies. In Burkina Faso, an assessment of diagnostic concordance between referrers and specialists over time showed that regular use of telemedicine in dermatology improved the knowledge of users (7).

### System Weaknesses and Improvement Pathways

### Patient Follow-Up Is Rarely Done After Receiving the Specialist's Answer

One of the reason given for 40% of referrers was an ignorance of the need to provide follow-up. When a specialist sends a response, having a follow-up on the relevance, its applicability in the field, the future of the patient and the effectiveness of the treatment are important points. Follow-up does not only maintain the motivation of specialists to respond to requests, it would allow quality and adapted responses. It is also important for evaluating and monitoring the system's quality. A monitoring and evaluation program is not available in all telemedicine systems. The telemedicine system of the state of Minas Gerais in Brazil has developed a quality program including regular assessments of specialist responses (8). A survey was sent to referrers systematically after each expertise and included three questions: Did the consultation avoid addressing the patient in consultation? Did the remote consultation answer your question? How satisfied are you with the current system? Surveys of satisfaction sent systematically allow to ensure system quality since each unsatisfied answer entails a reason evaluation and the installation of correction and adaptation system. In our study, the medical turnover as well as the initial absence of computerized patient medical records on the system have probably been an obstacle to patient follow-up. Quality program and trainings on telemedicine might have an interest to focus on this point.

### The Relevance of the Answers Depends on the Quality's Requests

Our study highlights the importance of the quality of the demand, whose relevance of the answer depends. A request for clarification or technical advice on the taking of photographs by the specialist concerned one out of four requests. Also, 40% of the photographs were judged of medium or bad quality. The diagnosis of the specialist was judged as not certain in 38% of the cases. These results reinforce the importance of completing the request forms exhaustively and providing good quality images. To date, the use of telemedicine is possible in each CDPS center, with general training at the Cayenne Hospital available since 2008 as well as an annual visit of all sites once a year by a physician and a computer scientist. However, the beneficiaries do not have any initial training and good practice recommendations for dermatology. Training is possible annually in the service but not mandatory. Physicians on CDPS sometimes come from outside the territory for a limited time and cannot attend. An online training system could be useful for users with significant turnover. Providing such training could increase the visibility of the system for more optimal use.

### A System Barely Used by Non-medical Health Workers

Telemedicine system for dermatology was mainly used by physicians (70% of requests) and to a lesser extent by nurses (18%). Of the 16 CDPS in French Guiana, eight centers are staffed only by nurses. Experiences in similar contexts where telemedicine is used to obtain specialist advice in front-line health centers show that requests can be sent by nurses. In Brazil, for example, 50% of TNMG requests are made by nurses (5). The telemedicine system in French Guiana is probably underused by paramedical teams.

### Technological Limits

The technology used in French Guiana requires the downloading of a software on a computer. Consulting the telexpertise forms must be done exclusively in the Cayenne dermatology unit and case management is not possible at a distance from a personal computer or a mobile phone. The exchanges are between centers and well-defined specialists who have the software. In the literature, publications on telemedicine are increasingly interested in mobile technologies that open new avenues for improving the supply of health care. There are many mobile apps especially for dermatology. Some may be useful for training. In 2017, there were more than 500 mobile applications in dermatology with a significant growth in telemedicine applications (9). One example is the SkinApp application developed to improve the management of dermatological diseases in front-line health centers. This application aims to present the most common pathologies in tropical context as well as neglected tropical dermatoses that require specific care and without delay. The increasing use of smartphones and the considerable improvement of their image resolution is no longer a limit to their use in teledermatology. Moreover, in France, three-quarters of physicians own a smartphone and more than 9/10 use it for professional purposes<sup>5</sup> and some applications have shown a good sensitivity (98%) in dermatological telexpertise (10). However, many areas in French Guiana still have very poor network and mobile internet coverage<sup>6</sup> which require tools adapted to these condition with

6

<sup>4</sup> "Coup d'envoi du remboursement de la télémédecine : 15 septembre 2018," Medicitus https://www.medicitus.com/coup-denvoi-du-remboursementde-la-telemedecine-15-septembre-2018.

<sup>5</sup>Livre blanc santé connectée, January 2015, https://www.conseil-national. medecin.fr/node/1558.

<sup>&</sup>quot;Mon Réseau Mobile|Arcep," https://www.monreseaumobile.fr.

low connectivity. Protection of personal data, health data and confidentiality are the main issues related to the development of such applications.

### CONCLUSION

Our study shows that teledermatology is useful in the context of French Guiana. First for patients who benefit from the expertise of a dermatologist without delay or displacement. Then for physicians and nurses who have direct access to specialists and benefit from a quick and adapted specialized advice, helping in their medical education. Finally for the healthcare system since the activity of specialists is part of the dermatology department and the technological investments are offset by the reduction of medical evacuations, hospitalizations, and consultations. The vast majority of CDPS patients discussed with the specialist were being treated via telemedicine. Efforts remain to be made on the quality of the requests sent and on patients follow-up, probably via improving users training in telemedicine particularly in the dermatology field.

### REFERENCES


### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

### ETHICS STATEMENT

Ethics permission was not required, because the work was a retrospective chart review conducted by the hospital staff.

### AUTHOR CONTRIBUTIONS

A-LM: article writing. PC and RB: rereading. SD: supervising and rereading.

### ACKNOWLEDGMENTS

We thank Dr. Dominique Sainte-Marie and Dr. Julie Dufour for their support to the telemedicine service, we thank Dr. Sophon Kheng, Dr. Thierry Legwen, and Dr. Paul Brousse for sharing data.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Messagier, Blaizot, Couppié and Delaigue. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Improving Mental Health Care in Developing Countries Through Digital Technologies: A Mini Narrative Review of the Chilean Case

Graciela Rojas 1,2,3,4, Vania Martínez 2,3,5, Pablo Martínez 1,2,3, Pamela Franco3,6 and Álvaro Jiménez-Molina2,3,4 \*

<sup>1</sup> Hospital Clínico, Universidad de Chile, Santiago, Chile, <sup>2</sup> Millennium Nucleus to Improve the Mental Health of Adolescents and Youths (Imhay), Santiago, Chile, <sup>3</sup> Millennium Institute for Research in Depression and Personality (MIDAP), Santiago, Chile, <sup>4</sup> Millennium Nucleus in Social Development (DESOC), Santiago, Chile, <sup>5</sup> Centro de Medicina Reproductiva y Desarrollo Integral del Adolescente (CEMERA), Facultad de Medicina, Universidad de Chile, Santiago, Chile, <sup>6</sup> Doctoral Program in Psychotherapy, Pontificia Universidad Católica de Chile, Santiago, Chile

#### Edited by:

Richard Wootton, University Hospital of North Norway, Norway

#### Reviewed by:

Suvira Ramlall, University of KwaZulu-Natal, South Africa Brian Grady, Minneapolis VA Health Care System, United States

#### \*Correspondence:

Álvaro Jiménez-Molina alvaro.jimenez@imhay.org; alvarojimol@gmail.com

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 09 April 2019 Accepted: 04 December 2019 Published: 20 December 2019

#### Citation:

Rojas G, Martínez V, Martínez P, Franco P and Jiménez-Molina Á (2019) Improving Mental Health Care in Developing Countries Through Digital Technologies: A Mini Narrative Review of the Chilean Case. Front. Public Health 7:391. doi: 10.3389/fpubh.2019.00391 The uneven distribution of mental health resources contributes to the burden of mental disorders in vulnerable groups, especially in developing countries. Internet-based interventions and digital technologies can contribute to reducing the gap between high prevalence of mental disorders, demand for treatment, and access to mental health care, thereby reducing inequities in mental health. This mini review summarizes the current state of the field of e-mental health research in Chile, showing its progress, limitations, and challenges. Internet-based interventions are at an early stage of development in Chile. The interventions included are heterogeneous in terms of participants (e.g., secondary students, patients, healthcare professionals) and contexts (e.g., rural, urban, schools, primary health care), aims, and modalities (e.g., website, online games). While these studies confirmed the feasibility of Internet-based interventions, the shortage of studies on effectiveness and cost-effectiveness makes it difficult to disseminate and scale up these Internet-based programs. However, the growing amount of knowledge accumulated in the Chilean context could guide practices in other developing countries for supporting the mental health of underserved populations.

Keywords: telemedicine, e-mental health, Internet, digital technologies, primary health care, developing countries

### INTRODUCTION

Developing countries are facing the impact of mental health problems while confronted with limited resources and inequities in access to mental health care (1–5). The use of Internet and digital technologies has the potential to address these gaps, facilitating the development of more equitable models of care in a variety of contexts (6–8).

Internet and digital technologies could be a powerful strategy for the delivery of mental health services in low-resource settings. Although Internet-based and digital interventions have shown their potential benefits in developed countries (8–10), there is a lack of studies in developing countries. In particular, randomized controlled trials (RCTs) investigating the effectiveness of Internet-based and digital interventions in developing countries is lacking (11–13). It is, therefore, crucial to closely study e-mental health experiences in developing countries, in order to learn from their successes and limitations.

Chile is a developing Latin American country with a high prevalence of mental disorders and marked socio-economic and geographical inequities in access to health care (14–17). Since the 1990s, Chile has developed community-based mental health services throughout the country to reduce the gap between mental health needs and access to treatment (18). The public health system, which serves approximately 70 per cent of the population including the most at-risk groups in Chile, between the years 2000 and 2010 increased the amount of human resources for specialized mental health care and the number of mental health centers, especially in primary health care (18, 19). In addition, in 2006 Chile introduced a policy that guarantees access to and quality of care, as well as financial protection for prioritized diseases (20), including four mental pathologies (depression in people over the age of 15, bipolar disorder, addictions in people under 20 years of age, and first episode of schizophrenia). In this way, Chile provides one of the first examples of an evidence-based depression intervention being scaled up in resource-constrained settings (21, 22).

However, after 25 years of relatively successful mental health policies, mental health in Chile still presents serious deficiencies and continues to be considered a great challenge for public health (23). There is still a striking imbalance between government spending on mental health and the related disease burden in Chile (4). Thus, the treatment gap remains high: only 38.5% of those with a diagnosis receive any type of mental health service (17), and there are long waiting lists for psychological treatments in primary health care settings (23, 24). This gap is unequally distributed among social strata. Multiple socioeconomic and territorial inequalities persist, with a high concentration of specialized mental health services in the capital city (18).

Digital technologies are highly utilized across all social strata in Chile (25). This high level of digital connectivity (26) has recently facilitated the development of e-mental health interventions in an effort to improve access to treatment for different populations across the country and to utilize preventative methods.

The aim of this article is to provide a description of the use of Internet and digital technology in mental health through a narrative mini-review of studies conducted in Chile. The lessons learned in the Chilean context can inform the delivery of mental health services in low-resource settings with access to Internet and digital technologies.

## METHODS

We conducted a literature search of available source describing the use of Internet and digital technologies in mental health in Chile. Anticipating a small number of randomized clinical trials, we decided to do a narrative mini-review of the literature. A systematic or scoping review method was not used because it would have required greater consideration of intervention effectiveness. A mini-review allowed for greater discussion of important areas in which we believe that Internet and digital technologies could yield considerable gains toward addressing mental disorders in developing countries.

### Procedure and Literature Search Strategy

We searched for published papers, written in English or Spanish, indexed from inception to November 2018 in PubMed, Embase, and SciELO databases. Given the shortage of studies, we searched for study protocols and unpublished data through contact with national experts in the field. Search strategies are detailed in the **Supplementary Material**.

The following study eligibility criteria were applied: (1) articles that relate the use of mental health interventions based on or supported by Internet and digital technologies; (2) feasibility and acceptability studies, pre-post evaluation (with no control group) and RCTs; (3) studies conducted in Chile.

## Study Selection and Data Collection Process

Study selection and data collection were carried out by teams of two reviewers in an independent manner (GR and VM, AJ-M and PF). Study records were compiled and duplicates were removed. The study selection process was documented using the PRISMA Extension for Scoping Reviews (PRISMA-ScR) flow diagram (**Figure 1**) in order to achieve a correct organization of the procedure (27). If the information was insufficiently detailed in the original article, we contacted the corresponding author asking for the preliminary study results. Narrative techniques were the selected approach for data analysis and synthesis, with due emphasis on study characteristics.

## RESULTS

After the full-text eligibility assessment, we found three RCTs (28–30), three acceptability and/or feasibility studies [(31, 32); Martínez et al. under review], and three on-going studies without published results [(33, 34); Irarrázaval et al. under review]. The responsible researcher from three studies (Gaete et al., under review; Martínez et al., under review; Irarrázaval et al., under review) provided details of it and its preliminary results. The main characteristics of each study are presented in **Table 1**.

### Remote Collaborative Depression Care Programs

Rojas et al. (28) tested the feasibility, acceptability, and effectiveness of a remote collaborative depression care (RCDC) intervention for patients living in rural areas. The RCDC methods used Web-based shared electronic health records (SEHR) between primary care teams and a specialized mental health team, remote supervision by psychiatrists through the SEHR and/or telephone, and telephone monitoring of patients. Once a week, the specialized mental health team reviewed data entered into the SEHR by the primary care team, and provided remote assistance by entering suggestions into the platform, and in special cases by giving indications over the telephone. Their results showed that the RCDC program achieved higher user satisfaction (odds ratio [OR] 1.94, 95% CI 1.25–3.00) and better treatment adherence rates (OR 1.81, 95% CI 1.02–3.19) at 6 months compared to usual care. There were no statically significant differences in depressive symptoms between the

RCDC program and usual care, but a trend was observed in favor of the intervention group. Significant differences between groups in favor of the RCDC program were observed at 3 months for mental health-related quality of life (beta 3.11, 95% CI 0.19–6.02).

Martínez et al. (29) tested the feasibility, acceptability, and effectiveness of a RCDC intervention for adolescents with depression. The intervention group received periodic remote supervision by psychiatrists located in Santiago, through SEHR and phone patient monitoring. The SEHR functioned as a discussion forum allowing for specialists to assist primary care teams during diagnostic processes and treatment during the acute phase of disorders through personalized, confidential, and real-time interaction. Phone calls to adolescents and his/her primary caregiver included monitoring of different aspects (e.g., adherence to pharmacological treatment). Primary care clinicians were satisfied with the RCDC intervention, valuing the usefulness of receiving timely specialized support. However, there were no significant differences in depressive symptoms or health-related quality of life between groups. The adolescents in the RCDC intervention group were more satisfied with psychological assistance than those in the enhanced usual care group. Satisfaction with psychological care, in both groups, was related to a significant change in depressive symptomatology at 12-weeks follow-up (beta = −4.3, 95% CI −7.2 to −1.3).

### Prevention Programs for Children and Adolescents

In the study of Gaete et al. (30), researchers from Chile and Finland collaborated on adapting Finland's KiVa anti-bullying program (35) for 4th and 5th grade students in Chile and on evaluating its effectiveness in this new context. KiVa is an evidence-based program to prevent bullying and tackle bullying cases. Students in the full and partial KiVa groups received universal actions and indicated actions. Universal actions focused mainly on preventing bullying and were delivered to the students in lessons lead by trained teachers. The indicated actions were intended to be used when a bullying case emerged and to be managed by a designated KiVa team in the school. Half of the schools in the KiVa condition also received an online game, which had the intention to raise awareness of the role of the group in TABLE 1 |Description of studies.


Improving Mental Health Through Technologies

Rojas et al.


bullying, increase empathy and promote strategies to support victimized peers. The study results showed that KiVa program (with or without the online game) was an effective intervention among 5th graders (initially 4th graders when they enrolled in the study) to reduce self-reported victimization (with game: beta = 0.19 95% CI 0.05 −0.33; without game: beta = −0.28 95% CI −0.43 −0.12) and peer-reported bullying actions (with game: beta = 0.33 95% CI 0.11 −0.55; without game: beta = −0.44 95% CI −0.68 −0.19), but no significant difference was found between the two interventions and the control regarding students reporting being a perpetrator (Gaete et al., under review). The online game did not increase the effectiveness of the intervention. Among 6th graders (initially 5th graders when they enrolled in the study), no significant differences were found between the intervention groups and the control, in all the assessments.

Martínez et al. (under review) conducted a study to evaluate the feasibility and acceptability of a stepped-care Internet-based program for the prevention and early intervention of adolescent depression, called Cuida tu ánimo [Take Care of Your Mood] (CTA). One hundred and ninety-five adolescents from two Chilean schools (n = 146) and two Colombian schools (n = 53) interacted with the program through monitoring and feedback messages, which were delivered every 2 weeks, and a website that allowed them to access psychoeducational content. In addition, the website provided emergency information and allowed users to contact a specialist via online chat appointments. Adolescents with severe depressive symptoms or suicidal risk were invited to participate in an online chat appointment, a phone session, or a face-to-face assessment with a mental health professional. Seventy percent answered at least three monitoring e-mails while participating in the program. The number of responses decreased after the 4th monitoring e-mail. During the 3-month program, three participants accessed an online counseling appointment, one participated in a phone call session, and five had a face-toface assessment. The results showed that implementation of the CTA program pilot were feasible. However, the authors point out that it is necessary to involve adolescents in the design of the intervention in order to encourage and maintain participation.

### Feasibility and Acceptability Studies for Depression Management

Carrasco (31) studied the acceptability of an online adventure video game as a psychotherapeutic tool for female adolescents in psychotherapy for mild or moderate depression. As part of the game design process, the author conducted focus groups with adolescent females. In the game, players follow the story of a female adolescent, who gets involved in interpersonal situations that require psycho-social reasoning. The scoring system provides cues about positive game behavior in the areas of: recognition and modification of negative cognitive bias; interpersonal skills and interpersonal problem solving; and behavioral activation and a healthy lifestyle. Fifteen adolescent females played the game as suggested by their therapists. Patients could play the game as many times as they wanted. The average playtime was 11:57 min (SD = 03:42). Most patients played once, four patients played twice. The majority of patients (n = 11) reported positive acceptability rates. This indicates that most of them valued the game and that they thought that they could obtain mental health-related benefits from playing it. The patient's therapists were also interviewed, and all gave positive feedback on the game and felt the game was useful for various reasons (e.g., that it was possible in post-game sessions to relate elements of it with aspects of the patients' lives).

Espinosa et al. (32) studied the feasibility and acceptability of the Chilean version of the Supportive Monitoring and Disease Management over the Internet program [SUMMIT (36)], which was called ASCENSO in Spanish. This program aims to monitor and support patients after being discharged from depression treatment. The monitoring consisted of an online assessment of symptoms every 2 weeks and automated feedback. In cases where a patient reported severe impairment, the ASCENSO team contacted the patient to explore the need for further professional support. In cases where a patient showed moderate impairment, the patient received a recommendation to access a personalized self-care plan on the website. Patients were able to book an online chat counseling appointment with a psychologist at any time. When the program monitoring displayed severe symptomatology impartment, the patient feedback reminded patients of the online chat counseling service. Although only half of the participants actively used the program, most of them displayed a good level of acceptance. Participants stated that the program was easy to use, helped them to learn about depression, taught them to selfmonitor their mood, and the program was generally regarded as a source of support and as beneficial. However, participants did not use the online chat counseling.

### Study Protocols

The Rojas et al. (33) project aims to study the effectiveness of a technology-assisted training and supervision program to enhance depression management in primary care clinics in Santiago. In order to develop a comprehensive supervision program, trained administrative staff will contact patients from a call center to support treatment adherence, and psychiatrists will supervise the primary care team, using a web-based platform.

The Mascayano et al. (34) project aims to develop and evaluate an online intervention for preventing suicide and improving mental health among adolescents. The program uses a web-based platform and a mobile application to cultivate a virtual community. It includes both informational and interactive features, ranging from suicide prevention strategies (e.g., a chat with a psychologist), to components designed to increase interactions between participants and promote a sense of belonging and connection. During the intervention, two psychologists will monitor the platform, identifying behaviors associated with suicide risk and proceeding to an established emergency protocol. A group of adolescents played a role in the creation of the program and have access to the platform to facilitate discussions.

The Irarrázaval et al. (under review) project aims to implement a telepsychiatry service to enhance the provision of primary care mental health for children and adolescents living in substitute care facilities. Two primary care mental health teams, who provide mental health care to vulnerable children and adolescents, will receive bimonthly videoconference supervision from psychiatrists located at the Faculty of Medicine of the University of Chile over a 6-month period.

### DISCUSSION

This mini-review shows that Chile has presented a slow but progressive development of e-mental health research. Nine studies that used Internet and digital technologies to address mental health are presented: three RCTs, three acceptability and feasibility studies and three on-going studies without published results. The interventions included are heterogeneous in terms of participants and contexts (e.g., adolescents, adults, patients, professionals, rural, urban), aims (e.g., collaborative care treatment, monitoring relapse prevention) and modalities (e.g., website, online games). This demonstrates the diversity of interventions that can be made through digital technologies to meet different needs. Despite this heterogeneity, the majority of the interventions reported in this study are meant to reduce the burden of depression, the most worrying mental health problem in Chile (16).

Studies have prioritized the use of digital technologies to assist vulnerable groups in low-resource settings, especially in primary care. This is especially relevant for low-resource health services located in rural hard-to-reach areas where there is no specialized mental health care nearby. Remote collaborative care and telepsychiatry interventions reviewed in this study show that it is feasible to use technologies for collaborative care in this particular context, with adequate acceptability and satisfaction levels among health care teams and patients.

The reviewed studies are addressing not only adult populations but also children and adolescents, who are in a critical period where many mental health disorders arise. Implementing Internet-based interventions to prevent or treat mental health problems in this age group can be a reasonable way of dealing with the severe scarcity of mental health care resources in developing countries, considering that this age group is familiar with digital technologies. The reviewed studies show that Internet-based interventions are feasible in children and adolescents, with adequate acceptability levels.

Despite the recent growth of this field, the number of conducted and on-going studies is still low when compared to developed countries. Most studies are mainly concentrated on the feasibility and acceptability of the interventions. The few effectiveness studies (RCTs) show that the interventions supported by digital technologies are feasible, but failed to demonstrate their effectiveness in reducing symptomatology. Demonstrating significant differences between interventions and control conditions is hindered by low sample sizes that lead to low statistical power. Therefore, there is a need to develop more effectiveness studies with larger sample sizes and substantial follow-up periods. Likewise, studies need to develop understanding on how interventions work, for whom, and why (mechanisms of change) they work. Given that one of the main arguments for introducing Internet-based interventions is their cost-saving potential, cost-effectiveness evaluations are also of crucial importance to the field (37).

Some of the reviewed studies reported particular concern for the social adaptation and cultural sensitivity of interventions, highlighting that the involvement of participants (patients and providers) in the design of interventions could be crucial for their success. For example, in the Mascayano et al. (34) study, the investigators included peer-adolescents in their research team as "experts by experience" to advise the creation of the intervention. Similarly, Carrasco (31) conducted focus groups with adolescents in order to get information for the design of their video game intervention. The Martínez et al. (under review) study also concluded that it is necessary to involve the target users in the design of the intervention in order to encourage participation.

The studies reviewed consistently show a gap between acceptability and usability/adherence of interventions. In this regard, future studies need to address challenges associated with high attrition in these programs. In the CTA program, in order to improve user retention, the authors suggested adding more personalized and interactive content, and more concrete tasks for users to perform during their interaction with it. Furthermore, in the evaluation of this program, Parada et al. (38) suggested using a persuasive systems design approach. Likewise, in the study of Espinosa et al. (32) participants who had completed the intervention suggested diversifying the monitoring assessments and having the possibility to interact with a therapist from the treatment center where they underwent face-to-face treatment. Both Martínez et al. (under review) and Espinosa et al. (32) suggested enabling peer communication. ASCENSO and CTA studies showed that email does not seem to be a good way to contact and monitor participants (27, 29). A possible way to improve the usability and effectiveness of e-mental health interventions is to facilitate access to the content of programs using mobile devices like smartphones (8, 9).

The case of ASCENSO and CTA reveals that it is probably relevant for participants to know that online personalized contact and support from a mental health professional is available if they require it. Similarly, a newer form of a blended care intervention that combines the strengths of Internet-based interventions and face-to-face therapy is increasingly being applied in mental health care (39). Even though blended care studies have not been conducted in Chile, this format is presented as a good alternative to address the growing need for access to psychological support and treatment for mental disorders in developing countries (13), without disproportionately increasing costs for health services.

A critical issue in mental health research is the gap between what is known about interventions and what is provided in daily care routines. To ensure the scaling-up of the interventions reviewed, it is necessary to produce methodological developments in effectiveness and cost-effectiveness studies, but also in implementation methods (40). Since a key strategy for increasing the use of digital technologies in mental health programs includes the development of translation-focused research, studies need to place greater emphasis on facilitators and (structural and cultural) barriers to the implementation of these technologies in the Chilean health system, especially in primary care services.

Most developing countries do not have sufficient mental health resources or highly trained professionals in mental health. The accelerated growth in mobile and Internet connectivity witnessed in the recent years have boosted the development of the e-mental health field in order to reduce this gap. The lessons learned in the Chilean context can provide local evidence for persuading policymakers and other stakeholders to support Internet-based interventions, which is critical to define them as a priority area for research and to ensure funding to widely disseminate and scale up these interventions. The Chilean case can also inspire initiatives in other developing countries.

## CONCLUSIONS

Internet-based interventions are at an early stage of development in Chile. There are few studies on effectiveness and no studies on cost-effectiveness, which makes it difficult to disseminate and scale up these interventions. However, the growing amount of knowledge accumulated in the Chilean context could guide practices in other developing countries for supporting the mental health of underserved populations.

### AUTHOR CONTRIBUTIONS

GR, VM, PM, and ÁJ-M contributed conception and design of the study. GR, VM, PM, and ÁJ-M organized the database. ÁJ-M and PF wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

### FUNDING

This study was supported by the Millennium Science Initiative of the Ministry of Economy, Development and Tourism, grant Millennium Nucleus to Improve the Mental Health of Adolescents and Youths, Imhay, grant Millennium Nucleus in Social Development, Desoc, and the Fund for Innovation and Competitiveness (FIC) of the Chilean Ministry of Economy, Development and Tourism, through the Millennium Science Initiative, Grant No. IS130005. PF received funding from CONICYT PFCHA/DOCTORADO NACIONAL/2019–21190745. This study was also supported by the Chilean National Fund for Scientific and Technological Development (FONDECYT), project No. 1180224.

### ACKNOWLEDGMENTS

This article is part of the work of the e-mental health network of the University of Chile, U-REDES-C\_2018\_07.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh. 2019.00391/full#supplementary-material

## REFERENCES


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Rojas, Martínez, Martínez, Franco and Jiménez-Molina. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.