# NEW HORIZONS IN HEALTH-PROMOTING TECHNOLOGIES: FROM DEVELOPMENT TO RATIONAL USE

EDITED BY : Luciane Cruz Lopes, Cristiane De Cássia Bergamaschi, Silvio Barberato-Filho, Marcus Tolentino Silva and Brian Godman PUBLISHED IN : Frontiers in Pharmacology

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88966-043-8 DOI 10.3389/978-2-88966-043-8

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# NEW HORIZONS IN HEALTH-PROMOTING TECHNOLOGIES: FROM DEVELOPMENT TO RATIONAL USE

Topic Editors:

Luciane Cruz Lopes, University of Sorocaba, Brazil Cristiane De Cássia Bergamaschi, University of Sorocaba, Brazil Silvio Barberato-Filho, University of Sorocaba, Brazil Marcus Tolentino Silva, University of Sorocaba, Brazil Brian Godman, Karolinska Institutet (KI), Sweden

Citation: Lopes, L. C., De Cássia Bergamaschi, C., Barberato-Filho, S., Silva, M. T., Godman, B., eds. (2020). New Horizons in Health-Promoting Technologies: From Development to Rational Use. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88966-043-8

# Table of Contents

*06 Editorial: New Horizons in Health-Promoting Technologies: From Development to Rational Use*

Luciane Cruz Lopes, Brian Godman, Cristiane De Cássia Bergamaschi, Silvio Barberato-Filho and Marcus Tolentino Silva


Orenzio Soler and Jorge Otávio Maia Barreto


Flávia Campos Barcelos, Guacira Corrêa de Matos, Mario Jorge Sobreira da Silva, Fabrício Alves Barbosa da Silva and Elisangela da Costa Lima


Renata Lima, Fernando Sá Del Fiol and Victor M. Balcão

*86 Development of a Complex Intervention to Improve Adherence to Antidiabetic Medication in Older People Using an Anthropomorphic Virtual Assistant Software*

Isa Brito Félix, Mara Pereira Guerreiro, Afonso Cavaco, Ana Paula Cláudio, Anabela Mendes, João Balsa, Maria Beatriz Carmo, Nuno Pimenta and Adriana Henriques

*99 Efficacy and Safety of Brinzolamide as Add-On to Prostaglandin Analogues or* b*-Blocker for Glaucoma and Ocular Hypertension: A Systematic Review and Meta-Analysis*

Yuanzhi Liu, Junyi Zhao, Xiaoyan Zhong, Qiming Wei and Yilan Huang


Luisa von Zuben Vecoso, Marcus Tolentino Silva, Mariangela Ribeiro Resende, Everton Nunes da Silva and Tais Freire Galvao


Jasleen K. Jolly, Holly Bridge and Robert E. MacLaren

*195 Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances*

M Sanni Ali, Daniel Prieto-Alhambra, Luciane Cruz Lopes, Dandara Ramos, Nivea Bispo, Maria Y. Ichihara, Julia M. Pescarini, Elizabeth Williamson, Rosemeire L. Fiaccone, Mauricio L. Barreto and Liam Smeeth

*214 Administrative Data Linkage in Brazil: Potentials for Health Technology Assessment*

M Sanni Ali, Maria Yury Ichihara, Luciane Cruz Lopes, George C.G. Barbosa, Robespierre Pita, Roberto Perez Carreiro, Djanilson Barbosa dos Santos, Dandara Ramos, Nivea Bispo, Fabiana Raynal, Vania Canuto, Bethania de Araujo Almeida, Rosemeire L. Fiaccone, Marcos E. Barreto, Liam Smeeth and Mauricio L. Barreto

*234 Kawasaki Disease and the Use of the Rotavirus Vaccine in Children: A Systematic Review and Meta-Analysis*

Natália Gibim Mellone, Marcus Tolentino Silva, Mariana Del Grossi Paglia, Luciane Cruz Lopes, Sílvio Barberato-Filho, Fernando de Sá Del Fiol and Cristiane de Cássia Bergamaschi

*245 Pharmacokinetic Aspects of Nanoparticle-in-Matrix Drug Delivery Systems for Oral/Buccal Delivery*

Renata Carvalho Feitosa, Danilo Costa Geraldes, Viviane Lucia Beraldo-de-Araújo, Juliana Souza Ribeiro Costa and Laura Oliveira-Nascimento

*260 Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis*

Clare Yuen Zen Lee, Pairote Chakranon and Shaun Wen Huey Lee

*272 Expression of the Biologically Active Insulin Analog SCI-57 in* Nicotiana Benthamiana

Adriana Muñoz-Talavera, Miguel Ángel Gómez-Lim, Luis A. Salazar-Olivo, Jörg Reinders, Katharina Lim, Abraham Escobedo-Moratilla, Alberto Cristian López-Calleja, María Cristina Islas-Carbajal and Ana Rosa Rincón-Sánchez

*291 Efficacy and Safety in the Continued Treatment With a Biosimilar Drug in Patients Receiving Infliximab: A Systematic Review in the Context of Decision-Making From a Latin-American Country*

Edward Mezones-Holguin, Rocio Violeta Gamboa-Cardenas, Gadwyn Sanchez-Felix, José Chávez-Corrales, Luis Miguel Helguero-Santin, Luis Max Laban Seminario, Paula Alejandra Burela-Prado, Maribel Marilu Castro-Reyes and Fabian Fiestas

*306 Robotic Therapy: Cost, Accuracy, and Times. New Challenges in the Neonatal Intensive Care Unit*

Ilaria Amodeo, Nicola Pesenti, Genny Raffaeli, Gabriele Sorrentino, Alessia Zorz, Silvia Traina, Silvia Magnani, Maria Teresa Russo, Salvatore Muscolo, Laura Plevani, Fabio Mosca and Giacomo Cavallaro

*316 Fusogenic Liposomes Increase the Antimicrobial Activity of Vancomycin Against* Staphylococcus aureus *Biofilm*

Andreia Borges Scriboni, Verônica Muniz Couto, Lígia Nunes de Morais Ribeiro, Irlan Almeida Freires, Francisco Carlos Groppo, Eneida de Paula, Michelle Franz-Montan and Karina Cogo-Müller

*327 Pain and Inflammation Management in Older Adults: A Brazilian Consensus of Potentially Inappropriate Medication and Their Alternative Therapies*

Fabiane Raquel Motter, Sarah Nicole Hilmer and Vera Maria Vieira Paniz

*340 Treatment of Cystitis by Hungarian General Practitioners: A Prospective Observational Study*

Ria Benko, Maria Matuz, Zoltan Juhasz, Julia Bognar, Reka Bordas, Gyongyver Soos, Edit Hajdu and Zoltan Peto

*347 Knowledge Translation for Improving the Care of Deinstitutionalized People With Severe Mental Illness in Health Policy*

Izabela Fulone, Jorge Otavio Maia Barreto, Silvio Barberato-Filho, Marcel Henrique de Carvalho and Luciane Cruz Lopes

# Editorial: New Horizons in Health-Promoting Technologies: From Development to Rational Use

Luciane Cruz Lopes 1\*, Brian Godman2,3,4, Cristiane De Ca´ ssia Bergamaschi <sup>1</sup> , Silvio Barberato-Filho<sup>1</sup> and Marcus Tolentino Silva<sup>1</sup>

<sup>1</sup> Pharmaceutical Science Graduate Course, University of Sorocaba, Sorocaba, Brazil, <sup>2</sup> Karolinska University Hospital Huddinge, Karolinska Institutet (KI), Stockholm, Sweden, <sup>3</sup> Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow, United Kingdom, <sup>4</sup> School of Pharmacy, Sefako Makgatho Health Sciences University, Garankuwa, South Africa

Keywords: health technology assessment, new horizon, rational use of medicines, implementation, drug utilization research

Editorial on the Research Topic

#### New Horizons in Health-Promoting Technologies: From Development to Rational Use

#### Edited by:

Vanda Markovic´-Pekovic´, University of Banja Luka, Bosnia and Herzegovina

#### Reviewed by:

Adina Turcu-Stiolica, University of Medicine and Pharmacy of Craiova, Romania

\*Correspondence:

Luciane Cruz Lopes Luciane.lopes@prof.uniso.br

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 11 June 2020 Accepted: 20 July 2020 Published: 06 August 2020

#### Citation:

Lopes LC, Godman B, Bergamaschi CC, Barberato-Filho S and Silva MT (2020) Editorial: New Horizons in Health-Promoting Technologies: From Development to Rational Use. Front. Pharmacol. 11:1180. doi: 10.3389/fphar.2020.01180 This Research Topic covers 30 articles focusing on recent advancements related to the balance in innovation technology and the rational use of medicine for better decision making.

Independent drug information comes from publications with no conflicts of interest, performed by transparent and robust methods that validate the results that can subsequently be used to improve health outcomes. The publication of studies of high methodological quality helps to identify evidence about the benefits of medicines and technologies that are widely used as well as for those that show substantial variation in their use but without improvement in health outcomes. No less important are the findings that demonstrate which technologies that have proven to be ineffective or have harmful effects, which continue to be used often driven by hype, hope, lack of national guidance, or other pressures including financial pressures. We have seen this with the considerable hype and misinformation surrounding the use of hydroxychloroquine in the treatment of patients with COVID-19.

In a current scenario of limited financial resources, health managers are under pressure, which requires greater efficiency in the use of public money. They need consistent and substantiated information about the benefits and harm of health technologies and their impact on patients and health services to guide future decision making. This Research Topic points out studies that can contribute to this theme, having as a main focus the adequate and rational use of a medicine and other technologies. Rational use of medicine occurs when patients receive the appropriate medicines in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost both to them and their community (WHO, 2004).

Inappropriate use of medicines can be costly and extremely harmful (related illness and deaths), both to the individual and the population as a whole (Holloway and Van Dijk, 2011). Few healthcare systems presently control the inappropriate use of medicines, which in some is due to a lack of consciousness of the size of the problem and its economic and health burden. In others, decisionmakers often lack knowledge of the most cost-effective ways to manage this important problem (Lima et al., 2017) or have a reluctance to instigate demand-side measures to improve appropriate use through pressures from key stakeholder groups (Lopes et al., 2010). Particularly, challenges and barriers to be overcome in low- and middle-income countries are still considerable (Salas et al., 2020).

**6**

Problems related to shortages, especially considering the global context of those considered essential, is a topic that can contribute to the irrational use of medicines. Acosta et al. address this issue through a scoping review in which the authors identified an appreciable number of countries that are introducing legislative actions to address shortages of medicines and discuss interest in international cooperation for their prevention and ways to facilitate actions that provide a timely response.

The responsible use of medicines implies that existing activities, capacities, resources, and key stakeholders are aligned to ensure the rational use of medicines (WHO, 2012). Rational use, including issues of adherence to medicines, is enhanced by the instigation of universal healthcare including the continual availability of a key list of essential medicines (Yamauti et al., 2015; Araujo et al., 2016; Yamauti et al., 2017). This is important particularly in low-income countries where the cost of medicines can account for up to 60% or so of total healthcare expenditure, much of which is out of pocket (Cameron et al., 2009) and should be addressed as part of Sustainable Development Goals.

Interventions, including effective drug monitoring and regulations, and encouraging rational use, have been well described in studies that have addressed, among others, the translation of knowledge through a brief political and deliberative discourse (de Araújo et al.; Fulone et al.). In addition, aspects related to the challenges and impact of control policies were addressed in the study by Ranabhat et al. Some authors have shown that when there is a lack of effective drug control and monitoring, either due to wrong selection of essential drugs for reimbursement or lack of demand-side measures (Godman et al., 2014; Fulone et al., 2016; Osorio-De-Castro et al., 2018) or lack of adherence of official guidelines, health and expenditures (Silveira et al., 2014; De Camargo et al., 2016) may be compromised.

It is worth mentioning that in this scenario, in addition to translating knowledge and control policies (Wettermark et al., 2009), the participation of health professionals in a responsible way to expand access and the responsible use of the medicines, mainly in the vulnerable population, assumes significant relevance (Soler and Barreto; Silva et al.). Observational studies carried out with health professionals showed that clinical practice also needs to be reviewed and improved, suggesting significant gaps in the knowledge of appropriate prescriptions (Benko et al.; Fadare et al.).

A good decision should include society's values, the interests of those potentially involved, and an appreciation of local policies. There is widespread recognition of the need to use information more effectively to inform public health policies, programs, and administrative decisions. To consistently contribute to this theme, some authors of this Research Topic have also included studies that provided important tools to improve health decision-making (Ali et al.; Barcelos et al.) to assist in clinical practice (Fé lix et al.; Motter et al.; Amodeo et al.) and the quality of the methods used in primary studies (Ali et al.).

New technologies are registered based on controlled and randomized clinical trials, in which, in most cases, they fail to capture important safety results that only appear in real-life studies (Lopes et al., 2014; Fulone et al., 2018; De Carmago et al., 2019). In addition, in the case of new cancer medicines, often licensed on the basis of limited information, which can cause concern and wasted resources (Pontes et al., 2020).

This Research Topic brings together interesting features specified for secondary (Andrade et al.; Lee et al.; Liu et al.; Mellone et al.; Mezones-Holguin et al.) and primary studies (Cavalcanti et al.; Gomes et al.) of technologies already used in several countries. However, doubts related to their effectiveness and safety still point out that they need to be better studied in different contexts. Part of the continued growth in health spending is attributable to the increasing production of new technologies and changes in the population's epidemiological profile. Studies that indicate whether a technology is cost-effective contribute to adjustments in health policies adopted. The cost-effectiveness study developed by Vecoso et al. showed that the chemoprophylaxis of influenza A (H1N1) is cost-saving in the Brazilian health system context. On the other hand, a number of Brazilian studies have cast doubt on the inclusion of insulin glargine within public health systems in Brazil given appreciable higher prices that existed versus NPH insulins (Caires De Souza et al., 2014; Marra et al., 2016).

In addition, the rapid emergence of high-priced innovations is another major challenge faced by decision-makers. In order to respond, the identification offuture innovations and trends should be carried out in a comprehensive, systematic, and sustainable manner so that policymakers and other interested parties can respond appropriately and improve the managed entry of new medicines starting with horizon scanning and budgeting activities, followed by funding and reimbursement decisions and subsequently post launch patient-level studies (Malmström et al., 2013; Godman et al., 2015). The comprehensive model in Sweden provides an exemplar to others (Eriksson et al., 2017; Eriksson et al., 2019). Authors have also discussed important aspects related to gaps in gene therapy assessment (Jolly et al.) and new technologies to combat multidrug-resistant bacteria (Lima et al.).

To improve the input from new developing technologies, this Research Topic also brought together interesting studies that show preliminary experiments, in "in vitro" phase, using, among others, biopharmaceutical techniques that modify the pharmaceutical form currently commercialized to increase the permeation of molecules in the skin (Vigato et al.) or through the encapsulation of antibiotics that result in increased antimicrobial effectiveness (Scriboni et al.) addressing concerns with the lack of new antibiotics being developed to address rising resistance concerns. In addition, discussions concerning combined delivery systems based on nanostructures to administer drugs by the oral cavity (Feitosa et al.). Tests with promising agents to treat diabetes (Muñoz-Talavera et al.) or osteoporosis have also been presented (Hou et al.). We will be following up their development.

### AUTHOR CONTRIBUTIONS

LL designed conception and wrote the draft of editorial. BG, CB, SB-F, and MS made substantial contributions, revising it critically for important intellectual content. All authors contributed to the article and approved the submitted version.

## REFERENCES


psoriasis in Brazil. Expert Opin. Drug Saf. 13, 1155–1163. doi: 10.1517/ 14740338.2014.942219


Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a past co-authorship with one of the authors BG.

Copyright © 2020 Lopes, Godman, Bergamaschi, Barberato-Filho and Silva. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Use of Assessment of Chronic Illness Care Technology to Evaluate the Institutional Capacity for HIV/AIDS Management

Andressa Wanneska Martins da Silva<sup>1</sup> \*, Micheline Marie Milward de Azevedo Meiners<sup>2</sup> , Elza Ferreira Noronha<sup>1</sup> and Maria Inês de Toledo<sup>1</sup>

<sup>1</sup> Tropical Medicine, Faculty of Medicine, University of Brasília, Brasília, Brazil, <sup>2</sup> College of Pharmacy, Faculty of Ceilândia, University of Brasília, Brasília, Brazil

#### Edited by:

Marcus Tolentino Silva, Federal University of Amazonas, Brazil

#### Reviewed by:

Alvaro Francisco Lopes Sousa, Universidade NOVA de Lisboa, Portugal Nyenty Franklin Tabe, University of Ngaoundéré, Cameroon

#### \*Correspondence:

Andressa Wanneska Martins da Silva wanneska.andressa@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 28 November 2018 Accepted: 11 February 2019 Published: 27 February 2019

#### Citation:

Silva AWM, Meiners MMMA, Noronha EF and Toledo MI (2019) The Use of Assessment of Chronic Illness Care Technology to Evaluate the Institutional Capacity for HIV/AIDS Management. Front. Pharmacol. 10:165. doi: 10.3389/fphar.2019.00165 The effectiveness of antiretroviral therapy has rendered HIV infection a manageable chronic condition. Currently, the health systems face the challenge of adopting organizational healthcare models capable of ensuring the delivery of comprehensive care. The Chronic Care Model has been reported for its effectiveness, particularly in terms of delivery system design. In this study, the Assessment of Chronic Illness Care (ACIC) questionnaire, a soft technology widely used for other chronic conditions, was employed on a teaching hospital to evaluate healthcare provided to people living with HIV/AIDS. The ACIC technology is a self-explanatory instrument which diagnoses, among the six components of the Chronic Care Model Framework, areas for quality improvements, indicating at the same time, intervention strategies and achievements. These components are healthcare network organization, delivery system design, selfmanagement support, decision support, clinical information systems, and community. From May to October 2014, the tool was applied to the multidisciplinary teamwork at the points of care identified, as well as to the hospital management board. Respondents broadly rated care as basic. A pronounced contrast was observed from evaluation by management board and health professional staff in some components like organization of healthcare and clinical information system. The self-management support and delivery system design were the components best evaluated by the multidisciplinary team. Combined with the array of services offered, the entry points available at the hospital can ensure healthcare comprehensiveness. However, some gaps were detected, precluding the delivery of an effective care. The ACIC was considered an adequate technology to provide knowledge of the gaps, to promote productive discussions and reflections within teams and to indicate actions to achieve improvements on healthcare for people living with HIV/AIDS.

Keywords: Chronic Care Model, ACIC, delivery system design, HIV/AIDS, assessment technology, health evaluation

**Abbreviations:** ACIC, Assessment of Chronic Illness Care; CCM, Chronic Care Model; HUB, Hospital Universitário de Brasília; IPD, infectious and parasitic disease; PLWHA, people living with HIV/AIDS; SUS, Unified Health System.

## INTRODUCTION

fphar-10-00165 February 26, 2019 Time: 16:1 # 2

In 2017, there were an estimated 37 million PLWHA worldwide and 21.7 million people receiving antiretroviral treatment. In Brazil, there were 880 000 registered cases of AIDS, of which more than 100 000 were pregnant women. Preventive measures have been adopted, including post-exposure prophylaxis, testing campaigns, condom distribution to populations at risk, and implementation of national treatment protocols with free provision of drug therapy. At the end of 2017, 87% of PLWHA had been diagnosed and 75% of all diagnosed were already on antiretroviral therapy (Ministério da Saúde [MS], 2017; United Nations Programme on HIV/AIDS [UNAIDS], 2018; World Health Organization [WHO], 2018).

In 1999, Brazil issued the National Policy for Sexually Transmitted Diseases and AIDS (STD/AIDS), containing guidelines and actions for the National Program of STD/AIDS. Objectives, guidelines and priorities were defined from the perspective of the Unified Healthcare System ("Sistema Único de Saúde," SUS) principles – equity, universality, integrality, decentralization and social participation – where the State and society interact in search of health promotion of users. It should be noted that the last three principles sustain SUS, therefore, they must be present in health actions and services (BRASIL, 1999). The National STD/AIDS Program incorporates three coordinated components: (1) Promotion, Protection and Prevention; (2) Diagnosis and Assistance; (3) Institutional Development and Management. Each component is detailed with guidelines, strategies, norms and procedures regarding PLWHA care (BRASIL, 1999).

In 2000, in order to assess the National Policy, the Ministry of Health supported the Qualiaids Research Team to develop and validate a questionnaire, a tool for external assessment based on the Qualiaids Program, as well as its recommendations book, as a monitoring and evaluation mechanism to improve HIV/AIDS, Universidade de São Paulo [USP] (2018). The questionnaire has 84 structure and process indicators and a set of best practice recommendations. The principles and clinical, epidemiological and ethical guidelines of the Program were translated into norms, criteria, indicators and quality standards for the questionnaire elaboration and validation.

Although these efforts have improved prognosis for PLWHA, the challenge to SUS is to adapt the current healthcare model: most PLWHA are retained in the specialized care not being referred to primary care setting. Then, it becomes necessary to change the healthcare model to ensure an effective, comprehensive, multidisciplinary model focused on chronic conditions, aptly integrated with primary healthcare. The traditional model focused in the specialist is unsustainable to the healthcare system (Ministério da Saúde, 2015b).

A global call has been made urging countries to foster research on innovative, optimized management of chronic conditions by healthcare systems, allowing clinical knowledge to be translated to the current healthcare context (World Health Organization [WHO], 2013). The CCM, developed in the United States in the 1990s, identifies six key elements that must function in a coordinated form in order to yield improved healthcare for chronic conditions. These elements are split into two groups: health systems and community (Improving Chronic Illness Care [ICIC], 2018). In **Figure 1**, we present the CCM model, adapted by us to consider national features from SUS settings, including two additional key elements: District health plan and nongovernmental organizations (Mendes, 2011; Moysés et al., 2012).

In the CCM, changes to the health system should address healthcare network organization, delivery system design, self-management support, decision support, and clinical information systems. The model advocates the establishment of partnerships and the use of resources available in the community to implement the intended changes and align these resources with public policies (Wagner et al., 2005; Mendes, 2011). The CCM implementation can be monitored and evaluated with its own innovative health technology, the ACIC questionnaire (**Supplementary Table 1**), which diagnoses the situation revealing the nature and degree of the improvements required, indicating intervention strategies and measuring the progress achieved after the interventions (Bonomi et al., 2002; Schwab et al., 2014).

The impact of the CCM on a variety of chronic diseases has been reported, including asthma, diabetes, and depression (Improving Chronic Illness Care [ICIC], 2018). Only seven studies, however, have been retrieved on the application of the CCM to HIV/AIDS (Goetz et al., 2008; Drabo et al., 2010; Tu et al., 2013; Clarke et al., 2015; Mahomed and Asmall, 2015; Massoud et al., 2015; Berenguer et al., 2018). These studies reported improved access and adherence to antiretroviral therapy, implementation of pertinent interventions, and increased involvement of PLWHA with their own care, resulting in clinical, immunological, and virological gains. Of note, a systematic review already published compiled data from 16 papers using CCM Framework for people living with HIV (Pasricha et al., 2013). This systematic review aimed to assess the effectiveness of decision support and clinical information system interventions, examining the outcomes: immunological/virological, medical, psychosocial, economic measures. However, the instruments applied for this assessment were others than ACIC. Therefore, the ACIC remains an innovative approach for HIV/AIDS management. Since our last review in 2018, only one study had, in fact, employed this questionnaire to evaluate HIV care (Drabo et al., 2010).

The purpose of the present investigation was to apply a validated Brazilian Portuguese version of the ACIC questionnaire to diagnose the capacity of care to PLWHA at a Brazilian teaching hospital, bringing the high importance of the CCM Framework as a technology which helps the improvement of the quality of care.

### MATERIALS AND METHODS

### Study Design, Site and Phases

From May to October 2014, this descriptive study was conducted with the staff of points of care for PLWHA and the management board of a Brazilian teaching hospital – HUB.

The entry points, the points of care, and the delivery system design available for PLWHA were identified by interviewing

nutritionists, psychologists, pharmacists, medical interns, and other healthcare professionals. The interviews followed a script comprising three questions concerning PLWHA treated at the HUB: "What are the patient's entry points?", "To which points of care is the patient assigned?", and "To which services is the patient subsequently referred?" In this study, a point of care was defined as any hospital location where PLWHA have their condition directly treated by health professionals (Mendes, 2011).

The term delivery system design, defined by Wagner et al. (2005) as structured reorientation of available healthcare services and interaction between general practitioners and specialists for achieving comprehensive care, was expressed in our questionnaire as line of care, term used in Brazil to represent the reorientation of care flow combined with the relationships that emerge from this flow and are constructed in the light of comprehensiveness (Malta and Merhy, 2010).

In the subsequent phase, the ACIC questionnaire was applied to healthcare professionals at all points of care and to the hospital's managing board in order to allow comparison of the views held by healthcare professionals and institutional leadership.

### Evaluation Tools and Data Collection

For this investigation was applied an ACIC questionnaire (version 3.5) from MacColl Institute for Health Care Innovation (2010) previously adapted, translated and validated to Brazilian portuguese by Moysés et al. (2012). This validated questionnaire was also adapted by the authors for PLWHA according to the hospital context and terminologies. Since Moysés' paper applied ACIC to a primary care context and our work take place in the specialized care scenario, necessary adjustments were made. Also, the term "...chronic conditions" was replaced by "... people living with HIV or PLWHA" along the ACIC tool (Moysés et al., 2012). The questionnaire was applied on different dates for 90 min, on average, to a group of at least three healthcare professionals at different points of care. The answers expressed the group consensus. Within the managing board, only the general manager completed the questionnaire. The researchers acted as facilitators and refrained from interfering with discussions or responses.

### How Does the ACIC Questionnaire Work?

The ACIC questionnaire is a self-explanatory instrument which diagnoses, among the six components of CCM Framework, areas for quality improvements, indicating at the same time, intervention strategies and achievements. It covers 35 qualitative indicators divided into seven blocks: one block for each component of the model and an integrative block termed Integration of CCM components. These indicators measure processes including technical and interpersonal ones, which can influence the way care is delivered and consequently its success. The interpersonal processes indicators are often ignored because it is not easily available, consisting of limitation

in most assessments of quality of care (Donabedian, 1988). In health system organization, the questionnaire assesses the management changes that are required to establish a proactive leadership, a healthcare and information flow and incentives to providers and PLWHA. These changes tend to integrate and refine work spaces within the organization (Mendes, 2011; Moysés et al., 2012).

In delivery system design, it evaluates how well-defined the tasks are among the health team to ensure a comprehensive and individualized care – adjusted to the social and cultural context of the user. It also measures the system for referral and returnreferral which are accountable for linking the points of care (Mendes, 2011).

In self-management support, it assesses the knowledge and capability of the user, aiming the patient empowerment – to make decisions, to understand the plan of care and treatment goals; additionally, the service provides emotional support and brings the user to the available community resources, for instance, support groups or peer groups (Epping-Jordan et al., 2004; Wagner et al., 2005; Mendes, 2011; Moysés et al., 2012; Improving Chronic Illness Care [ICIC], 2018).

In clinical decision support, the indicators basically focus the use of evidence-based guidelines, training, practical and opportune decisions by the health team, gathering user preferences and health conditions. In addition, the flow of communication between specialists and primary care or interdisciplinary team should improve care (Epping-Jordan et al., 2004; Wagner et al., 2005; Mendes, 2011; Moysés et al., 2012).

In clinical information system, it assesses the system of information, including registries, data of individual patients and populations of patients with specific conditions, as well as provides reminders and feedbacks. It should promote, especially, the exchange of information between the various levels of care, leading to a better coordination of information (Epping-Jordan et al., 2004; Wagner et al., 2005; Mendes, 2011; Moysés et al., 2012; Improving Chronic Illness Care [ICIC], 2018).

In community resources, it evaluates the implementation of intersectoriality for health, the articulations and partnerships with resources that exist in other sectors of public administration (such as education, sports, social assistance), as well as community organizations (clubs, churches, community centers and support groups such as Alcoholics Anonymous, Narcotics Anonymous, among others) (Wagner et al., 2005). Also, it verifies the District Plan of Health about the resources available to the HIV care (Mendes, 2011; Moysés et al., 2012).

Finally, the ACIC assesses the interrelationship among the six elements of the CCM, linking key elements that contribute to desired clinical and functional outcomes with a positive impact on PLWHA quality of life and health organization effectiveness (Improving Chronic Illness Care [ICIC], 2018).

### Criteria for Analysis

The 35 indicators of ACIC are evaluated individually inside of each block. Each indicator measures, on a scale of 0–11, an institution's capacity of care provision for chronic conditions. Scores are grouped into four levels: D (limited, 0–2), C (basic, 3–5), B (reasonably good, 6–8), and A (fully developed, 9–11) (Bonomi et al., 2002). ACIC guidelines were followed to analyze the results—i.e., for each completed questionnaire, the mean value of each CCM component was calculated and the average of these means was assigned to the questionnaire. To evaluate PLWHA care, a global score was calculated as the average value of the means obtained at the points of care and management board. Also, a global mean was obtained for each component based solely on the points of care.

The value of each component was analyzed considering the means obtained for each component and applying stratified analysis to identify items exhibiting deficits or limitations.

The Microsoft Office Excel 2013 software was employed for the construction of graphs and analyses.

### Ethics Statement

This study was carried out in accordance with the recommendations of the Resolution 466/12 of the Brazilian Health Council, Research Ethics Committee of the Universidade de Brasília School of Health Sciences with written informed consent from all subjects. The protocol was approved by the Research Ethics Committee of the Universidade de Brasília School of Health Sciences (permit 278.787).

### RESULTS

### Delivery System Design and Points of Care

"What are the entry points of PLWHA?", "To which points of care are these patients assigned?", and "To which services are these patients subsequently referred?"

The entry points reported by health professionals were three: the hospital's emergency service, outpatient pharmacy, and psychosocial support center (termed "Com-Vivência"). The identified points of PLWHA care were four: the Com-Vivência, the outpatient clinic for IPDs, the outpatient pharmacy, and the inpatient unit for IPDs.

Despite ongoing attempts to certify the HUB as an HIV/AIDS referral center for the Federal District, and particularly for the East Region Healthcare Network, none of the respondents reported return-referrals to other health services. This led us to conclude that patients bear the burden of finding additional healthcare services outside the hospital.

### Assessment of Healthcare to PLWHA at the HUB

The result of ACIC questionnaire at HUB yielded an overall score of 4 (in the 2–5 range), assigning level C (basic) to the hospital's capacity of care delivery to PLWHA. The outpatient pharmacy scored lowest (2, level D: limited capacity) (**Figure 2**).

When analyzing the mean scores of ACIC for each component (**Figure 3**) it was observed that the capacity to employ community resources and policies was rated as basic (mean score 4) by healthcare professionals. The management board acknowledged the importance of the District Healthcare Plan in the care delivery practiced in the HUB. Overall, the coordination between hospital Brasília, 2014.

fphar-10-00165 February 26, 2019 Time: 16:1 # 5

and community resources was regarded as limited. In the view of respondents, the care delivery was not shared between the HUB and community organizations.

A pronounced contrast was observed in the component "Organization of healthcare," rated as basic (mean, 3) by the point of care staff, but as good (mean, 8) by the managing board (**Figure 3**). Health professionals acknowledged the role of organizational leadership in effecting changes in PLWHA care. However, they viewed the organizational goals as unclear and the strategies for improvement as restricted to emergency solutions, which are devised and implemented on a case-by-case basis as problems arise. Incentives and regulations for professionals were not employed for PLWHA management purposes.

Next, the components mean scores were analyzed individually to assess how each indicator contributed to the mentioned results. The capacity for self-management support was deemed good (mean score 6) by health professionals. Particularly, they acknowledged the role played by the psychosocial support center in assisting the healthcare team to empower and provide psychosocial support to PLWHA. In general, all the points of care showed engagement with contexts related to treatment adherence, and commitment to seeking suitable solutions for each individual user.

The capacity for clinical decision support was rated as basic (mean score 5) by health professionals who viewed the involvement of other specialists in PLWHA care as limited. Among the healthcare team, continuing education was pursued either by holding weekly meetings to discuss clinical cases and scientific papers or at the personal initiative of staff members, often without acknowledgment from the managing board. The Ministry of Health rarely provided updates or refresher courses on national clinical guidelines. PLWHA had access to information on clinical guidelines (verbally or in the form of educational materials) only upon request. The Com-Vivência center continually added this information to the strategies for self-management support.

Delivery system design was rated as basic (mean score 5) by health professionals. The Com-Vivência and emergency service were described as the principal entry points. Although PLWHA can use the entire range of services available from the hospital, their delivery system design takes place primarily at the points of care identified, which, however, are not coordinated for multiprofessional teamwork. There is a chief of staff who heads each of these services, but leadership was not clearly perceived by respondents. A medical appointment management system is currently in operation, and periodic appointments with a single specialist are given priority. PLWHA monitoring complies with clinical guidelines or is tailored to the patient's needs, being mostly performed by the outpatient clinic for IPDs and the Com-Vivência center. Programmed care was only available for complications or when requested by users. Because neither referral nor return-referral system is in operation, it has been dealt with in a non-standardized, case-by-case manner. Communication between points of care was poor.

Health professionals assigned the lowest score to the clinical information system, evaluated as limited (mean score 1), since the HUB has no electronic outpatient registry or outpatient medical record system. In fact, each point of care has its own record system—paper-based, except at the inpatient unit for IPDs. The outpatient pharmacy employs an electronic system for drug dispensing control, managed by the Ministry of Health, but does not keep clinical records. Pharmacy staff has access only to data retrievable from medical records or directly informed by users. The healthcare team has standardized a care delivery plan for PLWHA.

The indicators rated as limited (ACIC mean score ≤ 2) are shown in **Figure 4**, based on a stratified analysis of all 35 ACIC indicators.

### DISCUSSION

The ACIC is a comprehensive tool designed to represent poor to optimal healthcare management and support, assessing technical and interpersonal processes which may influence the quality of care (Donabedian, 1988). It may be applied to all chronic conditions or constellations of conditions (Bonomi et al., 2002; Hibbard and Greene, 2013). The proposal of ACIC being applied for HIV/AIDS care in Brazil is innovative and it fulfills an important gap in the assessment of quality of care. Despite of being an external tool for assessment of quality of care, Qualiaids is not able to measure the nuances of interpersonal processes. For this, both questionnaires, ACIC and Qualiaids, could support the improvement of quality of care for PLWHA, as the ACIC tool complements the Qualiaids as a self-assessment tool in the perspective of improving results.

Moreover, the application of ACIC is fast and each indicator facilitates discussion, converging to a consensus. The highest score describes the optimal practice, situating the best position that an organization could reach during the diagnoses, the intervention or the assessment (Bonomi et al., 2002; Moysés et al., 2012). Therefore, this questionnaire quickly highlights which areas of the healthcare need to be improved, delivers guidance along this process and monitors progress over time in order to promote a comprehensive care (Bonomi et al., 2002; Drabo et al., 2010; Schwab et al., 2014).

The ACIC proved useful as a soft technology for the situational diagnosis of healthcare delivered to PLWHA at the teaching hospital in the Brazilian setting. The questionnaire fostered discussions within the healthcare team, encouraging its members to actively seek approaches for improvement.

The self-assessment of the hospital's capacity of care delivery to PLWHA, rated as basic (mean score 4), was the main finding emerging from the ACIC instrument, revealing that several aspects need to be improved for a proper management addressing chronic conditions. A literature survey retrieved a single study that applied the complete ACIC to HIV/AIDS care. The study, by Drabo et al. (2010), comprised three hospitals and eight healthcare centers randomly selected from three districts in Burkina Faso and yielded a mean score of 4, assigning basic capacity to PLWHA care (Drabo et al., 2010). In both studies, healthcare system

organization was rated as basic. In the present study, the organizational goals, strategies to improve healthcare delivery, and regulation and incentives to professionals, all of which were rated lowest, were the indicators that most influenced the global result.

The differences observed in the perceptions held by health professionals and managing board indicate the need for greater transparency in leadership responsibilities and organizational strategies for PLWHA care. Institutional goals and plans were viewed as poorly defined, a feature that can undermine the motivation of health professionals (Wagner et al., 2001). Proactive leaderships, capable of establishing rapport with team members, are associated with a more positive stance in the workplace and greater commitment of staff, translating to consistent, effective changes in care delivery (Wagner et al., 2001; Benzer et al., 2011). Lack of interest and poor commitment of team leaders, absence of committed professionals, and the unavailability of updated information technologies seem to negatively impact CCM implementation. Having contradictory results is intrinsic of ACIC questionnaire, that is why it should be applied periodically (at least once every 6 months), from the perspective that with each new assessment the results will come closer to reality in order to improve quality of care (Improving Chronic Illness Care [ICIC], 2018). Further studies to clarify conditions predictive of CCM success are warranted (Coleman et al., 2009; Davy et al., 2015).

Both studies highlighted low or ineffective use of community resources. In our study an interaction of the health service with the community resources and non-governmental organizations were highly limited. However, Drabo et al. (2010) reported that, despite the absence of formal partnership arrangements, the institutions investigated worked with community-based organizations, promoting joint efforts, with gains for PLWHA.

Using community resources minimizes duplication of effort, reduces healthcare system costs, and raises the quality of care delivered (Wagner et al., 2005; Mendes, 2011).

On the other hand, in both studies the best rating was assigned to self-management support, for its emphasis on providing advice at treatment outset, combined with individual appointments and peer group support. Studies have shown promising results of interventions designed to promote selfmanagement and user empowerment, even when support is provided via telephone calls (Wagner et al., 2001; Damush et al., 2010). Interventions should be user-centered, provide support and health education to enhance the user's ability in the management of their condition. Additionally, psychosocial support to users and their families should be offered (Damush et al., 2010; Malta and Merhy, 2010; Tu et al., 2013).

Both clinical decision support and delivery system design were rated as basic by Drabo et al. (2010) and likewise in the present study. Poor communication between specialists and other professionals, including primary care physicians, characteristic of the current referral and returnreferral system, was perceived as a hurdle to be overcome. A comprehensive care requires smoother communication among all providers to accomplish an individualized therapeutic plan (Barceló et al., 2010; Mendes, 2011).

At the teaching hospital, clinical decision support to PLWHA followed the Clinical protocol and therapeutic guidelines for management of HIV infection, in compliance with recommendations of the Ministry of Health (Ministério da

Saúde, 2015a). In contrast with the obstacles to the provision of antiretroviral therapy reported for Burkina Faso (Drabo et al., 2010), Brazil ensures free access to antiretroviral therapy to all PLWHA, managed through the National Medication Logistics Control System. This contrast reveals a weakness of the ACIC instrument in the evaluation of a crucial measure, the access of PLWHA to drug therapy, since the questionnaire assigned the same score to the two very different policies adopted by each respective country.

While Drabo et al. (2010) reported a lack of health professionals, the present study revealed obstacles in multidisciplinary training and in communication across points of care. The aggregation of pharmacists, nurses, and social workers to the service network and the promotion of communication across points of care as routine are expected to decrease teamwork fragmentation, ultimately allowing the monitoring of users, to meet the needs of this population. Multiprofessional teamwork has been associated with positive functional outcomes in users with chronic conditions (Bodenheimer et al., 2009; Carter et al., 2009).

The clinical information system was critical in both studies. Incomplete and paper-based records not only have a detrimental effect on the management of interventions, but also preclude reliable evaluation of the quality of care delivered, increasing the likelihood of medical error (Hillestad et al., 2005; Chaudhry et al., 2006; Kalogriopoulos et al., 2009). Safety can be increased with the use of electronic clinical records, as well as by employing more low-tech resources such as reminders, alerts, brochures, and letters tailored for users. Electronic prescriptions, combined with ready access to clinical information from different services and points of care, contribute toward comprehensiveness in care delivery. The benefits of clinical information systems have been observed in health promotion efforts and in the prevention of complications and risk factors (Hillestad et al., 2005; Chaudhry et al., 2006; Carter et al., 2009; Kalogriopoulos et al., 2009). However, these systems typically require large investments in material and human resources to become efficient and can often be met with resistance by health professionals. Nonetheless, clinical information systems can be convenient facilitating tools, although insufficient to transform the healthcare system by themselves (Tomasi et al., 2004; Hillestad et al., 2005; Kalogriopoulos et al., 2009).

Some of observed limitations about the ACIC questionnaire should be considered. It is a technology that qualifies but does not describe all the pieces of evidence related to care – as structure, process and outcomes at Donabedian evaluation; in this sense, it must be analyzed according to the local context and other supportive data. Importantly, this tool is not a "step-bystep," detailed evaluation about the care process, but it provides the pillars to reach high quality of care (Bonomi et al., 2002; Drabo et al., 2010). The consensus method is important to gather every opinion and summarize them in only one. However, it could conceal biases because of the opinion coming from a person in leadership role during the process. For this, we stratified datareal to board manager and health professionals. Besides, in the first assessment, we could observe that the teams frequently over- or underestimate the quality of care as a result from the misperception of the care they are providing. However, during the CCM Framework implementation process the teams notice what effective care is and their scores could decrease or increase depending on their recently acquired knowledge. When the capability of comprehensive care increases and teams continue implementing effective changes, these scores tend to be improved. Of note, most studies that applied CCM Framework and ACIC addressed a variety of chronic conditions other than HIV/AIDS (Mendes, 2011; Schwab et al., 2014). Therefore, there is a scarce evidence for HIV assessment with this technology, allowing mild consistency about the strengths and limitations in the tool application.

### CONCLUSION

Despite the limitations, we considered that the ACIC succeeds to evaluate the key components for a comprehensive healthcare, encourages reflection from the healthcare team at the HUB, generates helpful discussions, raises awareness among the professionals overwhelmed with service routines, and indicates goals to be pursued to improve the quality of healthcare for PLWHA.

In summary, the ACIC technology proved useful for the situational diagnosis of healthcare delivery to PLWHA at a teaching hospital in Brazil. ACIC concomitant application with Qualiaids provides interpersonal processes indicators, often disregarded in most assessments, which would improve the PLWHA quality of care. Additional aspects to be explored include the ACIC use in other settings, interventions evaluations and monitoring and the CCM implementation at institutions that provide healthcare improvement to PLWHA.

### AUTHOR CONTRIBUTIONS

AS and MM designed this work, drafted, and reviewed the manuscript. MT and EN reviewed the draft. All authors approved the manuscript for publication and agreed to be accountable for all aspects of this work.

## FUNDING

The investigation was part of the subproject entitled Evaluation of care delivery to and pharmacotherapeutic follow-up of people living with HIV/AIDS treated at the Hospital Universitário de Brasília, funded with resources from the project Improving management, surveillance, prevention, and control of STDs, AIDS, and viral hepatites among drug users, sex workers, inmates, and people living with HIV/AIDS, run by the Brazilian Ministry of Health (Grant No. 01/2013).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019. 00165/full#supplementary-material

## REFERENCES

fphar-10-00165 February 26, 2019 Time: 16:1 # 9


in Medicine and Biology Society, (Piscataway, NJ), 1730–1733. doi: 10.1109/ IEMBS.2009.5333561


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Silva, Meiners, Noronha and Toledo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Community-Level Pharmaceutical Interventions to Reduce the Risks of Polypharmacy in the Elderly: Overview of Systematic Reviews and Economic Evaluations

Orenzio Soler <sup>1</sup> \* and Jorge Otávio Maia Barreto<sup>2</sup>

<sup>1</sup> School of Pharmacy, Health Science Institute, Federal University of Pará, Belém, Brazil, <sup>2</sup> Fiocruz School of Government, Fiocruz Brasília, Osvaldo Cruz Foundation, Brasília, Brazil

Background: Patients over 65 years of age taking multiple medications face several risks, and pharmaceutical interventions can be useful to improve quality of care and reduce those risks. However, there is still no consensus on the effectiveness of these interventions aimed at promoting changes in clinical, epidemiological, economic, and humanistic outcomes for various service delivery, organizational, financial, and implementation-based interventions. The objective of this overview of systematic reviews was to summarize evidence on the effectiveness of community-level pharmaceutical interventions to reduce the risks associated with polypharmacy in the population over 65 years of age.

#### Edited by:

Brian Godman, Karolinska Institutet (KI), Sweden

#### Reviewed by:

Kurt Neumann, Independent Researcher, Hungary Robert L. Lins, Retired, Antwerpen, Belgium

> \*Correspondence: Orenzio Soler orenziosoler@ufpa.br

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 07 December 2018 Accepted: 11 March 2019 Published: 02 April 2019

#### Citation:

Soler O and Barreto JOM (2019) Community-Level Pharmaceutical Interventions to Reduce the Risks of Polypharmacy in the Elderly: Overview of Systematic Reviews and Economic Evaluations. Front. Pharmacol. 10:302. doi: 10.3389/fphar.2019.00302 Method: This overview used a previously described protocol to search for systematic review articles, with and without meta-analysis, and economic evaluations, without any language or time restrictions, including articles published up to May 2018. The following databases were searched: the Cochrane Library, Epistemonikos, Health Evidence, Health Systems Evidence, Virtual Health Library, and Google Scholar. The basic search terms used were "elderly," "polypharmacy," and "pharmaceutical interventions." The findings for outcomes of interest were categorized using a taxonomy for health policies and systems. Equity-related questions were also investigated. The studies were evaluated for methodological quality and produced a narrative synthesis.

Results: A total of 642 records were retrieved: 50 from Health Evidence, 197 from Epistemonikos, 194 from Cochrane, 116 from Health Systems Evidence, and 85 from the Virtual Health Library. Of these, 16 articles were selected: 1 overview of systematic reviews, 12 systematic reviews, and 3 economic evaluations. There is evidence of improvement in clinical, epidemiological, humanistic, and economic outcomes for various types of community-level pharmaceutical interventions, but differences in observed outcomes may be due to study designs, primary study sample sizes, risk of bias, difficulty in aggregating data, heterogeneity of indicators and quality of evidence included in the systematic reviews that were assessed. It is necessary to optimize the methodological designs of future primary and secondary studies.

**18**

Conclusion: Community-level pharmaceutical interventions can improve various clinical, epidemiological, humanistic and economic outcomes and potentially reduce risks associated with polypharmacy in the elderly population.

#### Keywords: elderly, polypharmacy, pharmaceutical interventions, overview, systematic review

### INTRODUCTION

It is estimated that 21% of the world's population will be over 65 years of age by the year 2050. The elderly have complex health needs as they often have multiple comorbidities. An estimated 30% of elderly persons are prescribed 5 to 12 medications (United Nations, 2017). The elderly not only use more medications but also experience physiological changes, i.e., pharmacokinetic and pharmacodynamic changes, that increase the risk of adverse events. Between 10 and 30% of hospitalizations in this population are consequences of drug-related complications, which are potentially avoidable through adequate management (World Health Organization, 2015). The provision of care to this population represents one of the greatest challenges for health systems worldwide.

Community-level care for the elderly can be provided in various types of facilities, including community pharmacies. The terminology used to describe care units for the elderly differs around the world: hospices, long-term care facilities, nursing homes, skilled nursing facilities, and assisted living facilities (Pinto and Von-Simson, 2012). These types of facilities vary with regard to their infrastructure, the profile of the professionals employed, and the type of care offered (partial or full and/or individual or collective).

Elderly persons, particularly those residing in nursing homes, are susceptible to polypharmacy. Polypharmacy is defined as the prescription of multiple drugs to an individual (Duerden et al., 2013). The negative consequences of polypharmacy include prescription errors (PE), potentially inappropriate prescription (PIP), and potentially inappropriate medication (PIM), which can lead to drug-related problems (DRP) and/or drug-related negative outcomes (DNO) such as adverse drug events (ADEs) and/or adverse drug reactions (ADRs). The prevalence of polypharmacy in the elderly is high, although it varies widely depending on the definitions used, the facility type and the geographical location (Santos et al., 2007; Brasil Ministério da Saúde, 2014; Leelakanok et al., 2017).

The differences between "adequate polypharmacy" and "inadequate polypharmacy" are now recognized. "Adequate polypharmacy" occurs when multiple drugs are prescribed to an individual with multiple morbidities, in an evidence-based manner; i.e., the combination of prescribed medications will ensure a good quality of life, improve longevity, and minimize drug toxicity. "Inadequate polypharmacy" occurs when multiple medications are inappropriately prescribed, beyond the clinical needs; that is, when the intended benefit with the drug is not achieved, leading to unnecessary risks and negative health outcomes (Duerden et al., 2013).

Improving the quality of medication prescribing for the elderly also involves reducing the irrational use of medications, leading to better health outcomes. To address this challenge, frameworks for the evaluation of key factors related to the occurrence of inappropriate prescriptions as well as interventions to improve this professional conduct have been developed.

According to the Canadian Agency for Drugs and Technologies in Health (CADTH), there are several types of interventions targeted at various levels and components of health systems (Higgins and Green, 2011):


These interventions require health professionals to analyze the pharmacotherapeutic strategy established for a patient. This is a continuous process that identifies and solves DRP and/or DNO based on need, efficacy and safety, with the goal of increasing effectiveness and decreasing the risks of pharmacotherapy. Examples include therapeutic strategy-related interventions, interventions related to the quantity of drugs and health education interventions (Brasil Ministério da Saúde, 2014). It is hypothesized, although there is no consensus, that professional, organizational, regulatory and financial interventions targeted at prescribers and consumers can be effective for improving the prescription and rational use of medications.

### Goal

This overview investigated the available evidence on the effects of community-level pharmaceutical interventions to reduce the risks associated with polypharmacy in the elderly population over 65 years of age.

### METHODS

### Search Strategy

This overview covered studies published in the following databases: Cochrane Library, Epistemonikos, Health Evidence, Health Systems Evidence, Virtual Health Library (Portuguese acronym: BVS), and Google Scholar. There was no language or time restriction, including articles published up to May 2018. Systematic reviews, with or without meta-analysis, and economic evaluations were included. The search strategy included medical subject headings (MeSH) and health sciences descriptors (DeCS), using the keywords "Elderly," "Polypharmacy," and "Pharmaceutical intervention." The search was adapted to the various electronic databases. Details of the search strategies are provided in **Supplementary Material 1**.

This study addressed the following question: Which community-level pharmaceutical interventions reduce the risks associated with polypharmacy in the elderly population over 65 years of age? In accordance with the PICO guidelines (Santos et al., 2007), studies with the following characteristics were included: Population (P): Individuals over 65 years of age; Intervention (I): Pharmaceutical interventions (pharmaceutical care); Control (C): No pharmaceutical intervention or any other intervention; and Outcome (O): Clinical, epidemiological, humanistic, and economic outcomes.

Studies focused on other age groups, such as adolescents and adults aged between 18 and 64 years, were excluded, along with studies that addressed interventions at other levels of care. The interventions of interest were those focused on identifying and solving problems related to polypharmacy, pharmaceutical care and reduction of the risks of medication use, at the community level.

### Review Process

### Data Selection, Categorization, and Extraction

The identification and selection of studies followed the Cochrane Collaboration methods for systematic reviews (Higgins and Green, 2011). The retrieved studies were imported into the Rayyan QCRI (Ouzzani et al., 2016) online platform, and the references from the included and excluded studies were imported into the Mendeley reference manager (Mendeley et al., 2017). The titles and abstracts of the retrieved studies were independently selected by two reviewers (OS; JB). All disagreements were resolved by consensus among the reviewers. The selection process was documented and is presented in the flowchart adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyzes (PRISMA) guidelines (Moher et al., 2009) (**Figure 1**).

For the categorization of pharmaceutical interventions, the taxonomy proposed by the CADTH (Canadian Agency for Drugs Technologies in Health., 2018) and the Cochrane Effective Practice and Organization of Care (EPOC) taxonomy (Khalil et al., 2017) were used to classify interventions as professional, organizational, financial, regulatory or multifaceted. The outcomes of these interventions were compared to the outcomes of interventions by other professionals, outcomes with no pharmaceutical intervention or outcomes with any other community-level intervention. As outcome criterion, clinical and humanistic outcomes, including access to services (output), as well as epidemiological and economic outcomes were used. Other definitions and concepts are available in **Supplementary Material 2**.

An extraction form was used to collect the data of interest: article title, authors, journal, year of publication, last year of research, objectives, methods, statistics, risk of bias, main results, gaps, limitations, recommendations, equity analysis, quality assessment, conflicts of interest, and unanswered questions.

### Excluded Articles

In total, 642 records were retrieved: 50 from Health Evidence, 197 from Epistemonikos, 194 from Cochrane, 116 from Health Systems Evidence, and 85 from the Virtual Health Library, with no articles retrieved from Google Scholar. Of these, 117 duplicates were removed, leaving 525 records. The titles and abstracts of the eligible studies were independently assessed by two reviewers (OS; JB). During the screening, 472 articles were excluded due to inadequacies regarding outcomes, target population, design and/or type of study and type of publication. A total of 53 articles were preselected based on the inclusion criteria. After reading the full text of the articles, 27 were excluded because of the scenarios, intervention types and outcomes. In the end, 16 articles were selected for this overview. The list of excluded articles is available in **Supplementary Material 3**.

### Data Synthesis

A narrative-descriptive synthesis was prepared. This synthesis describes the interventions and evidence found, including the main findings relevant to the aims of this overview.

### Quality Assessment of Included Studies

The quality assessment was performed independently for each study by two reviewers (OS; JB), and the results were compared. Disagreements were resolved by consensus. We used A MeaSurement Tool to Assess systematic Reviews (AMSTAR) (Shea et al., 2007) and the checklist for Assessment of Economic Evaluation Studies (AEES) (Silva et al., 2014) to evaluate the quality of systematic reviews. Individual evaluations are available in **Supplementary Materials 4, 5**.

### Equity Considerations

Equity aspects were considered for the included studies, in particular, design approaches and reporting of issues related to health inequities. We used the PROGRESS framework (National Collaborating Centre for Methods and Tools, 2015), which includes place of residence, race/ethnicity/culture/language, occupation, sex/gender, religion, education level, socioeconomic status, and social capital. The PROGRESS framework was used to identify whether there were approaches and/or issues related to inequities in the outcomes of interest, in order to develop research and/or adapt research evidence and inform the design of new interventions (O'Neill et al., 2014).

The protocol of this overview was previously registered in PROSPERO (Booth, 2013) under number CRD42018093788/2018 (Soler and Barreto, 2018).

### RESULTS

### Profile and Characteristics of the Reviews

Sixteen articles met the inclusion criteria. The included studies were published between 2007 and 2017. They included 1 overview of systematic reviews, 12 systematic reviews, and 3 economic evaluations. The countries where the primary studies in these reviews were conducted include high-, middle-, and lowincome countries. The characteristics of the included studies are

provided in **Supplementary Material 6** and the distribution of studies by country and continent in **Supplementary Material 7**.

### Categories of Interventions and Outcomes

The categorization of interventions and outcomes based on the adopted frameworks is shown in **Tables 1**, **2**. Clinical, epidemiological and humanistic outcomes, including access to services, were categorized for professional, organizational, financial, regulatory, and multifaceted interventions.

## Methodological Quality of Included Studies

The systematic reviews were graded based on the AMSTAR criteria, with scores varying from low to high quality (**Supplementary Material 4**). The economic evaluation


TABLE

1


 for clinical outcomes.



The references in parentheses indicate the studies that presented evidence for each intervention/outcome studied. \*Not studied and/or no evidence. Community-level pharmaceutical interventions to reduce the risks of polypharmacy in the elderly: overview of systematic reviews and economic evaluations.

change prescribing practices

Governmental

interventions - governmental

prescriptions

Multifaceted interventions

 interventions:

 regulatory

\* \*

\*

\*

\*

\*

 \*

\*

\*

\*

\*

\*

\*

\*

\*

 \*

\*

\*

\*

\*

 policies that regulate

studies were considered of high quality based on the AEES criteria (**Supplementary Material 5**).

The level of noncompliance with the AMSTAR criteria may be related to the year of publication of the article, as earlier studies tended to have lower scores. The least frequently met AMSTAR criteria, in ascending order, were list of studies (included and excluded), methods for aggregating study results, evaluation, and documentation of scientific quality of included studies, evaluation of probability of publication bias, the inclusion of gray literature, and declaration of conflicts of interest.

### Reported Results

### Synthesis of Evidence on Interventions for Clinical Outcomes

The results presented in **Table 1** show the effects of various categories of interventions (professional, organizational, governmental, and multifaceted) on clinical outcomes related to polypharmacy and medication errors, adherence, and drug-related problems. The reported outcomes include reduced prescription of potentially inappropriate medicines, improved use of appropriate and safe medications, improved adherence, and reduced adverse drug events, adverse drug reactions, drug-drug interactions, and drug-related negative health outcomes.

### Professional Interventions:

### **Review of drug use**

Hajjar et al. (2007), Kaur et al. (2009), Mathumalar et al. (2011), Patterson et al. (2012), Cooper et al. (2015), Jórdan-Sánchez et al. (2015), Olaniyan et al. (2015), Khalil et al. (2017), presented evidence regarding the reduction of polypharmacy, medication errors, prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence. Hajjar et al. (2007) also found evidence for clinical outcomes related to reduction of drug-related problems, adverse drug events, adverse drug reactions, drug-drug interactions, and drug-related negative health outcomes.

### Clinical Case Analysis and/or Evaluation

Kaur et al. (2009), Alldred et al. (2013), Olaniyan et al. (2015), and have published evidence on clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Educational Interventions Targeted at Prescribers

Kaur et al. (2009), Hajjar et al. (2007), Olaniyan et al. (2015), and Mathumalar et al. (2011) provided evidence for clinical outcomes related to the reduction of polypharmacy, medication errors and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Educational Interventions Targeted at Users and/or Caregivers

Kaur et al. (2009), Olaniyan et al. (2015), and Mathumalar et al. (2011), and presented evidence for clinical outcomes related to the reduction polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Organizational Interventions

### Use of Information and/or Communication Technology for Active Search of Data and User Information

Olaniyan et al. (2015) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Use of Information and/or Communication Technology for Drug Information Services

Kaur et al. (2009) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Use of Information and/or Communication Technology for Clinical Decision-Making Support Systems

Kaur et al. (2009), Mathumalar et al. (2011), and Cooper et al. (2015) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Use of Information and/or Communication Technology for Risk Screening Tools

Patterson et al. (2012) and Olaniyan et al. (2015) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Provision of Pharmaceutical Care Services

Mathumalar et al. (2011), Patterson et al. (2012), Lee et al. (2013), Sáez-Benito et al. (2013), Cooper et al. (2015), Olaniyan et al. (2015), and Babar et al. (2017) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence. Lee et al. (2013) also found data on clinical outcomes regarding the reduction of drug-related problems, adverse drug events, adverse drug reactions, drug-drug interactions, and drug-related negative health outcomes.

### Technical Management of Medications and/or Medication Logistics

Olaniyan et al. (2015) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Governmental Interventions: Regulation of Prescribing Practices

Kaur et al. (2009) presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence.

### Financial Interventions: Incentive Programs to Change Prescribing Practices

Among the evaluated articles, no article specifically addressed this type of intervention, although there is evidence that such interventions can be effective in improving prescribing quality. It should be noted, however, that the CADTH has published evidence on interventions related to the improvement of prescribing quality using financial interventions (Higgins and Green, 2011).

## Multifaceted Interventions

Hajjar et al. (2007), Alldred et al. (2013), Lee et al. (2013), Cooper et al. (2015), and Olaniyan et al. (2015), and presented evidence for clinical outcomes related to the reduction of polypharmacy, medication errors, and prescription of potentially inappropriate medications; improvement of the use of appropriate and safe medications; and improvement of adherence. Lee et al. (2013) also found data on clinical outcomes related to the reduction of drug-related problems, adverse drug events, adverse drug reactions, drug-drug interactions, and drug-related negative health outcomes.

### Synthesis of Evidence on Epidemiological, Humanistic, and Economic Outcomes and Access to Services

The **Table 2** shows the effects of different interventions (professional, organizational, governmental, financial, and multifaceted) on different outcomes: access to services (reduction of outpatient visits, home visits, visits to emergency and emergency services, hospitalizations hospital, time of hospital stay), epidemiological (morbidity and mortality), humanistic (improvement of health status, improvement of health-related quality of life), and economic (reduction of drug costs).

### Professional Interventions: Clinical Case Analysis and/or Evaluation

Hajjar et al. (2007) have presented evidence on epidemiological outcomes including reduction of morbidity and mortality.

### Review of Drug Use

Hajjar et al. (2007) have presented evidence on epidemiological outcomes including reduction of morbidity and mortality.

### Educational Interventions Targeted at Prescribers

Hajjar et al. (2007) have presented evidence on epidemiological outcomes including reduction of morbidity and mortality.

### Educational Interventions Targeted at Users

Hajjar et al. (2007) have presented evidence on epidemiological outcomes including reduction of morbidity and mortality.

### Organizational Interventions:

### Provision of Pharmaceutical Care Services

Lee et al. (2013) and Babar et al. (2017) presented evidence on the improvement of access to services (output) and reduced hospital admissions. Lee et al. (2013) also found evidence for humanistic outcomes related such improved health status in terms of both clinical and surrogate outcomes. Jórdan-Sánchez et al. (2015) found evidence for improvement of health-related quality of life (HRQoL). Bojke et al. (2010) and Jórdan-Sánchez et al. (2015) found evidence on the reduction of drug costs. There is evidence that, on average, pharmaceutical care is economically viable and cost-effective, with an 80% probability reported by Bojke et al. (2010).

### Equity Considerations in Included Studies

With regard to equity, we found that the criteria described in the studies were limited to place of residence in high-, middle-, and low-income countries. There was no mention of whether individuals lived in urban or rural areas, their race/ethnicity/culture/language or their sex/gender (**Table 3**). Thus, in general, the included studies did not address equity and did not include subgroup analyses to identify socioeconomic differences.

### DISCUSSION AND CONCLUSION

This study aimed to provide an overview of systematic reviews and economic evaluations that addressed community-level pharmaceutical interventions to reduce the risks associated with polypharmacy in the elderly over 65 years. The elderly constitute the age group most at risk of polypharmacy and most susceptible to adverse events. For this population group, care at the community level represents one of the greatest challenges for health systems, especially for universal healthcare systems.

Polypharmacy refers to the prescription of both adequate and inadequate medications. Prescriptions must be made in a way that explicitly considers the overall effects of the total drug regimen and should be based on strong evidence to ensure rational use of medications.

Pharmaceutical care is an important part of universal healthcare systems in regard to ensuring rational use of drugs. Services provided at the points of care that include pharmaceutical care, whether delivered individually or collectively, are particularly important.

#### TABLE 3 | PROGRESS framework.


P, Place of residence; R, Race/ethnicity/culture/language; O, Occupation; G, Sex/gender; R, Religion; E, Education level; S, Socioeconomic status; S, Social capital.

(+), High-, middle- and low-income countries, with no information about whether individuals live in urban or rural areas.

(–), No information.

♂, male; ♀, female.

\*, White and non-white; \*\*, Lack of formal education; \*\*\*, Mobility problem; \*\*\*\*, Living with a partner; \*\*\*\*\*, Own house.

Source: Pharmaceutical interventions at the community level to reduce risks of polypharmacy in the elderly: an overview of systematic reviews and economic assessments.

## LIMITATIONS

This overview used systematic methods and a rigorous approach to identify and provide an up-to-date global synthesis of community-level pharmaceutical interventions that reduce the risks associated with polypharmacy in the elderly over 65 years of age.

A limitation of this study was that the results found did not allow a comparison between the studies, the quality of the evidence presented and the ethical conflicts. It is possible that potentially eligible systematic reviews might have been missed because they used different synonyms of the key descriptors.

The authors of the selected systematic reviews often warned readers to be cautious in the interpretation of the results, especially in view of the difficulty of aggregating data and the heterogeneity of the studies in terms of the variety, types, intensity and multiplicity of indicators and the use of narrative synthesis, as a meta-analysis was not possible.

### IMPLICATIONS FOR PROFESSIONAL PRACTICE

The categories of interventions included in this overview (professional, organizational, regulatory, financial, and multifaceted) demonstrated the benefits of pharmaceutical care for improving outcomes in the elderly over 65 years of age. There is evidence that an adequate system for managing, prescribing, monitoring, and evaluating the use of medications is effective in reducing polypharmacy and improving adherence to medications, while decreasing drug costs, medication errors, drug-related problems, adverse drug reactions, drug-drug interactions, drug-related negative health outcomes, and hospital admissions. Such systems also improve access to services, the use of safe and adequate medications and health-related quality of life.

### Implications for Research

In terms of implications for research, there is a substantial number of international studies showing that community-level interventions that reduce the risks associated with polypharmacy are complex and varied; there is no single path. However, authors of the systematic reviews selected in this overview highlight issues that remain unanswered, namely:


There is a need for further investigation of the effect of various types of pharmaceutical interventions (professional, organizational, regulatory, financial, and multifaceted) on the improvement of pharmaceutical care in the elderly over 65 years of age.

### Implications for Policies and Programs

As for the implications for policies and programs, pharmaceutical care stands out from an economic perspective as it is an efficient intervention to optimize prescribed medications and improve the quality of life in elderly persons taking multiple medications. Results from a cost-utility analysis suggest that pharmaceutical care is cost-effective.

There is evidence—with 80% probability—that pharmaceutical care is economically viable and profitable. This supports its incorporation into pharmaceutical assistance programs and/or policies, especially in universal healthcare systems based on access, quality and rational and sustainable use of medications at all levels of health care.

We know how important it is to ensure the establishment and implementation of evidence-based policies. Thus, we reiterate that, in universal and sustainable healthcare systems, pharmaceutical assistance and/or pharmaceutical care must be based on evidence of the efficacy and safety of the drugs, the effectiveness of the medications and the efficiency of the treatments.

Finally, we recommend that pharmaceutical professionals committed to efficient health policies should be included in multidisciplinary care teams to ensure that the elderly have access to high-quality and safe pharmacotherapy and a better quality of life.

### AUTHOR CONTRIBUTIONS

OS defined the research questions and prepared the research protocol in conjunction with JB. OS designed the study and conducted the bibliographic searches and the analyses. JB refined

### REFERENCES


the research question in the original draft and contributed to the study design by helping with the literature review and article review. The authors read and approved the final version of the manuscript.

### FUNDING

DIREB 017 FIO 16—Project: Support for the Improvement of technology management in SUS through a platform for translation, exchange, and social appropriation of knowledge. Coordinator: JB. The content of the review is the sole responsibility of the individual authors and does not reflect the opinions of Fiocruz-Brasília or the Brazilian Ministry of Health.

### ACKNOWLEDGMENTS

The authors thank the Oswaldo Cruz Foundation, Regional Unit of Brasília, Fiocruz School of Government, the Graduate Program in Public Health Policies, and the Ministry of Health of Brazil.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar. 2019.00302/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Soler and Barreto. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Use Profile of Magnesium Sulfate in Anesthesia in Brazil

Ismar Lima Cavalcanti 1,2 \*, Fernando Lopes Tavares de Lima<sup>2</sup> , Mario Jorge Sobreira da Silva<sup>2</sup> , Rubens Antunes da Cruz Filho<sup>3</sup> , Estêvão Luiz Carvalho Braga<sup>1</sup> and Nubia Verçosa<sup>4</sup>

<sup>1</sup> Department of General and Specialized Surgery, Anesthesiology, Fluminense Federal University, Niterói, Brazil, <sup>2</sup> Coordination for Education, Brazilian National Cancer Institute (INCA), Rio de Janeiro, Brazil, <sup>3</sup> Department of Clinical Medicine, Fluminense Federal University, Niterói, Brazil, <sup>4</sup> Department of Surgery, Anesthesiology, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil

Objectives: The use of magnesium sulfate in the perioperative period has several benefits, including analgesia, inhibition of the release of catecholamines and prevention of vasospasm. The aim of this survey was to provide an overview of the use of magnesium sulfate in anesthesia.

Method: This was a prospective descriptive cross-sectional study. An online questionnaire was sent to 9,869 Brazilian anesthesiologists and trainees. The questionnaire comprised closed questions mainly regarding the frequency, clinical effects, adverse events, and doses of magnesium sulfate used in anesthesia.

#### Edited by:

Luciane Cruz Lopes, Universidade de Sorocaba, Brazil

#### Reviewed by:

Ludo Haazen, Self-employed, Belgium André Oliveira Baldoni, Universidade Federal de São João del-Rei, Brazil

#### \*Correspondence:

Ismar Lima Cavalcanti ismarcavalcanti@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 18 January 2019 Accepted: 04 April 2019 Published: 26 April 2019

#### Citation:

Cavalcanti IL, Lima FLT, Silva MJS, Cruz Filho RA, Braga ELC and Verçosa N (2019) Use Profile of Magnesium Sulfate in Anesthesia in Brazil. Front. Pharmacol. 10:429. doi: 10.3389/fphar.2019.00429 Results: Of the 954 doctors who responded to the survey, 337 (35.32%) reported using magnesium sulfate in anesthesia. The most commonly cited clinical effects for the use of magnesium sulfate in anesthesia were (n/%): postoperative analgesia (245/72.70%), reduction of anesthetic consumption (240/71.21%) and prevention and treatment of preeclampsia and seizures in eclampsia (220/65.28%). The most frequently reported adverse events were hypotension (187/55.48%), residual neuromuscular blockade (133/39.46%), hypermagnesemia (30/8.90%), and intravenous injection pain (26/7.71%). The intravenous doses of magnesium sulfate used in most general anesthesia inductions were between 30 and 40 mg.kg−<sup>1</sup> .

Conclusions: Magnesium sulfate is an important adjuvant drug in the practice of anesthesia, with several clinical effects and a low incidence of adverse events when used at recommended doses.

Keywords: anesthetics (MeSH), analgesics, magnesium sulfate, survey, adverse events

### INTRODUCTION

Magnesium is the fourth most common ion in the body, and it participates in several cellular processes, including protein synthesis, neuromuscular function and stability of nucleic acid, as well as regulating other electrolytes such as calcium and sodium. Magnesium acts as a cofactor in protein synthesis, neuromuscular function and stability and the function of nucleic acids. It is a component of adenosine 5-triphosphatases and an endogenous regulator of other electrolytes. It is a calcium antagonist because it is a non-competitive inhibitor of calcium channels with inositol triphosphate. Magnesium modulates sodium and potassium currents and, as a consequence, interferes with the transmembrane potential. It is a central nervous system depressant, antagonizing N-methyl-Daspartate (NMDA) and inhibiting the release of catecholamines (Herroeder et al., 2011).

Some studies have shown that the use of magnesium sulfate as an adjunct in anesthesia reduces intraoperative consumption of anesthetics (Koinig et al., 1998; Seyhan et al., 2006; Ryu et al., 2008; Forget and Cata, 2017). It also provides better analgesia and reduces the amount of morphine used in the postoperative period (Mentes et al., 2008; Dabbagh et al., 2009; Hwang et al., 2010). Studies in clinical practice have demonstrated the inhibitory effects of magnesium on the release of catecholamines (Herroeder et al., 2011) through better hemodynamic control during laryngoscopy (Puri et al., 1998; Shin et al., 2011) and pneumoperitoneum insufflation for videolaparoscopy (Mentes et al., 2008). Magnesium sulfate also reduces levels of noradrenaline and vasopressin during anesthesia (Jee et al., 2009).

Other benefits of using intraoperative magnesium have been reported, including hemodynamic control in surgeries for resection of pheochromocytoma (James and Cronjé, 2004), reduced incidence of atrial fibrillation in myocardial revascularization surgeries (Toraman et al., 2001), and prevention of vasospasm (Wong et al., 2006) and neurological protection after subarachnoid hemorrhage (Schmid-Elsaesser et al., 2006). The attenuation of the release of catecholamines by the adrenal glands and antagonism to calcium in smooth muscle cells of arterioles are possible mechanisms of action (Herroeder et al., 2011).

The clinical duration of nondepolarizing neuromuscular blockers is prolonged with the use of magnesium sulfate in anesthesia (Fuchs-Buder et al., 1995; Kussman et al., 1997; Czarnetzki et al., 2010; Rotava et al., 2013). Magnesium interferes with neuromuscular function by reducing the conductance of calcium in presynaptic membranes, decreasing the amount of acetylcholine released by motor neurons (Herroeder et al., 2011). It may also reduce post-synaptic sensitivity to acetylcholine or have a direct effect on the membrane potential of muscle cells (Del Castillo and Engbaek, 1954).

This survey was conducted to contribute evidence on the use of magnesium sulfate as adjunct of anesthesia due to its potential clinical benefits.

The primary objective of this study was to know the use profile of Magnesium Sulfate in Anesthesia in Brazil.

### MATERIALS AND METHODS

The descriptive study was approved by the Research Ethics Committee of the Fluminense Federal University, Niterói, RJ, Brazil (CAAE 35038614.0.0000.5243, opinion 884.839, dated 11/13/2014). The informed consent form was signed electronically.

All the anesthesiologists and trainees members of Brazilian Society of Anesthesiology in 2015 were invited to participate. A self-administered electronic questionnaire was sent via e-mail to 9,869 potential participants of the research using the Survey Monkey software. The invitation was sent by 3 times with the 10-day interval between them.

We did not find in the literature a data collection instrument on the subject of this research. The lead researcher created the electronic questionnaire used in this research, composed of 10 closed questions that addressed the following aspects: duration of practice of anesthesiology, use of magnesium sulfate and other anesthesia adjuvants, indications, complications and doses of magnesium sulfate in anesthesia (**Figure 1**).

The instrument was pre-tested in two stages. In the first stage, the relevance of the instrument was evaluated and was carried out by the researchers themselves. In the second stage, the questionnaire was evaluated by 8 anesthesiologists and the results were used to create the final version of the questionnaire used in the research.

Data were analyzed using descriptive statistics. The original data can be accessed in the **Supplementary Table 1**.

### RESULTS

Survey responses were received from 945 (9.57%) participants. The length of time of anesthesia practice among the respondents is shown in **Table 1**.

Of the 945 anesthesiologists who responded to this survey, 331 (35.02%) reported using magnesium sulfate in anesthesia. The frequency of use of adjuvant drugs in anesthesia is described in **Table 2**.

The number and percentage of clinical effects (n/%) for the use of magnesium sulfate in anesthesia were (in descending order, more than one response per participant allowed): postoperative analgesia (242/73,11%), reduction of anesthetic consumption (237/71.60%), prevention and treatment of preeclampsia and seizures in eclampsia (218/65.86%), prevention and treatment of arrhythmias (175/52.87%), reduction of the dose of neuromuscular blockers (168/50.75%), prevention of postoperative chronic pain (167/50.45%), bronchodilation (165/49.84%), prevention of hyperalgesia post remifentanil use (160/48.34%), hypomagnesemia prevention in large surgeries (128/38.67%), induced systemic arterial hypotension (112/33.83%), brain protection (95/28.70%), sedation (86/25.98%), reduction of surgical bleeding or reduction of perioperative blood replacement (74/22.35%), management of pheochromocytoma (72/21.75%), prevention and treatment of agitation in emergence from general anesthesia (64/19.33%), inhibition of preterm birth (59/17.82%), prevention of myocardial ischemia (54/16.31%), prevention and treatment of shivering (50/15.10%), facilitation of tracheal intubation without the use of neuromuscular blocker agent (44/13.29%), reduction of nausea and vomiting (39/11.78%), prevention and treatment of laryngospasm (38/11.48%), control of fasciculation and myalgia after succinylcholine (31/9.36%), prevention of myoclonus after intravenous injection of etomidate (24/7.25%), treatment of tetanus (20/6.04%), adjuvant in spinal anesthesia (19/5.74%), decrease in platelet aggregation (14/4.23%), attenuation of the sympathetic response to tracheal intubation (1/0.30%) and extension of duration of motor block on subdural anesthesia (1/0,30%).

All anesthesiologists reported using the intravenous route (331/100.00%) to administer magnesium sulfate. Other routes were used less frequently: muscular (16/4.83%), nerve plexus


FIGURE 1 | Electronic questionnaire used in research "Use of Magnesium Sulfate in Anesthesia in Brazil." Brazil, 2015.



Brazil, 2015.

TABLE 2 | Frequency of use of adjuvant drugs in anesthesia (n, %).


More than one response per participant was possible (n = 945). Brazil, 2015.

TABLE 3 | Frequency of adverse events during use of magnesium sulfate witnessed at least once by the anesthesiologist.


More than one response per participant was possible (n = 331). Brazil, 2015.

(6/1.81%), spinal (3/0.90%), regional intravenous anesthesia (3/0.90%), wound infiltration (2/0.60%), inhalation (2/0.60%), and oral (1/0.30%).

**Table 3** shows the frequency of adverse events during use of magnesium sulfate witnessed at least once by the anesthesiologist. The most commonly reported were hypotension, residual neuromuscular blockade, hypermagnesemia, intravenous injection pain, and respiratory depression.

Of the adverse events reported, 73.78% of the cases were considered of mild gravity (see **Table 4**). It should be noted that TABLE 4 | Rate of intensity level of adverse events witnessed by anesthesiologists using magnesium sulfate anesthesia (n = 305).


Brazil, 2015.

TABLE 5 | Magnesium sulfate intravenous doses most commonly used in the induction of general anesthesia and sedation (n = 331).


Brazil, 2015.

some adverse events were reported as severe, i.e., respiratory depression (4), hypotension (4), residual curarisation (4), hypermagnesemia (2) and bradycardia (1).

**Table 5** shows the dosages of intravenous magnesium sulfate commonly used for induction of general anesthesia and sedation.

### DISCUSSION

Little or no scientific literature exists that reports on surveys on the use of magnesium sulfate in anesthesia.

Approximately 10% of those who received the invitation to participate completed the survey, specifically, 945 anesthesiologists. Several medical polls have reported similar response rates (Naguib et al., 2010; Locks et al., 2015). Low adherence of participants can be explained by the electronic method used for data collection.

### Duration of Anesthesia Practice of the Survey Participants

In the present survey, anesthesiologists with more than 20 years of anesthesia practice (30.59%) reported using magnesium sulfate in anesthesia and sedation most frequently; this group was followed by those with between 1 and 5 years of clinical practice (25.40%). The frequent use of magnesium sulfate among the more experienced anesthesiologists may stem from common use in certain specialties, particularly obstetrics. The high frequency of use of magnesium sulfate among the younger group of anesthesiologists may be result of the recent attention being paid to this drug, as well as the introduction of multimodal analgesic and anesthesia techniques (Czarnetzki et al., 2010; Herroeder et al., 2011; Shin et al., 2011; Rotava et al., 2013).

### Adjuvant Drugs in Anesthesia

Anesthesia adjuvants are agents that are administered in association with anesthetics to increase effectiveness, improve delivery, or decrease required dosage. The survey showed that the drug most commonly used in Brazil as an anesthesia adjuvant is clonidine (85.18%); magnesium sulfate (35.02%) ranks fifth among the medicines included as possible survey responses.

Giovannitti et al. (2015) postulated that agonists of the α-2 adrenergic receptors, including clonidine and dexmedetomidine, are important tools in the arsenal of modern anesthesia because of their ability to induce calm without causing respiratory depression. They also promote cardiovascular stability and reduce anesthetic requirements.

The drug reported as the second most frequently used adjuvant was ketamine. Bakan et al. (2014) conducted a randomized clinical trial and showed that ketamine, when associated with remifentanil in total intravenous anesthesia in children, is well suited to rigid bronchoscopic procedures.

Although this survey found that lidocaine ranked third on the list of most used drugs, Kranke et al. (2015), in a systematic review, reported that there is only little or moderate evidence that a continuous infusion of lidocaine has an impact on pain intensity, especially in the early postoperative period, or on postoperative nausea. There is limited evidence that it has consequences in other clinical outcomes, such as gastrointestinal recovery, length of hospital stay and opioid use (Kranke et al., 2015).

Gupta et al. (2006) demonstrated that magnesium sulfate has anesthetic, analgesic and muscle relaxing effects and significantly reduces the need for anesthetic drugs and neuromuscular blockers.

### Clinical Effects of Magnesium Sulfate in Anesthesia

As noted in this survey, there is a wide range of clinical effects for the use of magnesium sulfate in anesthesia. The great variety of clinical effects could be explained by the substantial involvement of magnesium in the physiology of various organs and systems.

Magnesium participates in over 325 cellular enzyme systems and is the second most abundant intracellular cation after potassium. Magnesium participates in numerous physiological and homeostatic functions, such as binding of hormone receptors, the transmembrane flow of ions, regulation of adenylate cyclase, calcium release, muscle contraction, cardiac excitability, neuronal activity, control of vasomotor tone and release of neurotransmitters, blood pressure and peripheral blood flow. Mg2<sup>+</sup> modulates and controls the input of cell Ca2<sup>+</sup> and Ca2<sup>+</sup> release from the sarcoplasmic reticulum (Altura, 1994).

Magnesium is essential in the transfer, storage and utilization of energy in cells. The intracellular level of free Mg2<sup>+</sup> ([Mg2+]i) regulates intermediate metabolism, synthesis and structure of DNA and RNA, cell growth, reproduction and membrane structure (Altura and Altura, 1996).

Dubé and Granry (2003) cited the therapeutic use of magnesium in the following anesthesia, intensive care and emergency situations: prevention and treatment of hypomagnesemia, induction of anesthesia, control of pheochromocytoma, cardiac arrhythmias, preeclampsia and eclampsia, perioperative analgesia, asthma, myocardial infarction, hypertensive crisis, and insulin resistance.

Roscoe and Ahmed conducted a postal survey of cardiac anesthetists in the United Kingdom, to determine the extent of magnesium sulfate (MgSO4) use and the main indications for its administration. The most common indications for administration were arrhythmia prophylaxis and treatment, myocardial protection and treatment of hypomagnesemia (Roscoe and Ahmed, 2003).

All the clinical effects for the use of magnesium sulfate in anesthesia presented by the anesthesiologists participating in this survey have been reported in other publications, including various systematic reviews and meta-analyses, although some of them are still subjects of controversy Be¸sogul et al., 2009; Gozdemir et al., 2010; Rhee et al., 2012; Abdulatif et al., 2013; Rotava et al., 2013; Agrawal et al., 2014; Ahsan et al., 2014; Crowther et al., 2014; Kahraman and Eroglu, 2014; Kew et al., 2014; Marzban et al., 2014; Rodrigo et al., 2014; Srebro et al., 2014; Uludag et al., 2014; Berhan and Berhan, 2015; Kim et al., 2015; Safavi et al., 2015; Vigil-De Gracia and Ludmir, 2015; Demiroglu et al., 2016; Green, 2016; Griffiths and Kew, 2016; Jangra et al., 2016; Juibari et al., 2016; Maged et al., 2016; Naghipour et al., 2016; Rodríguez-Rubio et al., 2016, 2017; Soltani et al., 2016; Thomas and Behr, 2016; Ulm et al., 2016; Vendrell et al., 2016; Xie et al., 2016, 2017; Brookfield et al., 2017; Haryalchi et al., 2017; Kutlesic et al., 2017; Lecuyer et al., 2017; McKeown et al., 2017; Mendonca et al., 2017; Salaminia et al., 2018; Zhang et al., 2018.

### Adverse Events of Magnesium Sulfate Use and Classification of Intensity

Herroeder et al. (2011) reported that the vasodilator effect of magnesium is the likely cause of burning or heat sensations in the body. Prolonged PR and QT intervals as well as atrioventricular blockage may occur. Toxicity occurs with the administration of venous doses greater than 30 g or with plasma concentrations above 14.4 mg/dl (Herroeder et al., 2011). Hypermagnesemia is manifested by abolition of tendon reflex; treatment consists of calcium gluconate, furosemide furosemide and hemodialysis (Herroeder et al., 2011).

In this survey, 2.95% of respondents reported severe complications from the use of magnesium sulfate. It is worth mentioning that the occurrence of severe adverse events is of fundamental importance, demonstrating that the administration of magnesium sulfate is not risk free. As in the present research, Herroeder et al. (2011) related as severe adverse events from the use of magnesium sulfate: arterial hypotension, bradycardia, muscle weakness, and respiratory depression. The results of our survey demonstrated similar results. Despite the occurrence of reports of serious AEs, the use of magnesium sulfate can be safe in recommended doses with close monitoring of patients (Kutlesic et al., 2017).

Marret and Ancel (2016) used magnesium sulfate in obstetric patients at an initial venous dose of 4 g followed by 1 g/h, without exceeding the cumulative total dose of 50 g. In their analysis of short and medium-term outcomes, they found no serious maternal adverse effects nor adverse effects on the newborns.

Griffiths and Kew (2016) observed few adverse effects when intravenous magnesium sulfate was used for treatment of asthma in children in the emergency department.

Wilson et al. (2014) realized a retrospective cohort study to evaluated the tolerability and safety of high doses of intravenous magnesium sulfate for tocolysis in preterm labor. The frequency of severe adverse events was 5.3% while in our survey it was 2.95%. This difference can be explained because all patients in the study received high doses of magnesium sulfate. They concluded that side effects occurred in 9 out of 10 patients and were considered severe for 1 out of every 20 pregnant women.

### Intravenous Dose of Magnesium Sulfate Most Frequently Used in Induction of General Anesthesia and Sedation

Germano Filho et al. (2015), in a randomized controlled study, demonstrated a significant increase in magnesium plasma concentrations after infusions of 40 mg.kg−<sup>1</sup> solution containing magnesium sulfate among ASA 1 or 2 patients. This confirmed that this dose is capable of increasing magnesium serum levels.

The magnesium sulfate doses reported in this survey are in accordance with those found in other publications. There are reports of magnesium sulfate induction doses in general anesthesia from 15 mg.kg−<sup>1</sup> to 75 mg.kg−<sup>1</sup> (Be¸sogul et al., 2009; Gozdemir et al., 2010; Rotava et al., 2013; Kahraman and Eroglu, 2014; Rodrigo et al., 2014; Honarmand et al., 2015; Rower et al., 2017) and doses up to 50 mg.kg−<sup>1</sup> in sedation (Lecuyer et al., 2017).

We observed that the Brazilian anesthesiologist uses magnesium sulfate rationally. Clinical effects, doses and routes of administration are found in the literature.

This survey describes the wide range of purposes magnesium sulfate is used for in anesthesia in Brazil. Although anesthesiologists have free access to the use of magnesium sulfate, research data have shown that the drug has been used primarily in those indications approved by the Health Authorities and/or supported by critical evaluation of systematic reviews and meta-analyzes. The frequency of its use is related to the amount and strength of evidence of its effects reported in the literature.

This survey has some limitations. Only Brazilian anesthesiologists participated in the study. Further, the participation of the anesthesiologists was voluntary; those

### REFERENCES


who agreed to participate are likely those most interested in the use of magnesium sulfate in anesthesia. This may have created bias that could interfere with the generalization of the responses to the full population of anesthesia specialists. Only 10% effectively responded to the survey, that the results may thus be biased. The questionnaire was not validated.

We conclude that magnesium sulfate is among the five most commonly used adjuvants in anesthesia, along with clonidine, ketamine, lidocaine and dexmedetomidine. Several clinical effects for magnesium sulfate were reported, especially postoperative analgesia, reduction of anesthetic consumption and the prevention and treatment of preeclampsia and eclampsia seizures. Hypotension, residual neuromuscular blockade, hypermagnesemia and pain on intravenous injection were the most frequent adverse events and, in general, were considered mild. Magnesium sulfate intravenous doses used in most general anesthesia induction were between 30 and 40 mg.kg−<sup>1</sup> .

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Brazilian National Health Council (Resolution number 466, from December 12, 2012) with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Committee of the Fluminense Federal University, Niterói, RJ, Brazil (CAAE 35038614.0.0000.5243, opinion 884.839, dated 11/13/2014).

### AUTHOR CONTRIBUTIONS

IC, FL, and MS designed the study and performed the experiments, IC, RCF, EB, and NV analyzed the data and wrote the manuscript.

### FUNDING

The study was supported by the Fluminense Federal University, Niterói, Brazil and Brazilian Society of Anesthesiology, Rio de Janeiro, Brazil. There was no funding source for this study.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar. 2019.00429/full#supplementary-material


elective adenotonsillectomy surgery in children. Anesth. Pain Med. 4:e15960. doi: 10.5812/aapm.15960


propofol requirements, haemodynamic variables and postoperative pain relief in gynaecological surgery. Br. J. Anaesth. 96, 247–252. doi: 10.1093/bja/aei291


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Cavalcanti, Lima, Silva, Cruz Filho, Braga and Verçosa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Suspected Adverse Drug Reactions Related to Breast Cancer Chemotherapy: Disproportionality Analysis of the Brazilian Spontaneous Reporting System

### Edited by:

Luciane Cruz Lopes, Universidade de Sorocaba, Brazil

#### Reviewed by:

Adriano Max Moreira Reis, Federal University of Minas Gerais, Brazil Raquel Herrera Comoglio,

Universidad Nacional de Córdoba, Argentina

#### \*Correspondence:

Flávia Campos Barcelos fbarcelos1979@gmail.com Mario Jorge Sobreira da Silva mjsobreira@yahoo.com.br Elisangela da Costa Lima eclima.ufrj@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 31 January 2019 Accepted: 18 April 2019 Published: 08 May 2019

#### Citation:

Barcelos FC, Matos GC, Silva MJS, Silva FAB and Lima EC (2019) Suspected Adverse Drug Reactions Related to Breast Cancer Chemotherapy: Disproportionality Analysis of the Brazilian Spontaneous Reporting System. Front. Pharmacol. 10:498. doi: 10.3389/fphar.2019.00498 Flávia Campos Barcelos <sup>1</sup> \*, Guacira Corrêa de Matos <sup>2</sup> , Mario Jorge Sobreira da Silva<sup>1</sup> \*, Fabrício Alves Barbosa da Silva<sup>3</sup> and Elisangela da Costa Lima<sup>2</sup> \*

<sup>1</sup> Brazilian National Cancer Institute (INCA), Rio de Janeiro, Brazil, <sup>2</sup> Faculty of Pharmacy, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil, <sup>3</sup> Oswaldo Cruz Foundation (Fiocruz), Rio de Janeiro, Brazil

Spontaneous reporting systems may generate a large volume of information in real world conditions with a relatively low cost. Disproportionality measures are useful to indicate and quantify unexpected safety issues associated with a given drug-event pair (signals of disproportionality), based upon differences compared to the background reporting frequency. This cross-sectional study (2008 to 2013) aimed to analyse the feasibility of detecting such signals in the Brazilian Pharmacovigilance Database comprising suspected adverse drug reactions related to the use of doxorubicin, cyclophosphamide, carboplatin, trastuzumab, docetaxel, and paclitaxel for breast cancer chemotherapy. We first accessed overall database features (patient information and suspected adverse drug reactions) and further conducted a disproportionality analysis based on Reporting Odds Ratios with a confidence interval of 95% in order to identify possible signals of disproportionate reporting, only among serious suspected adverse drug reactions. Of all data reports of adverse reactions (n = 2603), 83% were classified as serious, with the highest prevalence with docetaxel (78.1%). The final analysis was performed using 1,309 reports with 3,139 drug-reaction pairs. The following signals of disproportionate reporting, some rare or not mentioned on labels, were observed: tachypnea with docetaxel; bronchospasm, syncope, cyanosis, and anaphylactic reaction with paclitaxel; and anaphylactic shock with trastuzumab. Structured management of spontaneous adverse drug reaction reporting is essential for monitoring the safe use of drugs and detecting early safety signals. Disproportionality signal analysis represents a viable and applicable strategy for oncology signal screening in the Brazilian Pharmacovigilance Database.

Keywords: pharmacovigilance, oncology, disproportionatily analysis, spontaneous reporting system, adverse drug reaction

### INTRODUCTION

Pharmacovigilance activities—aimed at monitoring the safe use of medicines—are particularly important in oncology due to the inherent toxicity of antineoplastic agents (Baldo and De Paoli, 2014). Given the high incidence, prevalence and mortality of breast cancer in the worldwide female population (International Agency for Research on Cancer., 2014), knowledge of the toxicity profile of the main drugs used in its treatment is important strategically for prevention, detection and early management of suspected adverse drugs reactions (ADRs) related to chemotherapy.

Spontaneous reporting of ADRs contributes substantially to signal detection in drug safety surveillance, especially for rare and acute reactions (Arnaud et al., 2017). In pharmacovigilance, a signal is reported information on a possible causal relationship between an adverse event and a drug exposure, which was previously unknown or incompletely documented. It also refers to an increased number adverse events compared to the frequency of reactions normally expected with the use of a given product (U.S. Department of Health and Human Services et al., 2005; World Health Organization (WHO)., 2018).

Reports of ADR screening from the national surveillance system may offer data pertaining to different drug-event combinations (or drug-event pairs). These include methods of disproportionality analysis (DPA) that represent the main class of analytical methods for spontaneous report systems (SRS) data analysis in pharmacovigilance (Harpaz et al., 2012; European Medicines Agency., 2016)

Such methods, aimed at drawing attention to unexpected associations by generating hypothesis, have been guiding pharmacovigilance experts from health agencies worldwide in their investigations to draw definitive conclusions (Courtois et al., 2018).

Quantitative signal detection methods for spontaneously reported data include Proportional Reporting Ratio (PRR), Reporting Odds Ratio (ROR), and Empirical Bayes Geometric Mean (EBGM). These methods may identify relevant associations in SRS databases, focusing on projections of lower data dimensionality, more specifically two-dimensional contingency tables. Frequentist approaches are usually accompanied by independence hypothesis tests (Bate and Evans, 2009).

Brazilian Reporting System in Health Surveillance (Notivisa) was created in 2008. Since then, no evaluations of signal detection through disproportionality analysis have been made using this national database (Mota et al., 2018).

This study aimed to analyse serious ADRs associated with the most commonly used drugs in first-line adjuvant chemotherapy for breast cancer, reported from the Brazilian Surveillance System, in order to verify the feasibility of identifying potential signals through DPA using this database.

### MATERIALS AND METHODS

We carried out a cross-sectional study related exclusively to spontaneous ADRs reported in Notivisa from 2008 to 2013 during breast cancer treatment with doxorubicin, cyclophosphamide, carboplatin, trastuzumab, docetaxel, and paclitaxel—which are the preferred regimens according to National Comprehensive Cancer Network and Brazilian guidelines (National Comprehensive Cancer Network., 2017; Brasil, 2018).

Notivisa is a computerized system developed by the National Health Surveillance Agency (Anvisa) to receive notifications of incidents, adverse events and technical complaints related to the use of products and services under sanitary surveillance (Mota et al., 2018; National Health System Surveillance (Anvisa)., 2018). Anvisa supplied data on the Notivisa website reports (Pharmacovigilance Module) as a Microsoft Excel <sup>R</sup> file, which was made available by collaboration research. The database provided did not allow patient identification. The study was approved by the Research Ethics Committee of the Federal University of Rio de Janeiro.

Suspected ADRs were analyzed for their origin (geographical location), the patient's age (in years) and suspected drug (Anatomical Therapeutic Chemical Classification System), and were also classified according to severity and System Organ Class (SOC/MedDRA). The findings were also compared to the information contained in drug labels (MedDRA, 2018).

We accessed only serious ADR reports by using the signals of disproportionate reporting (SDR) method and thresholds recommended by the European Medicines Agency (EMA), namely Reporting Odds Ratio (ROR) and thresholds based on the 95% confidence interval and number of individual cases (European Medicines Agency., 2016).

DPA methodologies use frequency analysis of twodimensional contingency tables to estimate surrogate measures of statistical association between specific drug–event combinations mentioned in databases of spontaneous reports. For instance, the Reporting Odds Ratio (ROR) measure is defined by the formula below:

$$\text{ROR} = \text{(a.d)} / \text{(c.b)}$$

Where:


Eudravigilance adopts the following criteria to define an SDR (European Medicines Agency., 2016):


• The event belongs to the Important Medical Event Terms (IME) list, as defined by EudraVigilance.

In this work, we adopted a conservative approach: ROR calculations were performed for all drugs with **a** ≥ 3, and defined the concept of SDR intensity as a measure directly related to the value 'a'. Therefore, we differentiated between drug-event pairs with **a** ≥ 5 (higher intensity) and pairs with 3 ≤ **a** < 5 (lower intensity).

A situation occurs when **c** = 0, i.e., when all database reports containing a specific suspected ADR are associated to only one drug. In this case, the ROR cannot be computed and the value of the ROR is arbitrarily set at 99.9 to reflect the presence of a possible SDR, according to EudraVigilance (European Medicines Agency., 2016).

We adopted the same approach in this work. The observed signals were listed and further classified according to their intensity and ADR frequency, described in drug labels as: very common, not common, rare or very rare.

### RESULTS

Overall analysis was performed on 1,309 reports with 3,139 drugevent pairs (4% of total Brazilian reports for all medicines in the study period).

All reports came exclusively from nine out of the 27 Brazilian States (Rio de Janeiro, São Paulo, Minas Gerais, Rio Grande do Sul, Santa Catarina, Paraná, Mato Grosso, Mato Grosso do Sul e Bahia). Most (64.7%) were from hospitals located in the state of Rio de Janeiro. The median age of the female patients was 51.7 years (range: 25–87 years).

Around 83% (n=2603) of total suspected ADR (drug-reaction pairs) were classified as serious, for causing: death (n = 6), life-threatening conditions (n = 79), hospitalizations (n = 57), permanent disabilities (n=19) and medically important events (n = 2472). Of all serious ADRs, the majority (78.1%) were associated with docetaxel, with the greatest prevalence of those classified as general disorders (**Table 1**).

### Disproportionality Analysis

SDRs of higher intensity (a ≥ 5) and lower intensity (a ≥3 and a < 5; a ≥ 3 and c = 0 ∗ ) were observed for docetaxel, paclitaxel, trastuzumab, doxorubicin and cyclophosphamide. The disproportionality analysis did not identify any SDR for carboplatin.

Unexpected events considered rare, not common or not even mentioned on their respective drug labels and also known as acute and life-threatening were identified. The following are noteworthy: tachypnea with docetaxel, bronchospasm, anaphylactic reaction, cyanosis and syncope with paclitaxel and anaphylactic shock with trastuzumab (**Table 2**).

### DISCUSSION

The overall analysis in Notivisa has identified the reported prevalence of serious suspected ADRs with docetaxel and paclitaxel, mostly due to infusion-related reactions (IRR) that usually present with: flushing, rash, pruritus, fever, tremor, rigor, dyspnea, chest/back pain, nausea, light hypotension and tachycardia. Nevertheless, these sets of reactions tend to be mild to moderate and also common, occurring during the first few minutes of a first or a second drug infusion (Picard and Castells, 2015).

In fact, severe hypersensitivity reactions with taxanes are considered rare (incidence 3–5%) but have the potential to evolve rapidly toward a high risk scenario, characterized by a significant drop in blood pressure (systolic ≤ 90 mmHg) and/or syncope, bronchospasm, oxygen desaturation and anaphylaxis, which requires immediate therapeutic intervention (Brown, 2004; Picard, 2017).

The analyzed data indicated that careful monitoring of such reactions likely should have been done in Brazilian breast cancer patients, in the view of the large number of suspected ADRs related to docetaxel reported to the Notivisa. However, such high prevalence must be observed with caution due to the fact that the study time frame (2008–2013) corresponds exactly to the period immediately after the Sanofi-Aventis patent of Taxotere (docetaxel) expired in Brazil. This scenario might have influenced many health professionals and also pharmaceutical companies toward notification of suspected ADRs, due to the entrance of new generic brands of docetaxel into the Brazilian market. In addition, patients and health professionals often believe that generic cancer drugs have less quality, effectiveness and safety than branded-name drugs, leading to continuous attention and

TABLE 1 | Serious ADRs related to breast cancer chemotherapy by system organ groups (Brazilian Health System Surveillance, 2008–2013).


ADRs, suspected adverse drug reactions; SOC, System Organ Class; MedDRA, Medical Dictionary for Regulatory Activities; GI, gastrointestinal.


TABLE 2 | Disproportionality analysis conducted on Notivisa (Brazil) data according to the methods recommended by EudraVigilance for routine signal detection.

SDR, Signals of Disproportionate Reporting (lower bound of the ROR 95% confidence interval >1); ROR, Reporting Odds Ratio (with 95% confidence interval calculated for each drug-reaction pair of serious ADRs in comparison to drug labels information).

\*ROR measure for a≥3 and c = 0 defined arbitrarily as 99.9 in order to reflect the presence of a possible SDR.

#Included in post-marketing period.

a higher tendency to report ADRs (Yang et al., 2016). It is also common to note that ADRs and therapeutic failure reports significantly increase after the entry of generic oncology drug into the marketplace (Pitts et al., 2016).

As trastuzumab, a monoclonal antibody, is also highly associated with IRR, symptoms such as fever, chills and tremor that were observed in Notivisa are considered common and may affect up to 40% of patients. However, urticaria, angioedema, anaphylaxis and anaphylactic shock resulting in hospitalization or death are quite infrequent (Lenz, 2007).

We also noticed that serious ADRs (cardiovascular, gastrointestinal and neurologic events) have been scarcely reported in Notivisa, despite being widely observed among patients undergoing chemotherapy for breast cancer with the studied drugs, both in the literature and in VigiAcces <sup>R</sup> –the free access platform of the UMC/WHO database VigiBase (Barbour, 2008; De Lullis et al., 2015; Frise et al., 2017; Martel et al., 2017).

Despite its importance in terms of drug safety monitoring, pharmacovigilance activities remain a challenge, mostly due to under-reporting. This common problem especially impacts the field of oncology due to the fact that ADRs are often considered "normal" (or inevitable) in cancer treatment. Additionally, the sensitivity and availability toward spontaneously reporting suspected ADRs can also vary between different kinds of health professionals and health systems all around the world (Baldo and De Paoli, 2014).

Therefore, using disproportionality measures, a quick, inexpensive and sensitive method of signal screening, has benefits and strengths in that it can provide valuable information on ADRs of greater clinical importance and higher risk in oncology (Montastruc et al., 2011; Dias et al., 2014; Tuccori et al., 2015).

In our study we managed to identify SDRs for docetaxel, paclitaxel, trastuzumab, and less frequently, for doxorubicin and cyclophosphamide, but not for carboplatin. The following drug-event pairs should be highlighted: allergy/ hypersensitivity, anaphylactic reaction and bronchospasm with paclitaxel and classical IRRs with docetaxel and trastuzumab.

In addition, DPA analysis could also identify some important drug-event pairs, whose ADRs were not clearly mentioned on labels or were considered uncommon in clinical trials, such as: docetaxel and tachypnea; paclitaxel and bronchospasm (not mentioned in label, but related to severe RRI); paclitaxel and anaphylactic reaction; paclitaxel and cyanosis; paclitaxel and syncope (not mentioned in label, but related to severe IRR); trastuzumab and tremor and trastuzumab and anaphylactic shock (both related to IRR).

However, such results should be interpreted with caution due to specific limitations of reporting system databases, with which it is not possible to ascertain causality. Thus, a relative increase in the proportion of notifications of a given drug-event pair may in fact be a false positive without any kind of causal relationship, particularly in cases with low report numbers, where the statistical disproportionality may reflect one or more biases (Dias et al., 2014; Hauben et al., 2017).

In addition, it is relevant to mention that some external factors, such as: (i) time on the market (new or old drugs); (ii) tendency to report only severe adverse events and (iii) selective reporting for a given drug may affect the reliability of detected disproportionality signals (Bate and Evans, 2009; Arora et al., 2017). These important points seem also to justify the great number of docetaxel reports in Notivisa (2008–2013).

Based on our findings, and in accordance with the literature, it would be possible to select specific events identified and investigate their relationship to all drugs (event-based approach) or to select specific drugs to monitor and their relationship with all possible events (drug-based approach) (Bate and Evans, 2009; Trifirò et al., 2009; Dias et al., 2014).

The study limitations result from characteristics related to the Brazilian Surveillance System Database (Notivisa) and other SRSs (biases of under-reporting, heterogeneity and selectivity), limits of DPA itself and the fact that the majority of suspected ADRs originated from only one Brazilian State. This last limitation seems to be due to the fact that one of the most important cancer treatment centers (National Cancer Institute/INCA—considered a reference standard in the treatment of breast cancer in Brazil) is located in the Rio de Janeiro State.

Finally, it must be said that data mining methods in pharmacovigilance are considered complementary and not substitutes for traditional signal identification strategies. It is necessary to evaluate, in advance, the accuracy of the signaling criteria used, the nature and the number of drugs and warning events to monitor, the potential impact of false positives or false negatives and the availability of resources (Bate and Evans, 2009; Montastruc et al., 2011; Hauben et al., 2017).

Nevertheless, our analysis indicates a potential use of Notivisa for signal detection in regards to clinical or regulatory applications, as it was able to identify relevant disproportionate signals considered severe and rare, and not mentioned on drug labels.

To our knowledge, this was the first study to apply DPA methods to the Brazilian surveillance database system (Notivisa).

## CONCLUSION

Our analysis using Notivisa showed a predominance of serious ADRs in regards to docetaxel and paclitaxel, as well as a greater tendency to report general, vascular and respiratory disorders, mostly related to IRR.

The DPA applied was able to identify some interesting signals worthy of further investigation related to antineoplastic agents in Notivisa, suggesting that such a method might be useful for other drug classes.

Despite its inherent limitations, SRS seems to benefit from this kind of approach, as it can potentially contribute to research, surveillance and the ever safer use of medicines in low and medium-income countries such as Brazil.

### AUTHOR CONTRIBUTIONS

FCB, ECL, and GCM conceived and designed the study. FABS contributed to the disproportionality analysis. GCM and ECL contributed to the acquisition and the interpretation of data for the work. FCB, ECL, and MJSS discussed the results. FCB drafted the manuscript with support from ECL and MJSS. All authors critically revised the work and approved the final manuscript.

### ACKNOWLEDGMENTS

The authors thank the National Council for Scientific and Technological Development (CNPQ) for the funding support

### REFERENCES


and National Health Surveillance Agency (ANVISA) for providing access to Notivisa data.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar. 2019.00498/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Barcelos, Matos, Silva, Silva and Lima. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Post-marketing Study of Linagliptin: A Pilot Study

Gabrielle Kéfrem Alves Gomes, Mariana Linhares Pereira, Cristina Sanches and André Oliveira Baldoni\*

Grupo de Pesquisa em Epidemiologia e Avaliação de Novas Tecnologias em Saúde, Universidade Federal de São João del-Rei, Divinópolis, Brazil

Introduction: Linagliptin is a high-cost oral antidiabetic that has been widely used, and studies on its effectiveness and safety for the treatment of type 2 diabetes mellitus (DM2) in the real world is rare and necessary.

Objective: To analyze the values of glycated hemoglobin (HbA1c) and adverse events before and after the use of linagliptin in the post-marketing context of a pilot study.

#### Edited by:

Marcus Tolentino Silva, Federal University of Amazonas, Brazil

#### Reviewed by:

Palsamy Periyasamy, University of Nebraska Medical Center, United States Godfrey Mutashambara Rwegerera, University of Botswana, Botswana Hiroshi Yokomichi, University of Yamanashi, Japan Tsvetelina Velikova, Lozenetz Hospital, Bulgaria

> \*Correspondence: André Oliveira Baldoni andrebaldoni@ufsj.edu.br

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 08 January 2019 Accepted: 06 May 2019 Published: 24 May 2019

#### Citation:

Gomes GKA, Pereira ML, Sanches C and Baldoni AO (2019) Post-marketing Study of Linagliptin: A Pilot Study. Front. Pharmacol. 10:576. doi: 10.3389/fphar.2019.00576 Methods: This is a descriptive observational and exploratory study with a retrospective longitudinal approach, conducted between January 2014 and December 2016. All patients who participated in the study were over 18 years of age, with DM2, assisted by the Brazilian Public Health System (Sistema Único de Saúde – SUS) and had been indicated for use of linagliptin. The users were followed up and the variables of interest were collected from a computerized health information system (sistema informatizado de saúde – SIS) and patient records. For effectiveness analysis, HbA1c before (T0) and after (T1) the use of linagliptin was considered in patients registered as having collected linagliptin at the pharmacy for at least three consecutive months. For safety analysis, registered adverse events (AE) were verified in patients' records. The sample was stratified according to the pharmacotherapeutic scheme of the users. To compare the means before (T0) and after (T1), a paired t-test (data with normal distribution) and Wilcoxon Signed Rank Sum test (non-normal distribution data) were performed.

Results: Considering the total population of the study, in a different pharmacotherapeutic regimen, a median reduction in HbA1c of −0.86% (p < 0.05) was observed. After stratification by pharmacotherapeutic regimen, the most significant reduction of HbA1c was −1.07% (p = 0.014) for the linagliptin group associated with insulins and oral antidiabetic agents (n = 13). On the other hand, patients taking linagliptin in monotherapy had the lowest HbA1c reduction, −0.48% (p > 0.05). AE occurred in 12 (36.4%) patients, and 16.7% were in monotherapy.

Conclusion: Linagliptin did not presented, in real world, the desired performance as showed in randomized premarketing clinical trials and it should be carefully evaluated in public health services.

Keywords: linagliptin, Dipeptidyl peptidase 4 inhibitors, diabetes mellitus type 2, effectiveness, safety, pharmacovigilance, pharmacoepidemiology

## INTRODUCTION

fphar-10-00576 May 22, 2019 Time: 18:9 # 2

Diabetes mellitus type 2 (DM2) is a chronic disease highly prevalent in the adult population. The main objectives of DM2 treatment are metabolic control, the reduction of microvascular and macrovascular complications associated with the disease, as well as the reduction of its acute manifestations. To meet these goals it is necessary that blood glucose reach normal levels, both in fasting and in the postprandial period. Regarding the choice of pharmacological therapy, this should take into consideration the mechanisms of insulin resistance, secretory capacity of the pancreas, metabolic disorders involved, and the complications of DM2 present (American Diabetes Association [ADA], 2018). In healthy individuals, glucagon-like peptide 1 (GLP-1) and glucosedependent insulinotropic polypeptide (GIP), which are intestinal hormones or incretins, account for up to 70% of the insulin response, as they contribute to the modulation of pancreatic betacell activity, stimulating insulin secretion (Morris et al., 2013; American Diabetes Association [ADA], 2018).

In DM2 there is a reduced response of the insulin effect, which alters the regulation of the amount of glucose present in the blood, contributing to a lack of glycemic control of the sick individual. In this sense, incretin analogs and inhibitors of the enzyme Dipeptidyl peptidase-4 (DPP-4) have been developed in order to potentiate the function of these endogenous hormones. Incretin-based therapy has been increasingly prominent among treatment options for type 2 diabetes (DM2) (Websky et al., 2013). Studies demonstrate the efficacy of these substances in glycemic control as well as in the weight reduction of these patients (International Diabetes Federation [IDF], 2017; American Diabetes Association [ADA], 2018).

As members of the incretin class, gliptins are the inhibitors of the DPP-4 enzyme. Linagliptin, a representative of this class, has a peculiar pharmacological profile: pharmacokinetics allowing only one daily administration and no dose adjustment requirement for patients with renal and hepatic dysfunction (Richard, 2014; American Diabetes Association [ADA], 2018). Linagliptin can be used both in monotherapy and in combination with other antidiabetic agents (Chen et al., 2015; Marx et al., 2015; Mikhael, 2016; Thrasher, 2016). Studies have demonstrated that this association of other antidiabetic agents with linagliptin has been shown to be effective and widely used in clinical practice in order to optimize treatment of DM2 (Defronzo et al., 2015; Haak, 2015).

Several clinical trials have shown that incretins, such as linagliptin, have been considered a great therapeutic promise in terms of effectiveness and safety (Chen et al., 2015; Defronzo et al., 2015; Haak, 2015; Marx et al., 2015; Mikhael, 2016; Thrasher, 2016). However, data on the use of this class in the real world in monotherapy or in combination are scarce (Barnett et al., 2013; Sortino et al., 2013; Richard, 2014). In addition, linagliptin, and other representatives of the class of gliptins, are on the list of medications to be avoided, according to data published in the journal "Prescrire" and there is concern about an unfavorable profile of adverse effects including urinary tract infections and upper respiratory tract infections (Prescrire International, 2017).

In addition, studies on the effectiveness and adverse event profile of linagliptin in the post-marketing context is rare and necessary. In this context, this study, considered a pilot, aims to reduce this gap between the use of linagliptin by patients in the "real world" and the evidence from randomized clinical trials in developed countries. This study aims is to analyze the values of glycated hemoglobin and adverse events before and after the use of linagliptin in the post-marketing context of a pilot study.

### MATERIALS AND METHODS

### Study Design

This is a descriptive observational and exploratory study with a retrospective longitudinal approach (Elseviers et al., 2016). The study was outlined and described following the recommendations of Kempen (2011).

### Setting

The city where the study was conducted has 21,3016 inhabitants, Human Development Index (HDI) of 0.764, and 43 primary health care units, and only one center of endocrinology.

### Participants

All patients served at health units of the Brazilian Public Health System (SUS) of the city of Divinópolis, in the state of Minas Gerais (MG) who received a medical indication for the use of linagliptin during the period from January 2014 to December 2016 were identified and considered eligible for the study. Identification was made through a computerized health information system (SIS) that records the medication dispensed to the patients.

All participants who took linagliptin for at least three consecutive months were considered. The 3-month period was established so that it was possible to analyze the effectiveness of linagliptin according to the time required for variation of HbA1c levels (Malta et al., 2010). Participants in concomitant use of other medicinal products of the DPP-4 inhibitor class were excluded. This information was collected together with the patient's medical record and dispensing record.

### Variables and Data Source

The outcome variables considered were the effectiveness and safety of linagliptin. The analysis of the effectiveness of linagliptin was performed by comparing patients' HbA1c values shortly before linagliptin (T0) and after the first 3 months of consecutive use of the medication (T1).

Safety was analyzed from the active search for adverse events (AE) registered in patients' medical records during the period of linagliptin use. A list of adverse events related to linagliptin was developed to direct and systematize the search for AE reported in patients' records. This list was constructed after a systematized search in the literature on adverse medication events (Andriolo and Vieira, 2005; Food And Drug Administration [FDA], 2013; ANVISA, 2017; Gomes et al., 2018). To investigate possible laboratory abnormalities, the results of microalbuminuria, urea, aspartate transaminase (AST), alanine aminotransferase (ALT),

and gamma-glutamyltransferase (GAMA-GT) were investigated. Only the AE that were recorded in the patients' records during the period of linagliptin use were considered for this analysis.

In addition to the outcome variables, the following variables were analyzed: (I) demographic data: gender, self-reported race; (II) clinical data: pharmacotherapy used for DM2, family history, presence of comorbidities, and time of diagnosis of DM2; (III) biochemical data: fasting glycemia, glycated hemoglobin, postprandial glucose, creatinine, and urea. Medical records were used to define the presence of alcoholism, degree of obesity, renal failure, and other diagnoses.

### Statistics

Data analysis was performed with STATA software – Data Analysis and Statistical Software, version 12.0. To compare the biochemical tests before (T0) and after (T1) the use of linagliptin, the normality of the data of each variable was analyzed through the value of skewness and kurtosis, after which comparative analyses were performed between groups. For the data with normal distribution the paired t-test was performed and for the data with non-normal distribution the Wilcoxon Signed Rank Sum test was used. For the variable DM2 diagnosis time, the data were classified into groups according to the interquartile ranges observed (<25%, between 25 and 50%, >50%). To analyses differences in the values of HbA1c before (T0) and after (T1) the treatment with linagliptin, and stratification by pharmacotherapy of the medications of the patients was used paired t-test. All analyses were performed considering the level of significance of 5% and confidence level of 95%.

### Ethics Statement

This research was approved by Ethics in Research Committee of the Federal University of São João del-Rei (UFSJ), whose approval protocol is 1,827,849.

### RESULTS

It was observed that 108 participants had access to linagliptin for at least 1 month, however, only 33 (30.6%) had access for at least three consecutive months (inclusion criteria of the study). **Table 1** shows the profile of the 33 patients. It was observed that the majority of the patients were female (72.7%), evenly distributed among the age groups. The majority of patients were mixed race (36.3%), non-alcoholic (75.8%) and non-smokers (72.7%), and 48.5% reported a sedentary lifestyle. Regarding baseline glycated hemoglobin values (T0), it was observed that the majority of the patients (63.7%) had values above 9%. However, a significant number of patients (33%) presented HbA1c values within the normal range before starting treatment.

About the clinical characteristics of the patients, concerning the time of diagnosis for DM2, a higher prevalence of diagnostic times of 7–15 years (48.5%) was observed in this population. Regarding the observed comorbidities, 78.8% of the patients had systemic arterial hypertension, 36.4% dyslipidemia, and 27.3% cardiovascular disease. As for family history, the most prevalent



Data collected at T<sup>0</sup> – prior to the use of linagliptin.

diseases were cardiovascular disease (30.3%), diabetes mellitus (21.2%), and systemic arterial hypertension (15.2%) (**Table 2**).

Regarding the laboratory parameters, there was no statistical difference before and after the use of linagliptin (T<sup>0</sup> and T1) (**Table 3**). In the results of microalbuminuria, AST, ALT, and GAMA-GT, which were investigated to analyze the safety associated with the use of linagliptin, no altered values were observed. However, it was not possible to carry out the statistical analyses due to the scarce recording of these data.

In relation to pharmacotherapy, the association of "linagliptin with other oral antidiabetics and insulin" was the most used pharmacotherapeutic scheme among patients (45.4%). Data observed at baseline showed patients with microvascular complications such as chronic kidney disease (21.2%), diabetic retinopathy (12.1%), diabetic neuropathy (6.1%), amputation (3.1%), and glaucoma (3.1%).

Regarding the effectiveness of linagliptin, it was observed that the mean HbA1c of the patients reduced from 8.94% (±2.2) to 8.08% (±1.7). These data correspond to an absolute reduction of −0.86% (p < 0.05) in HbA1c values. After stratification of the sample according to the pharmacotherapeutic scheme for DM2

TABLE 2 | Clinical characteristics of patients in continuous use of linagliptin attended by the Brazilian Public Health System (SUS) from 2014 to 2016 (n = 33).


Data collected at T<sup>0</sup> – prior to the use of linagliptin. <sup>1</sup>Cardiovascular diseases considered for this analysis were: cerebrovascular accident (CVA), congestive heart failure (CHF), unstable angina. <sup>2</sup>Patients with degrees of obesity type I and type II were grouped in this class of clinical condition. <sup>3</sup>Patients with degrees of renal failure III and III were grouped in this class of clinical condition. <sup>4</sup>Other family histories found less frequently: throat cancer, bowel cancer, breast cancer, and hearing loss.

used by the patients, the reduction of HbA1c was lower when linagliptin was used as monotherapy (**Table 4**).

Among the 12 (36.4%) patients who presented AE records during the use of linagliptin, 16.7% were on monotherapy with linagliptin and 83.3% in association with other antidiabetics. Occurrences of 25 types of AE were observed and hypoglycemia corresponded to 20.0% of the total; 60.0% of the complaints about hypoglycemia occurred in the association of "linagliptin with insulin and other oral antidiabetics" (**Table 5**).

### DISCUSSION

In the effectiveness analysis of linagliptin in the present study, a difference of −0.86% (p < 0.05) of the mean values of HbA1c was observed, when considering the total population in use of the medication. However, when analyzing the effectiveness of linagliptin in monotherapy, the difference in HbA1c values was −0.48% (p > 0.05). Results of a phase III study that demonstrated its efficacy in monotherapy are close to the results here presented, with a difference in HbA1c of −0.67% after 24 weeks of study (Nogueira et al., 2014) and −0.87% after 12 weeks of study (Tang et al., 2015). In addition, according to the consensus algorithm for initiation and adjustment of therapy for DM2, the expected difference in HbA1c with iDPP-4 in monotherapy is −0.50 to −0.80%. It is also worth noting that treatment with this class has a neutral effect on weight, has long-term safety, but is expensive pharmacotherapy (Kawamori et al., 2012).

Therefore, it is important to note that this pilot study has limitations that limit the generalization of results, such as, the study has a small sample size and it is not a randomized clinical trial with control group. In addition, it was not possible to control confounders. Another point concerns the follow-up time of patients taking linagliptin, which was relatively short, so that it was not possible to observe probable AE associated with the chronic use of the medication. Also, it is important to note that it was not possible to evaluate adherence to treatment by primary and direct methods. This study also presents limitations inherent in observational studies, such as the lack of control of the researcher on the scenario investigated. In addition, information biases can be attributed to data collection performed with secondary sources of information.

The study by Nathan et al. (2009) and Lauand et al. (2014) suggests that the effect of linagliptin on the reduction of HbA1c appears to be moderate when compared to other oral agents such as metformin and sulphonylurea, reducing from 1.0 to 2.0%, and thiazolidinedione of 0.5–1.4%. The authors consider that linagliptin has a relatively low occurrence of hypoglycemia. Therefore, this medication has been proposed to be used as a second line therapy associated with metformin in the treatment of adult patients with DM2, or even as a first line therapy in those patients intolerant to metformin. The study also points to linagliptin as an option to be used in a triple pharmacotherapeutic scheme, as observed in this investigation, which would be in combination with oral antidiabetics and insulin.

Regarding the pharmacotherapy for DM2 used by patients included in the effectiveness analysis of the present study, it was found that linagliptin was indicated as monotherapy or associated with other antidiabetics and insulins. Regarding the treatment of DM2, in the current protocols there is no specific and clear information about which stage of the disease linagliptin is indicated (American Diabetes Association [ADA], 2018). However, there are premarketing studies that demonstrate the efficacy and safety of associating linagliptin with insulin receptor sensitizers, such as biguanides and glitazones (Haak, 2015) or with other medicinal products that act to stimulate insulin production and secretion (Ross et al., 2016) and also with insulin (Haak et al., 2013; Lauand et al., 2014; Defronzo et al., 2015).

Among the four pharmacotherapeutic groups used in conjunction with linagliptin, the insulin group in combination with oral antidiabetics was the most used among patients (45.4%). Although linagliptin was not approved in Brazil by the National Agency of Sanitary Surveillance (ANVISA) for use with insulin, an off-label use of this medication was observed in this study. However, linagliptin-specific warnings and precautions given by the Food and Drug Administration (FDA) indicate that when this medication is being used with an insulin secretagogue (e.g., sulphonylurea) or insulin, we should consider reducing the dose of the insulin or insulin secretagogue to reduce the risk of hypoglycemia. Despite the divergences of indication for the association of linagliptin and insulins between regulatory agencies, phase III studies demonstrate that the association of basal insulin and other DPP-4 inhibitors significantly improves

TABLE 3 | Comparison of laboratory parameters before (T0) and after (T1) the continued use of linagliptin by patients attended by the Brazilian Public Health System (SUS) from 2014 to 2016 (n = 33).


<sup>∗</sup>Non-parametric data presented in median (interquartile range: 25–75%) and statistical analyzes performed by the Wilcoxon Signed Rank Sum Test. Source: ADA, 2018 and VII Brazilian Guidelines on Hypertension. HbA1c, glycated hemoglobin.



<sup>∗</sup>Parametric data presented on average (standard deviation) and statistical analyses performed by the paired t-test. HbA1c, glycated hemoglobin. Pharmacotherapeutic groups: LINA, linagliptin monotherapy; LINA + AO, linagliptin associated with oral antidiabetics; LINA + INS, linagliptin associated with insulins; LINA + INS + AO, linagliptin associated with insulin and oral antidiabetic agents.

glycemic control over placebo (Rosenstock et al., 2009; Barnett et al., 2013; Yki-Järvinen et al., 2013; Marra et al., 2017).

However, in spite of the investigations demonstrating efficacy and safety in the use of linagliptin associated with insulin, in none of them was justified the rationale of this association, since the progression of DM2 reflects in the reduction of the production of insulin by the organism, a consequence of the reduction of the functioning beta cells (International Diabetes Federation [IDF], 2017). Another important factor is that the studies do not define the time of diagnosis of the patients included, or an evaluation of the tests that prove the secretory capacity of the pancreas. In the study by Yki-Järvinen et al. (2013) and Lauand et al. (2014) they observed that the type of insulin used, basal or bolus, did not interfere with the efficacy and safety of the combination with linagliptin. The literature suggests that the use of iDDP-4 should be a co-adjuvant in the treatment of DM2 (Vilsbøl et al., 2010), but it is not yet clear what are the therapeutic regimens in which it is most effective.

Studies have shown the efficacy of linagliptin associated with other pharmacotherapeutic regimens such as with metformin, suggesting a 2.72% HbA1c difference (Haak, 2015). In the results found in this study, a difference of HbA1c of −0.44% (p = 0.150) was observed for the association of linagliptin and oral antidiabetics, which included metformin 850 mg, metformin XR 500 mg, glibenclamide 5 mg, and gliclazide 30 mg. The differences in the values found may be related to two main factors at T<sup>0</sup> of the study, being (a) the difference of clinical parameters (diagnosis time, comorbidities, etc.), and (b) mean HbA1c. In the study by Ross et al., HbA1c at T<sup>0</sup> was 9.80 (1.1)%, being higher than in this study's population, which was 8.94 (2.2)%. According to the ADA, in patients with HbA1c values greater than 9%, gliptins may be more effective (American Diabetes Association [ADA], 2018). A meta-analysis involving 98 observational studies with 24,163 patients using iDPP-4 in different associations, attributed the cause of HbA1c reduction of 36.0% at the baseline level of HbA1c in patients. The study also found that variables such as prior oral treatment, age, gender, and body mass index (BMI), and the treatment time of the participants had no significant additional effect on the HbA1c reduction variance (Esposito et al., 2014b).

In the present study, a greater prevalence of diagnostic times of 7–15 years was observed, suggesting a population with a reduced secretory capacity of insulin by beta cells of the pancreas. In the analysis between the time of diagnosis of DM2 and the reduction in HbA1c values it was not possible to establish a correlation between the two variables. Even if these variables did correlate, it is admitted that this is a heterogeneous population with different pharmacotherapeutic regimens associated with linagliptin. Therefore, the reduction of observed HbA1c could not be attributed in a restricted way to the effectiveness of linagliptin, since insulin behaves as a powerful agent for the reduction of glucose (Esposito et al., 2015).

The literature indicates that the HbA1c reduction profile of iDPP-4 reduces with the treatment time, showing greater effectiveness in the first weeks (Vilsbøl et al., 2010). However, the time of accomplishment of the present study did not allow for the observation of this effectiveness profile, which suggests the importance of additional investigations with longer followup times.

Regarding the safety results of linagliptin, it was observed that more than one third of the 33 patients who used the medication continuously had some adverse event described in the literature related to linagliptin. The pharmacotherapeutic regimen that presented the most adverse events (58.3%) was that of triple pharmacotherapy (linagliptin + oral antidiabetic + insulin), with hypoglycemia being the most reported AE in this group. In this sense, considering the pharmacodynamics of these medications,

TABLE 5 | Adverse events using linagliptin described in the records of patients served by the Brazilian Public Health System (SUS), from 2014 to 2016 (n = 33).


<sup>1</sup>Unspecified hypoglycemia (4); hypoglycemia at night (1). <sup>2</sup>Pain in lower limbs (2); lower back pain (1). <sup>3</sup>Gastrointestinal events: intestinal constipation (1); vomiting (1); diarrhea (1). <sup>4</sup>Other: hepatomegaly, polydipsia, polyuria, polyphagia, weight loss, nocturia, altered sleep-wake cycle, edema, fever, fetid urine, weight gain, decreased visual acuity, dizziness, and otitis.

it is valid to consider that this event may be related more strictly to the use of insulin than to linagliptin. However, the study design and the co-medications used do not allow to infer the causality of the AE. In addition, information on the insulin doses used was not available. It is important to note that in this study only those AE that occurred after starting treatment with linagliptin were considered.

The total frequency of hypoglycemia in the 33 patients was approximately 15.0%. In the study by Gomis et al. (2012) and Esposito et al. (2014a), a similar frequency of hypoglycemia of 14.6% was observed in patients using linagliptin with other antidiabetics over a period of 24 weeks, twice the time of the present study (Esposito et al., 2014a).

In monotherapy with linagliptin the observed frequency of hypoglycemia was 16.7%, being a higher frequency when compared to studies by Haak et al. (2013), Defronzo et al. (2015), and Ross et al. (2016) whose incidence of this adverse event was lower than 8.0%. The present study differs from the clinical trials regarding the follow-up time of the participants, which ranged from 24 to 52 weeks in these studies, and also the characteristics of the population, since the clinical trials were controlled and since they excluded from the study any participants presenting comorbidities and who were inserted into the real world of polypharmacy.

Gomis et al. (2012) and Inagaki et al. (2013) evaluated hypoglycemia in the linagliptin-associated groups of biguanide, glinid, glitazone, sulfonylurea, and α-glucosidase inhibitors. In that study, only in the groups treated with linagliptin associated with sulphonylurea did hypoglycemia occur (9.5 and 5.9%). The incidence of hypoglycemia was significantly lower (<4.0%) in other studies using this AE as one of the outcomes (Kawamori et al., 2012; Inagaki et al., 2013; Tang et al., 2015).

The literature reports that hypoglycemic events are rare because of the glucose-ingestion dependent action (Haak et al., 2013), but they occur predominantly when a DPP-4 inhibitor is associated with sulfonylureas (American Diabetes Association [ADA], 2018). In the present study six patients (18.2%) were using glibenclamide or gliclazide, which are representatives of sulfonylureas associated with hypoglycemia.

It is important to note that in most premarketing studies, patients with these clinical conditions were not eligible for the study because of exclusion criteria, or family history data were not assessed (Haak et al., 2013; Inagaki et al., 2013; Lauand et al., 2014; Defronzo et al., 2015; Tang et al., 2015). In view of this, the importance of the post-marketing studies that accompany, record, and analyze data on the use of the medication in the real world stands out. All these factors can justify the differences found in this study, both in the effectiveness results and those related to medication safety.

Although the laboratory parameters analyzed did not present statistically significant differences between T<sup>1</sup> and T0, a reduction was observed in fasting glucose, postprandial glucose, serum creatinine and serum urea levels. The fact that these parameters did not show significant improvement in their values can be explained by the small sample size and the short followup period. In addition, it is important to note that the scarcity of recording in the patient's medical record of safety parameters such as microalbuminuria, AST, ALT, and GAMA-GT suggests absence of clinical monitoring or non-occurrence of an adverse event.

A systematic review by Gomes et al. (2018) presented results from 16 randomized clinical trials, evaluating the effectiveness and safety of linagliptin. The study identified that 93.8% of the studies were funded by the pharmaceutical industry, which evidences the need for studies free of conflicts of interest (Andriolo and Vieira, 2005).

Regarding the strengths of the study, it should be considered that this is the first real-world investigation conducted with Brazilian patients who used linagliptin, free from the influence of the pharmaceutical industry, in which the pharmacotherapy studied is immersed in a complex scenario which is related to the existing comorbidities and the presence of other factors extrinsic to the participants. In this sense, it is valid to consider that 93.8% of the studies evaluating the safety of linagliptin are financed by the pharmaceutical industry, and most of them had a comparison with placebo rather than with conventional pharmacotherapies (Andriolo and Vieira, 2005). On the other hand, the results of this study cannot be generalized, given the small sample size and the specificity of the participants.

In summary, the relevance of post-marketing studies as a tool for decision makers is recognized, especially in the face of unfavorable economic scenarios. Pharmacoeconomic studies and with a greater number of patients are needed to subsidize information for more assertive choices, maximizing the benefits of investments, without compromising the sustainability of the public health system.

### CONCLUSION

In the real world, linagliptin presented lower performance than in randomized premarketing clinical trials. These results reinforce the relevance of post-marketing studies as a tool for decision, especially in the face of unfavorable economic scenarios. In addition, it is important that further research be conducted through pragmatic clinical trials to be performed to assess possible confounding variables of real-world, such as adherence and access to medications. Because in public health system is not feasible that the therapeutic alternative has only efficacy. It needs to be effective and efficient.

### ETHICS STATEMENT

fphar-10-00576 May 22, 2019 Time: 18:9 # 7

This research was approved by Research Ethics Committee (CEP) involving Human Subjects of Federal University of São João del-Rei (UFSJ), Central-West Campus (CCO), whose approval protocol is 1,827,849.

### AUTHOR CONTRIBUTIONS

GG, MP, CS, and AB contributed to conception and design of the study. GG organized the database and performed the statistical analysis. MP, CS, and AB wrote the first draft of the manuscript.

### REFERENCES


All authors contributed to manuscript revision, and read and approved the submitted version.

### FUNDING

The present work was carried out with the support of the Coordination of Improvement of Higher Education Personnel – Brazil (CAPES) – Financing Code 001.

### ACKNOWLEDGMENTS

We thank the Federal University of São João del-Rei (UFSJ), Dona Lindu Center – West Campus (CCO) for infrastructure and institutional support. We also thank the editors of Frontiers Pharmacology for their support, especially Dr. Brian Boyle and Dra. Luciane Cruz Lopes.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Gomes, Pereira, Sanches and Baldoni. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Physician's Knowledge of Appropriate Prescribing for the Elderly—A Survey Among Family and Internal Medicine Physicians in Nigeria

*Joseph O. Fadare1\*, Abimbola Margaret Obimakinde2,3, Okezie O. Enwere4, Olufemi O. Desalu5 and Raphael Olasoji Ibidapo6*

*1 Department of Pharmacology and Therapeutics, Ekiti State University College of Medicine, Ado-Ekiti, Nigeria, 2 Family Medicine Unit, Department of Community Medicine, College of Medicine, University of Ibadan, Ibadan, Nigeria, 3 Family Medicine Department, University College Hospital, Ibadan, Nigeria, 4 Department of Medicine, Imo State University, Orlu, Nigeria, 5 Department of Medicine, College of Health Sciences, University of Ilorin, Ilorin, Nigeria, 6 Department of Medicine, Ekiti State University Teaching Hospital, Ado-Ekiti, Nigeria*

#### *Edited by:*

*Luciane Cruz Lopes, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Amer Hayat Khan, University of Science, Malaysia Syed Imran Ahmed, International Medical University, Malaysia*

#### *\*Correspondence:*

*Joseph O. Fadare joseph.fadare@eksu.edu.ng jofadare@gmail.com*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

> *Received: 31 January 2019 Accepted: 08 May 2019 Published: 31 May 2019*

#### *Citation:*

*Fadare JO, Obimakinde AM, Enwere OO, Desalu OO and Ibidapo RO (2019) Physician's Knowledge of Appropriate Prescribing for the Elderly—A Survey Among Family and Internal Medicine Physicians in Nigeria. Front. Pharmacol. 10:592. doi: 10.3389/fphar.2019.00592*

Background: Prescription and use of inappropriate medications have been identified as a major cause of morbidity among the elderly. Several screening tools have been developed to identify inappropriate medications prescribed for elderly patients. There is dearth of information about the knowledge of Nigerian physicians regarding these screening tools and appropriate prescribing for the elderly in general. The primary objective of this study was to assess the knowledge of Nigerian physicians about these screening tools and appropriate prescribing of medications for the elderly.

Methods: The study was a cross-sectional questionnaire-based study conducted among physicians working in Family Medicine and Internal Medicine departments of four tertiary health care facilities in Nigeria. The questionnaire consisted of sections on general characteristics of respondents and their knowledge of four selected screening tools for inappropriate medications in the elderly. Ten clinical vignettes representing different therapeutic areas (using the best option type questions) about medicine use in the elderly were included with a score of 1 and 0 for correct and wrong answers, respectively. The knowledge of respondents was classified as (total score, over 10): poor (score, < 5), average (score, 5-6), and good (score, 7-10).

Results: One hundred and five physicians returned completed questionnaires. Twenty percent of respondents knew about Beers criteria, whereas 15.6% were familiar with the STOPP criteria. Majority (83; 84.7%) of the respondents were confident of their ability to prescribeappropriately for elderly patients. The mean knowledge score was 5.3 ± 2.0 with 32 (30.5%), 41 (39%), and 32 (30.5%) having low, average, and good scores, respectively. The association between the knowledge score, duration of practice, and seniority was statistically significant (OR, 3.6, *p* = .004 and OR, 3; *p* = .012), respectively.

Conclusion: There are significant gaps in the knowledge of Nigerian physicians about screening tools for inappropriate medications. There is a need for stakeholders involved in

**52**

the care of elderly Nigerian patients to develop new strategies to improve services being offered. These may include introduction of modules on appropriate prescribing in the curriculum of undergraduate and postgraduate medical education and the routine use of some screening tools for inappropriate medications in daily clinical practice.

Keywords: inappropriate prescribing, elderly patients, rational prescribing, adverse drug reactions, physicians

### INTRODUCTION

Drug treatment of elderly patients is associated with potential adverse drug reactions (ADRs) with prescription and use of inappropriate medications being responsible in many cases (Dormann et al., 2013; Oscanoa et al., 2017). A systematic review on the prevalence and risk factors for ADRs in the elderly found a mean prevalence of 10% (Alhawassi et al., 2014). Patel and Patel (2018) demonstrated a four-fold increase in mortality due to ADR in elderly patients when compared to children. Identified factors for ADR in the elderly include polypharmacy and multiple co-morbidities on the background of declining physiological functions of the liver and kidneys (Joshua et al., 2007; Alhawassi et al., 2014; Sonnerstam et al., 2016). Also, changes in cognitive functions may affect adherence and other medicine-use issues (Wucherer et al., 2017).

Several tools/criteria have been developed over time to address the problem of inappropriate prescribing for elderly patients and these include the Beers criteria, Screening Tool of Older Persons' potentially inappropriate Prescription (STOPP) criteria, Medication Appropriateness Index (MAI), and the Zhan criteria (Beers et al., 1991; Hanlon et al., 1992; Zhan et al., 2001; Gallagher et al., 2008). Other recently developed tools are the European Union list of potentially inappropriate medications (EU(7)-PIM list) (Renom-Guiteras et al., 2015), PRISCUS (developed in Germany) (Holt et al., 2010), and Improved Prescribing in the Elderly Tool (IPET) (Naugler et al., 2000) developed in Canada. Emerging economies like Brazil and Thailand have also developed their own contextspecific instruments to screen for inappropriate medications in the elderly building on the foundation of the older instruments (Almeida et al., 2018; Prasert et al., 2018). The Beers list, an explicit criterion initially developed in 1991, has been regularly updated to cater for new scientific findings regarding appropriateness of medications in the elderly. Its latest update was in 2015 by the American Geriatric Society (AGS), and most of the newer tools were developed using it as a model (American Geriatrics Society, 2015). Negative patient outcomes, such as falls and delirium, have been associated with some of the drugs listed in the Beers criteria (Rothberg et al., 2013; Bazargan et al., 2018). The main emphasis of the STOPP criteria is in the area of avoidable adverse drug events (ADEs) and potential drug–drug interactions. Results from a meta-analysis of four randomized control trials (RCTs) showed a reduction in falls and episodes of delirium when STOPP criteria were applied (Hill-Taylor et al., 2016). The MAI is a 10-item question-based implicit criterion that takes into consideration the indication, effectiveness, dose, and direction for use among others (Hanlon and Schmader, 2013). The Zhan criteria, a modification of the Beers criteria, classify into 33 drugs into three categories according to their appropriateness or otherwise (Zhan et al., 2001).

The body of research in the area of prescription of potentially inappropriate medications and its consequences in elderly patients continues to grow in developing countries, like Nigeria and South Africa (Fadare et al., 2013; Fadare et al., 2015; Van Heerden et al., 2016; Akande-Sholabi et al., 2018; Saka et al., 2018). With improvement in the management of communicable diseases, such as HIV/AIDS, malaria, and tuberculosis, people are living longer in many developing countries and as such the proportion of elderly patients is expected to rise significantly in the next decade. Nigeria, the most populous country in Africa, has an estimated population of over 190 million with the elderly comprising just over 5% [Nigerian Demographic and Health Survey (NDHS), 2013]. The health care system is made up of private, faith-based, and public health care facilities. The public health care system of the country currently comprises three levels of care: primary, secondary, and tertiary. The tertiary level facilities, which are made up of Federal Medical Centres and University Teaching Hospitals, are the best equipped and have the most qualified health care personnel in the country. They are also training centers for medical students and postgraduate resident doctors in various specialties of medicine. These centers are the most patronized by the public because of the poor level of functionality of many primary and secondary level facilities in Nigeria. Therefore, there is a need for prescribers to be knowledgeable about potentially inappropriate medications and to actively screen prescriptions written for elderly patients. There are reports from some countries about the knowledge and attitude of physicians toward prescription of potentially inappropriate medications for elderly (Maio et al., 2011; Ramaswamy et al., 2011). There is, however, a dearth of information regarding this important issue in Nigeria and thus is the rationale for this study.

The primary objective of this study was to assess the knowledge of Nigerian physicians working in Family Medicine and Internal Medicine about appropriate prescribing of medications for the elderly. Secondary objectives included familiarity with screening tools for inappropriate medications and identification of barriers limiting appropriate prescription for the elderly.

### METHODOLOGY

The study was a cross-sectional questionnaire-based study conducted among physicians working in four tertiary health care facilities in Nigeria.

### STUDY SETTING

The study was conducted in the Family and Internal Medicine departments of four tertiary level health care facilities located in the South-Western, South-Eastern, and North-Central regions of Nigeria between January and April 2017. The health care facilities are: Ekiti State University Teaching Hospital, Ado-Ekiti (South-West), University College Hospital, Ibadan (South-West), University of Ilorin Teaching Hospital, Ilorin (North-Central), and the Imo State University Teaching Hospital, Orlu (South-East). The choice of these two specialities is because elderly outpatients are seen mainly in the clinics manned by residents and consultants from the departments. The departments of family and internal medicine in each of these facilities run daily outpatient clinics for ambulatory patients. The participating centers were selected through convenient sampling as some of the co-investigators were based there.

### STUDY PARTICIPANTS

### Inclusion Criteria

Physicians (residents and consultants) working in the Family and Internal Medicine Departments of the participating health care facilities who consented to the study.

### Exclusion Criteria

House physicians working in the Family and Internal Medicine Departments of the participating health care facilities because of their limited knowledge and experience with prescribing.

### STUDY INSTRUMENT

The self-administered questionnaire, developed in English language, was adapted from similar studies conducted in Italy and India among primary care physicians (Maio et al., 2011; Ramaswamy et al., 2011) and consisted of two parts: bio-demographics details, prescription experience for the elderly and respondent's perception about factors affecting appropriate prescribing for the elderly are captured in the first section. The second part was made up of 10 clinical vignettes about medication use in the elderly in the form of best option multiplechoice questions with a score of 1 and 0 allocated for a correct and wrong answer, respectively (**Supplementary Material Appendix A**). The scenarios described in the vignettes consisted of therapeutic problems of the central nervous (Nos. 3, 6, 10), endocrine (No. 9), musculoskeletal (No. 5), and cardiovascular systems (Nos. 1, 2, 7, 8). The knowledge of respondents was classified as (total score over 10): poor (score <5), average (score 5–6), and good (score 7–10). The questionnaire was reviewed for reliability and content validity by the principal researcher and two co-authors who are experts in clinical pharmacology and therapeutics, geriatrics and family medicine. Subsequently, it was pilot tested among 10 physicians working in the Family Medicine Department of a tertiary health care facility located within the same region, which was not part of the study. Based on their responses, the questionnaire was revised and necessary adjustments made before administration to study participants.

All physicians working in the departments of Family Medicine and Internal Medicine of the four participating hospitals were approached to participate in the study. The chief residents of the selected departments were asked to distribute the questionnaires to the physicians during mandatory departmental programs such as grand rounds and seminars. A time frame of 30 min was allowed for the completion of the questionnaire.

### SAMPLE SIZE

A convenience sampling method that included all 142 physicians working in the Family Medicine and Internal Medicine Departments of four tertiary health care facilities in three geopolitical areas of the country was used for this study. All the 142 physicians were invited to participate in the study.

### ETHICAL CONSIDERATION

Ethical approval was obtained from the Research and Ethics Committee of the Ekiti State University Teaching Hospital, Ado-Ekiti, Nigeria before commencement of the study. Because the study was questionnaire-based and noninvasive, permission was granted by the other centers based on approval from the primary site.

### STATISTICAL ANALYSIS

The information obtained from the questionnaire was coded, entered, and analyzed using IBM SPSS version 19. Analysis was done using descriptive statistics, which was used to obtain the general characteristics of the study participants. Chi-square was used to determine the level of significance of groups of categorical variables. P values < 0.05 were considered significant.

## RESULTS

### Socio-Demographic Details

One hundred and five physicians returned the completed questionnaires for analysis—a response rate of 73.9%. Majority of respondents (69; 65.7%) were males, senior medical doctors, and worked in internal medicine (65.7%, 52.8%, and 57.3%, respectively). **Table 1** shows the demographic characteristics of respondents. The mean duration of practice was 8.3 ± 6.7 years with 23.8%, 45.7%, and 30.5% of the respondents having practiced for <5 years, 5 to 10 years, and more than 10 years, respectively.

### Respondents' Knowledge of Screening Tools for Inappropriate Medications and Sources of Drug Information

The sources of drug information used by respondents were the British National Formulary (BNF), consultations with colleagues, and online sources (**Figure 1**). Regarding respondents' knowledge about screening tools for inappropriate medications, 20.4% and 22.7% of them knew about Beers criteria and MAI, while 15.6% were familiar with the STOPP criteria. **Figure 2** highlights respondents' knowledge of the screening tools. Majority (83; 84.7%) of the respondents were confident of their ability to prescribe rationally for elderly patients.

TABLE 1 | Demographic details of respondents.


### Barriers Against Appropriate Prescribing for the Elderly

Multiple medications (81; 82.6%), potential drug interactions (79; 81.4%), and cost of medicines to the patients (80; 80%) were the most commonly identified barriers against appropriate prescribing for the elderly (**Table 2**).

### Respondents' Knowledge of Presented Case Vignettes and Association With Other Study Variables

The mean knowledge score was 5.3 ± 2.0 with 32 (30.5%), 41 (39%), and 32 (30.5%) having low, average, and good scores, respectively. Overall, most respondents had good knowledge of appropriate use of medicines in the elderly in the following therapeutic areas: non-steroidal anti-inflammatory drugs (NSAIDs) (90; 87.6%), centrally acting antihypertensives in the elderly (92; 87.6% and 76; 76.4%), the use of first generation sulfonylureas (81; 77.1%), and the correct use of Nifedipine in hypertensive emergency (80; 76.2%). Poor level of knowledge was demonstrated in therapeutic areas of insomnia (19; 18.1%) and depression (25; 27.2%). The clinical scenario on the effect of NSAIDs on heart failure was answered correctly by 61 (58.1%) respondents.

The variables duration of practice, position, and knowledge score were dichotomized for ease of inferential analysis into the following: younger and older than 5 years, junior (junior residents and medical officers) and senior (senior residents and consultants), and score less than 5 and greater than 5. **Table 3** shows that the association between the knowledge score, duration of practice, and seniority was statistically significant. The strength of this association is further confirmed by the odds ratios (OR) of 3.6 and 3, respectively.

### DISCUSSION

This study assessed the knowledge of physicians working in Internal and Family Medicine departments regarding appropriateness of medicines for the elderly and its screening tools. Overall, the knowledge score of respondents was above average with majority of them dealing correctly with clinical scenarios affecting the cardiovascular, endocrine, and gastrointestinal systems.

Screening tools, such as the Beers criteria, STOPP, MAI, and Zhan criteria, are a group of explicit and sometimes implicit classification of medicines used for elderly patients. Medications on these lists have been found to contribute significantly to morbidity in elderly patients (Price et al., 2014; Sarwar et al., 2018). Only about a fifth of respondents in this study knew about the Beers criteria and the MAI with another 15% having information about the STOPP criteria. The knowledge level among respondents in our study is slightly better than that reported in qualitative studies among general practitioners in Australia and Germany (Magin et al., 2015; Pohontsch et al., 2017). In the just-cited studies, respondents showed a high level of confidence in appropriate

TABLE 2 | Identified barriers against appropriate prescribing for the elderly.


TABLE 3 | Association between the knowledge score and some variables.


*\*The variables duration of practice, position, and knowledge score were dichotomized for ease of inferential analysis into the following: less than or greater than 5 years, junior (junior residents and medical officers), and senior (senior residents and consultants).*

prescribing for elderly patients despite their poor knowledge of the screening tools. Similarly, over 80% of our respondents were also confident of appropriate prescribing for the elderly despite their poor knowledge of the screening tools. This might be due to the fact that many physicians may have used their residual knowledge of clinical pharmacology and understanding of the clinical scenarios to address prescribing issues in the elderly.

Respondents in this study identified high cost of medicines, potential drug interactions, and difficulty in communicating with other prescribers as barriers in addition to the ones cited in literature. Barriers to appropriate prescribing for the elderly from literature include limited knowledge of potentially inappropriate medications, extra time needed to consult the criteria, high number of prescribed medications, poor communication, and lack of formal education of prescribing guidelines (Maio et al., 2011; Clyne et al., 2016; Voigt et al., 2016).

Close to 70% of respondents scored above average in the clinical vignettes, indicating a good understanding of therapeutic issues in elderly patients. Our findings are similar to that from an Italian study with seven clinical scenarios where only 17% of respondents had low scores (Maio et al., 2011). Similarly, the mean score in an Indian study among family and internal medicine physicians was above average (3.9/6) (Ramaswamy et al., 2011). Respondents in our study were more knowledgeable in therapeutic areas of hypertension, diabetes mellitus, and osteoarthritis. This is not surprising because these are the most common disease conditions found among Nigerian elderly patients (Fadare et al., 2013, Fadare et al., 2015; Cadmus et al., 2017). This is in contrast to areas like arrhythmias, insomnia, depression, and other central nervous system (CNS) problems where low scores were recorded. The poor knowledge in therapeutic areas of insomnia and depression also resonates with the findings from Nigerian studies on prescription of inappropriate medicines where benzodiazepines (diazepam) and antidepressants (amitriptyline) were among the most common inappropriate medicines identified (Fadare et al., 2013, Fadare et al., 2015; Akande-Sholabi et al., 2018). On the contrary, respondents in the earlier-cited Italian study had better than average knowledge in therapeutic areas of insomnia and depression, whereas a similar poor performance was reported in the clinical vignette on the use of amiodarone in the elderly (Maio et al., 2011). The association between the knowledge score, duration of practice, and position was strong with OR of 3.62 and 3, respectively. In contrast, the study by Maio et al. (2011) reported respondents with low score had been in practice for longer duration. The reported higher score in our study is likely due to the working environment of respondents with many of them at various levels of postgraduate specialist training or were consultant physicians in charge of the training programs.

The BNF, physician colleagues, and the Internet were the most common sources of medicines information used by respondents. Medical textbooks, journals, and physician colleagues were the most identified sources in an earlier study conducted in Italy (Maio et al., 2011). Similarly, report from an earlier Nigerian survey among 163 medical doctors indicated information from colleagues, reference books, and pharmaceutical sales representatives as the main sources (Oshikoya et al., 2011). The rapid growth of information and communication technology (ICT) globally and especially its use in medicine is likely responsible for the change in trend.

### STUDY LIMITATIONS

This study is associated with some limitations. The study was conducted in a few tertiary health care facilities and hence may not reflect the reality among physicians working in primary and secondary care facilities worldwide nationwide. Since the questionnaire was also self-administered, there is also the possibility of bias while responding. The lack of similar studies from Nigeria and other African countries with which to compare our findings may also have affected the discussion of this study. However, the use of previously validated questionnaire, relatively high proportion of respondents from the sampling frame, representation from different cadres of physicians and different regions of the country are potential strengths of the study.

### CONCLUSION

There are significant gaps in the knowledge of Nigerian physicians about screening tools for inappropriate medications. There is a need for stakeholders involved in the care of elderly Nigerian patients to develop new strategies to improve services being offered. This may include introduction of modules on appropriate prescribing in the curriculum of undergraduate and postgraduate medical education in Nigeria.

### IMPLICATIONS FOR PRACTICE (FUTURE DIRECTIONS)

There is a need to incorporate the use of some of the established screening tools, such as the Beers and STOPP criteria in the daily clinical practice of Nigerian physicians. The development of a context-specific screening tool may be the direction to go for stakeholders in geriatric care in Nigeria.

### AUTHORS CONTRIBUTIONS

JF conceptualized and designed the study. He also analyzed the data and developed the first draft of the manuscript. AO took active part in conceptualizing the study. She was involved in data acquisition and analysis and reviewed the initial draft of manuscript for critical intellectual content. OE was involved in conducting the literature search for the work and also in data acquisition, and reviewed the initial draft of the manuscript. OD was involved in data acquisition and analysis and reviewed the draft manuscript before final approval. RI was involved in the literature search, data acquisition, and analysis, and also reviewed the draft manuscript. All co-authors approved the final draft of the manuscript.

### ACKNOWLEDGMENTS

The authors are grateful to the physicians who took the time to complete the questionnaires.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00592/ full#supplementary-material

### REFERENCES


inappropriate medication to elderly patients? *BMC Fam. Pract.* 17, 93. doi: 10.1186/s12875-016-0482-3


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Fadare, Obimakinde, Enwere, Desalu and Ibidapo. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# How to Prevent or Reduce Prescribing Errors: An Evidence Brief for Policy

Bruna Carolina de Araújo, Roberta Crevelário de Melo, Maritsa Carla de Bortoli, José Ruben de Alcântara Bonfim and Tereza Setsuko Toma\*

Department of Health, Institute of Health, Government of the State of São Paulo, São Paulo, Brazil


#### Edited by:

Cristiane De Cássia Bergamaschi, Universidade de Sorocaba, Brazil

#### Reviewed by:

Sabrina Anne Jacob, University of Strathclyde, United Kingdom Muhammad Usman, University of Veterinary and Animal Sciences, Pakistan Maria Elizete Almeida Araujo, Federal University of Amazonas, Brazil

#### \*Correspondence:

Tereza Setsuko Toma ttoma@isaude.sp.gov.br

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 14 January 2019 Accepted: 05 April 2019 Published: 12 June 2019

#### Citation:

Araújo BC, Melo RC, Bortoli MC, Bonfim JRA and Toma TS (2019) How to Prevent or Reduce Prescribing Errors: An Evidence Brief for Policy. Front. Pharmacol. 10:439. doi: 10.3389/fphar.2019.00439


Keywords: inappropriate prescribing (MeSH term), prescription errors, pharmaceutical services (MeSH), evidence brief for policy, patient safety

### PRESCRIBING ERRORS: A WORLDWIDE PROBLEM

Patient safety became the focus of attention of the World Health Organization (WHO), which in 2004 launched the World Alliance for Patient Safety (World Health Organization (WHO), 2017). During the second Global Ministerial Summit on Patient Safety in 2017, the WHO Director-General announced a third challenge to be faced: drug safety.

Medication errors are a relevant problem to face, in terms of patient damage and health systems sustainability, since worldwide their costs are estimated to reach 42 billion US dollars per year. The goal proposed by WHO is to reduce the level of serious and preventable drug-related harm by 50% within a 5-year period. One of the recommendations is the development of specific action programs to improve safety in situations where a drug can cause unintended harm, including health professionals' behavior and medication practices and systems (Donaldson et al., 2017).

In this context, it is important to distinguish "medication error" and "prescribing error," often used interchangeably in the literature. A medication error can be characterized as "a failure in the treatment process that leads to, or has the potential to lead to, harm to the patient," which encompasses prescribing errors (**Table 1**), dispensing errors and administration errors (Ferner and Aronson, 2006; Ferner, 2014). Nevertheless, medication errors are difficult to assess because of the variety of terms that are misused for this purpose. Several types of errors can be influenced by different factors and result in a variety of outcomes that may require specific courses of action

**60**

#### TABLE 1 | Classification of prescribing errors.


Adapted from Lavan et al. (2016).

(Rosa et al., 2009; Ferner, 2014; Lavan et al., 2016). It is worth noting that the errors committed by prescribers are the major factor behind the occurrence of medication errors (Qureshi et al., 2011; Porter and Grills, 2016).

Prescribed drugs are considered to rank as the third leading cause of death in the United States and Europe, surpassed only by heart disease and cancer. While about 100,000 deaths each year in the United States could be related to people taking drugs correctly, another equivalent number of deaths would occur due to errors like the use of contraindicated drugs or in very large doses. Impotent drug regulation, corruption of scientific evidence, drug marketing, and bribery of physicians are pointed out as factors that contribute to this situation (Gøtzsche, 2014).

In India, drug misuse is also common, and the major determinants of the problem include the lack of effective regulation and education on the appropriate use of these products. It is estimated that ∼50% of the average family spending on medicines is unreasonable or unnecessary (Porter and Grills, 2016).

In Brazil, Martins et al. (2011) analyzed medical records of 103 patients from three different hospitals and found that the occurrence of avoidable adverse events was 2.3%, whereas the mortality rate related to adverse events was about 8.5%. Among the elderly individuals, a use prevalence of 11.5–62.5% of potentially inappropriate drugs was associated with adverse effects, hospitalization, morbidity, mortality, and a higher cost of health services (Lucchetti and Lucchetti, 2017).

In this context, this study was aimed at identifying evidence in the scientific literature of effective interventions to avoid or reduce prescribing errors.

### SUPPORT TOOLS FOR DRAWING UP EVIDENCE BRIEFS FOR POLICY

This is an evidence brief for policy that followed the methodological guidelines proposed by the SUPPORT collaboration group—Supporting Policy Relevant Reviews and Trials (Lavis et al., 2009).

Evidence briefs for policy are documents that identify, through the most reliable scientific evidences, interventions to deal with a policy-related issue. They are tailored to inform decision makers on the best available and efficient actions to handle with health policy problems, without posing a recommendation, since the process of decision making depends on a variety of factors, including the local context. Within this structure, it is usually found a problem and its relevance for health policies, options to deal with the problem, considerations regarding implementation and equity (Bortoli et al., 2017).

The search for studies was carried out in December 2017, in nine databases: BVS Regional Portal; PubMed, Health Systems Evidence; Health Evidence, PDQ-Evidence; Center for Reviews and Dissemination; Embase; Cochrane Database of Systematic Reviews; and Epistemonikos. In our search strategy, we used the terms "Inappropriate Prescribing" and "Prescription Errors." Search filters were used for identifying systematic reviews published in English, Spanish, and Portuguese. This process was performed by a researcher from our team, and no limits were placed on the publication date.

Article selection and data extraction were carried out independently by two investigators, and disagreements were resolved by a third investigator. The studies thus identified that did not fit our inclusion criteria (systematic reviews, strategies/interventions to enhance prescribing, strategies that involved not only physicians) were excluded after reading their titles, abstracts and full texts (**Supplementary Table S1**). Data from the selected systematic reviews were extracted into a spreadsheet containing information related to the study population, interventions administered, outcomes, and countries according to their income (**Supplementary Table S2**). From this extraction, we came up with a range of interventions, which were arranged in groups according to their similarity, resulting in options for dealing with the problem.

The methodological quality of the selected systematic reviews was assessed independently by two investigators who used the Assessing Methodological Quality of Systematic Reviews tool— AMSTAR (Shea et al., 2007). Any divergences were settled by consensus.

In order to implement health policies, it is necessary to reflect on their implications so as not to cause or increase health iniquities. In this study, we used the tool PROGRESS—an acronym standing for Place of residence; Race/ethnicity/culture/language; Occupation; Gender/sex; Religion; Education; Socioeconomics status; Social capital (Evans and Brown, 2003)—for making considerations on equity in the policy options.

Most systematic reviews included in the options were developed in HIC (high-income countries), thus, in order to best address considerations about the process of implementation, for each one of the options we searched qualitative articles at the BVS Regional Portal. This step aimed to identify, preferably, strategies performed in Brazil, our context, and that could be relatable to other UMIC (Upper-Middle-Income Countries).

### POLICY OPTIONS FOR PREVENTING OR REDUCING PRESCRIBING ERRORS

Of the 1,191 systematic reviews identified, 40 were selected and analyzed in order to draw up the options provided (**Figure 1**). From the set of interventions extracted from the

systematic reviews, we devised four options for dealing with prescribing errors, which we present below: (1) Promoting educational actions on prudent prescribing directed to prescribers; (2) Incorporating computerized alert systems into clinical practice; (3) Implementing the use of tools for guiding medication prescribing; and (4) Encouraging patient care by a multidisciplinary team, with the participation of a pharmacist.

### Option 1. Promoting Educational Actions on Prudent Prescribing Directed to Prescribers

Thirteen systematic reviews addressed the effectiveness of educational actions in preventing or reducing prescribing errors, of which six were deemed as having high methodological quality, three as moderate, and four of low quality.

The following studies highlighted the positive effects produced by educational actions through different approaches: educational performance of pharmacists (Ross and Loke, 2009; Tesfaye et al., 2017); actions that improve the transfer of information among prescribers and discussion of cases in the multidisciplinary team in long–term care facilities for the elderly (Alldred et al., 2016); educational actions with multidisciplinary teams (Chiatti et al., 2012); distribution of clinical protocols and therapeutic guidelines, educational meetings, audit and feedback (Arnold and Straus, 2005); small group workshops, use of decision trees, sharing of quarterly reports, and annual educational actions (Kaur et al., 2009); educational initiatives dissemination, targeted studies and meetings with the participation of professionals (Forsetlund et al., 2011); use of web-based education program, use of performance feedback, along with patient and clinician education, direct and individualized education actions (Brennan and Mattick, 2013); multifaceted interventions (Brennan and Mattick, 2013; Ivanovska and Holloway, 2013; Roque et al., 2014; Coxeter et al., 2015); educational actions that promote behavior change (Tonkin-Crine et al., 2011) tailored to antimicrobial stewardship teams (Davey et al., 2017); interactive educational workshops with reinforcement by a local opinion leader (Fleming et al., 2013).

All reviews concluded that different educational interventions can be effective in reducing inappropriate prescriptions.

### Option 2. Incorporating Computerized Alert Systems Into Clinical Practice

Eighteen systematic reviews, of which eleven were classified as high methodological quality, three of moderate one and four as low quality, addressed the use of electronic systems and showed the effectiveness of using different systems in reducing prescribing errors.

The studies emphasized a positive effect on improving prescription writing or reducing prescribing errors by using: alert systems (Schedlbauer et al., 2009; Davey et al., 2017); drug dose adjustment supported by information technology (Mekonnen et al., 2016); electronic archives in hospitals (Sánchez et al., 2014); electronic prescribing resources for undergraduate students (Ross and Loke, 2009); medical reminders, information provided at the time of prescription writing on an online prescription editor (Arnold and Straus, 2005); a Clinical Decision Support System (Kaushal et al., 2003; Yourman et al., 2008; Kaur et al., 2009; Pearson et al., 2009; Reckmann et al., 2009; Lainer et al., 2013; Maaskant et al., 2015; Clyne et al., 2016); a Medical Order Entry System (Kaur et al., 2009) at an intensive care unit (Kaushal et al., 2003; Hodgkinson et al., 2006; Van Rosse et al., 2009; Khajouei and Jaspers, 2010); a Prescription Automatic Screening System (Yang et al., 2012).

Nevertheless, some studies have shown increased medication and prescribing errors when using complex Physician Order Entry Systems (Khajouei and Jaspers, 2010), due to excessive available information (Lainer et al., 2013).

### Option 3. Implementing the Use of Tools for Guiding Medication Prescribing

Nine systematic reviews, four of which were considered to be of high methodological quality, four of moderate and three of low quality, provided information on the use of medication prescribing tools.

The findings showed that the tools that may be useful for improving prescribing quality and reducing inadequate prescription are: STOPP/START (Cooper et al., 2015; Santos et al., 2015; Hill-Taylor et al., 2016; Hyttinen et al., 2016) and Beers criteria (Garcia, 2006; Jano and Aparasu, 2007; Soares et al., 2011; Cooper et al., 2015; Santos et al., 2015; Hyttinen et al., 2016). In addition, these tools can be combined with other actions, such as educational ones (Alldred et al., 2016; Valencia et al., 2016).

STOPP - Screening Tool of Older Persons' Prescriptions and START - Screening Tool to Alert to Right Treatment are prescribing screening tools for older people (Mahony et al., 2010).

Beers criteria are lists of potentially inappropriate drugs for the elderly (DeSevo and Klootwyk, 2012).

### Option 4. Encouraging Patient Care by a Multidisciplinary Team, With the Participation of a Pharmacist

Nine systematic reviews, of which four were regarded as being of high methodological quality, three of moderate and three of low quality, showed that working as a multidisciplinary team reduces prescribing errors, especially when there is a pharmacist in the team (Chiatti et al., 2012; Sánchez et al., 2014; Alldred et al., 2016; Clyne et al., 2016).

These studies indicated that, as far as patient care is concerned, a multidisciplinary team is better indicated to reduce inappropriate or multiple prescribing (Garcia, 2006; Kaur et al., 2009), decrease inappropriate prescribing in elderly patients (Riordan et al., 2016; Walsh et al., 2016), and antibiotic inappropriate prescribing (Fleming et al., 2013; Maaskant et al., 2015).

### CONSIDERATIONS ABOUT IMPLEMENTING POLICY OPTIONS AND THEIR EQUITY

Although the options presented, do not necessarily have to be implemented together nor in a comprehensive way, their practical implementation should consider local feasibility and whether they can be integrated into the governability of decision making, irrespective of a health system's size (whether national, regional, or local). When implementing health policy options, managers usually need to tackle several types of obstacles. Not only it is necessary to consider them, but also to find ways to overcome them, especially those related to cultural and social representations of health care users and workers. The following are some difficulties that may be encountered when implementing each of the options and issues that may give rise to iniquities, especially in Upper-Middle Income Countries.

### Option 1. Promoting Educational Actions on Prudent Prescribing Directed to Prescribers

Implementing these interventions may aggravate iniquities when the prescriber does not participate in those activities, whatever the reasons, which may be a consequence of institutional disorganization, lack of personal motivation, or overvaluation of the knowledge they already have.

In the literature, the barriers that must be overcome may be encountered both at the individual level (courses and training of their interest and a belief that empirical knowledge is enough on its own), and at the collective level (communication difficulties among teams, infrastructure, a lack of available time to perform those activities, and punitive management, all of which can have a negative impact on professionals). In addition, difficulties may arise due to insufficient human resources or in complying with previously established guidelines (Carvalho et al., 2011; Bonadiman et al., 2013; Marchon and Mendes, 2014; Ugarte and Acioly, 2014; Santos, 2016; Silva, 2016).

### Option 2. Incorporating Computerized Alert Systems Into Clinical Practice

It should be highlighted that the implementation of these electronic resources requests some infrastructure (for example, computer or Internet access, human resources for support), as well as actions to raise awareness about and encourage the use of these technologies by prescribers.

The obstacles observed include a lack of rapid and simplified access to information by means of electronic systems in emergency situations (Cassiani et al., 2003; Gimenes et al., 2006), a lack of culture regarding the adequate inputting of information into the system (Cassiani et al., 2003; Marchon and Mendes, 2014), and a lack of participation in trainings aimed at enhancing the understanding of how the electronic system actually works. It is also important to note that these systems require financial resources, which can make them difficult to deploy (Freire et al., 2004).

### Option 3. Implementing the Use of Tools for Guiding Medication Prescribing

These tools are tailored for use mostly in the elderly population, which therefore limits their use in the entire population. Furthermore, the difficulty of access or even the lack of knowledge about these resources precludes them from being used in the clinical practice (Jano and Aparasu, 2007; Soares et al., 2011; Hill-Taylor et al., 2016; Hyttinen et al., 2016; Valencia et al., 2016).

Based on the tools, it can be noted that the lack of knowledge about the resources (Miasso et al., 2006), not considering specific characteristics of the patient (Hyttinen et al., 2016) and the constant updates (Soares et al., 2011) are all obstacles to their incorporation and use.

### Option 4. Encouraging Patient Care by a Multidisciplinary Team, With the Participation of a Pharmacist

Among the barriers that we found, there are a reduced number of professionals, work overload, a lack of communication among team members (Silva et al., 2007), not to mention resistance to incorporating the pharmacist into the care management staff.

In addition, we have also observed that verbal interaction among professionals (pharmacists and doctors) alone, does not produce significant results (Silva, 2016). Not sharing the patients' clinical data (medical records, for example) with all professionals that exert an influence over the therapeutic conduct, hamper prescription validation (Cardinal and Fernandes, 2014). It should also be emphasized that inadequate resources may prevent professionals from being employed or replaced.

### EVIDENCE GAPS

Further studies should be conducted on factors influencing prescribing and evaluating specific strategies (Davey et al., 2017). High-quality studies assessing the effectiveness of educational actions are still scarce in the literature (Alldred et al., 2016).

Pearson et al. (2009) reported that further studies should analyze the benefits of automated prescribing screening systems, since there is a lack of studies on the impact of the system

### REFERENCES

Alldred, D. P., Kennedy, M., Hughes, C., Chen, T. F., and Miller, P. (2016). Interventions to optimise prescribing for older people in care homes. Cochrane Database Syst. Ver. 2:CD009095. doi: 10.1002/14651858.CD009095.pub3

on drug-related adverse events, safety, quality, cost, and patient outcomes (Yang et al., 2012). Evidence of effective interventions based on computerized systems to prevent medication errors in the pediatric inpatient population is also incipient (Maaskant et al., 2015). Further research is also needed to check the effectiveness of the strategies found in the implementation of computerized alert tools (Kaushal et al., 2003; Hodgkinson et al., 2006), as well as to assess the impact of interventions on legibility and completeness of electronic prescriptions (Reckmann et al., 2009).

The use of the STOPP/START criteria remains incipient in health services, except in emergency services, and further studies are thus needed to assess this tool's efficacy in detecting potentially inappropriate prescriptions (Hill-Taylor et al., 2016).

### CONCLUSION

There are several options indicated in the scientific literature that are effective and safe to assist professionals in order to avoid or reduce medication prescribing errors in health services. Our evidence brief for policy present four options that may be useful to deal with this problem, although there is no recommendation on which one is the best. The decision to implement one or more options depends on the context where the decision makers are inserted.

The options are not exclusive and can be used together, according to the local reality of implementation.

When implementing these options, however, it should be taken into account that the number of studies is still incipient and confidence in the results could be improved with further research with high methodological quality.

### AUTHOR CONTRIBUTIONS

BA, RM, JB, and TT contributed with the design and conception of the study. BA and RM wrote the first draft of the manuscript. BA, RM, and TT participated in the study selection process. MB and TT contributed to the revision of the manuscript, read and approved the submitted version.

### ACKNOWLEDGMENTS

The authors are grateful to the Librarians of the Library of the Faculty of Medicine of the University of São Paulo, for providing articles of restricted access.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar. 2019.00439/full#supplementary-material

Arnold, S. R., and Straus, S. E. (2005). Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database Syst. Rev. 4:CD003539. doi: 10.1002/14651858.CD003539.pub2

Bonadiman, R. L., Bonadiman, R. L., Bonadiman, S. L., and Silva, D. A. (2013). Estudo das prescrições medicamentosas em uma farmácia básica de Itapemirim, Espírito Santo - Brasil. Acta Biomed. Bras. 4, 114−123.


tomadas em quatro hospitais brasileiros. Rev. Esc. Enferm. USP. 40, 524–532. doi: 10.1590/S0080-62342006000400011


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Araújo, Melo, Bortoli, Bonfim and Toma. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Bergenin Activates SIRT1 as a Novel Therapeutic Agent for Osteogenesis of Bone Mesenchymal Stem Cells

*Weiduo Hou1,2†, Chenyi Ye1,2†, Mo Chen3, Weixu Li1,2\*, Xiang Gao1,2, Rongxin He1,2, Qiang Zheng1,2\* and Wei Zhang1,2\**

*1 Department of Orthopedics, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China, 2 Research institute of Orthopaedics, Zhejiang University, Hangzhou, China, 3 Department of Rheumatology, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China*

*Edited by:*

*Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Brya Matthews, The University of Auckland, New Zealand Yun Qian, Shanghai Sixth People's Hospital, China Lakshman Singh, The University of Melbourne, Australia*

#### *\*Correspondence:*

*Wei Zhang zhangweilook@zju.edu.cn Qiang Zheng 2100057@zju.edu.cn Weixu Li zrlwx@zju.edu.cn*

*†These authors have contributed equally to this work.*

#### *Specialty section:*

*This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology*

*Received: 25 October 2018 Accepted: 15 May 2019 Published: 14 June 2019*

#### *Citation:*

*Hou W, Ye C, Chen M, Li W, Gao X, He R, Zheng Q and Zhang W (2019) Bergenin Activates SIRT1 as a Novel Therapeutic Agent for Osteogenesis of Bone Mesenchymal Stem cells. Front. Pharmacol. 10:618. doi: 10.3389/fphar.2019.00618*

Bone mesenchymal stem cells (BMSCs) are important candidates for bone regeneration. The role of Bergenin, a C-glucoside of 4-O-methyl gallic acid obtained from the species, Bergenia, in BMSC osteogenesis has not yet been elucidated. We therefore investigated the effects of Bergenin on the osteogenesis of BMSCs and found that Bergenin enhanced osteoblast-specific markers and downregulated the adipocyte-specific markers *in vitro*. Furthermore, using a rat calvarial defect model, we found that Bergenin significantly improved bone healing, as determined by imaging and histological analyses. Moreover, it also upregulated SIRT1 expression. A SIRT1 inhibitor (EX 527) decreased the enhanced bone mineral formation caused by Bergenin. Taken together, these findings show that Bergenin accelerated the osteogenic differentiation of BMSCs, at least partly through the activation of SIRT1.

Keywords: Bergenin, bone mesenchymal stem cells, osteogenesis, SIRT1, bone regeneration

### INTRODUCTION

In clinical practice, approximately 5–10% of fractures result in delayed healing or nonunions, followed by morbidities and functional limitations (Dozza et al., 2018). Bone-marrow-derived mesenchymal stem cells (BMSCs) have the potential to differentiate into bone tissue, making them attractive candidates for bone regeneration (Walmsley et al., 2016). Accordingly, it is essential to identify therapeutic strategies to enhance BMSC osteogenesis.

Bergenin is a colorless, crystallin isocoumarin primarily obtained from the species, Bergenia. It is a C-glucoside of 4-O-methyl gallic acid (Barai et al., 2018) and has been reported to engage in antioxidant, anti-inflammatory, antiarthritic, immunomodulatory, antinarcotic, wound-healing, antidiabetic, and *in vitro* neuroprotective activities. Barai et al. (2018) reported that Bergenin might prevent neurodegenerative disorders. Bergenin inhibited methylglyoxal-induced oxidative stress and inflammation-induced cytokine expression in MC3T3-E1 cells (Lee and Choi, 2018). Wang et al. (2017) reported that Bergenin ameliorated experimental colitis in mice by enhancing expression of SIRT1 to inhibit NF-κB-mediated macrophage activation. Based on the promising beneficial role of SIRT 1 on osteogenesis and bone metabolism (Feng et al., 2016; Deng et al., 2017; Zainabadi et al., 2017; Qu et al., 2018; Zhang et al., 2018; Wang et al., 2019), it is of great interest to explore the possible impact of Bergenin on osteogenesis.

To date, the pharmacological actions of Bergenin during osteogenesis have not yet been elucidated. We hypothesized that Bergenin may promote the osteogenic differentiation of BMSCs through the activation of SIRT1. The results of our study showed that Bergenin enhanced the osteogenic differentiation of BMSCs both *in vitro* and *in vivo*.

### MATERIALS AND METHODS

### Cells and Reagents

Human BMSCs, as previously reported (Zhang et al., 2015), were purchased from Cyagen Biosciences (from multiple healthy adult donators aged 18–45 years, HUXMA-01001, Guangzhou, China). These cells can differentiate into osteoblasts, adipoblasts, and chondrocytes under specific induction conditions. Adherent cells were trypsinized and passaged after reaching 80% confluence (2–3 days after seeding). Cells from passages 3–5 were used in subsequent experiments. As previously reported (Wang et al., 2017), Bergenin (purity > 99%) was purchased from JingZhu Biological Technology (Nanjing, China). EX 527, a SIRT1 inhibitor, was prepared by Selleck Chemicals (Houston, TX, USA). This study was carried out in accordance with the principles of the Basel Declaration and recommendations of Zhejiang University. The protocol was approved by the Animal Ethics Committee of Zhejiang University.

### Cytotoxicity Assay

To evaluate the impact of Bergenin on the viability of BMSCs, CCK-8 (Dojindo, Kumanoto, Japan) assay and 3-(4, 5-dimethylthiazolyl-2)-2,5-diphenyltetrazolium bromide (MTT) assay (Beyotime, Shanghai, China) were applied. The cells (5000/ well) were seeded into 96-well plates and allowed to adhere for 24 h. After that, cells were treated with different concentrates of Bergenin. After the treatment for 1, 5, and 7 days, the medium was removed, and the cells were treated with 10% CCK-8 solution or 0.5 mg/ml MTT solution in 100 μl of low-glucose Dulbecco's Modified Eagle's Medium without fetal bovine serum for 3 h at 37°C. Absorbance at 450 nm, which was directly proportional to cell proliferation, was measured by a microplate reader (ELX808; BioTek, Winooski, VT, USA).

### Osteogenic Differentiation and Adipogenic Differentiation Protocol of BMSCs

BMSCs (3 × 104 /cm2 ) were cultured in complete growth medium (HUXMA-90011, Cyagen Biosciences) and incubated at 37°C under 5% CO2. For osteogenic differentiation, the cells were subsequently cultured in osteogenic induction medium (HUXMA-90021; Cyagen Biosciences). For adipogenic differentiation of BMSCs, cells were induced in adipogenic induction medium (HUXMA-90031; Cyagen Biosciences). The cells were maintained by the addition of fresh osteogenic induction medium every 2–3 days.

### Measurement of Alkaline Phosphatase (ALP) Activity

According to a previous study (Zhou et al., 2015a), we first lysed the cells with RIPA lysis buffer (Beyotime, Shanghai, China) to measure the ALP activity of cells. Then, ALP activity was measured using a *p*-nitrophenyl phosphate colorimetric determination (Sigma-Aldrich, Shanghai, China). All tests were performed according to the manufacturer's protocols. The ALP activity was also measured in blood samples taken at the time of animal sacrifice.

### Alizarin Red (ARS) and Oil Red O Staining

After the induction of osteogenic differentiation, mineral deposition was assessed by ARS (Cyagen Biosciences). Cells were fixed in 4% paraformaldehyde (Sangon Biotech, Shanghai, China) for 15 min at room temperature and then washed with distilled water twice. A 1% solution of Alizarin Red was added and incubated for 10 min at room temperature; this was followed by rinsing with distilled water. Photographs of images were then taken using an inverted microscope (Leica, Wetzlar, Germany).

After the induction of adipogenic differentiation, fat droplet was assessed by Oil Red O staining kit (Cyagen Biosciences). Cells were fixed in 4% paraformaldehyde (Sangon Biotech, Shanghai, China) for 15 min at room temperature and then washed with distilled water twice. An Oil Red O staining was added and incubated for 30 min at room temperature; this was followed by rinsing with distilled water. Photographs of images were then taken using an inverted microscope (Leica, Wetzlar, Germany).

### Immunofluorescence

Cells (3 × 104 /cm2 ) were cultured in induction medium in a 12-well plate, and runt-related transcription factor 2 (RUNX2) and SIRT1 were detected using a fluorescence microscope (EU5888; Leica) after 3 days' induction. Briefly, the cells were fixed in 4% paraformaldehyde for 15 min at room temperature after treatment. They were then blocked for 30 min in 0.01% Triton X-100 and 5% bovine serum albumin. Fixed cells were washed and incubated overnight with anti-RUNX2 (#12556S; 1:400; Cell Signaling Technology, Shanghai, China) and SIRT1 (#8469S; 1:100; Cell Signaling Technology). Cells were then incubated with a fluorescence-conjugated secondary antibody (ab150077 or ab150075, Abcam, Shanghai, China) for 120 min at room temperature, and nuclei were stained with 4',6-diamidino-2-phenylindole (DAPI; Sigma-Aldrich, Shanghai, China) for 5 min; they were then observed using an inverted fluorescence microscope (Leica).

### Quantitative Real-Time Polymerase Chain Reaction

Total cellular RNA was isolated using RNAiso reagent (Takara Bio, Kusatsu, Japan) and reverse-transcribed into cDNA in a reaction volume of 20 μl with Prime Script RT Master Mix (Takara Bio) according to our previous study (Zhang et al., 2018). After that, 1 µl of cDNA was used as the template for the quantitative real-time polymerase chain reaction (qPCR). All gene transcripts were quantified by PCR using the Power SYBR® Green PCR Master Mix (Takara Bio) on the ABI StepOnePlus System (Applied Biosystems, Warrington, UK). According to the manufacturer's instructions, the cycle conditions of PCR were as follows: 95°C for 30 s, 40 cycles of 95°C for 5 s, and 60°C for 30 s. The relative target gene expression levels were calculated using the 2−∆∆Ct method. The mRNA of the target genes and the housekeeping gene (GAPDH) were quantified in separate tubes. All primers were synthesized by Sangon Biotech. Primers used are followed: GAPDH, Forward: CGGACCAATACGACCAAATCCG; Reverse: AGCC ACATCGCTCAGACACC; ALP, Forward: TTGACCTCCTCGG AAGACACTCTG; Reverse: CGCCTGGTAGTTGTTGTGAGCA TAG;RUNX2, Forward: ACTTCCTGTGCTCGGTGCT; Reverse: GACGGTTATGGTCAAGGTGAA; COL1A, Forward: GAGA GCATGACCGATGGATT; Reverse: CCTTCTTGAGGTTGCC AGTC; PPARγ, Forward: GGGATCAGCTCCGTGGATCT; Reverse: TGCACTTTGGTACTCTTGAAGTT; SIRT1 Forward: TAGCCTTGTCAGATAAGGAAGGA; Reverse: ACAGCTTCA CAGTCAACTTTGT.

### Western Blot Analysis

Cells were lysed in RIPA lysis buffer (Beyotime) with a proteasome inhibitor (Beyotime). Total proteins were separated by 10% sodium dodecyl sulfate-polyacrylamide gel electrophoresis and then transferred to a polyvinylidene fluoride membrane (Millipore, Shanghai, China). After blocking in 5% nonfat milk for 2 h, the membranes were incubated overnight at 4°C with antibodies specific for glyceraldehyde 3-phosphate dehydrogenase (A00227- 1; 1:8,000; Boster Biological Technology, Wuhan, China), RUNX2 (#12556S; 1:1,000; Cell Signaling Technology), PPARγ (#2435S; 1:1,000; Cell Signaling Technology), or SIRT1(#8469S; 1:1,000; Cell Signaling Technology). Horseradish-peroxidase-conjugated goat anti-rabbit IgG (BA1056; 1:5000; Boster Biological Technology) was used as a secondary antibody for 2 h at room temperature. The immunoreactive bands were detected using an enhanced chemiluminescent detection reagent (Millipore, Shanghai, China). Signal intensity was measured using a Bio-Rad XRS chemiluminescence detection system (Bio-Rad, Hercules, CA, USA).

### *In Vivo* Evaluation of Rats

The accelerated bone-forming ability of Bergenin was assessed in a calvarial defect model in Sprague–Dawley rats (Aghaloo et al., 2007; Sun et al., 2014). All experiments were conducted in accordance with the Animal Care and Use Committee guidelines of Zhejiang province and the Institutional Animal Care and Use Committee of Zhejiang University. Threemonth-old male (approximately 200 g) Sprague–Dawley rats were obtained from the Academy of Medical Sciences of Zhejiang province. According to our previous studies (Ye et al., 2018; Zhang et al., 2018), rats were anesthetized with 0.3% sodium pentobarbital (Sigma-Aldrich) intraperitoneally at 30 mg/kg body weight. A trephine drill was utilized under constant irrigation to create a 4-mm, critically sized defect in the parietal bone. Care was taken to avoid injury to the underlying dura mater. All rats received the above surgical procedures. The rats were divided randomly into two groups: a control group (sham) and an experimental (Bergenin) group (*n* = 6/group). To conduct effective statistical analysis, sample size ≥ 4/group is required. In this study, *n* = 6/group was set, which was consistent with our previous study (Chen et al., 2017) and other published studies (Chen et al., 2017; Deng et al., 2018; Wang et al., 2018). As reported in previous studies (Gao et al., 2015; Yun et al., 2015; Wang et al., 2017), the Bergenin group was intraperitoneally treated with Bergenin in phosphate-buffered saline (PBS) at 50 mg/kg body weight weekly after surgery, throughout the 8 weeks; the sham group was treated with an equal volume of PBS.

The rats were sacrificed in a CO2 chamber at 8 weeks after surgery. The cranium was collected for radiographic and histological analyses, and the serum was assessed for ALP activity.

### Microcomputed Tomography Evaluation

To evaluate callus formation and bridging bone formation at bone defect sites 8 weeks postoperatively, the craniums were scanned using a μCT-100 imaging system (Scanco Medical, Brüttisellen, Switzerland) with X-ray energy settings of 70 kVp, 1,024 reconstruction matrix, 14.8-μm slice thickness, and an exposure time of 300 ms. According to previous studies (Shinozaki et al., 2014; Toda et al., 2014), after threedimensional (3D) reconstruction using the manufacturer's software was conducted, a square region of interest (ROI) centered on the area of the defects was selected for further qualitative and quantitative analyzes. The bone volume fraction (bone volume/total volume, BV/TV) was calculated by 3D standard microstructural analysis.

### Histological Evaluation

Samples were fixed with 4% paraformaldehyde for 24–48 h at room temperature and decalcified using 10% EDTA (Sigma-Aldrich) with a solution change once weekly for more than 8 weeks at 4°C before embedding in paraffin. Serial sections with a thickness of 5 μm were cut and mounted onto polylysine-coated slides. Consistent with previous studies (Shinozaki et al., 2014; Toda et al., 2014), the cross section of the central area of the defects was serially cut at 5 μm thick for further histological evaluation. Hematoxylin and eosin and Masson staining were performed separately on consecutive tissue sections, as described in our previous study (Zhang et al., 2016).

### Statistical Analysis

Statistical analysis was performed using SPSS statistical software for Windows, version 19.0 (IBM, Armonk, NY, USA). All experiments were performed in at least triplicate, and the data are presented as the mean ± standard deviation. Statistical significance was determined using a two-tailed Student's *t* test when comparing two groups and by a one-way analysis of variance followed by Bonferroni's *post hoc* test when comparing more than two groups. A value of *P ≤* 0.05 was considered to represent a statistically significant difference.

### RESULTS

### Bergenin had no Adverse Effect on the Viability of BMSCs

To determine the cytotoxic potential of Bergenin, its effects on BMSC viability were evaluated by the CCK-8 and MTT assay. No significant cytotoxic effect was observed between groups treated with and without Bergenin (**Figure 1**).

### Bergenin Upregulated the Levels of Osteo-Specific Markers Under Osteogenic Conditions

ALP activity is an important marker for the osteogenesis of BMSCs. After treatments using 1–100 μM Bergenin, the ALP activity of the experimental group was increased on days 3 and 5 after the induction of osteogenic differentiation, compared with that of the control group (**Figure 2A**).

To assess the role of Bergenin in the osteogenic differentiation of BMSCs, the levels of osteo-specific genes and proteins, including ALP, RUNX2, and COL1A1, were determined. The qPCR analysis revealed that the ALP, RUNX2, and COL1A1 mRNA levels were significantly increased on days 3 and 5 after the induction of osteogenic differentiation in BMSCs in the presence versus the absence of Bergenin (1 or 100 μM) (*P* < 0.05; **Figure 2B**–**E**).

Western blot analysis revealed that RUNX2 protein expression was increased by 10–100 μM Bergenin treatment on day 3 after the induction of osteogenic differentiation. Moreover, on day 5, there was a higher level of RUNX2 protein expression in BMSCs treated with certain doses of Bergenin, when compared with those from the control group (**Figure 2F**–**K**). Moreover, Alizarin Red staining showed significantly more calcium deposits in the 1–100 µM Bergenin treatment group (**Figure 2L**). Using immunofluorescence analysis, we also found higher levels of Runx2 and COL1A1 due to Bergenin on day 3 (10 or 100 µM) (**Figure 2M**).

### Bergenin Downregulated the Levels of Adipo-Specific Markers Under Adipogenic Conditions

PPARγ is a master regulator of adipogenic differentiation of BMSCs. Western blot analysis revealed that PPARγ protein expression was decreased by 1–100 μM Bergenin treatment on days 3 and 5 after

the induction of osteogenic differentiation (**Figure 3A** and **B**). Furthermore, oil Red O staining found significantly less fat droplet in the 1–100 µM Bergenin treatment group (**Figure 3C**).

### Bergenin Activated SIRT1 Expression

Based on the vital beneficial role of SIRT1 on osteogenesis (Feng et al., 2016; Deng et al., 2017; Zainabadi et al., 2017; Qu et al., 2018; Zhang et al., 2018; Wang et al., 2019) and the results of previous studies (Wang et al., 2017), we measured the SIRT1 mRNA and protein levels. The results of qPCR and Western blot analyses indicated lower expression of SIRT1 in the control group, compared with certain dose of the Bergenin-treated group (**Figure 2D** and **E**). Moreover, using immunofluorescence analysis, we also found increased levels of SIRT1 due to Bergenin on day 3 after the induction of osteogenic differentiation (10 or 100 µM) (**Figure 2H**).

### Accelerated Osteogenic Differentiation of BMSCs Due to the Presence of Bergenin was Partially Impaired by a SIRT1-Specific Inhibitor (EX 527)

To confirm the role of SIRT1, we investigated the effect of its specific inhibitor (EX 527) on the osteogenic differentiation of BMSCs. According to previous studies, EX-527 is the most specific and potent SIRT1 inhibitor (Li et al., 2011; Sasca et al., 2014; Nikseresht et al., 2018). EX-527 (10 μM) was added to osteogenic induction medium (Li et al., 2011); after 3 days, the expression of osteo-specific markers was determined. **Figure 4A** and **B** shows that lower levels of RUNX2 were found in the Bergenin + inhibitor-treated cells compared with cells treated with Bergenin alone. Blocking SIRT1 also decreased the levels of mineralization (**Figure 4C**). In a similar manner, the downregulation of SIRT1 by EX-527 inhibited the expression of osteo-specific genes (*ALP*, *RUNX2*, and *COL1A1*) (**Figure 4D**).

### Bergenin Accelerated Bone Formation in a Calvarial Defect Model in Rats

To assess the *in vivo* effect of Bergenin on osteogenesis, a rat calvarial defect model was used. The morphology of new bone formation was characterized using microcomputed tomography (micro-CT)

FIGURE 4 | Effects of a SIRT1-specific inhibitor (EX 527) on the enhanced expression of an osteoblast-specific gene of bone marrow mesenchymal stem cells by Bergenin. (A and B) The expression of RUNX2 and SIRT1 in blank, control + EX 527, Bergenin (10 μM), and Bergenin (10 μM) + EX 527 groups was determined by Western blot analysis. EX 527 (10 μM) was applied for 1 h, followed by culture in osteogenic induction medium with Bergenin for 3 days. Protein expression levels were normalized to glyceraldehyde-3-phosphate dehydrogenase. Data are expressed as the mean ± standard deviation (SD) of three independent experiments, and one of three independent experiments is shown. Data are expressed as the mean ± SD. \**P* < 0.05 vs. group with osteogenic induction medium alone. (C) Alizarin red staining and quantification of mineralization at day 12 of osteogenic differentiation. (D) The mRNA expression of RUNX2, ALP, and COL1A1 in blank, control + EX 527, Bergenin (10 μM), and Bergenin (10 μM) + EX 527 groups was determined by quantitative reverse transcription polymerase chain reaction. EX 527 (10 μM) was applied for 1 h, followed by culture in osteogenic induction medium with Bergenin for 3 days. mRNA expression levels were normalized to GAPDH. \**P* < 0.05 vs. BMSCs treated with osteogenic induction medium alone.

analysis. Representative images are shown in **Figure 5A**. Micro-CT revealed that the Bergenin group showed increased bone formation in the calvarial defect compared with the sham group at 8 weeks after surgery. Quantification of the mineralized areas also showed a significant increase in mineralization tissues in the Bergenin group (**Figure 5B**). Additionally, higher levels of ALP activity were detected in the serum of the Bergenin group (**Figure 5C**).

Representative histological photographs of each group, including hematoxylin and eosin and Masson's trichrome staining, are shown in **Figure 5D** and **E**. In the Bergenin group, a thick callus consisting of newly formed woven bone tissue was observed in the defect area, with a narrower distance between bone defects, compared with the control animals.

### DISCUSSION

To the best of our knowledge, this is the first demonstration that the SIRT1 agonist Bergenin effectively promoted both the *in vitro* and *in vivo* osteogenesis. First, it promoted the osteo-specific markers and mineralization forming of BMSCs and inhibited the adipogenesis *in vitro*. It also enhanced bone formation in a bone defect model and upregulated the expression of SIRT1. Moreover, blocking the activation of SIRT1 decreased the enhanced osteogenesis of BMSCs due to the presence of Bergenin. Taken together, these findings indicated that Bergenin accelerated the osteogenic differentiation of BMSCs, at least partly through upregulation of SIRT1.

Bone defects and nonunions occur frequently in clinical settings. Bergenin is one of the plant-derived chemical constituents in traditional medicine and engages in multiple biological activities. Importantly, previous studies have reported that it can affect bone metabolism. Nazir et al. (2007) reported that it showed antiarthritic activity through the possible modulation of the Th1/Th2 cytokine balance. Kwang et al. revealed that Bergenin had protective effects on methylglyoxal-induced cytotoxicity in MC3T3-E1 osteoblasts (Suh et al., 2018). Pretreatment with Bergenin before methylglyoxal exposure reduced mitochondrial dysfunction,

FIGURE 5 | Bergenin accelerated bone formation in a calvarial defect model in rats. (A) Microcomputed tomography analysis for bone healing. (B) Bone volume was analyzed by microcomputed tomography. Data are expressed as the mean ± standard deviation (SD). Reactions were performed in triplicate. \**P* < 0.05 vs. bone defects with PBS (sham group). (C) ALP activity in serum, \**P* < 0.05 vs. bone defects with PBS (sham group). (D) Histologic analysis for bone healing. HE, hematoxylin and eosin staining; Magnification ×40 (bar = 500 μm) and ×100 (bar = 200 μm). (E) Masson, Masson's trichrome staining. Magnification ×40 (bar = 500 μm) and ×100 (bar = 200 μm).

indicating that Bergenin may prevent the development of diabetic osteopathy (Lee and Choi, 2018). In our study, we found that Bergenin upregulated the levels of osteo-specific markers (ALP, RUNX2, and COL1A1) and accelerated the mineralization of BMSCs *in vitro*. Moreover, increased bone formation was found after Bergenin treatment *in vivo*. Of note, numerous studies have demonstrated that fat-induction factors inhibit osteogenesis, and, conversely, bone-induction factors hinder adipogenesis (Chen et al., 2016a; Chen et al., 2016b). Likewise, in our study, we found that Bergenin could inhibit the levels of adipo-specific marker (PPARγ) and suppress the lipid droplet formation.

Increasing evidence has demonstrated that SIRT1 can promote bone formation (Cohen-Kfir et al., 2011; Chen et al., 2014; Qu et al., 2016). Tseng et al. (2011) reported that resveratrol promoted the osteogenesis of human mesenchymal stem cells by upregulating RUNX2 gene expression *via* the SIRT1/FOXO3A axis. Moreover, SIRT1 also inhibits adipogenesis (Zhou et al., 2015b). A previous study indicated that Bergenin could improve the expression of SIRT1 by inhibiting inflammation (Wang et al., 2017). Consistent with these results, we also found that Bergenin acted as an activator of SIRT1. Bergenin upregulated protein and mRNA expression of SIRT1 in a dose-independent manner. Furthermore, the accelerated osteogenic differentiation of BMSCs due to the presence of Bergenin was partially impaired by the addition of a SIRT1-specific inhibitor (EX 527).

Our study has several limitations. First, although our results indicated that Bergenin upregulated SIRT1 to enhance BMSC osteogenesis, the underlying mechanism remains unclear. Second, as the long-term dose–response relationship and safety of Bergenin were not adequately characterized, the translational relevance of these findings needs to be confirmed. In addition, in this study, we did not compare the *in vivo* results with the positive control group, which absolutely causes osteogenesis, by classic drugs or gold standard clinically. Future studies are required.

### REFERENCES


### CONCLUSION

Based on our data, we found that Bergenin enhanced the osteogenic differentiation of BMSCs, partly through activation of SIRT1. Bergenin may therefore be a novel therapeutic agent for the treatment of bone defects.

### ETHICS STATEMENT

All experiments were conducted in accordance with the Animal Care and Use Committee guidelines of Zhejiang province and the Institutional Animal Care and Use Committee of Zhejiang University.

### AUTHOR CONTRIBUTIONS

WZ and QZ contributed to the design and funding sources to this study. WZ, WH, and MC drafted the manuscript. WL, XG, RH, and CY did all the *in vitro* and *in vivo* parts of the study. All authors have contributed significantly and read and approved the final manuscript.

### FUNDING

This study was supported by a grant from the National Natural Science Foundation of China (No. 81802221, No. 81672147, No. 81501906, and No. 81572124), China postdoctoral science foundation (2018M640567) and the Zhejiang Provincial postdoctoral preferred foundation (zj20180131).

### ACKNOWLEDGMENTS

This work was also supported by a grant from China postdoctoral science foundation (2018M640567) and the Zhejiang Provincial postdoctoral preferred foundation (zj20180131).

*vitro* and enhancement of bone repair *in vivo*. *Theranostics* 7 (5), 1072–1087. doi: 10.7150/thno.18193


DPSCs through Wnt/beta-catenin signal. *In Vitro Cell Dev. Biol. Anim.* 52 (10), 1001–1011. doi: 10.1007/s11626-016-0070-9


**Conflicts of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Hou, Ye, Chen, Li, Gao, He, Zheng and Zhang. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Prospects for the Use of New Technologies to Combat Multidrug-Resistant Bacteria

*Renata Lima1\*, Fernando Sá Del Fiol2 and Victor M. Balcão3,4*

*1 LABiToN—Laboratory of Bioactivity Assessment and Toxicology of Nanomaterials, University of Sorocaba, Sorocaba, Brazil, 2 CRIA—Antibiotic Reference and Information Center, University of Sorocaba, Sorocaba, Brazil, 3 PhageLab—Laboratory of Biofilms and Bacteriophages, i(bs)2—intelligent biosensing and biomolecule stabilization research group, University of Sorocaba, Sorocaba, Brazil, 4 Department of Biology and CESAM, University of Aveiro, Campus Universitário de Santiago, Aveiro, Portugal*

The increasing use of antibiotics is being driven by factors such as the aging of the population, increased occurrence of infections, and greater prevalence of chronic diseases that require antimicrobial treatment. The excessive and unnecessary use of antibiotics in humans has led to the emergence of bacteria resistant to the antibiotics currently available, as well as to the selective development of other microorganisms, hence contributing to the widespread dissemination of resistance genes at the environmental level. Due to this, attempts are being made to develop new techniques to combat resistant bacteria, among them the use of strictly lytic bacteriophage particles, CRISPR–Cas, and nanotechnology. The use of these technologies, alone or in combination, is promising for solving a problem that humanity faces today and that could lead to human extinction: the domination of pathogenic bacteria resistant to artificial drugs. This prospective paper discusses the potential of bacteriophage particles, CRISPR–Cas, and nanotechnology for use in combating human (bacterial) infections.

#### *Edited by:*

*Brian Godman, Karolinska Institute (KI), Sweden*

#### *Reviewed by:*

*Amit P. Bhavsar, University of Alberta, Canada Yun Qian, Shanghai Sixth People's Hospital, China*

> *\*Corespondence Renata de Lima renata.lima@prof.uniso.br*

#### *Specialty section:*

*This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology*

*Received: 15 February 2019 Accepted: 28 May 2019 Published: 21 June 2019*

#### *Citation:*

*Lima R, Del Fiol FS and Balcão VM (2019) Prospects for the Use of New Technologies to Combat Multidrug-Resistant Bacteria. Front. Pharmacol. 10:692. doi: 10.3389/fphar.2019.00692*

Keywords: multidrug-resistant bacteria, bacteriophage particles, phage therapy, CRISPR–Cas, nanotechnology

### BACTERIAL RESISTANCE

Since their discovery in 1929, antibiotics have been widely used in human and veterinary medicine, either for treatments or in attempts to prevent bacterial infections. The excessive use of antibiotics, whether for prevention or treatment, has significantly increased the level of bacterial resistance worldwide (Ali et al., 2018). The associated numbers of human deaths are alarming, reaching 50,000 per year in the United States and Europe (Simlai et al., 2016), with an estimated 10 million deaths per year by 2050, surpassing the current deaths resulting from all types of cancer (approximately 8.2 million) (Jansen et al., 2018).

The first list of antibiotic-resistant pathogens was published by the World Health Organization (WHO) in 2017. This list showed that out of the 12 resistant pathogens, seven were noted to be resistant to beta-lactam antibiotics. Consequently, there is renewed focus on the production of new antibiotics, establishing a goal for future research strategies (WHO, 2017).

The overuse and misuse of antibiotics in humans have led to the selective emergence of bacteria resistant to the currently available antibiotics, as well as resistant non-pathogenic microbiota, hence leading to the generalized dissemination of resistance genes at the environmental level (Nitsch-Osuch et al., 2016). There is greatest concern when this phenomenon occurs with *Enterococcus*

spp., *Staphylococcus aureus*, *Klebsiella pneumoniae*, *Acinetobacter baumannii*, *Pseudomonas aeruginosa*, and *Enterobacter* spp., together given the acronym ESKAPE, which highlights the ability of these microorganisms to escape the action of antimicrobial agents (Boucher et al., 2009).

Antimicrobial resistance has become globalized, following the first reports of its appearance in India, with its subsequent spread to Pakistan, the United States, Canada, Japan, and the United Kingdom (Rios et al., 2016). This resistance can occur in different ways, depending on the acquired and selective genetic changes or insertion of external genes, which leads to previously non-existent responses. Several mechanisms of resistance have emerged in recent times, including alteration of the target (by a DNA gyrase), increased efflux (export of a drug out of the microorganism), inactivation of fluoroquinolones (by an aminoglycoside N-acetyltransferase), inhibition of the 30S ribosomal subunit (by aminoglycosides), and protection of the target by DNA-binding proteins (the Qnr family) (Redgrave et al., 2014; Munita and Arias, 2016; Kapoor et al., 2017).

Some of these changes are already well known, such as alteration of the chemical structure of antimicrobial agents (Alekshun and Levy, 2007), decrease of the concentration of the antimicrobial at its site of action (Gonzalez-Bello, 2017; Willers et al., 2017), changes in the target of antimicrobial action (Sieradzki and Markiewicz, 2004), and alteration of membrane permeability (Hao et al., 2018). There are mechanisms of permeability reduction that do not involve porin expression, such as changes in the cell envelope of *P. aeruginosa* that are associated with resistance to polymyxin B (Falagas and Kasiakou, 2005). In addition to antibiotics that act on the cell wall, such as penicillins and glycopeptides, the activities of other antimicrobials that act on the bacterial ribosome may also decrease due to changes in their primary target. This phenomenon mainly affects macrolides and tetracyclines (Poehlsgaard and Douthwaite, 2005; Wu et al., 2005).

The presence of these mechanisms of resistance is increasingly common in large numbers of microorganisms, due to the selective pressure exerted by antimicrobials, leading to a natural selection that results in the dominance of certain groups of resistant bacteria, with concomitant death of sensitive microorganisms (Tello et al., 2012).

In a meta-analysis carried out by Bell et al. (2014), in which 243 studies were evaluated, it was concluded that "Increased consumption of antibiotics may not only produce greater resistance at the individual patient level but may also produce greater resistance at the community, country, and regional levels, which can harm individual patients." Another study of the same year evaluated the consumption of antibiotics worldwide between 2000 and 2010. It was found that the consumption of antibiotics increased by around 36%, with the countries of the BRICS group (Brazil, Russia, India, China, and South Africa) accounting for approximately 76% of the increase (Van Boeckel et al., 2014).

Therefore, the data reflect a worrying trend regarding the treatment of infectious diseases, since not only are these drugs being increasingly used (Van Boeckel et al., 2014), but also their use is directly proportional to the increase in resistance indicators (Bell et al., 2014). In the absence of any significant discovery of new molecules for the control of resistant microorganisms (Hogberg et al., 2010), there is an urgent need for redefining the relationship of humans with infectious diseases.

In summary, the problem faced in relation to bacterial resistance is a concern that must be urgently addressed, since functional meta-genomic studies of soil microorganisms have revealed a wide range of genetic determinants that confer resistance to antibiotics, of which only one fraction has been described in human pathogens (Forsberg et al., 2014).

Hence, there is a pressing need for a new generation of antimicrobials able to mitigate the spread of antibiotic resistance and preserve beneficial microbiota. Among the possibilities for the solution of problems related to bacterial resistance, the use of nanotechnology, CRISPR–Cas9, and therapy with bacteriophage particles can be highlighted as potential future strategies. These techniques could be employed individually to directly combat microorganisms, as well as in combination in integrated strategies.

The scientific community has indicated that there are no perspectives for any significant clinical introduction of new antimicrobials in the short term. The main recommended approach is rational use of the classical antibiotics that have been used for the past 50 years, together with techniques that enhance their activity. This may be achieved using substances that increase antibiotic activity by reducing or blocking the resistance mechanism, such as beta-lactamase, efflux pump, and quorum sensing inhibitors, as well as bacteriophages and new drug delivery systems, among other techniques (Moo et al., 2019; Mulani et al., 2019; Pham et al., 2019; Vikesland et al., 2019).

### CRISPRs

CRISPRs (clustered regularly interspaced short palindromic repeats) are adaptive immune systems derived from bacteria and archaea. CRISPR–Cas systems use RNA for target DNA recognition and the Cas enzyme for subsequent destruction of nucleic acids, so they require only one protein for binding and cleavage. Due to this simplicity, researchers have developed a new molecular tool based on natural CRISPRs (**Figure 1**). This tool has different applications, one of them being the possibility of antimicrobial action, since they are cytotoxic systems that can be directed to kill bacteria, immunizing them against resistant plasmids (Sorek et al., 2013; Bikard et al., 2014; Hsu et al., 2014).

For medical purposes, CRISPR–Cas systems can enable the selective and specific removal of microorganisms. Although there are other antimicrobial approaches, they offer only partial solutions, while CRISPR systems are generalized and programmable strategies (Gomaa et al., 2014) that can be employed to selectively and quantitatively remove individual bacterial strains, based purely on sequence information, hence creating opportunities in the treatment of multidrug-resistant infections.

In studies of the use of these systems as antimicrobials, Gomaa et al. (2014) reported that both heterologous and endogenous systems could selectively kill bacterial species and strains. It was shown that all sequences in the target

genome led to cell death, suggesting that, theoretically, any genomic location could be a distinct target for antimicrobials based on CRISPRs. Another possibility would be the use of this technology for antimicrobial action using RNAguided nucleases (RGNs), targeting specific resistance genes or undesirable polymorphisms, allowing programmable remodeling of the microbiota (Citorik et al., 2014).

In a study carried out by Fuente-Núñez and Lu (2017) concerning CRISPR–Cas constructs designed to function as precision antimicrobials, these were shown to be capable of eliminating drug-resistant microbes, with CRISPR–Cas selectively targeting genes involved in antibiotic resistance, biofilm formation, and virulence. However, although studies show that CRISPRs are effective, there are still problems to be overcome in relation to an efficient delivery vehicle, which is the next step for the implementation of CRISPR–Cas systems as antimicrobial agents (Beisel et al., 2014). Focusing on the problem of CRISPR transportation and delivery, Pan et al. (2017) were able to identify eight depolymerases in the multi-host bacteriophage K64-1, which, together with K64dep (S2-5), characterized elsewhere, gave a total of nine capsule depolymerases.

Currently, obtaining bacteriophages as carriers of CRISPRs is still a challenge. Shen et al. (2018) succeeded in obtaining positive results in studies aimed at obtaining a *Klebsiella* bacteriophage by genome alteration, which was suggested as a possibility for the use of targeted CRISPRs. One option is to use nanotechnology for the delivery of CRISPRs, which could provide surface modifications that ensure the desired specificity (Yan et al., 2015). As pointed out by Pursey et al., (2018), there is still a great deal to discover concerning the use of CRISPR–Cas in the fight against resistant bacteria, with further research especially needed in relation to its safe use.

Another concern is the possibility that bacteria could present resistance against CRISPR–Cas, since the original mechanisms are present in them. However, in a study by Chen et al. (2019), performed with multidrug-resistant *Shigella*, it was shown that the bacteria that presented resistance genes also presented a decrease in the activity of natural CRISPR–Cas.

If we consider the different possibilities of target genes for CRISPR–Cas, we can conclude that there is a need for an interdisciplinary study, where there is collaboration of researchers who study sequences, find a safe way of delivery, and evaluate the existence of resistance to technology. Different studies show that bacteria tend to store different genes, and different combinations between virulence and resistance are an alarming threat, as it suggests the feasibility of adaptability. A study carried out by Oliveira Santos et al. (2018), where they showed the possible adaptability of the KPC-2 gene to different mobile elements, is an example of the need to consider different possibilities for the application of CRISPR–Cas. Regarding the onset of carbapenem-resistant *K. pneumoniae*, recent publication showed the introduction of two new DNA editing systems. One is the plasmid pCasKP-pSGKP and the other is the plasmid system pBECKP, where both systems showed efficiency in genome editing, which will facilitate further investigations for treatment of resistance to carbapenems (Wang et al., 2018).

Although the CRISPR–Cas tool offers a new possibility of fighting multidrug bacteria, some studies show that they do not present activity in some strains, as demonstrated by Hullahalli et al. (2017, 2018) in studies with *Enterococcus faecalis* where they present a study that determines the genetic basis of phenotypes associated with CRISPR–Cas tolerance, showing the importance of having a better knowledge of the response of organisms and possible strategies for dealing with conflicts induced by the use of CRISPRS, which may lead to tolerant phenotypes to this tool. Therefore, these studies show that knowledge of the genome and the metabolic pathways of the different resistant multidrug bacteria should be investigated so that resistance problems will not occur in the future in relation to new strategies used to fight resistant bacteria.

### NANOTECHNOLOGY IN THE FIGHT AGAINST RESISTANT BACTERIA

Nanotechnology applied to the synthesis of new antibiotics is an important approach, since the use of nanometric size materials can result in greater contact between the compound and the bacteria, with improved bioavailability, increased absorption, faster passage of the drug into the cell, and enhanced mucoadhesion. There is also the possibility of producing controlled release systems for the targeted delivery of encapsulated or surface adsorbed drugs (Zaidi et al., 2017; Jamil and Imran, 2018). One new approach is to use nanoparticles (NPs) of a metal such as silver, which can affect the bacterial respiration system, inducing the generation of reactive oxygen species (ROS). This approach could be used synergistically with antimicrobials, with effects such as inhibition and alteration of the synthesis of the cell wall, as well as its rupture (Shahverdi et al., 2007; Kumar et al., 2018).

One of the concerns regarding the use of nanoparticles is in relation to the resistance that bacteria can present to them, or the possibility of stimulating the transmission of MultiDrug-Resistant (MDR) genes. An example is provided by the work of Ansari et al. (2014), where Al2O3 nanoparticles were observed to promote the horizontal conjugative transfer of MDR genes, hence increasing the resistance to antibiotics.

The use of NPs to eliminate microorganisms can involve microbicidal or microbiostatic effects. In the latter case, the growth of bacteria is interrupted and the metabolic activities are halted, with microbial death then induced by the immune cells of the host. Nanotechnology can also solve problems related to drug solubility, since encapsulation can improve permeation through the membrane, increase circulation times, and enhance efficiency, while there is also the possibility of directing the drug towards the desired site of action in the body (Rodzinski et al., 2016).

The use of nanoparticles appears to have potential for the treatment of infectious diseases, especially considering that NPs may be able to access locations where the pathogens are present. However, there are a number of issues to be resolved, such as the scarcity of toxicity data, few existing preclinical studies, and the need for regulation (Zaidi et al., 2017).

### Polymeric Nanoparticles and Nanocrystals

The use of polymeric nanocapsules as carriers for antibiotics, or the use of drug nanocrystals that are stable during delivery, can be successfully applied to a range of commonly used drugs. Polylactide-*co*-glycolide (PLGA) is an especially useful substance that can be employed in nanotechnological drug delivery applications (Kalhapure et al., 2014; Hemeg, 2017; Boya et al., 2017; Shaaban et al., 2017).

Hong et al. (2017) used bacitracin A (BA) modified with PLGA for synthesis of nano-BA, resulting in a core–shell structure with an average diameter of 150 nm. It was found that the nanoparticles strongly increased the antibacterial activity, than does free BA, with effective inhibition of the growth of various types of Gram (+) and Gram (−) bacteria. The formulation provided improved wound healing in rats than did use of a commercial Polysporin® ointment.

Yu et al. (2016) reported the development of a multifunctional release system with encapsulation of gentamicin sulfate/ zirconium bis(monohydrogen orthophosphate) (α-ZrP) using chitosan (CHI). The formulation (α-ZrP CHI) extended the release of the drug, than did unencapsulated α-ZrP, which was attributed to the unique lamellar structure and the CHI encapsulation. The methodology provided a model for the future development of new delivery vehicles.

Shaaban et al. (2017) reported that nanoantibiotics produced by incorporating imipenem in PLGA or PCL nanocapsules provided better results, than did classical imipenem. The nanoencapsulated formulations showed antimicrobial and anti-adherent activities in evaluations using clinical isolates of imipenem-resistant bacteria.

Other types of nanoparticles that have received attention are lipid nanoparticles (liposomes) (Derbali et al., 2019) and nanoceramics applied in orthopedic surgeries where systemic drug administration has limitations (Kumar and Madhumathi, 2016).

Gaspar et al. (2017) reported the use of solid lipid nanoparticles containing rifabutin (RFB) for pulmonary administration to treat tuberculosis. The nanoparticles increased the activity of the drug against *M. tuberculosis* infection, suggesting that RFB-solid lipid nanoparticles (SLN) encapsulation could be a promising approach for tuberculosis treatment. A major advantage of encapsulation is that it provides sustained release of the drug, resulting in greater efficiency of treatment, as well as easier absorption, enabling satisfactory results to be achieved with a smaller amount of the active agent.

Although the use of nanoparticles can be advantageous, some studies have shown that the microenvironment where they are released (such as blood and lung fluid) may alter the creation of the nanoparticle–pathogen complex, due to the formation of a corona around the nanoparticle. Siemer et al. (2019) exposed nanoparticles to different bacteria and showed that formation of the pathogen–nanoparticle complex was assisted by its small size and that the presence of a corona significantly inhibited formation of the complex. Therefore, in addition to *in vitro* analyses, new studies are needed that consider the microenvironment in which the nanoparticle will be released and exert its action.

### Metallic Nanoparticles

The use of metallic nanoparticles can be a good option in the fight against resistant bacteria. Studies have reported the synthesis and use of different nanoparticulate metals, metal oxides, metal halides, and bimetallic materials showing antimicrobial activity. Nanoparticles have been synthesized consisting of Ag, Au, Zn, Cu, Ti, and Mg, among other metals (Zakharova et al., 2015; Hajipour et al., 2012; Sunitha et al., 2013; Dizaj et al., 2014; He et al., 2016; Senarathna et al., 2017; Eymard-Vernain et al., 2018). However, consideration should be given to their potential toxicity (Lima et al., 2012; Dakal et al., 2016; Durán et al., 2016a).

Eymard-Vernain et al. (2018) showed that MgO nanoparticles presented bactericidal action, mainly affecting the expression of genes related to oxidative stress, together with membrane alteration. Verma et al. (2018) reported excellent antibacterial activity of ZnO nanoparticles, with a size-dependent effect, since the use of smaller nanoparticles resulted in more ROS and increased cell membrane rupture.

Other studies have investigated the bactericidal potential of carbon nanotubes, either plain or functionalized, as well as their use to assist the transport and translocation of antibiotics (Cong et al., 2016; Mocan et al., 2017).

With the development of nanotechnology, many studies have been carried out concerning the application of nanoparticles as antimicrobials. These nanomaterials present different diameters, structures, and modes of action. Some of them have produced good results, showing that nanotechnology can be used as one of the strategies in the fight against multidrug-resistant bacteria in the future (**Supplementary Table 1**).

Silver nanoparticles are the most studied metallic nanoparticles, with their antimicrobial activity having been recognized by the United States Food and Drug Administration (FDA) since the year 1920. The mechanisms of action of silver nanoparticles (AgNP) on bacteria have been exhaustively investigated. There is a consensus that adhesion of the nanoparticles to the cell membrane can lead to electrostatic changes, porosity alteration, rupture, leakage of cytoplasmic content, interference in bacterial respiratory processes, blocking of enzyme activity, and DNA destruction. It has also been observed that there is the production of ROS, with consequent effects on the DNA (Choi and Hu, 2008; Durán et al., 2010; Prabhu and Poulose, 2012; Rai et al., 2012; Kon and Rai, 2013; Yuan et al., 2017).

The adhesion of nanoparticles to bacterial membranes mainly occurs due to the presence of proteoglycans (Kim et al., 2017) and results in rupture or increased porosity of the membrane. This enables access of the nanoparticles into the cell, where they can interact with enzymes and DNA (Grigor'eva et al., 2013; Kasithevar et al., 2017). AgNPs may also interact with membrane proteins, leading to cell stress, or may interact with the lipid part of the membrane, affecting its fluidity (Morones et al., 2005; Chwalibog et al., 2010). Some studies have suggested that the observed effects are actually caused by silver ions released from AgNPs (Jung et al., 2008; Xiu et al., 2011; Xiu et al., 2012; Chernousova and Epple, 2013). Accordingly, the AgNPs only act as vehicles for the delivery of ions that cause adverse effects in the respiratory chain and protein synthesis, as well as DNA alterations (Chen et al., 2011; Li et al., 2014).

The biogenic synthesis of silver nanoparticles (**Figure 2**) has received increasing attention in recent years. These nanoparticles present positive characteristics in terms of their improved stability and dispersion, due to the coating formed during the synthesis. In addition, there may be a positive effect of synergy between the nanoparticles and the compounds originating from the organism used. Biogenic synthesis is considered simple, low cost, and suitable for large-scale nanoparticle production (Lima et al., 2012; Kasithevar et al., 2017).

Biogenic nanoparticles have been found to present lower toxicity, while providing effective bactericidal activity against both Gram (−) and Gram (+) bacteria (Durán et al., 2016b; Kasithevar et al., 2017). These nanoparticles have also shown potential for use in the control of fungi (Balashanmugam and Kalaichelvan, 2015; Ahmad et al., 2016; Guilger et al., 2017).

### Nanocages

Nanocages are hollow and porous nanometric structures that may be used for the transport and delivery of antibiotics. They can be synthesized from various substances, including metals, proteins, and polymers, and have been investigated in terms of their potential for combating multidrug-resistant bacteria. Reported advantages of these structures are that they provide greater adhesion, retention at the site of infection, increased systemic circulation, and good biocompatibility (Wang et al., 2016; Mekeer et al., 2018).

Wang et al. (2018) synthesized gold nanocages using membrane coating of macrophages pretreated with *S. aureus*. Clinical treatments performed with local or systemic injection showed that the system provided increased bactericidal effectiveness. Ruozi et al. (2017) synthesized apoferritin-based nanocages, which were used for the encapsulation of streptomycin. The system showed promise for the delivery of antimicrobials, although further characterization, biocompatibility, and efficacy studies were still required. A study by Wu et al. (2019), using silica, silver, and gold nanospheres, showed that the Au–Ag@SiO2 nanocage had broadspectrum bactericidal properties. The nanocage could be used for antibiotic transport, as well as for infrared-induced hyperthermia therapy against bacterial infection.

### BACTERIOPHAGES

Bacteriophages (or phages, for short), which are viruses that only infect bacterial cells, are among the most ubiquitous biological entities, with a total estimated abundance of at least 1,030 types (Chibani-Chennoufi et al., 2004). Despite being known for more than 100 years, only now is renewed interest in phages driving studies of them as potential alternatives or complements to current antibiotics, due to their unique affinities and ability to kill bacteria resistant to antibiotics (Hagens and Loessner, 2010; Hyman and Abedon, 2010; Summers, 2012). The interaction between phage particles and bacteria generally involves specific receptors located in the outer membranes of bacteria. Despite the great potential of phages for treating and/or controlling infections caused by antibiotic-resistant bacteria, only a few clinical trials have been performed in humans and are accepted by public health authorities such as the FDA and the European Medicines Agency (EMA) (Rios et al., 2016).

Phages are ubiquitous in the biosphere and are highly specific to particular bacteria species, acting as natural predators of bacteria. They exhibit high tissue permeability and do not affect the beneficial intestinal microflora (so they do not promote secondary infections). Their exponential growth results in their accumulation in extremely high concentrations where they are needed the most, as long as the bacterial host still exists (Hagens and Loessner, 2010; Wittebole et al., 2013; Rios et al., 2016; Harada et al., 2018). However, phagebased therapy requires that the bacterium responsible for the infection is firstly isolated, before the identification and isolation of a specific and strictly lytic phage can be achieved. In addition, due to their protein nature, plain phage particles may be recognized by the immune system, resulting in a drastic reduction of their therapeutic efficacy (Chan and Abedon, 2012; Wittebole et al., 2013).

Bacterial resistance to phage particles generally occurs due to non-adsorption, membrane coating due to mucilage production by bacteria, and destruction of viral genetic material by restriction endonucleases (Wittebole et al., 2013).

Following oral or intravenous administration, phage particles may affect the major body systems, namely, the cardiovascular, digestive, immune, and nervous systems (Moutinho et al., 2012). Furthermore, due to their protein nature, phage particles are prone to denaturation by conformational changes that may be either reversible or irreversible, or to destruction by the immune system. The solution lies in protecting them, either by encapsulation within nanocarriers (Rios et al., 2018) that are invisible towards the digestive and immune systems, or by binding them to a macroscopic support so that they become insoluble (Balcão et al., 2013; Balcão et al., 2014). The combination of these strategies can provide phages with structural and functional stabilization (Balcão and Vila, 2015), enabling them to be potentially used for the eradication of antibiotic-resistant bacteria.

Several studies have described phage-based CRISPR-driven techniques for the prevention of bacterial drug resistance (Barrangou, 2015; Bikard and Barrangou, 2017; Doss et al., 2017;

Hatoum-Aslan, 2018; Pursey et al., 2018). In this approach, bacteriophages are designed to carry and deliver CRISPR–Cas in bacteria, in order to combat multidrug-resistant bacteria. Such systems are being developed by biotechnology companies such as Locus Biosciences (Morrisville, NC, USA) and Eligo Bioscience (Paris, France) (Reardon, 2017).

Recent biotechnological advances therefore open the door to the possibility of tailoring bacteriophage particles to improve their characteristics, including i) enhancing the ability of phages to penetrate bacterial biofilms; ii) increasing phage efficacy; iii) broadening the spectrum of phage lytic activities to infections caused by different bacteria; and iv) making phages more stable and specific (Maura and Debarbieux, 2011; Rios et al., 2016; Harada et al., 2018).

At the present time, due to the increase in bacterial resistance to antibiotics, together with the likely ineffectiveness of antibiotics within a few years, there is an urgent need to develop new antimicrobial strategies. This is a new era, in which the emergence of new solutions and discoveries will be crucial.

### FUTURE TRENDS AND POSSIBLE SOLUTIONS

The use of new technologies to combat multidrug-resistant bacteria is ever more necessary, because although there are still effective antibiotics, resistance to them is constantly increasing. The strategies discussed in this paper may provide new ways of fighting multidrugresistant bacteria. This could include associations between different strategies, as well as their use in combination with antibiotics, in order to combat this critical emerging problem (**Figure 3**).

The use of CRISPRs, a relatively new technology, may be one of the available solutions. Coupled with nanotechnological delivery methods, this technique could be sufficiently specific and provide the activity required to combat multidrug-resistant bacteria. For this, nanocapsules could be synthesized that are able to reach specific targets, which would facilitate the delivery of CRISPRs.

Biogenic metal nanoparticles, such as silver nanoparticles, may be an option in conjugated treatments to combat MDR bacteria. These nanoparticles offer the benefits of synergy between the effects of the metal and the metabolites of the organism used for their production. They present low toxicity and can act to disrupt existing mechanisms of resistance in bacteria.

Bacteriophages can be used successfully to fight multidrugresistant bacteria, but although it is not difficult to find the correct virus for each specific bacterium host, the task is nevertheless not straightforward. Consequently, the use of bacteriophage particles as carriers for CRISPRs seems to be a faster and more efficient solution, although such delivery may not always be guaranteed. Recent studies show that CRISPR technology can assist in the modification of bacteriophages, making them more specific for the intended purpose.

To conclude, a deeper understanding of these new and innovative therapeutic strategies is of utmost importance. Until such new strategies have been mastered, structured, and made commercially available, it is imperative to control the use of the currently available chemical antibiotics. It is also essential that

health professionals use wisely, and only as a last resort, new antibiotics that may become available in the near future, in order to prevent the emergence and spread of bacterial resistance to them.

### AUTHOR CONTRIBUTIONS

All authors participated in writing the manuscript, specifically RL with the themes nanotechnology and CRISPRs, FF with multidrug resistance, and VB with bacteriophage technology.

### FUNDING

The funding for this work was provided by the São Paulo State Research Foundation [FAPESP, grants #2016/08884-3

### REFERENCES


(PneumoPhageColor project), #2016/12234-4 (TransAppIL project), #2018/05522-9 (PsaPhageKill project, BPE fellowship granted to VB), and #2017/13328-5 (Biogenic Metal Nanoparticles project)]. Support was provided by the National Council for Scientific and Technological Development (CNPq), in the form of Research Productivity (PQ) fellowships awarded to VB (grants #306113/2014-7 and #308208/2017-0) and RL (grant #303967/2015-3). Funding support was also provided by CESAM (UID/AMB/50017/2019) and FCT/MCTES.

### ACKNOWLEDGMENTS

The authors are grateful to Universidade of Sorocaba/UNISO for supporting the publication charges.

for bacterial biosensing: bacterium diffusion studies. *Appl*. *Biochem. Biotechnol.* 172, 1194–1214. doi: 10.1007/s12010-013-0579-2


diseases society of America. Clinical infectious diseases: an official publication of the Infectious. *Dis. Soc. Am.* 48 (1), 1–12. doi: 10.1086/595011


nanoparticle-induced. *Biomaterials* 192, 551–559. doi: 10.1016/j.biomaterials. 2018.11.028


microorganisms: the role of efflux inhibitors. *Expert Opin. Ther. Targets* 21 (1), 23–36. doi: 10.1080/14728222.2017.1265105


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Lima, Del Fiol and Balcão. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Development of a Complex Intervention to Improve Adherence to Antidiabetic Medication in Older People Using an Anthropomorphic Virtual Assistant Software

*Isa Brito Félix1, Mara Pereira Guerreiro1,2\*, Afonso Cavaco3, Ana Paula Cláudio4, Anabela Mendes1, João Balsa4, Maria Beatriz Carmo4, Nuno Pimenta5 and Adriana Henriques1*

#### *Edited by:*

*Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Kurt Neumann, Independent Researcher, Hungary Sabrina Anne Jacob, University of Strathclyde, United Kingdom*

*\*Correspondence:*

*Mara Pereira Guerreiro mara.guerreiro@esel.pt*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 01 February 2019 Accepted: 24 May 2019 Published: 21 June 2019*

#### *Citation:*

*Félix IB, Guerreiro MP, Cavaco A, Cláudio AP, Mendes A, Balsa J, Carmo MB, Pimenta N and Henriques A (2019) Development of a Complex Intervention to Improve Adherence to Antidiabetic Medication in Older People Using an Anthropomorphic Virtual Assistant Software. Front. Pharmacol. 10:680. doi: 10.3389/fphar.2019.00680*

*de Investigação Interdisciplinar Egas Moniz (CiiEM), Instituto Universitário Egas Moniz, Monte de Caparica, Portugal, 3 iMed. ULisboa, Faculty of Pharmacy, University of Lisbon, Lisbon, Portugal, 4 Biosystems & Integrative Sciences Institute (BioISI), Faculty of Sciences, University of Lisbon, Lisbon, Portugal, 5 Sport Sciences School of Rio Maior, Polytechnic Institute of Santarém, Santarém, Portugal*

*1 Unidade de Investigação e Desenvolvimento em Enfermagem (UI&DE), Lisbon Nursing School, Lisbon, Portugal, 2 Centro* 

Introduction: Improving adherence to antidiabetic medication is crucial, resulting in improved health outcomes, cost reduction, and minimization of waste. A lack of underlying theory in existing interventions may explain the limited success in sustaining behavior change. This paper describes the development of a theory and evidence-based complex intervention to improve adherence to oral antidiabetics in older people *via* a software prototype with an anthropomorphic virtual assistant.

Methods: The Behavior Change Wheel (BCW) was used to develop a theoretical understanding of the change process, corresponding to the first phase of the Medical Research Council Framework for developing and evaluating complex interventions. At the BCW core is a model of human behavior (COM-B), which posits that human behavior (B) results from the interaction between capabilities (C), opportunities (O), and motivation (M). Literature-derived medication adherence determinants were mapped onto COM-B components. Then, intervention functions (IFs) were selected employing the APEASE criteria. Finally, standardized behavior change techniques (BCTs) were chosen based on their suitability and their effectiveness on medication adherence trials. The prototype was developed for android devices; its core was implemented in Unity3D, using a female 3D virtual assistant, named Vitória.

Results: Two COM-B components were identified as main targets for behavior change—psychological capability and reflective motivation; these were linked with four IFs—education, persuasion, enablement, and environmental restructuring. Eleven BCTs were, in turn, linked with the IFs. An example of a BCT is "problem solving"; it requires users to pinpoint factors influencing non-adherence and subsequently offers strategies to achieve the desired behavior. BCTs were operationalized into the dialogues with Vitória and into supplementary software features. Vitória communicates with users verbally and

**86**

non-verbally, expressing emotions. Input options consist of buttons or recording values, such as medication taken.

Conclusion: The present approach enabled us to derive the most appropriate BCTs for our intervention. The use of an explicit bundle of BCTs, often overlooked in interventions promoting medication adherence, is expected to maximize effectiveness and facilitates replication. The first prototype is being refined with users and health professionals' contributions. Future work includes subjecting the prototype to usability tests and a feasibility trial.

Keywords: relational agent, older adult, type 2 diabetes, medication adherence, virtual assistant, behavior change wheel, intervention development, complex intervention

### INTRODUCTION

Globally, diabetes affects more than 425 million people, of which one-third is older than 65 years (International Diabetes Federation, 2017). Type 2 diabetes (T2D) is the most prevalent form around the world, affecting 90% of the diabetic adults (Zheng et al., 2018). Glycemic control, which is key to prevent complications, requires sustained adherence to an appropriate diet, adequate physical activity, and, frequently, antidiabetic medication.

A systematic review including 27 studies found that adherence to antidiabetic medication can be as low as 38.5% (Krass et al., 2015). A recent study reported an overall antidiabetics adherence in older people of 72.4% (Juste et al., 2018). Additionally, Spanish elderly patients receiving oral antidiabetics showed rates of discontinuation for oral antidiabetics of 46.8% (Menditto et al., 2018).

Poor medication adherence in T2D is associated with negative outcomes, including increased morbidity and mortality, increased costs of ambulatory and hospital care, as well as loss of quality of life (Iuga and McGuire, 2014).

A wide array of factors associated with non-adherence has been described, which may differ according to study designs. For example, key nonpatient factors (e.g., healthcare systems limitations), non-modifiable patient factors (e.g., young age, low income level), and modifiable patient factors (e.g., beliefs about perceived treatment efficacy and medication burden) may be associated with non-adherence (Polonsky and Henry, 2016). Capoccia et al. (2016) mentioned lower adherence when medications were not tolerated or were taken more than twice daily.

Strategies to overcome poor medication adherence should consider the reduction of the medication burden, not only from a drug perspective (e.g., using fixed combinations) but also tools for an easier therapy management, plus responding to negative medication beliefs (Polonsky and Henry, 2016; Capoccia et al., 2016). Modifiable psychosocial factors belong to the overarching domains of knowledge, beliefs (and related cognitive constructs), behavioral skills, and coping (Gonzalez et al., 2016); thus, interventions targeting these aspects are expected to improve antidiabetic medication adherence.

Until recently, very few interventions to improve medication adherence were technology-based. Williams et al. (2014)

reported a systematic review on effective interventions to improve medication adherence in T2D, in which the literature was searched from 2000 to 2013. The authors only found five studies using mobile phones or the Internet.

The advantages of technology-based interventions include automation and convenience of access. While simple electronic reminders, such as SMS, have shown improvement in medication adherence in chronic diseases (Vervloet et al., 2012), they are not without caveats. These reminders target essentially unintentional non-adherence and their long-term effects are uncertain (Vervloet et al., 2012). Technological development brought supplementary approaches, such as web-based interfaces, mobile applications, or smartwatches.

Currently, one of the most used technologies is referred to as mHealth apps (mobile health applications, typically for iOS or android devices). The rapid growth of these apps occurs in the context of aging population. The mHealth advances align well with the growing interest of older adults to integrate technology in the self-management of chronic conditions (Gilbert et al., 2015; Kim and Lee, 2017).

Concerning medication adherence, Anglada-Martinez et al. (2015) concluded that 65% of the mHealth application trials reported positive results. Virtual humans, designed to build long-term socioemotional relationships with users, can be incorporated in mHealth applications. Such virtual humans are commonly referred to as relational agents, although the term is often used interchangeably with virtual agent or assistant. There is a considerable body of knowledge on the effect or acceptability of relational agents in different populations. The acceptability to older people, including those with low literacy, has been studied (Bickmore et al., 2010). These authors found that individuals with inadequate literacy generally report higher levels of satisfaction with the relational agent and ask more questions than those with adequate health literacy. This has implications for our project, as the health literacy of older people in Portugal is limited (Mota-Pinto et al., 2010; Paiva et al., 2017).

Additionally, research suggests that a relational agent software is effective in promoting medication adherence and accepted by people with schizophrenia (Bickmore et al., 2010). Recently, the effect of an embodied conversional agent on antiretroviral therapy adherence was researched, using a 3-month pre-post design (Dworkin et al., 2019). This study demonstrated a positive effect on pill count, which improved from 62% at baseline to 88% at follow-up. The use of these agents to support diabetes self-care, including medication adherence, seems to have received less attention. Exceptions are papers describing coaching interventions for diabetes patients, which focus on medication adherence (op den Akker et al., 2011; Monkaresi et al., 2013). However, these papers lack data on usability or the effect on endpoints of interest.

The present paper aims to describe the development of a theory and evidence-based complex intervention to improve adherence to oral antidiabetics in older people *via* a software prototype with an anthropomorphic virtual assistant (VASelfCare). This is part of a wider intervention, targeting also lifestyle change of elderly T2D patients.

### MATERIALS AND METHODS

Complex interventions include several interacting components that impact on the length and complexity of the causal chain and are influenced by features of the local context (Craig et al., 2008). The components usually include behaviors, characteristics of behavior (e.g., timing) and methods of organizing, and delivering those behaviors (e.g., setting and location). Many healthcare interventions are considered complex. The Medical Research Council (MRC) framework for developing and evaluating complex interventions has been widely employed (Bleijenberg et al., 2018); it provides guidance on the methodological and practical decision-making on healthcare interventions. The rationale for resorting to this framework is that interventions carefully designed and tested in an early stage are more likely to prevent negative or inconclusive trials (Craig et al., 2008).

**Figure 1** depicts the MRC framework phases. Phase 1 encompasses three elements (Craig et al., 2008), two of which are focused in the present paper: "identifying the existing evidence" and "identifying or developing theory." Identifying the existing evidence included a mapping review on the effectiveness of relational agents, on the design of these interventions and on successful medication adherence interventions. We followed guidelines for self-management of people with T2D, focusing on key dimensions of lifestyle (diet and physical activity) plus medication adherence (Powers et al., 2015). It has been suggested that interventions targeting behaviors in a stepwise approach are more effective in promoting change than targeting multibehaviors simultaneously (Sweet and Fortier, 2010).

The mapping review informed the intervention components. One of the intervention components is the nurse interaction with older people upon referral from the software prototype, in predefined circumstances. Moreover, the experience of Bleijenberg and colleagues (2018) with the MRC framework indicates that it is important to consider the elements of context, e.g., how the receiver (older people with T2D) and the provider (nurse) interact within the intervention. In Portugal, diabetes nursing consultations in primary care have been formally established as supplementary to medical consultations.

Another key task in phase 1 (**Figure 1**) is to develop a theoretical understanding of the underlying processes of change. The section Deriving the Active Ingredients of the Intervention describes the use of theory for deriving the active ingredients of the intervention. Then the section Development of the VASelfCare Software Prototype provides a brief description of how the software prototype was developed.

### Deriving the Active Ingredients of the Intervention

We chose the Behavior Change Wheel (BCW) (Michie et al., 2014) as the theoretical framework underpinning the intervention. The BCW is a recent approach to design research of behavior change interventions through a systematic and structured process (Michie et al., 2011; Cane et al., 2012). This process enables the evidencebased selection of intervention components, ensuring that the intervention targets the underlying determinants of behavior.

At the BCW core is a model of human behavior—the COM-B—which posits that human behavior (B) results from the interaction between physical and psychological capabilities (C), opportunities provided by the physical and social environment (O) and reflective and automatic motivation (M) (Michie et al., 2014). The COM-B is supported by the Theoretical Domains Framework (TDF), which describes 14 factors derived from 33 theories of behavior change. These factors fall under the categories of capability, opportunity, and motivation (Cane et al., 2012).

In the next sections, we will describe the stepwise approach to derive behavior change techniques (BCTs) for improving medication adherence based on the BCW.

#### Stage 1: Understanding the Behavior

We tackled this stage by reviewing the evidence on adherence and its determinants, which includes both barriers and facilitators to medication use. Several recent systematic literature reviews on determinants of adherence have been published. In particular, we draw on the seminal WHO on adherence determinants (Sabaté, 2003) and on systematic reviews and literature with the same scope to understand influences to medication use (Gellad et al., 2011; Kardas et al., 2013; Zeber et al., 2013; Capoccia et al., 2016; Yap et al., 2016; Choi and Smaldone, 2018). Subsequently, we mapped these determinants into the COM-B and identified key domains more influential to behavior change, through a multidisciplinary consensus discussion within the research team. This team has expertise in nursing, medication adherence, clinical pharmacology, and clinical communication.

#### Stage 2: Identifying Intervention Functions

Intervention functions (IFs) are defined as "broad categories of means by which an intervention can change behavior" (Michie et al., 2014, 192). Based on a systematic review of frameworks of behavior change interventions, Michie et al. (2011) put forward nine IFs. The BCW links COM-B domains to IFs and subsequent policy categories most likely to achieve behavioral change (Michie et al., 2011, 2014). Policy categories represent "types of decisions made by authorities that help to support and enact the intervention" (Michie et al., 2014, 235). We did not address these categories as they fell outside the scope of our intervention.

The IFs and their relation to the COM-B domains were organized in a mapping matrix (Michie et al., 2014). The former were selected based on their links to COM-B domains, as prescribed by the authors (Michie et al., 2014). The same IF can be linked to different COM-B domains. IFs most likely to affect behavior change were selected in a multidisciplinary consensus discussion within the research team, using the APEASE criteria: 1) affordability, 2) practicality, 3) effectiveness and costeffectiveness, 4) acceptability, 5) side-effects/safety, and 6) equity (Michie et al., 2014).

#### Stage 3: Identifying Behavior Change Techniques

A BCT is an observable, replicable, and irreducible component of an intervention designed to change behavior (Michie et al., 2013). BCTs are known as the active ingredients of the intervention. They have been organized in a taxonomy, designated in brief by

BCTTv1 (Michie et al., 2013; Cane et al., 2015), which provides a definition for each of the 93 techniques, enabling a consistent application.

There is guidance on the BCTs better suited to certain IFs and their underlying theoretical domains (Michie et al., 2008; Cane et al., 2015). From the list of 93 BCTs, the most appropriate for our intervention were selected based on:


### Development of the VASelfCare Software Prototype

We decided that the application would run in tablets with android system without requiring Internet connection. This decision was grounded on the need to secure inclusive access, considering that presently a significant percentage of older adults' homes do not have Internet available (INE, 2014). Such decision, in turn, influenced software development options.

The overall development of the application was guided by usability principles for older people with T2D (Arnhold et al., 2014). For example, redundancy of both audio and written information may help reduce any communication shortcomings, such as lower eyesight accuracy and hearing deficits.

The core of VASelfCare solution was implemented in Unity3D (https://unity3d.com/pt), a software for the development of video games.

The virtual assistant chosen for this first prototype was a female 3D model with a realistic look, obtained from Daz3D (https://www.daz3d.com/). The choice for a 3D character is supported by studies that concluded that realistic-looking virtual agents are more appropriate for medical tasks (Ring et al., 2014; van Wissen et al., 2016). Other applications have also employed female figures as relational agents, for example (Bickmore et al., 2005a; Bickmore et al., 2005b).

Body animations and facial expressions are activated by the software that controls the virtual assistant behavior and verifies the context of the on-going interaction. Conveying emotions to users, beyond messages content, intends to establish an affective and effective communication. Furthermore, this has been acknowledged as the cornerstone for engagement and long-term use (Bickmore et al., 2010).

To support the virtual assistant speech, first dialogues had to be created, then the corresponding sound is generated and mouth animations that simulate the articulation of the sound were provided.

Dialogues were iteratively created by team members with a nursing and pharmacy background. Quality of the output was ensured by agreeing on principles at the outset of the process, by discussing dialogue structured regularly and through independent double-checking of scripts. To streamline this creative process, we adopted Yarn (https://github.com/InfiniteAmmoInc/ Yarn), a graphical dialogue editor simple to use and compatible with the other software tools that were being used. Yarn provides an interface where the speech lines of the virtual assistant and the options that the user may choose (i.e., buttons in the interface) are inserted in text boxes. These are sequentially connected in a way that expresses the possible progressions of the dialogue during the interaction between the user and the virtual assistant.

General principles were deployed to write a comprehensive and optimal dialogue for each individual user (Migneault et al., 2006). One principle was tailoring content, which has been defined as "any combination of information and behavior change strategies intended to reach one specific person based on characteristics that are unique to that person related to the outcome of interest and derived from an individual assessment" (Kreuter et al., 2000). A meta-analysis showed that tailored webbased interventions have a significantly greater effect in health behavior outcomes than non-tailored counterparts (Lustria et al., 2013). Another principle was personalization, which broadly involves giving an indication that the software "knows who is talking to" (Migneault et al., 2006). A non-judgmental approach was employed to create the dialogues, resorting to a helpfulcooperative communication style (Niess and Diefenbach, 2016). Such an approach aims at creating rapport and a trust-bond, so that effectiveness of BCTs may be enhanced, along with other psychological benefits, as reported in the literature on virtual assistant-based interventions (Velicer et al., 2015).

A rule-based component was implemented to control the dialogue flow of the interaction. This component corresponds to the definition of a set of if–then rules (rules of the form "if *some conditions hold* then *execute some action"*), where the *conditions* may include context information regarding the interaction (e.g., user characteristics, such as age, or the date when interaction takes place) and the *action* represents the subsequent locutionary act performed by the virtual assistant. This approach allows greater flexibility in the dialogue definition and in the generation of diversified interactions.

Speech2Go from Harpo (https://harposoftware.com) was used to generate the audio files. This is a text-to-speech (TTS) software, which converts the written dialogues to audio speech. Since the virtual assistant is currently represented by a female character, the female voice of Inês, a European Portuguese voice, has been used. The speech rate has been slowed down, taking into consideration the target population.

Visemes describe the animations of the virtual assistant's mouth that simulate phonemes' articulation. Conversion of written dialogues to the viseme files is performed by a software developed in a previous project (Cláudio et al., 2015).

SQLite (https://www.sqlite.org/index.html) was the database engine chosen to control all the information stored locally in each tablet as it encompasses a secure storage mechanism. The local database, initially empty, will store clinical data registered by the care nurse, such as the prescribed antidiabetic medication, weight, and HbA1c. It will then be filled with data inserted during user interactions, including clinical data such as blood glucose levels, medication taken, and number of daily steps. These data are deployed in future interactions with the user and enable

analysis at a later stage, contributing to the further development of the software prototype. Periodically, the database in each tablet is copied to an institutional computer using a secure connection and the access to this repository is reserved for credentialed personal. Privacy is further ensured by restricting access to this repository only to the researcher responsible for data analysis; data will be extracted omitting any reference that could identify the user.

### RESULTS

### Deriving the Active Ingredients of the Intervention

### Stage 1: Understanding the Behavior

The target behavior of this intervention is medication adherence, defined briefly as the behavioral response to an agreed medical recommendation (Sabaté, 2003). Recently, medication adherence has been defined as a process consisting of three stages: initiation, implementation, and discontinuation (Vrijens et al., 2012). The target behavior for this intervention was further defined as taking between 80% and 100% of the prescribed antidiabetic doses. The 80% cut-off point is underpinned by previous research that stratified adherent and non-adherent individuals based on predicting subsequent hospital admission across prevalent chronic conditions, including diabetes (Karve et al., 2009). The clinical implications of non-adherence to antidiabetic agents have been discussed earlier in the Introduction section.

To be effective the software prototype should target modifiable determinants of adherence (Allemann et al., 2016). Therefore, determinants with a negative effect on adherence, amenable of change by the prototype or other intervention components, were extracted from the literature (Gellad et al., 2011; Kardas et al., 2013; Zeber et al., 2013; Capoccia et al., 2016; Yap et al., 2016; Choi and Smaldone, 2018). For example, medication cost is not directly modifiable by the software prototype, but can be addressed *via* referral to the primary care nurse. The same applies to suspected adverse drug reactions, which the prototype cannot manage on its own. **Table 1** depicts the determinants influential on behavior change mapped into key domains of the COM-B.

### Stage 2: Identifying Intervention Functions

**Table 2** presents the IFs selected and the reasons for their inclusion or exclusion. Overall, two IFs—"modeling" and "training"—were excluded because they were deemed impracticable to be applied *via* the software.

TABLE 1 | Adherence determinants mapped into the COM-B model.



#### Stage 3: Identifying Behavior Change Techniques

Eleven BCTs were selected to be included in the software prototype (**Table 3**).

The selected BCTs can serve more than one IF. For example, "feedback on behavior" could be linked to the "education" and "persuasion" functions. This BCT was operationalized by providing information on medication-taking behavior *via* a chart (**Figure 2**), described by the virtual assistant, plus giving supportive messages about the user's abilities to perform the desired behavior. **Figures 3**, **4**, and **5** depict software interfaces representing "biofeedback," "self-monitoring of behavior," and "problem-solving," respectively. While the software prototype is in Portuguese, we present figures in English to facilitate readers' understanding.

### Development of the Medication Adherence Component of the VASelfCare Prototype

Prior to the user's first interaction with the prototype, registration by a nurse providing care in diabetes consultations is required. There is a specific interface for this purpose that includes mandatory clinical information, such as the prescribed antidiabetic medication and the last HbA1c level.

When a user interacts with the prototype for the first time, a number of features are available. A key feature, described in the next section, is the dialogue view, in which the user interacts with Vitória, an anthropomorphic virtual assistant. Supplementary features are offered in other views. The prototype is designed to allow no more than one full interaction with Vitória per day. Access to the other views is unrestricted.

#### Interaction With the Dialogue View

Vitória is capable of speaking, by means of a synthetic voice, and features also text subtitles, reproducing the verbal content. Furthermore, Vitória expresses herself non-verbally with users, providing emotions through facial and body animations, depending on the user's response. The user communicates with Vitória using buttons or by recording values, such as medication taken.

To attain a more convincing experience, the 3D scenario where Vitória is displayed changes, for example, according to the time of the day and season of the year.

Initially, the user is prompted to select a component to start with (medication adherence or lifestyle changes—diet or physical activity). Assuming the medication component is selected, the first 3 days are devoted to evaluation. The main purpose of this phase is to collect data on user's knowledge about antidiabetic agents, current behavior on medication-taking and perceived self-efficacy in medication management, for future tailoring of the intervention.

The evaluation phase was structured as a six-step interaction: opening; social talk; assessment; feedback; pre-closing; and closing. This structure has been adapted from the literature (Bickmore et al., 2005b). The "opening" step consists of greeting the user and it is followed by the "social talk," which includes inquiries about the user's general emotional and physical state (e.g., "how are you feeling today?"). "Assessment" is the step in which questions about the aforementioned constructs are posed. In the "feedback" step, brief information on the answers is provided. Finally, the contents of the next interaction are described ("pre-closing") and a farewell is delivered ("closing").

#### TABLE 3 | BCTs selected for the VASelfCare software prototype (antidiabetics adherence).


# Environmental restructuring

The second phase, designated by follow-up, has a main purpose of promoting the desired behavior or to maintaining it. The dialogues in this phase are also structured in repeated sequential steps, described in the literature (Bickmore et al., 2005b). The first two (opening and social talk) and the last two steps (pre-closing and closing) correspond to those described in the evaluation phase. In the "review tasks" step, information is collected about previously agreed tasks or behaviors. Then, feedback is provided and behavior determinants are discussed where pertinent ("assess"). The "counseling" step has a twofold end. Firstly, it offers strategies to overcome previously identified adherence barriers, if applicable. Secondly, it provides tailored educational contents to users. This is exemplified by information about adverse drug reactions, according to knowledge gaps identified in the evaluation phase. In the "assign tasks" step, the user negotiate new behavioral goals and tasks with Vitória.

The selected BCTs are incorporated into different steps of the dialogues, as shown in **Table 4**.

Tailoring was applied not only at the information level, but also in what concerns specific BCTs. For instance, social support

(**Table 4**) is only employed when non-adherence determinants cannot be addressed by the software prototype.

### DISCUSSION

This article illustrates a systematic, evidence-driven, and theorybased approach to specify the content and active ingredients of a software prototype, devised to improve medication adherence in older adults with T2D.

Older age has been related to better medication adherence; for example, a systematic review identified increasing age as a factor positively associated with adherence to diabetes medication (Krass et al., 2015). Likewise, a meta-analysis by Choi and Smaldone (2018) indicates that the elderly are more likely to take diabetes medication as prescribed, when compared with younger adults. While *per se* older age seems to affect positively adherence, it is widely recognized that non-adherence is multidimensional. Other factors such as multiple morbidities, regimen complexity, cognitive decline, dexterity, and medication cost can impair adherence (Gellad et al., 2011). This case is illustrated by recent Portuguese data on medication adherence in type 2 diabetic patients measured by the Morisky Medication Adherence Scale (MMAS-4), which categorizes adherence in good, median, and bad (Pinto et al., 2019). The 60 to 79 years group presented both a higher proportion of good and bad medication adherence in relation to lower age groups. The fact that adherence interventions in older people merit research efforts is illustrated by the plethora of trials addressing this population group (Marcum et al., 2017).

We departed from published literature on potential determinants of medication adherence to ensure that the intervention targets known modifiable factors. Four IFs associated with psychological capability and reflective motivation in the COM-B model were subsequently selected—education, persuasion, enablement, and environmental restructuring resorting to structured criteria (APEASE). Finally, 11 standardized BCTs were chosen from the BCTTv1, based on their suitability to the selected IFs and their effectiveness on medication adherence research trials. This developmental process was guided by the work of Cane and colleagues (2015) and the mapping work from Michie and collaborators (2008).

Morrissey et al. (2016) surveyed 166 smartphone medication adherence applications on the market to identify the BCTs present. From a range of 93 possible techniques (Michie et al., 2013), only 12 were found in the evaluated applications. In accordance with our work, the most common ones were "action planning" and "prompt/cues," followed by "self-monitoring" and "feedback on behavior" (Morrissey et al., 2016).

Interventions to improve medication adherence often use BCTs combinations. Future research could examine which particular BCTs or combinations of BCTs are most effective in changing the medication-taking behavior in older people *via* a relational agent software. A recent study based on a different BCTs taxonomy found that specific BCTs configurations were effective in promoting medication adherence in the context of non-digital interventions (Kahwati et al., 2016). Overall, there is paucity of evidence on this topic, in particular concerning BCTTv1, maybe related to its novelty.

From the software standpoint, we have obtained a first prototype of a multi-behavior intervention, designed in light of usability requirements of older adults. It features an anthropomorphic virtual assistant, designed as a relational agent, able to communicate verbally and non-verbally in an empathic fashion. The agent tailors information to users' level of knowledge, self-reported adherence, and perceived selfefficacy in managing medication. Moreover, it personalizes the

#### TABLE 4 | Description of the operationalization of each BCT.


interaction by, for example, mentioning the user's name and referring to a relative or friend. The prototype offers several features in addition to the dialogue with the virtual assistant, such as a calendar depicting antidiabetics-taking behavior and a chart with HbA1C levels.

The question of whether applications to promote medication adherence using anthropomorphic virtual agents are advantageous when compared to their simpler, text-only counterparts, remains essentially unanswered. The few studies published present limitations, such as relatively small and younger nonclinical populations, which raises generalizability issues for older populations with prevalent chronic diseases. For instance, one study compared an application for behavior change in healthy eating using a female virtual agent versus simple text messages in a sample of 60 higher education students (Mazzotta et al., 2009); information quality and persuasion strength were better rated for the virtual agent modality. By contrast, a study comparing a virtual coach for healthy lifestyle versus text messages in a sample of 43 office workers was unable to demonstrate the virtual agent superiority (op den Akker et al., 2016). Once our intervention is matured, it can get trialed, contributing to answer this question.

### Strengths and Limitations

One of the strong points of our work is the theory-driven approach we pursued to the development of the medication adherence intervention. Absence of a theoretical underpinning has been associated with limited effectiveness in these interventions in older people and other patient groups (Nieuwlaat et al., 2014; Patton et al., 2017). Resorting to theory enables a better understanding of the behavior change process (Patton et al., 2017) and tends to have a significant effect on outcomes (Webb et al., 2010).

Another strength is providing a detailed description of the chosen BCTs. Published work on the development of behavior change interventions does not always offer such a description (Wells et al., 2012; Glasziou et al., 2014), which is critical to tout reproducibility. Portraying the content used in an intervention is crucial for reporting, replication, and synthesizing evidence (Proctor et al., 2013; Hoffmann et al., 2014).

However, this work presents also limitations. One of the limitations is whether the behavior change strategies devised by us truly represents a particular BCT. For instance, we were of the understanding that monitoring and recording blood glucose levels did not represent "self-monitoring of outcomes of behavior (2.4)", as we did not provide a method for that purpose, indicating timing and frequency, as others did (Goyal et al., 2016). We attempted to overcome this uncertainty by in-depth discussion within the research team, aided by examples of BCTs in the scientific literature.

Another drawback is the fact that evidence on the effect of BCTs in medication adherence is limited. For instance, while a Cochrane review reported that education, counseling, daily treatment support, and additional support from family or peers were effective for improving adherence (Nieuwlaat et al., 2014), the interventions description may be insufficient to allow coding with a taxonomy. Furthermore, there are cases in which no standardized description of the active components is provided, nor referenced to a taxonomy. This is exemplified by a systematic review on effective interventions to improve medication adherence in T2D (Williams et al., 2014). Problemsolving emerged as effective in improving medication adherence and HbA1c levels (Williams et al., 2014), but it is arguable what it actually means. In other studies interventions addressed problemsolving through action plans and goal setting (Walker et al., 2011; Bogner et al., 2012; Sapkota et al., 2015); it is unclear whether these techniques are identical to their counterparts in BCTTv1. Describing interventions using a comprehensive taxonomy with agreed definitions could promote a better integration of research findings into practice and consequently to building a cumulative knowledge of intervention effectiveness (Presseau et al., 2015).

From an operational standpoint, one of the limitations of the current prototype is that medication adherence and tolerability is assessed only in respect to oral antidiabetics. Future efforts should be directed to the full medication regimen of users and not only blood glucose lowering medication.

The virtual assistant was designed to establish rapport and trust and to provide automated, long-term support to older adults with T2D, supplementing nursing and other health consultations. As previously mentioned, the virtual assistant is able to discuss nonadherence determinants and, according to the adherence barrier identified, provides strategies to overcome them. For example, beliefs about medicines may be tackled by providing information about antidiabetics medication and the disease. This represents a step ahead of many currently available applications. However, it is unlikely that the relational agent is able to manage all instances of non-adherence. The prototype is programed to refer to the primary care nurse or other healthcare professional if problems persist, potentially preventing sustained lack of adherence.

### Future Work

We have obtained a first prototype of a software application with an anthropomorphic virtual assistant, which has been used to elicit informed opinions on pre-requisites from nine older people with T2D and 19 health professionals' in five primary care units of the Portuguese national health service (ethical approval 6104/CES/2018/ARSLVT). Results were overall positive and highlighted opportunities for improvement. These were prioritized and actioned.

In parallel with the software development, we are conducting a cross-sectional study in three primary care units to characterize the study population in respect to variables of interest, such as functional and cognitive status, HbA1c levels, and medication adherence. These results will inform the feasibility trial.

The next step is subjecting the prototype to usability tests in a purposive sample of around 10 older patients with T2D, employing the Portuguese version of the System Usability Scale (SUS) (Martins et al., 2015). Usability will also be evaluated by up to five experts, chosen by their experience in aspects such as interface design and diabetes management. This iteration will contribute to further fine-tune the prototype.

Subsequently, the software prototype will be tested in a nonrandomized, non-controlled feasibility trial in participating primary care units. For this trial, we envisage to recruit about 20 end users enrolled in nursing consultations. Inclusion and exclusion criteria will be informed by the aforementioned crosssectional study. Acceptability to users will be researched by focus groups, conducted at the end of the feasibility trial. This represents the second phase of the MRC framework.

### CONCLUSIONS

The BCW enabled the identification of the most appropriate BCTs for a novel intervention to promote adherence to antidiabetic medication in older people *via* an anthropomorphic virtual assistant. This stepwise approach involved the identification of four IFs, linked to two key COM-B categories (psychological capability and reflective motivation). Subsequently, 11 BCTs were considered potentially effective. The process outlined here can be used by researchers to guide a comprehensive intervention development, maximizing effectiveness, and facilitating replication.

## AUTHOR CONTRIBUTIONS

IF, MG, and AH conceived the idea for the paper. IF guided the team through the behavior change wheel theoretical framework. All authors contributed to the first draft of the manuscript. IF and MG performed a first critical review, which was then commented by all authors. All authors approved the manuscript in its final version for submission and agreed to be accountable for the work presented.

### FUNDING

This project is supported by FCT and Compete 2020 (grant number LISBOA-01-0145-FEDER-024250, 02/SAICT/2016). It is also supported by UID/MULTI/04046/2019 Research Unit grant from FCT, Portugal (to BioISI).

### ACKNOWLEDGMENTS

The authors are indebted to other VASelfCare team members for their contribution to the software development (http://vaselfcare. rd.ciencias.ulisboa.pt/index-en.html).

### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Félix, Guerreiro, Cavaco, Cláudio, Mendes, Balsa, Carmo, Pimenta and Henriques. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Efficacy and Safety of Brinzolamide as Add-On to Prostaglandin Analogues or **β**-Blocker for Glaucoma and Ocular Hypertension: A Systematic Review and Meta-Analysis

*Yuanzhi Liu1, Junyi Zhao1,2, Xiaoyan Zhong1, Qiming Wei1 and Yilan Huang1\**

*1 Department of Pharmacy, The Affiliated Hospital of Southwest Medical University, Luzhou, China, 2 College of Pharmacy, Southwest Medical University, Luzhou, China*

#### *Edited by:*

*Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Pooja Dhupkar, University of Texas, United States Fathi M. Sherif, University of Tripoli, Libya*

*\*Corresponding author: Yilan Huang*

*hyl3160131@163.com*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 11 February 2019 Accepted: 24 May 2019 Published: 25 June 2019*

#### *Citation:*

*Liu Y, Zhao J, Zhong X, Wei Q and Huang Y (2019) Efficacy and Safety of Brinzolamide as Add-On to Prostaglandin Analogues or β-Blocker for Glaucoma and Ocular Hypertension: A Systematic Review and Meta-Analysis. Front. Pharmacol. 10:679. doi: 10.3389/fphar.2019.00679*

Background: Brinzolamide as a carbonic anhydrase inhibitor could be combined with other intraocular pressure (IOP) lowering drugs for glaucoma and ocular hypertension (OHT), but the efficacy was controversial. So, this study was used to assess the efficacy and safety of brinzolamide as add-on to prostaglandin analogues (PGAs) or β-blocker in treating patients with glaucoma or OHT who fail to adequately control IOP.

Methods: We searched PubMed, Embase, MEDLINE, Cochrane Library, and clinicaltrials. gov from inception to October 4, 2018. Randomized controlled trials of brinzolamide as add-on to PGAs or β-blocker for glaucoma and OHT were included. Meta-analysis was conducted by RevMan 5.3 software.

Results: A total of 26 trials including 5,583 patients were analyzed. Brinzolamide produced absolute reductions of IOP as an adjunctive therapy for patients with glaucoma or OHT. Brinzolamide and timolol were not significantly different in lowering IOP as add-on to PGAs (9 am: *P =* 0.07; 12 am: *P =* 0.66; 4 pm: *P =* 0.66). Likewise, brinzolamide was as effective as dorzolamide in depressing IOP (9 am: *P =* 0.59; 12 am: *P =* 0.94; 4 pm: *P =* 0.95). For the mean diurnal IOP at the end of treatment duration, there were no statistical differences in above comparisons (*P* > 0.05). Compared with brimonidine (b.i.d.), there was a significant reduction of IOP in brinzolamide (b.i.d.) at 9 am (*P* < 0.0001); however, the difference was cloudy in thrice daily subgroup (*P =* 0.44); at 12 am, brinzolamide (b.i.d.) was similar to brimonidine (b.i.d.) in IOP-lowering effect (*P =*  0.23), whereas brimonidine (t.i.d.) led to a greater effect than brinzolamide (t.i.d.) (*P =*  0.02). At 4 pm, brinzolamide (b.i.d.) was superior IOP-lowering effect compared with brimonidine (b.i.d.) (*P =* 0.0003); conversely, the effect in brinzolamide (t.i.d.) was lower than brimonidine (t.i.d.) (*P* < 0.0001). For the mean diurnal IOP, brinzolamide was lower in twice daily subgroup (*P* < 0.00001); brimonidine was lower in thrice daily subgroup (*P* < 0.00001). With regard to the safety, brinzolamide and dorzolamide had a higher incidence of taste abnormality; moreover, brinzolamide resulted in more frequent blurred vision;

**99**

dorzolamide resulted in more frequent ocular discomfort and eye pain. Timolol resulted in more frequent blurred vision and less conjunctival hyperemia. Brimonidine resulted in more frequent ocular hyperemia. As to other adverse events (AEs) (conjunctivitis, eye pruritus, foreign body sensation in eyes, and treatment-related AEs), brinzolamide was similar to other three active comparators.

Conclusions: Brinzolamide, as add-on to PGAs or β-blocker, significantly decreased IOP of patients with refractory glaucoma or OHT and the AEs were tolerable.

Keywords: brinzolamide, prostaglandin analogues, **β**-blocker, glaucoma, ocular hypertension, systematic review

### INTRODUCTION

Glaucoma is an acquired disease of irreversible blindness and the second leading cause of blindness worldwide, characterized by optic neuropathies and intraocular pressure (IOP) elevation (Peters et al., 2014). Primary open-angle glaucoma (POAG), one of the most prevalent types, will have threatened 76.0 million people by 2020 and 111.8 million people by 2040 (Tham et al., 2014). There are no significant symptoms in the early stage of glaucoma, but once showing impaired vision, the patients have lost nearly 1 million of their retinal ganglion cell (RGCS) (Weinreb et al., 2014; Sharif, 2018). Therefore, the early diagnosis and treatment are particularly important for glaucoma.

Currently, pharmacotherapy is still a common and effective way to treat glaucoma and ocular hypertension (OHT). There are a variety of IOP-lowering agents containing carbonic anhydrase inhibitors (CAIs), beta-blockers, α2-adrenergic agonists, and prostaglandin analogues (PGAs). PGAs are the first-line treatment option, while their monotherapies may offer insufficient IOP control, so they need to be combined with other therapies, such as latanoprost and travoprost, which are combined with brinzolamide or dorzolamide or brimonidine or timolol for patients failing to control IOP (Cheng et al., 2009; Dzhumataeva, 2016; Lusthaus and Goldberg, 2017). Timolol, one of the β-blockers, has an obvious effect on diurnal IOP, but it is also insufficient to hold a stable IOP over the long term (Konstas et al., 2016). Brimonidine, an α2-adrenergic agonist, is popularized due to the positive effect of AQH and neuroprotective actions (Lusthaus and Goldberg, 2017). Brinzolamide and dorzolamide could inhibit carbonic anhydrase in ciliary epithelium to reduce IOP, increase retinal blood flow, and the efficacy of brinzolamide would be enhanced after improving the drug-delivery system (Iester, 2008; Konstas et al., 2013; Dong et al., 2018; Wang et al., 2018). However, brinzolamide and dorzolamide are restricted by lacking efficacy and brimonidine has a higher AE. It is, therefore, essential to combine multiple agents.

According to the differences of mechanisms, brinzolamide could be used in combination with other IOP-lowering drugs for glaucoma and OHT. However, there were no relevant systematic reviews to compare the efficacy and safety between brinzolamide and other active drugs as add-on treatment. Thus, basing on published and unpublished randomized controlled trials (RCTs) of patients with glaucoma or OHT, we did a systematic review

to assess the efficacy and safety of brinzolamide compared with other anti-glaucoma agents as add-on treatment.

### METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Knobloch et al., 2011).

### Data Sources and Search Strategy

We systematically searched using databases including PubMed, Embase, MEDLINE, and Cochrane Library from inception to September 4, 2018, with a language restriction (English). The unpublished data were also searched from clinicaltrials.gov. We used the following terms: "brinzolamide," "CAS No. 138890- 62-7," "carbonic anhydrase inhibitors (CAI)," "glaucoma," and "ocular hypertension." These terms were adjusted to adhere to the relevant rules in each database.

Two independent reviewers screened titles and abstracts of all retrieved citations, and subsequently examined potentially eligible studies in full text. All discrepancies were resolved through discussion and added to the third reviewer when necessary.

### Study Selection and Data Extraction

We included RCTs if they met the following criteria: 1) patients aged > 18 years; 2) a clinical diagnosis of glaucoma (POAG, exfoliation glaucoma, pigmentary glaucoma) or OHT in at least one eye (study eye); 3) the patients without lowering IOP adequately by the monotherapies of antiglaucomatous drugs (PGA: IOP ≥ 18 mmHg; β-blocker: IOP ≥ 20 mmHg) or the patients with IOP ≥ 20 mmHg without medication (including washout schedule); 4) the patients using brinzolamide as a monotherapy or a combination therapy for safety analysis; 5) no history of glaucoma surgery before the study; 6) Snellen visual acuity ≥ 0.1 or Snellen score ≥ 20/100 in the study eye(s); 7) duration: follow-up time ≥ 4 weeks; and 8) outcome variables: a) IOP changes from baseline; b) the mean diurnal IOP at the end of treatment duration; c) AEs.

Exclusion criteria were as follows: 1) a history of chronic or recurrent severe ocular inflammatory disease; 2) ocular trauma or intraocular surgery within 6 months or laser eye surgery within 3 months of screening; 3) ocular infection, endophthalmitis, or retinal disease; 4) hypersensitivity to any of the excipients in the study medications; 5) maximum corrected visual acuity ≤ 0.2 (decimal acuity) or an anterior chamber angle grade < 2 in either eye; 6) quantify visual acuity < 0.6 logarithm of the minimal angle resolution; 7) optic nerve with a cup-disc ratio > 0.8; 8) previous or current evidence of a severe illness or any other condition that could make the patient unsuitable for the study; 9) treatment with stable doses of any medication within 30 days of the start of the study that could affect IOP; and 10) pregnant or lactating, or intending to become pregnant during the study period.

### Data Extraction and Risk of Bias Assessment

The data extraction was implemented by two independent reviewers (YL and QW) according to the inclusion criteria. The information extracted from the trials includes study characteristics, interventions, types of glaucoma, duration of treatment, background therapy, and efficacy outcomes and AEs.

The methodological quality of eligible studies was assessed using the Cochrane risk-of-bias tool (Higgins et al., 2011). The predefined key domains included: randomization, allocation concealment, blinding, intent-to-treat (ITT) analysis, and a description of losses to follow-up.

We chose doses of the study drugs including brinzolamide 1% b.i.d. or t.i.d., which were the most commonly used doses in clinical treatments. In addition, our studies included 23 articles published and 3 articles unpublished. All studies are assessed under the same criteria.

### Statistical Analysis

The statistical analysis was performed by 5.3 software and Stata 12 software. For the efficacy (IOP changes from baseline, the mean diurnal IOP at the end of treatment duration), we assessed them by the weighted mean difference (WMD) with 95% confidence intervals (CIs). For the safety, we assessed the incidences of AEs by risk ratios (RRs) with 95% CIs. Heterogeneity was evaluated with the chi-square test and the I2 statistic. We planned to explore heterogeneity with a sensitivity analysis when I2 was higher than 50% (Higgins et al., 2003). We also conducted egger analysis to assess the potential publication bias when three or more studies offered relevant data, and defined significant publication bias with the *P* value < 0.1.

### RESULTS

## Search Results and Study Characteristics

We identified 831 articles from four databases search through the search strategy, and 472 with duplicate were removed. After excluding reviews, meta-analysis, non-human studies, and nonclinical human studies, 109 were left. By further reviewing the full text, we included 26 articles with a total of 5,683 patients (**Figure 1**). The basic characteristics of the included studies were shown in **Supplementary Table 1**. All trials were randomized and active-controlled involving the study drugs added on PAG in 11 articles (Hollo et al., 2006; Reis et al., 2006; Feldman et al., 2007; Day and Hollander, 2008; Miura et al., 2008; Bournias and Lai, 2009; Nakamura et al., 2009; Pfeiffer, 2011; Konstas et al., 2013; Alcon, 2016; Aihara et al., 2017), added on timolol 0.5% in two articles (Michaud and Friren, 2001; Martinez and Sanchez-Salorio, 2009), and added on the combination therapy of latanoprost and a beta-blocker in one article (Tsukamoto et al., 2005). Main clinical diagnosis of patients were POAG and OHT; a few were other glaucoma (exfoliation glaucoma, pigmentary glaucoma). Duration of intervention ≥ 4 weeks.

## Bias Risk Analysis

**Supplementary Table 2** presented the bias risk analysis of the included RCTs. All studies were randomized, multicenter clinical trials; six trials (Silver, 1998; Miura et al., 2008; Bournias and Lai, 2009; Manni et al., 2009; Katz et al., 2013; Aung et al., 2014) described the sequence generation. Twelve studies (Silver, 1998; Sall, 2000; March and Ochsner, 2000; Hollo et al., 2006; Feldman et al., 2007; Day and Hollander, 2008; Kaback et al., 2008; Miura et al., 2008; Bournias and Lai, 2009; Pfeiffer, 2011; Katz et al., 2013; Aung et al., 2014) offer the details of concealment procedures (Martinez and Sanchez-Salorio, 2009; Research, 2013a; Research, 2013b; Alcon, 2016). Sixteen trials performed ITT analyses (Sall, 2000; March and Ochsner, 2000; Michaud and Friren, 2001; Hollo et al., 2006; Feldman et al., 2007; Kaback et al., 2008; Bournias and Lai, 2009; Martinez and Sanchez-Salorio, 2009; Manni et al., 2009; Pfeiffer, 2011; Research, 2013a; Research, 2013b; Katz et al., 2013; Nguyen et al., 2013; Whitson et al., 2013; Aung et al., 2014) and all studies described withdraws or dropouts. All studies were funded by the company.

### Efficacy Analysis Brinzolamide vs Timolol

The changes of IOP from baseline between brinzolamide and timolol were shown in **Figure 2**. Both drugs significantly decreased IOP as adjunctive therapies to PGAs. There were no statistically significant differences (9 am: WMD 0.50 mmHg, 95%CI [−0.04 to 1.04], *P =* 0.07, *I =* 37%; 12 am: WMD 0.25 mmHg, 95%CI [−0.70 to 1.19], *P =* 0.61, *I =* 60%; 4 pm: WMD 0.41 mmHg, 95%CI [−1.16 to 1.97], *P =* 0.66, *I =* 87%). For the high level of heterogeneity at 12 am, we removed one trial (Hollo et al., 2006) whose designs slightly differ from others, and the heterogeneity was eliminated without affecting the overall estimate (WMD 0.77 mmHg, 95%CI [−0.02 to 1.57], *P =* 0.06, *I =* 0%). At 4 pm, we did not use a sensitivity analysis due to only including two trials. Likewise, the mean diurnal IOPs at the end of treatment duration did not differ between brinzolamide and timolol (WMD 0.38 mmHg, 95%CI [−0.18 to 0.94], *P =* 0.18, *I =*  21%) (**Figure 3**). There was no publication bias on egger test (*P* ≥ 0.1; **Supplementary Table 3**).

### Brinzolamide vs Dorzolamide

The changes of IOP from baseline between brinzolamide and dorzolamide were shown in **Figure 4**. Both drugs significantly decreased IOP as adjunctive therapies to PGAs and/or betablocker, and the brinzolamide was as effective as dorzolamide

in depressing IOP (9 am: WMD −0.04 mmHg, 95%CI [−0.20 to 0.11], *P =* 0.59, *I =* 0%; 12 am: WMD −0.01 mmHg, 95%CI [−0.16 to 0.14], *P =* 0.94, *I =* 0%; 4 pm: WD 0 mmHg, 95%CI [−0.16 to 0.17], *P =* 0.95, *I =* 0%). The mean diurnal IOPs at the end were also similar (WMD 0.07 mmHg, 95%CI [−0.20 to 0.34], *P =* 0.63, *I =* 0%) (**Figure 3**). No publication bias on egger test was found (*P* ≥ 0.1; **Supplementary Table 3**).

### Brinzolamide vs Brimonidine

The changes of IOP from baseline between brinzolamide and brimonidine were shown in **Figure 5**. Both drugs significantly decreased IOP as adjunctive therapies to PGAs. At 9 am, a significant reduction of IOP was found in the brinzolamide (b.i.d.) compared to brimonidine (b.i.d.) (WMD −1.11 mmHg, 95%CI [−1.60 to −0.61], *P* < 0.0001, *I =* 0%); however, the difference was not significant in thrice daily subgroup (WMD −0.60 mmHg, 95%CI [−2.13 to –0.96], *P =* 0.44). At 12 am, brinzolamide (b.i.d.) was similar to brimonidine (b.i.d.) in IOPlowering effect, with a statistically significant heterogeneity (WMD −0.53 mmHg, 95%CI [−1.40 to −0.34], *P =* 0.23, *I =*  53%). When thrice daily, brimonidine led to a greater IOPlowering effect than brinzolamide, with statistically significant

FIGURE 2 | Forest plot for IOP change from baseline (Brinzolamide group vs. Timolol group). The plot of change value for IOP at 9 am (A), 12 am (B), 4 pm (C).

heterogeneity (WMD 2.07 mmHg, 95%CI [0.37 to 3.78], *P =*  0.02, *I =* 70%). In this analysis, we did not perform a sensitivity analysis due to only including two trials in each dose group. At 4 pm, brinzolamide (b.i.d.) had a superior IOP-lowering effect compared with brimonidine (b.i.d.) (WMD −0.97 mmHg, 95%CI [−1.51 to −0.44], *P =* 0.0003, *I =* 0%); conversely, the effect in brinzolamide (t.i.d.) was lower than brimonidine (t.i.d.) (WMD 1.19 mmHg, 95%CI [0.74 to 1.64], *P* < 0.0001, *I =* 0%). With regard to the mean diurnal IOPs at the end, brinzolamide was lower in twice daily subgroup (WMD −1.20 mmHg, 95%CI [−1.31 to 1.08], *P* < 0.00001, *I =* 0%), brimonidine was lower in thrice daily subgroup (WMD 1.41 mmHg, 95%CI [1.02 to 1.80], *P* < 0.00001, *I =* 0%), and the results were also consistent with their IOP changes. There were no publication biases on egger test (*P* ≥ 0.1 for each group; **Supplementary Table 3**). The changes of IOP between brinzolamide and brimonidine could be related to their plasma concentrations and pharmacokinetics (detailed descriptions in the Discussion section).

### Safety Analysis

To obtain a more comprehensive safety profile, we compared the common AEs between brinzolamide and other anti-glaucoma agents, including monotherapies and add-on therapies.

### Blurred Vision and Conjunctival Hyperemia

Blurred vision was one of the most common AEs of brinzolamide. Compared with active comparators (dorzolamide, brimonidine), a greater proportion of patients suffered from blurred vision in brinzolamide (**Table 1**). However, the difference between brinzolamide and timolol was not significant (**Table 1**). For conjunctival hyperemia, the incidence was significantly increased in brinzolamide compared to timolol (**Table 1**); there were no significant differences when comparing brinzolamide with other active comparators (dorzolamide, brimonidine) (**Table 1**).

### Occurrence of Taste Abnormality

Occurrence of taste abnormality was analyzed to have a similar incidence between brinzolamide and dorzolamide (**Table 1**). However, compared with other active comparators (timolol, brimonidine), the reports of occurrence of taste abnormality were significantly higher in brinzolamide (**Table 1**).

### Ocular Discomfort, Eye Pain, and Ocular Hyperemia

Ocular discomfort and eye pain were analyzed to have significantly lower incidences in brinzolamide compared to dorzolamide (**Table 1**); but the differences between

Brinzolamide group and Timolol group (6.1.1); between Brinzolamide group and Dozorlamide group (6.1.2); between Brinzolamide group and Brimonidine group (b.i.d) (6.1.3), (t.i.d) (6.1.4).

brinzolamide and other active (timolol, brimonidine) were not significant (**Table 1**). For ocular hyperemia, the incidence was significantly lower in brinzolamide than brimonidine (**Table 1**); nevertheless, the differences between brinzolamide and other active comparators (timolol, dorzolamide) were not significant (**Table 1**).

### Other Adverse Events

There were no significant differences in the incidence of occurrence of conjunctivitis, eye pruritus, foreign body sensation in eyes, and treatment-related AEs when we compared brinzolamide with active comparators (timolol, dorzolamide, and brimonidine) (all *P >* 0.05; **Table 1**). Besides, no severe AEs were reported in most studies.

### DISCUSSION

In the present systematic review, we assessed 26 RCTs, containing a comparison between brinzolamide and timolol in 7 studies (Silver, 1998; March and Ochsner, 2000; Hollo et al., 2006; Reis et al., 2006; Kaback et al., 2008; Miura et al., 2008; Pfeiffer, 2011), brinzolamide and dorzolamide in 10 studies (Sall, 2000; Michaud and Friren, 2001; Tsukamoto et al., 2005; Bournias and Lai, 2009; Martinez and Sanchez-Salorio, 2009; Manni et al., 2009; Nakamura et al., 2009; Sezgin Akcay et al., 2013; Alcon, 2016; Aihara et al., 2017), and brinzolamide and brimonidine in 9 studies (Feldman et al., 2007; Day and Hollander, 2008; Katz et al., 2013; Konstas et al., 2013; Nguyen et al., 2013; Research, 2013a; Research, 2013b; Whitson et al., 2013; Aung et al., 2014). Patients with POAG or OHT have a common characteristic as elevated IOP, which is closely associated with progression of visual field deterioration. Currently, IOP level control is a primary goal for the treatment of POAG and OHT.

Our analyses found that brinzolamide had similar efficacies to timolol in lowering IOP at three time points (9 am, 12 am, 4 pm) and holding the mean diurnal IOP at the end of treatment duration, as add-on therapies to a PGA, which were not inconsistent with the effects as monotherapies. In a previous meta-analysis, monotherapies were adopted to treat patients with POAG or OHT, and the relative

peaks of reduction in IOP were 17% and 27% for brinzolamide and timolol, respectively (van der Valk et al., 2005). Timolol and brinzolamide could reduce formation of AQH; the former decreases blood flow to the iris root–ciliary body while the latter inhibits CAI, and timolol has a stronger effect on the process than brinzolamide (Costagliola et al., 2002; Shoji et al., 2005). PGAs, besides exporting the first-line effect, could enhance the activity of CAI, so brinzolamide is added on PGAs benefiting patients with glaucoma or OHT to achieve further reductions in IOP (Puscas and Coltau, 1995; Miura et al., 2008). However, there is not an interaction between timolol and PGAs. Thus, the efficacy of brinzolamide was similar to timolol as added on PGAs in daytime. Moreover, brinzolamide also lowered the nighttime IOP, although the effect was smaller than during daytime (Liu et al., 2004; Liu et al., 2009). The performance may be explained by the fact that brinzolamide is weaker in reducing the formation of AQH during the nocturnal period than during the diurnal period (Ingram and Brubaker, 1999). In contrast, timolol has no similar effect, because there are normally reductions of endogenous circulating catecholamines in night-time (Topper and Brubaker, 1985; Liu et al., 2004; Liu et al., 2009; Konstas et al., 2016).

As for brinzolamide and dorzolamide, they had same mechanisms that lowered IOP by inhibiting the activity of carbonic anhydrase and enhanced ocular hemodynamic function by retarding the release of intracellular Ca2+(Chandra et al., 2016; Dong et al., 2018). Accumulating evidences showed that visual field defect was highly related to the reduction in ocular blood flow (Deokule et al., 2010; Calvo et al., 2012). Therefore, it was reasonable to our results that brinzolamide and dorzolamide had a similar effect in lowering IOP at three time points (9 am, 12 am, 4 pm) and same mean diurnal IOPs at the end of treatment duration, as add-on therapy to a PGA or beta-blocker.

In terms of the comparisons between brinzolamide and brimonidine where some changes had been generated, we implemented subgroup analysis on medication times. At 9 am, brinzolamide was more effective than brimonidine when added to PGAs, and a similar tendency occurred in the brinzolamide group (b.i.d.) at 4 pm. Interestingly, this tendency had reversed in brimonidine (t.i.d.) at 12 am and 4 pm, respectively. However, the difference was not statistically significant comparing brimonidine (b.i.d.) with brimonidine (b.i.d.) at 12 am. In addition, brinzolamide

#### TABLE 1 | Meta-analysis for the efficacy and safety outcomes.


had a lower mean diurnal IOP in twice daily subgroup, but brimonidine had a lower mean diurnal IOP in thrice daily subgroup. The reasons leading to variabilities for therapeutic effect were listed as follows. On the one hand, brinzolamide worked within 30 min and reached a peak after 1–2 h after administration (Silver, 1998). But for brimonidine, the effect was only observed within 1 h, and the peak effect occurred by 2–3 h (Walters, 1996). Nevertheless, the IOP lowering effect of two drugs dropped back to a trough over 10 h after doses (Anderson, 2003; Lusthaus and Goldberg, 2017). On the other hand, the plasma concentrations of two drugs (t.i.d.) were greater than the drugs (b.i.d.), which were more beneficial to the effect of brimonidine than brinzolamide (Fudemberg et al., 2008).

Safety profile of brinzolamide was similar to the other three active comparators when they were used as either monotherapies or adjunctive therapies, but there were some AEs with diverse frequencies. The incidences of blurred vision and taste abnormality were outstanding in brinzolamide. Differing from brinzolamide, ocular discomfort and eye pain were common in dorzolamide; ocular hyperemia was common in brimonidine; timolol led to a low risk of conjunctival hyperemia. All AEs of the four drugs were usually mild and superficial depending on their unique structures; furthermore, brinzolamide with a physiological pH could also ameliorate tolerability and adherence (Silver, 1998).

The present study still had some limitations. First, all articles in our study were published in English; there was a potential risk that we failed to involve some papers that were published in other languages. Second, people with glaucoma and OHT eventually ended up with visual field loss. The patient's visual field directly reflected the disease progression. However, there was not a precise and accepted visual field detection method at present. Therefore, we evaluated the treatment effect of study drugs by IOP changes from baseline as well as the mean diurnal IOP at the end of treatment duration to restrict the potential bias. Third, due to the lack of data, we did not compare the nocturnal IOP, which also played a dominant role responding to the control level of the disease. It was necessary that more researches were still needed for the available guidance. Finally, there was a lack of costeffectiveness studies of brinzolamide as an adjunctive therapy.

### CONCLUSION

This meta-analysis indicated that brinzolamide, as add-on to PGAs or β-blocker, could significantly decrease IOP of

### REFERENCES


people with refractory glaucoma or OHT, and the AEs of brinzolamide were tolerable. Therefore, it could be used as a replacement therapy for patients whose IOP became uncontrollable with a PGA or timolol alone; or as an alternative treatment to patients with contraindications of timolol and brimonidine.

### AUTHOR CONTRIBUTIONS

YH contributed to devising the topic and writing the manuscript. YL and JZ contributed equally to this work (contributed by writing the manuscript and analyzing the data). XZ and QW contributed to checking the data.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00679/ full#supplementary-material


or brimonidine 0.2%. (NCT01297920). Available online at: https://www. clinicaltrials.gov/


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Liu, Zhao, Zhong, Wei and Huang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Medicine Shortages: Gaps Between Countries and Global Perspectives

*Angela Acosta1,2\*, Egdda Patricia Vanegas1,3, Joan Rovira1,4, Brian Godman1,5,6,7 and Tomasz Bochenek1,8*

*1 ISAGS, South American Institute of Government in Health, UNASUR, Rio de Janeiro, Brazil, 2 RAM Group, National University of Colombia, Bogotá, Colombia, 3 SEPRO Research Group, National University of Colombia, Bogotá, Colombia, 4 Andalusian School of Public Health, Granada, Spain, 5 Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow, United Kingdom, 6 Division of Clinical Pharmacology, Karolinska Institute, Karolinska University Hospital Huddinge, Stockholm, Sweden, 7 School of Pharmacy, Sefako Makgatho Health Sciences University, Garankuwa, South Africa, 8 Department of Drug Management, Faculty of Health Sciences, Jagiellonian University Medical College, Krakow, Poland*

Introduction: Over the last decade, global health policies and different research areas have focused on the relevance and impact of medicine shortages. Published studies suggest there have been difficulties with access to medicines since the beginning of the 20th century, and there have been advances in our understanding and management of the problem since then. However, in view of global and regional health care concerns with shortages, we believe this phenomenon needs to be characterized and described more fully regarding the types of medicines affected, possible causes, and potential strategies to address these. The aim of this scoping review was to identify, compare if possible, and characterize the recent literature regarding the situation of medicines shortages between countries, and provide different perspectives, including a global context and national approaches.

Methodology: A scoping study presented as a narrative review of the situation and findings principally based on published articles.

Results: Based on the reported cases in the literature, a typology of medicines shortage and supply interruption episodes and their causes were proposed; national approaches to notify and manage the medicines shortages cases were described and classified by update frequency; principal differences between market and supply chain management perspectives of the situation were identified and global and countries' perspectives were described.

Conclusion: Policy makers require solutions that prevent those cases in which the population's health is affected by episodes of medicine shortages and/or interruption in the supply chain. There is also a need to generate a glossary related to logistics management and the availability of medicines which will be useful to understand and overcome shortages. In addition, recognize that potential solutions are not only related with actions linked to research, development and innovation, but much wider. Overall, we believe this article can act as a basis for future discussions in this important area.

Keywords: medicine shortages, medicine access, pharmaceutical policy, medicine supply, South America, Europe, North America, Western Asia

#### *Edited by:*

*Sam Salek, University of Hertfordshire, United Kingdom*

#### *Reviewed by:*

*Robert L. Lins, Retired, Belgium Muhammad Usman, University of Veterinary and Animal Sciences, Pakistan*

#### *\*Correspondence:*

*Angela Acosta angelaacosta@isags-unasur.org*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 31 January 2019 Accepted: 12 June 2019 Published: 19 July 2019*

#### *Citation:*

*Acosta A, Vanegas EP, Rovira J, Godman B and Bochenek T (2019) Medicine Shortages: Gaps Between Countries and Global Perspectives. Front. Pharmacol. 10:763. doi: 10.3389/fphar.2019.00763*

## INTRODUCTION

The World Health Organization (WHO) defined "access to medicines" as a multidimensional problem in view of the rising prices of new medicines and persisting problems of medicine shortages among others (World Health Organization, 2018). Other concerns include out-of-pocket payments, which are especially important in lower- and middle-income countries (LMICs) where expenditure of medicines can be up to 70% of total health care expenditure and potentially catastrophic for patients and their families if they become ill (Cameron et al., 2009; Ofori-Asenso and Agyeman, 2016; World Health Organization, 2018). However, this is outside the scope of this article, which principally deals with issues of shortages of medicines.

The last WHO Director-General's report included a variety of terms on global medicine shortages such as "shortage," "scarcity" (only in the Spanish version), and "stock outs" (only in the English version) that demand a comprehensive approach across countries (World Health Organization, 2018). Consequently, there is an urgent need to develop a set of terminologies related to the problems identified by countries regarding the continual availability of essential medicines.

There are different situations leading to out of stock of medicines. Some of them can be solved without causing obstacles to health care provision or in the availability of the best therapeutic option, whereas others may require additional efforts and ways to overcome affected health conditions. There have been several important efforts to document countries' experiences and potential ways forward to address concerns with medicine shortages. The study of ISAGS UNASUR published in 2017 characterized and analyzed the situation among eight South American countries: Bolivia, Chile, Colombia, Ecuador, Paraguay, Peru, Venezuela, and Uruguay (ISAGS, 2017). In 2018, Bochenek et al. (2018) systematically characterized, compared, and evaluated current measures, as well as legislative and organizational frameworks, to address medicines shortages among a wide range of European and Western Asian countries. These included 20 countries of the European Union (EU) and the European Free Trade Association (EFTA)—Austria, Belgium, Croatia, the Czech Republic, Estonia, France, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Malta, Norway, Poland, Portugal, Slovakia, Slovenia, Spain, and Switzerland. In addition, eight non-EU/EFTA countries: Albania, Azerbaijan, Israel, Kosovo, Montenegro, Republic of Srpska (Bosnia and Herzegovina), Serbia, and Turkey.

Both publications involving descriptions from different regions of the world with similar findings about formal definitions as well as strategies that had been developed to prevent or mitigate against medicine shortages. Both publications emphasized the urgent need to explore further the phenomena to identify mechanisms which introduce possible solutions for situations affected by medicine shortages, as well as facilitate the monitoring and assessing of signals and subsequent actions. However, there is a need to consolidate current findings to provide additional direction given current concerns, building on the experiences of other countries including Australia, Canada and the United States, as well as more recent research findings (Gupta and Huang, 2013; The Society of Hospital Pharmacists of Australian, SHPA, 2017; Videau et al., 2019).

Learning from other countries' experiences should not be underestimated or underutilized in shaping local or national pharmaceutical policies (Godman et al., 2010; Gupta and Huang, 2013; Godman et al., 2014; Moon et al., 2014; Godman et al., 2015; Moorkens et al., 2017; The Society of Hospital Pharmacists of Australian, SHPA, 2017; Godman et al., 2018; Kwon et al., 2018; Videau et al., 2019). Lessons can be drawn from crosscountry comparisons, even if a given country's characteristics do not perfectly correspond in terms of geographical location, size, demography, economy, or type of health care system. Consequently, the objectives of this article are to consolidate current findings as well as characterize and describe more fully the situation across continents to provide further evidence about the types of affected medicines, identified causes, and potential strategies to address shortages. Subsequently, use the findings to provide suggestions to address this important topic.

### MATERIALS AND METHODS

The design of this study is a scoping review. Scoping reviews have been useful to describe broad topic and provide an overview of diverse literature, including different study designs and methodologies, both widely available and gray scientific articles and reports (Pham et al., 2015).

Previous efforts have helped identify the main characteristics to describe the medicine shortage situation including potential definitions, general characteristics of the problem, description of information systems, potential and perceived causes of shortages, and implemented solutions (UNASUR, 2014; De Weerdt et al., 2015a; De Weerdt et al., 2015b; Pauwels et al., 2015; de Weerdt et al., 2017; Nurse-Findlay et al., 2017; Bochenek et al., 2018). We have built on this including the studies of Bochenek et al. (2018) and ISAGS (2017).

Full information on the content of the ISAGS survey form, including all the detailed questions which were posed in study on South American countries, can be found in the **Supplementary Material** to this paper. Likewise, detailed information on the content of the survey performed in European and Western Asian countries can be found in the **Supplementary Material** to the published study of Bochenek et al. Both surveys asked about existence of definitions of shortages of medicines, general characteristics of the problem, description of information systems, groups of medicines in shortage and particular molecules, potential and perceived causes of shortages, existence of processes, protocols and indicators to address shortages when they appear and to monitor their dynamics, and implemented solutions among others.

### Search Process

### First Step

**Figure 1** shows the search process. In 2017, eligible studies were first identified from a search of PubMed with the following general MeSH terms strategy "DRUG OR MEDICINE AND SHORTAGE." The results were subsequently filtered by period (5 years from May 2012 to 2017), with references filtered by tittle and abstract taking into account representative descriptions of countries.

Other gray literature was retrieved from websites of the WHO, ministries of health and national health authorities. In addition, legal acts, as well as information gathered and disclosed within the public domain by organizations involved in pharmaceutical markets.

#### Second Step

**Figure 2** shows the search process for the update implemented in 2019. The same search strategy was used filtered by period (January 2016 until April 2019) to help recover additional eligible studies. The strategy was also translated into Portuguese and Spanish to run a search in Lilacs, a specific database for Latin America countries.

To retrieve other potential studies, free text search terms and a snowball literature review was undertaken. We ran the term "SHORTAGE" with specific countries and continents including Africa, Australia, Canada, China, Costa Rica, Japan, Mexico, Panama and the United Kingdom.

#### Inclusion Criteria

This review includes studies that describe the medicines shortage situation within a large jurisdiction or system of care, and settings could be regional, national, or international.

#### Selection Process

Two reviewers (EV, AA) performed the screening of abstracts based on the tittle/abstract review. The full text of potential eligible studies was retrieved if one or both authors thought they were relevant. Disagreements were resolved by discussion and, when necessary, including a third author (TB) in the discussion.

#### Data Extraction

Data extracted from each study included the authors and year of publication, jurisdictional level, existence of definitions, general characteristics of shortages, medicines involved, description of information systems including four categories of frequencies of

shortages reports (high, medium, low, unspecified), potential and perceived causes of shortages, and possible implemented solutions at jurisdictional levels.

### RESULTS

### Description of Studies

By 2017, the literature review had identified 50 references to include in this scoping review, and by 2019 another 6 new references where identified. Of these 56 references, 9 studies had a regional broad

focus for Europe and Latin America. We found studies with country description of medicines shortage for one East Asia country (China), 7 European countries (Belgium, Finland, France, Ireland, Slovenia, Spain and the United Kingdom), two Latin America countries (Brazil and Venezuela); two North American countries (Canada and the United States); two Oceania countries (Australia and Fiji); and 4 Western Asia countries (Iran, Iraq, Jordan and Israel) (**Table 1**).

Three of the identified articles had developed surveys addressed to the ministries of health, state medicines agencies and local health authorities of various countries from three regions to discuss potential ways to address medicine shortages (Bogaert

#### TABLE 1 | Distribution of included references by geography.


*(Continued)*

TABLE 1 | Continued


*(Continued)*

#### TABLE 1 | Continued


et al., 2015; ISAGS, 2017; Bochenek et al., 2018). One study described the literature review in terms of the characteristics of shortages and country definitions of shortages (De Weerdt et al., 2015a). Another regional study described the shortage situation from the perspective of hospital pharmacists and prescribers (Pauwels et al., 2015).

The United States (US) is the country with more publications in the field (26 references) than many other countries. Many of these publications describe the professionals' perspective of the situation at the health care jurisdictional level (Morrison, 2011; Golembiewski, 2012; Griffith et al., 2012; Becker et al., 2013; McKeever et al., 2013; McLaughlin et al., 2013; Bible et al., 2014; Goldsack et al., 2014; Butterfield et al., 2015; Caulder et al., 2015; McLaughlin and Skoglund, 2015; Gabrielli et al., 2016; Parsons et al., 2016; Setayesh and Mackey, 2016; Bocquet et al., 2017; Fox and Tyler, 2017; Mazer-Amirshahi et al., 2017; Rinaldi et al., 2017; Schwartzberg et al., 2017). Another study performed a comparison of medicines shortages between two hospital settings, one from Arabia Saudi and the other from the US (Alsheikh et al., 2016).

### Contexts and Reasons of Medicines Shortage

The description of the shortage situations may be associated with 4 principal causes or determinants, namely: market, supply chain management, manufacturing, and political issues (**Table 2**).

#### TABLE 2 | Most frequent reasons for medicine shortages.


Some authors have concluded that these categories are interrelated and have one aspect in common, that is, many cases are related to the availability of safe and effective medicines with low profitability or with low sales making them non-viable commercially (McKeever et al., 2013; Alevizakos et al., 2016; Schwartzberg et al., 2017).

In South American countries, medicine shortages generally occur with mature products without suppliers in the market due to lack of market viability, and correspond mostly to parenteral medicines with low profitability (ISAGS, 2017).

Some countries issue alerts about medicines that are simply not available on the market in their country even if there is enough money to pay for them within their health care systems (Bochenek et al., 2018).

Finally, Bochenek et al. (2018) addressed an increasing amount of evidence where medicines were unavailable in countries even if the products complied with current regulations and were financed within the health care system. As a result, these situations show that ethical and political issues could be affecting the timely availability of first-line therapeutic alternatives. These situations threaten the ability of clinicians and governments to fulfill their moral obligations to patients and society to provide benefit to patients, minimize harm, and promote equity.

Published studies in hospitals allow a better follow up of the health consequences of medicine shortages (Pauwels et al., 2015; de Weerdt et al., 2017); which is more difficult in ambulatory care (Golembiewski, 2012; Becker et al., 2013; Goldsack et al., 2014; Jagsi et al., 2014; Chen et al., 2016). This scoping review identified a number of published articles describing inpatient challenges regarding medicine shortages in the US (Morrison, 2011; Golembiewski, 2012; Griffith et al., 2012; Becker et al., 2013; McKeever et al., 2013; McLaughlin et al., 2013; Bible et al., 2014; Goldsack et al., 2014; Butterfield et al., 2015; Caulder et al., 2015; McLaughlin and Skoglund, 2015; Gabrielli et al., 2016; Parsons et al., 2016; Setayesh and Mackey, 2016; Bocquet et al., 2017; Fox and Tyler, 2017; Mazer-Amirshahi et al., 2017; Rinaldi et al., 2017; Schwartzberg et al., 2017).

### Medicines Involved

The characterization of supply shortages, their frequency, and the main groups of medicines affected among countries and regions are described in **Table 3**. The same characterization though is not made for the impact of shortages on patients' health and health care systems. Descriptions in this respect generally correspond to descriptions of cases regarding either the impact of shortages on health conditions or medical specialties, or limitations in obtaining data and estimates made from surveys.

The shortage of essential medicines, including the active ingredients mostly used in injectable chemotherapy medicines, antibiotics, and anesthesia, is causing growing concern across regions including Europe, North America, Asia, and South America. However, the problem is much broader, affecting other classes of medicines - mainly parenteral medicines (Becker et al., 2013; Gulbis et al., 2013; Gupta and Huang, 2013; Jagsi et al., 2014; Caulder et al., 2015; De Weerdt et al., 2015b; De Weerdt et al., 2017; Schwartzberg et al., 2017) including anesthetics, nutrition and electrolyte solutions, enzyme replacement medicines, radiopharmaceuticals and antibiotics. The shortage of medicines has also been observed and documented for instance in Australia, Canada, China and Israel (Gray and Manasse, 2012; Kaposy, 2014; Schwartzberg et al., 2017). Most of the published evidence from countries of South America shows that shortages can occur even with essential medicines and especially injectable forms (ISAGS, 2017) (see **Figure 3**).

The groups of medicines, which were most frequently reported to be in shortage, were the cardiovascular medicines in Canada and Belgium; nervous system medicines in Australia, China, and Israel; and anti-infective medicines for most of the countries in South America (Bogaert et al., 2015; Yang et al., 2016; ISAGS, 2017; Schwartzberg et al., 2017; Videau et al., 2019).

Brazil has reported the national shortage of penicillins (firstline treatment) as a result of the lack of specific raw materials for their production in the international market. This episode

#### TABLE 3 | Description of medicines classes with shortages in the selected countries.


was described because of the increased incidence of syphilis, including also congenital syphilis, during the last 5 years and the concern this causes (see **Figure 4**). The alternative second-line antibiotics to treat acquired syphilis and the partners of pregnant women in Brazil, i.e., doxycycline and ceftriaxone, have doses of between 8 and 15 days, and there are concerns with adherence in practice (Lazarini and Barbosa, 2017).

In France (Bocquet et al., 2017), supply shortages with 71 types of medicine were identified. According to the ATC

classification (Norwegian Institute of Public Nealth, 2018), the most prevalent medicine were antibiotics, oncological medicines, antidepressants, antipsychotics, tuberculosis medicines, vaccines, and immunoglobulins.

In Iran, at least 73 cases of medicine shortages were identified, among which 44% were essential. Potentially, they impacted on successful management of disease areas including HIV, epilepsy, hemophilia, thalassemia, and patients undergoing organ transplants (Setayesh and Mackey, 2016).

## Definitions

### National Definitions

**Table 4** contains definitions adopted by countries, considering attributes with the formulation and agreement of potential definitions.

One of the most important findings of this scoping review is that consulted sources, literature and institutional websites, have identified a considerable number of countries that are introducing legislative actions to cope the medicines shortages and had included formal definitions or related terms of medicine shortages.

Recently, Bochenek et al (2018) identified four countries in Europe with specific definitions, and ISAGS in 2017 (ISAGS, 2017) described three definitions for Latin American countries. This scoping review found progress in both regions, i.e., eight European countries and seven Latin America now have national medicines shortage definitions. In addition, by using free terms in the search strategies, we found definitions for one more region, North America (Canada and the United States), and for one country, Australia (**Table 4**). There are two primary sources of definitions: one from health authority agency websites (Argentina, Australia, Brazil, Colombia, Spain, US, Uruguay, Norway), and the other is the published literature for countries including Belgium, Canada, France, Greece, Hungary, and Italy (Bogaert et al., 2015; Bocquet et al., 2017; Bochenek et al., 2018; Videau et al., 2019).

Few countries have currently established a specific definition that includes the term "shortages" alone or in combination with the terms "medicines" or "drugs." These include Belgium, Canada, Colombia, France, Hungary, Greece, Italy, Spain, and the US (**Table 4**).

Some countries have used logistics and market related terms, including Uruguay (declaration of interruption of sale or interruption of marketing), Argentina (medicines lacking), Croatia (disturbance on the medicines' market), Norway (temporary disruption of a medicine's marketing), Brazil, which uses the term "discontinuation" complemented with a term related to the temporality (definitive or unplanned or temporary), Italy (short supply), and Peru, which uses three terms "supply shortage," "unavailable pharmaceutical products," and "pharmaceutical product with limited supply" (**Table 4**).

The United States Food and Drug Administration (US FDA) has focused more on aspects related to scarcity and established a definition: "a period when the demand or projected demand for a medically necessary drug in the United States exceeds its supply" (CDER C for DE and research, 2018).

Bolivia, Chile, Croatia, Ecuador, Greece, Hungary, Norway, and Venezuela currently do not have an official definition of medicine shortage. These countries consider terms associated with medicine shortages but currently do not define the term medicine shortages (Bochenek et al., 2018). Finally, Venezuela differentiates scarcity as the situation in which there is insufficient quantity of a medicine to meet current demand and a supply shortage when the product is just not available (Ministerio de Salud Venezuela, 2017).

### Country Approaches

A new website at the European Medicines Agency (EMA) includes information about national legislation and local report mechanisms for 24 of the 30 European countries1 . In South America, Brazil, Argentina, and Uruguay have regulations that make the reporting of situations that could potentially lead to shortages mandatory. EMA also have a regulation to report these cases.

Two qualitative studies have documented countries' legislation and report mechanisms for three regions: Europe, Western Asia, and South America (ISAGS, 2017; Bochenek et al., 2018). As mentioned, this scoping review described one new region, North America (Center for Drug Evaluation and research C, 2018; Videau et al.,

<sup>1</sup> European Medicines Agency. Medicine shortages. https://www.ema.europa.eu/ en/human-regulatory/post-authorisation/medicine-shortages

#### TABLE 4 | Countries definitions.


*1 Administración Nacional de Medicamentos, Alimentos y Tecnología Médica. https://www.argentina.gob.ar/anmat*

*2 The Therapeutic Goods Administration. Medicine Shortages Information Initiative Available from: https://apps.tga.gov.au/prod/MSI/search#furtherinformation*

2019). In addition, it was possible to include information from 12 new countries: Australia, Bulgaria, Canada, Czech Republic, Denmark, Finland, Germany, Iceland, Romania, Sweden, United Kingdom (UK), and the US (All-party Pharmacy, 2015; Schwartzberg et al., 2017; The Society of Hospital Pharmacists of Australian, SHPA, 2017; CDER C for DE and research, 2018; Videau et al., 2019).

### Information Systems and Vigilance

Countries are taking actions regarding legislation and reporting systems in public websites (**Table 5**). Most of them are at the National Regulatory Agency level (28 countries), whereas others are at the Ministries of Health or National Health System level, including Colombia, Israel, Malta, Poland, Spain, Switzerland,


TABLE 5 | Frequency of update on the publicly available databases to report medicines shortages.

*(Continued)*

#### TABLE 5 | Continued


*(Continued)*

#### Acosta et al. Different Perspectives on Medicines Shortages

#### TABLE 5 | Continued


UK, and Uruguay). Five countries were documented by Bochenek et al. in 2018 (Gulbis et al., 2013; Bochenek et al., 2018) where pharmacist professional organizations and other stakeholders are involved in medicines shortages reports. Finally, Canada has a website operated by a telecommunication company (**Table 5**).

Some of the main features of these systems include the frequency of updates, the obligation of pharmaceutical companies or marketing authorization holders to notify key stakeholder groups, and public institutions in charge of the database. To enhance our understanding of the characteristics of databases on medicine shortages in selected countries of Europe, Western Asia, North America, and Latin America, all databases have been divided into four groups, depending on frequency of their updating: high (daily), medium (weekly), low (less often than weekly), and unspecified (**Table 5**).

Most of the countries have only a national reporting system, where the Ministry of Health or the National Regulatory Agency manage the database and request mandatory reports. However, other countries have more than one system involved in the gathering of information, such as professional associations in France, Ireland, Malta, Portugal, Spain, and Switzerland (Bochenek et al., 2018). Finally, 12 of 40 countries have not reported databases on medicine shortage, six from Europe (Albania, Cyprus, Lichtenstein, Montenegro, Netherlands, Serbia), seven from South America (Guyana, Suriname, Ecuador, Bolivia, Paraguay, Peru, Venezuela), and Azerbaijan from Western Asia. None of the databases reported affected health conditions, or the possible impact caused by the reported episode.

Although all analyzed European countries have mandatory reporting systems, this scoping review revealed information systems with high frequency of shortages reports in countries, such as Australia, Canada, and the US. Countries of Latin America and Europe have low or unspecified update frequency of their medicines shortages websites on their (**Table 5**).

### Networks and Initiatives

In 2014, the South American Council for Health from UNASUR issued a declaration on "Access to medicines and problems of medicines shortage," which stated that medicines shortage is a global and regional problem that manifests itself in diverse and changing ways, with various effects since there is insufficient information to determine the magnitude and features of the problem (ISAGS, 2017).

During 2016, several South American countries proposed to document the shortages situation of essential medicines in the region and to formulate strategies as part of the South American Institute of Government in Health (ISAGS UNASUR) actions for 2017. ISAGS UNASUR, together with the Andean Health Organization (Hipólito Unanue Agreement), undertook a study which described the situation in the member countries of the Andean and South American regions (ISAGS, 2017). An analysis of the medicines shortage situation of essential medicines was developed through the collection of information on decisions related to the problem, types of medicines identified by each country, identified causes, protocols of approach, solutions implemented, management and impact indicators, as well as limitations and experiences both at the country and regional levels that may be relevant to helping overcome the problem.

In 2016, the research collaboration initiative funded by the European Union was started. It is named COST CA15105— European Medicines Shortages Research Network—addressing supply problems to patients (Medicines Shortages) and it aims to stimulate and develop scientific research, as well as to propose solutions by end of 2020 (COST, 2015; COST, 2018). The COST CA15105 network encourages systematic sharing of information and research about shortages of medicines and nutritional products. It also aims to respond to the diverse interests of clinical and financial parties and patients' quality of life, to achieve analytical clarity on the causes of shortages, to simulate appropriate decision making in manufacturing and the trade of medicines, to highlight legal and economic frameworks, to disclose disincentives in the supply chain, as well as to reflect on best coping practices to help ensure patients' health is not compromised by ongoing shortages.

At the EU level, in 2016, the EMA and the Heads of Medicines Agencies (HMA) created an HMA/EMA Task Force on the Availability of Authorized Medicines for Human and Veterinary Use, aiming to provide strategic support and advice to tackle disruptions in the supply of human and veterinary medicines and to ensure their continued availability (EMA, 2019). The key priorities of the HMA/EMA Task Force included i) looking at ways to minimize supply disruptions and avoid shortages; ii) developing strategies to improve prevention and management of shortages caused by disruptions in the supply chain; iii) encouraging best practices within the pharmaceutical industry to prevent shortages; iv) improving sharing of information and best practices among EU regulatory authorities to better coordinate actions across the EU, and v) fostering collaboration with key stakeholders and enhancing communication of supply problems to EU citizens. A set of documents was published by EMA to support regulators involved in coordinating shortage situations due to good manufacturing practice (GMP) non-compliance. A public catalogue for shortages has been established by EMA2 , which is designed to communicate clear information on shortages to patients, health care professionals, and other stakeholders.

### DISCUSSION

### Studies Description

The studies included in this scoping review describe regional and countries contexts of medicine shortages and current perceptions of stakeholders, as well as their different perspectives. Indexed literature from the last 6 years presents important challenges, such as research on health and economic implications caused by supply disruptions and medicines shortages (Cousins, 2014; Steers, 2014; de Weerdt et al., 2017; Videau et al., 2019). More research efforts are needed to fully estimate the impact of medicine shortages on patients' health especially in ambulatory care.

The descriptive cases reported from the US identified parenteral and hospital medicines with shortages episodes and their implications. Moreover, published studies for countries, including Brazil, help to estimate the health implications of shortages of essential antibiotics (Griffith et al., 2012; Galvao et al., 2013; McKeever et al., 2013; Goldsack et al., 2014; Taylor et al., 2016; Mazer-Amirshahi et al., 2017). Next, efforts are needed to compare the current situation between country and regional levels.

### Shortage Definitions and Global Context

Some ministries and health authorities have developed important initiatives to the timely identification of potential medicine shortages alongside initiatives to manage episodes of interruption in the supply chain (Bocquet et al., 2017; Schwartzberg et al., 2017; Administración Nacional de Medicamentos A y TMA, 2018; CDER C for DE and research, 2018; INVIMA, 2018).

De Weerdt et al. (2015a) identified some elements included in the definitions, i.e., "supply," "delivery," "availability," "permanent discontinuation of drugs," and "time frame," to consider a uniform definition for drug shortages among European countries. Definitions from **Table 4** do not involve "delivery" and "availability" as these are not reflected in the current formal definitions. Instead, other common terms are included, such as "market" and "lack."

Other countries, including France, Belgium, Italy, Spain, Brazil, and Colombia, have adopted definitions that include the perspective of shortage and some aspects related to supply chain management and market determinants. However, it is worth mentioning that the US FDA defines the problem of medicines shortage only from the perspective of scarcity and includes also a lead time. In addition, the US FDA, unlike other countries including Brazil and France, states that they will never ask a producer to make medicines or change the amount of medicines to be manufactured as solutions for dealing with drug shortages. They do not see this as their role (CDER C for DE and research, 2018).

A further finding is that a few countries have established a specific period into their definition to confirm the shortage situation: Australia, Belgium, France, Norway, and Uruguay (Bogaert et al., 2015; Bocquet et al., 2017; ISAGS, 2017; The Society of Hospital Pharmacists of Australian, SHPA, 2017; Bochenek et al., 2018). There are though some differences based on the perspective of the definition. Belgium, France, Norway, and Uruguay establish the shortage situation in a specific point of the supply chain, for instance, community pharmacies, or hospital pharmacies, or wholesalers. These countries have included periods for shortages, which varies from 72 h to 30 days (ISAGS, 2017; Bochenek et al., 2018). On the other hand, Australia defined a period of 6 months considering the lack of a medication in the whole country (The Society of Hospital Pharmacists of Australian, SHPA, 2017). Uruguay uses a period less than 30 days as an "interruption of marketing" and more than 30 days as a "declaration of interruption of sale" (ISAGS, 2017).

One main feature of a specific shortage situation is the identity of the involved medicine, e.g., simvastatin (as an individual medicine) or the class of medicines (statins, lipid modulators). Shortages can also refer to a certain pharmaceutical form (e.g., injectable, capsule, or lotion), administration route (e.g., parenteral or oral), or a certain concentration or package size. The way the product episode is defined—broadly or narrowly—depends on the purposes of the analysis and the availability of information.

It should be highlighted that in 2017, a set of two co-existing definitions had been proposed by the WHO. On the supply side: "A 'shortage' occurs when the supply of medicines, health products and vaccines identified as essential by the health system is insufficient to meet public health and patient needs. This definition refers only to products that have already been approved and marketed, to avoid conflicts with research and development agendas." On the demand side: "A 'shortage' will occur when demand exceeds supply at any point in the supply chain and may ultimately create a 'stockout' at the point of appropriate

<sup>2</sup> European Medicines Agency—https://www.ema.europa.eu/en/humanregulatory/post-authorisation/medicine-shortages/shortages-catalogue

service delivery to the patient if the cause of the shortage cannot be resolved in a timely manner relative to the clinical needs of the patient" (WHO, 2017). The needs of patients about essential medicines are an important link in these definitions.

The report of the Director General of the WHO presented at the last Assembly on global shortage of medicines and access to them in 2018 (A71/12) gives considerable relevance to situations that are recognized as a shortage of medicines, which coincide with those recently reported by South American countries (ISAGS, 2017; Organizaciòn Mundial de la Salud O, 2017). They show how different points of the medicines' value chain are affected by concerns with demand, which are caused both by the lack of commercial attractiveness and by logistics, and supply factors that prevent having medicines in a timely manner.

In contrast, the title of the WHO's report refers to the global scarcity of medicines, and throughout the document, the term "shortage" is being used (Organizaciòn Mundial de la Salud O, 2017). In general terms, the difference between the two terms ("scarcity of medicines" and "shortage of medicines") can be minimal. However, it is worth specifying that "scarcity" refers only to those situations in which for a specific need the medication does not exist, whereas "shortage" covers those cases in which it is not possible to obtain the medicine in a timely manner. These latter problems are not solved by activities linked to research, development, and innovation.

Terms, such as "availability" and "affordability," have also to be clearly distinguished to describe the shortage phenomena where these occur to avoid confusion and concentrate on appropriate activities to address the situation.

Regarding "availability," at the jurisdiction/country level, this term usually implies that a medicine has marketing authorization and is marketed (can be bought or obtained from the health system). At the pharmacy level, this usually means that the medicine could be dispensed at demand or within a short period.

To contrast this, the term "affordability" refers to the extent the product is available at a reasonable price/cost for the patient or health care system considering the purchasing capacity of the individuals and/or the health system and does not endanger the financial sustainability of the purchasers. As mentioned earlier, this is particularly important for patients in LMICs where there are currently high co-payments. Affordability is also a growing issue in medium- to high-income countries especially with regard to new cancer medicines and those for orphan diseases with ever increasing prices (Howard et al., 2015; Godman et al., 2017; Godman et al., 2018; Kwon et al., 2018).

The shortage of medicines is related to two main activities planned by the WHO to be implemented between 2019 and 2023. These include i) research and development of medicines, and ii) supply chain management (States, 2019). Previous studies have reported essential medicines that need rapid solutions to address current concerns (ISAGS, 2017), i.e., the penicillins for syphilis and congenital syphilis.

The description of cases, the identified causes, and definitions of medicine shortages typically involve attributes that facilitate the identification of key solutions to address the main challenges of health conditions affected by the shortages. Currently, the specific actions described in the last version of WHO roadmap for 2019 to 2023 (States, 2019) do not discuss ways of approaching this and possible solutions to these cases, which is a concern.

We have highlighted the difficulty of building a simple typology of medicine shortages based on the multiple possible combinations of criteria. It may be more useful to define a checklist of relevant criteria that allows the characterization of the episode and to predict/estimate the potential impact/effects according to the likely magnitude and length of the shortage. This again will be researched further in the future.

### Causes and Characteristics of the Context That Determines the Identified Episode

The causes and context that define the shortage episodes or interruptions in the supply chain are different from other areas of policy that affect medicine availability, such as the nonavailability of medicines without a marketing license, or those still in the research phase, or issues of affordability, leading to a lack of reimbursement or funding.

Overall, we believe medicine shortages can be defined according to several criteria (**Figure 5**).

In fact, medicine shortages might have multiple causes. Some of the causes might have a sequential cause-effect relationship, such as a reduction of future envisaged demand due to a reduction of the incidence of relevant diseases or reduced use due to the availability of new valued medicines, replacing existing ones, which will negatively impact on future profitability. This might cause the manufacturer to stop investing in the manufacturing process and disregard safety concerns, which may in turn lead to an increase in side effects and subsequent closure of the manufacturing plant.

Beyond an agreed single definition of medicine shortages, the concepts that could be developed must identify precisely the causes and points in the supply chain for which they apply. Topics, such as "shortage episode" could be more related to causes included in the market and policy dimensions (**Table 2**) or "interruption of supply chain episodes" for those situations caused or detected from the dimension of supply chain management.

### Global Framework

Once a shortage episode is identified and characterized, including the population and health condition affected, the appropriate pharmaceutical and health policy tools, as well as rational management of medicines and their stocks, could help identify the most effective and efficient available intervention to address the problem. This could include fast tracking authorization and importation of the medicine or an equivalent from another country or recommending physicians to prescribe a substitute medicine; alternatively, potentially increasing reimbursed prices where profitability is a concern. Coordination of national efforts and international cooperation is urgently needed on various levels to address this, including UNASUR, the Andean Health Organization-Hipólito Unanue Agreement, the European Union, and the WHO. We will be following this up in the future as well.

There have been issues in Europe with older cancer medicines when the originator company has sold their molecule to a generic manufacturer, and this was re-launched at a considerably higher price, prompting potential counter measures (Pagliarulo, 2017;

Hawkes, 2017; Kahn, 2017; UK Department of Health and Social Care, 2016). Such activities will also be followed up, mindful of the need for manufacturers to make a profit for sustained supply.

Overall, we believe that specific strategies and solutions should be formulated at national and global levels. However, shortages must be clearly distinguished from scarcity. We believe this will lead to improved approaches to address episodes of medicine shortages and interruptions in the supply chain, as well as more pertinent solutions and their implementation to address current concerns.

There is a growing interest in international cooperation in the area of medicine shortages research, its prevention and ways to facilitate actions that provide a timely response. For South American countries, the development of a consensus to build or adapt a definition of medicines shortages demands the compilation of the countries' definitions to generate a glossary related to logistics management and the availability of medicines. Some approaches, such as risk management and a panel of logistical indicators for medicines, can be adapted to each country according to its complexities, including the health system, epidemiological alerts, and geographical peculiarities. At the regional level, it has also been proposed to design a regional platform for horizon scanning and undertaking a pilot project to identify medicines that may be at risk of stock-outs, or possible withdrawals from the market, by reviewing the license status and reviewing databases related to imports and pharmaceutical market. No less important is to develop pilots for those priority medicines that are affecting the population health in the region.

### Approaches to Medicine Shortages

The proposed categorization of causes of medicines shortages can be used as a guidance to find potential solutions, as well as considering the supply stage of medicines, market competition, and possible therapeutic substitutes.

The Ministries and Health Authorities must consider which health conditions are affected by the shortage or supply chain interruption. This includes the proposed therapeutic use of the medicine and the availability of other therapeutic options, considering the optimal treatment for the disease or health condition involved within available resources.

Maybe the most critical causes of medicine shortages are those related to market aspects because these situations depend on the will of the producer, seller, or buyer, and the power relationship between each part. Some common situations are when a pharmaceutical company decides to suspend the production of a medicine, or the seller decides to modify the price of a specific pharmaceutical product, affecting its funding and supply. In view of this, possible solutions cannot just be focused on new possible innovations.

Likewise, countries initiatives should not just consist only of designing and implementing systems to compile information about cases of medicine shortages. The specific interventions should be moved forward to address potential shortages, as stated earlier.

Other approaches could include designing and implementing of risk management programs that allow the anticipation of shortage episodes as well as implementing activities, including horizon scanning to identify "programmed obsolescence" of a medicine based on market withdrawals or changes in the pricing of products. We have seen horizon scanning and budgeting successfully used in countries to better plan for the future, with Sweden and exemplar for other countries (Wettermark et al., 2010; Malmström et al., 2013; Eriksson et al., 2017; Godman et al., 2018), and these activities are likely to grow.

On the other hand, problems related to supply chain management are focused on logistics management and the level of the supply chain or network. These problems can be mitigated through the design of health care supply chains, with objectives that look to maximize the service level (availability and accessibility) and optimize the financial component, with recent changes in the public system in South Africa, a recent example among LMICs (Meyer et al., 2017). Ministries of Health should set key performance indicators for the country's supply chain as well as implement improvements among the different agents in the supply chain over the different process including inbound and outbound transportation, fleet management, warehousing, materials handling, order fulfillment, logistics network design, inventory management, supply/demand planning, and the management of third-party logistics services providers (Council of Supply Chain Management Professionals (CSCMP), 2013). This will help reduce shortages in the future.

Some countries, such as France (Ordre National des Pharmaciens, 2015) have implemented actions focused on logistic aspects including defining a dead line to establish supply interruptions and length of shortage episodes. France also established a decree in 2012 to overcome the dysfunctions of the pharmaceutical distribution channel based on supply chain interventions, as well as introducing constraining and coercive measures for the various actors of the distribution channel. The decree defines shortages as the inability for a community or hospital pharmacy to deliver a medicine to a patient within 72 h if the medicine is currently not available, as well as possible therapeutic alternatives if needed, or which supply difficulties may lead to a risk of public health for patients, thereby creating an emergency. France established a procedure whereby all the actors of the medicine's distribution channel should join forces to help address current concerns. The decree also established some regulatory obligations for pharmaceutical companies, including ensuring an appropriate and continuous supply of medicines to wholesalers to meet public and patients' needs, implementing emergency call centers to inform agents about an anticipated or a current medicine shortage, informing the French authorities of emergency supplies, specifying the quantities and addresses, and informing of any risk concerns associated with any medicine shortage. The wholesaler must also declare its distribution territory, meet public obligations, have a selection of medicines at least nine tenths of commercialized pharmaceutical products in France, and ensure to deliver medicines within 8 h from any request (Ordre National des Pharmaciens, 2015; Bocquet et al., 2017). However, there are still challenges especially in the field of medicines with major therapeutic interest, and after 7 years of the introduction of the decree, there are still tensions between the different agents in the supply chain in France due to other causes not related to supply chain management (Biétry, 2018).

### CONCLUSIONS

Descriptions of medicines shortage extracted from the multiple studies included in this scoping review allowed the authors to extract main characteristics, as well as contexts and reasons of medicines shortage, which medicines are involved, definitions of shortages, countries information systems on shortage reports and alerts management, and finally any networks between countries and stakeholders' initiatives.

We believe this is first time such a comprehensive study has been undertaken reporting the most representative description of the current medicine shortage situation across multiple countries and giving inputs regarding national and global health policies related to the most frequent reasons reported for medicine shortages. This includes the description of the most likely medicine classes with shortages, current national definitions, and related terms and available databases to report medicine shortages. The study also describes frameworks and approaches among countries to help address these issues in the future. Overall, we believe our findings can help with the development of definitions related to the shortage of medicines that can be universally applied to prevent future confusion.

Signals linked with unattended or affected health conditions because of shortages must also be documented to establish effective interventions from a health authority perspective to ensure patients' health is not compromised. Indeed, policy makers require solutions that prevent those cases in which the population's health is affected by episodes of medicine shortages and/or interruption in the supply chain.

As part of this, we believe it is necessary to develop a consensus to build or adapt definitions of shortage and scarcity, as well as identifying all local definitions, to generate a glossary related to logistics management and the availability of medicines to facilitate aggregation of data from different sources and to better manage future situations. We have started this process with this article.

We also believe it is important to emphasize that for those cases where shortages are due to logistical and supply factors, and it is not possible to have the medicines in a timely manner, that potential ways forward are not related to improving research, development and innovation as some bodies are proposing. This is a very different situation, especially if alternative solutions are available including importation, as well as permitting increased prices if pertinent along with advocating and documenting possible alternatives, and these must be explored first.

Overall, we believe it would also be useful to identify options and best practices on how to address the most common causes of shortages and share these among all countries to provide future direction. These are research topics for the future building on this review across countries.

Our findings show that part of the problem of shortages is focused on provision, especially those cases related to the permanent suspension of production or voluntary suspension, due to, for instance, to the lack of commercial interest, either by low demand, end of the patent, price regulations, use of compulsory licenses and/or the introduction of new medicines. Regarding suspending commercialization, it is important to review specific cases that raise considerable public health concerns, such as antineoplastic and antibiotics, to suggest potential ways to overcome the situation. We have started this process and will continue.

### LIMITATIONS

Although the findings presented in this study allows us to make comparisons between countries related with, for instance, national definitions and involved medicines, there are other attributes that may well broaden the situation which we have not discussed. These can be addressed going forward.

### RECOMMENDATIONS

Further research is needed to fully assess the clinical and economic impact of medicine shortages at different jurisdictional levels, including both hospital and ambulatory care levels. Additionally, cross-country and regional studies will help to identify current essential medicines supply disruptions and shortages episodes, which need prompt solutions.

### REFERENCES


### AUTHOR CONTRIBUTIONS

AA and EV conducted the study and prepared the first draft of the article and the literature review. JR brought methodologic inputs and assessed preliminary results. JR, BG, and TB commented on and contributed to subsequent iterations on the manuscript.

### ACKNOWLEDGMENTS

The work on this article was facilitated by the ISAGS UNASUR institutional program of associate researchers, promoted by Carina Vance, its current Executive Director.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00763/ full#supplementary-material

aimed at preventing or mitigating drug shortages in 28 European and Western Asian Countries. *Front. Pharmacol.* 8. doi: 10.3389/fphar.2017.00942


Meyer, J. C., Schellack, N., Stokes, J., Lancaster, R., Zeeman, H., Defty, D., et al. (2017). Ongoing initiatives to improve the quality and efficiency of medicine use within the public healthcare system in South Africa; A preliminary study. *Front. Pharmacol.* 8, 1–16. doi: 10.3389/fphar.2017.00751

Ministerio de Salud Venezuela (2017). Comunicación.


FARMACOTERAPÉUTICA DE LA COMARCA. Vitoria - Gasteiz. Available from: http://www.osakidetza.euskadi.eus/contenidos/informacion/cevime\_ infac/es\_cevime/adjuntos/INFAC\_Vol\_23\_N\_7\_Desabastecimientos.pdf.


**Conflict of Interest Statement:** TB is an active member (Action Chair) of COST CA15105—European Medicines Shortages Research Network—addressing supply problems to patients (Medicines Shortages) funded by the European Union.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Acosta, Vanegas, Rovira, Godman and Bochenek. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Bleeding Risk in Patients Using Oral Anticoagulants Undergoing Surgical Procedures in Dentistry: A Systematic Review and Meta-Analysis

*Natália Karol de Andrade1, Rogério Heládio Lopes Motta1\*, Cristiane de Cássia Bergamaschi2, Luciana Butini Oliveira3, Caio Chaves Guimarães1, Jimmy de Oliveira Araújo1 and Luciane Cruz Lopes2*

*1 Division of Pharmacology, Anesthesiology and Therapeutics, Faculdade São Leopoldo Mandic, Instituto de Pesquisas São Leopoldo Mandic, Campinas, Brazil, 2 Pharmaceutical Science Graduate Course, University of Sorocaba, Sorocaba, Brazil, 3 Division of Paediatric Dentistry, Faculdade São Leopoldo Mandic, Instituto de Pesquisas São Leopoldo Mandic, Campinas, Brazil*

#### *Edited by:*

*Natalie Schellack, Sefako Makgatho Health Sciences University, South Africa*

#### *Reviewed by:*

*Elmien Bronkhorst, Sefako Makgatho Health Sciences University, South Africa Monique Coetzee, Mediclinic Southern Africa, South Africa*

#### *\*Correspondence:*

*Rogério Héladio Lopes Motta rogerio.motta@slmandic.edu.br*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 24 April 2019 Accepted: 08 July 2019 Published: 09 August 2019*

#### *Citation:*

*Andrade NKd, Motta RHL, Bergamaschi CdC, Oliveira LB, Guimarães CC, Araújo JdO and Lopes LC (2019) Bleeding Risk in Patients Using Oral Anticoagulants Undergoing Surgical Procedures in Dentistry: A Systematic Review and Meta-Analysis. Front. Pharmacol. 10:866. doi: 10.3389/fphar.2019.00866*

The management of patients who undergo dental surgical procedures and receive oral anticoagulant therapy requires particular attention due to the risk of bleeding that may occur during the procedure. Bleeding rates in these trans- or post-operative patients tend to be unpredictable. The aim of this study was to conduct a systematic review in order to assess the risk of bleeding during and after performing oral surgery in patients administered oral anticoagulants compared with a group that discontinued anticoagulant therapy. For the purposes of this review, we searched the databases of the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (*via* Ovid), EMBASE (*via* Ovid), and the Virtual Health Library (VHL) from inception of the database to December 2018. The primary outcome was defined as the occurrence of local bleeding during and after oral surgical procedures. Four reviewers, independently and in pairs, screened titles and abstracts for full-text eligibility. Data regarding participant characteristics, interventions, and design and outcomes of the included studies were extracted. The data were pooled using randomeffects meta-analyses and described as risk ratios (RRs) with a 95% confidence interval (95% CI). The confidence for the pooled estimates was ascertained through the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, and the protocol of this review was recorded in PROSPERO (CRD42017056986). A total of 58 eligible studies were identified, of which three randomized controlled trials were included in the meta-analysis, covering a total of 323 adult participants, among whom 167 were taking anticoagulants at the time they underwent dental surgery. Of these patients, 14.2% had reported bleeding. The risk of bleeding was found to be one to almost three times greater in patients taking warfarin compared with patients who discontinued the use of anticoagulant during the trans-operative period (RR = 1.67, 95% CI = 0.97 to 2.89) and in the post-operative period (RR = 1.44, 95% CI = 0.71 to 2.92), although the quality of evidence was very low. The results indicate that there is no evidence that the use of anticoagulants eliminates the risk of bleeding during surgical dental procedures.

Keywords: oral surgery, oral anticoagulant, bleeding, safety, systematic review, meta-analysis

### INTRODUCTION

The routine use of oral anticoagulants is related to hemostatic imbalance between clotting and blood anticoagulation, and significant variations in this relationship may increase the risk of hemorrhage or thromboembolism (Dahlback, 2000; Leiria et al., 2007; Barco et al., 2013; Harter et al., 2015; Yeh et al., 2015; Gerzson et al., 2016).

Anticoagulants can be classified according to their route of administration. Oral anticoagulants include vitamin K antagonists, such as warfarin, that inhibit factors II, VII, IX, and X of the coagulation cascade; new oral anticoagulants that directly inhibit thrombin (coagulation cascade factor II), such as dabigatran; and those that inhibit factor Xa, such as rivaroxaban. A commonly used parenteral anticoagulant is low molecular weight heparin (Barco et al., 2013; Lijfering and Tichellar, 2018).

The increasing use of these medications raises the probability of anticoagulant therapy in patients undergoing dental treatment. Thus, it is necessary to promote safe and preventive treatment in order to avoid complications and comorbidities in these patients (Thacil and Gagg, 2015).

According to the Guidelines of the American College of Chest Physicians (2012), the desirable value of the International Normative Ratio (INR) for patients taking oral anticoagulants is between 2 and 3. When this index indicates anomalies, treatment should be adjusted by the health professional in charge, in order to maintain the patient in a favorable health condition while promoting good short- and long-term prognoses.

Some studies have demonstrated that exodontias performed in patients with a recommendable INR range can be safely conducted without oral anticoagulant interruption or antiplatelet drugs (Campbell et al., 2000; Barrero et al., 2002; Aframian et al., 2007; Morimoto et al., 2008; Goodchild and Donaldson, 2009; Nematullah et al., 2009; Bajkin et al., 2015).

Although previous systematic reviews have assessed the risk of bleeding during oral surgical procedures in patients using oral anticoagulants (Dunn and Turpie, 2003; Madrid and Sanz, 2009; Nematullah et al., 2009; Diermen et al., 2013; Kämmerer et al., 2015; Yang et al., 2016; Shi et al., 2017), the safety of performing dental surgical procedures remains uncertain, on account of the various methodological discrepancies or inconsistencies in these studies. For example, the findings reported by Nematullah et al. (2009) and Madrid et al. (2009) were not restricted to dental surgeries, whereas Diermen et al. (2013) did not perform a metaanalysis of the results, and Kämmerer et al. (2015), Yang et al. (2016), and Shi et al. (2017) used observational studies instead of clinical trials and did not restrict INR values. Furthermore, none of these systematic reviews used GRADE (Grading of Recommendations Assessment, Development and Evaluation), a tool that is routinely used to evaluate the quality of a body of evidence and the strength of recommendations of the findings.

The present review aims to answer the following PICO question: "What is the risk of bleeding in patients who take oral anticoagulants and will undergo oral surgical procedures with or without antithrombotic therapy interruption?" Accordingly, by conducting this systematic review, we aim to determine the risk of bleeding during and after dental surgery in patients undergoing anticoagulant therapy, as well as providing guidance that will assist dental professionals in making informed clinical decisions.

### METHODS

### Protocol and Registration

This systematic review was conducted in accordance with the recommendations described in the Cochrane Handbook for Systematic Reviews of Interventions (Moher et al., 2009). The study was performed in accordance with the checklist of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (Moher et al., 2009; Higgins and Green, 2011). The protocol of this review has been recorded in PROSPERO (International Prospective Register of Systematic Reviews) (protocol CRD42017056986) (http://www.crd.york.ac.uk/ PROSPERO) (Motta et al., 2017).

## Eligibility Criteria

#### Study Selection Criteria

Studies considered eligible for inclusion were randomized controlled trials (RCT) involving adult volunteers of both genders, users of oral anticoagulants (vitamin K antagonists, factor Xa inhibitors, or direct thrombin inhibitors), and requiring dental surgeries such as exodontias and dental implants. Included studies were also required to have an experimental group containing participants who were submitted to dental surgeries without interruption of anticoagulant therapy and a control group containing participants who were submitted to dental surgeries with interruption of anticoagulant therapy.

#### Exclusion Criteria

Exclusion criteria included studies in which patients under combined anticoagulant therapy (oral anticoagulant associated with a platelet antiaggregant) represented more than 20% of the sample; studies in which the population was not clearly representative (e.g., patients presenting different bleeding risks due to recent episodes of stroke and recent ablation surgeries); and/or an INR value higher or lower than the desirable range between 2 and 3 in more than 20% of the sample.

#### Outcomes Assessed

Included studies should have reported one of the following outcomes: trans-operative or post-operative local bleeding measured at least 48 h after oral surgical intervention. The definition of bleeding was accepted as described in each study. Oral complications (infections, implant failures, and healing problems at the surgical site) were considered as secondary outcomes.

### Search for Primary Studies

#### Electronic Databases

For the purposes of the review, we performed searches for eligible studies in CENTRAL (Cochrane Central Register of Controlled Trials) part of The Cochrane Library and MEDLINE (*via* Ovid), and EMBASE (*via* Ovid) and VHL (Virtual Health Library) databases from the inception of the database until December 2018, without restrictions relating to language or year of publication. The management of the references (listing and removal of duplicates) was carried out using Endnote X8 software.

#### Other Search Resources

Two reviewers (NA and CB) performed a manual search by reading the reference lists of each selected study or citations found in secondary studies in order to verify potentially eligible studies.

#### Search Strategies

For search purposes, the terms describing the risk of bleeding and oral surgical procedure were combined. The search was conducted using MeSH (Medical Subject Headings) terms for each minor oral surgical procedure (oral surgery, exodontias, and dental implant installation), risk of hemorrhagic events and their synonyms, and several oral anticoagulants (vitamin K antagonists and new oral anticoagulants). The search strategy was adapted for each database and is described in **Supplementary Material A**.

### Study Selection

Four reviewers (NA and CB; LO and JA), working independently and in pairs (as indicated), selected potentially relevant titles and abstracts, to which the eligibility criteria were applied. The full texts of potentially eligible studies were obtained, and two reviewers (NA and RM) independently assessed the eligibility of each study. All disagreements were resolved through consensus.

### Data Extraction

Initially, four reviewers (RM and CG; LO and NA), working independently and in pairs, extracted the data from two articles, and a consensus was reached. Discordances were resolved by consensus, and contentious issues were discussed with a third reviewer (LL).

The same reviewers, independently and in pairs, extracted data and recorded information regarding patients, methods, interventions, outcomes, and absence of significant results. A data extraction sheet was used in accordance with the instruction manual prepared by the lead author of this review (NA).

### Assessment of the Risk of Bias

The Cochrane Collaboration tool was used to evaluate the risk of bias (Altman et al., 1990; Higgins and Green, 2011). All reviewers, independently and in pairs, assessed the risk of bias for each clinical trial according to randomization; allocation concealment; blinding of the patient, the healthcare professional, and the outcome assessors; reporting of incomplete outcomes; and selective reporting of outcomes and imbalance in baseline measurements of the sample.

For each domain, the reviewers assigned response options of "definitely yes," "probably yes," "probably not," or "definitely not," with "definitely yes" and "probably yes" ultimately indicating a low risk of bias and "definitely not" and "probably not" indicating a high risk of bias (Akl et al., 2013). In cases of disagreement, a consensus was reached through discussion or consultation with a third author (LL).

### Data Synthesis

Analysis was performed for each anticoagulant and outcome of interest. Confidence in the findings was determined by estimates for each body of evidence. For studies that reported dichotomous outcomes, the relative risk (RR) was calculated combined with a 95% confidence interval (95% CI).

Continuous outcomes were not considered. However, if available they would be described according to the information available in the published protocol of this review (Motta et al., 2017). Details regarding the methods adopted for data synthesis can be found in the same protocol. The random effects associated with meta-analysis were determined using STATA Software (version 10.1).

### Quality of Evidence

The quality of the evidence was independently assessed (confidence in estimates of the effect) for each outcome reported using the GRADE system (Guyatt et al., 2011a; Guyatt et al., 2011b). In this approach, randomized trials that were initially considered to have a high quality of evidence may have their quality status diminished according to assessment of one or more of the following five categories of limitation: risk of bias (assessed for each study as described previously), inconsistency, indirect evidence, imprecision, and publication bias (Guyatt et al., 2013a, Guyatt et al., 2013b).

The heterogeneity was also evaluated in association with the estimates of the effects using a χ2 test and the I2 statistic (Higgins and Thompson, 2002) and was classified as follows: 0% to 25% (low heterogeneity), 50% (moderate heterogeneity), and 75% (high heterogeneity) (Higgins and Thompson, 2002).

## RESULTS

### Literature Search Results

From our searches within the aforementioned four databases, we extracted 2,266 publications for consideration, of which 70 articles were duplicates. Accordingly, a total of 2,196 publications were deemed to be potentially eligible. A further 26 publications were identified through manual searches. Of these studies, 58 were considered to fulfil the eligibility criteria established for this review, three of which were included in the qualitative synthesis (Campbell et al., 2000; Evans et al., 2002; Al-Mubarak et al., 2007), and two (Evans et al., 2002; Al-Mubarak et al., 2007) were included in the meta-analysis (**Figure 1**).

### Synthesis Results

The characteristics of the three studies subjected to qualitative synthesis are shown in **Table 1**. These studies evaluated 348 users of oral anticoagulants, most of whom were using vitamin K antagonists. The studies, however, did not present information relating to the underlying diseases that justified the prescription of anticoagulants. All studies compared the risk of bleeding in patients undergoing oral surgical procedures with or without interruption (a few days prior to the dental procedure) of anticoagulant therapy but did not report other outcomes. The

list of excluded studies and the reasons for their exclusion are presented in **Supplementary Material B**. Local hemostatic measures, such as compression with gauze and sutures (Campbell et al., 2000; Evans et al., 2002; Al-Mubarak et al., 2007) and sutured oxidized cellulose sponges, were used in the surgical beds in order to control bleeding (Evans et al., 2002).

Two of the three studies provided details of patient follow-up during the first week of the post-operative periods. In the third


#### TABLE 1 | Characteristics of included studies.

*VKA, vitamin K antagonist; INR, International Normative Ratio.*

study by Campbell et al. (2000), the results indicated a 2-day follow-up time, although this was not stated explicitly in the text.

The study conducted by Evans et al. (2002) reported the concomitant use of other drugs during the research period, in which some patients received antibiotic prophylaxis. None of the studies received industrial funding. Although all three studies reported the primary outcome "local bleeding," none reported secondary outcomes.

Campbell et al. (2000) performed a qualitative evaluation of the bleeding associated with minor oral surgeries in patients with VKA (the study does not specify the anticoagulant). Bleeding was assessed in terms of "surgical sponge weight" in the pre- and post-operative periods. In this regard, it was considered that each gram of blood is equivalent to a volume of 1 ml, and accordingly the volume of blood lost in the pre- and post-operative periods was determined from the difference in sponge weight. Twentyfive patients were allocated to one of two groups (mean INR of 2.0 ± 0.5): the "experimental group" (*n* = 12) in which patients' use of anticoagulants before and after surgical procedures was maintained, and the "control group" (*n* = 13) in which anticoagulant use was interrupted 72 to 96 h before the surgical intervention. None of the patients presented trans- or post-operative bleeding requiring any therapeutic intervention, and there was no significant difference in blood loss between the groups. Although the authors have suggested the need for additional investigations, they also consider that it would be possible to perform minor oral surgical procedures in patients taking oral anticoagulants without additional pharmacological interventions.

Evans et al. (2002) evaluated the risk of bleeding in 109 individuals taking warfarin, who underwent simple and multiple exodontia. Bleeding was assessed by the use of gauze, which the patient was instructed to bite on for a period of 10 min. If the hemostatic measures were not sufficient to contain the bleeding, the event was considered as "immediate bleeding." Patients were randomly assigned to one of the two study groups by allocation concealment (57 to the experimental group and 52 to the control group). Individuals in the experimental group continued using warfarin, whereas those in the control group interrupted the use of warfarin 2 days prior to the surgical procedure. All surgical beds were covered with oxidized cellulose sponges, and the patients were sutured. The patients were instructed to bite a gauze pad for 10 min. If such local hemostatic measures were not sufficient to contain bleeding, it was considered "immediate bleeding." Twenty-two patients presented complications related to bleeding (post-surgery only), 15 (26%) of whom were in the experimental group and seven (13.5%) (*n* = 7) were in the control group. The study thus indicates the possible heightened risk of complications associated with bleeding when warfarin is maintained. In most cases, however, these events can be controlled by administering local treatment.

Al-Mubarak et al. (2007) evaluated the incidence of postoperative hemorrhage in patients taking warfarin (dose of 2 to 10 mg daily) undergoing exodontia. The bleeding was assessed by the use of gauze on which the patient was instructed to bite for a period of 6 to 10 min. If these hemostatic measures were not sufficient to contain the bleeding, the event was considered as bleeding. Additionally, the volunteers were followed for up to 7 days in order to verify the presence of hemorrhagic sites (evaluated in terms clot formation and the local repair process). A total of 214 patients were randomly divided into four groups. Patients in groups 1 and 3 interrupted warfarin intake (2 days before the surgical procedure): group 1 (*n* = 48, mean INR 1.8 and no suture) and group 3 (*n* = 56, mean INR 1.9 and with suture). Patients in groups 2 and 4 maintained warfarin use: group 2 (*n* = 58, mean INR 2.4 and without suture) and group 4 (*n* = 52, mean INR 2.7 and with suture). All patients received gauze compression for 6 to 10 min and were followed for up to 7 days. It was observed that among the patients in groups 1 and 2, 12% (*n* = 6) and 21% (*n* = 12) presented trans-operative hemorrhagic events, respectively. During the postoperative period, 4% (*n* = 2) of the patients in group 1 and 3% (*n* = 2) in group 2 presented bleeding.

### Risk of Bias Assessment

As shown in **Figures 2** and **3**, the three included studies had a high risk bias. However, the three studies did not provide sufficient data regarding the randomization process to enable an evaluation of potential selection bias (Campbell et al., 2000; Al-Mubarak et al., 2007). Evans et al. (2002), however, describe the generation of random sequences of patients in the groups.

In the studies reported by Campbell et al. (2000) and Al-Mubarak et al. (2007), concealment of the allocation was not guaranteed, and none of the studies provided sufficient information in order for us to determine whether there was blinding of the patients or of those involved in the research, which may have caused detection bias, as the outcomes evaluated are likely to be influenced by the absence of blinding. Only Al-Mubarak et al. (2007) provided an indication that there was blinding of the professionals who evaluated the outcomes, although they did not clearly describe how the procedure was performed.

Furthermore, none of the studies clarified whether there was follow-up loss, although all included studies described the evaluated outcomes. However, apparently, all patients enrolled in the studies initiated and completed the proposed treatments.

The incidence of thromboembolic events among patients who interrupted anticoagulant therapy was not assessed in clinical trials. Other secondary outcomes, such as bruising, ecchymoses, or post-operative infections, may have been considered by the studies; however, these were not reported. None of the studies presented the protocol record, which would have enabled us to infer the risk of uncertain bias.

None of the included studies obtained industrial funding, and based on a reading of the studies, no other problems were identified in addition to those already mentioned; thus, we assume that the studies were free of other potential sources of bias.

### Results of Evaluated Outcome and Quality of Evidence

Random-effects meta-analysis revealed no statistically significant difference between the groups that continued or interrupted the use of anticoagulants. Nevertheless, the results have demonstrated a one to almost three times greater bleeding risk in patients taking warfarin compared with patients who discontinued the use of anticoagulant in the trans-operative (RR = 1.67, 95% CI = 0.97 to 2.89) and post-operative (RR = 1.44, 95% CI = 0.71 to 2.92) periods (**Figure 4**). Sub-group analyses were not possible due to the low number of studies included.

The quality of evidence according to GRADE (**Table 2**) concerning "bleeding risk" (the main outcome) in both trans- and post-operative periods was considered very low, which indicates a very small confidence in the estimated effect. Furthermore, we also noted that the studies were associated with a high risk of bias, and thus the relative estimate of results may not be reliable. We did, however, observe an overlap in the confidence intervals of the studies, and therefore, the inconsistency was considered


unimportant. Moreover, low heterogeneity was observed between the results.

The included studies presented details of interventions and outcomes of interest, and therefore the evidence was considered direct. Considerable imprecision was noted with regard to the number of total events and the small number of samples, in addition to the wide confidence intervals with no true effect.

Although each of the included studies reported few bleeding events, the lack of any details on other meaningful outcomes, including the incidence of thromboembolism or local complications, limits the scope of the clinical decision making, such as the interruption or maintenance of antithrombotic therapy prior to dental surgical procedures. The absence of such details may suggest a publication bias.

### DISCUSSION

Excessive trans-or post-operative bleeding is a negative outcome in dentistry that limits several surgical interventions, and in this regard, an evaluation of the risk of bleeding in users of oral anticoagulants may contribute to minimizing the complications experienced by dental patients.

In this study, we evaluated the risk of bleeding in patients using oral anticoagulants who underwent dental surgical procedures. For the purposes of the descriptive analysis, we performed a detailed assessment of three relevant studies (Campbell et al., 2000; Evans et al., 2002; Al-Mubarak et al., 2007), two of which were included in the meta-analysis (Evans et al., 2002; Al-Mubarak et al., 2007).

The assessments undertaken in the present review are restricted to warfarin, for which no recent RCTs have been published. Although our meta-analysis indicates an increased risk of bleeding when patients maintain the use of this anticoagulant, the findings of the assessed studies present very poor-quality evidence, which represents a constraint in terms of making any definite recommendations.

It is noteworthy, however, that apart from bleeding, no other important relevant outcomes, such as ecchymoses and thromboembolism, were reported in the trials included in this review and therefore could not be evaluated. Further, we were unable to perform sub-group analyses, on account of the small number of studies included. Given that the frequency of thromboembolic events is an important outcome for assessing safe antithrombotic therapy interruption, the absence of any report on this outcome in clinical trials may suggest bias due to the selective reporting of outcomes.

According to the guidelines of the American College of Chest Physicians Holbrook et al. (2012), a safe INR for dental interventions ranges from 2 to 3. This INR range has not always been considered as an eligibility criterion in previous studies, as comparisons are difficult and a larger amplitude of INR becomes an inconclusive factor, particularly for those patients who are users of vitamin K antagonists. In the present study we used patient INR values ranging from 2 to 3 as an inclusion criterion for clinical trials, which necessarily limited the number of studies we were able to select for the review. In addition to the INR interval adopted by the studies, other disparities were observed in the systematic reviews previously published on this subject (Dunn and Turpie, 2003; Madrid and Sanz, 2009; Diermen et al., 2013; Kämmerer et al., 2015; Yang et al., 2016; Shi et al., 2017), which accordingly highlight the relevance of the present study.

In a systematic review conducted by Dunn and Turpie (2003), the authors identified a low incidence of thromboembolic events in patients who had discontinued anticoagulant therapy (1.6% of patients). However, this finding was based on a consideration of both medical and oral procedures. Moreover, the authors of this review did not assess the risk of bias or the quality of the evidence of the findings.

Some systematic reviews have indicated that there is no difference in the risk of bleeding among patients who interrupt and those who maintain anticoagulant therapy prior to undergoing surgical procedures (Madri and Sanz, 2009; Yang et al., 2016), which corroborates the findings of the present study. However, the findings of Kämmerer et al. (2015) and Shi et al. (2017) have indicated an increased risk of bleeding in patients who maintain anticoagulant therapy during the period in which they undergo dental procedures. Nevertheless, they concluded that such complications can be


readily treated using local hemostatic measures. However, none of these studies included only RCTs, and none took INR values into account.

Kämmerer et al. (2015) emphasized that the risk of potentially fatal thromboembolism due to interruption of anticoagulant therapy outweighs the risk of post-operative bleeding episodes. The authors suggest that minor procedures, such as exodontias and dental implants, can be safely performed if the INR is within the therapeutic range and local hemostatic measures are used.

Diermen et al. (2013) evaluated the level of evidence and grade of recommendation (regardless of study design and including clinical practice guidelines) of studies that examined the risk of discontinuing treatment with antiplatelet agents and anticoagulants prior to oral surgical procedures. The results indicated that antithrombotic therapy should not be interrupted for simple dental procedures.

Although the review performed by Yang et al. (2016) also included the studies of Evans et al. (2002) and Al-Mubarak et al. (2007), it was essentially a qualitative analysis and considered other study designs in addition to RCTs. The authors suggested that patients who maintain oral anticoagulant therapy are not at any greater risk of bleeding after dental extractions than are patients who have discontinued oral anticoagulant therapy.

### Strengths and Limitations of This Study

The present study was carried out with methodological accuracy, including an evaluation of the risk of bias and an assessment of the quality of evidence, which have not featured in the previously published systematic reviews of this topic (Dunn and Turpie, 2003; Madrid and Sanz, 2009; Nematullah et al., 2009; Diermen et al., 2013; Kämmerer et al., 2015; Yang et al., 2016; Shi et al., 2017). Thus, we have explicitly highlighted the eligibility criteria, the comprehensive database search, and the independent and paired evaluation of each study. Moreover, the use of GRADE made it possible to evaluate the strength and the quality of the body of evidence in relation to the effect of bleeding risk.

The specific criteria used for the selection of included studies do, however, represent a limiting factor for the findings of this review, owing to our requirement regarding the methodological quality of the RCT. This, nevertheless, highlights the necessity for a greater number of primary studies on this subject, with greater methodological accuracy in order to increase the reliability of the findings.

### Implications for Clinical Practice and Future Research

Our findings have revealed that there is no statistical difference in the risk of bleeding in warfarin users undergoing dental surgery without anticoagulant therapy interruption compared with those who discontinue the therapy. However, due to the poor quality evidence, the most appropriate practice with respect to whether oral anticoagulant therapy should be interrupted remains uncertain. Meticulous methodological quality should be encouraged with regard to answering the question posed at the outset, in addition to emphasizing the use of INR as a standardized criterion for future research.

On the basis of our findings, we recommend dental surgical planning based on a diagnosis of the general health condition of the patient. This diagnosis implies the accomplishment of a comprehensive anamnesis, including the diseases involved and use of the drugs related to blood hemostasis. This evaluation should additionally include the request for a complete blood exam and determine whether the patient is decompensated or presents comorbidities that contribute to a greater risk associated with the surgical procedures. Moreover, it is important that they be referred to the doctor in charge for a more specific opinion and classification of risk.

To date there have been no RCTs that have evaluated the outcomes of interest of this research in relation to new oral anticoagulants. Given this scenario, new studies with variable control and the use of standardized methods for determining outcomes should be developed.

### CONCLUSION

The findings of this analysis indicate that there is no evidence of a greater risk of bleeding in patients using oral anticoagulants who undergo surgical dental procedures. However, the findings should be interpreted with caution and new studies on the subject should be initiated.

### REFERENCES


### AUTHOR CONTRIBUTIONS

NA is the main investigator and led the development and the writing of the manuscript. LL, RM, and CB are the project managers and co-investigators, who contributed to the development, writing, and revision of the manuscript. LO, CG, and JA are co-investigators and contributed to the development and revision of the manuscript. All authors have read and approved the final manuscript.

### ACKNOWLEDGMENTS

We thank the researchers Professors Marcus Tolentino Silva and Lauren Giustti Mazzei for their support in the data analysis.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00866/ full#supplementary-material


Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest. 141 (2 Suppl), e152S–e184S. doi: 10.1378/chest.11-2295


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Andrade, Motta, Bergamaschi, Oliveira, Guimarães, Araújo and Lopes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Situation, Impacts, and Future Challenges of Tobacco Control Policies for Youth: An Explorative Systematic Policy Review

#### *Edited by:*

*Cristiane De Cássia Bergamaschi, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Juman Abdulelah Dujaili, Monash University Malaysia, Malaysia Juan Carlos Sanchez-Salgado, Institute of Molecular Medicine and Advanced Sciences, Mexico Robert L Lins, Independent Researcher, Antwerp, Belgium HyunSook Lee, Kongju National University, South Korea*

#### *\*Correspondence:*

*Chhabi Lal Ranabhat chhabir@gmail.com Chun-Bae Kim kimcb@yonsei.ac.kr*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 20 November 2018 Accepted: 31 July 2019 Published: 09 September 2019*

#### *Citation:*

*Ranabhat CL, Kim C-B, Park MB and Jakovljevic M(M) (2019) Situation, Impacts, and Future Challenges of Tobacco Control Policies for Youth: An Explorative Systematic Policy Review. Front. Pharmacol. 10:981. doi: 10.3389/fphar.2019.00981*

*Chhabi Lal Ranabhat1,2,3,4\*, Chun-Bae Kim2,5\*, Myung Bae Park4 and Mihajlo (Michael) Jakovljevic6,7*

*1 Policy Research Institute, Kathmandu, Nepal, 2 Institute for Poverty Alleviation and International Development, Yonsei University, Wonju, South Korea, 3 Manmohan Memorial Institute of Health Science, Kathmandu, Nepal, 4 Department of Gerontology, Pai Chai University, Daejeon, South Korea, 5 Department of Preventive Medicine, Yonsei University Wonju College of Medicine, Wonju, South Korea, 6 Department of Global Health Economics and Policy, University of Kragujevac, Kragujevac, Serbia, 7 Division of Health Economics, Lund University, Lund, Sweden*

Background: Tobacco use in youths is a major public health challenge globally, and approaches to the challenge have not been sufficiently addressed. The existing policies for tobacco control are not well specified by age.

Objective: Our study aims to systematically investigate existing tobacco control policies, potential impacts, and national and international challenges to control tobacco use targeting the youth.

Data sources: We used the statistics of the Global Youth Tobacco Survey (GYTS), studies, and approaches of tobacco control policies targeting youth. Considering country, continent, age, and significance, PubMed, Health Inter-Network Access to Research Initiative (HINARI), Scopus, the Cochrane Library, Google, and Google Scholar were searched. The related keywords were tobacco control, youth, smoking, smoking reduction policies, prevalence of tobacco use in youth, classification of tobacco control policies, incentives to prevent young people from using tobacco, WHO Framework Convention on Tobacco Control (FTCT), etc. The search strategy was by timeline, specific and popular policies, reliability, significance, and applicability.

Results: We found 122 studies related to this topic. There were 25 studies focusing on situation, significance, and theoretical aspects of tobacco control policies associated with youth; 41 studies on national population polices and challenges; and 7 studies for global challenges to overcome the youth tobacco epidemic. All national policies have been guided by WHO-MPOWER strategies. Increases in tobacco tax, warning signs on packaging, restriction of tobacco product advertisements, national law to discourage young people, and peer-based approaches to quit tobacco are popular policies. Smuggling of tobacco products by youth and ignorance of smokeless tobacco control approach are major challenges.

**141**

Limitation: Our study was flexible for the standard age of youth and we were not able to include all countries in the world and most of the studies focused on smoking control rather than all smokeless tobaccos.

Conclusion: The policies of tobacco control adopted by many countries are based on the WHO Framework Convention on Tobacco Control but not necessarily focused on youth. Due to the physical and economic burden of tobacco consumption by youth, this is a high priority that needs to be addressed. Youth-focused creative policies are necessary, and more priority must be given to tobacco prevention in youth. Tobacco control should be a social, public health, and quality-of-life concern rather than a business and trade issue.

Implication of key findings: There is limited research on how and in what ways tobacco control policies reach young people and their engagement with these policies from physical, physiological, and psychological aspects. Analysis of these aspects, popular polices practiced in different countries, and creative strategies support the need to review current practices and future ways to discourage youth from tobacco use.

Keywords: tobacco control, smoking, policies, youth, preventive measure, tobacco products

### INTRODUCTION

Tobacco consumption is a major challenge for the 21st century because tobacco-related deaths are increasing, destroying the young generation and promoting an environmental threat. Globally, tobacco has killed 100 million people in the 20th century, much more than all deaths in World Wars I and II combined, and tobacco-related deaths will number around 1 billion in the 21st century if current tobacco use patterns continue (Eriksen et al., 2015). Of the 100 million projected tobacco-related deaths over the next 20 years, about half will be of people in the productive ages of 35–69 (Centers for Disease Control and Prevention, 2000). Regardless of many national bans on tobacco sales to minors, approximately 25% of people under 18 years old are using tobacco and 12.6% are using more than two types of tobacco products (Arrazola et al., 2014). There are multiple impacts (economic, health, social, family, and peer groups) of tobacco use in youth because they are losing high amount of money, as the tax on tobacco increases year by year, risk factors for many disease, vulnerability to alcohol use and drugs, and copying his/her tobacco use by juniors in schools and sibling in family put them at risk for tobacco use.

There are a variety of programs and policies for tobacco control, but policy analyses on age-specific tobacco control are very rare. It is important because the resources, efforts, and approaches to quitting tobacco for people 60 years of age and those 16 years age do not have a similar impact. A study pointed out that there is a need for a comprehensive multifaceted approach to tobacco control policies for youth (Grimsaw and Stanton, 2017; Cancer Council., 2017a). There is a need to observe that as a foundation and productive age group, youth should be a high priority because interventions would be cost-effective and more productive to family and the nation. Moreover, approaches to controlling tobacco in youth are easy in comparison with late adults and the elderly because nicotine addiction in the late stage is difficult to overcome. From the point of its effectiveness, preventive and nominal remedial approaches are sufficient for youth. In contrast, more resources and complex medical approaches are necessary in late-stage addiction. Likewise, different kinds of medical risk can be prevented in the early stage (youth) but are hardly possible in the late stage. Previous studies, research, policies, and programs are not clearly distinctly age-specific tobacco control approaches, and in our study, we explore the different dynamics of tobacco control policies focusing on youth.

### Prevalence of Tobacco Use Among Youth

Tobacco use among youth remains a major public health concern worldwide. Globally, there are about 1.2 billion smokers, of whom more than 50% are young people; the prevalence varies by region, country, and gender (Khuder et al., 2008; Lim et al., 2010; Al-Sadat et al., 2010). By gender, smoking among boys (16%) is almost three times than that among girls (6%) globally. In the West Pacific, the prevalence of smoking among boys (18%) is four times than that among girls (4%), whereas in the United States and Europe, the gap between boys and girls is less than double. Smokeless tobacco is also gaining popularity and is currently used globally by 8% (6% in boys and 2% in girls). The highest proportion of girls using smokeless tobacco (17%) is found in the West Pacific and the lowest (2%) is found in Europe (World Health Organization., 2012). India, with 327 million adolescents, has one of the youngest populations in the world; the number of adolescents using tobacco is approximately 21% of the country's population (Vidhubala

**Abbreviations:** CDC, Centers for Disease Control and Prevention; FCTC, Framework Convention on Tobacco Control; GYTS, Global Youth Tobacco Survey; NRF, National Research Foundation; STM, Sales to minor; WHO, World Health Organization; IARC, International Agency for Research on Cancer; BAT, British American Tobacco.

Ranabhat et al. Tobacco Control Policies for Youth

et al., 2014). Since 1980, large reductions in the estimated prevalence of daily smoking have been observed at the global level for both boys and girls (e.g., in the United States, about 10% from 1980 to 1990 of 12th-grade students), (Nelson et al., 2008) but because of population growth, the gross number of smokers has increased significantly (Ng et al., 2014). According to the Global Youth Tobacco Survey (GYTS) 2011, the top three countries for tobacco consumption rates were Papua New Guinea (43.8%), Chile (31.5%), and Lithuania (30.8%), while Cambodia consumption rate was the lowest (0.2%) (World Health Organization, 2011). Prevalence of youth tobacco use reduces life expectancy (Ranabhat et al., 2018; Ranabhat et al., 2019). Socially, youth from disadvantaged groups are more vulnerable to smoking because of their social context (Hefler and Chapman, 2014). Despite the variation in statistics between different classifications of youth, youth smoking is a major threat in every aspect.

### Understanding Youth

Youth is a critical and foundation period of human life; however, there is no consistent definition. Youth is the time of life when one is young, but often means the time between childhood and adulthood (maturity) (Walker et al., 2013). Around the world, the English terms youth, adolescent, teenager, kid, and young person are interchanged, often meaning the same thing (Konopka, 1973). The United Nations has defined the contextual definition of youth: age between 15 and 24 by UN secretaries, UNESCO, and ILO; age of 15–32 by UN habitat; age of 10–24 by UNFPA and WHO; child until 18 by UNICEF; and age between 15 and 35 by the African youth charter (United Nation., 2016). Beyond this, different countries have defined the youth in their context, and we have used the term youth as a wide concept as used by different scholars and organization. The focus of this study is on tobacco control policies applicable for youth despite the age variation.

### Youth Period and Risk for Tobacco Use/Smoking

There are theoretical and empirical studies about the risk of youth tobacco use/smoking. Youth may initiate smoking by i) social learning theory; youth are eager to try something new, i.e., attention, retention, motor reproduction, and reproduction and motivation process; (Bandura and McClelland, 1977) ii) psychological development theory; they decide whether they should use or not, i.e., age, self-control, academic achievement, growth trend, and distal structure (parents or peer); (Jessor and Jessor, 1977) iii) behavior theory; the attitude of acceptance and to continue or not, i.e., person's behavior is a function of behavioral intention, which is determined by attitude toward the act; (Ajzen and Fishbein, 1970), and iv) self-conceiving theory; every activity of human is determined by self-conception (Rosenberg, 1986). Blum has explored different aspects of human emotions, motivation, and perceptions, particularly during the period of youth (Blum, 2009). Due to their age, physiological changes, and family and social environments, youth often perpetuate smoking practices, and they ultimately become addicted. Indeed, youth tobacco use is a primary source of substance use and other social deviations. High school male students who have smokers in family and school, smoker friends, media and advertisement influence, and easy access to the purchase of cigarettes are factors to youth smoking; smokers often lack self-control even though they know smoking is harmful (Simons-Morton et al., 1999; Alexander et al., 2001; Ertas, 2007).

### Key Questions

KQ1—What is the current situation of youth tobacco use in terms of prevalence and control policy pattern?

KQ2—What are the major and popular policies to control tobacco in the national and global context targeting youth?

KQ3—What could be the best creative policies for tobacco control for youth?

### Objective of the Study

The aim of this study is to systematically investigate the situation regarding youth tobacco use, existing policies, effectiveness, and challenges in the national and global context.

## METHODS

### Eligibility Criteria

We focused precisely on tobacco control policies, targeting youth, policies adopted by countries, challenges to implementing those policies, publications in English, and availability of latest prevalence data.

Studies were included based on the following PICOS strategies.

	- a. Disconnect youth from tobacco
	- b. Smoking/tobacco use cessation

### Exclusion:


The details of excluded data with reason are available in the PRISMA flowchart (**Figure 1**).

### Data Searching Strategy

Literature search strategies were developed using Medical Subjects Headings (MeSH) terms and keywords. Different health and social science search engines and databases were used (PubMed, Scopus, the Cochrane Library, HINARI, Google, and Google Scholar) to find sources regarding global and national policies mostly focusing on youth tobacco use. We used single, double, or multiple MeSH terms, free text, and specific terms under a subheading to identify relevant studies from the online data sources. Search strategy also included content, synonyms, year, and country names (**Box 1**). We downloaded and analyzed relevant journal articles, books, survey results, analytical views related to the WHO FCTC, and unpublished reports.

### Data Source Management

We, all authors, established the inclusion and exclusion criteria and data search strategies. CR and MBP searched all data and CBK and MJ verified those data. There were some data that were ambiguous and less relevant to our study but we decided to include some in our study.

### Ways of Screening and Selection of Data Sources

The title of the study was screened by search engines using major keywords and titles. Many times, search engines were used to find the appropriate titles. After that, the titles were selected. In a second step, abstract and full-length studies were selected by database search engines. In the third stage, full-length articles were screened based on inclusion criteria. Studies were assessed using criteria developed, for example, effective for public health implications, representativeness of study samples, comparability, credibility of data collection tools, and attributability to the intervention. An additional criterion of "generalizability" assessed whether findings were likely to be transferable at a global, regional, or national level. Particular attention was paid to internal and external validity; important quality and validity issues are discussed alongside study results.

### Quality Assessment

For quality assessment of resources, we used the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for

#### BOX 1 | Electronic data search strategy.

#### *Terminology used for search*

Tobacco control policies, Smoking reduction policy, Youth and smoking, Tobacco use by adolescents, Comparative studies of tobacco use by boys and girls, Systemic review of tobacco control policies, Classification of tobacco control policies, Incentives on preventing and cessation of smoking, Legal provision for tobacco control, Effectiveness of WHO FCTC, Interventions for tobacco control on youth, MPOWER strategies, tobacco control policies by country continent, age gender, Effectiveness of MPOWER etc.

*PubMed Search Options*


*Scopus search options* 


• Source type: Journal, Book Series and Conference preceding *Cochrane library*


*Google Scholar*


*Health Inter Network Access to Research Initiative (HINARI)* 


*Google* 


Intervention (Higgins and Green, 2011) as reference. All research studies applied in this paper were searched and screened by two authors (Chhabi Ranabhat and Myung-Bae Park), with any disagreement resolved by consensus or arbitration of authors during meetings. For the risk of bias, we considered only selection bias and data collection method.

### RESULTS

### Search Outcome

The cumulative total records found in our search screen were 2,836 from PubMed (*n* = 668) Google Scholar (*n* = 760), Google (*n* = 1,085), Health Inter-Network Access to Research Initiative (HINARI *n* = 118), Cochrane library (*n* = 150), and Bibliographic Search (*n* = 55). The total excluded 2677 records in the first stage: duplicates (*n* = 657) and mismatching keywords in titles in the first stage (*n* = 2020) were removed. From record screening, we found *n* = 159 full text sources, and out of 159 full-length articles, we excluded 37 articles due to age not matching, scope is very limited, joint study with drinking, study with a small sample size, and more links to other social policy, and we finally included 122 articles in our full study (see the PRISMA flowchart).

### Study Characteristics

**Table 1** shows the general characteristics of the study. Almost fifty percent (44.94%, *n* = 89) of our retrieved studies were about youth-targeted tobacco control in youth and most of the data (87.60%, *n* = 121) are from 2005 to 2019. We set the time frame for the study because before 2005, there was a situation of tobacco epidemic and individual countries were doing their own efforts on controlling tobacco use after 2005 (actually after signature on WHO FCTC by different countries). Regarding national policies from different continents, more than 1/3 (41.46%) are from Asian countries.

### Analysis on Risk of Bias

Risk of bias analysis is not perfectly attractive with our study. We did not analyze the attrition bias because our study is a comparison of interventional studies and attrition bias occurs for randomized control trials. Moreover, we had very low influence of selection bias because all the interventions in this study were based on WHO MPOWER strategies. Studies by country and continent are not in equal proportion. We found that there were few studies in Africa, and more studies were available for Asia, but the creative policies we presented have no geographical boundaries. The number of studies for Asian countries is more than that from other continents because Asia has more than 2/3 of the world population and there are

TABLE 1 | General characteristics of included studies based on key questions, time interval, and region.


a higher number of countries compared to other continents. Most importantly, we focused on the policies related to youth and policies applicable to all age groups; gender and location were not prioritized. We applied almost all national representative survey results and global studies. There were no significant issues on data collection from the aspect on risk of bias.

### Synthesis of Result

We synthesized the results *via* a descriptive approach in three perspectives. The first one is youth-targeted current pattern of tobacco control policies and their impact. We have divided the policy pattern into two categories: i) protect the youth form tobacco use and ii) help the youth quit tobacco use. Likewise, we compared the effectiveness and challenges of tobacco control in different countries. Other important aspects are global challenges and ways on controlling tobacco use among youth.

### Patterns of Youth Tobacco Control Policies

Over the past two decades, a number of tobacco control policies have been implemented to prevent smoking initiation and encourage cessation among adolescents. To expand the fight against the tobacco epidemic, WHO introduced MPOWER, an initiative that includes six strategies: monitor tobacco use and prevention policies; protect people from tobacco smoke; offer help to people who want to quit using tobacco; warn about the dangers of tobacco; enforce bans on tobacco advertising, promotion, and sponsorship; and raise taxes on tobacco (World Health Organization, 2008). The effectiveness of MPOWER on youth is distinct. The International Agency for Research on Cancer (IARC) found that after increasing the tax and price of tobacco, there was significant reduction in tobacco use because youth are price sensitive and have a limited amount of resources (World Health Organization, 2011). Similarly, the smoke-free air law has a protective effect on young people and reduced smoking prevalence among boys of high socioeconomic status (Tauras et al., 2013). Most youth are influenced by tobacco advertisements, and according to the WHO, 24 countries have implemented a complete ban on direct and indirect tobacco advertising, promotion and sponsorship (TAPS) (World Health Organization, 2013). A study in 19 developing countries showed that there was a positive correlation between smoking and exposure to advertising (Kostova and Blecher, 2013). There were no appropriate research examining the effectiveness of offering to quit tobacco/smoking and warning of the harmful effects of tobacco applicable for youth. Therefore, in high-, (Dupont and Ward, 2002) low-, and middle-income countries, (Joseph, 2010) raising the price of tobacco and the tax on tobacco is more effective in reducing youth tobacco use than other strategies. The US Centers for Disease Control and Prevention (CDC) identified seven basic principles for tobacco control; three of these policies are relevant to youth: reducing tobacco use among adolescents; reducing the initiation of tobacco use among children, adolescents, and young adults; and increasing smoking cessation attempts among adolescent smokers (Centers for Disease Control Prevention (CDC)., 2007). A tobacco control plan in the United Kingdom has implemented three strategies to protect youth from tobacco use: reducing tobacco consumption, supporting parents and youths, and reinforcing the benefits of clean air spaces (National Health Service., 2015). Policy researchers have classified tobacco control initiatives into two groups: those intending to prevent first-time tobacco users (adolescents) and those aiming for cessation of tobacco use among current users, including both occasional and frequent users. These initiatives are based on multipronged approach, described below.

### Disconnect Youth From Tobacco and Use Advertising Campaigns and Laws to Demotivate Youth From Tobacco Consumption

Popular policies within this guideline include cigarette tax increases, smoke-free air laws, and youth access laws (laws on sales to minors and laws against youth possession, use, and purchase) (Warner et al., 2003). Ross and Chaloupka highlighted that in the United States, higher cigarette prices reduced the probability of youth smoking, and the teen-specific perceived price of cigarettes had a negative effect on demand (Ross and Chaloupka, 2003). Similar findings have been reported by other researchers, and some have found that increased price decreased current smoking prevalence and the number of cigarettes smoked per day among youth and young adult smokers (Wasserman et al., 1991; Tauras, 2004; Levy et al., 2004). Other approaches to discourage adolescent smoking are restrictions on smoking at home, more extensive bans on smoking in public places, and enforced bans on smoking at school (Wakefield et al., 2000; Farkas et al., 2000). Sales to minors (STM) laws, which penalize merchants and retailers for selling tobacco to youth, and possession, use, and purchase laws, which punish youth themselves for possessing, using, or purchasing tobacco products, have also been applied (Rabin and Sugarman, 2001). Some studies have revealed an association between youth access and STM laws, but a sustained relationship between those laws and decreased youth smoking prevalence has been questioned. A study from Bangladesh shows that youth are more vulnerable due to a tobacco-friendly environment in school (Kabir et al., 2013). When the price increase policy was endorsed in the United States, it had a mixed impact, but a systematic review revealed no significant change in youth smoking. This law was also controversial, and cases were brought to court (Jason et al., 2005). Raising taxes on tobacco and STM laws have thus shown mixed effects, along with legal restrictions on adolescent smoking in the United States.

### Increasing Smoking Cessation

Tobacco cessation, especially smoking cessation, depends on patterns of smoking such as age, peer pressure, influences of electronic media, and effective counseling to users. Among youth, quitting smoking has had no special program, and only 2–8% of youth smokers have attempted to quit in the Grimshaw et al., 2003 study. Hodder et al. suggested that universal schoolbased interventions could be an effective way to get youth to quit using tobacco and alcohol (Hodder et al., 2014).

In family and school, both approaches can be equally useful because some family members, teachers, and seniors need to quit smoking and pupils at risk must be disconnected from the first puff of smoking. Preventing youth from starting tobacco use is more effective and costs less than helping users quit, but no proper comparative studies have examined the different patterns and effects of youth tobacco control policies.

### Major National Tobacco Control Policies Intervention, Outcomes, and Challenges

**Table 2** shows the national tobacco control policies targeting youth. In 26 studies, researchers explored the overview and impact of youth-related tobacco control policies from China, India, Nepal, Thailand, Japan, South Korea, Namibia, Chad, Seychelles, Mauritius, Niger, Eritrea, Madagascar, South Africa, North Africa, France, United Kingdom, Australia, Uruguay, Panama, Colombia, Guatemala, Brazil, United States, and Canada. Likewise, in 15 studies, the challenges to control tobacco use in those countries were investigated. Seven studies were used as representative of global situations and challenges.

The direction of current policies has shifted significantly after WHO FCTC in tobacco control movement. The average strength of policies adopted varies significantly by country efforts. Cultural diversity leads to different pros and cons of tobacco control initiatives to any age, gender, geography, etc., and these subtleties must be taken into account when forming policies, and the FCTC was implemented on a national and local level.

A report of the US Surgeon General in 2012 showed that the global youth tobacco use rate was decreasing satisfactorily; the rate was 28% in 2000 and had declined to 8% by 2013 (Health UDo, Services H., 2012), but that survey did not match the GYTS of 2011. WHO FCTC has a comprehensive and global impact, specific to reducing the burden of youth tobacco use, but countries are reluctant to take proactive roles and prepare specific strategies due to lack of strong monitoring by WHO (Warner and Tam, 2012). Nevertheless, taxation, clean indoor air policies, and warning labels are ranked as the highest priorities but are general approaches, and there is still a need for more specific guidelines, integrated approaches, and age-specific tobacco control initiatives.

Tobacco companies around the world are focusing on strategic plans to block the implementation of Article 11 of the WHO FCTC, which sets guidelines on packaging and labeling of tobacco products (Sebrié et al., 2010) because youth are their target group as after they have become addicted, youth become regular customers. Tobacco companies have the power and money to influence legislators (Patel et al., 2007) and challenge the government's legislative powers through litigation (Holden and Lee, 2009). The British American Tobacco (BAT) company's former quality controller appealed against the ban on public smoking in Uganda; however, it was not successful. The tobacco industry in Kenya went to court to challenge the Tobacco Control Act of 2007, which created challenges for public smoking ban in public places (Tumwine, 2011). Similar cases have also been seen in the US high courts without decisions, representing a challenge to implement the WHO FCTC. Tobacco companies have litigated against new cigarette labeling policies in Uruguay, Brazil, and Paraguay to stop or delay the implementation of pictorial warnings (Sebrié et al., 2010) Thus, tobacco control stakeholders and tobacco production, manufacturing, and distribution companies need to continue to fight. It shows that tobacco companies may be a negative influence on youthfriendly tobacco control policies in the future.

### DISCUSSION

The WHO FCTC sought to substantially reduce and sustain reductions in indoor smoking and had high levels of public support and a strong public commitment with close monitoring in France, (Fong et al., 2013) China, (Levy et al., 2014) the United States and Canada, (House of Representative, USA., 2009; Levy et al., 2011; Hitchman et al., 2013) South Africa, (Reddy et al., 2013) South Korea, (Levy et al., 2010) and Brazil (Levy et al., 2012) (**Table 1**).The popular policies were tax increase, restriction on advertisement, and smoking ban in public places, but the most effective policy is raising the tax on tobacco products (Health NCfCDPaHPUOoSa., 2012) Abuse of youth to promote smoking by companies and smuggling are major challenges, particularly in Africa and South America. The above policies showed that hard policies (legal provision) could be more effective and long lasting. Some creative soft policies (program and interventions) were also equally effective but policies related to tobacco cessation are not enough. Primarily, strong monitoring and creative preventive policies led to satisfactory reduction on prevalence on youth tobacco use.

In our study, we explored the need to formulate tobacco control policies by age groups, and the vulnerable age group is the youth; there is a need for a preventive way that is cost-effective and a lesser burden to the disease both clinically and economically. There are some similar conclusions in our studies. Saleheen et al. suggested that there is need for targeted policies for youth and burden of diseases produced by tobacco (Saleheen et al., 2014). Another systematic review by Jawad with 36 studies from 15 countries yielding 125 elasticity estimates found that a 10% price increase would reduce demand by 8.3% for cigars, 6.4% for roll your owns, 5.7% for bidis, and 2.1% for smokeless tobacco (Jawad et al., 2018), mostly to youth (Levy et al., 2018). Another systematic review from 16 studies by Duncan found that a tobacco control approach in health care settings is more effective than in school and at home (Duncan et al., 2018). A policy review by Glantz concluded that e-cigarettes are replacing conventional cigarettes, but it is not a good approach to reducing nicotine dependency (Glantz and Bareham, 2018). Now, the JUUL lab's mission is to eliminate cigarettes, but it does not relieve nicotine dependency as it only replaces traditional tobaccos; the challenge for JUUL is to eliminate nicotine dependency in the future. The tobacco control project period is also a significant factor because comprehensive tobacco control programs lead to an 8% short-term relative reduction, increasing to a 12% long-term relative reduction in smoking prevalence through the greater impact on youth smoking (Levy et al., 2018).

#### TABLE 2 | National tobacco policies and challenges to control.


#### TABLE 2 | Continued.


Similar policies were applied by Canada and reduced the youth smoking from 25%–12% from 2002–2011 (Canadian tobacco use monitoring survey 2011; current smoking from 1985 to 2011; youth aged 15-19, 2011).

### Way Forward

We discussed the situation of youth tobacco use, its effectiveness, and major challenges. Now, there is the question of the way forward. UN sustainable development goal 3a indicates strengthening the implementation of the WHO Framework Convention on Tobacco Control in all countries, as appropriate. Multiple impacts of tobacco control focusing on youth directly reduce adult and premature mortality rate (Ranabhat et al., 2017). Thus, there is a need for specific policies by age because growth and development, psychology, social environment, responsibilities, and pathophysiology are not similar for all ages. In this line, there should be different policies, programs, interventions, and remedies. Similarly, the priority of tobacco control should be different because intervention for youth and the elderly does not provide similar outcomes. Other public health projects like maternal health, child reproductive health, disease control, etc., are a matter of resource allocation, and we expect measurable outcomes in each micro activity. There is no debate that tobacco prevention for youth ensures low cost and higher output. The funding organizations for tobacco control have not encouraged or disappointed because such organizations have not provided significant outcomes. If we distinguish by age, specific tobacco control, and prevention program, funding projects may revisit their policies and invest more on youth due to their economic productive life. It is possible because youth have short exposure to tobacco addiction, can make strong commitments to quit tobacco, are easy to motivate to quit tobacco, and have family and social pressure in comparison with adults and the elderly. A similar analysis can be found in *The State of Youth Tobacco Prevention and Control Spending in Alabama: Struggles, Consequences, and Solutions* report 2014 (Dunlap and McCallum, 2014).

The average strength of policies varied significantly by country. The success of tobacco control initiatives was significantly associated with the number and types of policy adopted (Wipfli and Huang, 2011). In European countries, limiting youth exposure to smoking in movies might be an effective way to prevent adolescent smoking onset (Morgenstern et al., 2013). In addition, a family smoking prevention law in the United States has reduced the prevalence of youth smoking (Ribisl, 2012). Recently, Nepal drafted a clear provision that people who use tobacco (including smokeless tobacco) will not be eligible as government employees; this policy is appealing because it directly affects unemployed youth (Sinha et al., 2012). Peer-based approach projects are best models in youth tobacco control in Australia. Youth are active users of social media such as Facebook, Twitter, YouTube, Messenger, and other apps. Creative mobilization of information technology could be useful for implementation and monitoring of youth tobacco use (Freeman, 2012). Social drama, documentary, family movies, cartoons about tobacco use impact on youth from elementary school could also discourage youth from using tobacco (Merchant, 2013). Hence, appropriate use of mass media and social network is more effective. The e-cigarette is another option for tobacco control, but regulatory issues surrounding audit of electronic cigarettes is unknown (Jovanovic and Jakovljevic, 2015). A systematic review by Park et al. concluded that the use of multimedia, tailored approaches, personalized feedback, and interactive feature programs could positively affect tobacco prevention and cessation (Park and Drake, 2015). Likewise, receptivity to tobacco advertising was significantly associated with progression toward use in adolescents (Pierce et al., 2018).

### Strengths and Limitations

This article describes tobacco control policies with a focus on the younger generation. This is an issue that is significant but often neglected. This article is intended for a wide range of readers (basic readers to policy makers, policy researchers, and other stakeholders related to tobacco control). However, this paper also has some limitations. We used the term youth as used by different authors in their articles and context than any fixed age. Here, we have used terms such as boys, girls, adolescents, elder children, and teenagers synonymously with youth. Tobacco use is focused mostly on smoking, though smokeless tobacco is also a major problem that the world also needs to face. Being a comprehensive review, all components of systematic review and meta-analysis are not applicable to use.

### CONCLUSION

About 1/5 of youth used tobacco globally and it has multiple impacts on health economy and family integrity. The prevalence of tobacco use is decreasing but not at a satisfactory rate. The policies of tobacco control adopted by many countries are based on the WHO Framework Convention on Tobacco Control but have not necessarily focused on youth. Due to the physical and economic burden of tobacco consumption by youth, this is a high priority that needs to be addressed. In this digital age, creative tobacco control policies focusing on youth must be applied. Successful policies of tobacco control for youth that need to be replicated by context, country, region, and gender can be recommended. Tobacco control should be a social, public health, and quality-of-life concern rather than a business and trade issue.

### REFERENCES


### AUTHOR CONTRIBUTIONS

CR prepared the research concept and framework, collected the articles, prepared the manuscript, and overall pursued the article. C-BK verified the concept, verified the reference, and reviewed the manuscript. MBP verified all references and prepared the composition of contents. MJ reviewed the manuscript and rearranged some parts.

### FUNDING

This study was supported by a National Research Foundation Grant of Korea, Korean Government (NRF-2016S1A5B892520) and the Korean Medical Association (RIHP-2015-02).

### ACKNOWLEDGMENTS

We sincerely acknowledge Prof. Indeok Kong, Dr. Shambu Prasad Acharaya (WHO), Dr. Nabin Baral and Arthur Atkinson for their quick review and Margaret Storey for English language improvement.

Cancer Council. (2017a). Approaches to youth smoking prevention.


policy and programs in Australia: protocol for a mixed-method prospective study. *BMC Public Health* 13 (1), 879. doi: 10.1186/1471-2458-13-879


Tamil Nadu-A School Based Survey. *J. Indian Assoc. Child Adolesc. Ment. Health* 10 (2), 110–131.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Ranabhat, Kim, Park and Jakovljevic. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Cost-Effectiveness Analysis of Influenza A (H1N1) Chemoprophylaxis in Brazil

*Luisa von Zuben Vecoso1\*, Marcus Tolentino Silva2, Mariangela Ribeiro Resende1, Everton Nunes da Silva3 and Tais Freire Galvao1\**

*1 University of Campinas, Campinas, Brazil, 2 Universidade de Sorocaba, Sorocaba, Brazil, 3 University of Brasilia, Brasilia, Brazil*

Background: Oseltamivir and zanamivir are recommended for treating and preventing influenza A (H1N1) worldwide. In Brazil, this official recommendation lacks an economic evaluation. Our objective was to assess the efficiency of influenza A chemoprophylaxis in the Brazilian context.

#### *Edited by:*

*Tahir Mehmood Khan, University of Veterinary and Animal Sciences, Pakistan*

#### *Reviewed by:*

*Amer Hayat Khan, University of Science, Malaysia Brian Godman, Karolinska Institute (KI),Sweden*

#### *\*Correspondence:*

*Tais Freire Galvao taisgalvao@gmail.com Luisa von Zuben Vecoso luisavecoso@gmail.com*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 15 December 2018 Accepted: 24 July 2019 Published: 10 September 2019*

#### *Citation:*

*Vecoso LvZ, Silva MT, Resende MR, da Silva EN and Galvao TF (2019) Cost-Effectiveness Analysis of Influenza A (H1N1) Chemoprophylaxis in Brazil. Front. Pharmacol. 10:945. doi: 10.3389/fphar.2019.00945*

Methods: We assessed the cost-effectiveness of oseltamivir and zanamivir for prophylaxis of influenza for high risk population, compared to no prophylaxis, in the perspective of Brazilian public health system. Quality-adjusted life years (QALY) and effectiveness data were based on literature review and costs in Brazilian real (BRL) were estimated from official sources and micro-costing of 2016's H1N1 admissions at a university hospital. We used a decision-tree model considering prophylaxis and no prophylaxis and the probabilities of H1N1, ambulatory care, admission to hospital, intensive care, patient discharge, and death. Adherence and adverse events from prophylaxis were included. Incremental cost-effectiveness ratio was converted to 2016 United States dollar (USD). Uncertainty was assessed with univariated and probabilistic sensitivity analysis.

Results: Adherence to prophylaxis was 0.70 [95% confidence interval (CI) 0.54; 0.83]; adverse events, 0.09 (95% CI 0.02; 0.18); relative risk of H1N1 infection in chemoprophylaxis, 0.43 (95% CI 0.33; 0.57); incidence of H1N1, 0.14 (95% CI 0.11; 0.16); ambulatory care, 0.67 (95% CI 0.58; 0.75); hospital admission, 0.43 (CI 95% 0.39; 0.42); hospital mortality, 0.14 (CI 95% 0.12; 0.15); intensive care unit admission, 0.23 (95% CI 0.20; 0.27); and intensive care mortality, 0.40 (95% CI 0.29; 0.52). QALY in H1N1 state was 0.50 (95% CI 0.46; 0.53); in H1N1 inpatients, 0.23 (95% CI 0.18; 0.28); healthy, 0.885 (95% CI 0.879; 0.891); death, 0. Adverse events estimated to affect QALY in –0.185 (95% CI –0.290; –0.050). Cost for chemoprophylaxis was BRL 39.42 [standard deviation (SD) 17.94]; ambulatory care, BRL 12.47 (SD 5.21); hospital admission, BRL 5,727.59 (SD 7,758.28); intensive care admission, BRL 19,217.25 (SD 7,917.33); and adverse events, BRL 292.05 (SD 724.95). Incremental cost-effectiveness ratio was BRL –4,080.63 (USD –1,263.74)/QALY and –982.39 (USD –304.24)/H1N1 prevented. Results were robust to sensitivity analysis.

Conclusion: Chemoprophylaxis of influenza A (H1N1) is cost-saving in Brazilian health system context.

Keywords: cost-effectiveness, cost-utility, neuraminidase inhibitor, prophylaxis, influenza, Brazil, Unified Health System

### INTRODUCTION

Influenza A (H1N1) prophylaxis with neuraminidase inhibitors is recommended by the World Health Organization (WHO), and health agencies of most developed and underdeveloped countries (World Health Organization, 2010; World Health Organization, 2019). Population at risk for influenza A complications includes pregnant and postpartum women, the elderly, children, indigenous people, immunosuppressed persons, health professionals, and long-term residents among others (Uyeki et al., 2018; Martinez et al., 2019). Influenza A accounted for 97% of the specimen circulating in the firsts months of 2019, of which 60% were influenza A (H1N1) 2009 pandemic (World Health Organization, 2019). Deaths associated with respiratory diseases from seasonal influenza accounts 300,000 to 650,000 annually (Iuliano et al., 2018). Higher burden of death is observed in less developed regions and in the elderly (Iuliano et al., 2018).

Complete efficacy data of neuraminidase inhibitors were published in 2014 and updated in 2016 (Jefferson et al., 2014a; Jefferson et al., 2014c; Heneghan et al., 2016). Before this effort, 60% of the patient data from phase III clinical trials have never been published; previous evidence could have been biased in favor of chemoprophylaxis (Jefferson et al., 2014b). Biases and conflicts of interests involved in research on influenza treatment and prevention translate into a need for studies on the drugs' clinical performance vis-à-vis health systems' financial investments (Jefferson et al., 2014b). Economic evaluations that take into consideration complete efficacy evidence are not available.

The efficiency of Influenza A (H1N1) chemoprophylaxis is also absent in in the Brazilian context, in which it is recommended and funded by the Ministry of Health (Brasil. Ministério da Saúde, 2017). Our objective was to assess the cost-effectiveness of influenza A (H1N1) chemoprophylaxis in the Brazilian public health system.

### METHODS

### Target Population and Subgroups

Our target population were non-vaccinated or vaccinated for less than 15 days, people groups with high risk for influenza complications (the elderly, children, indigenous people, obese individuals, people with chronic diseases or immunodeficiency, pregnant or puerperal women), health care and laboratory workers exposed to samples or cases of influenza, and residents of nursing homes or inpatients during an outbreak (Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde, 2017).

### Setting and Location

The Unified Health System (*Sistema Único de Saúde*, SUS) is a public and universal health system (Paim et al., 2011). SUS is the public health sector responsible for primary care, access to medicines, immunization programs, complex services (cancer treatment and HIV/AIDS care), sanitary regulation, and sentinel surveillance, which monitors influenza by means of mandatory reports on flu syndrome and severe acute respiratory syndrome (World Health Organization, 2010). Access to these services has been largely improving since the system's birth in 1988 (Paim et al., 2011). Despite this gradual improvement over the decades, SUS is systematically underfunded (Paim et al., 2011).

### Study Perspective

We adopted the SUS perspective and considered costs in the SUS context and excluded societal costs such as absence from work and patient personal costs. This involved costs for drug acquisition, health care services expenditure in cases of symptomatic diseases (ambulatory treatment, medical consultation, hospital admission, and procedures), and treatment of prophylaxisrelated adverse events.

### Comparators

We assessed influenza A (H1N1) chemoprophylaxis in the aforementioned high-risk population, comparing oseltamivir and zanamivir prophylaxis with no prophylaxis.

Oseltamivir is an oral antiviral drug that inhibits the neuraminidase surface enzyme [Anatomical Therapeutic Chemical (ATC) code: J05AH02]. Its market availability was scientifically supported by experimentally infecting healthy subjects with influenza A and B (EPAR summary for the public, 2015). The drug effectively prevented influenza A infection after individuals were exposed to it (Jefferson et al., 2006), and was also able to reduce cases of symptomatic influenza within households (Dobson et al., 2015), as well as the time for alleviation of symptoms in infected adults. Oseltamivir significantly increased the incidence of nausea, vomit, and psychiatric events (Jefferson et al., 2006). Adults and children with more than 40 kg should take a 75 mg dose orally every 12 h, for 10 days. For children below this weight, the dosage is adjusted to 3–3.5 mg/kg for infants; 30 mg for children up to 15 kg; 45 mg, for over 15 to 23 kg; and 60 mg, until 40 kg (Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde, 2017).

Zanamivir is an antiviral selective neuraminidase inhibitor (ATC code: J05AH01) administered intranasally (Relenza, 2015). *In vitro* assays showed that low concentrations of the drug were able to inhibit influenza A and B neuraminidase. Symptom duration was reduced in healthy adults (median reduction 1.5 days; 1.0–2.5 days), but the mean time for symptom alleviation in elderly (>65 years) and in 5 or 6 year-old children was not significantly reduced. It has no documented benefits against non-febrile disease (body temperature < 37.8°C) (Relenza, 2015). Zanamivir is employed only in cases where oral oseltamivir is not feasible. Adults and children older than 5 years should receive two 5 mg inhalations per day for 10 days (Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde, 2017).

### Time Horizon and Discount Rate

We evaluated the outcomes of influenza A (H1N1) prophylaxis based on the duration of influenza infection, which is less than 21 days. No discount rate was applied.

### Choice of Health Outcomes

Quality-adjusted life years (QALY) was the primary outcome. Willingness-to-pay (WTP) threshold was considered to be 30,000 Brazilian real (BRL) per QALY (Soarez and Novaes, 2017). Prevented influenza A (H1N1) was also assessed, as a secondary outcome.

### Measurement of Effectiveness

#### Search Strategy

Data on oseltamivir's and zanamivir's effectiveness in preventing symptomatic flu and its complications was gathered from search on the literature held on March, 2017. The following search strategy was employed in the MEDLINE (via PubMed) database: (oseltamivir OR tamiflu OR zanamivir OR relenza OR "neuraminidase inhibitors") AND (H1N1 OR influenza) AND ("clinical trial"[Filter] OR "systematic"[Filter] OR cost OR economic). The same strategy was adjusted to Embase, Scopus, and Cochrane Library databases. Additional searches were performed to ascertain effectiveness and cost data in the Brazilian scenario. Results were imported to Covidence (www. covidence.org) for identifying duplications; pair selection was performed by two independent researchers. Systematic reviews, randomized clinical trials, and observational studies were included.

Complementary non-systematic searches were performed in order to gather specific data on prevalence, hospitalization, death in hospital, and other variables included in the model. Information was also collected from SUS electronic systems whenever needed. When estimates from different studies were available, random-effect meta-analysis was performed using *Stata* (version 14.2).

### Quality Assessment of Included Studies

We assessed the quality of all the included studies using standard instruments: A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2) for systematic reviews (Shea et al., 2017), Newcastle–Ottawa scale for cohort and case–control studies (Wells et al., 2000), and the Joanna Briggs Institute checklist for prevalence studies (Munn et al., 2015).

### Estimating Resources and Costs

Costs of oseltamivir and zanamivir acquisition were obtained from 2016 purchase data, provided by the Brazilian Ministry of Health, using information made available by the Pharmaceutical Assistance Department. Health care assistance costs were obtained from the SUS reimbursement system (http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio. jsp). We considered the dosage and administration according to Brazilian guidelines (Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde, 2017).

Health expenditures were obtained from micro-costing of all inpatients admitted in 2016 for H1N1 treatment at the Clinics' Hospital of the University of Campinas, Campinas, São Paulo a 400-beds high complexity hospital.

### Currency, Price Date, and Conversion

Costs were calculated in BRL acquisitive value in 2016. Costs gathered from the literature from previous years were corrected to 2016 using the Brazilian consumer's price index *(Índice de Preços ao Consumidor*, IPCA) (https://ww2.ibge.gov.br/home/estatistica/ indicadores/precos/inpc\_ipca/defaultinpc.shtm). The obtained incremental cost-effectiveness ratio (ICER) was converted to United States dollars (USD) using the exchange rate for July 1st, 2016 provided by Brazil's Central Bank (1 USD = 3.229 BRL) (https:// www4.bcb.gov.br/pec/taxas/ingl/ptaxnpesq.asp?id = quotations).

### Choice of Model

TreeAge Pro 2018 (R.2.0) software was used to build a decisiontree model. Two scenarios were considered: chemoprophylaxis and no chemoprophylaxis. In both scenarios, the following probabilities were assessed: H1N1 infection, ambulatory care, hospital admission, intensive care admission, patient discharge, and death. In the prophylaxis scenario, we included adherence to prophylaxis and incidence of adverse events (**Figure 1**).

Costs for outcomes were calculated considering that all flu cases were influenza A (H1N1) type; half-cycle correction was used to calculate costs for cases with death as the final outcome. Clinical plausibility was evaluated by an infectious disease specialist doctor, who was part of the research team (MRR) and had experience in influenza management.

### Assumptions

We considered that all symptomatic patients would seek outpatient care. Hospital admission was assumed as a probability for those seeking ambulatory care, and admission to the intensive care unit as a probability for people admitted to the hospital. Death was assumed as possible only for people admitted to the hospital or to the intensive care. Subjects who did not develop flu were considered healthy. No sequelae or late effects of influenza were considered.

### Analytical Methods

Uncertainties of the model were estimated according to variations in the adopted parameters. A tornado diagram of minimum and maximum values was used for univariate sensitivity analysis.

Sensitivity-enhancing model parameters were chosen for bestand worst-case scenario analyses.

Probabilistic sensitivity analysis was performed using Monte Carlo, employing a 10,000 simulation count and threshold analysis to identify the maximum cost of the technology, all other parameters unchanged (*ceteris paribus*). We used variables as distribution; beta distribution was adopted for probabilities and outcomes, gamma for costs, and log-normal for relative risk (Bilcke et al., 2011).

### Ethics Approval Statement

The study was approved by the University of Campinas Ethics Committee, report number 2,357,158 issued on October, 30th 2017. The study was exempt from consent procedure, once patient data would be from medical records.

### RESULTS

### Study Parameters

#### Effectiveness Data

Probabilities of efficacy adopted are described in **Table 1**.

TABLE 1 | Probabilities of outcomes, distribution parameters adopted in the analytical model, and sources.


*abeta distribution. bJoanna Briggs Institute checklist. cAMSTAR 2. drelative risk. elog-normal distribuiton. f Newcastle-Ottawa scale. CI, confidence interval; SD, standard deviation.*

Prophylaxis adherence was considered to be 70%, according to adherence data from health professionals exposed to H1N1 virus during the 2009 pandemic (Upjohn et al., 2012). The incidence of adverse events among those who adhered to the prophylaxis was estimated as 9%, based on the incidence of headaches, nausea and psychiatric events—the most frequent and significant adverse events (**Appendix A**).

Risk of H1N1 infection in the high-risk population was considered to be 14%, based on the incidence of symptomatic infection among households which had contact with infected patients (Lau et al., 2012). The relative risk of H1N1 infection with prophylaxis was considered to be 0.43 [95% confidence interval (CI) 0.33; 0.57], according to meta-analysis for the prophylaxis with the antivirals (Jefferson et al., 2014a) (**Data Sheet S1**). Since scientific evidence showed no efficacy for preventing complications (a proxy for seeking for medical care), hospital or intensive care admission and death from influenza (Jefferson et al., 2014a; Jefferson et al., 2014c; Heneghan et al., 2016), these variables had the same probability in both prophylaxis and no prophylaxis branches: the probability of seeking medical care (ambulatory care) was 0.67, the incidence of symptomatic illness after experimental influenza infection (Carrat et al., 2008), assuming that all people who developed symptoms would seek medical care.

Incidence of hospital (43%), and intensive care (23%) admission, hospital (23%), and intensive care mortality (40%) were based on Brazilian studies held during the 2009–2010 pandemics (Duarte et al., 2009; Lenzi et al., 2012; Calmona, 2013). Complete quality assessment of studies that provided data to the model are available at **Data Sheet S2**.

### Utility

QALY for H1N1 infections managed in outpatient services was 0.50 and those admitted to hospital or intensive care was 0.23 based on a study with patients infected with H1N1 during the 2009 pandemic (Hollmann et al., 2013). Adverse events reduced QALY in 0.195 (**Appendix A**). The QALY for the healthy state was 0.885, the mean QALY measured in two population-based Brazilian studies (Zimmermann et al., 2016; Silva et al., 2017). QALY for death was 0 (**Table 2**).

#### Costs

Cost with prophylaxis was BRL 39.42, based on average expenditure of Brazilian Ministry of Health with the antivirals (**Appendix B**). Treatment of prophylaxis' adverse cost BRL 292.05, calculated from the cost of each main adverse event (headache, nausea, and psychiatric event) weighted to each adverse event incidence (**Appendix A**).

Outpatient care cost BRL 12.47 according to SUS reimbursement for an urgent care consultation. Cost of hospital admission was estimated in BRL 5,727.59 and for intensive care, BRL 19,217.25 (**Table 3**).

### Incremental Costs and Outcomes

The prophylaxis scenario was undominated, while no prophylaxis was absolutely dominated (**Table 4**). The incremental cost of prophylaxis was BRL –54.45, and QALY increased 0.013, resulting in an ICER of BRL –4,080.63 per QALY (USD –1,263.74/QALY). For the secondary outcome prevention of H1N1 infection, incremental QALY was 0.055, and ICER was BRL –982.39 per prevented case (USD –304,24/prevented H1N1).

## Characterizing Uncertainty

### Univariate Sensitivity Analysis

The tornado-diagram sensitivity analysis demonstrated the robustness of our model when using expected intervals for each variable (**Figure 2**). None of the variables changed the costeffectiveness profile of the technology given the adopted WTP threshold (BRL 30,000.00/QALY). The ICER remained robust after best- and worst-case scenario analysis with highest impact variables in the tornado (**Table 5**). Threshold analysis led to BRL 134.00 limit for chemoprophylaxis cost-effectiveness.

### PROBABILISTIC SENSITIVITY ANALYSIS

In the probabilistic sensitivity analysis, 68% of ICER would be in fourth quadrant (higher effectiveness and lower cost) and 18% of ICER, in first quadrant (higher cost and effectiveness). The probability of the technology being under the WTP threshold (BRL 30.000/QALY) was 97.9% (**Table 6**, **Figure 3**).


*abeta distribution. bNewcastle-Ottawa scale. creduction on QALY due to adverse events. ddata from previous economic evaluation which used multiple sources. eJoanna Briggs Institute checklist (both studies had this score). QALY, quality-adjusted life years; SD, standard deviation.*

#### TABLE 3 | Costs included in the model, in Brazilian real.


*agamma distribution. bavailable from: http://sigtap.datasus.gov.br/tabela-unificada/app/ sec/inicio.jsp; SD, standard deviation.*

TABLE 4 | Costs, effectiveness and incremental cost-effectiveness ratio (ICER) of prophylaxis compared to no prophylaxis.


*QALY, quality-adjusted life years; BRL, Brazilian real (1 USD = 3.229 BRL); USD, United States dollar.*

### DISCUSSION

H1N1 prophylaxis compared to no prophylaxis was cost-saving in the context of the Brazilian health system for both QALY and prevention of H1N1 outcomes. The mean cost calculated from micro-costing are aligned to previous Brazilian studies that TABLE 5 | Incremental cost-effectiveness ratio for best- and worst-case scenarios (variables with the highest impact in the univariate sensitivity analysis).


estimated the cost of hospital admission to influenza A (H1N1) (Silva et al., 2012).

The chemoprophylaxis reduces the cost and the increases the effectiveness of influenza A (H1N1) prevention. Its effect on QALY (0.013), however, may be clinically irrelevant. In any case, preventing a single influenza A (H1N1) case by means of prophylaxis could save nearly BRL 1,000. At the same time, Brazil has no official WTP threshold (Soarez and Novaes, 2017). Whether present represents a cost-effective alternative is subject for debate. Effects of neuraminidase inhibitors on prophylaxis came from clinical trials in which exposure to H1N1 and treatment onsets were highly controlled. The effectiveness for chemoprophylaxis is limited to strict conditions according to a mathematical modelling and computer simulations, and stockpiling for this situation is questioned (Parra-Rojas et al., 2018). Despite a protocol to start the drug in the first 24 hours post-exposure, pragmatic clinical trial revealed late initiation of oseltamivir at the hospital setting without reduction of clinical failures among the assessed groups (Ramirez et al., 2018). This potentially unrealistic efficacy data may have inflated the effects of prophylaxis.

We obtained influenza prevention efficacy data from systematic reviews carried out as the offspring of a Cochrane Collaboration and The BMJ campaign to obtain complete clinical trials data from Roche, the drug manufacturer. The campaign's efforts led to the publication of the systematic review in 2014; it was then updated in 2016, with no changes in the results (Jefferson et al., 2014a; Jefferson et al., 2014c; Heneghan et al., 2016). Sixty percent of patient data in phase III clinical trials had never been published, suggesting that

FIGURE 2 | Univariate sensitivity analysis of incremental cost-effectiveness ratio (ICER) of chemoprophylaxis compared to no prophylaxis. QALY, quality-adjusted life years; ICU, intensive care unit.


TABLE 6 | Probabilities (p) of incremental cost-effectiveness ratio (ICER) in each quadrant according to 10,000 Monte Carlo simulations, chemoprophylaxis versus no prophylaxis.

previously-published research was biased in favor of the technology (Jefferson et al., 2014a). Publication bias was reduced once all clinical trials with the drugs were taken into consideration in such efforts (Jefferson et al., 2014a; Jefferson et al., 2014c; Heneghan et al., 2016).

Some of our probabilities were based on data from studies held during the 2009 influenza A (H1N1) pandemic, a period marked by greater virulence of influenza in Brazil and worldwide (Ministério da Saúde, 2012). In 2009, cases of severe acute respiratory syndrome in Brazil reached more than 44 per 100,000 inhabitants; later in 2010, its occurrence decreased to 4.6 cases per 100.000 inhabitants, finally reaching 2.5 in 2011. Influenza vaccine has been part of programmed vaccination for the elderly (>65 years of age) since 1999, and its use was expanded in 2010 to people >60 years of age. In 2011, pregnant women, children between six months and two years of age, indigenous people and health workers were included; since then, vaccine coverage has hovered above 80% (Ministério da Saúde, 2012). The probabilities adopted in our model led to more severe consequences for influenza, favoring the prophylaxis performance. We assumed that all patients with symptoms would seek for medical care, therefore we did not consider "out-of-pocket" expenses in cases that patients would treat themselves without seeking for medical consult, as did an economic study of dengue in Brazil (Godoi et al., 2018). Adherence to prophylaxis was based on health professionals during the 2009 pandemic, period with greater concern about infection. Such assumptions and use data from the pandemia brought to a more conservative scenario that probably does not reflect the current scenario, where more people vaccinated and greater herd immunity is granted. Deterministic and probabilistic sensitivity analysis attested robustness of cost-effectiveness when probabilities of infection, hospital admission, and death by H1N1 ranged, partially circumventing these limitations.

The primary outcome of our study was based on QALY from the Spanish context, due to lack of utility data for influenza in Brazil. QALY for healthy state was based in Brazilian population data (Zimmermann et al., 2016; Silva et al., 2017). We evaluated the prevention of influenza as a secondary outcome, which does not involve population perception and favored prophylaxis. The Brazilian protocol for influenza states that chemoprophylaxis should be administered to non-vaccinated or vaccinated for less than 15 days people (Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde, 2017). Data on the effectiveness of the antiviral drugs segregated by vaccination status were not available for a specific analysis of the target-population not under the vaccine's effect. Influenza vaccination showed to reduce healthcare utilization in the elderly (Doyon-Plourde et al., 2019), as well as antibiotic usage in health adults (Buckley et al., 2019). While maintaining consistency with the national guideline, ignoring the effect of vaccination in our model may have favored the need and effectiveness of the chemoprophylaxis.

Our study is similar to previous health economic evaluations on influenza chemoprophylaxis, which also adopted a decisiontree model with a time horizon shorter than one year and favored the prophylaxis. In the Canadian health system, post-exposure prophylaxis in institutionalized and vaccinated elderly was dominant for preventing influenza-like illnesses when compared to no prophylaxis (Risebrough et al., 2005). This evaluation was based on three alternatives – prophylaxis with amantadine, prophylaxis with oseltamivir and no prophylaxis – and predicted viral resistance and adverse effects on the amantadine branch, influenzalike illnesses, complications, death, survival, and treatment in hospital or institution (Risebrough et al., 2005). The research was sponsored by oseltamivir manufacturer, F. Hoffmann-La Roche.

In the United Kingdom, post-exposure prophylaxis for interfamily contacts was probably cost-effective in the context of the National Health System, considering 2002's cost data (Sander et al., 2006). The model compared prophylaxis to no prophylaxis with or without oseltamivir treatment in the case of symptomatic influenza, and predicted complications, outpatient care, hospital admission, recovery and death, and assessed QALY and avoided cases of influenza-like illness. Probabilistic and sensitivity analysis attested the robustness of the model (Sander et al., 2006). The study was also sponsored by F. Hoffmann-La Roche, and the last author was an employee of the company.

United States analysis of post-exposure prophylaxis with oseltamivir in children up to 12 years was cost-effective in the perspectives of society and the payer, with 2008's costs (Talbird et al., 2009). The model compared prophylaxis to no prophylaxis and predicted development of influenza, hospital admission, outpatient care, death, and survival (Talbird et al., 2009). The research was commissioned by Roche, and the last author was its employee.

The National Health System in the United Kingdom funded a systematic review about efficacy and effectiveness of seasonal and post-exposure prophylaxis, with subsequent analysis of costeffectiveness using amantadine, oseltamivir, and zanamivir in vaccinated and non-vaccinated individuals (Tappenden et al., 2009). Six subgroups were considered: children, adults and elderly, in high-risk or healthy states, using cost data for 2006. Influenzalike illnesses, search for outpatient care, antiviral treatment,

### REFERENCES


complications, death, and survival were considered in the analysis (Tappenden et al., 2009). The model predicted adverse effects to amantadine, vaccination, and prophylaxis abandonment ranging from 1.3% to 14.7%. Post-exposure prophylaxis was under 30,000.00 British pounds/QALY for non-vaccinated children and the elderly. None of these economic assessments considered herd immunity, adverse events of the studied drugs, and the complete efficacy data with lower risk of publication bias (Jefferson et al., 2014a; Jefferson et al., 2014c; Heneghan et al., 2016).

## CONCLUSION

Post-exposure prophylaxis for influenza A (H1N1) is cost-saving in the context of the Brazilian public health system. Current Brazilian guidance for influenza A (H1N1) prevention is supported by the findings, but a lack of national efficacy and effectiveness data is noticed. Both oseltamivir and zanamivir are already incorporated for this purpose, changes to current guidelines are unnecessary.

### ETHICS STATEMENT

Our study was approved in October 30, 2017 by Unicamp Ethics Committee (2.357.158) de 30/10/2017, certificate number 78192417.0.0000.5404. The study was exempt from consent procedure, once data was from patient medical records.

### AUTHOR CONTRIBUTIONS

LV, TG and MS designed the work, LV collected the data, LV and TG did the analyses and drafted the work, MS, ES and MR interpreted the data and revised the work critically. All authors approved the version to be published and agree to be accountable for all aspects of the work.

### FUNDING

The study was funded by HAOC – PROADI SUS. Nonfinancial support was provided by Clinics' Hospital and Faculty of Pharmaceutical Sciences of the University of Campinas.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00945/ full#supplementary-material

the brazilian private healthcare system. *JBES Braz. J. Health Econ. J. Bras. Econ. Saúde* 6 (3).


*tratamento de Influenza 2017* (Brasília: Ministério da Saúde). bvsms.saude.gov. br/bvs/publicacoes/protocolo\_tratamento\_influenza\_2017.pdf.


occupational influenza exposure. *Am. J. Infect. Control* 40 (10), 1020–1022. doi: 10.1016/j.ajic.2011.11.014


Zimmermann, I. R., Silva, M. T., Galvao, T. F., and Pereira, M. G. (2016). Healthrelated quality of life and self-reported long-term conditions: a populationbased survey. *Rev. Bras. Psiquiatr.* doi: 10.1590/1516-4446-2015-1853

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor and reviewer BG declared their involvement as co-editors in the Research Topic, and confirm the absence of any other collaboration.

*Copyright © 2019 Vecoso, Silva, Resende, da Silva and Galvao. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## APPENDIX

### Appendix A. Cost and Utility of Adverse Events

We used the risk difference of each significant adverse event reported in the systematic review (Jefferson et al., 2014a) to calculate the probabilities of adverse events.


*a2014's costs corrected to 2016. b"suspected serious psychotic/suicidal adverse events (including hallucination, psychosis, schizophrenia, paranoia, aggression/hostility and attempted suicide)" (Jefferson et al., 2014a). c2009's costs corrected to 2016. dStandard deviation = 736.34. QALY, quality-adjusted life years.*

### Appendix B. Expenditure With Acquisition of Oseltamivir and Zanamivir by the Pharmaceutical Services Department, Brazilian Ministry of Health in 2016

It is worth noting that we did not consider stockpiling costs and loss due to product expiration, since such data was unavailable. This would be important information for calculating the total cost of chemoprophylaxis.


*BRL, Brazilian real; astandard deviation = 17.94. bexpenditure in BRL from 01/01/2016 to 08/23/2017. cminimum value adopted on the univariate sensitivity analysis. dmaximum value adopted on the univariate sensitivity analysis.*

# Adverse Events in Patients With Rheumatoid Arthritis and Psoriatic Arthritis Receiving Long-Term Biological Agents in a Real-Life Setting

*Mayara Costa de Camargo1, Bruna Cipriano Almeida Barros2, Izabela Fulone1, Marcus Tolentino Silva1, Miriam Sanches do Nascimento Silveira1, Iara Alves de Camargo1, Silvio Barberato-Filho1, Fernando de Sá Del Fiol1 and Luciane Cruz Lopes1\**

*1 Pharmaceutical Sciences Graduate Program, University of Sorocaba (UNISO), Sorocaba, Brazil, 2 Hematology Graduate Program, Federal University of São Paulo (UNIFESP), São Paulo, Brazil*

#### *Edited by:*

*Deslypere Jean Paul, Besins Healthcare, Thailand*

#### *Reviewed by:*

*Marc Henri De Longueville, UCB Pharma, Belgium Gaurav Deshpande, Bristol Myers Squibb, United States*

#### *\*Correspondence:*

*Luciane Cruz Lopes luslopesbr@gmail.com; luciane.lopes@prof.uniso.br*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 30 January 2019 Accepted: 29 July 2019 Published: 11 September 2019*

#### *Citation:*

*de Camargo MC, Barros BCA, Fulone I, Silva MT, Silveira MSN, Camargo IA, Barberato-Filho S, Del Fiol FS and Lopes LC (2019) Adverse Events in Patients With Rheumatoid Arthritis and Psoriatic Arthritis Receiving Long-Term Biological Agents in a Real-Life Setting. Front. Pharmacol. 10:965. doi: 10.3389/fphar.2019.00965*

Background: Biological agents used for the treatment of psoriatic arthritis (PsA) and rheumatoid arthritis (RA) are associated with serious adverse effects (SAEs). Although several biologics have demonstrated good efficacy and tolerability in short-term trials, treatment guidelines recommend them as third line therapies due to a relative lack of long-term safety data.

Objective: To determine the frequency and severity of adverse effects associated with the long-term use of biologics in the treatment of PsA and RA, and possible risk factors for such events in a real-life setting.

Methods: We conducted a longitudinal study in PsA and RA patients only taking longterm biological agents from 2003 to 2011. Sources of information included dispensing pharmacy data and interviews with patients. Research staff conducted telephone interviews with patients inquiring about any apparent medication-related adverse drug reactions (ADRs) or SAEs. ADR/SAE's data was based on pharmacy reports. We conducted a multivariate analysis to identify the factors associated with the risk of ADRs.

Results: Of the 305 patients identified, we interviewed 268 patients. Most of these were taking adalimumab 127 (47.4%), 52 (19.4%) etanercept, 42 (15.7%) infliximab, 25 (9.3%) rituximab, 10 (3.7%) abatacept, 9 (3.4%) efalizumab, and 3 (1.1%) tocilizumab. Of the 268 patients, 116 (43.3%) experienced one or more adverse events related to biological agents with 1.6 events per patient, and of these 29 (25%) experienced one or more SAEs, with majority subjected to hospitalizations. The most frequently reported ADRs were administration site reactions as observed in 73 patients (27.2%), infections in 30 patients (11.2%), effects on nervous system in 22 patients (8.2%), and 15 (5.6%) patients withdrew due to ADRs. The use of rituximab was related with less risk of ADR [PR 0.42, 95% CI 0.18–0.96; *p =* 0.04] than other agents. No other predisposing factors were associated with risk of ADR. The monitoring of patients (medical consultation and laboratory test) was only completed by 48 patients (30.4%).

**165**

Conclusion: These data showed the early biological experience in Brazil that were associated with ADRs, withdrawals due to ADRs and SAEs. The quantification of adverse effects (serious or nonserious) considering close monitoring and patients' perceptions are increasingly important for future decision-making.

Keywords: adverse reaction, biologic agents, psoriatic arthritis, rheumatoid arthritis, safety

### INTRODUCTION

Biologic agents, introduced in the late 1990s, have improved the treatment outcomes of autoimmune disease, inflammatory disease, and tumour therapy (Chen et al., 2006; NICE, 2012; Coates et al., 2013). Additionally, the application of biological processes involving recombinant DNA technology, which allowed the production of proteins like cytokines and humanized antibodies, must be credited (Mazurek and Jahnz-Rozyk, 2012).

These drugs include tumour necrosis factor (TNF) inhibitors (e.g. adalimumab, certolizumab, etanercept, golimumab and infliximab), anti-CD28 agent (abatacept), anti-cytokine agents (anakinra and tocilizumab), anti-B-cell agent (rituximab), T-cell modulating agent (alefacept), and inhibitors of interleukin (IL)- 12 and IL-23 (ustekinumab) (Rosman et al., 2013). Indications for use vary between the countries in which they have been approved for marketing.

The wide use of biological agents in modern medicine is a challenge for physicians and requires constant learning, with distinct knowledge and familiarity of the disease to be treated. Additionally, these biological agents are expensive and compel the physicians to consider the economic burden on patients. Biologic agents have been associated with high rates of total adverse events and withdrawals due to adverse events (Singh et al., 2011).

Tumor Necrosis Factor-alpha (TNFα) is essential for increasing phagocytic activity of macrophages and other killer cells; therefore, anti-TNFα medication can lead to common and opportunistic infections (Curtis et al., 2011). These include tuberculosis, atypical mycobacteriosis, listeriosis, histoplasmosis, aspergillosis, pneumocystis, and legionellosis (Murdaca et al., 2015).

There is increasing evidence of the paradoxical induction of autoimmune processes associated with biological agents (Lee and Kavanaugh, 2005; Katz and Zandman-Goddard, 2010; Karmacharya et al., 2015). Autoimmune diseases secondary to biological therapies comprise a variety of both, systemic illnesses including lupus, vasculitis, sarcoidosis, and antiphospholipid syndrome (Pichler, 2006). Biological agents have also been associated with organ-specific autoimmune processes including interstitial lung disease, uveitis, optic neuritis, peripheral neuropathies, multiple sclerosis, and autoimmune hepatitis (Hausmann et al., 2010).

The majority of adverse effects manifest between one month to one year after initiating the therapy with a biological agent; however, they may also manifest years after treatment suspension (Mazurek and Jahnz-Rozyk, 2012). Biological agents may also manifest adverse effects that are yet unknown, suggesting that monitoring of ongoing patients is essential (NICE, 2012; Silveira et al., 2014; Gonzalez-Alvaro et al., 2015; Deighton et al., 2009).

Patient reports are an important source of information on patient safety (Fowler et al., 2008; Kuzel et al., 2004) and are useful in evaluating the adverse events (Hibbard et al., 2005; Zhu et al., 2011). In Brazil, patients reported one or more adverse reactions associated with biological agents in 67% of the cases. These patients had no close clinical monitoring (Lopes et al., 2014; Camargo et al., 2016).

To further elucidate the adverse effects and predisposing factors associated with the use of biologics in clinical practice, we performed an observational study in patients with psoriatic arthritis (PsA) and/or rheumatoid arthritis (RA) who had been using the same biological agent for at least six months. The main objective was to evaluated the mediumand long-term safety of biologics in patients from a middleincome country.

### METHODS

### Design and Setting

We utilized a retrospective longitudinal design to investigate the adverse drug reactions (ADRs) (Gonzalez-Alvaro et al., 2015) occurring in patients using long-term biologics (abatacept, adalimumab, efalizumab, etanercept, infliximab, rituximab, and tocilizumab) for treatment of PsA and RA in Brazil. The protocol was authorized by the Health State Department and approved by the Ethics Committee for Clinical Research of University of Sorocaba (August 17, 2009; protocol number 011/2009). Each patient provided an informed consent.

### Definitions

We defined ADR as "a response to a medicine or medicinal product that is noxious and unintended, and which occurs in doses normally used in humans for the prophylaxis, diagnosis, or therapy of disease or for the modification of physiological function" (WHO, 2002). Therefore, we excluded events that resulted from drug errors, therapeutic failures, intentional or accidental poisoning, and drug abuse.

A serious adverse event (SAEs) was defined under the same code as 'any untoward medical occurrence that at any dose results in death; is life-threatening; requires inpatient hospitalization or prolongation of existing hospitalization; creates persistent or significant disability/incapacity, or a congenital anomaly/ birth defect' (International Council on Harmonisation, 1994).

### Eligibility Criteria

Eligible patients were those who underwent treatment with biologics for PsA/RA for at least 6 months, during 2003–2011.

### Identification of Patients and Collection of Patient Data

To identify eligible patients, two researchers abstracted data from all the dispensing orders from the database of the government (CODES-SP). The patients with PsA were identified by ICD code M07, and those with RA were identified by ICD code M05. Patient details such as name, address, telephone number, gender, age, healthcare provider, type of biologic dispensed, and duration of treatment and diagnoses were collected.

We contacted these patients by telephone, and if they proved eligible and agreed to participate in the study, we conducted interviews by telephone using a questionnaire. The questionnaire included the following: name of the drug that patient was using for the treatment of PsA and RA; time of diagnosis of the disease; comorbidities; adverse drug reaction and whether it led to discontinuing medications; and whether patients was informed about the risk of taking such drugs.

Interviews were conducted by telephone using computerassisted telephonic interview technology with a microcomputer handset with headphones. This system allows recording and monitoring of the conversation. Research staff working in pairs independently recorded data from the interviews, with discrepancies (if any) resolved by the principal investigator (LCL). This interview approach was developed by local dermatologists and rheumatologists in accordance with the recommendations of Brazilian and others important guidelines. Each interviewer (pharmacists) received training on use of language, related to each question in the interview schedule.

In order to dispense the biologics drugs in the pharmacy of the government, all pharmacists monitoring the patients with a formal structured checklist for ADR validated by rheumatologist and dermatologists.

We crosschecked patients' reports with data obtained from pharmacy records and from the database of the government. If discrepancies were found between sources of information, we considered the information from the pharmacy records as definitive. Definitive information about the name of the biologic and the duration of its use was obtained from the pharmacy, and definitive information regarding the time of diagnosis, use of previous medicines, and laboratory results were obtained from the patient.

This study is a part of a protocol published elsewhere (Camargo et al., 2016; Silveira et al., 2014).

### ADR Reporting

We collected information regarding the reported ADRs to understand their onset, nature of the reaction (system or organ affected), causality (Naranjo et al., 1981), and severity (ICH, 1994). The Naranjo algorithm (Naranjo et al., 1981) provided guidance for establishing causality and ADRs were ranked in three categories (definite, probable, and possible). The ADRs ranked as "definite" and "probable" were classified as likely caused by the biological

agent. Pairs of reviewers (SBF, BCAB, FSDF, MCC) independently classified potential ADRs as "present" or "absent," and if present, classified them according to the causality classification. In case of disagreement a third reviewer (LL) provided adjudication.

### Predisposing Factors

We considered the following variables as possible predisposing factors for ADRs. For each factor, we priori postulated the direction of the possible effects: i) **age** (older [≥60 years] versus younger [19–59 years], with a higher risk in older) (Girolomoni et al., 2012), ii) **presence of comorbidities** (none versus one or more, with a higher risk in one or more illnesses) (Mazurek and Jahnz-Rozyk, 2012), iii) **diagnosis of more than one immunosuppressive disease** (only one versus more than one, with a higher risk for diagnostic with one or more) (Coates et al., 2013), iv) **concomitant use of other medications** (none versus one versus two or more, with a higher risk with more than one medication) (Girolomoni et al., 2012), v) **concomitant use of disease modifying anti rheumatic drugs—DMARD** (none versus one or more, with a higher risk with more than one DMARD) (Mazurek and Jahnz-Rozyk, 2012), vi) **physician provided warnings regarding risk of medication** (higher risk when warnings were provided) (Girolomoni et al., 2012), vii) **health insurance** (private versus public health insurance, with a higher risk for private insurance) (Mariette et al., 2011), viii) **biological agent**, iv) **duration of use of biologics** (6 to 12 months versus 13 months or more).

### Follow-Up and Clinical Monitoring

Clinical monitoring was done only in patients who were taking biologics during the interviews. We referred the guidelines of a few countries, such as Brazil (BRAZIL, 2014; BRAZIL, 2006), England (Smith et al., 2009), Canada (Bykerk et al., 2012), Germany (Wollenhaupt et al., 2013), European League against Rheumatism – EULAR (Gossec et al., 2016), Group for Research and Assessment of Psoriasis and Psoriatic Arthritis – GRAPPA (Coates et al., 2016) and took into account the common recommendations such as medical consults, laboratory blood examination, and radiograph.

As there is lack of consensus regarding the best interval for patient monitoring, we adopted the recommendations present in the Brazilian clinical protocols of the time, which considers as follows: **i) number of consults** (at least two annual medical consults); **ii) laboratory blood examination** (complete Blood Count, liver function tests and C-Reactive Protein test performed once a year) and **iii) radiography** (at least once a year). The IGRA sign test (interferon gamma release assay) was only introduced in public health service in 2014.

Owing to their high cost, biologics are only provided by government pharmacies, and therefore the follow-up was in accordance with the Brazilian official protocol. At the time of the study, annual radiography was considered as a test for monitoring tuberculosis progression, a condition common in Brazil, given the high number of patients acquired immunodeficiency syndrome (AIDS) in the general population.

### Statistical Analysis

Initially, we obtained the descriptive statistics of the variables studied through frequencies. Later, the variables were stratified by patients, with and without ADRs and SAEs. We calculated the prevalence ratio (PR) to detect factors associated with the risk of ADRs using a Poisson regression.

We used with bivariate analysis (unadjusted) as a first step, and then, the analysis was adjusted for age, presence of comorbidities and use of other concomitant drugs. A significance level of *p* < 0.05 and a confidence interval of 95% were adopted. All analyses were performed using STATA software.

### RESULTS

Of the 305 patients identified for using biologics for PsA or RA, 10 patients refused to participate, 13 were deceased and 14 used biological agent for less than 6 months. The resulting group of interviewees included, 268 plaintiffs of whom 158 (58.9%) were still using a biological agent at the time of the interview (**Figure 1**).

**Table 1** presents characteristics of the patients with PsA and RA. Most of the patients were female (73.1%), less than 60 years old (mean age 55.8 ± 13), with rheumatoid arthritis only (73.1%), with one or more comorbidity (51.5%), using the biologic for 13 to 36 months (mean duration 35.7 ± 20).

Patients with RA showed different comorbidities as compared to those with PsA. Approximately, 102 (52%) patients with RA showed comorbidities and 46 (45.1%) had cardiovascular diseases. Twenty-two (21.5%) patients had metabolic diseases (mainly diabetes and obesity), while 17 (16.6%) had muscle pain that rendered work disabilities. On the other hand, 16 (25%) patients with PsA showed comorbidities and 8 (50%) had cardiovascular disease, 6 (37.5%) had dermatological problems, 4 (25%) had inflammatory bowel disease, 3 (18.8%) had metabolic disease, 3 (18.8%) had musculoskeletal problems including osteoporosis and ophthalmic disorders was observed in 2 patients (12.5%).

**Table 2** depicts the characteristics of the adverse events in patients taking biological agents. Of the 268 patients, 116 (43.3%) experienced one or more adverse events related to the use of biological agents, at the rate of 1.6 events per patient. The most frequently [n = 73 (27.2%)] reported ADRs were administration site reactions (hypersensitivity reactions and cytokine-release syndrome), followed by incidences of respiratory and other types of infections [n = 30 (11.2%)] (opportunistic infections, urinary tract infections, skin infections, other systemic fungal infections and meninges infection)and effects on nervous system [n = 22 (8.2%)] (headaches and neuropathies). Of 116 patients with at least one ADR, 29 (25%) experienced SAE. The main causes for SAE included serious infections, malignancies and major cardiovascular events. There was no case of tuberculosis.

TABLE 1 | Characteristics of the patients with psoriatic arthritis and rheumatoid arthritis.


*\*Patients may have more than one comorbidity.*

*PsA, Psoriatic arthritis; RA, rheumatoid arthritis.*

We could not access the medical records to gain further details about these events. Eight patients required hospitalization, 6 had prolongation in existing hospitalization and 8 experienced lifethreatening events.

**Table 3** shows the characteristics of population of the study with respect to predisposing factors of ADR and the association with ADRs. Overall, rituximab showed less risk to ADR [PR 0.42, 95% CI (0.18–0.96); *p =* 0.04] than other agents. The factors such as age, private healthcare assistance, provision of information about risk of ADRs, showed no association with ADRs.

**Table 4** presents the clinical follow-up and outcome in patients with on-going biologic treatments. One hundred fifty-one (95.6%) patients visited a doctor at least once a year, however, 48 patient (30.4%) did not undergo the laboratory tests (complete blood count, liver function test, reactive protein test), while 75 (47.5%) did get radiography done, whereas 58 (36.7%) patients had at least two medical consultations, underwent a laboratory blood test at least once, and had a radiography examination once a year.

### DISCUSSION

### Main Findings

The typical long-term users of biological drugs in this study were women aged 19–59 years, with one or more comorbidities, using biological drugs for 13–36 months and having access to public healthcare. Around 40% of the patients using biologics drugs for a long-term, had one or more ADR related to these agents, with 1.6 events per patient. The most common ADRs were administration site reactions, infections and symptoms related to nervous system (headache and neuropathic pain). The occurrence of SAEs was less than 10% of all patients. Nevertheless, the majority of SAEs did not lead to drug discontinuation. Notably, no case of tuberculosis or mortality were detected. None of other risk factors studied were associated with ADRs. Adalimumab was the biological agent most used for the majority of patients and ADRs related with this drug included serious infections disease and injections site reactions. The use of rituximab was related with lower risk of ADR than other agents. Complete clinical follow-up was done by 36.7% of patients, implying that remain the patients, despite receiving government medicines failed to follow-up, according to the official Brazilian guidelines.

### Relation to Prior Literature

Biologicals have become potent and effective therapeutic alternative for many inflammatory and autoimmune diseases like RA and PsA, focus of this cohort population. Their direct and focused effect makes them superior to classic immunosuppressive, whose use is frequently limited by undesirable and often severe generalized adverse effect. Biologic agents targeting specific immune mediators have emerged as other treatment option for patients with RA, PsA and others immune disease who are unresponsive to, or intolerant of, non-biologic systemic agents. (Yazici, 2018; Michet, 2018).

Furthermore, conventional treatments for PsA have limited efficacy for nail disease, enthesitis or axial involvement, and some are unable to control moderate and severe peripheral joint and skin disease (Yazici, 2018). The introduction of biologic treatments offered the possibility of controlling multiple aspects of these diseases using a single drug, minimizing the need for additional therapies (Elyoussfi et al., 2016).

Although several biologics have demonstrated good efficacy and tolerability in short-term trials, treatment guidelines recommend them as third line therapies due to a relative lack of long-term safety data. Here, we have reviewed the long-term (>6 months) safety data. In our study, 35% of patients used biological agents for more than 37 months, reflecting the real scenario of its long-term use in Brazil.

Evidently, the harms of biologics must be balanced against their use benefits, when making a risk–benefit assessment of its use for a patient with systemic autoimmune conditions such as RA or PsA. Patients and physicians worry about risks including TABLE 2 | Characteristics of adverse events in patients taking biological agents to treat psoriatic arthritis and rheumatoid arthritis.


*A patient could have more than one event for system.*

*ADR, adverse drug reactions; SAE, serious adverse event.*

*\*Adverse events ranked as possible, probable and definite.*

*\*\*Adverse event ranked as only definite or probable.*

*¨Hypersensitivity reactions and cytokine-release syndrome.*

*+Opportunistic infections, infections of the urinary tract, infections of the skin, other systemic fungal infections and infection of the meninges.*

*&Headache and neuropathies.*

*&&Life-threatening: requires inpatient hospitalization or prolongation of existing hospitalization; creates persistent or significant disability/incapacity, or a congenital anomaly/birth defect' (ICH, 1994).*

de Camargo et al.

#### TABLE 3 | Association between predisposing factors and ADRs.


*\*ADR ranked as definite or probable.*

*Adjusted to: age, comorbidity and concomitant use of others drugs.*

*\*\*DMARDs not included.*

TABLE 4 | Clinical follow up and outcome judgment in patients with psoriatic arthritis and rheumatoid arthritis still taking biologics.


*efalizumab was removed from the market in 2009.*

*¹At least two annual medical consults.*

*²Laboratory blood tests performed at least once a year (CBC – Complete Blood Count; Liver function tests; CRP test – C-Reactive Protein test).*

*³Adiography performed at least once a year.*

*\*\*Clinical follow up was checked in pts still taking the medication during the interview (n = 158).*

not only common ADRs such injection site reactions but also infections that may be less common.

The clinical representation of ADR to biologicals can be ambiguous with regard to pathogenesis of the reaction, because different pathomechanisms may lead to similar symptoms. This is especially important for infusion reactions, whereby no clear clinical distinction between allergic, IgE-mediated, and the more frequent no allergic, most probably complementmediated reactions are possible. In our study, despite the long-term use of biological agents, mainly the anti-TNFα drugs (infliximab, etanercept and adalimumab) they still have reactions related with administrations. Infusions/Injections site reactions are a major complication of all anti-TNFα drugs with studies showing an incidence rate 3–40% (Henderson Berg and Carrasco, 2017).

The most common cutaneous side effects are injection site reactions, which are often defined as a constellation of symptoms, including swelling, erythema, pruritus, and pain around the site of injection (Zeltser et al., 2001; Clarke, 2010; Scherer et al., 2010). Administration site reactions can be divided into two types according to their mechanism of action: i. Type α - irritative reactions (immediate) commonly at the injection sites of subcutaneously administered biologics caused by proinflammatory actions of the substances (Corominas et al., 2014); ii. the Type hypersensitivity reactions categorized into Types I–IV, which are induced by IgE, IgG/IgM, complement or T-cells. Injection site reaction after etanercept injection produce a T-cell-mediated delayed-type hypersensitivity reaction, as approximately 8% of patients developed "recall injection site reaction," reactions at sites were medication was previously injected (Scherer et al., 2010).

The injections site reactions with etanercept, can occur in up to 37% of patients and characteristically these reactions consist of mild to moderate erythema, pain, pruritus and edema immediately evident or appear within 24–48 h and the mean duration of the reaction is 3–5 d and there is a gradual decrease in frequency and severity with continuation of injections (Kim et al., 2015; Murdaca et al., 2012). This didn't happen in the population of this cohort. Patients using adalimumab, etanercept and infliximab more than 6 months in our cohort had 36, 34 and 11% of infusion/injections site reactions. Data from the Hong Kong Biologics Registry that followed up 1,345 patients from 2005–2013 that reported the most frequent SAE (per 100-paties-year) was infusion/injection site reaction (0.75) (Mok et al., 2014).

We found that inappropriate injection techniques, injection close to blood vessels, the chemical and physical properties of the injected drug and a reaction to the vehicle component are several causes described in the literature resulting in irritative reactions (Corominas et al., 2014).

Severe infusion reactions, such as angioedema and shock, have been reported in patients under infliximab therapy. As infliximab is a chimeric human/mouse anti- TNF-α antibody, it may induce the synthesis of neutralizing antibodies which could reduce the efficacy of the drug. Therefore, methotrexate is usually co-administered to control both the rheumatic disease and the development of neutralizing antibodies (Murdaca et al., 2013). In spite of its fully human sequence, the production of antibodies to adalimumab has been also reported, which may reduce the efficacy of the drug and induce the development of adverse drug reactions and exanthema (Murdaca et al., 2012).

However, in the majority of cases the injections site reaction with adalimumab in our study were mild-to-moderate severity, and do not necessitated drug discontinuation.

Studies comparing the intravenous and subcutaneous route of administration of these two agents did not show any difference in clinical efficacy and safety, except that injection site reactions were more common with the subcutaneous access which is the case of adalimumab and etanercept use differently of infliximab that is indicate to infusion use (Gabay et al., 2013).

Few biologics are associated with a higher rate of some of the ADRs than others, and potential SAEs with short-term and long-term use (Singh et al., 2011). The safety of rituximab was consistent with earlier findings, which indicated that there was no increase in ADRs or SAE with its prolonged use (>5 years) (Van Vollenhoven et al., 2013; Winthrop et al., 2018).

This cohort is related to the use of long-term biologicals, which shows that the adverse effects may be different from those studies that indicate short-term adverse effects. The biologic use can increase the risk of serious infections in the first months of treatment with respect to disease-modifying antirheumatic drugs (DMARDs). Data from the British Society for Rheumatology Biologics Register (BSRBR) comparing the risk of serious infections between TNFi-treated patients and traditional DMARD-treated patients showed that the risk of serious infections with TNFi was increased in the first 6 months of initiating therapy for RA and that this risk was higher compared to traditional DMARDs (Galloway et al., 2011).

Another largest observational studies of infection in patients with autoimmune diseases found that compared to traditional DMARDs, the initiation of TNF-alpha antagonists was not associated with an increased risk of hospitalizations for serious infections (Grijalva et al., 2011).

Serious infections (such as infections requiring hospitalization or intravenous administration of antibiotics, opportunistic infections, including tuberculosis, systemic fungal infections and herpes zoster) were uncommon in population currently using the biological agents for more than six months.

Moreover, long-term use of biological could be related with non-serious infections, mainly of the upper respiratory tract. This ADR were common among the users of biologics in our study, endorsing the findings from earlier studies (Salliot et al., 2007; Winthrop et al., 2018).

Emerging data also suggest that the incidence of serious infections is dependent on, past history of serious infections, corticosteroids anti-inflammatory doses, and older age as important predictors of risk of serious infections in patients treated with biologics. The duration of treatment is also an important risk factor with the highest rate being observed during the first months of therapy and the risk of infection decreases over time (Singh, 2016). Our study included patients who used biologics for a prolonged period, which could explain the reduced number of opportunistic infections. No case of tuberculosis was reported.

A study in Latin America showed that the risk of serious infections may vary depending on the region and the characteristics of the patient (Ranza et al., 2019). This study compared a database not available publicly (BIOBADABRASIL) from Brazil versus a database of Argentina (BIOBADASAR) and showed that the risk of infections in Brazil was decreasing over time, corroborating our findings.

Nowadays, safety profile has changed mainly because we know more about the disease and about biologic agent. Furthermore, not only the function of these composites has to be understood, but also the subjacent immunology (which is very complex). Therefore, current patients do have less prolonged and severe chronic inflammation, key element for decrease of cardiovascular events or malignancy risk.

The delayed reactions related to dysfunction of the cellular response such as autoimmunity or cancer, may appear after many months or years of the cessation of the biologic therapy (Mazurek and Jahnz-Rozyk, 2012). However, recent cohort study showed no evidence of change in risk of solid cancer with increasing exposure to biologics in the first five years (Mercer et al., 2015).

The characteristics of the population of this study are similar to those found in studies carried out in Europe and USA (Sfriso et al., 2009; Schneeweiss et al., 2017). The most of patients were women, age around 50–55 years old, with at least one comorbidity.

Comorbidities in our study also were similar to those observed in other studies (Ruiz et al., 2014; Deus et al., 2015; Camargo et al., 2016; Ryan et al., 2011). Patients with RA had more cardiovascular diseases, metabolic diseases like type 2 diabetes mellitus and obesity, and muscle pain rendering in work disabilities. Patients with PsA also had more cardiovascular disease, metabolic disease, inflammatory bowel disease and autoimmune ophthalmic disease. Comorbidities in this study were not related to increase in risk of ADR, as found in patients with only psoriasis in other study (Lopes et al., 2014).

Considering the high costs of biologic therapies and their adverse events profile in long-term use, the follow up of the patients and individual monitoring is essential (Mazurek and Jahnz-Rozyk, 2012). The big problem here with the use of biologics is that the prescription is made by a private doctor and accessibility to the medicine is dependent on public service, disallowing appropriate monitoring.

Furthermore, the wide use of biological agents in modern medicine is a challenge in clinical practices, as it is a case of how fast new therapeutic principles based on novel knowledge and modern techniques can enter clinical practice, and that constant learning is required. Their use often requires a special knowledge and familiarity with the disease to be treated. Scientific data shows that ADRs to these drugs are clinically very heterogeneous. It makes clear, the monitoring of them seems essential (Pichler, 2006; Zemkova et al., 2007). Patients on biologic therapy should be monitored closely with routine blood tests, regular doctor's visit and outcome measures about effectiveness (considering the response to therapy), safety (presence of ADRs and SAEs) and quality of health for prolonged period (Emer et al., 2010; RCN, 2017).

There is a lack of data regarding the best interval for monitoring the patients. We would like to emphasize that we have adopted the recommendations of the Brazilian Clinical Protocols of the time. These are official government protocols, since these medications are only provided in government pharmacies. Most of the patients had medical consultations, and laboratory tests for liver function and blood cell counts, at least annually, according to the recommendations of several guidelines from England, Brazil, Canada, etc. (BRAZIL, 2006; Smith et al., 2009; Bykerk et al., 2012; BRAZIL, 2014; Michet, 2018, Coates et al., 2016; Gossec et al., 2016). The recommendations for radiography vary according to the adopted guideline. Usually, radiographic assessments are encouraged according to clinical manifestations and discretion of physician (Ritchlin et al., 2008). In this study, we decided to follow the Brazilian guideline, and the radiography was assumed as a test for monitoring the progression of tuberculosis.

The most appropriate way to monitor disease activity in PsA is under defined (Gossec et al., 2016). The recommended core set for PsA comprises of peripheral joints, pain, physical and global function assessment, quality of life and fatigue (Kimball et al., 2008). It is also recommended that radiographic monitoring for erosions and osteolysis of the hands and feet be done annually (Michet, 2018).

The effectiveness of the biologics in RA and PsA is unquestionable, but their association with potentially adverse effects can doubt the benefit risk ratio. Therefore, closer monitoring and education of patient and their caregivers about the nature of their condition, benefits and risks of treatment are essential for improving treatment outcome and overall patient satisfaction (Emer et al., 2010). The patients need to be advised to report any worsening of symptoms (neurological, cardiac, pulmonary, skin, uveitis and/or malignancies), avoid exposure to potential risk factors for infection, and to promptly communicate to their physician possible signs and symptoms of infection (RCN, 2017).

### Strengths and Limitations

The strengths of this study included an extensive questionnaire, with questions about previous treatments, diagnosis, comorbidities, and if any adverse effects and its interference with the current treatment. Besides that, our sample size was relatively large, we contacted 305 patients and interviewed 268. Our data was checked twice, as we obtained data about use of medication from the pharmacy records and then confirmed the information from patient interview to ensure its accuracy.

This registry study has mounting importance in medical research and decision-making processes. In fact, despite the inherent limitations of such studies, including the lack of randomization, the relatively high frequency of missing data, and the presence of patients with different diseases, registries usually include larger populations than clinical trials, and therefore have a higher power to detect rare adverse events. In addition, registry studies may better reflect clinical practice with respect to randomized clinical trials, whose results may not immediately be extended to "real-life". Today, several registries in Europe are collecting data on the use of biological Drugs: ARTIS (Antirheumatic Therapies In Sweden), BIOBADASER (Base de Datos de Productos Biológicos de la Sociedad Española de Reumatología, Spain), BSRBR (British Society for Rheumatology Biologics Register, United Kingdon), DANBIO (Danish Database for Biological Therapies in Rheumatology, Denmark), RATIO (Research Axed on Tolerance of Biotherapies, France) etc. Though most patients included in these registries are taking biologic agents for the treatment of RA/PsA, the safety information gained from these sources can be applied in clinical practices (Nikiphorou et al., 2017).

Brazil have had limited initiatives to build up an important national database linking with important clinical outcomes in RA/PsA/Psoriasis. There are databases restricted to a private insurance sustained by societies of rheumatologist and funding by pharmaceutical industry. An example of this is a multicentre prospective observational cohort in Brazil is the REAL (Rheumatoid arthritis in real life) that is following up 1,300 patients from 11 canters in 4 regions from Brazil, since 2015, funding by Bristol-Myers Squibb, Eli Lily and others pharmaceutical industries (Da Rocha Castelar-Pinheiro et al., 2018). Our study is one of the few studies in Brazil that had access to patients in public health sector, without any conflict of interest with long term follow-up.

Recall bias may have led to obtaining inaccurate information about previous treatments, side effects, and clinical monitoring. We try to reduce this limitation by only asking about the clinical monitoring to those who were still using the biological agents. Another limitation is the fact that ADR were not validated by a treating physician. However, some studies point that data reported by patients are potentially important to support and improve the care (Dawson et al., 2010; Nelson et al., 2015). Furthermore, in many countries, pharmacists are recognized as one of the most important healthcare providers in ADR reporting like in the Netherlands, Spain, Portugal, and Korea (Van Grootheest et al., 2004; Kim et al., 2010; Yu et al., 2016).

### Implications

This study showed the association of biological agents with ADRs, withdrawals due to adverse events and SAEs in long-term use. Patients using biological agents must be aware of these risks and should be subjected to careful monitoring throughout the treatment to prevent or at least treat a possible ADR. Our data

### REFERENCES


suggests that, patients need comorbidity warning of possible adverse events and recommended enhanced surveillance.

In the long-term, the possible risk of SAE requires caution and further monitoring and investigation. Therefore, review and further investigations of their safety are warranted.

### ETHICS STATEMENT

The protocol was authorized by the Health State Department and approved by the ethics committee for clinical research of University of Sorocaba on August 17, 2009, with protocol number 011/2009.

### AUTHOR CONTRIBUTIONS

LCL had the original idea and reviewed all of steps of the manuscript conception. MCC, IAC and MSNS collected the data. SBF, BCAB and FSDF performed the adverse reactions analysis and cross-checked the data. MTS performed data statistical analysis. BCAB and IF drafted the manuscript.

### FUNDING

The authors have received support from FAPESP (Foundation for Research Support of the State of São Paulo, process number 2009/530841).

### ACKNOWLEDGMENTS

Part of this work was shown as poster presentation in the European Drug Utilisation Research Group Conference 20171 . The correspondent author holds the copyright and has the permission for reuse of the published conference paper.


<sup>1</sup> Barros B, Camargo M, Silva MT, Castilho B, Mariano A, Lopes LC. Adverse events in patients with rheumatoid arthritis and psoriatic arthritis receiving long-term biological agents in a real-life setting. In: European Drug Utilisation Research Group Conference; 2017 Nov. 15–17; Glasgow, UK. p. 79.

Camargo, I. A., Barros, B. C. A., do Nascimento Silveira, M. S., Osorio-de-Castro, C. G. S., Guyatt, G., and Lopes, L. C. (2016). Gap between official guidelines and clinical practice for the treatment of rheumatoid arthritis in São Paulo, Brazil. *Clin. Ther.* 38, 1122–1133. doi: 10.1016/j.clinthera.2016.02.020


review. *Exp. Opin. Drug Safety* 14 (4), 571–582. doi: 10.1517/14740338.2015. 1009036


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 de Camargo, Barros, Fulone, Silva, Silveira, Camargo, Barberato-Filho, Del Fiol and Lopes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Physico-Chemical Characterization and Biopharmaceutical Evaluation of Lipid-Poloxamer-Based Organogels for Curcumin Skin Delivery

*Aryane Alves Vigato1, Samyr Machado Querobino2, Naially Cardoso de Faria1, Ana Carolina Bolela Bovo Candido3, Lizandra Guidi Magalhães3, Cíntia Maria Saia Cereda4, Giovana Radomille Tófoli4, Estefânia Vangelie Ramos Campos1,5, Ian Pompermayer Machado6, Leonardo Fernandes Fraceto5, Mirela Inês de Sairre1 and Daniele Ribeiro de Araujo1\**

#### *Edited by:*

*Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Sandra Donnini, University of Siena, Italy Narasimham L. Parinandi, The Ohio State University, United States*

#### *\*Correspondence:*

*Daniele Ribeiro de Araujo daniele.araujo@ufabc.edu.br; draraujo2008@gmail.com*

#### *Specialty section:*

*This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology*

*Received: 11 February 2019 Accepted: 08 August 2019 Published: 12 September 2019*

#### *Citation:*

*Vigato AA, Querobino SM, de Faria NC, Candido ACBB, Magalhães LG, Cereda CMS, Tófoli GR, Campos EVR, Machado IP, Fraceto LF, de Sairre MI and de Araujo DR (2019) Physico-Chemical Characterization and Biopharmaceutical Evaluation of Lipid-Poloxamer-Based Organogels for Curcumin Skin Delivery. Front. Pharmacol. 10:1006. doi: 10.3389/fphar.2019.01006*

*1 Human and Natural Sciences Center, ABC Federal University, Santo André, Brazil, 2 Department of Biomedical Sciences, State University of Minas Gerais, Passos, Brazil, 3 Research Group on Natural Products, Center for Research in Sciences and Technology, University of Franca, Franca, Brazil, 4 São Leopoldo Mandic Research Unit, São Leopoldo Mandic Faculty, Campinas, Brazil, 5 Department of Environmental Engineering, State University "Júlio de Mesquita Filho", Sorocaba, Brazil, 6 Department of Fundamental Chemistry, Institute of Chemistry, University of São Paulo, São Paulo, Brazil*

Organogels (ORGs) are semi-solid materials, in which an organic phase is immobilized by a three-dimensional network composed of self-organized system, forming the aqueous phase. In this context, lipid–Pluronics (PLs) ORGs form a two-phase system which can be effectively used as skin delivery systems, favoring their permeation across the skin. In this study, we presented the development of ORG skin drug-delivery systems for curcumin (CUR), a liposoluble phenolic pigment extracted from the turmeric rhizome. In special, we designed the formulation compositions in order to carry high amounts of CUR soluble in oleic acid (OA), as organic phase, entrapped into an aqueous phase composed of micellar PL-based hydrogels by associating two polymers with different hydrophilic–lipophilic balances, Pluronic F-127 (PL F-127), and Pluronic L-81 (PL L-81), to enhance the permeation across the skin. Results revealed that the incorporation of PL L-81 favored the CUR incorporation into micelle–micelle interface. CUR insertion into OA-PL F-127/L-81 reduced both G'/G" relationship (~16 x) and viscosity values (η\* ~ 54 mPa.s, at 32.5°C), disturbing the ORG network structural organization. *In vitro* permeation assays through Strat-M® skin-model membranes showed that higher CUR-permeated amounts were obtained for OA-PL F-127/L-81 (4.83 µg.cm−2) compared to OA-PL F-127 (3.51 μg.cm−2) and OA (2.25 μg.cm−2) or hydrogels (~1.2 μg.cm−2, p < 0.001). Additionally, ORG formulations presented low cytotoxic effects and evoked pronounced antileishmanial activity (IC50 < 1.25 µg.ml−1), suggesting their potential use as skin delivery systems against *Leishmania amazonensis.* Results from this study pointed out OA-PLbased ORGs as promising new formulations for possible CUR topical administration.

Keywords: organogel, pluronic, skin-delivery, curcumin, oleic acid

## INTRODUCTION

Curcumin (CUR) and its derivatives have shown a wide variety of biological activities, such as anti-oxidant (Dall'Acqua et al., 2016), anti-inflammatory (Zhu et al., 2016), anti-tumor (Han et al., 2011), antimicrobial (Cetin-Karaca and Newman, 2015), and antiparasitic effects (Morais et al., 2013), as well as for the treatment of ulcers (Magalhaes et al., 2009) and skin diseases (Patel et al., 2009; Rachmawati et al., 2015), among others (Aggarwal and Harikumar, 2009). Despite its efficacy and safety, CUR has not yet been approved as a therapeutic agent (Anand et al., 2008). In addition, due to its physico-chemical limitations such as low aqueous solubility and low bioavailability (Priyadarsini, 2014), several studies have been devoted to developing new pharmaceutical formulations to overcome those limitations. In fact, CUR extensive first-pass biotransformation and low aqueous solubility became an interesting molecule for skin delivery (Anand et al., 2008).

In this context, skin delivery is an important strategy for drug administration, since this procedure is non-invasive and avoids first-pass biotransformation and enables the use of self-administered pharmaceutical forms, improving patient compliance. However, the clinical efficacy of this type of administration depends on the drug physico-chemical and pharmacological properties, as well as its bioavailability at the site of action, which is limited by the low permeability of the stratum corneum (Godin and Touitou, 2007; Prausnit and Langer, 2008). Considering that most skin pathological processes occur locally, CUR topical application may offer the advantage for delivering the molecule into the site of action. Nanocarrier systems such as gels and nanoemulsions can therefore provide the chemical stabilization and permeation of the CUR molecule (Rachmawati et al., 2015).

Among the various nanostructured systems are those formed by the poloxamers or Pluronics® (PL), copolymers used in preformulations such as hydrogels and as aqueous phase of organogels (ORGs). Particularly, as recent hybrid systems, ORGs stand out as semi-solid colloidal systems that has an oil phase dispersed in an aqueous phase, being used as reservoirs for lipophilic molecules. Those systems present advantages over conventional formulations (creams, ointments, hydrogels) due to their ability to incorporate higher concentrations of lipophilic molecules (Patel et al., 2009; Esposito et al., 2018) such as CUR, being capable to modulate the time and rate of skin permeation according to their composition. In fact, lipid (oil or organic phase) and PL (aqueous phase) gels form a two-phase system which can be effectively used as skin delivery promoters for hydrophilic and lipophilic drugs, favoring their permeation across the stratum corneum. Additionally, ORGs present other advantages such as (i) adhesion to the skin due to the formation of a homogeneous film on the stratum corneum surface, (ii) increases the contact area with the application site, (iii) improves the chemical stability of incorporated molecules, and (iv) absence of organic solvents in formulation preparation, which increases the ORG biocompatibility (Vintiloiu and Leroux, 2008; Iwanaga et al., 2012; Esposito et al., 2018).

In this study, we have developed ORGs with organic phase (OP) composed of oleic acid (OA), a free fatty acid (C18H34O2) welldescribed as permeation enhancer, incorporated into an aqueous phase (AP) containing Pluronic F-127 (PL F-127) isolated or in association with Pluronic L81 (PL L-81). The PL physico-chemical features such as molecular weight (PL F-127 = 12,400 g.mol−1 and PL L-81 = 2,800 g.mol−1), hydrophilic–lipophilic balances (HLB, PL F-127 = 22 and PL L-81 = 2), and polypropylene oxide (PPO):polyethylene oxide (PEO) relationships (PL F-127 = ~1:3 PPO : PEO and for PL L-81 = ~7:1 PPO : PEO) provide a differential structural organization (Oshiro et al., 2014) to modulate the CUR permeation. Then, we have studied those ORG systems regarding to their physico-chemical, structural, and biopharmaceutical properties e.g., the micellization process, the sol–gel transition, rheological features, structural organization and, especially, the influence of OA-PL F-127/L-81 association on CUR permeation profile, photostability, citotoxicty, and its biological activity.

### MATERIALS AND METHODS

### Chemicals and Reagents

Pluronic® F-127 (PL F-127), Pluronic® L-81 (PL L-81), OA, and CUR were purchased from Sigma–Aldrich (St. Louis, MO, USA). All chemicals and solvents were analytical grade.

### High-Performance Liquid Chromatography (HPLC) Analysis

CUR analysis was performed using an HPLC system (Ultimate 3000, Chromeleon 7.2 software, Thermo Fisher Scientific, Waltham, USA) with a gradient pump, DAD detector, and C18 column (150 x 4.6 mm, 5 µm; Phenomenex). Drug samples were analyzed at 425 nm, 0.8 ml/min flow rate at 25°C. The mobile phase was composed by a mixture of acetonitrile and water with acetic acid (0.05%) solution (70:30). The drug retention time was 3.4 min. All results represent three experiments, in 3 days, performed in triplicate. The limits of detection (LD) and quantification (LQ) were determined from a standard curve of CUR at 10, 20, 40, 60, 80, and 100 µg/ml. The LD and LQ values were 0.31 and 0.94 µg/ml, respectively. CUR concentration was determined using the equation y = 2.616x ± 0.003 with correlation coefficient (R²) value of 0.998.

### ORG Preparation

Initially, the AP was prepared by mixing appropriate amounts of PL F-127 (20% wt) isolated or in association with PL L-81 (0.6%) in HPLC-grade water under ice bath with continuous stirring (350 rpm, for at least 12 h) until the solution became transparent. The hydrogels formulations, used in this study as AP, were previously designed and characterized (Oshiro et al., 2014). For OP preparation, CUR (2 mg) was added to 2 ml of OA under magnetic stirring (350 rpm, at 25°C) until the complete drug dissolution. Then, OP was added to the AP (1:4 v/v) and magnetically stirred (350 rpm) until the formation of an homogeneous gel (Boddu et al., 2014; Vigato et al., 2019). Finally, sodium benzoate (0.25% wt) was added to all formulations as a preservative. For comparisons regarding to morphology and permeation profiles, hydrogels were prepared at the same composition from ORG AP. All formulation compositions are presented on **Table 1**.



*Organic: aqueous phase ratio of 1:4 (OP : AP, v/v); CUR final concentration was 0.1%. H—indicates hydrogels formulations (without oleic acid); OA, oleic acid; PL F-127, Pluronic® F-127; PL L-81, Pluronic® L-81; CUR, curcumin.*

### Organoleptic Characterization, Drug Content, and pH Determination

The ORG formulations were evaluated by color, odor, and phase separation. The pH measurements were performed for all ORG formulations inserting the probe into the ORGs until the equilibrium determination. For drug content determination, samples of ORGs (0.05 g) were weighted, mixed with 10 ml of acetonitrile:water (70:30 v/v) and sonicated for ~ 20 min. Samples of 1 ml were filtered (nylon syringe filter, 0.22-µm pore) and analyzed by HPLC for determining CUR concentration. Drug content was expressed as a percentage.

### Structural and Morphological Analysis

ORG formulations were analyzed by atomic force microscopy (AFM). For samples preparation, a thin film of each formulation was disposed in a glass slide and dried and dripped on a silicon plate. Samples were analyzed in a Nanosurf easyScan 2 Basic microscope (Nanosurf, Switzerland) in non-contact mode in an instrument equipped with a TapAl-G cantilever (BudgetSensors, Bulgaria) operated at a scan rate of 90Hz. Images (256×256 pixels, TIFF format) were captured in time mode and were analyzed using Gwyddion software. In addition, the formulation structural properties were analyzed by X-ray diffraction (XRD) technique, using a Rigaku Miniflex II instrument with CuKα1 radiation (λ = 1.5406 Å). ORG samples were pressed against a glass sample holder to obtain a homogeneous surface and analyzed over the 5–70° 2θ range, employing a 0.05° step with 1 s of integration time.

### Differential Scanning Calorimetry (DSC) and Rheological Analysis

Differential scanning calorimetry (DSC) experiments were carried out by a Netzsch DSC Polyma Calorimeter (NETZSCH, Selb, Germany). ORG samples (20 mg) were placed in a sealed aluminum pan and analyzed by three

cycles (heating-cooling-heating) from 0 to 50°C at 5°C/min rate. Thermograms were presented as heat flux (J/g) against temperature (°C). For all analyzes, an empty pan was used as the reference.

For rheological analyses, an oscillatory Kinexus rheometer (Malvern Instruments Ltd., UK) with cone-plate geometry was employed. In order to determine the sol–gel transition temperature (Tsol–gel), the frequency was set at 1 Hz, and a temperature range from 10 to 50°C was used. Additionally, for frequency sweep mode, the temperature was kept at 32.5°C, and formulations were analyzed from 0.1 to 10 Hz. For both measurements, the oscillatory mode was used to obtain the elastic (G') and viscous modulii (G"), as well as viscosity (η\*) values for each formulation. Data were analyzed with the RSpace for Kinexus® software.

### *In Vitro* Permeation Studies

For *in vitro* permeation assays, vertical Franz-type diffusion cells (Vision Microette Plus; Hanson Research, Chatsworth, CA, USA) were used. The cells presented two compartments, donor (1.72 cm2 permeation area) and receptor (7 ml), separated by an artificial skin-model membrane (Strat-M® membranes, 25-mm discs, Millipore Co., USA, ultrafiltration membrane, 325 µm thick) (Uchida et al., 2015; Kaur et al., 2018). Each formulation (0.3 g/cm2) was applied to the donor compartment (in contact with the upper surface of the artificial membrane). The receptor compartment was filled with 7 ml of pH 7.4 sodium phosphate (5 mM) with sodium chloride (154 mM) buffer and magnetically stirred (350 rpm) at 32.5 ± 0.5°C for 48 h. During the time interval from 15 min to 48 h, aliquots (1 ml) from the receptor compartment were collected and analyzed by HPLC. All experiments were performed in triplicate. The cumulative amounts of permeated CUR were expressed as µg.cm−2, and the results were plotted as a function of time (h). For data analyzes, flux values were obtained from the slope of the curve over the 8-h period. Data were analyzed according to the equation (eq. 1):

$$J = P.Cd\tag{1}$$

where J (µg.cm−2.h−1) is the drug flux across the membrane, P (cm.h−1) is the permeability coefficient, and Cd (µg.cm−3) is the drug concentration in the donor compartment. The lag time was calculated by extrapolating a straight line to time axis (de Araujo et al., 2010).

### *In Vitro* Cytotoxicity and Antileishmanial Activity

Epidermal keratinocytes (HaCaT cell line, Thermo Fisher Scientific, Waltham, Massachusetts, USA) were used for the cytotoxicity experiments. Cells were seeded for 48 h in 96-well plates (2.104 cells/well), in Dubelcco's Modified Eagle Medium (DMEM; Gibco Laboratories, Grand Island, NY, USA) with 10% (v/v) fetal bovine serum (pH 7.2–7.4), humidified atmosphere at 37°C and 5% CO2) and 100 μg.ml−1 of penicillin/ streptomycin. For experiment design, ORG formulations were previously diluted in DMEM medium on concentration range from 10 to 100 mg.ml−1, and 200 µl from each solution were used for cell treatment during 24 h. Then, 100 µl of MTT solution (5 mg/ml, in phosphate buffered saline) was added to each well and incubated with cells for 4 h. After that, MTT solution was removed and 50 μl of DMSO added to the wells for 10 min. Absorbance was measured at 570 nm. For comparisons with non-toxicity, cells were treated only with DMEM at the same volume used for ORGs.

For pharmacological assays, in order to evaluate the antileishmanial activity, *Leishmania amazonensis* promastigote forms (MHOM/BR/PH8) were maintained in RPMI 1640 (Gibco) culture medium supplemented with 10% fetal bovine serum, penicillin (100 UI/ml), and streptomycin (100 μg/ ml). Subsequently, about 1 x 106 parasites were seemed in 96-well plates, and ORGs dissolved in culture medium were added at concentrations from 1.25 to 10 μg.ml−1 to the cultures. Amphotericin B (Sigma Aldrich Chem. Co., 97% purity) was added to cultures at concentrations ranging from 0.05 to 0.40 μg/ml and used as positive control. CUR was dissolved in dimethyl sulfoxide (DMSO) and added at same concentrations described before. Cultures were incubated at 25°C for 24 h, and the antileishmanial activity was determined by verifying the growth of the promastigote forms was inhibited, as revealed by counting the total number of live promastigotes using a Neubauer chamber according to the flagellar motility. The results were expressed as the mean of the percentage of growth inhibition relative to the negative control (RPMI 1640 medium+0.1% DMSO or RPMI 1640 medium). Two experiments were performed in triplicate. The 50% inhibitory concentration (IC50) values were determined by means of non-linear regression curves using GraphPad Prism version 5.0 software for Windows (GraphPad software, USA) (Bezerra et al., 2006).

### Statistical Analysis

Results were presented as mean ± standard deviation. For statistical comparisons, one-way analysis of variance (ANOVA) with Tukey–Kramer *post hoc* test was used. Statistical significance was defined as p < 0.05.

### RESULTS

### Structural and Morphological Characterization, pH, and Drug Content Determination

Before CUR incorporation, ORGs presented white-opaque aspect while hydrogels were colorless and transparent, as observed on **Figures 1 A**, **C**. After CUR incorporation, all formulations became yellow (opaque or clear, **Figures 1 B**, **D**), and neither particulate materials nor any phase separation were observed. For morphological characterization, AFM images revealed smooth surfaces with small and sparse protuberances for hydrogels (control formulations), as a result of the dry process before analysis. On the other hand, ORG morphology was characterized by wrinkles distributed for all surface, which can be attributed to the incorporation of the OP into the hydrogels forming a system with low water content resulting in a different morphology from hydrogels. Morphological differences between hydrogels and ORGs were also previously described using scanning electron microscopy (SEM) (Vigato et al., 2019). The ORGs were also characterized by X-ray diffraction (XRD), as shown in **Figure 2**. All formulations exhibited the same pattern, with two diffraction peaks at 19 and 23°, which are correspondent to the PL F-127 and crystalline structure (Shin et al., 2000; Saxena et al., 2012). The very broad diffraction peak observed in the diffractograms indicates that the ORGs also exhibit an amorphous character. This feature can be attributed to the presence of OA as OP,

FIGURE 1 | Micrographs for organogels and hydrogel formulations obtained from atomic force microscopy (AFM). (A) F-127/L-81-H, (B) F-127/L-81-CUR-H, (C) OA-F-127/L-81, and (D) OA-F-127/L-81-CUR. H—indicates hydrogel formulations (without oleic acid). OA, oleic acid; F-127, Pluronic® F-127; L-81, Pluronic® L-81; CUR, curcumin.

which disturbs the well-known crystalline nature of PL after incorporation into the formulations.

PL F-127, Pluronic® F-127; PL L-81, Pluronic® L-81; and CUR, curcumin.

The pH values for ORG formulations were from 5.5 to 5.9, for OA-PL F-127 and OA-PL F-127/L-81, even after CUR incorporation. Those results are in agreement with previous reports about other ORG compositions such as lanolin-PL F-127 (Vigato et al., 2019), ricinoleic acid-PL F-127 (Boddu et al., 2015), and lecithin-PL F-127 (Agrawal et al., 2010), reflecting no possible risk of skin irritation. For all formulations, the CUR content was ~ 95.2%, confirming the homogeneous drug distribution throughout the ORGs.

### Differential Scanning Calorimetry (DSC) and Rheological Analysis

All ORG formulations were analyzed regarding to micellization and sol–gel transition processes considering their initial (Tonset), peak (Tpeak), final (Tendset) phase transition temperatures; enthalpy change (ΔH); and rheological parameters such as elastic (G') and viscous (G") moduli, as well as viscosity (η\*). All results are presented on **Table 2** and **Figure 3**.

DSC analysis revealed that, in general, Tonset and Tpeak values were similar for all formulations, but small shifts were observed on phase transition temperatures in response to CUR and/or PL L-81, since Tendset values were reduced from 14.2 to 12.8°C after PL L-81 incorporation, while CUR insertion increased the Tendset value for 15.3°C, being observed only for OA-PL F-127/L-81. Regarding to enthalpy variation, a more pronounced CUR interference was observed for the systems composed of OA-PL F-127, since similar ΔH values were obtained before and after CUR incorporation into the OA-PL F-127/L-81. Those results reflect the CUR influence on the phase transition process, considering the possible drug dispersion into the ORG oil phase and the potential hydrophobic interactions between the OA carbon backbone (C18) and CUR.

The rheological parameters elastic (G') and viscous (G") moduli, as well as apparent viscosity (η\*), were determined for all ORG formulations. Additionally, the sol–gel transition temperature was also obtained in order to predict the possible influence of CUR incorporation and both PL on ORG AP composition, their compatibility and structural organization. In this context, ORG formulations were also analyzed specially


TABLE 2 | Temperatures (T), enthalpy variation (ΔH), and rheological parameters relative to the organogels phase transition before and after curcumin incorporation.

*Tonset, Tpeak, and Tendset represent the initial, peak, and final temperatures for phase transitions. G' (elastic) and G" (viscous) moduli, apparent viscosity (*η*\*), and sol–gel transition temperatures (Tsol-gel). OA, oleic acid; F-127, Pluronic*® *F-127; L-81, Pluronic*® *L-81; CUR, curcumin.*

according to the PL types forming binary systems into the AP (**Table 2** and **Figure 3**).

Rheological analysis revealed similar Tsol–gel values ranging from 14.8 to 15.9°C for isolated PL F-127 and its binary system with PL L-81, as well as after CUR incorporation. All ORG formulations presented viscoelastic behavior, being stable under temperature variation, since G' > G" values. However, the presence of CUR reduced the G'/G" relationships (from ~ 60 to 16), specially for the system OA-PL F-127/L-81. Similar effects were also observed on η\* parameter (**Table 2**), which can be attributed to the influence of CUR molecules into the gels three-dimensional network formed by the AP, disturbing their structural organization (Mady et al., 2016).

Regarding to the frequency sweep analysis, results revealed that the parameters G' and G" were not significantly affected by the applied frequency range, since G' > G" were observed for all formulations. After CUR incorporation, the system OA-PL F-127/L-81 presented the lowest G'/G" relationship value, compared to OA-PL F-127, showing the drug influence on ORG structural organization in addition to the presence of L-81 into the AP, as described before for other parameters such as viscosity and Tsol–gel.

TABLE 3 | Curcumin permeation parameters across Strat-M® membranes from organogel formulations.

(without oleic acid). a- OA-F-127/L-81 *vs*. F-127/L-81-H (PL-F-127-H; b- OA-F-127/L-81 *vs*. OA-F-127. \*\*\*p < 0.001.


*F-127, Pluronic*® *F-127; L-81, Pluronic*® *L-81; OA, oleic acid; H—indicates hydrogel formulations (without oleic acid). Data presented as mean ± SD, Statistical differences are expressed as: a- OA-PL F-127/L-81 vs. hydrogels (F-127/L-81-H and F-127-H); b- OA-F-127/L-81 vs. OA or OA-F-127. \*\*\*p < 0.001 and \*\*p < 0.01.*

### *In Vitro* Permeation Studies

In order to characterize the CUR permeation from ORG formulations, experiments were performed using Strat-M® artificial membranes as barrier. In addition, hydrogels composed of each AP used for preparing the ORGs were also included as formulations, for evaluating the influence of OA- and PL-type on CUR permeation profiles. Results from those assays are summarized on **Figure 4** and **Table 3**.

Results from permeation experiments across Strat-M® presented different profiles according to the presence of OA and the AP composition. All formulations showed CUR gradual permeation during the experiment (48 h), and no different profiles were observed until 12 h. However, from this time point, formulations were segregated in different profiles where higher CUR-permeated concentrations were obtained for the systems OA-PL F-127 and OA-PL F-127/L-81 than those determined for OA, PL F-127-H, and PL F-127/L-81-H. In fact, CUR permeation rate was significantly enhanced (p < 0.001) by OA-PL F-127/L-81 (4.83 µg.cm−2) compared with OA-PL F-127 (3.51 μg.cm−2), OA (2.25 μg.cm−2), PL F-127-H (1.16 μg.cm−2), and PL F-127/L-81-H (1.21 μg.cm−2) (**Figure 4**). Similar results were also observed after comparisons among the parameters drug flux and permeability coefficient for OA-PL F-127/L-81, which were significantly lower in relation to hydrogels (PL F-127-H and PL F-127/L-81-H) and OA-PL F-127, with p < 0.001 and p < 0.01, respectively (**Table 3**). On the other hand, latency times were very close for all formulations (from 3.14 to 3.66 h), and no statistical differences were observed, indicating a possible drug retention into the formulation.

### *In Vitro* Cytotoxicity and Antileishmanial Activity

*In vitro* cytotoxicity assays were carried out in order to assess the effects of the vehicle OA, and the formulations (OA-PF F-127 and OA-PL F-127/L-81) in keratinocytes from HaCat cell line, evaluated by the MTT reduction test. All results are presented on **Figure 5**.

The increase on OA concentrations evoked toxic effects to the cells. However, the cell treatment with both ORG formulations did not induce pronounced cell toxicity. In addition, OA-PF F-127 presented the lowest cytotoxic effects (p < 0.001), with percentages ranging from 100 to 73.6%, compared to OA-PL F-127/L-81 (from 87.1 to 45.5%) and OA (from 87.4 to 10.1%), since the incorporation of OA into the PL-based AP reduced its cytotoxicity, being associated to the highest cell viability percentages. In addition, the reduced cell viability percentage after PL L-81 incorporation can be due to its lower HLB value

FIGURE 5 | Effects of organogels formulations on HaCat cells determined by MTT reduction test. Data expressed as mean ± standard deviation with n = 6 replicates/concentration). F-127, Pluronic® F-127; L-81, Pluronic® L-81; OA, oleic acid. a- OA-F-127 and OA-F-127/L-81 *vs*. OA; b- OA-F-127 *vs*. OA-F-127/L-81. \*\*\* p < 0.001.

TABLE 4 | Percentage of inhibition of growth *L. amazonensis* promastigote forms/concentrations (µg.ml−1).


*Results were expressed as the mean percentage of growth inhibition relative to the negative control. Two experiments were performed in triplicate. The 50% inhibitory concentration values (IC50) and 95% confidence limits (in parenthesis) were determined by non-linear regression curves. Positive control: amphotericin B, Amph. B; negative control: medium RPMI +0.1% dimethylsulfoxide. F-127, Pluronic® F-127; L-81, Pluronic® L-81; OA, oleic acid.*

compared to PL F-127, which possibly enhanced the PL cell membrane partitioning.

Additionally, the potential leishmanicidal activity was also investigated. All ORGs were incubated with *L. amazonensis* promastigote forms during 24h and their effects compared with amphotericin B, as positive control. In general, all ORG formulations OA, OA-PL F-127, and OA-PL F-127/L-81 evoked pronounced antileishmanial activity (IC50 < 1.25 µg.ml−1) before and after CUR incorporation. Additionally, it is necessary to highlight that all formulations presented lower IC values compared with isolated CUR (IC50 = 9.45 µg.ml−1), as observed on **Table 4**. Even considering that low CUR concentration can be permeated from ORGs, all formulations exhibited leishmanicidal effects suggesting their potential use as skin delivery systems against *L. amazonensis*.

### DISCUSSION

Natural products have been reported as a source of medicines for thousands of years. Since the discovery of pure compounds as bioactive molecules, the art of exploring natural products has become part of the molecular sciences. Many drugs used to treat different pathologies have been extracted from plants, and CUR has a long application for the treatment of several pathological processes including inflammatory, immunogenic, wound healing, and infectious conditions (Priyadarsini, 2014; de Moraes, 2015; Vaughn et al., 2016).

Despite its extensive pharmacological activities, CUR presents physico-chemical limitations, such as low aqueous solubility and bioavailability, reducing its permeation across the skin, since the clinical efficacy of bioactive molecules administered by topical route depends mainly on their physico-chemical and pharmacological properties, as well as their bioavailability at the site of action, which is limited by the low permeability of the stratum corneum. Then, due to the special structure and skin properties, novel formulations, such as lipid-based ORGs, have been developed in attempt to overcome those limitations.

In this study, we have presented the development, physicochemical characterization, and biopharmaceutical evaluation of ORGs for CUR skin-delivery. In special, we designed the formulation compositions in order to carry high amounts of CUR soluble in OA (OP) and, then, entrapped into a tridimensional PL-based micellar AP, associating two polymers with different HLB values (PL F-127 and PL L-81) for promoting the permeation enhancement across the skin.

In this context, comparisons between the systems OA-PL F-127 and OA-PL F-127/L-81 suggest that the differences on thermodynamics parameters can be attributed to the presence of L-81 on AP. Since, PL L-81 is more hydrophobic (HLB = 8) compared to PL F-127 (HLB = 22), this feature can favor the possible CUR incorporation into micelle–micelle interface, as previously described by other hydrophobic drugs (Sharma et al, 2008; Esposito et al., 2018; Vigato et al., 2019). Additionally, those observations can suggest that different endothermic processes are capable to promote changes on ORG structural organization, particularly considering variations on aqueous and/or OP compositions.

As expected for hydrophobic molecules, such as CUR (log P = 3.62), there is a high partition on OP, but is possible that these molecules could be interacting with micellar central propylene glycol hydrophobic blocks from PL molecules, especially for the system composed by the association PL F-127/L-81. Additionally, it is necessary to point out CUR reduced both G'/G" relationship and η\* values, as an indicative of a disturbance on the ORG network structural organization caused by the CUR molecule insertion into the system. Those results are also in agreement with previous reports describing the structural organization of PL-based systems for delivering hydrophobic molecules (Sharma et al., 2008; Sharma et al., 2018; Nascimento et al., 2018; Vigato et al., 2019), also corroborating the calorimetric results. Although this effect has been observed, ORG formulations maintained the high G'/G" ratio, an important feature to obtain adequate spreadability, forming a thin film on skin, but without loss of the formulation structural organization and potentially prolonging the contact time with the skin.

Despite the differences between *in vitro* (using artificial membranes) and *ex vivo* skin permeation profiles, Strat-M® has been used as skin model for evaluating the drug diffusion profiles from new delivery systems during the early stages of the development. In fact, its lipid matrix composition (ceramides, free fatty acids, cholesterol, and phospholipids) can simulate the skin barrier, being useful for determining permeation parameters for different types of pharmaceutical formulations such as hydrogels, nanoparticulate systems, emulsions, and ORGs (Uchida et al., 2015; Simon et al., 2016; Haq et al., 2018; Grillo et al., 2019; Vigato et al., 2019).

Several studies have been reported regarding the development of new skin delivery systems for CUR such as monoolein aqueous dispersion and lecithin ORGs (Esposito et al., 2014), liquid crystalline systems composed of OA, polyoxypropylene/polyoxyethylene cetyl alcohol (Fonseca-Santos et al., 2016), Pluronic F-127/P-123 micelles (Akbar et al., 2018), Pluronic F-127 hydrogel (Yen et al., 2018), and methoxy poly (ethylene glycol)-block-poly (ε-caprolactone) (MPEG-PCL) hydrogels (Zhou et al., 2019) for antioxidant, anti-inflammatory, antileishmanial, and wound-healing purposes. However, the influence of structural parameters on drug permeation, the presence of OA, and association of polymers on AP have been not discussed. In this context, we can postulate that the high permeated CUR amounts from the system OA-PL F-127/L-81 can be attributed to some structural factors: (i) the hydrophobic interactions between CUR and OA, into the OP, promoting the drug solubilization and acting as a permeation enhancer; (ii) the association of L-81 into the AP that, possibly, allowed the CUR interaction with its PPO hydrophobic units on organic-AP interface and also promoted CUR incorporation into Strat-M® lipid matrix; and (iii) the formation of a more fluid ORG system, OA-PL F-127/L-81, presenting lower viscosity and G'/G" relationship, compared to OA-PL F-127, but capable to maintain the formulation in contact with the skin-membrane model and enhance the drug permeation. Additionally, different PL-based systems have been used as new therapeutic strategies for several purposes, such as leishmaniosis treatment. Recent reports described formulations based on PL F-127 micelles encapsulating a naphthoquinone derivative (Mendonça et al., 2019), clioquinol (Tavares et al., 2019), amphotericin B (Mendonça et al., 2016), and PL F-68 micellar systems for amphotericin B (Espuelas et al., 2000). In this study, we present the development of lipid-PL formulations containing CUR; in particular, OA has been described as an important component of skin formulations for enhanced efficacy on leishmaniosis treatment (Pinheiro et al., 2016), as well as due to its involvement on transition from promastigotes to amastigotes (Bouazizi-Ben et al., 2017). In this context, further experiments will be necessary in order to evaluate the performance of each formulation and its isolated components on proliferation process for both promastigote and amastigote host-cell stages as well as on different *Leishmania* strains. In summary, results from this study pointed out OA-PL-based ORGs as promising new formulations for CUR skin delivery with potential pharmacological activity against *L. amazonensis*.

### AUTHOR CONTRIBUTIONS

AV, NF, SQ and IM were responsible for physico-chemical characterization experiments, summarized the data and wrote the manuscript. EC and LF performed microscopy analysis. CC and GT were responsible for cell culture assays. AC and LC carried out antileishmanial activity assays. MS and DA contributed to the design, review and wrote the manuscript.

### FUNDING

This research work was supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP 2014/14457- 5, 2018/04036-3, 2018/02482-6, 2016/18045-9, 2016/24456-1),

### REFERENCES


Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq 309207/2016-9, 402838/2016-5, 303946/2018-0) and UFABC Multiuser Central Facilities (CEM-UFABC).

### ACKNOWLEDGMENTS

The authors are grateful to Prof. Hermi Felinto de Brito (Institute of Chemistry, University of São Paulo) and Maria Claudia França da Cunha Felinto (Nuclear and Energy Research Institute, São Paulo, SP, Brazil) for their support on X-ray diffraction experiments.


a transcriptomic approach. *Mol. Biochem. Parasitol.* 187, 91–97. doi: 10.1016/j. molbiopara.2012.11.006


*in vivo-in vitro* correlation. *Int. J. Pharm.* 512, 234–241. doi: 10.1016/j. ijpharm.2016.08.052


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Vigato, Querobino, de Faria, Candido, Magalhães, Cereda, Tófoli, Campos, Machado, Fraceto, de Sairre and de Araujo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Outcome Measures Used in Ocular Gene Therapy Trials: A Scoping Review of Current Practice

*Jasleen K. Jolly1,2,3, Holly Bridge2 and Robert E. MacLaren1,3\**

*1 Nuffield Laboratory of Ophthalmology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 2 Wellcome Integrative Neuroimaging Centre, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 3 Oxford Eye Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom*

Multiple gene therapy trials are occurring for a variety of ophthalmic diseases around the world. The safety of gene therapy in the eye has been established, and the next step is to reliably assess efficacy. This is primarily done through the use of imaging techniques and visual function measures. Standardized visual function assessments, however, were originally developed for a clinical setting and may not be suitable for detecting and quantifying therapeutic changes. This scoping review takes a comprehensive look at current practice in terms of the outcome measures defined at trial registration. These were compared to the outcome measures reported in the literature. All published trials reported the pre-registered primary outcome measure. A range of additional secondary outcomes were reported that were not originally planned. Gaps in gene therapy assessment exist and further discussion are required to find a way forward, particularly as more conditions progress to phase 2 and 3 trials. Several factors impacting on trial design and outcome measure choice are discussed.

Keywords: clinical trial, gene therapy, genetic eye disease, outcome measure, retinal imaging, vision, visual function

## INTRODUCTION

The eye presents the perfect organ for gene therapy. It is an immune privileged site, which is protected by the blood retinal barrier. The target cells, such as photoreceptors and retinal pigment epithelium are frequently non-dividing, meaning any intervention is likely to last for life. The different structures in the eye can be visualized due to the optical clarity inherent in the eyeball or can be imaged with well-documented techniques (Zysk et al., 2007; Fleckenstein et al., 2014). The structures in the eye can be targeted by various surgical procedures. Finally, many disease processes have a degree of symmetry. This means that, when treating one eye, the other eye can act as a control for comparison. For these reasons, ocular gene therapy is being trialed as an experimental treatment for an increasing number of conditions. There are established techniques to measure both structural and functional changes, with work ongoing in this field to evaluate the different diseases being treated. The success of gene therapy can be determined by the pattern of change seen in visual function measurement. Since visual function is the major marker for success of gene therapy, it is critical to establish guidelines for best practice.

Gene therapy can follow several different strategies. Most commonly, it is the supplementation of a defective gene with a working copy in affected target cells, as happens in achromatopsia and choroideremia. However, in some cases, such as with neovascular age-related macular

#### *Edited by:*

*Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Kurt Neumann, Independent researcher, Kerékteleki, Hungary Stephen Tsang, Columbia University, United States*

*\*Correspondence: Robert E. MacLaren enquiries@eye.ox.ac.uk*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 25 December 2018 Accepted: 23 August 2019 Published: 18 September 2019*

#### *Citation:*

*Jolly JK, Bridge H and MacLaren RE (2019) Outcome Measures Used in Ocular Gene Therapy Trials: A Scoping Review of Current Practice. Front. Pharmacol. 10:1076. doi: 10.3389/fphar.2019.01076*

degeneration, the gene expression may introduce a factor to help dampen the disease response. The mode of action is less important than the disease being investigated in determining the appropriate measures to use for trial monitoring. The Monaciano Symposium identified the measurement of treatment outcome as an area requiring priority review in order to aid the robustness of clinical interventional trials (Thompson et al., 2015). It calls for the investigation into appropriate outcome measures for each disease to measure structure and function without adding an unreasonable burden on the patient. They propose a standardization of testing protocols and data analysis. The reproducibility and reliability of tests should also be pre-defined.

With the increase in the number of trials using ocular gene therapy, the importance of adequate outcome measures is gathering interest. The success of gene therapy relies on three key aspects. The viral vector is developed over several years and optimized in animal models before reaching human trials (Koilkonda et al., 2014; Patrício et al., 2017). Much work has been conducted on optimizing the delivery of the therapeutic vector (Salvetti et al., 2017; Xue et al., 2017). Another component required for success of clinical trials is the adequate measurement of therapeutic impact. This requires a combination of the evaluation of ocular structure *via* imaging, and measurement of visual function. Standardized clinical visual function measures were largely developed for use in a clinical setting rather than for the assessment of novel interventions and may not always be adequate for measurement of a therapeutic effect. For example, the 100-hue test for color vision has wide normative ranges, making interpretation of longitudinal data difficult (Kinnear and Sahraie, 2002). The relationship of the outcome measures to disease progression, and therefore, the therapeutic window should also be better understood to interpret clinical trial findings. In addition, disease features such as visual field loss may make the conduct the test difficult.

A systematic review of gene therapy for retinal disease has been registered on the PROSPERO database (CRD42017056500) by London City University, but not yet completed. This specifies visual outcome as the outcome measure for assessing the success of trials, but the type of vision measure being looked at is not detailed, demonstrating the importance of providing further guidance on this topic. Additionally, as bilateral gene therapy will become more common, there will no longer be a control eye to provide a comparison as is done in many phase 1 trials, making vision outcome even more critical (MacLaren, 2016). Visual function is a combination of many aspects of vision, including detail, color, contrast, speed of vision, and night vision. The objective of this paper is to review the outcome measures listed and published for registered gene therapy trials in order to establish current practice, and to consider the scope for development of relevant outcome measures.

### METHOD

All clinical interventional trials must be registered on a publically available database. The databases on Clinicaltrials.gov (RRID: SCR\_002309), EU clinical trials register (RRID SCR\_005956), and the NIH clinical trials register were searched for all registrations by the end of October 2018, using the following search terms: gene therapy, subretinal injection, intravitreal injection, STX eye trial, Nightstar, Applied genetic, MeiraGTx, Hemera, Oxford Biomedica, Sanofi, Spark, ProQR, GenSight, and Genzyme.

Duplicate records were omitted from analysis. Natural history studies or studies specifically for long-term follow up of patients in a previous trial were also excluded in order to focus on the primary interventional trials. We then searched for any results from studies with a registered start date of greater than 12 months before October 2018. This was done *via* PubMed, study group websites, and Scopus. Searches were conducted using the investigator details and registered study name. Publications for the same study were grouped together and analyzed as an integrated dataset, with discrepancies between the primary outcome measure on the clinical trials record *versus* the final publications being noted.

### RESULTS

### Listed Outcome Measures

We identified 50 unique clinical trials on the registers for 17 ophthalmic indications (**Supplementary Figure**). Lebers congential amaurosis, Leber hereditory optic neuritis, and choroideremia are the only conditions currently in phase 3 trials. Outcome measures were analyzed according to clinical trial phase and were separated into four categories: safety, validated tests, novel test methods, and non-specific (**Supplementary Table**).

Visual acuity was included in almost all studies as either a primary or a secondary measure. Various forms of perimetry also featured highly in the outcome measures list. Out of the 50 trials, 16 used broad descriptors which did not make clear what data were being collected or how it was going to be used. This included descriptors such as visual function or specifying imaging techniques with no details of the aspect of the images to be examined. One did not specify any outcome measures.

### Published Trials

One trial marked as completed has not yet been published (NCT00001735). Eighteen trials have associated results in the peer-reviewed literature. **Table 1** details the correlation between the trial register record and the outcomes reported in the peerreviewed papers. Where primary and secondary outcomes were explicitly stated in the paper, these were recorded if also reported in the results. Anything not reported in the results or supplemental sections was not counted. In reports where primary or secondary was not made clear, all measures reported were recorded, and primary *versus* secondary was inferred from emphasis and context.

There was 100% compliance with reporting on the prespecified primary outcome measures. Sixteen trials prespecified one or more secondary outcome measure. These were met in full by 68% (11) trials. Five trials did not meet all of the


TABLE 1 | Comparison of clinical trial record and published outcomes in ocular gene therapy trials.

(*Continued*)

#### TABLE 1 | Continued


*References provided in brackets. AF, fundus autofluorescence; AMD, age-related macular degeneration; ERG, electroretinogram; FST, full-field stimulus threshold; IOP, intraocular pressure; OCT, ocular coherence tomography; RNFL, retinal nerve fiber layer; VA, visual acuity; VEP, visual evoked potential.*

outcome measures but did meet some of them. In addition, 83% (15/18) trials reported additional outcome measures, including a range of features, such as retinal imaging, aspects of visual function, and fMRI imaging. **Figure 1A** demonstrates the number of additional secondary outcomes carried out in the published literature but not originally included in the register as a histogram. **Figure 1B** shows the array of the visual function tests reported. VA was the most commonly used assessment of visual function. Perimetry was also commonly used but could take several different forms; each of which is targeting different areas of the visual field. Mobility testing is not standardized and appears in almost half the published trials. Electroretinograms are measured in a similar number of trials but are standardized due to ISCEV standards. The tests listed in low number of trials are generally non-standardized exploratory techniques such as fixation analysis and darkadapted perimetry.

### DISCUSSION

Adverse events are a key part of phase 1 trials as would be expected. Visual acuity is also a frequent factor in determining treatment effects and is reported in 94% of trials. It is a widely accepted measure, both clinically and by medical regulatory authorities. VA has been reported to have higher variability in low vision patients so strategies to optimize VA measurements in patients with disease need to be better established (Kiser et al., 2005).

Even within trials for the same disease, visual outcome measures used differ across sites, making direct comparison difficult. This is especially problematic due to the small numbers of patients involved in these highly specialized trials. The ideal way forward would be to conduct formal natural history studies and additional validation studies of novel outcome measures where existing measures are not sufficient or appropriate. The length of natural history studies should be determined by the nature of the disease being investigated. Fast progressing conditions will require a shorter follow-up period of 1 or 2 years. Slower progressive conditions should ideally have a longer follow-up period. An initial audit of imaging and functional data collected in the clinical environment can provide guidance on the speed of progression as a starting point such as conducted by Jolly et al., 2016 and 2017 in choroideremia (Jolly et al., 2016, Jolly et al., 2017). Combining structural and functional data will be helpful in better understanding the disease process as well as treatment impact in both natural history trials as well as in final outcome measures chosen.

Outcome measures should ideally be based on the biology of the disease and related to measurement of the therapeutic target within the eye, in order to maximize the chance of measuring a real therapeutic effect. This may change in end-stage disease *versus* trials designed for early disease states. The balance of structural *versus* functional measures is likely to change in late stage *versus* early stage disease. If the novel outcome measures are established in the disease prior to the interventional trials, the data can be submitted to the regulatory authorities in

advance of the interventional trials. This would increase the acceptability of these measures. Outcome measures should be based on an understanding of the underlying disease process as well as the impact of the gene therapy as determined by the expected impact of the viral vector and cells likely to be transfected. A significant advantage of this approach would be the likelihood of reaching the final outcome more quickly due to the use of targeted and sensitive markers of disease.

Patient quality of life is highly dependent on their perception of the world. Subjective assessment using validated instruments can provide insights into visual perception from the patients' perspective and can be considered as part of the battery of outcome measures (De Boer et al., 2004). Defining success based on clinical (such as repeatability) and patient (such as improvement required for greater quality of life) factors will make a stronger case for the success of therapy, particularly for phase 2 and 3 trials (Mcglothlin and Lewis, 2014). Although questionnaire results may be considered biased due to the patients motivation, the subjective feedback can provide very powerful evidence for the real world impact of any therapy in a way that clinical measures are unable to achieve. Many funding bodies in the United Kingdom encourage the use of patient and participant involvement in research as the insights they can provide can have an influence on guiding researchers to improved clinical trial design, as well as impact when reporting results (Boote et al., 2011).

Despite visual function being highlighted as an important factor in the success of gene therapy trials, little progress has been made on developing a coherent approach worldwide (Thompson et al., 2015). Other fields have highlighted similar issues for gene therapy (Lähteenvuo and Ylä-Herttuala, 2017). As more diseases are targeted by a gene therapy approach, and trials progress to phases 2 and 3, this will become ever more important. Cataract formation is a side effect of the invasive vector delivery techniques (Gupta et al., 2007; Hasler et al., 2015). Moreover, patients are followed up over long periods of time, increasing the chance of age-related cataract formation. Thus, it is necessary to account for the effect of cataract on visual function measures to ensure that any deficits do not interfere with determining the impact of gene therapy. Otherwise, results may be skewed in a negative way masking the therapeutic effect. Greater investment is needed in exploring disease parameters in more detail in order to complete gene therapy trials in an effective, timely, and cost-effective manner. This review provides an important starting point for clinical trial design.

## AUTHOR CONTRIBUTIONS

JJ and RM contributed conception and design of the study. JJ performed the data analysis and wrote the first draft of the manuscript. HB and RM contributed to manuscript revision, read and approved the submitted version.

### FUNDING

This study is funded by the National Institute for Health Research (NIHR) [Clinical Doctoral Research Fellowship CA-CDRF-2016-02-002 for Jasleen K Jolly]. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The sponsor and funding organization had no role in the design or conduct of this research.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01076/ full#supplementary-material

### REFERENCES


ocular subretinal injection of adeno-associated virus gene vector: results of a phase I trial. *Hum. Genet.* 135 (3), 327–343. doi: 10.1007/s00439-016-1637-y


**Conflict of Interest Statement:** RM is the scientific founder of Nightstar Therapeutics, receives grant funding from Nightstar Therapeutics, and is a consultant to Nightstar Therapeutics and Spark Therapeutics.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Jolly, Bridge and MacLaren. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances

*M Sanni Ali1,2,3\*, Daniel Prieto-Alhambra2,4, Luciane Cruz Lopes5, Dandara Ramos3, Nivea Bispo3, Maria Y. Ichihara3,6, Julia M. Pescarini3, Elizabeth Williamson1, Rosemeire L. Fiaccone3,6,7, Mauricio L. Barreto3,6 and Liam Smeeth1,3*

*1 Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, United Kingdom, 2 Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), Center for Statistics in Medicine (CSM), University of Oxford, Oxford, United Kingdom, 3 Centre for Data and Knowledge Integration for Health (CIDACS), Instituto Gonçalo Muniz, Fundação Osvaldo Cruz, Salvador, Brazil, 4 GREMPAL Research Group (Idiap Jordi Gol) and Musculoskeletal Research Unit (Fundació IMIM-Parc Salut Mar), Universitat Autònoma de Barcelona, Barcelona, Spain, 5 University of Sorocaba–UNISO, Sorocaba, São Paulo, Brazil, 6 Institute of Public Health, Federal University of Bahia (UFBA), Salvador, Brazil, 7 Department of Statistics, Federal University of Bahia (UFBA), Salvador, Brazil*

#### *Edited by:*

*Ileana Mardare, Carol Davila University of Medicine and Pharmacy, Romania*

#### *Reviewed by:*

*Bogdan Ileanu, Bucharest Academy of Economic Studies, Romania Douglas Faries, Eli Lilly, United States*

> *\*Correspondence: M Sanni Ali Sanni.ali@lshtm.ac.uk; sanni.ali@ndorms.ox.ac.uk*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 18 April 2019 Accepted: 31 July 2019 Published: 18 September 2019*

#### *Citation:*

*Ali MS, Prieto-Alhambra D, Lopes LC, Ramos D, Bispo N, Ichihara MY, Pescarini JM, Williamson E, Fiaccone RL, Barreto ML and Smeeth L (2019) Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances. Front. Pharmacol. 10:973. doi: 10.3389/fphar.2019.00973*

Randomized clinical trials (RCT) are accepted as the gold-standard approaches to measure effects of intervention or treatment on outcomes. They are also the designs of choice for health technology assessment (HTA). Randomization ensures comparability, in both measured and unmeasured pretreatment characteristics, of individuals assigned to treatment and control or comparator. However, even adequately powered RCTs are not always feasible for several reasons such as cost, time, practical and ethical constraints, and limited generalizability. RCTs rely on data collected on selected, homogeneous population under highly controlled conditions; hence, they provide evidence on efficacy of interventions rather than on effectiveness. Alternatively, observational studies can provide evidence on the relative effectiveness or safety of a health technology compared to one or more alternatives when provided under the setting of routine health care practice. In observational studies, however, treatment assignment is a non-random process based on an individual's baseline characteristics; hence, treatment groups may not be comparable in their pretreatment characteristics. As a result, direct comparison of outcomes between treatment groups might lead to biased estimate of the treatment effect. Propensity score approaches have been used to achieve balance or comparability of treatment groups in terms of their measured pretreatment covariates thereby controlling for confounding bias in estimating treatment effects. Despite the popularity of propensity scores methods and recent important methodological advances, misunderstandings on their applications and limitations are all too common. In this article, we present a review of the propensity scores methods, extended applications, recent advances, and their strengths and limitations.

Keywords: bias, confounding, effectiveness, health technology assessment, propensity score, safety, secondary data, observational study

### INTRODUCTION

Randomized clinical trials (RCTs) are generally accepted as the gold-standard approaches for measuring the "causal" effects of treatments on outcomes (Sibbald and Roland, 1998; Concato et al., 2000) and the design of choice for health technology assessment (HTA). In causal inference terminology using Rubin's potential outcomes framework (Rubin, 2005), the effect of a certain treatment (Z = 1) versus a control or comparator (Z = 0) on an outcome (Y) involves comparison of potential outcomes under treatment (Y1)) and an alternative treatment (Y0)). In RCT, with sufficient numbers of participants and adequate concealment of allocation, randomization ensures that individuals assigned to treatment and control or comparator groups are comparable in all pretreatment characteristics, both measured and unmeasured (Sibbald and Roland, 1998). The only difference is that one group received the treatment (Z = 1) and the other received no treatment or the alternative treatment (Z = 0); hence, any difference in outcomes between the two groups can be attributable to the effect of the treatment. In other words, the "causal" effect of treatment in the study population (the average treatment effect, ATE) on outcomes can be estimated by a direct comparison of the outcomes between the treatment and the comparator groups (Equation 1) (Concato et al., 2000). However, even adequately powered RCT may not always be feasible for reasons such as cost, time, ethical, and practical constraints (Sibbald and Roland, 1998). RCTs also rely on data collected on selected, homogeneous population under highly controlled conditions; hence, they provide evidence on efficacy rather than on effectiveness of interventions or treatments (Eichler et al., 2011).

$$ATE = E[Y\_1 - Y\_0] = E[Y\_1] - E[Y\_0] \tag{1}$$

With steadily increasing costs of health care and the introduction of novel, yet very expensive, pharmaceutical products and diagnostics, HTA agencies such as the UK National Institute for Health and Care Excellence (NCIE) are inquiring robust methods for evaluation of relative effectiveness and safety of medications, devices, and diagnostics in daily clinical practice. In contrast to efficacy, relative effectiveness of an intervention or treatment is "the extent to which an intervention does more good than harm, when compared to one or more alternative intervention(s)" when used under the routine setting of health care practice" (Eichler et al., 2011; Schneeweiss et al., 2011). In addition, for medical devices and diagnostics, waiting for evidence from RCTs when the health technology is diffusing in the clinical practice could be costly for the payers, inefficient from policy perspective, and methodologically questionable (Tarricone et al., 2016). On the other hand, regulators' and HTA agencies' perception of the importance of real-world data in complementing evidence on the relative effectiveness of health technologies has been steadily increasing (Makady et al., 2017; Yuan et al., 2018).

The effect of a particular health technology, e.g., a medication, on a certain outcome event could also be investigated using nonrandomized studies (i.e., observational or quasi-experimental) using routinely collected data (Schneeweiss et al., 2011, Ali et al., 2016, Bärnighausen et al., 2017). In observational studies, however, treatment selection is mainly influenced by the patient, the physician, and, to a certain extent, the health system characteristics. Hence, treated and untreated groups differ not only in receiving the treatment but also in other pretreatment characteristics, leading to non-comparability or non-exchangeability, a phenomenon leading to confounding bias (Greenland and Morgenstern, 2001). This means that differences in outcomes between the two groups, treated versus untreated, could be explained by either the treatment, or other pretreatment variables, or both. In other words, direct comparison of outcome events between the two groups leads to biased estimate of the treatment effect. Hence, any systematic difference in pretreatment characteristics between treatment should be accounted for by design, or analysis, or both (Rubin, 1997). Over the years, several methodologies have been developed to control for confounding bias in observational studies (**Figure 1**); the propensity score methods (Rosenbaum and Rubin, 1983) are among the popular approaches in pharmacoepidemiology and health technology evaluations (Ali et al., 2015).

Propensity score approaches were first introduced by Rosenbaum and Rubin in 1983 (Rosenbaum and Rubin, 1983), and their use to control for confounding has been increasing in the previous decade. Propensity score (PS) is a scalar summary of all measured pretreatment characteristics (often called potential confounders); stated formally, the propensity score e(X) is the conditional probability of receiving a certain treatment, versus a comparator or no treatment, given the measured pretreatment characteristics (Rosenbaum and Rubin, 1983), X, denoted as

$$e(X) = \text{pr}(Z = 1 | X),\tag{2}$$

where Z = 1 for individuals in the treatment group and Z = 0 for individuals in the comparison group (Rosenbaum and Rubin, 1983; Rosenbaum and Rubin, 1984). Treated and untreated individuals with similar propensity scores have, on average, similar or comparable pretreatment characteristics, a situation similar to an RCT. However, this comparability, conditional on the propensity score, of the treatment groups is limited only to measured pretreatment characteristics included in the propensity score model and may not hold for unmeasured ones (Rosenbaum and Rubin, 1983). Hence, balancing these pretreatment potential confounders through propensity scores enables researchers to obtain a "quasi-randomization" of treatment groups to reduce confounding and hence to get a better estimate of the treatment effect. Implicitly, researchers assume "Strongly Ignorable Treatment Assignment" (SITA) given the measured covariates; this comprises "unconfoundedness" and "positivity" (Rosenbaum and Rubin, 1983). Unconfoundedness implies that all relevant pretreatment characteristics are measured and included in the propensity score model; hence, given these measured covariates are included in the propensity score, there is no unmeasured confounding. Positivity, on the other hand, implies that every individual has a non-zero (positive) probability of receiving all values of the treatment variable: 0 < *P*(*Z =* 1|*X*) < 1 for all values of Z (Rosenbaum and Rubin, 1983).

In the last decade, the propensity score methods have been popular among clinical researchers, their use in pharmacoepidemiology and HTAs has been ubiquitous, and they have undergone substantial methodological advances. On the other hand, confusions and misunderstandings on what a propensity score method can and cannot do as well as errors in the design, analysis, interpretation, and reporting of propensity score-based analyses are unfortunately all too common (Ali et al., 2015). With increasing availability of routinely collected electronic medical records for evaluation of effects (both comparative effectiveness and safety) of health technologies, and relatively rapid development of the methods, an up-to-date review of the methods and their characteristics is necessary. In this article, we aim to introduce propensity score methods with an emphasis on important aspects of the methods; describe their extended applications and recent developments; and discuss their strengths and limitations.

Schneeweiss (2006), Uddin et al. (2016), and Zhang et al. (2018)).

The manuscript, including the introduction, is organized into eight sections: the section *Introduction* has introduced RCT, observational studies, and propensity score in relation to HTA; the section *Variable Selection and Propensity Score Estimation* discusses variable selection and propensity score estimation approaches; the section *Covariate Balance Assessment* describes methods for assessment of covariate balance in propensity score methods; the section *Propensity Score Methods* summarizes the different types of propensity score methods; the section *Extended Applications* describes extended applications of propensity scores; the section *Advantages and Limitations of Propensity Score Methods* summarizes strengths and limitations of the propensity score methodology; the section *Reporting* highlights on reporting of propensity score based analysis; and the section *Conclusion* concludes the discussion.

### VARIABLE SELECTION AND PROPENSITY SCORE ESTIMATION

Observational studies using administrative or clinical databases often involve high dimensionality with respect to the number of pretreatment covariates available for analysis including socioeconomic characteristics, demographics, comorbidities, comedications, and health system characteristics, among others. The inclusion of a large number of covariates in conventional regression models, particularly in nonlinear models such as logistic regression and Cox regression models, requires sufficient number of outcome events (approximately 10 outcome events per covariate) (Peduzzi et al., 1995; Peduzzi et al., 1996; Cepeda et al., 2003). For example, to adjust for 5 confounders using logistic regression model, one would need to have 5\*10 = 50 outcome events. However, many practical settings in pharmacoepidemiology and other HTAs involve relatively few or rare outcome events; hence, confounding adjustment using regression methods requires selection of a limited number of covariates to avoid problems such as over-fitting (Peduzzi et al., 1995). Alternatively, the use of propensity score methods to summarize a large pool of covariates into a single score, the propensity score, avoids over-fitting and collinearity issues in estimating treatment effects (Cepeda et al., 2003). When the number of covariates available in the study dataset is relatively small, it is common practice to include all the pretreatment covariates in the propensity score model; however, covariate selection might be required when researchers are presented with very large number (several hundreds) of covariates and limited number of outcome events (Schneeweiss et al., 2009).

Covariates selection in propensity score is often based on prior subject-matter knowledge on the relationships underlying the covariates in the study data, statistical tests on the association between the covariates and the outcome event (using p-values or change in effect estimates) (Brookhart et al., 2006; Patrick et al., 2011; Ali et al., 2015; Adelson et al., 2017), strength of associations with the treatment and/or the outcome event (Patrick et al., 2011; Ali et al., 2015; Adelson et al., 2017), and machine learning methods such as generalized boosted models (McCaffrey et al., 2004). Each approach has its own strengths and limitations; however, emphasis should be given to achieve balance on important prognostic pretreatment characteristics (Rosenbaum and Rubin, 1983) and not to improve model fit or to predict treatment as well as possible. Hence, the use of p-values, goodness-of-fit tests, and model discrimination tests such as c-statistics should be avoided (Weitzen et al., 2005; Patrick et al., 2011; Westreich et al., 2011). The iterative approach of model fitting, by including interactions and square terms of the covariates, and subsequent balance assessment, which was recommended in the seminal paper by Rosenbaum and Rubin (1983), is still a more robust approach. This application helps to achieve the goal of propensity score modelling, "improving balance" of potential confounders between treatment groups so that the groups are comparable or exchangeable conditional on the propensity score.

One of the greatest strengths of propensity score approaches is the separation of design from analysis, i.e., propensity score methods purposefully disregard outcome information at this stage of the design (Rubin, 2004b; Leacy and Stuart, 2014). That would also mean, as in the classical implementation of the methods, association between the covariates and the outcome event in the study data is not assessed for selection of covariates while constructing the propensity score model. However, this approach is not without disadvantages: failure to exclude colliders (variables that are common effects of the treatment and the outcome) and strong instruments (variables that are strongly related to treatment but independent of both the confounders and the outcome) can lead to increased bias in the estimated treatment effect (Pearl, 2011; Myers et al., 2011a, Myers et al., 2011b; Pearl, 2012; Ali et al., 2016).

It is important to emphasize that, similar to conventional regression modelling, intermediates (variables on the causal pathway between the treatment and the outcome) and colliders should not be included in the propensity score model (Greenland and Morgenstern, 2001) since including these variables will tend to increase (rather than reduce) bias. In addition, strong instruments should also be excluded, particularly when strong unmeasured confounding is a concern thereby avoiding any amplification of the residual bias (Pearl, 2011; Myers et al., 2011a; Myers et al., 2011b; Pearl, 2012; Ali et al., 2016). However, it is not common to come across with such a scenario; the use of propensity score method is meaningful when the assumption of "Strongly Ignorable Treatment Assignment", SITA, is met (i.e., there is no unmeasured confounding given the measured covariates and also there is positivity) (Rosenbaum and Rubin, 1983). Compared to residual confounding by unmeasured characteristics, bias amplification should be considered a secondary concern; hence, researchers should be cautious and are advised to err on the side of including rather than excluding any potential confounder (Myers et al., 2011b; Ali et al., 2017c). Alternatively, when a strong instrument—essentially a proxy measure of difference in treatment—is identified that is independent of confounders and outcome, instrumental variable analysis can be a powerful tool to account for any unmeasured confounding (Angrist et al., 1996).

A common question asked by clinical researchers who have not used propensity score methods is "why do we need to estimate the probability that an individual receives a certain treatment versus a comparator while we certainly know from the data whether that particular individual has received the treatment?" A brief answer to this important question is as follows: propensity score exists both in RCT and in observational studies (Joffe and Rosenbaum, 1999; Rubin, 2004b; Ali et al., 2016). In RCT, the true propensity score is known by design or the treatment allocation mechanism, i.e., randomization. For example, consider a simple two-arm RCT in which individuals are assigned to a treatment versus a comparison group by flipping of a fair coin (also assume that the sample sizes are equal in both treatment groups). The propensity score for every individual, the probability of being assigned to the treatment group versus the comparator group, is equal to 0.5, apart from chance variations. In contrast, in observational studies, the true propensity score for individuals is unknown and is dependent on several pretreatment characteristics, both clinical and nonclinical, under consideration by the physician. As a result, the propensity score should be—and can often be estimated using the study data (Joffe and Rosenbaum, 1999; Rubin, 2004b; D'Agostino, 2007; Ali et al., 2016). Estimation of the propensity score is needed to create a "quasi-randomized experiment" by using the individual's probability of receiving the treatment as a summary score of all measured pretreatment covariates. It enables appropriate adjustment for measured potential confounders to estimate the effect of the treatment. This explains one of the key properties of the propensity score method: if we find two individuals with the same propensity score, one in the treated group and one in the untreated group, we can assume that these two individuals are more or less "randomly assigned" to one of the treatment groups in the sense of being equally likely to be treated or not, with respect to measured pretreatment characteristics (Ali et al., 2015; Ali et al., 2016).

In practice, the propensity score is often estimated using ordinary logistic regression model, in which treatment status is regressed on measured pretreatment characteristics (Austin, 2008a; Ali et al., 2015). The estimated propensity score is the predicted probability of receiving the treatment derived from the fitted logistic regression model. Logistic regression has several advantages: it is a familiar and well-understood statistical tool for researchers as well as easy to implement using standard statistical software packages (Setoguchi et al., 2008; Westreich et al., 2010; Ali et al., 2016). However, logistic regression is not the only approach; other methods have also been used including recursive partitioning (D'Agostino, 2007) and several machine learning methods, for example, classification and regression trees (CARTs), neural networks, and random forests (Setoguchi et al., 2008; Lee et al., 2010; Westreich et al., 2010; Lee et al., 2011). Comparative simulation studies favor the use of machine learning methods over logistic regression when there is moderate or high nonlinearity (square or cubic terms of covariates) and non-additivity (interactions between pretreatment covariates) in the propensity score models. This could be explained by the fact that machine learning methods include interactions and square terms by default (Setoguchi et al., 2008), compared to logistic regression where the researcher should "manually" include interactions and square terms. When important interaction and square terms are included, the performance of logistic regression is as good as other machine learning methods (Ali et al., 2017b).

### COVARIATE BALANCE ASSESSMENT

The aim of propensity score methods is to balance covariates between treatment groups and hence control for measured confounding (Rosenbaum and Rubin, 1983). Therefore, the quality of propensity score model should be assessed primarily on the covariate balance achieved. It should not be evaluated based on how well the propensity score model discriminates between treated and untreated individuals, i.e., whether the treatment assignment is correctly modeled (Rubin, 2004b; Westreich et al., 2011; Ali et al., 2015, Ali et al., 2016) or whether the subsequent estimates of treatment effect are smaller or larger than expected (Rosenbaum and Rubin, 1984; Hansen, 2004). Hence, propensity score modelling can be considered as an iterative step where the propensity score model is updated by adding different covariates, interactions between covariates, or higher-order terms of continuous covariates until an acceptable level of balance on important confounding variables is achieved (Rosenbaum and Rubin, 1984). It is also important to underline that variable selection and covariate balance are inseparably linked; however, covariate balance is often checked on a preselected list of pretreatment covariates (Ali et al., 2015). On the other hand, there are propensity score modelling techniques that optimize covariate balance while estimating the propensity score (Imai and Ratkovic, 2014; Austin, 2019).

It is helpful to start propensity score analysis by examining the distribution of propensity scores using histograms or density plots. This facilitates subjective judgment on whether there is sufficient overlap, also called "the common support," between propensity score distributions of treated and untreated groups (Dehejia and Wahba, 2002). However, such plots should not be considered as proper measures of covariate balance; they can guide the choice of matching algorithms in propensity score matching and the number of strata in propensity score stratification (Ali et al., 2015; Ali et al., 2016). For example, when there is very little overlap in the propensity score distributions, matching treated and untreated individuals with replacement, with or without caliper, can be a better option because it will be challenging to find sufficient number of untreated individuals for the treated individuals (Ali et al., 2016). Inadequate overlap in the propensity score distributions, which can be quantified using overlapping coefficient (Ali et al., 2014), should also warn researchers that the dataset, no matter how large, could not support any causal conclusion about the effect of the treatment on the outcome of interest without relyng o untrustworthy model assumptions (Rubin, 1997; Ali et al., 2016).

To assess covariate-specific balance, several metrics have been proposed in the literature (Austin, 2009; Belitser et al., 2011; Ali et al., 2014). Each balance metric has its own advantages and limitations; the absolute standardized difference in means or proportions (ASMD) (Austin, 2009) is more robust in terms of sample size and covariate distribution requirements in comparison to other balance diagnostics, such as overlapping coefficients (Ali et al., 2014; Ali et al., 2015; Ali et al., 2016). The ASMD is also a familiar, easy-to-calculate and present, and well-understood statistical tool (Austin, 2009; Ali et al., 2015; Ali et al., 2016). Hence, it is recommended for checking and reporting covariate balances in propensity score methods (Austin, 2009; Belitser et al., 2011; Ali et al., 2014; Ali et al., 2015; Ali et al., 2016). The ASMD is calculated for each covariate and can be averaged to compute an overall covariate balance and to compare propensity score models (Belitser et al., 2011; Ali et al., 2014). The covariate-specific ASMD is useful to identify the variable that is still imbalanced and to modify the propensity score model with squares and interaction terms of the variable to improve its balance. Although there is no universal threshold below which the level of covariate imbalance is always acceptable (Imai and Van Dyk, 2004; Ali et al., 2016), the use of arbitrary cutoffs for balance diagnostics (e.g., < 10% for the ASMD) is common in the medical literature (Ali et al., 2015; Ali et al., 2016). Covariate balance is not only a property of the sample means but also of the overall distribution of the covariate; hence, higher-order sample moments of the covariate distribution such as variance should also be evaluated (Rosenbaum and Rubin, 1985; Rubin, 2001; Ho et al., 2007; Austin, 2009; Linden and Samuels, 2013). Rubin (2001) proposed the ratio of variances of treated and untreated groups as an additional check on balance; a variance ratio of 1.0 in the propensity score matched sample indicates a good matching and acceptable balance, and a variance ratio below 2 is generally considered acceptable balance (Rubin, 2001; Linden and Samuels, 2013).

In addition to numerical quantification of the covariate balance achieved by the specified propensity score model, graphical methods such as (weighted) side-by-side box plots, quintile-quintile (Q-Q) plots, plots of ASMD, and empirical density plots of continuous pretreatment covariates provide a simplified overview on whether balance on individual pretreatment covariates has improved, compared to prematching, pre-stratification, or pre-weighting (Rosenbaum and Rubin, 1983; Ali et al., 2016).

### PROPENSITY SCORE METHODS

Once the propensity score has been estimated, researchers have several options of using the propensity score in the design or analyses, including matching, stratification (also called subclassification), covariate adjustment using the propensity score, inverse probability of treatment weighting, and combinations of these methods (Rosenbaum and Rubin, 1983; Rosenbaum and Rubin, 1984; Rubin and Thomas, 2000; Hirano and Imbens, 2001; Johnson et al., 2018). Each method has its own advantages and disadvantages; the choice of a specific propensity score method is in part determined by the inferential goal of the research (i.e., the type of treatment effect estimand: the average treatment effect in the entire population, ATE, versus the average treatment effect in the treated population, ATT) (Imbens, 2000; Stuart, 2008; Ali et al., 2016). Although it is possible to estimate both ATT and ATE using all of the four propensity score methods, for example, by assigning different weights for the treated and untreated individuals, the default approach in each method might give slightly different estimand. For example, propensity score matching primarily estimates the treatment effect in the treated group, ATT (Imbens, 2004; Stuart, 2008). Therefore, to get an estimate of the average treatment effect in the entire population, ATE, one has to use either full matching (Hansen, 2004) or different weighting (Stuart, 2008, Stuart, 2010; Ali et al., 2015; Ali et al., 2016). The use of a specific propensity score method has also direct implication on the covariate balance assessment (Rosenbaum and Rubin, 1983; Rosenbaum and Rubin, 1984; Ali et al., 2016) and interpretation of the estimated treatment effect (Stuart, 2008; Ali et al., 2015; Ali et al., 2016).

### Propensity Score Matching

Propensity score matching, the most common application of propensity score (Ali et al., 2015), entails forming matched groups of treated and untreated individuals having a similar value of the propensity score (Rosenbaum and Rubin, 1983; Rubin and Thomas, 1996). The matching could be done in many ways: one-to-one or one-to-many (1:n, where n is the number of untreated individuals often up to five), exact or caliper matching, matching with or without replacement, stratified matching, and full matching (Hansen, 2004). However, one-toone caliper matching without replacement is the most common implementation of propensity score matching (Ali et al., 2015; Ali et al., 2016). For detailed discussion on different matching approaches, we refer to the literature (Rosenbaum and Rubin, 1985; Hansen, 2004; Stuart, 2010).

Once a matched sample has been formed, covariate balance can be easily checked between the matched groups using one of the balance diagnostics, preferably ASMD, and then treatment effect can be estimated by directly comparing outcomes between treated and untreated individuals in the matched sample (Rosenbaum and Rubin, 1983; Rubin and Thomas, 1996). With dichotomous or binary outcomes such as the presence or absence of a disease ("Yes" or "No"), the effect of the treatment can be estimated as the difference or the ratio between the proportion of individuals experiencing the outcome event in each of the two treatment groups (treated vs. untreated) in the matched sample. If the outcome is continuous, for example blood pressure measurement or HBA1c level, the effect of the treatment is estimated as the difference between the mean outcome for treated and the mean outcome for untreated individuals in the matched sample (Rosenbaum and Rubin, 1983).

If matching is done with replacement or in one-to-many matching, weights should be incorporated to account for the multiple use of the same untreated individual to match with several treated individuals or the multiple use of the same treated individual to match with several untreated individuals, respectively (Stuart, 2010). Whether or not to account for the matched nature of the data in estimating the variance of the treatment effect, for example, using paired t-test for continuous outcome or McNemar's test for binary outcome, is an ongoing discussion (Schafer and Kang, 2008; Stuart, 2008; Austin, 2008a; Austin, 2011).

The most appealing feature of propensity score matching is that the analysis can partly mimic that of an RCT, meaning that the distribution of measured pretreatment covariates will be, on average, similar between treatment groups. Hence, direct comparison of outcomes between treated and untreated groups within the propensity score matched sample has the potential to give unbiased estimate of the treatment effect, depending on the extent to which the measured variables have captured the potential confounding factors (Rosenbaum and Rubin, 1983). However, RCT, on average, guarantees balance on both measured an unmeasured confounders, whereas propensity score improves balance on measured confounders but those of unmeasured confounders only to the extent that they are related to the measured confounders included in the propensity score model (Rubin, 2004b; Austin, 2011). Other useful features include: separation of the design from analysis *via* preprocessing of the data to improve covariate balance without using outcome data, thereby a minimal reliance on model specification; relatively easy assessment, visualization, and communication of covariate balance using simple statistics or plots; and qualitative indication of whether the dataset at hand is good enough to address the causal question without relying on untrustworthy "modeldependent" extrapolations (Rubin, 2004b; Ho et al., 2007; Ali et al., 2016).

Recently, the use of propensity score for matching has been criticized on the basis of an argument that propensity score matching approximates complete randomization and not completely blocked randomization; hence, it engages in random pruning or exclusion of individuals during matching. "Unlike completely blocked randomization, random exclusion of individuals in propensity score Ali et al. Propensity Score Methods in HTA

matching, as in complete randomization, means a decrease in sample size leading to covariate imbalance and more model dependence, so called the 'propensity score paradox'" (King and Nielsen, 2016). At first this might seem a valid argument; however, the practical implication of this paradox is very limited, if any (Ali et al., 2017a). This is partly due to the fact that propensity score matching could do better than complete randomization with respect to the balance of measured covariates if variables related to treatment are included in the propensity score model (Joffe and Rosenbaum, 1999). In addition, the use of matching algorithms such as caliper matching or matching with replacement retains the best matches thereby avoiding random pruning or exclusion, and hence the paradox is not a big concern. Furthermore, it is currently a standard practice to check covariate balance in the propensity score matched sample before estimating the treatment effect, further minimizing any risk of exacerbating covariate imbalance (Ali et al., 2015).

Similar to RCT, when there are residual differences in pretreatment characteristics between treatment groups in propensity score matched sample, regression adjustment can be used on the matched sample to reduce bias due to residual differences in important prognostic factors (Rubin and Thomas, 2000; Imai and Van Dyk, 2004; Schafer and Kang, 2008). This method has been described as a doubly robust (DR) approach, i.e., correct specification of either the matching or the regression adjustment, but not necessarily both, is required to obtain unbiased estimate of the treatment effect (Schafer and Kang, 2008; Funk et al., 2011; Nguyen et al., 2017). Propensity score matching primarily estimates the effect of treatment in the treated individuals (ATT), not the effect of treatment in the population (treated and untreated individuals, ATE) (Imbens, 2004; Stuart, 2008). This is because the closest untreated and treated individuals are matched and the remaining untreated individuals that were not matched are often excluded from the analysis (Stuart, 2008; Ali et al., 2016). It is important to emphasize that exclusion of unmatched individuals from the analysis not only affects the precision of the treatment effect estimate but also could have consequences for the generalizability of the findings, even for the ATT (Lunt, 2013; Ali et al., 2016). For example, exclusion of treated individuals due to a lack of closer untreated matches could change the estimand from the effect of treatment in the treated (ATT) to the effect of treatment in those treated individuals for whom we can find untreated matches (ATT) (Lunt, 2013; Ali et al., 2016). However, it is possible to estimate the ATE in the matched sample with slight modifications of the matching algorithms. For example, using full matching that retains all the treated and untreated individuals in the study data, one can estimate either the ATE or ATT (Hansen, 2004; Stuart, 2010). Generally, matching discards some data (often unmatched untreated individuals); however, it may increase the efficiency, reducing the estimated standard error, of the treatment effect estimate by reducing heterogeneity of observations (Ho et al., 2007; Ali et al., 2016).

### Propensity Score Stratification

Propensity score stratification, also called propensity score subclassification, involves grouping individuals into strata based on their propensity scores (often 5 groups using quintiles or 10 groups using percentiles). Within these strata, treated and untreated individuals will have a similar distribution of measured covariates; hence, the effect of the treatment can be estimated by direct comparison of outcomes between treated and untreated groups within each strata (Rosenbaum and Rubin, 1984; D'Agostino, 2007; Ali et al., 2017a). The stratum-specific treatment effects can then be aggregated across subclasses to obtain an overall measure of the treatment effect (Rosenbaum and Rubin, 1984).

Rosenbaum and Rubin (1983, 1984) proposed quintile stratification on the propensity score based on their finding that five equal-size propensity score strata removed over 90% of the bias due to each of the pretreatment covariates used to construct the propensity score. However, it is recommended that researchers examine the sensitivity of their results to the number of subclasses by repeating the analysis using different quantiles of the propensity score (Imai and Van Dyk, 2004; Adelson et al., 2017). Similar to matching, residual imbalances after stratification can be accounted for using regression adjustment within each stratum (Rosenbaum and Rubin, 1984; Rubin, 2001). Alternatively, the propensity score, defined as quintiles and deciles, can be used as a categorical variable in a model-based adjustment to estimate treatment effects (Rosenbaum and Rubin, 1984; Ali et al., 2016).

Propensity score stratification can estimate the stratum-specific ATT, or the overall ATT across strata, or the ATE, depending on how the subclass treatment effect estimates are weighted. Weighting stratum-specific estimates by the total number of individuals (treated and untreated) in each stratum yields the ATE. On the other hand, weighting stratum-specific estimates by the proportion of treated individuals in each stratum provides ATT (Stuart, 2010; Ali et al., 2016). Similarly, pooling stratum-specific variances provides pooled estimates of the variance for the pooled ATT or ATE estimate (Imbens, 2004; Ali et al., 2016). Pooling the stratumspecific treatment effect is straightforward when treatment effect is homogeneous among the propensity score strata (Ali et al., 2016). When there is heterogeneity of treatment effect among the strata even after automated iterations of the number and boundaries of propensity score strata (Imbens, 2004; Imbens and Rubin, 2015; Ali et al., 2016), pooling the stratum-specific treatment effect might complicate interpretation of the treatment effect estimate (Ali et al., 2014; Ali et al., 2016). In the presence of treatment effect modification regardless of the presence of confounding, Mantel-Haenszel methods do not estimate a population parameter (ATE); hence, estimating the effect of treatment in the treated (ATT) rather than the whole population (ATE), for example, using propensity score matching is preferable (Stürmer et al., 2006b). Alternatively, one could standardize the stratum-specific estimates to a specified distribution of propensity scores, for example, to calculate a standardized mortality ratio (AMR) from the stratum-specific estimates (Stürmer et al., 2006b; Lunt et al., 2009)

Stratification has several advantages: it is an easy and wellunderstood method to implement; it is straightforward to evaluate and communicate covariate balance, and to interpret particularly to non-technical audiences; it separates the design of the study from the analysis, like propensity score matching, hence less dependent on parametric models (Rosenbaum and Rubin, 1984); it is less sensitive to nonlinearities in the relationship between propensity scores and outcomes; and it can accommodate additional modelbased adjustments (Rosenbaum and Rubin, 1983; Rosenbaum and Rubin, 1984). However, this propensity score approach is prone to residual confounding, which might be an issue due to propensity score heterogeneity within the strata.

### Regression Adjustment Using Propensity Score

The propensity score, as a single summary of all covariates included in the propensity score model, can be included as a covariate in a regression model of the treatment, i.e., the outcome variable is regressed on the treatment variable and the estimated propensity score (Rosenbaum and Rubin, 1983; Ali et al., 2016). Although this approach is very easy to implement, it is generally considered to be a sub-optimal application of the propensity score for several reasons: 1) The treatment effect estimation is highly model-dependent because it mixes the study design and data analysis steps; hence, it requires correct specification of the propensity score model (Rubin, 2004b; Johnson et al., 2018). 2) It also makes additional assumptions unique to regression adjustment; the relationship between the estimated propensity score and the outcome must be linear and there should be no interaction between treatment status and the propensity score (Rosenbaum and Rubin, 1983; Austin, 2011; Ali et al., 2016). However, both assumptions can be checked with the data, and can be relaxed if necessary, for example, by combining with propensity score stratification. 3) It enables estimation of the ATE; however, its interpretation is complicated particularly in nonlinear models such as logistic regression or Cox regression where the estimand of interest is non-collapsible. Non-collapsibility refers to a phenomenon in which, in the presence of a non-null treatment effect, the marginal (overall) treatment effect estimate is different from the conditional (stratum-specific) treatment effect estimate, even in the absence of confounding (Greenland et al., 1999; Austin, 2008b). In addition, assessment and communication of covariate balance are not straightforward (Ali et al., 2016).

### Inverse Probability Treatment Weighting

Inverse probability weights (IPW) calculated from propensity score can also be used to create a weighted "artificial" population, also called a "pseudo-population" in which treatment and measured pretreatment characteristics included in the propensity score are independent (Hernán et al., 2000; Robins et al., 2000; Cole and Hernán, 2008; Ali et al., 2016). Hence, treated individuals will be assigned weights equal to the inverse of their propensity scores (1/ PS, as they have received the treatment) and untreated individuals will be assigned weights equal to the inverse of one minus their propensity scores [1/(1 – PS)] (D'Agostino, 2007). A particular diagnostic concern in using propensity score weighting is that individuals with extremely large weights may disproportionately influence results and yield estimates with high variance (Lee et al., 2011). When some individuals have probabilities of receiving the treatment close to 0 or 1, the weights for such individuals become extremely high or extremely low, respectively (Ali et al., 2016). Weight stabilization to "normalize" the range of the inverse probabilities is often considered: the "1" in the numerator of the inverse probability weights can be replaced with the proportion of treated individuals and the proportion of untreated individuals for treated and untreated individuals, respectively (Hernán et al., 2000; Ali et al., 2016).

Alternative approaches such as weight trimming and weight truncation have been suggested (Cole and Hernán, 2008; Lee et al., 2011). Weight trimming involves removing individuals in the tails of the propensity score distributions using percentile cut-points (Cole and Hernán, 2008; Lee et al., 2011), i.e., individuals who have extreme values of the propensity score—both very high and very low are excluded. On the other hand, weight truncation involves setting a maximum allowable weight, W*ma*), such that individuals with a weight greater than W*ma*) will be assigned W*ma*) instead of their actual weights. Both approaches may help stabilize weights, reduce the impact of extreme observations, and can improve the accuracy and precision of parameter estimates; however, both involve biasvariance trade-offs (Lee et al., 2011). For example, trimming the tails excludes some individuals with extreme values and hence changes the population, which might introduce bias depending on the cutoff (Cole and Hernán, 2008). Recently, Li et al. (2018) suggested a different set of weights called "overlapping weights" which weight each individual proportional to its probability of receiving the alternative treatment, i.e., the overlap weight is defined as 1-PS for a treated individual and PS for an untreated individual. Unlike standard IPW, the overlap weights are bounded between 0 and 1; hence, they are less sensitive to extreme weights. It also means that there is no need for arbitrary choice of a cut-off for inclusion in the analysis as well as exclusion of individuals, unlike weight trimming (Li et al., 2018).

In the weighted population, weighted standardized difference can be used to compare means, proportions, higher-order moments, and interactions between treated and untreated individuals. In addition, graphical methods can be employed to compare the distribution of continuous covariates between treated and untreated individuals (Austin and Stuart, 2015). Once sufficient covariate balance is achieved, the effect of the treatment can be estimated by direct comparison of outcomes between treated and untreated groups. The weights can also be used in weighted regression models to estimate the effect of the treatment; and adjustment can be made for covariates that are not sufficiently balanced in the weighted sample. This method focuses on estimating the average treatment effect in the entire population (ATE); modification of the weights allows to estimate the average treatment effect in the treated population (ATT) (Stuart, 2010; Ali et al., 2016). Most importantly, the variance estimation should take into account the weighted nature of the "pseudo-population" since some observations can have weights that are unequal to one another (hence, potentially inducing a withinindividual correlation in outcomes), for example, by using the sample weights in robust variance estimation (Hernán et al., 2000; Cole and Hernán, 2008; Austin and Stuart, 2015). Alternatively, bootstrapping could be used to construct 95% confidence intervals, which also takes into account the estimation of the propensity score, in addition to the lack of independence between duplicate observations in the weighted sample (Hernán et al., 2000; Austin and Stuart, 2015; Ali et al., 2016; Ali et al., 2017b).

Inverse probability of treatment weights (IPTW) can be also be used to estimate parameters of marginal structural models (MSMs) to deal with time-varying confounding (Hernán et al., 2000), time-modified confounding (Platt et al., 2009), and Ali et al. Propensity Score Methods in HTA

competing risks (Hernán et al., 2000; Ali et al., 2017b). Hence, the implementation of propensity scores as inverse probability weights is often referred to as MSM using IPTW. All other propensity score approaches can only be extended to time-varying confounding and treatment settings under certain conditions as described in **Figure 2**. Comparison of the four propensity score approaches is summarized in **Table 1**.

### EXTENDED APPLICATIONS

### Time-Varying Treatments

In clinical practice, it is common for patients to start on a certain medication, stop or switch to another one (for example, due to intolerance or lack of adequate response); in such cases, treatment might be treated as a time-varying exposure. Consider a cohort study to estimate the effect of antiretroviral zidovudine treatment (AZT) in HIV (human immunodeficiency virus) positive individuals, on progression to AIDS (acquired immune deficiency syndrome), where CD4 count is a confounder. Assuming individuals show up for clinical visits at baseline/ pretreatment (t = 0) and then every 6 months (t = 1, 2, 3,…), and CD4 counts are recorded at these visits (CD4*<sup>t</sup>* ), represented as CD40, CD41, CD42,…). If AZT is a time-varying dichotomous treatment variable indicating whether the individual is on antiretroviral treatment at each of the visits (AZT*<sup>t</sup>* , represented as AZT0, AZT1, AZT2,…), this means, an individual's treatment plan, at each subsequent visit (t = 1,2,…), is time-varying: the clinician in consultation with the individual decides treatment AZT*<sup>t</sup>* based on the changing values of the individual's clinical and demographic history recorded during the previous and current visits. These include prior treatment history, current CD4 count, and other confounders, which are not included in this discussion and ignored for now for the sake of simplicity. The relationships between treatment, confounder, and outcome are presented using directed acyclic graphs (DAGs) for clarity.

In **Figure 2**, we considered two time points or visits t = 0 (baseline/pretreatment) and t = 1; hence, CD40 refers to baseline CD4 count and AZT0 refers to treatment at the first visit. Treatment decision at the first visit AZT0 is influenced by pretreatment CD4 count (CD40), represented in **Figure 2A** by the arrow from CD40 to AZT0. In the second visit (t = 1), treatment decision AZT1 is based on previous treatment (AZT0) and CD4 count at the current visit (CD41), represented in **Figure 2A** by the arrows from AZT0 and CD41 to AZT1.

In settings such as DAG of **Figure 2A**, where there is no arrow from AZT0 to CD41 implying previous treatment does not affect current CD4 count, all the standard propensity score approaches can deal with the time-varying confounder CD4 count by matching, conditioning, stratification, or weighting, for example, by combining with time-varying Cox models to estimate the treatment effect. However, this is not biologically plausible; RCTs have proved that antiretroviral treatment indeed affects CD4 count. It is important



*aConstructs treated and untreated matched groups with similar propensity scores. bConstructs subgroups of treated and untreated individuals, often quintiles or deciles of PS. cPS is used, as a single summary of all covariates included in PS model, in regression model. dPSs are used as weights to create a pseudo-population in which exposure and measured covariates included in the treatment (PS) model are independent (Ali et al., 2016). 1Estimation of stabilized weights as well as extension to time-varying treatment and confounding setting in MSMs framework can be complex (Ali et al., 2016). 2Requires correct specification of PS and outcome model, apart from the basic assumptions that there is positivity and no unmeasured confounding (Ali et al., 2016). 3Weight trimming excludes some individuals in the tails of the propensity score distribution. 4In PSM, when treated individuals are excluded, interpretation of the treatment effect may change, not just ATT and in Stratification, when there is treatment effect modification by the PS, in regression adjustment using PS, when non-collapsible effect measures such as odds ratios are used. 5Modification of the matching or weighting method enable to estimate either ATT or ATE. 6When timevarying confounder is affected by previous treatment, all the propensity score based methods fail to correctly control for the confounding bias including standard IPWs; however, MSMs using IPWs.*

to mention that there are many practical examples where both treatment and confounders are time-varying or dynamic, but previous treatment does not affect time-varying confounder; hence, the DAG in **Figure 2A** may still be valid in other situations.

When a time-varying confounder (such as CD4 count in our example, CD41) is affected by previous treatment (AZT0) as in the DAG of **Figure 2B**, the time-varying confounder (CD41) is also an "intermediate" for the effect of previous treatment (AZT0) on the outcome (progression to AIDS), represented by the path AZT0 → CD41 → AIDS. Furthermore, if there is an unmeasured common cause (U) of both the time-varying confounder (CD41) and the outcome (progression to AIDS) as in DAG of **Figure 2C**, the timevarying confounder (CD41) is also a "collider" on the path AZT0 → CD41 ← U → AIDS (the arrows from U and CD40 collide on CD41). Hence, the path AZT0 → CD41 ← U → AIDS is a closed or noncausal path because it is blocked at CD41 (using DAG terminologies). It also means that there is no association between AZT0 and U unless one conditions, matches, or stratifies on this collider, CD41 (Hernán et al., 2000; Robins et al., 2000). Such a time-dependent variable is a confounder, an intermediate, and also a collider all at the same time; hence, adjustment requires careful consideration.

Conventional statistical approaches including propensity score methods (matching, stratification, and regression adjustment) that condition or stratify on such a covariate will result in a biased estimate of the treatment effect (Hernán et al., 2000; Robins et al., 2000). This happens because conditioning or stratifying on an intermediate will adjust away the indirect effect of the treatment mediated by the cofounder, in this case CD41; and conditioning or stratifying on a collider creates a spurious association between the treatment and the unmeasured common cause that did not exist before conditioning (creating an open backdoor path AZT0 → CD41,... U→ AIDS), which is indicated by using dotted lines in the DAG of **Figure 2D**, leading to collider-stratification bias (Hernán et al., 2000; Cole et al., 2009; Ali et al., 2013).

In such settings, MSM using inverse probability weighting is the method of choice; unlike conditioning or stratification, weighting creates a "pseudo-population" in which the association between the time-varying confounder and treatment is removed (Hernán et al., 2000; Robins et al., 2000). Additional methods are also available to deal with time-varying treatment and confounding including other classes of marginal structural models (g-formula and g-estimation of structural nested models) (Hernán et al., 2000; Robins et al., 2000).

It is straightforward to hypothesize that such a time-varying confounding can also be time-modified, which means not only the confounder (CD4 count) change over time but also its association with the treatment and its impact on the outcome (progression to AIDS) varies during these times. The effects of the confounder change over time mean that the strength of association between CD40 and AIDS (CD40 → AIDS) is different from that of CD41 and AIDS (CD41 → AIDS) (Platt et al., 2009). However, time-modified confounding might still exist in longitudinal treatment settings where the confounder is time-invariant or fixed. Standard methods are sufficient to deal with time-modified confounding unless the confounder is both time-varying and affected by previous treatment, which requires the implementation of marginal structural models, such as using inverse probability weighting.

### Multiple Treatments

Propensity score methods are often used to estimate the effect of a binary treatment (whether treatment is received: Yes = 1 or No = 0) in observational data. However, with more than two levels of treatment, which is common in pharmacoepidemiology such as comparison of three or more statins (e.g., simvastatin, atorvastatin, fluvastatin, lovastatin, pravastatin, and rosuvastatin) or of multiple doses of a certain medication (e.g., low, medium and high doses), estimation of treatment effects requires additional assumptions and modelling techniques (Imbens, 2000; McCaffrey et al., 2004). These include the use of multinomial logistic and multinomial probit models for nominal treatments and ordinal logistic regression or the proportional odds model for ordinal treatments (Imbens, 2000). Alternatively, generalized boosted model, a machine learning approach involving an iterative process using multiple regression trees to capture complex, nonlinear, and non-additive relationships between treatment assignment and pretreatment covariates without the risk of over-fitting the data, can be used to fit inverse probability weighting for multiple treatments (McCaffrey et al., 2004). However, applications in pharmacoepidemiology using observational data are infrequent partly due to methodological complexities in fitting the models and understanding their assumptions as well as limited availability of guidance documents on these methods.

### Multilevel Treatments

Propensity score methods have been extensively studied and widely applied in a single-level treatment (no clustering among participants); however, most healthcare data have a multilevel structure such that individuals are grouped into clusters such as geographical areas, treatment centers (hospital or physician), or insurance plans (Goldstein et al., 2002). The unknown mechanisms that assign individuals to clusters may be associated with individuallevel measured confounders (such as race, age, and clinical characteristics) and unmeasured confounders (such as unmeasured severity of disease, aggressiveness in seeking treatment) (Li et al., 2013). These measured and unmeasured confounders might also create a cluster-level variation in treatment and/or outcome. If this variation is correlated with group assignment at the group or cluster level, it might lead to confounding (Greenland, 2000; Li et al., 2013). Hence, the use of standard regression or propensity score methods ignoring the cluster structure should be avoided. This is because ignoring the cluster structure often leads to invalid inferences: not only the standard errors are inaccurate but also the cluster-level effects could be confounded with individual-level effects.

Propensity score matching and weighting are often used in such settings (Arpino and Mealli, 2011; Li et al., 2013). One might consider the use of within-cluster PSM (of treated and untreated individuals), which automatically achieves perfect balance on all the measured cluster characteristics. However, it is very unlikely, particularly in small clusters, to find a sufficient number of untreated matches to treated individuals in the same cluster. Alternatively, PSM could be performed across clusters taking into account the cluster structure in the propensity score estimation model. Preferably, cluster structure should be taken into account in estimation of both the propensity score and the treatment effect (Li et al., 2013).

Multilevel regression models that include fixed effects and/ or random effects have been developed (Greenland, 2000; Goldstein et al., 2002), and extended to propensity scores approaches (Arpino and Mealli, 2011). Empirical applications of such methods in medication and device effectiveness and safety are rare. However, simulations studies have shown that multilevel propensity score matching (Arpino and Mealli, 2011) and weighting approaches (Li et al., 2013), without imposing a within-cluster matching or weighting requirement, reduce bias due to unmeasured cluster-level confounders.

### Missing Data

Missing data is a common problem in the estimation of treatment effects using routinely collected data. The impact of such missing data on the results of the treatment effect estimation depends on the mechanism that caused the data to be missing and the way missing data are handled. Missing data can be categorized into three distinct classes based on the relationship between the missing data mechanism and the missing and observed values: i) Missing Completely at Random (MCAR), when the missing data mechanism is unrelated to the values of any variable, whether missing or observed. Hence, the observed values are representative of the entire sample without missing values. ii) Missing at Random (MAR), when the missing data mechanism is unrelated to the missing values but may be related to the observed values of other variables. iii) Missing Not at Random (MNAR), when the missing data mechanism is related not only to the observed values of other variables but also to the missing values (Rubin, 1996). For each of the missing data patterns, different statistical techniques are used to correct for its impact on the quality of the inference. It is important to emphasize that MCAR, MAR, and MNAR could exist for different variables in a specific data. However, if one variable is MAR or MNAR, generally, the dataset is considered MAR or MNAR, respectively.

Complete case analysis, including only those individuals who have no missing data in any of the variables that are required for the analysis, performs well when data are MCAR and may be valid under some MAR and MNAR conditions. However, it often results in biased estimate of the treatment effect if missing is at random (MAR) (Rubin, 1996; Sterne et al., 2009). In MAR, as stated before, any systematic difference between the missing values of a variable and the observed values of the variable can be explained by differences in observed data (Sterne et al., 2009). Furthermore, missing data in several variables often lead to exclusion of a substantial proportion of the original sample, which leads to a substantial loss of precision (i.e., power) and hence estimates with wider confidence intervals (Cummings, 2013). Other approaches to deal with missing data include: 1) replacing missing values with values imputed from the observed data (for example, using the mean of the observed values); 2) using a missing category indicator; and 3) using the last observed value to replace missing values particularly in longitudinal studies [often called "last observation carried forward" (LOCF)]. These three approaches are generally statistically invalid, except under certain conditions, and they might lead to serious bias (Rubin, 1996; Sterne et al., 2009). Missing category indicator and LOCF approaches require specific assumptions for validity that are distinct from the MCAR, MAR, and MNAR categorization. On the other hand, single imputation of missing values (mean imputation) usually results in too small standard errors, because it fails to account for the uncertainty about the missing values (Sterne et al., 2009).

A relatively flexible approach to allow for the uncertainty in the missing data is multiple imputation. Multiple imputation involves creating multiple different copies of the dataset with the missing values replaced by imputed values (Step 1); estimating treatment effects in each copy of the data (Step 2); averaging the estimated treatment effects to give overall estimated measure of association and calculating standard errors using Rubin's rules (Step 3) (Rubin, 1996; Rubin, 2004a). Applications of propensity score methods in data with missing values involve a similar approach: 1) creation of multiple copies of imputed data; 2) estimation of propensity scores and treatment effects in each of the imputed copies of the dataset (Qu and Lipkovich, 2009; Leyrat et al., 2019); and 3) pooling of treatment effects by averaging across the multiple datasets and estimation of standard errors using Ruben's rule (Crowe et al., 2010; Leyrat et al., 2019) (**Figure 3A**). An alternative approach is pooling the propensity scores from the multiple copies of data, in step 2, and conducting the analysis in the pooled data (**Figure 3B**); however, this method has been proved sub-optimal in terms of bias reduction (Leyrat et al., 2019).

### ADVANTAGES AND LIMITATIONS OF PROPENSITY SCORE METHODS

Previous literature reviews of observational studies have found that results from both traditional regression and propensity scores analyses are similar (Shah et al., 2005; Stürmer et al., 2006a). These findings may be in part due to sub-optimal implementations of propensity score methods (Shah et al., 2005; Austin, 2008a; Ali et al., 2015); however, similarity of findings has been used to question the need for propensity score methods if they do not provide better ways to improve confounding control. Despite these findings, propensity score methods will remain advantageous for several reasons compared to covariate-adjustment techniques, which correct for covariate imbalances between treatment groups by conditioning them in the regression model for the outcome.

### Transparency

Propensity score methods primarily aim at balancing treatment groups with respect to covariate distributions; when sufficient covariate balance is achieved, it is relatively easy to check and communicate the balance (Ali et al., 2015, Ali et al., 2016) by using simple graphical tools or quantitative statistics. In addition, propensity score methods, unlike regression adjustment, can

give investigators an insight into the quality of the data at hand. Inadequate overlap in propensity score distributions (also called poor "common support") between treatment groups should be considered as a warning that the data set at hand may not be sufficient to reliably address the causal question without "modeldependent" extrapolations based on untrustworthy assumptions (Dehejia and Wahba, 2002; Rubin, 2004b; Rubin, 2007; Ali et al., 2016). In some cases, the researcher might decide to focus on individuals only in the overlapping regions using propensity score matching or trimming; as a consequence, the conclusions of the findings should be restricted to individuals that are sufficiently represented in the overlapping regions of the propensity score distributions (Ali et al., 2016). Conventional regression methods do not provide the researcher with these possibilities. Furthermore, covariate balance in regression methods is a "black-box" and, irrespective of inadequate overlap (i.e., when the treated and untreated groups are disparate on pretreatment covariates), conventional models use extrapolations to estimate treatment effects that may not be generalizable to the entire population in the data set.

### Design Tools

Similar to RCTs, propensity score methods can be considered as design tools for pre-processing of the data (matching, stratification, and weighting) without using any outcome information at this stage. As a result, formal causal inference models (also called the potential outcomes framework) (Rubin, 2005) can be applied to clearly specify the causal question without conflating with the modeling approach (Vandenbroucke et al., 2016); hence, it allows for a simple and transparent analysis. In addition, this approach minimizes bias from potential misspecification of the outcome model (Rubin, 2004b). Furthermore, matched, stratified, and weighted analyses do not make strong assumptions of linearity in the relationship of propensity score with the outcome. If a non-parametric preprocessing of the data using propensity score methods does not reduce model dependence, it is reasonable to accept that the data do not have enough information to reliably support the causal inference by any other statistical method. In fact, this knowledge in itself should still be useful and the conclusion may be correct (Rubin, 2004b; Ho et al., 2007; Rubin, 2007; Ali et al., 2016).

### Dimension Reduction

Propensity score typically summarizes a large number of measured pretreatment covariates to a single score; hence, it is called a "summary score." This is particularly useful in high-dimensional data with a substantially large number of pretreatment covariates compared to the number of outcome events including rare events, typical of most medication safety studies in pharmacoepidemiology (Glynn et al., 2006). In this setting, maximum likelihood estimations used in conventional regression techniques such as logistic and Cox regression require several outcome events for each parameter included in the regression model; the rule of thumb is that ≥ 10 outcome events are required per confounder included in a model (Peduzzi et al., 1995; Peduzzi et al., 1996). On the other hand, Cepeda et al. (2003) suggested using propensity score when there are fewer than eight outcomes per included covariate to effectively improve estimation.

## Doubly Robust Estimations

Generally, doubly roubst estimations (DR) estimation methods apply different procedures or models simultaneously and produce a consistent estimate of the parameter if either of the two models, not necessarily both, has been correctly specified (Imai and Ratkovic, 2014). Several applications of propensity scores have been described as DR in terms of estimating the effect of a certain treatment, including:


CBPS is optimizing covariate balance so that even when the propensity score model is misspecified, there will still be a reasonable balance of the covariates between the treatment and comparator groups. Unlike standard DR estimators, however, the CBPS approach does not require estimation of the outcome model.

4) Calculation of DR estimators using different approaches, for example, using the propensity score, predicted, and observed outcome (Ŷ and Y, respectively). This approach involves specifying regression models for the treatment (Z) and the outcome (Y) as a function of covariates (X) and combining these subject-specific values to calculate the DR estimate for each individual. First, treatment is modelled as a function of covariates to estimate propensity scores for each individual using the observed data. Second, the relationships between measured confounders and the outcome are modelled within treated and untreated groups separately. The resulting parameter estimates are then used to calculate predicted outcomes (Ŷ1, Ŷ0) for each individual in the population that is treated (setting Z = 1) and not treated (setting Z = 0) given covariate values. Third, the DR estimates of the outcome are calculated for each individual both in the presence and absence of treatment (DR1 and DR0), respectively) using the subject-specific predicted (Ŷ) and observed (Y) outcomes weighted by the propensity score. Finally, the means of DR1 and DR0 are calculated across the entire study population and these means will be used to calculate the effect of the treatment (Funk et al., 2011).

### Unmeasured Confounding

Propensity score methods, like other conventional regression methods, can account for only measured confounding factors and not unmeasured factors (Rosenbaum and Rubin, 1983). Therefore, propensity score analyses are only as good as the completeness and quality of the potential confounding variables that are available to the researcher. The only way to convince a critical reader that the study is not subject to unmeasured confounding is to have a rich set of covariates for constructing the propensity score model. Therefore, it is important to provide a detailed account of the variables collected and included in the propensity score model (Ali et al., 2015).

Modifications of the standard propensity score applications have been suggested to further reduce the risk of unmeasured confounding including the use of high-dimensional propensity score and propensity score calibration. High-dimensional propensity score refers to the use of a large number (in the range of several hundreds) of covariates to improve control of confounding; the underlying assumption is that the variables may collectively be proxies for unobserved confounding factors (Schneeweiss et al., 2009; Rassen et al., 2011). Propensity score calibration refers to the use of a "gold standard" propensity score estimated in a separate validation study, with more detailed covariate information unmeasured in the main study, to correct the main-study effect of the drug on the outcome (Stürmer et al., 2005; Stürmer et al., 2007).

Furthermore, sensitivity analyses (Rosenbaum and Rubin, 1983; Rosenbaum, 2005) are useful to assess the plausibility of the assumptions underlying the propensity score methods and how violations of them might affect the conclusions drawn (Stuart, 2010). Methods to deal with unmeasured confounding are summarized in **Figure 1**.

### Effect Modification

In estimating treatment effects, there is often an interest to explore if the effect of treatment varies among different subgroups (for example, men versus women) of the population under study, often called "treatment effect modification." There are many ways to utilize propensity score methods to adjust for confounding in a subgroup analysis; however, common implementation of propensity score matching in the medical literature is sub-optimal (Wang et al., 2017; Ali et al.,

TABLE 2 | Summary of considerations when planning, conducting, and reporting propensity score analysis.


2018a). The use of propensity score matched (PSM) cohort for subgroup analysis breaks the matched sets and might result in imbalance of covariates (Ali et al., 2018a). Depending on the frequency of treatment or outcome, small changes in the matched cohort might lead to large fluctuations for measures of association (Rassen et al., 2012).

To account for covariate imbalances, subgroup analyses of propensity score matched cohorts involve: i) adjusting for covariates in the outcome model or ii) re-matching within the subgroups either using the propensity score estimated in the full cohort or fitting new propensity score within subgroups (**Figure 4**) (Rassen et al., 2012; Wang et al., 2017). The choice of a specific method should take into account several factors: prevalence of the treatment and the outcome; strength of association between pretreatment covariates and the treatment; the true effect size within subgroups; and the amount of confounding within the subgroups (Wang et al., 2018).

### REPORTING

The credibility of any research depends on a critical assessment by others of the strengths and weaknesses in study design, conduct, and analysis. Hence, transparent and adequate reporting of critical aspects of propensity score-based analysis (Ali et al., 2015), like other observational studies, helps readers follow "what was planned, what was done, what was found, and what conclusions were drawn" (Von Elm et al., 2007). It also makes it easier for other researchers to replicate the study findings using other data sources and to judge whether and how results can be included in systematic reviews (Von Elm et al., 2007). Despite substantial methodological developments and common applications of the propensity score methods, in general, reporting on important features of the propensity score analysis is poor, incomplete, and inconsistent in the medical literature (Austin, 2008a; Ali et al., 2015; Ali et al., 2016; Wang et al., 2017). This could in part be due to a lack of standards for the conduct and reporting of propensity score based studies in guidelines. Therefore, critical items relevant to propensity score analyses should be incorporated in guidelines on the conduct and reporting of observational studies, such as the STROBE statement (Von Elm et al., 2007; Ali et al., 2015) and the ENCePP guide on methodological standards in pharmacoepidemiology (Blake et al., 2012; Ali et al., 2015) to improve the quality of the conduct and reporting of propensity score based studies (Ali et al., 2015; Ali et al., 2016). **Table 2** summarizes important consideration when planning, conducting, and reporting propensity score analysis and list of items that should be reported are summarized by Ali et al. (2016).

### REFERENCES


### CONCLUSION

Propensity score methods will remain important design and analytic tools to estimate effects of treatment from observational data. Preferably, they should be utilized in the design stage as tools for preprocessing of the data and they should be considered complementary tools, and not replacements, to conventional regression adjustments. In fact, when appropriate, propensity score methods should be used in combination with other model-based regression techniques. In addition, propensity score methods should not be regarded as magical remedies for the inadequacies of observational studies such as residual or unmeasured confounding (Rubin and Thomas, 2000; Ali et al., 2016). The ability of propensity score methods to overcome confounding is entirely dependent on the extent to which measured variables capture potential confounding. Taking full advantage of these methods requires explicit definition of the research question and appropriate choice of the propensity score method, transparent and detailed description of all subsequent statistical analyses to be conducted, and adequate reporting of the important aspects of the propensity score analyses (Ali et al., 2016).

## AUTHOR CONTRIBUTIONS

MA, DP-A, RF, MB, and LS contributed to the conception and design of the study. MA wrote the first draft of the manuscript. DR and NB wrote sections of the manuscript. All authors contributed to manuscript revision and read and approved the submitted version.

### FUNDING

This work is part of The 100 Million Brazilian Cohort project funded by the Wellcome Trust. Grant code: 202912/B/16/Z.


**Conflict of Interest Statement:** LS holds research grants from GSK, Wellcome, MRC, NIHR, BHF, and Diabetes UK and is a Trustee of the British Heart Foundation. DP-A holds research grants from NIHR, AMGEN, and UCB.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Ali, Prieto-Alhambra, Lopes, Ramos, Bispo, Ichihara, Pescarini, Williamson, Fiaccone, Barreto and Smeeth. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Administrative Data Linkage in Brazil: Potentials for Health Technology Assessment

*M Sanni Ali1,2,3\*, Maria Yury Ichihara3,4, Luciane Cruz Lopes5, George C.G. Barbosa3, Robespierre Pita3, Roberto Perez Carreiro3, Djanilson Barbosa dos Santos6, Dandara Ramos3, Nivea Bispo3, Fabiana Raynal7, Vania Canuto7, Bethania de Araujo Almeida3, Rosemeire L. Fiaccone3,4,8, Marcos E. Barreto3,9,10, Liam Smeeth1,3† and Mauricio L. Barreto3,4†*

*1 Faculty of Epidemiology and Population Health, Department of Non-communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, United Kingdom, 2 Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), Center for Statistics in Medicine (CSM), University of Oxford, Oxford, United Kingdom, 3 Centre for Data and Knowledge Integration for Health (CIDACS), Instituto Gonçalo Muniz, Fundação Osvaldo Cruz, Salvador, Brazil, 4 Institute of Public Health, Federal University of Bahia (UFBA), Salvador, Brazil, 5 University of Sorocaba–UNISO, São Paulo, Brazil, 6 Centro de Ciências da Saúde, Federal University of Recôncavo of Bahia (UFRB), Salvador, Brazil, 7 Department of Management and Incorporation of Health Technology, Ministry of Health (DGITS/MS), Brasília, Brazil, 8 Department of* 

#### *Edited by:*

*Brian Godman, Karolinska Institute (KI), Sweden*

#### *Reviewed by:*

*Gaurav Deshpande, Bristol Myers Squibb (United States), United States Tanja Mueller, University of Strathclyde, United Kingdom*

#### *\*Correspondence:*

*M Sanni Ali sanni.ali@lshtm.ac.uk*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*†These authors contributed equally to this work and share senior authorship*

*Received: 01 March 2019 Accepted: 31 July 2019 Published: 23 September 2019*

#### *Citation:*

*Ali MS, Ichihara MY, Lopes LC, Barbosa GCG, Pita R, Carreiro RP, dos Santos DB, Ramos D, Bispo N, Raynal F, Canuto V, de Araujo Almeida B, Fiaccone RL, Barreto ME, Smeeth L and Barreto ML (2019) Administrative Data Linkage in Brazil: Potentials for Health Technology Assessment. Front. Pharmacol. 10:984. doi: 10.3389/fphar.2019.00984*

*Statistics, Federal University of Bahia (UFBA), Salvador, Brazil, 9 Department of Computing, Federal University of Bahia (UFBA), Salvador, Brazil, 10 Institute of Health Informatics, University College London, London, United Kingdom* Health technology assessment (HTA) is the systematic evaluation of the properties and impacts of health technologies and interventions. In this article, we presented a discussion of HTA and its evolution in Brazil, as well as a description of secondary data sources available in Brazil with potential applications to generate evidence for HTA and policy decisions. Furthermore, we highlighted record linkage, ongoing record linkage initiatives in Brazil, and the main linkage tools developed and/or used in Brazilian data. Finally, we discussed the challenges and opportunities of using secondary data for research in the Brazilian context. In conclusion, we emphasized the availability of high quality data and an open, modern attitude toward the use of data for research and policy. This is supported by a rigorous but enabling legal framework that will allow the

Keywords: administrative data, Brazil, data linkage, epidemiological studies, health technology assessment, record linkage

conduct of large-scale observational studies to evaluate clinical, economical, and social

### INTRODUCTION

Health technology assessment (HTA), also known as healthcare technology assessment or medical technology assessment, is the systematic evaluation of the properties, intended and unintended effects and/or impacts of health technologies and interventions (Batarrita, 1999; Banta, 2009). It is an investigative process that evaluates the clinical (effectiveness and safety), economical (cost or costeffectiveness), ethical, and social consequences of using new or existing technologies in health with the main goal of improving "value for money" in health care (Banta, 2009; Kristensen et al., 2009). Health technologies can be any intervention in health with the aim of promoting health and prevent, diagnose, or treat disease, examples include: drugs, devices, procedures, and the organizational, educational,

impacts of health technologies and social policies.

Ali et al. Administrative Data Linkage in Brazil

informational, and support systems within which health care is delivered to the population (Banta, 2009). HTAs are useful to a wide range of decision makers in healthcare: government policy makers, insurance companies, and other payers, industries, planners, administrators, clinicians, and patients. Although the goal of HTA is to support policy decision and not sole knowledge generation, it must be carried out with integrity and using solid scientific methods to yield valid results (Banta, 2009).

In the mid-1980s, a new constitution in Brazil ruled that health is a right of all citizens and a duty of the State. This was the starting point for the building up of the Brazilian unified national health system [Sistema Único de Saúde (SUS), literally the "Single Health System"]. Since that moment, the Brazilian government showed strong interest in HTA, particularly after organizing and leading, in collaboration with Pan American Health Organization/World Health Organization (PAHO/WHO), on a HTA conference in Brasilia, Brazil (Banta, 2009; Guimarães, 2014). The discussions involved the political aspects of HTAs, including the questionable effectiveness of technologies used in health, cost and costeffectiveness, and the process of technology transfer. In the years from 2000 to 2008, substantial actions were taken in HTA: (1) Several seminars and consultations were held. (2) Key institutional changes were made at the Ministry of Health (MoH), including the formation of the Department of Science and Technology (DST); the Secretariat of Science, Technology and Strategic Inputs (SSTSI); and the Commission for the Incorporation of Technologies (the Commission). Later, the DST joined the International Network of Agencies in HTA. (3) Policies were developed by the federal government to encourage HTA and its application for clinical, management, and policy decisions. The SSTSI was then given the mandate for policy implementation in relation to pharmaceuticals within the SUS. At the same time, a process flow for incorporating technologies under the auspices of SUS and Supplementary Health System (SHS) was established, which was later redefined. Furthermore, the SSTSI was assigned to oversight the Commission; to evaluate and recommend the incorporation, alteration, or exclusion of products for the SUS and SHS procedure lists; to propose the revision of therapeutic guidelines; and to order and carry out HTA-specific studies. (4) Several academic institutions, such as the Federal University of Rio de Janeiro and the State University of Sao Paulo, developed HTA research initiative and the Brazilian Network for HTA (REBRATS), coordinated by MoH, was established by the adhesion of HTA groups scattered throughout universities, medical schools, and teaching hospitals (Banta, 2009; Guimarães, 2014).

In December 2011, the National Committee for Health Technology Incorporation into SUS (CONITEC) was created by the Brazilian Government through a Federal Law (12.401/2011) and regulated by Presidential Decree (7.646 from December 21, 2011). Aiming to provide greater agility, transparency, and efficiency to the health technologies, incorporation or disinvestment processes, CONITEC has set a new milestone to health management with innovative precepts in HTA in Brazil. With the objective of providing administrative, technical, and scientific support to CONITEC, the Executive Secretariat of CONITEC was created, which is in charge of the Department of Management and Incorporation of Health Technologies (DGITS) of the SSTSI of the MoH. The actions developed by DGITS, assisted by a network of national institutions (hospitals and universities), which are partners of CONITEC, have been successful in promoting public consultations, making decisions about claims, and acquiring health technology products and services (Guimarães, 2014). Currently, most of the HTA procedures within the REBRATS realm make predominant use of secondary sources of information, especially studies in the field of meta-analyses and other publications, for the incorporation of technologies (Guimarães, 2014).

Brazil has a long tradition of keeping records of healthrelated information for administrative purposes owing to the establishment of SUS, its informatics department (DATASUS), and the substantial progress made toward Universal Health Coverage (UHC) (Paim et al., 2011). Despite the efforts made by the MoH to harmonize the recording of information, a great disparity exists among various health institutions related to data collection processes. In addition, individual data collected by different health services (for example, hospital and mortality registries) lack unique key identifiers for individuals, hence, combining these data sources is not a trivial task. These factors, in addition to the technological infrastructure and skilled human resource constraints, have limited the use of routinely collected data to generate evidence to support clinical and policy decisions and to answer important epidemiological questions.

However, in the past decade, several big data and record linkage initiatives, different record linkage software packages (for example, Reclink, AtyImo, and CIDACS-RL) (Camargo and Coeli, 2000; Pita et al., 2018), and international collaborations on research and capacity building have emerged. The use of record linkage technology to integrate data that are not available in a single data set by supplementing information from other data sources and/ or validate information collected in one data source has made the conduct of health outcomes research possible in the Brazilian setting (Camargo and Coeli, 2000; Pinto et al., 2017). The HTA field in Brazil will benefit from such institutional and technological advances in data processing and analysis to produce evidence on the (cost-)effectiveness and safety of health technologies, as well as their impact of social, economic, and health policies. Hence, the objective of this manuscript was to review the main health care and socioeconomic databases, recent advances in the use of big data and data linkage tools; and to highlight the potentials and challenges of using secondary data and data linkage for health outcomes and policy research, as well as HTA.

This manuscript is organized as follows: the section *Databases Used in Health Outcomes/Policy Research in Brazil* describes the major databases used in health outcomes/policy research in Brazil, the section *Data Linkage* introduces important concepts in data linkage, the section *Data Linkage Initiatives in Brazil* describes major initiatives in creation of data centers and development of data linkage in Brazil, the section *Record Linkage Tools Developed and/or Used in Brazil* summarizes record linkage algorithms developed/used in Brazilian databases, the section *Challenges and Opportunities* highlights major challenges in the use of secondary data for health research in Brazil, and the section *Conclusion* concludes the manuscript.

### DATABASES USED IN HEALTH OUTCOMES/POLICY RESEARCH IN BRAZIL

In Brazil, the main databases storing health-related information are generated from SUS (de Mello Jorge et al., 2010; Paim et al., 2011; Souza et al., 2016). These databases can be classified into: (1) epidemiological (such as the Live Births Information System/SINASC; the Mortality Information System/SIM; the Information System for Notifiable Diseases/SINAN), which are used for surveillance, evaluation, and research to address public health questions; (2) administrative (such as the Outpatient Information System/SIA-SUS and the Hospital Information System/SIH-SUS), which are used for accounting and control of the production of the services provided; and (3) clinical, which are used to store clinical data on patients for future reference (Souza et al., 2016).

In addition, other government sectors also generate and manage data on demographic and socio-economic characteristics of the Brazilian population. For example, the Ministry of Social Development maintains an electronic database ("Cadastro Único," CadUnico) for provision of social services, such as the Conditional Cash Transfer Program (BFP) and the Housing Program (MCMV). These databases, in combination with others, have been used to study social determinants of health and evaluations of social policies on health (Rasella et al., 2013; Nunes et al., 2016; Machado et al., 2018). **Table 1** summarizes some details about the main databases publicly available from SUS and other governmental sources in Brazil.

### Cadastro Único

Large social inequalities and poverty are major historical characteristics of Latin America and Brazil, in particular (Bértola and Williamson, 2017). To reduce poverty and inequalities, Brazil has implemented several social protection policies including the Conditional Cash Transfer Program—"Bolsa Familia Program," the housing program "Minha Casa Minha Vida," and the access to water program "Cisterns," among others. In 2003, the Brazilian government created the unified registration for social programs, Cadastro Único (CadUnico), to facilitate implementation and to support decisions related to applications for any of the available social protection programs (Mostafa and Silva, 2007). CadUnico is an electronic database comprising individual records of 114 million people (57% of the Brazilian population, until 2015). It has information on the household characteristics and individual members, who applied to any of the 20 social benefits (from 2004 onward) and those who received any social benefit (since 2001), including detailed demographic, economic, and social conditions of the household (Mostafa and Silva, 2007; Rodrigues, 2017). It is continuous for new applicants, and those already registered have to update the information every 2 years. For those who had their benefits turned down but want to reapply and those who are already receiving a benefit, the information update is mandatory. The extensive coverage of this social registry, the availability of individually identified data, and the possibility of linking them to other health care databases, such as SIM/SINASC/SINAN, allow TABLE 1 | Databases From the Brazilian Public Health System (SUS) and Other Government Sources.


for designing individual level longitudinal studies to evaluate the impact of social protection programs on health outcomes (such as diseases, hospitalizations, and deaths) (Paim et al., 2011; Rasella et al., 2013; Nunes et al., 2016; Machado et al., 2016a) and has inspired the development of the 100 Million Brazilian Cohort (Pinto et al., 2017; Pita et al., 2018).

### Bolsa Família Program

The Brazilian government introduced the largest conditional cash transfer program in the developing world called "Bolsa Família Program" in 2003 as a merger of the pre-reform cash transfers (Lindert et al., 2007). The aim was to reduce current poverty and inequality, by providing a minimum level of income for extremely poor families and to break the inter-generational transmission of poverty by conditioning these transfers on beneficiary compliance with human capital requirements. The conditionalities include: 1) children aged 7 to 17 years have to attend a minimum of 85% schooling days; 2) children up to 7 years of age must complete vaccination and growth monitoring; and 3) beneficiary families with pregnant women, nursing mothers, or children younger than 7 years should follow a health and nutrition agenda (pre- and postnatal care, vaccination, and health and nutrition surveillance). It is implied that making the benefits conditional on "positive" behaviors can further increase the chances of breaking out of the poverty cycle through increased education or improved health. The program also seeks to help empower BFP beneficiaries by linking them to other complementary services, such as health and education (Lindert et al., 2007; Paes-Sousa et al., 2011).

BFP targets were identified through geographic and household assessment methods based on per capita household income. Geographic targeting is applied at federal and municipal levels where as family eligibility is determined based on household registry data that was collected locally and transmitted to the central database, the CadUnico (Lindert et al., 2007). The cash transfers are intended for poor and extremely poor households, with additional payments when a household include children up to 17 years of age (up to two payments per family), or pregnant women (up to nine monthly payments) or lactating women (up to six monthly payments). The original income ceilings for eligibility to the BFP program were set at a fixed monthly per capita household income of R \$100 (US \$48) for poor families and R \$50 (US \$25) for extremely poor families. To account for increases in the cost of living, the thresholds were increased in 2006 to R\$120 (US \$57) for poor families and R\$60 (US \$29) for extremely poor families (Machado et al., 2018). Additional adjustments were made in 2009 (R\$140 for poor and R\$70 for extremely poor families) and in 2014 (R\$154 for poor and R\$77 for extremely poor families). BFP covers 23% of the Brazilian population with the benefits ranging from \$18 to a maximum of \$175 per month. The mother, when present, must receive the monthly payment on behalf of the whole family (Paiva et al., 2013).

The BFP has attracted significant attention both in Brazil and beyond. As such, several studies have been conducted to evaluate the impact of this program on several health-related outcomes, such as poverty reduction (Soares et al., 2006), inequalities (Soares et al., 2006), crime (Chioda et al., 2016; Machado et al., 2018), leprosy incidence (Nery et al., 2014), and child mortality and hospital admissions (Rasella et al., 2013). Information recorded for each household include date of start of the benefit, period of receipt, and amount of monthly cash transferred (Ministro da Cidadania, 2017). This database, in combination with CadUnico, provides socio-economic information for nearly half of the Brazilian population in the lower income category.

### SINASC

The Live Births Information System (Sistema de Informação Sobre Nascidos Vivos [SINASC]), created in 1990 by the MoH, contains vital information on live births in Brazil with the most significant characteristics about the newborn, the mother, the pregnancy, and the delivery. The system includes consolidated data since 1994 and operates with a standardized model of the birth certificate (the Declaration of Live Birth, DNV, a "declaração de nascido vivo"), a legal document completed by the health provider who assisted the delivery and then collected by health secretariat (Frias et al., 2014).

SINASC includes information on place of delivery (hospital or home), the mother who gave birth (including name, age, place of residence, marital status, education, number of children, and number of previous live and still births), the pregnancy (number of prenatal appointments, length of gestation, type of pregnancy: singleton or twin, type of delivery); and the newborn (gestational age, birth weight, sex, ethnicity, the presence and type of birth anomalies for live and stillbirths, and 1- and 5-min APGAR score for live births) (da Saúde, 2011). SINASC uses the International Classification of Disease Version 10 (ICD-10) for coding congenital defects (do Nascimento et al., 2018).

Data must be uploaded by SUS's local level manager, the Municipal Health Secretariat, who are also responsible for processing, consolidating, evaluating, and analyzing these data to support decision-making at local level. Data completeness and coverage are very high, with more than 90% completeness for most variables at country level and capturing 97% of Brazilian registered births (Pedraza, 2012; Oliveira et al., 2015). However, this coverage is heterogeneous within the country, with large variations among the states and some with low percentages particularly those located in the North and Northeast regions. In addition, under-registration of births is still common in some regions of the country and inconsistency of records in variables, such as mother's education, race, and number of prior childbirths, is still high in North and Northeast regions (Oliveira et al., 2015; Hunter and Sugiyama, 2018). SINASC, in combination with SIM (the Mortality Information System) and SIH (the Hospital Information System), has been used to study the impacts, burden, and/patterns of diseases (Paixao et al., 2018), pregnancy-related hospitalizations (Moura et al., 2018), impact of socio-economic inequalities on prenatal consultation (Mallmann et al., 2018), factors affecting neonatal mortality (Kropiwiec et al., 2017; Paixao et al., 2018), the use of ICD-10 coding system on congenital disease ascertainment (do Nascimento et al., 2018). SINASC and SIM/SIH also provide data that are used as parameters in the HTA studies.

### SIM

The Mortality Information System (Sistema de Informação sobre Mortalidade [SIM]) was the first subsystem of health information created in 1975 and managed by MoH, containing records of all deaths in Brazil, including fetal deaths. These records are based on the standard death certificate (called the Declaration of Death; DO, "declaração de óbito"), a required legal document, and fetal death certificate collected by the state health secretariat, which contributes to the improvement in the registration of data (Oliveira et al., 2015). Information recorded include: name, date of birth, date of death, sex, ethnicity, educational level, marital status, occupation, place of death, type of health service where death occurred (hospital, another type of health unit, home, or elsewhere), ICD-10 code causes of death (main and secondary) and comorbidities (up to two) (Victora and Barros, 2001). The correct coding of the cause of death, according to ICD-10, is of great importance for the good quality of SIM data. Like SINASC, the coverage in SIM is heterogeneous within the country, with large variations among the states and some with low percentages particularly those located in the North and Northeast regions (Victora and Barros, 2001).

SIM, with SIH/SINASC, has been used in linkage studies (Kropiwiec et al., 2017; Paixao et al., 2018), characterization of trends and regional patterns in (cause-specific) maternal and infant mortality (Victora and Barros, 2001), trends and disparities in cancer mortality (Alves et al., 2009; Girianelli et al., 2014; Braga et al., 2017; Prado da Fonseca et al., 2018), among others. SIM has good coverage and quality, and death characterization. However, delays in data processing, under-reporting of deaths, high numbers of ill-defined cause of death, variation of the quality and coverage in different geographical areas, as well as incorrect filling of death certificates, are some of the limitations (Victora and Barros, 2001; Santos et al., 2008).

### SINAN

The Notifiable Diseases Information System [Sistema de Informação de Agravos de Notificação (SINAN)] was implemented gradually and disorderly from 1993. In 1998, it became mandatory to feed the system with data on diseases of compulsory notification, such as tuberculosis, leprosy, human immunodeficeincy virus/acquired immune deficiency syndrome (HIV-AIDS), leishmaniasis, dengue, and Zika. There is a national list of these diseases; however, some states could also include their own specific health problems or outbreaks. Information on disease, using ICD-10 code, is collected through forms filled by health professionals who attend patients with suspected diseases. There are three documents: 1) Individual Notification Form (FIN), filled at the hospital when there is a suspicion of obligatory notifiable disease, outbreak, or new/unknown diseases. This form is followed by: 2) Negative Notification, when there is no disease confirmation; 3) Individual Investigation Form (FII) on identification of the source of infection and transmission mechanism. Each disease record includes different variables, but all include: name, sex, date of birth, place of residence, years of education, date of onset, and clinical aspects of the disease such as symptoms, laboratory tests, disease severity, and sometimes the outcome of the treatment (Paixao et al., 2018). SINAN facilitates the study determinants of obligatory notifiable diseases; indicates the risks of diseases; and facilitates standardization of procedures, investigations, and forms for notifiable diseases. However, there is under-reporting especially of patients from private practices, delay in data processing and correction, and long and complicated information flow (Laguardia et al., 2004).

SINAN, linked to SIM and/or SINASC, has been used to evaluate maternal and child health outcomes (Paixao et al., 2018), incidence and prevalence studies (Tanaka et al., 2017), tuberculosis (Oliveira et al., 2012a; Saraceni et al., 2018), and HIV studies after linkage with other administrative databases SISCEL (Laboratory Tests Control Systems) and SICLOM (Medication Logistics Control System) administrative databases made available in 2000 and 2006, respectively (Saraceni et al., 2018). SICLOM database covers all people living with HIV and receiving ART (antiretroviral therapy), both in public and private health care sectors. SISCEL database, on the other hand, covers only those people living with HIV who had CD4 and viral load tests conducted in public laboratories (Saraceni et al., 2018).

### SIH

The Hospital Information System [Sistema de Informações Hospitalares (SIH)] is the national administrative database established in 1991 and comprises information on patient admissions in the network of public hospitals under SUS and private hospitals contracted by the SUS. It has information on over 75% of the country's hospitalizations that are covered/ funded by the SUS. Hospitalizations in SUS require completion of a standard form (authorization for hospitalization) that captures patients' personal data, symptoms, and ICD-10 codes of the initial diagnosis. This form and other information recorded by the SIH-SUS on diagnoses, treatment, test results, and billing are standardized throughout Brazil. The resulting data are checked and validated by local health authorities and subsequently transmitted to regional and national levels (Coelho et al., 2016).

Variables recorded include: sex, age, number of hospitalizations, the total amount and value of reimbursed hospital services, days and average length of stay, mortality, among others (Melione and Jorge, 2008; Coelho et al., 2016). SIH has high agility, good morbidity information; it is regularly submitted to audit and payment review; and also allows for monitoring of surveillance epidemiology. However, it mainly covers the public health system which accounts for about 70% of the total admissions and it is constantly changing (Medeiros et al., 2005; Melione and Jorge, 2008; Coelho et al., 2016; Machado et al., 2016a). Although it was created with an administrative purpose, it has been frequently used to monitor population health states and observational studies of adverse drug events (Martins et al., 2018) and health care costs (Quarti Machado Rosa et al., 2018).

### SIA-SUS

The Outpatient/Ambulatory Information System of SUS [Sistema de Informações Ambulatoriais do Sistema Único de Saúde (SIA-SUS)] was implemented throughout the country in 1995 and records outpatient visits through the Ambulatory Production bulletin (BPA). Data processing occurs in a decentralized way in which each state and municipality, duly qualified, can register, program, process, and pay for the production of its health facilities under its management. For the generation of information, SIA uses some basic systems, such as the SUS Procedure Chart Management System (SIGTAP), capture application, such as magnetic ambulatory production bulletin, and authorization of magnetic ambulatory procedures. Both capture applications allow recording of basic-, medium-, and high-complexity care procedures. SIA is widely used for HTA studies, since it provides, in addition to the quantitative procedure performed in the SUS, the cost of these procedures for SUS (Machado et al., 2016b).

### APAC-SIA

The System of High Complexity Procedures Authorization (APAC-SIA) is a SIA sub-system, established in 1996, constituted by individual registers of high-cost ambulatory procedures and highcost medicines for specific diseases such as biologics (Brito et al., 2005; Peres et al., 2016; Machado et al., 2016b). Access to high-cost medicines is *via* SUS's Specialized Component of Pharmaceutical Service through a form that comprises useful clinical information of the patient (Machado et al., 2016b). This database contains information on name, national health card number, age, sex, mother's name, address, main procedure code and name, amount of procedures, brief description of diagnosis, ICD-10 code, concomitant diseases, and health care professional number and register code.

The High Complexity Oncology Procedures Authorizations (APAC-ONCO) database contains additional information, including diagnosis date, primary cancer site; histopathology description and final diagnosis; ICD-10 topography; lymph node invasion (yes/no); metastasis locations; tumor, node, metastasis (TNM) stage; and stage by different system. It also records information on previous treatment and current treatment (surgery, chemotherapy, or radiotherapy), including description and start date, scheme, planned duration, and irradiated areas. APAC-ONCO has been used in cancer studies after linkage with other databases, such as SIM (Machado et al., 2016b; Peres et al., 2016), SIH, the breast cancer screening information system (SISMAMA) (Peres et al., 2016), and the hospital-based cancer registry (RCBP) (Peres et al., 2016).

### Cancer Information Systems

The cancer registry is a service for collecting, storing, analyzing, interpreting, and systematically disseminating cancer data and includes: 1) The Hospital Cancer Registry (RHC) for recording information about cancer patients seen in a particular hospital (public or private) responsible for oncology care. RHC has administrative purposes, such as estimation of future demand, equipment needs, and human resources. It is considered highly representative of the baseline population and is useful to determine diagnosis efficiency, stage at diagnosis, and treatment. 2) The Population-Based Cancer Registry (RCBP) which was established in 1967 with 26 centers located mostly in major cities. It monitors the frequency of new cancer cases between regions and over time by collecting diagnoses from different sources (clinicians and pathologists) or the death data (when the main cause is cancer). Both registries record socio-demographic information about the patient (age, education level, marital status and place of residence), family history of cancer, source and year of referral, date of diagnosis, diagnosis and previous cancer treatment, characteristics of the tumor (synchronous tumor and laterality), date of first appointment and initiation of the treatment, type of treatment received, stage at diagnosis and tumor evolution after the treatment, and cost of diagnosis and treatment (Ferreira et al., 2017).

The National Cancer Institute (INCA) branch of the MoH in partnership with DATASUS has also implemented the Information System for the Control of Breast Cancer (SISMAMA), an online tool that register information about breast and gynaecological cancer screening. In Brazil, mammograms are encouraged by SUS targeting the female population older than 50 years and has been performed every 2 years, or annually in the case of altered clinical examinations (Cecilio et al., 2015). Women presenting with familial history are encouraged to undergo annual screening of the breasts (Lima-Costa and Matos, 2007). It is estimated that 50% of Brazilian women older than 50 years have had at least one mammography in their life (Lima-Costa and Matos, 2007; Anderson et al., 2011). SISMAMA was conceived as a management tool that capture, organize, and make available data about the population tested; test results (mammograms and ultrasounds, and breast cytopathology and histopathology); follow-up of abnormal cases; the quality of the

services; as well as other essential information generated in the course of providing screening tests (Passman et al., 2011). Data collection begins in the primary care setting, typically with a physician's order for a screening or diagnostic mammogram. Mammography results are classified using the Breast Imaging Reporting and Data System (BI-RADS) developed by the American College of Radiology (ACR). SISMAMA has been used in several studies after linkage with SIM, SIA-APAC, and SIH-SUS (Freire et al., 2015; Peres et al., 2016; Tomazelli et al., 2018a; Tomazelli et al., 2018b).

### SI-PNI

The National Immunization Program Information System [Sistema de Informação do Programa Nacional de Imunização (SI-PNI)] contains records on dispensed immunobiologicals. It was developed by PNI in partnership with DATASUS and is comprise several subsystems: 1) the Information System of the Immunization Program Assessment (SI-API) which provides data on vaccination coverage (routine and campaigns), dropout rate, immunization control bulletins. API can be used by the federal, state, regional, and municipal levels; 2) the Immunobiological Inventory and Distribution Information System (SI-EDI) which controls the supply and distribution of immunobiologicals at the state and federal levels; 3) the Information System of adverse events following vaccination (SI-EAPV), which allows the vigilance of adverse events after administration of the vaccine; 4) the Information System of the Instrument Evaluation Program (SI-PAIS) and the Information System of the Evaluation Program of the Supervision Instrument in Vaccine Room (SI-PAISSV), which contribute for standard evaluation profile and fast delivery of tabulated results; 5) the Information System for the Assessment of Immunobiologicals Used (SI-AIU), which evaluates the lost and utilized doses; 6) the Information System of the Reference Center for Special Immunobiological (SI-CRIE) which informs adverse events and utilization of special immunobiologicals (da Nóbrega et al., 2010).

SI-PNI enables quantitative analysis of vaccination coverage by vaccine type, doses given, and dropout rate throughout the country by age group, time, and geographical area (Assis Moura et al., 2018). Within SI-PNI, it is also possible to perform monthly follow-up of vaccination activities regarding the quantity of distributed and applied doses, coverage, and adverse events post-vaccination (EAPV). SI-PNI uses single identifying number shown on the "National Health Card" [Cart ao Nacional de Sa ude (CNS)], hence, the vaccinated and their origins can be identified, allowing to find unvaccinated ones and give them a dose. Linkage to different national databases enables conduct of observational studies on vaccine effectiveness (Domingues and Teixeira, 2013; Sato, 2015). In addition, SI-PNI can be used as parameter to assess and modeling economics evaluation of new vaccines.

### SIAB

The Basic Health Care Information System [Sistema de Informação da Atenção Básica (SIAB)] was created in 1998 by DATASUS, in conjunction with the co-ordination of community health/ health care Secretariat (COSAC/SAS). It assists monitoring and evaluation of activities carried out by the community health agents (ACS), aggregating and processing the data from the home visits, as well as the medical and nursing care performed in households and health unit (Da Silva and Laprega, 2005; Frias et al., 2012). Data are collected using the forms for enrolment, and follow-up of families is served by the family health teams and community health agents. It contains data on socio-economic characteristics; health (morbidity); residences of households and their individuals; and medical follow-up data on priority groups such as pregnant women, diabetics, hypertensives, and leprosy patients having tuberculosis, and children younger than 2 years (the mother's name and address, age of the child, date of death, and cause of death). In addition, medical and nursing consultations, request for additional examinations, referrals, as well as notification of some diseases, for example, pneumonia in children younger than 5 years are recorded.

### SISLAB

Brazil has a national network of public laboratories, the National System of Public Health Laboratories [Sistema Nacional de Laboratórios de Saúde Pública (SISLAB)]. The laboratories are organized hierarchically (national, regional, state, and municipal level) by the degree of complexity of activities, in accordance with the principles of SUS, related to health surveillance including epidemiological surveillance, surveillance in environmental health, sanitary surveillance, and medical assistance (da Saúde, 2004). In 2008, the MoH, aiming to improve laboratory information through the General Coordination of Public Health Laboratories (CGLAB) and DATASUS, elaborated the Laboratory Environment Management System [Gerenciador de Ambiente Laboratorial (GAL)]. GAL is a free software with its own communication patterns, distributed, robust and flexible architecture, and multiplatforms (Jesus et al., 2013; Júnior et al., 2017). The national module of the GAL manages, monitors, and concentrates the results of the laboratory tests informed by the State Modules of the following six areas: Medical Biology, Environmental and Worker Health, Animal, Quality Control, Management, and Quality and Biotechnology. Therefore, GAL is a computerized system applied to the examinations and tests of samples of human, animal, and environmental origins, following the protocols of the MoH.

GAL sends laboratory test results from suspected or confirmed cases of Compulsory Notifications (flu, tuberculosis, leishmaniasis, dengue, zika, yellow fever, pertussis, and meningitis, among others) to the SINAN. It also contains data on viral hepatitis markers, serological diagnosis of HIV, tumor markers, diagnosis of zoonosis and related biological factors, analysis of water quality in health facilities and environmental health surveillance service (Jesus et al., 2013; Júnior et al., 2017). The SISLAB-GAL data contribute decisively to surveillance in Brazil, but its integration with other social-economic information (for example, data from CadUnico) and health care data (for example, SIM/SIH) would allow to conduct several epidemiological studies.

### NOTIVISA

The National Notification System for Health Surveillance [Sistema de Notificações em Vigilância Sanitária (NOTIVISA)], created in 2008, is an online computerized information system of the National Sanitary Surveillance Agency (ANVISA) that receives spontaneous reports of suspected cases of Adverse Drug Events. It covers the Brazilian territory and is considered the largest and most important repository of Adverse Drug Events data from the National Pharmacovigilance System (SINAF) of the country (de Vigilância Sanitária, 2008). NOTIVISA allows the obtaining and circulation of information on health problems to users, sudden or undesirable effect, and/or malfunctions related to health products marketed in Brazil. The NOTIVISA system has enabled adoption of adequate measures of control, safety alerts, besides providing information to update the existing legislation and/or to propose new legislation as well as sanitary recommendations for the adoption of measures that ensure the protection and health promotion of the population (Branco et al., 2015).

Data from NOTIVISA have been used in studies to investigate the occurrence of reports related to health products in the post-marketing phase, such as the occurrence of Adverse Events and Technical Complaints related to the use of a vascular catheter (Oliveira and Rodas, 2017), to describe the adverse events related to healthcare products that resulted in death in Brazil (Maia et al., 2018).

### SNGPC

The National System for Management of Controlled Products [Sistema Nacional de Gerenciamento de Produtos Controlados (SNGPC)], implemented in late 2007 and early 2008, is a sanitary surveillance information system that captures dispensing movement data (inputs and outputs) of the drugs subject to the special control as well as antimicrobials and updates in pharmacies and private drug stores in the country. The SNGPC main objectives include: to monitor the dispensation of drugs and narcotics, and psychotropic substances and their precursors; to optimize the book keeping process; to allow monitoring of prescription habits and consumption of controlled substances in a given region to propose control policies; to collect data that allow the generation of up-todate and reliable information for the National Health Surveillance Service (SNVS) for decision making; and to streamline the actions of health surveillance (de Vigilância Sanitária, 2010)

The data feeding the system comes from the prescription of qualified medical professionals, retained at the time of dispensing the drug in the pharmaceutical establishment, and invoices for the purchase of medicines suppliers. The main operational actors of the SNGPC are the pharmacists in charge of the pharmacies and drug stores, and SNVS health surveillance professionals. SNGPC has been used in studies to examine the consumption of appetite suppressant drugs (Mota et al., 2014), the consumption of psychotropic anorectic drugs (Martins et al., 2012), and the frequency as well as distribution of the consumption of benzodiazepine anxiolytics in private pharmacies and drug stores (Azevedo et al., 2016).

### PFPB

The Popular Pharmacy Program [Programa Farmácia Popular do Brasil (PFPB)] was created in 2004, within the scope of the SUS, to expand access to medicines for the most common diseases among citizens. One of the objectives of the program was to favor low-income people by making treatment feasible in the face of the high price of medicines. It also supports the population of the private health network as an alternative, since they have access to medicines with prices more affordable. PFPB was also aimed to contribute to the reduction of the expenses generated by the purchase of medicines and minimize the expenses of the SUS with hospitalizations that are caused by the abandonment of the treatment (Inocencio and De Vivo, 2011).

The PFPB developed two axes of action: the own network of public Popular Pharmacies (rede Própria) and accredited private retail pharmacies of PFPB (PFPB-E, "Aqui Tem Farmácia Popular" or ATFP). Popular Pharmacies, operating since 2004, have a list of 112 medicines which are dispensed at cost representing a reduction of up to 90% of the market value. The PFPB-E, considered as an expansion of PFPB in partnership with pharmacies and drugstores of the private network, was created with the objective of expanding the coverage of pharmaceutical assistance and promoting the integrity of health care (Coelho Filho et al., 2004). In this modality, the MoH subsidizes 90% of the reference value for diseases, such as dyslipidemia, Parkinson's, glaucoma, osteoporosis, rhinitis, contraceptives, and geriatric diapers. As of 2011, with the creation of "Health Without Price" ("Saúde Não tem Preço"), the two axes of action started to count on free medicines for asthma, diabetes and hypertension. It has the potential for linkage since it contains tax number (Cadastro de Pessoas Físicas [CPF]) of individual patients (Coelho Filho et al., 2004).

### SINITOX

The National Toxic-Pharmacological Information System [Sistema Nacional de Informações Tóxico-Farmacológicas (SINITOX)] was created in 1980 and is linked to FIOCRUZ. It is responsible for the collection, compilation, analysis, and dissemination of cases of intoxication and poisoning registered by the National Network of Information and Assistance Centers Toxicological—RENACIAT. RENACIAT is currently composed of 36 units located in 19 states and the federal district which provide information and guidance on the diagnosis, prognosis, treatment and prevention of intoxications, as well as on the toxicity of chemical and biological substances and the risks they cause to health (Bortoletto and Bochner, 1999).

The differences in structure, setup, and content of all these different databases can lead to significant challenges in use of these data for HTA and decision-making. In addition, there are considerable challenges regarding the lack of governance. Most often, there are poor or no standards for collaboration; there is a lack of incentives for data sharing; and there are issues with regard to patient consent, privacy, and data security that may severely hamper access to such data. As a result, the costs for data protection would be very high to comply with relevant regulation (Annemans, 2017).

Despite the availability of all these SUS and related databases, there are still key challenges in the use of secondary data for HTA, pharmacovigilance, and supporting decision-making. Data linkage is one of the available approaches that can be used to mitigate lack of integration and standardization observed in such databases. Record linkage can help generate useful and high quality data sets to conduct research, and support formulation and evaluation of public policy. However, linking these databases is not a trivial task mainly due to the lack of common key identifiers amongst all the databases, as well other technical issues related to data quality, standardization, availability, and volume (number of records). **Table 2** summarizes some attributes encountered in most of these databases, which are potential candidates for linkage purposes.

### DATA LINKAGE

Data linkage, also called record linkage, is the process of combining records about the same individual or entity from two or more different data sources (Winkler, 2006; Jurczyk et al., 2008a) or the process of identifying duplicate records in the same data set (Jurczyk et al., 2008a). In principle, record linkage problem consists of developing a classifier that categorizes record pairs as "linked" or "non-linked" with reasonable accuracy (Jurczyk et al., 2008a). It enables the aggregation of data not available in a single data set thereby supplementing information on an individual with information from other data sources, validating information collected in one data source, or to de-duplicate records within a single data source (Winkler, 2006; Jurczyk et al., 2008a). Record linkage also has additional applications, such as building longitudinal profile of individuals and case-identification in capturerecapture studies (Sayers et al., 2015).

There are two main types of linkage algorithms: deterministic and probabilistic. Deterministic linkage methods vary from a one-step procedure using a single unique identifier or a set of several attributes (called "exact" deterministic linkage) to step-wise algorithmic linkages involving a series of progressively less restrictive steps to allow variation between record attributes (called "iterative" deterministic linkage). A record pair is classified as "linked" if it meets the criteria or parameters at any step; otherwise is classified as "non-linked" (Dusetzina et al., 2014). Probabilistic linkage methods, on the other hand, takes advantage of differences in the discriminatory power of each attribute and apply calculation of similarity scores, as well as decision rules, to classify record pairs as linked, potentially linked (treated as dubious records in most linkage tools) and non-linked (Newcombe et al., 1959; Fellegi and Sunter, 1969; Dusetzina et al., 2014). It can also

TABLE 2 | Potential Linkeage Attributs amongst SUS databases.


deal with some inconsistencies between records with missing data, i.e., it has the capacity to link records with errors in the linking fields (Dusetzina et al., 2014).

Since its introduction by Newcombe (Newcombe et al., 1959) and mathematical formalization by Fellegi and Sunter (Fellegi and Sunter, 1969), several variations of record linkage and computerized tools have emerged to meet different requirements and challenges, such as accuracy, speed, and scalability. Many of these tools have a general purpose, allowing a combination of existing configurations and methodologies (Camargo and Coeli, 2000; Elfeky et al., 2002; Christen et al., 2004; Christen, 2008; Schnell et al., 2009; Pita et al., 2018). While most of these methods are probabilistic, some of them apply a combination of deterministic and probabilistic linkages (called "hybrid" methods) (Pita et al., 2018). In general, a successful linkage processing involves several main steps: pre-processing, blocking and indexing, field comparison, weight vector classification, and accuracy assessment (Christen, 2008) as depicted in **Figure 1**.

The pre-processing step involves data cleansing and standardization whereby incomplete and incorrectly formatted data is converted into well-defined, consistent form (Christen et al., 2004; Christen, 2008). Specific approaches to deal with missing data can be applied at this step to i) remove missing fields or entire records or ii) impute missing values based on standard or calculated values. Pre-processing may also involve anonymization using different privacy-preserving techniques, such as Bloom filters (Inan et al., 2008; Pita et al., 2018), to protect sensitive data from disclosure and unauthorized use.

Executing a linkage routine between data sets A and B will result in a number of field comparisons defined by the product |*A*| ∗ |*B*|. In a big data context, these numbers make pairwise comparisons impractical and lead to a number of infrastructure, data processing, and data analysis challenges (Peek et al., 2014; Harron et al., 2017). To circumvent scalability challenges over big data sets, different approaches have been used in the literature, such as parallelism/distribution and blocking (or indexing) strategies, as well as their combinations (Christen, 2008; Pita et al., 2018). Other initiatives have also proposed the use of cluster-based platforms, multi-processors or graphics processing units (GPUs) (Boratto et al., 2018; Pita et al., 2018). Blocking and indexing step generates pairs of candidate records pertaining to the same comparison blocks (Christen, 2012). These methods drastically decrease the number of candidate record pairs to a feasible number thereby speeding up the linkage performance over big data sets while still maintaining linkage accuracy. Several indexing techniques used in linkage solutions are well described in the literature (Christen, 2012).

The field comparison step involves using several functions to measure the similarity of attributes for each record pair. The choice of the functions is dependent on the content of the field: string comparison functions are used for names and addresses whereas numerical comparison functions are used for fields, such as date, age, and numerical values (Christen, 2012). Once a vector of numerical similarity values is calculated for each record pair, the candidate record pairs are classified as linked (i.e., candidate pairs that are linked deterministically or probabilistically by the linkage software), non-linked or possibly linked, based on one or more cutoff (threshold) points, in the weight vector classification step as shown in **Figure 2**.

During the final step—accuracy assessment—evaluates the linkage algorithm and the quality of the linkage (i.e., it estimates rates of linkage errors: missed matches and false matches).Linkage accuracy is often assessed using a gold standard dataset where the true match status of each pair of records is known. Comparing the probabilistically linked dataset to the gold-standard dataset will identify true matches, true non-matches, false matches, and missed matches. Hence, measures of linkage quality such as sensitivity, positive predictive value, and F-measure can be easily derived. When gold standard dataset is not available,alternative approaches such as sensitivity analysis, comparison of characteristics of linked and nonlinked data, and identification of implausible matches could be used to quantify the rate of linkage errors (Christen, 2008).

linkage pipeline. Three cut-off points can be chosen according to specific needs. Any pair in between sensitivity and specificity cutoffs is considered a dubious match and thus passed for manual review. When manual review is not possible, a common approach is to choose a cut-off point that averages sensitivity and specificity, maximizing accuracy.

### DATA LINKAGE INITIATIVES IN BRAZIL

In the last decade, the use of big data for research in Brazil has increased substantially due to several factors: data access, creation of research groups and data centers, development of efficient record linkage tools, and international research collaborations, among others. In this section, we describe three data centers specialized in the use of big data as well as development of record linkage tools:

### CIDACS

The Centre for Data and Knowledge Integration for Health [Centro de Integração de Dados e Conhecimentos para Saúde (CIDACS)] is a data linkage center managed by the Oswaldo Cruz Foundation (FIOCRUZ),´ officially launched in December 2016 and located in Salvador, State of Bahia, Brazil. It houses the 100 Million Brazilian Cohort (Pita et al., 2018) and is responsible for housing other large databases, such as SINAN, SIM, and SINASC, as well as the development of other new and innovative studies using these large databases. An agreement signed between the Ministry of Social Development (MoSD), FIOCRUZ, UFBA, and the University of Brasilia (UnB); the MoSD permitted the acquisition, after ethical approvals, of a copy of CadUnico and BF payments from 2004 to 2015. Negotiations with the MoH, in particular, the Department for Health Information (DATASUS) granted CIDACS copies of SINASC, SINAN and SIM from 2000 to 2015. Copies of SIH, SISVAN, and "Minha Casa Minha Vida" (housing program) are also available, whereas access to other databases, such as "Cisterns" (Wells), is in negotiation at the time of writing this paper.

The center operates with a strong governance; an advanced data platform comprising the computational infrastructure needed for receipt, storage, curation, and integration of large databases, and extraction of data sets for specific analysis; and a physical structure carefully designed to give full physical protection for the data when

handling non-anonymized data sets as well as to manage access and analysis of de-identified or anonymized data sets. All standard operation procedures (SOPs) for data manipulation, cleaning, linkage, and for meta-data production are being defined according to international standards. In the past few years, CIDACS has developed algorithms for data anonymization and data linkage including the two linkage algorithms ("AtyImo," a tool used in less safe environments where the identifier information must be masked or anonymized, and "CIDACS-RL," which is used under extremely safe environments without masking). Linkage using both tools were already validated and, for the optimum threshold (best trade-off of sensitivity and specificity), the accuracy of both algorithms is above 90% (Pita et al., 2018).

### Minas Gerais

The research group at the Federal University of Minas Gerais had also made significant achievement in record linkage. In partnership with the MoH's team, the team has been working on the National Health Database Centered on Individual: a 15-year cohort of individual-level historical data, preserving patient privacy, integrating SIH, SIA, SIM, SINASC, and SINAN (Guerra et al., 2018). It will allow researchers to generate real-world evidence using clinical, pharmacological, and pharmacoeconomic studies.

The group has also developed a parallel deduplication algorithm, called FER-APARDA, using probabilistic record linkage, as well as PAREIA (Santos et al., 2007; dos Santos Filho, 2008). PAREIA's crucial contributions are two-fold: 1) the proposed blocking scheme uses predicates from fields or portions of them, making a junction of disjunctions to prevent input errors to separate true matches from the right blocks. 2) The use of high-performance computing techniques and programming languages to guarantee its suitability to big data scenarios. This initiative has enabled several scientific investigations, such as the study of a criminal network by identifying distinct offenders on a graph-based police event database (dos Santos Fraga, 2009).

### Rio de Janero

The research group at the State University of Rio de Janeiro has made substantial contribution on data science and record linkage in Brazil, including the largely used probabilistic record linkage tool based on Fellegi-Sunter model, RecLink (Camargo and Coeli, 2000; Camargo and Coeli, 2006). Reclink has been used in many epidemiological studies by academic institutions and the MoH which were mostly published in *Reports in Public Health*, 1 a scientific repository maintained by the Oswaldo Cruz Foundation. They have also employed record linkage on administrative databases to study mortality rate on patients submitted to high complex cardiology procedures (Migowski et al., 2011) and to assess the under notification of tuberculosis cases in Brazil (Oliveira et al., 2012b).

Main contributions of this group, beyond the reported linked databases, comprise the use of the Expectation Maximization (EM) algorithm to predict the best settings for model tuning (Junger, 2006), the use of phonetic code in blocking step (Coeli

<sup>1</sup> https://scielosp.org/grid/csp/

Ali et al. Administrative Data Linkage in Brazil

and Camargo, 2002), and the rule-based matching. Recent work on the development of an open source version, the OpenRecLink (de Camargo and Coeli, 2015) has provided a multi-platform solution suitable to international users. Additional efforts have been made to define a cut-off point on probabilistic record linkage results (Verzinhasse Peres et al., 2014) and building a data warehouse for the integration of three Brazilian health information systems concerned with the production of ambulatory and hospital procedures for cancer care, and cancer mortality: SIH-SUS, APAC-ONCO, and SIM (Freire et al., 2015).

### RECORD LINKAGE TOOLS DEVELOPED AND/OR USED IN BRAZIL

In the past decade, the use of secondary data for research in Brazil has grown substantially. This is due to several factors: availability of large data sources, development of efficient linkage tools, legislation in Brazil favoring the used of secondary data for Brazil, the need to evaluate several public policies, and international collaborations, among others.

### RecLink

RecLink (RecLinkIII) is an open source C++ based probabilistic linkage algorithm specifically developed for the Portuguese language phonetics (Camargo and Coeli, 2000). Its flexible Graphical User Interface (GUI) allows the user to customize the tool and read different data sets easily. The interface is also used to define the behavior of the tool, such as the pre-processing steps, blocking and matching parameters (Camargo and Coeli, 2006). RecLink uses a custom format for input, which means the original data sets first have to be converted to standard formats to be linked. It has functions related to the standardization of common fields, including manipulation of names (case sensitive conversions, preposition and accent removal in names, and removal of commas and punctuation marks), standardization of date formats, and correct classification of missing values. Standardization of character attributes, such as date of birth and sex, is performed using the dBASE database manager, whereas a Soundex code developed for this purpose is used for strings, such as names and municipality (Camargo and Coeli, 2000; Camargo and Coeli, 2006).

RecLink performs record linkage in two steps: 1) blocking step, which separates the two data sets to be linked into smaller data sets according to the configuration provided by the user and 2) matching weight calculation and pairwise comparison of records that belong to the same block (Camargo and Coeli, 2000). The blocking stage can be conducted in one-step procedure using a single attribute, such as municipality(Paixao et al., 2018) or multi-step procedure using different combinations of attributes (Capuani et al., 2014). The Levenshtein string comparator is used to compare names; it is defined as the minimum number of insertions, deletions, or substitutions necessary to change one string into another (the values varying between 1, perfect similarity and 0, total disagreement). RecLink uses three different weight systems that can be selected by the user: 1) the pure and simple comparison, which only classifies records as matches if their attributes are strictly identical; 2) the character sequence comparison, which evaluates each pair of attributes from both records and returns how many different characters they have; and 3) the fuzzy comparison, which returns a normalized score consisting of the size of the longest common sequence of characters divided by the size of the longest attribute. The default m-probabilities and u-probabilities of 0.9 and 0.1, respectively, are often used (Oliveira et al., 2012a; Paixao et al., 2018).

RecLink is the most popular linkage tool in Brazil that has been used in several population linkage-based studies (Oliveira et al., 2012a; Capuani et al., 2014; Paixao et al., 2018). It also has several functions for other applications, such as de-duplication, standard query language (SQL) exporting, and frequency tables calculation. The software is available for free use and licensed under GPL (Camargo and Coeli, 2000; Camargo and Coeli, 2006).

### Python Linkage Algorithm

Python linkage algorithm (PLA) is a Python based deterministic algorithm developed for passive data collection with cohorts of HIV-infected patients at FIOCRUZ Rio de Janeiro. The tool aims to maximize accuracy and to minimize the need for clerical (manual) review in data linkage (Pacheco et al., 2008b). It was primarily implemented to assist in retrieval of information on the vital status of people living with HIV/AIDS (PLWHA) who are lost to follow-up in two large urban HIV/AIDS cohorts: Rio de Janeiro cohort database (Schechter et al., 1994) and TB-HIV in Rio (THRio) (Pacheco et al., 2011). The Rio de Janeiro cohort database was originally designed to validate the WHO-HIV staging system in a developing country, whereas the THRio cohort was designed to assess the impact of implementing isoniazid prophylactic therapy among HIV-positive patients with indications for prophylaxis in Rio de Janeiro (Pacheco et al., 2011; Saraceni et al., 2014). PLA has also been adapted to crossreference PLWHA public databases to both tuberculosis and AIDS cohort databases (Pacheco et al., 2008a; Grinsztejn et al., 2013).

The algorithm has a hierarchical structure and correlates records using exact comparisons. It allows for specific errors in names and dates, measured by means of phonetic codes and a string similarity score based on a recursive longer common substring algorithm, implemented in the "difflib" library from Python, which helps dealing with specific differences between sequences and dates. PLA runs both in a fully automated procedure (PLA-FAP) and in association with clerical/manual review of records that are not classified as true matches or non-matches (PLA-MR). Patient name, mother name, and date of birth are used as matching fields with parameter estimates obtained with the Expectation– Maximization algorithm. Date of birth is allowed to have only one digit mistake in any position or the common swap between day and month (only if they were exactly the same but swapped) (Pacheco et al., 2008b). The Levenshtein distance string comparator measure is used to compare the field name and mother name (Fonseca et al., 2010). The algorithm uses score values chosen empirically during its development using different data sources (Pacheco et al., 2008b).

The combination of these measurements and score values determine several levels of inclusion and exclusion, called automatic codes, which depends on how much information is available or missing. Records with complete information are treated independently from records with missing information. Whenever a pair of records is neither automatically included nor automatically excluded by the criteria, this pair is kept in the final merged database, marked as an unresolved pair for possible further manual review. The algorithm is hierarchical in the sense that lower codes mean more similar records hence perfect matches, but codes used for records with full information, even if higher, are more robust than codes for missing records. The algorithm is not "greedy": the same record in the test database linked with a lower code (exact match) to one record could also be linked to another one with a higher code (poor match). This feature is useful in dealing with databases with one-to-many relations, for example in the case of tuberculosis surveillance databases (Pacheco et al., 2008b; Fonseca et al., 2010).

PLA has been validated using several cohorts Pacheco et al. (2008b); Pacheco et al. (2011); Saraceni et al. (2014); de Paula et al. (2018) and has comparable accuracy to RecLink, which intrinsically require manual review, and outperformed RecLink significantly in the presence of incomplete data without manual inspection (sensitivity: 98.4% for PLA versus 94.6% RecLink, p *<* 0.05) Pacheco et al. (2008a, 2008b, 2009, 2011); Grinsztejn et al. (2013).

### AtyImo

AtyImo was developed by UFBA and CIDACS between 2013 and 2016 to support a joint Brazil–UK project aimed at developing large population-based cohorts. It was written in Python, freely available on Github, and runs distributed over Spark (Pinto et al., 2017; Pita et al., 2018) or in parallel over CUDA over hybrid (multicore+multi-GPU) architectures. It implements a pipeline comprising data pre-processing (cleansing, standardization, blocking, and anonymization), pairwise comparison and matching decision, and accuracy assessment (Pita et al., 2018).

Data pre-processing in AtyImo is responsible to clean and standardize names, filling null/missing fields with default values, and remove duplicate records. Blocking is based on different predicates built with five linking attributes (name, mother name, date of birth, sex, and municipality). In an effort to reduce errors due to typos or missing values that could lead records being inserted in wrong blocks, AtyImo uses a predicate of attributes in its blocking stage. Anonymization is based on Bloom filters, which guarantee privacypreserving requisites related to sensitive (identifiable) data, allowing AtyImo to run within less protected environments, if needed. A Bloom filter is a 128-bit vector in which bigrams (pair of characters) are represented as 0 or 1 depending on some hash functions. It is useful to reduce the effort during pairwise comparison; instead of comparing strings directly, one can compare binary vectors using the Sørensen's Dice similarity function (Pita et al., 2018). Dice is defined as: Dice = (2 h)/(a + b), where h is the total of 1's at the same positions in both filters, and a and b are the total of 1's in the first and second filters, respectively. A Dice = 1 means filters are completely equal, decreasing to 0 (zero) depending on existing differences. The current implementation normalizes Dice indices between 0 and 10.000. Dice costs less to compute than other editing distance functions, improving AtyImo's speed (Pita et al., 2018).

AtyImo implements a two-round linkage step in which a mixture of deterministic and probabilistic methods can be

used together to generate high accurate data marts (domain specific data). The weights of each attribute are determined by the amount of bits they occupy in the Bloom, meaning that important attributes, such as name will have a larger bloom length over less important attributes. In the hybrid approach, categorical attributes are matched exactly whereas names and dates (both more prone to errors) are probabilistically classified as: exact (Dice = 10,000), strong (10,000 ≥ Dice ≥ 9,000), weak (9,000 *>* Dice ≥ 8,000), and unpaired (8,000 ≤ Dice). This approach results in some flexibility in the combinations of exact and approximate comparisons. As a result, three output data sets are produced: true positive (TP) pairs, true negative (TN) pairs, and "dubious records" [false positive (FP) and false negative (FN) matches]. This classification is based on upper and lower cutoff points representing boundaries for TP and TN matches, respectively. Further analysis of the cutoff points is performed to retrieve more true (positive and negative) pairs and an iterative second round analysis of dubious pairs, shifting these points in each iteration, is conducted to retrieve additional records into these two groups (Pinto et al., 2017; Pita et al., 2018).

Finally, accuracy assessment can be performed manually based on gold standards (when existent) to certify small data marts or automatically based on supervised machine learning methods (Pita et al., 2017) in big data marts. Supervised methods use the same data produced during the accuracy assessment of previous linkage to fit a model that can be later used to classify new records. AtyImo, in comparison to previous linkage tools freely available, has reasonably better accuracy and shorter execution time with a major advantage to scale upward to huge databases (Pita et al., 2018). The current version of AtyImo based on the NVIDIA's CUDA library is able to probabilistically link databases of up 80 million records in around 60 s over multiple GPU architectures (Boratto et al., 2018).

### CIDACS-RL

CIDACS-RL, created at CIDACS in Salvador, is a Java-based search engine indexing linkage tool. It was developed to mitigate accuracy and scalability challenges in linking huge administrative electronic health and socioeconomic data sets, in the order of millions of records, stored within the center. To achieve this, instead of using usual blocking strategies, CIDACS-RL uses indexing, query and scoring modules provided by Apache Lucene (Białecki et al., 2012) and inverted index and term frequency-inverse document frequency (TF-IDF) to reduce the number of comparisons. The TF-IDF weight is composed by the normalized TF (the number of times a word appears in a document divided by the total number of words in that document) and the IDF (computed as the logarithm of the total number of documents divided by the number of documents where the specific term appears).

Within the CIDACS environment, all data sets are submitted to data cleansing and quality assurance processes after entering data linkage step. Those processes guarantee that linkage attributes are standardized and cleansed. Similarly to other methods, CIDACS-RL performs record linkage in two steps. If two databases A and B were to be linked and | · | denotes the number of records in a given database, assuming |*A*| *>* |*B*| (i.e., A is the largest database: the indexing module take as input the linkage attributes from data set A (larger data set) and builds an index Ai (Białecki et al., 2012). A challenging issue in linking huge data sets is to reduce the number of pairwise comparisons, therefore, CIDACS-RL uses the query module as a blocking stage. Hence, instead of comparing each record of data set B with every record of data set A, CIDACS-RL query a small subset of similar records from Ai and apply comparison functions on them.

As Apache Lucene provides different query types, CIDACS-RL uses a mixture of queries functions (exact, semi-exact or fuzzy) to overcome different errors expected to exist in data linkage attributes (Białecki et al., 2012). Exact query takes each linkage attribute as a parameter and returns only records in which every attribute is equal to those used for querying. Semi-exact query is a modification of exact query composed of an arrangement of n1 linkage attributes, hence, enabling retrieval of candidate pairs where only one attribute is different between the query record and result pairs. Unlike exact and semi-exact queries, fuzzy query allows differences on any number of attributes. Each query function (exact, semi-exact or fuzzy) takes each record in data set B to query in Ai and returns a set of similar records based on TF-IDF.

Since some attributes may have semantic meaning which the TF-IDF does not account for, CIDACS-RL relies on a custom scoring function tailored for Brazilian data sources to compare record pairs. This function is based on different metrics and approaches, depending on the type of attribute. CIDACS-RL supports four kinds of attributes: string, categorical, date, and IBGE municipality code. The IBGE code is a seven-digit numeric code where the first two digits represent one of Brazil's 27 states, the following four digits represent one of 5,570 municipalities and the last digit is used for verification purposes. Next, each record in data set B which was used as source for the query is compared with all records retrieved from Ai and returns the most similar record based on the custom scoring function. The function returns all pairs matched along with the score obtained; if any record with a score greater than the threshold is found on exact or semi-exact queries, the pair is added to the resulting set and fuzzy query is not executed.

### FRIL

Fine-grained Records Integration and linkage Tool (FRIL) is a Java based tool providing a set of highly customizable functions. Data integration (or reconciliation) is supported through different merging and splitting functions. It uses searching methods to determine which pairs of records will be compared from both data sets to be linked. FRIL has two different search methods: nested loop join and sorted neighborhood. Nested loop method performs "all to all" comparison, which is the same as no blocking. Sorted neighborhood defines a window limit in a way that records outside the window are not compared, reducing the number of comparisons (Jurczyk et al., 2008a; Jurczyk et al., 2008b).

To compare record pairs, FRIL implements four types of distance functions: edit distance, soundex, Q-gram and equality. All distances are normalized between 0 (total disagreement) and 1 (total agreement). Edit distance consists of the number of changes needed to make both attributes equal. Soundex transforms the attributes to a new form that takes the word sound into account, which can be useful when dealing with attributes, such as names and addresses, which are informed using speech. Equality just assigns 1 if the attribute is equal and 0 otherwise (Jurczyk et al., 2008a; Jurczyk et al., 2008b). Regarding the decision process, FRIL allows for matching weights to be assigned to each attribute by the user. The matching weights are then used to compute a normalized weighted average that is the similarity score for the pair. Similarity score is used to make a decision classifying the pair as match, non-match, or uncertain, according to thresholds defined by the user (Jurczyk et al., 2008a). Uncertain pairs can be later manually labeled as matches or non-matches, if needed. FRIL has been used in Brazilian databases and comparative studies of linkage algorithms, however, the performance in huge data sets was suboptimal compared others, such as AtyImo (Pita et al., 2018).

### Febrl

Freely extensible biomedical record linkage (Febrl) is a Python-based data linkage pipeline, implementing data cleansing, de-duplication, and pairwise comparison. Its modular architecture contains a handful of functions that can be used in the linkage process. The tool has a graphical interface (GUI) that can be used to customize settings according to the data sets and desired results. Febrl is organized in two main parts: pre-processing and the linkage itself. There is also a data exploration functionality that can be used to visualize the data and make sure it was loaded correctly (Christen et al., 2004; Christen, 2008).

Febrl supports several text-based data set formats: columnar text files, CSV, and SQL databases. Pre-processing comprises data cleaning and standardization through different standard functions for names, dates, and addresses. There is also an implementation of Hidden Markov Models (HMM) for name and address segmentation. For example, a single attribute name can be split into first name and second name. The HMMs can also be applied to addresses to extract multiple attributes (such as ZIP code, house number, city name, and state) from a single one. Febrl also has functions for date standardization, which are useful when two data sets have different date formats (Christen et al., 2004; Christen, 2008).

Pairwise comparison includes blocking methods and multiple functions to compare attributes. Febrl contains seven blocking implementations. They aim to reduce the amount of comparisons, and thus reducing the computational cost of the linkage. Febrl allows for each attribute to be compared using a different comparison function. Hence, it is possible to explore different strategies to find the one that better fits the set of attributes in the given data set; the tool has more than 20 different comparison functions. When comparing two records, each comparison function returns a score for attribute pair. On top of the score produced by each comparison function, it is necessary to use a definitive classifier, for weight vector classification, that generates a single score for the pair. The classification process can be supervised or unsupervised. After the classification step, it is also possible to customize the output of the system. This option allows the user to specify if the output will be one-to-one or one-to-many comparison (Christen, 2008). Similarly, comparative studies on data linkage tools in huge Brazilian databases showed slightly sub-optimal performance compared to RecLink and AtyImo (Pita et al., 2018). **Table 3** compares the five recording linkage tools using different attributes.


TABLE 3 | Comparative analysis of existing linkage tools.

### CHALLENGES AND OPPORTUNITIES

Linked administrative data sets hold the potential to change the research landscape in the HTA arena, since linkage constitutes a valuable tool for combining individual-level information (biological, behavioral, socio-economical, clinical, and environmental) from different sources. This combined information can be used for population-based research applications with implications for public health, as well supporting public policy decision making.

Administrative data differ substantially from data generated by the academic community. Government data is collected in a logical format over time and refers to the totality of populations or specific groups; academic data is limited in scope, generally collected over a defined period for specific purposes. The use of linked administrative data for research, compared to those of primary data, often has several challenges and limitations. However, it can have several advantages: 1) large sample size, enabling statistical power for stratified analysis, making possible to explore epidemiological questions in different sub-populations; 2) it can help rebuild the prospective characteristic of the data, allowing for longitudinal studies at relatively lower cost with retrospective data; 3) it can also help answer questions that require detailed data on hard-to-reach populations, such as children; and 4) it can also help generate evidence with a high level of external validity and applicability for policy making as it captures the real-world setting. In the area of HTA, administrative data holds the potential to contribute to the development of high-quality and powerful research that furnishes scientific evidence for use by policy makers.

Brazil has made substantial effort to improve the quality of the data collected as part of service delivery. Although huge variation exists between regions and states, most data sets contain incomplete, inconsistent, inaccurate data that vary in content, format, and structure. This has substantial impact on data pre-processing requirements, quality of linked data, and internal validity of research findings from the data (Harron et al., 2017). Although considerable proportion of missing values in non-mandatory variables in a specific data set can be expected, for example education and occupation, key variables are more likely to be complete. In addition, similar variables are recorded in different data sets: a missing variable in one data set can be recorded in another data set; hence, in some circumstances, the linkage process can recover this missing variable.

Importantly, due to lack of unique key identifiers, all linkage tools used in Brazilian data including RecLink, AtyImo, and CIDACS-RL, rely on names, sex, date of birth, and municipality as linkage attributes (Camargo and Coeli, 2000; Pita et al., 2018). Brazilian names are often recorded in different ways, for example, an individual with five names might have only the first and last name recorded in one data set but all the five names recorded in the other data set (Harron et al., 2017; Paixao et al., 2018). In addition, there are misspellings, abbreviations, and punctuation marks with names and municipalities, and different date formats with date of birth which require a time-consuming data cleansing and sometimes sophisticated techniques for standardization. The level of data cleansing performed should take into account preserving the discriminative power of individual identifiers and the ability to distinguish one record from another (Harron et al., 2017). To minimize errors and improve comparison of record pairs, RecLink (Camargo and Coeli, 2000; Camargo and Coeli, 2006) and CIDAS-RL use string comparators and phonetic coding adapted to Brazilian names, whereas AtyImo uses a predicate of attributes in its blocking stage and Bloom filters (Pita et al., 2018).

The availability of huge volumes of data also provides opportunities to explore the effect of interventions or policies on frequent as well as rare health outcomes and in sub-populations including vulnerable groups (e.g., children, woman, ethnic minorities). At the same time, storage, processing, and analysis of such huge data has proved challenging for research institutions despite recent technological advances. The 100 Million Brazilian Cohort, an electronic database comprising individual records of approximately 114 million people (57% of the Brazilian population) is one example. The extensive coverage in the social registry: CadUnico, the availability of individually identified data, and the possibility to link to other health care data sets made it possible to design individual level longitudinal studies to evaluate the impact of BFP and other social protection programs on health and healthrelated outcomes.

CIDACS, the data center located in Salvador, needed to utilize sophisticated infrastructures for storage and processing of data, yet it took several days to link some data sets. Again, the lack of unique key identifiers in the databases to be linked required the use of techniques (for example, the use of predicates and Bloom filters in AtyImo or search engine indexing and scoring in CIDACS-RL) to reduce computation time at both blocking and comparison steps of the linkage process. Freely available linkage software, such as FRIL, Febril, or RecLink, crashed when attempting to run the linkage to build the 100 Million Brazilian Cohort, even on high-speed computers, prompting the need for developing efficient linkage tools: AtyImo and CIDACS-RL. Furthermore, the large size of the linked data may pose analytic challenges for standard statistical packages, such as SPSS and R, to run advanced statistical methods, for example propensity score matching.

Providing access to and usage of administrative data sets containing personal identifiable information for linkage purposes also presents a range of privacy challenges, mainly with regard to ethical and legal issues in the effort to protect personal data. Processing, linkage, and analysis of data should be conducted in accordance with the principles and criteria designed to ensure individuals' privacy, data security, and the ethical use of data containing personal information. This could vary from country to country or from center to center. For example, CIDACS's big data platform utilizes: 1) a combination of physical and virtual environments including the separation of data curation, linkage and analysis, 2) a hierarchical data access policy ensuring that only a specified number of individuals possess the highest level of access to all data elements for treatment purposes, and 3) privacy-preserving linkage tool, for example, AtyImo uses hash functions to anonymize relevant fields before the record linkage stage (Pinto et al., 2017; Pita et al., 2018). The hash functions, in addition to anonymization, help to speed up the linkage process (Pita et al., 2018) despite complicating the process of assessing the similarity between identifiers on different records (Harron et al., 2017). On the other hand, CIDACS-RL uses safe heaven to protect privacy while processing and linking data sets.

In the center's access and analysis environment, researchers have exclusive permission to access anonymized linked data sets *via* the coded data variables relevant to their field of study, after obtaining approval from the institutional review board. Access to data sets can be in person at the data center or through a Virtual Private Network (VPN), in accordance with sound information security practices. These includes: 1) submission of detailed research project accompanied by favorable ethical opinion and filled forms for the data plan provided by CIDACS to support the linkage and extraction of variables contained in the available databases, which should be restricted to those necessary to answer the questions in the proposed study. Detailed descriptions of the processes to be applied and the analyses to be conducted on the data are also desirable to avoid methodological biases. 2) Signature of the "Terms of Responsibility" related to the access and use of data.

Administrative data are generally collected by government departments or agencies for specific purposes, which contain personal information that may be confidential and/or sensitive, such as data collected for the execution of social programs or health service delivery. The use of these data for secondary use in restricted research requires a case-by-case analysis taking into account the balance between risks and benefits to individuals and the predominant public interest. Linking of records between different data sources, administrative or non-governmental, requires individualized data for the application of record linkage techniques, in the absence of unique identifier in the different data sources. As a result, access, processing, and analysis of data containing personal information for the purpose of research and the generation of evidence for decision making in public health policy require legal basis, physical and virtual security arrangements, exclusive use for a purpose previously specified, appropriate credentials for access, and favorable ethical opinion of the proposed study (Harron et al., 2017).

In Brazil, the Law on Access to Information (LAI) provides guidelines for the organs and entities of the federal public administration to adjust their information management policies by promoting the necessary adjustments to the registration, processing and archiving of documents and information. However, LAI does not address the use of information collected or stored by government for use in research because it is more focused on public transparency. A general law for the protection and processing of personal data including data for research purposes, the Personal Data Protection Bill, processed by the Chamber of Deputies since 2012 was recently sanctioned presidential. The General law on the Protection of Personal Data (Law 13709/2018) determines the rights of citizens to their personal data and the criteria that public and private agents will have to obey in dealing with them. It requires the regulator to request privacy risk reports to make sure that personal data are being safely processed, stored, and accessed. Hence, the law might present challenges for governance and management of the entire life cycle of the data requiring investments in computer infrastructure and specialized personnel, and adherence to good information security practices, to maintain the privacy and confidentiality of personal data. The law, which will come into force in February 2020, is the first Brazilian law on the subject and will establish specific norms for the treatment of personal information for public health research.

### CONCLUSION

Brazil has high quality of health care records, growing number of linkage centers, and an open, modern attitude toward use of data for research and policy including HTA, supported by a rigorous but enabling legal framework. Despite the technical, infrastructural, and legal challenges with the use of huge secondary data for research, data linkage creates a unique opportunity to conduct large-scale observational studies to generate evidence on the impact of health technologies and health/social policies.

### AUTHOR CONTRIBUTIONS

LS, MSA, and MLB contributed to the conception and design of the study. MSA wrote the first draft of the manuscript with substantial contributions by other authors on the different

### REFERENCES


sections of the manuscript. All authors have read and revised the manuscript critically for important intellectual content, and approved the final version of the manuscript.

### FUNDING

This work is part of the The 100 Million Brazilian Cohort project funded by the Wellcome Trust, Grant code: 202912/B/16/Z.

used to track blood donors in the mortality information system database. *Cad. Saude Publica* 30, 1623–1632. doi: 10.1590/0102-311X00024914


**Conflict of Interest Statement:** LS holds research grants from GSK, Wellcome, MRC, NIHR, BHF, and Diabetes UK and is a Trustee of the British Heart Foundation.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor is currently co-organizing a Research Topic with one of the authors LL, and confirms the absence of any other collaboration.

*Copyright © 2019 Ali, Ichihara, Lopes, Barbosa, Pita, Carreiro, dos Santos, Ramos, Bispo, Raynal, Canuto, de Araujo Almeida, Fiaccone, Barreto, Smeeth and Barreto. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

### ABBREVIATIONS



# Kawasaki Disease and the Use of the Rotavirus Vaccine in Children: A Systematic Review and Meta-Analysis

*Natália Gibim Mellone1, Marcus Tolentino Silva1, Mariana Del Grossi Paglia1, Luciane Cruz Lopes1, Sílvio Barberato-Filho1, Fernando de Sá Del Fiol1 and Cristiane de Cássia Bergamaschi1\**

*1 Pharmaceutical Science Graduate Course, University of Sorocaba, Sorocaba, Brazil, 2 Federal University of Amazonas, Faculty of Medicine, Manaus, Brazil*

Background: The vaccine against the rotavirus is an effective measure in reducing hospitalizations and mortality caused by the virus. However, its use can result in serious adverse effects. The available evidence on Kawasaki disease has not yet been reported in the literature. This study investigated the risk of developing Kawasaki disease with the

#### *Edited by:*

*Brian Godman, Karolinska Institute (KI), Sweden*

#### *Reviewed by:*

*Isabella Piassi Godói, Federal University of South and Southeast of Pará, Brazil William Gustavo Lima, Federal University of Minas Gerais, Brazil*

#### *\*Correspondence:*

*Cristiane de Cássia Bergamaschi cristiane.motta@prof.uniso.br*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 11 January 2019 Accepted: 23 August 2019 Published: 24 September 2019*

#### *Citation:*

*Mellone NG, Silva MT, Paglia MDG, Lopes LC, Barberato-Filho S, Del Fiol FdS and Bergamaschi CdC (2019) Kawasaki Disease and the Use of the Rotavirus Vaccine in Children: A Systematic Review and Meta-Analysis. Front. Pharmacol. 10:1075. doi: 10.3389/fphar.2019.01075*

use of rotavirus vaccines in children. Methods: This is a systematic review of data collected from studies retrieved on the following databases: Cochrane, MEDLINE, Embase, CINAHL, Scopus, Web of Science, HealthSTAR, Lilacs, Clinical trial.gov, and International Clinical Trials Registry Platform, up

to the 15th of August 2018, with no restrictions on language or date of publication. The outcomes measured were incidence of Kawasaki disease, risk of developing the disease, and rate of discontinuation of the vaccination schedule. Four reviewers independently selected the studies, performed data extraction, and assessed the quality of evidence. A meta-analysis of random effects was performed.

Results: A total of 13 publications were included, with a population of 164,434 children included in the meta-analysis. The incidence of Kawasaki disease (24 cases per 100,000, 95% CI = 11.98–48.26) in the vaccinated children was low. No difference between the vaccines was found in the prevalence rate of adverse effects (RR = 1.55, 95% CI = 0.41–5.93). Use of the vaccines was not associated with risk of developing Kawasaki disease (low-quality evidence). None of the studies reported the rate of discontinuation of the vaccination schedule.

Conclusions: The vaccines were associated with a low incidence of developing Kawasaki disease, showing no association with this serious adverse effect.

Keywords: Kawasaki disease, rotavirus vaccine, safety, systematic review, adverse effect

## INTRODUCTION

The rotavirus is the leading cause of severe diarrhea in infants and children worldwide, particularly in developing countries representing one of the main causes of morbidity in young children globally (Uhlig et al., 2014; Yin et al., 2015). The deaths caused by gastroenteritis due to infection by the rotavirus have been high in low-to-medium income Latin American countries (Bryce et al., 2005) and the Caribbean (Linhares et al., 2011).

Two vaccines are recommended by the World Health Organization (WHO) for use against the rotavirus, referred to as the pentavalent vaccine and the monovalent vaccine, introduced into immunization programs of some countries from 2006 onwards (Uhlig et al., 2014; World Health Organization, 2014). Other vaccines are commercially available, such as the oral monovalent Lanzhou lamb rotavirus in China (Yin et al., 2015), the monovalent Rotavin-M vaccine in Vietnam, and the monovalent Rotavac vaccine in India (Kollaritsch et al., 2015).

In 2009, and again in 2013, the WHO recommended the introduction of one of these vaccines into all national immunization programs (Soares-Weiser et al., 2012). The vaccines are administered *via* the oral route in babies, in two doses for the monovalent vaccine and three for the pentavalent vaccine. The monovalent vaccine is administered at between the 6th and 15th week of life while the pentavalent vaccine is given as a three-dose series between the 6th and 32nd week (Shatsky and Vaccine, 2006).

The most common adverse reactions associated with the use of the vaccine are cough, nasal discharge, diarrhea, irritability, loss of appetite, fever, and vomiting (Bravo et al., 2014). However, its use can also cause serious adverse effects, such as intussusception (Maglione et al., 2014) and Kawasaki disease (Soares-Weiser et al., 2012).

Kawasaki disease was included as a serious adverse effect in the package insert of the pentavalent vaccine after being reviewed by the manufacturer and approved by the Food and Drug Administration (FDA) in 2007, when a pre-licensure clinical trial revealed the presence of this effect in children after use of the vaccine (Hua et al., 2009).

Kawasaki disease has features compatible with common viral infections and mainly affects children, where almost 100% of cases occur in children younger than 5 years and genetically predisposed individuals (Makino et al., 2018). Children of Japanese and Asian-Pacific Island descent have the highest rates, and males have higher rates than females (Fuller, 2019).

In Asian countries, the incidence rates of Kawasaki disease are high (Singh et al., 2015). Recent Japanese data revealed the highest global annual incidence of disease in children under 5 years of age (Makino et al., 2018), with second and third highest rates reported for South Korea (Kim et al., 2014) and Taiwan (Lin et al., 2015), respectively. However, other Asian countries have a steadily increasing incidence of this disease (Saundankar et al., 2014; Wu et al., 2017). In Canada (Lin et al., 2010), the United States (Uehara and Belay, 2012), and Europe (Salo et al., 2012; Jakob et al., 2016), Kawasaki disease rates are significantly lower. The marked differences in incidence rates among different ethnicities strongly support the idea of a strong genetic basis of susceptibility (Holman et al., 2010).

The disease presents a variety of signs and symptoms, such as persistent fever for longer than 5 days, non-exudative bilateral conjunctivitis, erythema of the lips and oral mucosa, swelling of the extremities, cutaneous eruption, gastrointestinal symptoms, and lymphadenopathy. Aneurysm of the coronary artery or ectasia can develop in 20–25% of cases, along with other complications in untreated children, where the condition may evolve to myocardial infarction, ischemic heart disease, and death (Abrams et al., 2015).

Although most epidemiologic and immunologic evidence suggests that an infectious agent causes Kawasaki disease (Geier et al., 2008), this is not conclusive. Besides, the infectious agent and the genetic characteristics of susceptible children have yet to be elucidated (Principi et al., 2013); it is possible that vaccination play some role in the pathogenesis of Kawasaki disease (Burgner and Harnden, 2005; Rowley and Shulman, 2007). The distinctive immune system characteristics of children with Kawasaki disease could suggest that they respond to all antigenic stimulations, including those due to vaccines (Esposito et al., 2016) in a way that differs from that observed in healthy children. However, the motive, based on biochemical and immunological mechanisms, by which the rotavirus vaccines leads to Kawasaki disease, were not found in the literature.

The available evidence on risk of Kawasaki disease with the use of rotavirus vaccines has not yet been reported in the literature. This knowledge can help guide health professionals in clinical decision-making. This systematic review sought to answer the following PICO question: "what is the risk of Kawasaki disease in children who made use of rotavirus vaccines compared to those who did not?" Therefore, the objective of this study was to investigate the risk of developing Kawasaki disease with the use of rotavirus vaccines in children.

### METHODS

### Protocol and Registration

The systematic review was performed according to the recommendations specified in the Cochrane Manual of Interventionist Reviews and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist (Liberati et al., 2009; Moher et al., 2010; Higgins JPT, 2011)

This protocol was registered on the International Prospective Register of Systematic Reviews (PROSPERO: CRD4201604633, https://www.crd.york.ac.uk/prospero/display\_record. php?RecordID=46334).

### Eligibility Criteria Inclusion Criteria

This study included randomized clinical trials (RCT) and quasi-randomized and observational studies (case report, ecological, case series, adverse event report, cross-sectional, case-control, and cohort studies) involving children up to 32 weeks of age in use of vaccines (monovalent or pentavalent) against rotavirus.

**Abbreviations:** CINAHL, Cumulative Index to Nursing and Allied Health Literature; GRADE, Grading of Recommendations Assessment Development, and Evaluation; FDA, Food and Drug Administration; WHO, World Health Organization; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomized clinical trials; CENTRAL, Cochrane Central Register of Controlled Trials; PROSPERO, International Prospective Register of Systematic Reviews; VHL, Virtual Health Library; ICTRP, International Clinical Trials Registry Platform.

### Exclusion Criteria

Abstracts published in congresses not providing data on the incidence of the adverse effect were excluded.

### Search for Primary Studies

#### Electronic Searches

The following electronic databases were searched: Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (*via* Ovid), Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science, HealthSTAR (*via* Ovid), Scopus, LILACS, Clinical trial.gov, and International Clinical Trials Registry Platform, up to the 15th of August 2018, with no restrictions on language or date of publication.

#### Other Search Resources

The references from the eligible studies, the systematic reviews on the rotavirus vaccine, and FDA data were reviewed to identify other eligible studies. The National Health Surveillance Agency in Brazil and the National Brazilian Immunization Program were contacted by email *via* the individuals in charge to check for the existence of reports of Kawasaki disease associated with the vaccines. Information was requested for notifications of the disease as well as related signs and symptoms. Google Scholar (to find unindexed journals), ProQuest Dissertation and Theses Database, the Brazilian Digital Library of Thesis and Dissertations, and the Thesis and Dissertation Catalog of Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) were also consulted.

### Search Strategies

The following Mesh descriptors and their combinations (entry terms) were used: rotavirus vaccines (or vaccines, rotavirus) and mucocutaneous lymph node syndrome (or Kawasaki syndrome or lymph node syndrome or mucocutaneous, or Kawasaki disease) for the article search. Search strategies for each database used are available in **Supplementary Data Sheet 1**.

#### Outcomes Assessed

The primary outcome was the incidence of Kawasaki disease associated with the vaccines (number of cases of the disease/total number of children vaccinated for rotavirus). The secondary outcomes were risk of Kawasaki disease comparing each rotavirus vaccine with a control group, and the rate of discontinuation of the vaccination schedule. Given the lack of a standard case definition for the disease, the diagnostic criteria adopted by the studies was used.

The rate of adverse effects was expressed according to the following categories: very common ≥1/10 (≥10%), common ≥1/100 and <1/10 (≥1% and <10%), uncommon ≥1/1,000 and <1/100 (≥0.1% and <1%), rare ≥1/10,000 and <1/1,000 (≥0.01% and <0.1%), and very rare <1/10,000 (<0.01%) (Meyboom and Egberts, 1999).

### Study Selection

The reviewers (NM and CB), working independently, selected the potentially relevant studies and applied the eligibility criteria. The full texts of all the potentially eligible articles were obtained and then the reviewers (NGB and MP, FSD-F, and SB-F) assessed the eligibility of each full article. Disagreements were resolved by consensus and, when necessary, submitted to a third reviewer (LL). The Endnote X7 software package was employed for study selection.

#### Data Extraction

The data were extracted by all reviewers, working independently, using a data extraction form. In the case of articles published only as abstracts or those with key information missing, their respective authors were contacted to obtain the necessary information. Disagreements were again resolved by consensus and, when necessary, submitted to a third reviewer.

Information was collected on: i) the characteristics of the studies (objectives, study design, country where the study was conducted, type of vaccine, data collection period, and conclusions) and ii) the study population (age, gender, race, sample size [number of children vaccinated]), vaccination status (doses of rotavirus vaccine given to child prior to onset of Kawasaki disease), diagnostic method for Kawasaki's disease, time to disease onset after vaccination (days of rotavirus vaccination until the cases developed the disease), and other concomitant vaccinations, when available.

Subgroup analyses were proposed for age, gender, race, and country, where applicable. The heterogeneity of the studies was determined using the χ2 test and I2 statistic. The following heterogeneity was considered: 0–25% (low heterogeneity), 50% (moderate heterogeneity), and 75% (high heterogeneity) (Higgins et al., 2003).

#### Assessment of Risk of Bias

The quality of the observational studies was determined using the tool described by Munn et al. (2014). This step was performed by all reviewers, working in pairs and independently. This tool includes 10 items for critical assessment of the methodological quality of prevalence studies. For each criterion, the study was attributed "yes" or "no" or "not applicable." The total number of "yes" answers per study was tallied. A higher number of "yes" answers indicates a lower risk of bias of the study. The risk of bias of clinical trials was that reported by the systematic review (Soares-Weiser et al., 2012), which used the Cochrane risk of bias tool to assess the following criteria: sequence; allocation concealment; blinding of the patient, healthcare professionals, outcome assessors, data collectors, and data analysts; incomplete outcome data; selective outcome reporting; and major baseline imbalance.

### Data Synthesis

The random-effect meta-analysis was performed using the STATA software package (version 14.2) (Montori et al., 2008). Given that this was a systematic review of adverse effect, RCTs and cohort studies were included in the meta-analysis, where both study designs allow information on adverse effects to be collected. Data were summarized according to incidence of the disease per 100,000 vaccinated children and relative risk (RR) with a 95% confidence interval (95% CI). When the metaanalysis was not suitable, a narrative summary of the studies was provided.

### Quality of Evidence

The quality of evidence of the studies was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (Guyatt et al., 2011). In this approach, RCTs start with high-quality evidence but can be assessed by one or more of the five categories of limitation of the studies: risk of bias, inconsistency, indirect measurement, imprecision, and publication bias. Observational studies start with low-quality evidence, which can increase according to the assessment of the categories.

### RESULTS

### Literature Search Results

A total of 1,051 publications were revised (after duplicates removed) and 52 potential eligible publications selected. Of these articles, 13 publications (total 14 studies) were included.

One of the publications, a package inserts by the FDA, contained information on two studies: a phase III trial and a phase IV study (**Figure 1**). There were no Brazilian studies or notifications reported in Brazil on Kawasaki disease due to use of the rotavirus vaccines. A search for articles in the references of the systematic review (Soares-Weiser et al., 2012) led to the identification of a further three clinical trials (Phua et al., 2005; Phua et al., 2009; Salinas et al., 2005), which were subsequently included in the study.

### Description of Studies Included in Narrative Syntheses

The characteristics and outcomes of the studies included are given in **Table 1** (descending order of year of publication). This review found information in: two case reports (Uhlig et al., 2014; Chang and Islam, 2018), four cohort studies (Belongia et al., 2010; Loughlin et al., 2012; Layton et al., 2018; RotaTeq, 2017),

TABLE 1 | Characteristics of the studies and outcomes found in children vaccinated against the rotavirus.


*USA, United States of America; AHA, American Heart Association; KD, Kawasaki disease; AE, adverse effect; OR, odds ratio; RR, relative risk; 95% CI, 95% confidence interval; NR, not reported; RV1, monovalent vaccine; RV5, pentavalent vaccine; ICD, International Classification of Disease; LLR, Lanzhou lamb rotavirus; RCT, randomized clinical trial.*

one cross-sectional study (Oberle et al., 2010), three adverse event reports (Geier et al., 2008; Hua et al., 2009; Paulke-Korinek et al., 2013), and four RCTs (Phua et al., 2005; Salinas et al., 2005; Phua et al., 2009; RotaTeq, 2017).

One case report described a 4-month-old Caucasian child, presenting classic Kawasaki disease shortly after receiving vaccines (pneumococcal 13, diphtheria–tetanus–pertussis, polio, Hepatitis B, and rotavirus vaccine). The case supports that vaccines may be associated with vasculitis and Kawasaki disease and that ongoing, systematic surveillance of such events is warranted (Chang and Islam, 2018).

Only one study reported the use of the Lanzhou lamb rotavirus vaccine. It consisted of a clinical case report of a 20-month-old Chinese child with Kawasaki disease after use of the rotavirus vaccine concomitantly with the hepatitis A vaccine. According to the authors, the rotavirus vaccine may have played a key role in the development of the disease, but no causal relationship between the effect and the vaccine could be established based on a single case (Yin et al., 2015).

A cohort study of commercial insurance data investigated the risk of adverse events associated with rotavirus vaccines. Most children received a concomitant diphtheria–tetanus–pertussis vaccine. Of a total of 2,468,002 vaccines, 23 cases of Kawasaki disease were found (Layton et al., 2018).

An adverse event report study determined the prevalence of adverse events associated with the monovalent and pentavalent vaccines using the sentinel surveillance system. A total of 833 adverse events were associated with the vaccines, but no cases of Kawasaki disease were reported (Paulke-Korinek et al., 2013).

The cohort study identified post-marketing adverse effects after routine use of the pentavalent vaccine from records held on an electronic database. Of the 85,150 children that received the rotavirus vaccine, there were 3 cases of Kawasaki disease in the intervention group (only 1 case was confirmed to have occurred within 30 days of vaccination) and 1 in the control group (recipients of diphtheria–tetanus–acellular pertussis vaccine). No increased risk of developing the disease due to the vaccine was observed (Loughlin et al., 2012).

Another cohort study assessed the risk of intussusception and other adverse events using the Vaccine Adverse Event Reporting System (VAERS) database among children aged 4–48 weeks who received the pentavalent vaccine (intervention group) compared to a non-exposed group. Similarly, no association of developing Kawasaki disease with use of the pentavalent vaccine was found (Geier et al., 2008).

A cross-sectional study used the database of the Paul Ehrlich Institute to assess the cases of Kawasaki disease associated with the use of monovalent and pentavalent vaccines (between 2001 and 2010). Four cases of the disease were reported from a total of 1,088 adverse events associated with the use of the pentavalent vaccine in children aged 2–5 months. The study reported the day on which the effect occurred (average of 6.3 days after vaccination). Three children were in use of another vaccine concomitantly (Oberle et al., 2010).

An adverse event report study assessed the data on adverse events associated with the pentavalent vaccine using the VAERS database. A total of 1,526 adverse events were associated with the pentavalent vaccine. Of 16 children found to have Kawasaki disease, 5 used this vaccine (Belongia et al., 2010).

Another study also searched the VAERS information to identify Kawasaki disease in children following use of vaccines with licensure in the United States. The authors identified a total of 239,535 events, including 107 cases of Kawasaki disease, 16 of which were following the use of the pentavalent vaccine. The study did not specify the number of pentavalent vaccines administered (Hua et al., 2009).

The package inserts by the FDA for the pentavalent vaccine described the results of two studies (a post-marketing study—phase IV and phase III clinical trials—the Rotavirus Efficacy Safety Trial/ REST). The cohort study compared the data on the disease in 17,433 children who received the pentavalent vaccine *versus* a control group of 12,339 that received a diphtheria, tetanus, and pertussis vaccine, revealing only one case of Kawasaki disease within 30 days of vaccination (RotaTeq, 2017).

The clinical trial found five cases of Kawasaki disease in 36,160 vaccinated children and one case in 35,536 children who received a placebo (42 days after the use of the pentavalent vaccine). The other three clinical trials also found no association of the adverse effect with the use of the monovalent vaccine (Phua et al., 2005; Phua et al., 2009; Salinas et al., 2005).

### Risk of Bias of the Studies

The cohort studies had risk of bias, having failed to account for possible confounding factors and/or to perform subgroup analyses. The cross-sectional and adverse event report studies had shortcomings for a larger number of assessed criteria that included the problems observed in the cohort studies plus those of data analysis with problems of underestimation of prevalence data for Kawasaki disease (**Table 2**).

It was not possible to assess risk of bias of the studies described in the package insert of the pentavalent vaccine because some information pertaining to these studies could not be accessed (RotaTeq, 2017). According to a systematic review (Soares-Weiser et al., 2012), one of the clinical studies (Phua et al., 2009) fulfilled all the assessment criteria for risk of bias and, therefore, had minimum bias risk. Another clinical trial (Salinas et al., 2005) had risk of bias for allocation and reporting of selective outcomes, whereas one study (Phua et al., 2005) had risk of bias for most of the criteria assessed.

### Results of Outcome Evaluated and Quality of Evidence

None of the studies reported the rate of discontinuation of the vaccination schedule. Some studies were not included in the meta-analysis because they failed to report data on the incidence of Kawasaki disease.

The meta-analysis revealed a rare incidence of cases of Kawasaki disease, with 24 cases per 100,000 vaccinated children for both vaccines (95% CI = 11.98–48.26). No differences between vaccines were found for incidence of the adverse effect (relative risk = 1.55 95% CI = 0.41–5.93) (**Figure 2**).


The risk of developing Kawasaki disease in the group of children receiving the vaccines did not differ to the comparator group, where no statistical difference was found between them. No heterogeneity was observed among the studies (**Figure 3**). However, the quality of the evidence according to the GRADE criteria for this outcome was considered low for both the vaccines, due to the high risk of bias, and imprecision in the results obtained.

Subgroup analyses were performed between studies conducted in the West (Salinas et al., 2005; Loughlin et al., 2012) and Asian-Pacific (Phua et al., 2005; Phua et al., 2009) countries. There was a higher incidence of disease in Asian-Pacific (57 per 100,000, I2 = 49.4%) compared to Western (23 per 100,000, I2 = 60.5%) countries. Subgroup analyses were not carried out for age, gender, and race due to the absence of information in the eligible studies.

### DISCUSSION

### Summary of Findings and Their Interpretation With the Available Literature

The findings of this review showed that the occurrence of Kawasaki disease is rare in children that received the monovalent or pentavalent vaccines. In addition, the risk of having the disease in the children receiving the vaccines against the rotavirus did not differ to comparator group.

A total of 13 publications reporting the frequency of the Kawasaki disease were included, the majority of which were conducted in the United States of America and Asian countries. The studies reported information on both the monovalent and pentavalent vaccines, where only one case report was found on the Lanzhou lamb rotavirus, marketed solely in China.

In general, the studies concluded that there was no increased risk or causal relationship of the adverse effect with use of the vaccines. However, some publications (Geier et al., 2008; Hua et al., 2009; Belongia et al., 2010) reported the need for monitoring the use of these vaccines in order to gather further information on their safety.

The rate of discontinuation of the vaccination schedule was not reported in included studies. This occurred either because the cross-sectional studies described the adverse effects reported in the databases or because the longitudinal studies did not bother to collect this outcome, important for endemic control programs.

The studies that involved the collection of information in databases were not included in the meta-analysis owing to the absence of data enabling the incidence of the adverse effect to be determined. The authors of these studies highlighted some shortcomings in the collection of the information in the databases used. The difficulty establishing a causal relationship between Kawasaki disease and the use of the vaccine stems, in part, from uncertainties regarding the information reported in the databases (Paulke-Korinek et al., 2013), and from the concomitant use of the rotavirus vaccine with vaccines against diphtheria, tetanus, and pertussis and/or pneumococcus, among others (Hua et al., 2009; Oberle et al., 2010; Layton et al., 2018), demonstrating that other vaccines could also cause the disease.

In the researched literature, we found studies that evaluated the use of vaccines in children, such as measles, mumps, rubella,

and varicella (MMRV) vaccine; measles, mumps, and rubella (MMR) vaccine (Holman et al., 2010); and pneumococcal vaccine (Park et al., 2011) and observed no association between vaccination and Kawasaki disease. We found no studies that evaluated the risk of Kawasaki disease due to combination of the rotavirus vaccine with any other vaccine (diphtheria, tetanus, and pertussis and/or pneumococcus).

In the present study, higher incidence of disease in Asian-Pacific countries compared to Western countries was observed. These results agree with the literature that have reported higher incidence of the disease in children of Asian and Japanese ethnicities (Holman et al., 2010; Park et al., 2011). However, the genetic characteristics of those children susceptible remain only partially elucidated (Principi et al., 2013). Subgroups analysis for age, gender, and race were not performed due to the absence of information in the eligible studies.

In addition, some studies failed to report how Kawasaki disease was diagnosed or reported, such as the studies based on data from the VAERS (Geier et al., 2008; Hua et al., 2009). The authors noted that the rates of notifications obtained by the system could not be interpreted as real, given possible underreporting of the adverse effects, which in turn may have been largely due to difficulties confirming the disease diagnosis and to the way this is recorded on the systems.

The revision of the package insert of pentavalent vaccine carried out in 2007, which included Kawasaki as a serious adverse effect, which led to an increase in the number of notifications of this disease (Hua et al., 2009; Hua et al., 2010; Oberle et al., 2010). The study of Hua et al. (2009) noted a rise in the number of annual cases from 0.65 to 2.78 per 100,000 children less than 5 years of age followed for up to 30 days post-vaccination, after amendment of the insert.

### Assessment of Study Validity: Limitations and Strengths

The difficulty in the description of the information registered on the systems/databases reported by the studies is a limitation of the present investigation. Measurement bias was also observed, owing to problems concerning the description of method of disease diagnosis (or absence of standard for diagnosis), which was not reported by some of the studies. This might be explained, in part, by difficulties studying Kawasaki disease, given the lack of a standard case definition coupled with insufficient knowledge of etiology.

The lack of a standard for diagnosing the disease can lead to the inherent underreporting of data on these databases. However, this occurred mainly in cross-sectional and adverse event report studies, which were not included in the meta-analysis.

In cases where the information on some RCTs proved inaccessible, the diagnostic criteria for the disease were described as "not reported." However, the design of this type of study ensures, with some confidence, that the adverse effect was measured rigorously. With regard to the observational studies, it is important to emphasize that the results found in the meta-analysis do not imply causation, since there is always the possibility of residual confounding in these studies.

Despite the broad search of scientific articles in several different databases and exhaustive attempts to obtain the missing information for some of the selected studies, few studies contained all the information required, as exemplified by some information from clinical trials provided by the systematic review and the absence of details in some studies described (such as number of doses of the rotavirus vaccine given prior to onset of Kawasaki disease, time to onset of Kawasaki disease after rotavirus vaccination, use of concomitant vaccinations, among others), hampering the analysis of bias risk. This study did not search specific databases for Asian and Japanese children, in whom the literature indicates a possible higher prevalence of the disease.

Notwithstanding, this systematic review and meta-analysis pooled the incidence of Kawasaki disease due to use of the rotavirus vaccine. The results exhibited no heterogeneity across studies, partly due to the high number of participants. Moreover, the method employed in this study was rigorous with explicit eligibility criteria and a broad search. The assessment of the quality of evidence was based on an independent assessment of bias risk, imprecision, consistency, indirect measures, and publication bias.

### Clinical Implications and Future Perspectives

The results of the present study suggest no association of Kawasaki disease with the use of monovalent and pentavalent vaccines. However, these results should be interpreted with caution due to low quality of the evidence from the studies included in the meta-analysis. The difficulty in conducting studies that did not associate rotavirus vaccine with other vaccines is due to the age of children who are receiving many of them. This difficulty finds robust epidemiological evidence to associate this adverse effect with the rotavirus vaccine. Then, future studies should be concerned with minimizing these biases.

The study also showed that observational studies assessing the incidence and causality of this adverse effect in the literature are scarce. This information underscores the importance of the use of vaccines, in view of the risks of contamination by the rotavirus in children under 5 years and the efficacy of these in preventing the infection (Tate and Parashar, 2014; Kollaritsch et al., 2015; Santos et al., 2016). However, the literature studied highlights the need for notification of adverse effects related to the vaccines in order to ensure continuous monitoring of these and other possible effects associated with the vaccines.

Although the results indicate low incidence of Kawasaki disease in children that used the rotavirus vaccines, it is important that health professionals and society at large report these and other adverse effects associated with the vaccines, rendering notification common practice, thereby contributing to the monitoring of safety data on the use of vaccines. This study reports the evidence on risk of Kawasaki disease with the use of rotavirus vaccines, and this finding can help guide health professionals in clinical decision-making.

### CONCLUSION

The results of the present study indicate that the monovalent and pentavalent vaccines were associated with a low incidence of developing Kawasaki disease, showing no association with this serious adverse effect. However, further studies involving larger samples are needed to confirm these findings.

### AUTHOR CONTRIBUTIONS

NM is the principal investigator, participated in all stages of the study, and oversaw the writing of the manuscript. CB and MS are the project managers and co-investigators, were involved in study selection and extraction and statistical analysis, and contributed to the writing and revision of the manuscript. LL, MM, FDF, and SB-F are co-investigators, took part in study

### REFERENCES


selection and extraction and contributed to the writing and revision of the manuscript. All authors read and approved the final manuscript.

### FUNDING

This project is funded by the governmental Program Graduate Education Institutions-PROSUP-CAPES/UNISO.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01075/ full#supplementary-material


countries. *Pediatr. Int. Off. J. Jpn. Pediatr. Soc.* 54, 770–772. doi: 10.1111/ j.1442-200X.2012.03692.x


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor is currently organizing a Research Topic with MS, LL, SB-F, and CB and confirms the absence of any other collaboration.

*Copyright © 2019 Mellone, Silva, Paglia, Lopes, Barberato-Filho, Del Fiol and Bergamaschi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Pharmacokinetic Aspects of Nanoparticle-in-Matrix Drug Delivery Systems for Oral/Buccal Delivery

*Renata Carvalho Feitosa1,2, Danilo Costa Geraldes1,2, Viviane Lucia Beraldo-de-Araújo1, Juliana Souza Ribeiro Costa1,2 and Laura Oliveira-Nascimento1\**

*1 Laboratory of Pharmaceutical Technology (Latef), Faculty of Pharmaceutical Sciences, State University of Campinas, Campinas, Brazil, 2 Institute of Biology, State University of Campinas, Campinas, Brazil*

Oral route maintains its predominance among the ones used for drug delivery, especially when medicines are self-administered. If the dosage form is solid, therapy gains in dose precision and drug stability. Yet, some active pharmaceutical substances do not present the required solubility, permeability, or release profile for incorporation into traditional matrices. The combination of nanostructured drugs (nanoparticle [NP]) with these matrices is a new and little-explored alternative, which could bring several benefits. Therefore, this review focused on combined delivery systems based on nanostructures to administer drugs by the oral cavity, intended for buccal, sublingual, gastric, or intestinal absorption. We analyzed published NP-in-matrix systems and compared main formulation characteristics, pharmacokinetics, release profiles, and physicochemical stability improvements. The reported formulations are mainly semisolid or solid polymers, with polymeric or lipid NPs and one active pharmaceutical ingredient. Regarding drug specifics, most of them are poorly permeable or greatly metabolized. The few studies with pharmacokinetics showed increased drug bioavailability and, sometimes, a controlled release rate. From our knowledge, the gathered data make up the first focused review of these trendy systems, which we believe will help to gain scientific deepness and future advancements in the field.

Keywords: nanoparticle, oral delivery, buccal delivery, matrix delivery, drug absorption

### INTRODUCTION

Medicines administered by the oral cavity have different fates, according to their varied processes: (i) immediate delivery and absorption, (ii) slow delivery from an adherent drug delivery system (DDS), followed by absorption or local action; and (iii) transport to the gastrointestinal region for absorption or local action. The latter is the gold standard for medicines and the most common process used for self-administered drug intake. Recently, buccal permeation (items i and ii) strategies increased in number of developments; this route bypasses hepatic and gastrointestinal effects, which is advantageous to sensitive drugs (Barua et al., 2016).

Regardless of the process, drug solubility can impair efficient release from DDS and subsequent mucosal crossing. Therefore, several solubilization strategies are available, such as salt forms of active molecules, pH modifiers, cosolvents, amorphization, solid dispersions, inclusion complexes, microemulsions, and nanotechnology (Kalepu and Nekkanti, 2015). The latter represents a large portion of published research but a few marketed products and no buccal-based options.

*Edited by: Silvio Barberato-Filho, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Han Qiao, Shanghai Jiao-Tong University School of Medicine, China Bruno Fonseca-Santos, São Paulo State University, Brazil*

*\*Correspondence:*

*Laura Oliveira-Nascimento lauraon@unicamp.br*

#### *Specialty section:*

*This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology*

*Received: 26 February 2019 Accepted: 20 August 2019 Published: 24 September 2019*

#### *Citation:*

*Feitosa RC, Geraldes DC, Beraldo-de-Araújo VL, Costa JSR and Oliveira-Nascimento L (2019) Pharmacokinetic Aspects of Nanoparticle-in-Matrix Drug Delivery Systems for Oral/Buccal Delivery. Front. Pharmacol. 10:1057. doi: 10.3389/fphar.2019.01057*

**245**

In addition to the solubility enhancement, nanoparticles (NPs) aid drug efficacy in at least four different ways (**Figure 1**). The simpler way relates to transport throughout the gastrointestinal lumen and local release (mode 1). Another possibility consists of carrier adherence to the mucosal surface/mucus to enhance drug absorption, but without carrier crossing (mode 2). Nanoparticles may also be absorbed and transport the drug systemically, enhancing its plasma half-life (mode 3). Absorbed NPs can then perform passive targeting due to charge, size, morphology, and constituents. For a detailed review on mechanisms of NP/ absorption and uptake, see Griffin et al. (2016). Mode 4 happens

when specific surface molecules or pH-responsive mechanisms allow the carrier to perform an active targeting after absorption. Modes 3 and 4 can still avoid first-pass metabolism if absorption happens in the buccal mucosa or lymphatic vessels. However, like other delivery strategies, NPs can fail to liberate drugs at an expected rate or target, as described in the following paragraphs.

degradation. 2: Adherence to mucus or mucosal surface for enhanced drug absorption (no NP permeation), followed by NP excretion or degradation. 3: NP absorption for enhanced plasma half-life, followed by NP liver degradation or lymphatic drainage. 4: Tissue or cell targeting. Figure created for this review in CorelDRAW Graphics Suite X7.

For drug uptake enhancement, NPs need to interact with the respective mucosal surface/mucus during an adequate amount of time. Some particles do stick to mucosa under continuous rinsing, such as thiolated chitosan NPs. But, in this case, NPs were freeze dried and applied on top of the mucosa in an *ex vivo* test (Bernkop-Schnürch et al., 2006). Since powder formulations require specific devices or encapsulation for a precise dosage, they generally require an additional delivery strategy to be administered.

A noteworthy NP disadvantage is the burst effect: a rapid initial drug release followed by a sustained or controlled release. Rapid plasma peaks or local drug concentration are desirable in some therapies; however, burst behavior is frequently uncontrollable and irreproducible. The phenomenon occurs due to several reasons, including weakly bound drugs, molecule migration to the particle surface, and nanomatrix heterogeneity (Kamaly et al., 2016). Strategies to prevent it include coating or nanomatrix reformulation (Kamaly et al., 2016), not always straightforward because both processes change NP physicochemical characteristics.

The majority of oral DDSs are biodegradable, so they will erode or degrade at some point. In fact, some systems rely on both properties as a release mechanism. If the carrier liberates as desired, it may not be resistant to pH/enzymes during traffic or at the action site. An earlier instability may interfere with the desired drug liberation profile, such as for regular gelatin NPs: their degradation in the stomach requires coating, chemical modification or an outermatrix embedment to allow intestinal release (Imperiale and Sosnik, 2013).

Apart from degradation, NPs can change their fate, uptake, or release rate due to *in vitro* or *in vivo* aggregation. The *in vitro* facet shows that cell media for cell culture is quite different from lumen or saliva environments, which makes it hard to predict aggregation during *in vivo* absorption (Moore et al., 2015).

All failure modes described above can be solved or attenuated with NP dispersion in micromatrices and macromatrices (**Figure 2**), reasons why this technique became important for buccal and oral delivery. As opposed to coating and core modifications, matrix embedment brings an extra added value: it can be applied to several kinds of NPs, with a better general prediction of the release behavior.

Considering all points above, this review intends to analyze pharmacokinetic improvements with the use of NPs incorporated in matrices for drug delivery (NP-in-matrix). Therefore, we focused here on pharmacokinetic data from animal and humans (when available). Several NP-in-matrix systems were tested only with *in vitro* assays. However, correlation of *in vivo* behavior is not guaranteed, and an extensive array of protocols impairs model comparisons (discussed in the following topic). Combined systems resulting in nanosized carriers (NP-in-NP) were not included in this work. Based on the theoretical analysis, we intend to verify and clarify matrix role on permeation and bioavailability of nanocarriers.

### PHARMACOKINETICS AND EVALUATION MODES FOR ORAL AND BUCCAL ADMINISTRATION

Pharmacokinetics describes absorption, distribution, metabolism, and excretion processes after drug intake. This knowledge remains essential for safety and efficacy assessments of the therapy in

FIGURE 2 | Scheme (not to scale) of the failure modes of nanoparticles (NPs, yellow circles) loaded with drug (blue circles) after oral/buccal administration: early washout in mouth or intestinal cavities; early degradation due to pH or enzymes; burst release upon contact with aqueous media (saliva, stomach acid); aggregation due to pH, osmotic environment, and protein binding. The network on the left shows matrix protection through adhesion (wash out elimination), shielding (degradation and burst elimination/decrease), and physical separation (aggregation elimination). Figure created for this review in CorelDRAW Graphics Suite X7 and Adobe Photoshop CS6.

the search for the optimal DDS. However*, in vivo* tests must be a final step toward product development. It should succeed an extensive physicochemical characterization, together with promising *in vitro* data assessing biorelevant properties. **Table 1** lists the most common assays used to evaluate and predict *in vivo* parameters related to pharmacokinetics, together with drug and NP predictors. Because matrix-based formulations aid mainly in absorption, this process will be discussed in more detail below.

Nanoparticle permeation happens only after mucus barrier crossing (gastrointestinal) and subsequent cellular uptake for transcytosis (M cells, enterocytes) or paracellular transport. *In vivo* absorption is generally better predicted by *ex vivo* mucosal permeation than *in vitro* tests with synthetic membranes; the extracted tissue offers a biochemical, anatomical, and structural resemblance to its *in vivo* counterpart that is difficult to replicate (Berben et al., 2018).

When it comes to gastrointestinal absorption, Caco-2 cell model stands out among the options, joining cell layer with synthetic membrane permeation. This system is frequently used to classify drugs in a permeation rank to direct dosage form development. However, a review covering Caco-2 correlation with *in vivo* intestinal permeability stressed the test fragility. Based on several studies, the authors concluded that the model is applicable to hydrophobic drugs but fails to predict hydrophilic molecule absorption. Even with hydrophobic drugs, variability is high yet minimized with internal standards. The paracellular route and active transport seem to correlate with higher variability and *in vivo*–*in vitro* lack of correlation (Larregieu and Benet, 2013).

Regardless of the *in vitro–ex vivo* model, most of them do not promote predigestion of samples. One cannot estimate the effect of gastrointestinal fluids upon NPs without this assessment. Even nondegradable polystyrene NPs (unmodified, aminated, or carboxylated) promoted different *ex vivo* permeation profiles with predigestion, showing its relevance for *in vivo* prediction (Westerhout et al., 2017).

Drug/NP *in vivo* absorption can be estimated by plasma detection if metabolism does not influence their integrity. *In vivo* pharmacokinetics focuses on blood sampling and drug quantification at several time points, generally with rats, rabbits, or humans. The most common parameters include the amount of drug detected over time (area under the curve [AUC]),


*aIn vivo models include generally mice, rat, rabbit, dog, or monkey species. Monkeys are considered more reliable than other animal species to infer human pharmacokinetic profiles (Furukawa et al., 2014). bDrugs that are substrates for biological transporters do not present predictive permeability with only these parameters. cUSP dissolution methods vary depending on the dosage form. They can be predictive for drug with dissolution rate-limited absorption, but complex release kinetics and NP environmental changes may increase in vivo correlation failure. dTo check ligand properties, the most common method is molecular docking (Meng et al., 2011).*

#### TABLE 1 | Pharmacokinetic parameters and evaluation strategies for oral/buccal medicines.

the highest drug level detected (*C*max), and the time the latter happened (*t*max). On the other hand, NPs are seldom classified based on plasma levels. Nanoparticle tracking relies mainly on marked particles with fluorescent dyes to observe biodistribution in tissues. An important reminder is that plasma levels do not discriminate buccal from gastric or intestinal absorption.

### NP-IN-MATRIX DDSS

The following items present a discussion of all articles we could find with *in vivo* pharmacokinetic data of NP-in-matrix DDSs. The NP type directed subdivisions based on their main constituent. Exact values and formulation details are presented in **Table 2**.

### Polymeric NPs

Polymers vary on their degradation, polarity, source, and chain size properties (Hallan et al., 2016). Polymeric based NPs present scalable manufacturing methods and capability to load a wide range of drug types (Crucho and Barros, 2017). The most common natural polymers for NPs are chitosan, sodium alginate, dextran, gelatin, and albumin. These hydrophilic proteins and carbohydrates degrade in physiological conditions, besides their biocompatibility, which helps to avoid side effects (Boateng and Areago, 2014; Madkhali et al., 2019; Bronze-Uhle et al., 2017; Kim et al., 2001). Alginate and chitosan still exhibit bioadhesive nature, which increases NP efficacy for mucosal delivery (Boateng and Areago, 2014). In turn, albumin discharges drugs *via* desorption without significant burst effects (Jiang and Stenzel, 2016). We have not found any pharmacokinetics data for NP-in-matrix systems based on gelatin or chitosan NPs; however, several studies discussed in this article address these excipients as matrix components.

Concerning alginate NPs, Garhy et al*.* loaded carvedilol in these carriers and further incorporated this system in buccoadhesive gels. The *in vitro* release assay showed burst behavior for all 12 different NP formulations. However, NP-ingel formulations (FG1 and FG2) indicated that the gelling agent delayed carvedilol release, which diminished burst. FG1 gel contained hydroxypropyl methylcellulose (HPMC), whereas le FG2 presented the same HPMC concentration and sodium carboxymethylcellulose. Rabbits treated with FG2 formulation showed a two-fold increase in relative bioavailability compared to the market product. The increase in bioavailability occurred probably due to NP enhancement in drug solubility and the buccal bypass of the first-pass effect (Garhy et al., 2018).

Albumin properties were tested in exenatide-loaded bovine serum albumin/dextran NPs; the peptide was adsorbed to the carrier protein and released in a sustained manner due to dextran crosslinkings. This loaded NP was incorporated in gastroresistant microparticles of Eudragit L/HPMC. The incorporation led to macromolecule protection and decreased release rates of the peptide cargo. Consequently, pharmacokinetic data showed that NP-in-matrix resulted in a high relative oral bioavailability of 77% compared to a subcutaneous injection of the commercial equivalent medicine (Soudry-Kochavi et al., 2015).

Among the synthetic polymers, the aliphatic polyesters and their copolymers are the most used for drug delivery because of their biodegradability and biocompatibility. One of the most popular is the poly(lactic acid), often combined with glycolide to form the hydrophobic copolymer poly(lactic-co-glycolic acid) (PLGA) (Washington et al., 2017); PLGA exhibit high stability in biological fluids and long clinical experience; it was the only synthetic polymer to compose a nanostructure part of an NP-inmatrix dosage form with pharmacokinetic data. All the drugs entrapped into these studied systems are considered poorly water-soluble drugs.

Three different studies reported buccal films as matrices for PLGA NPs. The incorporated drugs were acyclovir (Al-Dhubiab et al., 2015), selegiline (Al-Dhubiab, 2016a) and zolpidem (Al-Dhubiab, 2016b). *In vitro* studies showed that film formulations prolonged drug liberation in a compositiondependent mode. The results of *ex vivo* studies with rabbit buccal mucosa showed that these nanospheres can permeate the tissue. *In vivo* results confirmed the predictions, whereas male rabbits demonstrated an increased drug bioavailability with the NP-in-film strategy. After incorporating NPs in polymeric films, the bioadhesive properties increased the residence time in oral cavity. It was also observed that *C*max, AUC, and *t*max improved with the use of combined systems when compared to controls (oral drug solutions, results in **Table 2**).

The PLGA NP improvements may come as well from a micro matrix strategy. Nassar et al*.* (Nassar et al., 2011) used docetaxelloaded PLGA nanocapsules in entero-coated microparticles. The microcarrier released NPs that penetrated the enterocytes of rats, bypassed permeability-glycoprotein pump, and apparently circumvented gut metabolism of the drug. An oral administration of NP-in-matrix resulted in higher bioavailability than intravenous solution of the free drug (commercial formulation, 276%) and its NP formulation (400%). A subsequent study with minipigs confirmed the pattern obtained with rats; the superiority of NP-inmatrix over NPs was attributed to lymphatic transportation that changed drug biodistribution (Attili-Qadri et al., 2013).

### Lipidic NPs

Lipid NPs are well-established DDSs due to their high biocompatibility, biodegradability, low toxicity, and applicability to various administration routes (Chime and Onyishi, 2013; Wissing et al., 2004; Shastri, 2017; Teixeira et al., 2017; Puri et al., 2009). The main lipid NPs reported are liposomes, solid lipid nanoparticles (SLNs), nanostructured lipid carriers, and nanoemulsions. The hydrophobic cores enhance drug solubility and protect it from the environment; the surfactant layer separates particles by steric or electrical hindrance. The exception is the liposome vesicle form, which allows loading of hydrophobic and hydrophilic drug molecules into its outer layer or aqueous core, respectively (Karamanidou et al., 2016). Among the cited particles, only NLC did not present *in vivo* data, probably because it is the most recent development among this group.

Lipidic NPs may deform under mechanical stress, such as tableting; they can also undergo lipid phase transitions upon heating, such as for film casting. Therefore, Hazzah et al. incorporated curcumin SLN in freeze-dried polymeric sponges

Feitosa et al.

TABLE 2 | Pharmacokinetic data for NP-in-matrix systems.


*(Continued)*

#### TABLE 2 | Continued


*(Continued)*

Nanoparticle-in-Matrix Delivery Systems

Feitosa et al.

TABLE 2 | Continued


*(Continued)*

Feitosa et al.


#### TABLE 2 | Continued


*(Continued)*

Frontiers in Pharmacology | www.frontiersin.org TABLE 2 | Continued

Lyophilized tablet (LT)

Porous fumed silica, lactose, and microcrystalline

Microparticle Eudragit L + HPMC Polymeric

(nanocapsule)

(El-Say et al., 2017)\*\*

(Attili-Qadri et al.,

2013 )


Vitamin K • Model: male human volunteers.

*t*max = 2.0 ± 1.0 h;

∞ = 866.14

system (1.25 mg/kg).

± 215.45 (ng∙h/mL).

• Administration: oral dose of Taxotere (commercial) and combined

AUC

Docetaxel • Model: minipigs.

•Administration: subjects were classified into 3 groups (6 per each). Group I: vitamin K-in-SNELTs; group II: commercial tablet; group III:

*(Continued)*

Nanoparticle-in-Matrix Delivery Systems

Glyceryl tributyrate + oleoyl polyoxylglycerides + PLGA

References Matrix type Matrix excipients NP-Type NP excipients Drug Pharmacokinetics

Labrasol, and Transcutol

Self-nanoemulsion (SNE)

TABLE 2 | Continued


*NP, nanoparticle; AUC, area under the curve that describe drug concentration over time, measured in plasma; Cmax, the maximum drug concentration measured in plasma; tmax, the time taken to reach the maximum drug concentration; PVA, polyvinyl alcohol; PLGA, poly(lactic-co-glycolic acid); HPMC, hydroxypropyl methylcellulose; PEG, polyethylene glycol; SDS, sodium dodecyl sulfate; PVP, polyvinylpyrrolidone; BSA, bovine serum albumin. \*Studies comparing pharmacokinetic data for NP-in-matrix and free NPs.*

*\*\*NP-in-matrix systems for oral delivery.*

to avoid the cited issues. Human studies showed that curcumin SLN-in-polycarbophil sponge has higher *C*max, *t*max, and AUC than the SLN-in-HPMC sponge (**Table 2**). In accordance, polycarbophil formulation adhered to the mucosa for a longer time (15 h, compared to 4 h) and presented higher matrix porosity and homogeneous distribution of SLNs. The decreased porosity of HPMC sponges diminished swelling and consequent interaction with mucin. Also, SLNs remained onto the surface of HPMC sponges, lowering its adhesion property to the oral mucosa and releasing the NPs faster than polycarbophil. Although pure NPs were not tested *in vivo*, *in vitro* release showed that NP-insponge eliminated the burst effect of the nanodispersions, which improves prediction of therapy outcomes and avoids possible plasma peaks (Hazzah et al., 2015).

Ahmed et al. also used freeze-dried polymeric matrix (gelatin "tablets," no compression step) to load lipidic self-nanoemulsions (SNEs) of finasteride. Different from the mucoadhesive sponges, the freeze-dried tablets melt on the mouth upon contact with saliva. These macrocarriers aim as fast release as possible for buccal and gastrointestinal absorption. Both formulated tablets (with or without SNE) presented in human higher *C*max and shorter *t*max of the drug than the marketed tablets. Likewise, AUC and Mean Residence Time (MRT) indicated superiority of NP-in-tablet system. The improved drug bioavailability may enhance the therapeutic effects (Ahmed et al., 2018).

A variety of this technique decreased the low bioavailability of oral vitamin K. The SNE was loaded on porous silica carriers and later incorporated in lyophilized tablets. SNE-in-lyophilized tablets increased absorption rate and extent of vitamin K in humans compared to marketed tablets. Even better, the NP-inmatrix system presented AUC similar to the commercial intramuscular injection (El-Say et al., 2017). Powdered SNE loaded with vitamin K1 was also incorporated into regular tablets, but with different vitamin loadings. Because SNEs form nanoparticles upon contact with gastrointestinal fluid, NP deformation was not a concern. Beagle dogs administered with SNE-in-tablet showed a 2.3-fold increase in vitamin K1 *C*max and 1.98-fold in AUC (Tong et al., 2018). The increment in bioavailability probably happened due to higher surface area and consequent higher drug dissolution rate in the gastrointestinal tract (Gong et al., 2016).

SNE-in-tablet was also applied to rosuvastatin delivery, increasing solubility and bypassing hepatic first-pass metabolism. Male humans taking SNE-in-tablet had rosuvastatin AUC increased 2.45 times and *C*max increased 2.78 times compared with the intake of commercial rosuvastatin tablets. Similarly, *t*max decreased with drug administration in SNE tablet, highlighting its benefits (Salem et al., 2018).

The latter lipid NP-in-matrix with published *in vivo* pharmacokinetics is the phospholipid-bile salts-mixed micelles. Cucurbitacin B was loaded in these micelles to address its water insolubility, toxicity, and gastrointestinal side effects. The use of fast-dissolving oral films of pullulan as a matrix maintained the *in vivo* absorption properties like micelles and promoted a significant increase in the oral bioavailability of cucurbitacin B in Wistar rats (highest values of *C*max and AUC, lowest *t*max), when compared to the free suspension drug. Also, the matrix did not interfere with Cu B-micelles original structure (Lv et al., 2014). These micelles in carboxymethyl chitosan buccal film rendered a mucoadhesive formulation, which released the drug for a longer period. It resulted in 2.69-fold increase in bioavailability in rabbits when compared to marketed tablets and 10.46 times the film formulation without NP. The team reported that buccal mucosa barrier probably explains higher *C*max and lower *t*max of oral tablets; high AUC from film formulations may be owed to the presence of permeation enhancers in it and the nanosized drug, ensuring the increase in the amount of drug penetration into the blood (Lv et al., 2015). It would be interesting to compare the influence of the different matrices in cucurbitacin B kinetics, but the animal models differ in species, drug dose, and period of evaluation.

### Inorganic NPs

Inorganic nanoparticles are flexible carriers that allow surface modification, drug targeting, and modified drug release. This group includes silica, clay, and metals as excipients arranged in nanoparticles, nanotubes, or nanorods/nanoparticles, respectively (Kerdsakundee et al., 2017; Rao et al., 2011; Zhang et al., 2018). The silica-based ones were the only inorganic NP-inmatrix compositions tested *in vivo* for oral administration. Although biocompatible and with a well-defined/modifiable structure, they do not adhere to mucosa and are good candidates for matrix incorporation (Slowing et al., 2008).

To improve the oral delivery of the poorly soluble drug cinnarizine, an antihistamine and calcium-channel blocker, Rao S. et al. developed a pluronic functionalized silica–lipid hybrid microparticle. Pluronic acts as a polymeric precipitation inhibitor, avoiding recrystallization of cinnarizine dissolution; the silica–lipid hybrid microparticle improved drug partition by producing a hydrophobic microenvironment. Bioavailability was compared between the cinnarizine loaded functionalized silicalipid hybrid system, unformulated cinnarizine and a cinnarizine loaded nonfunctionalized silica-lipid system; the in vivo design included a single dose of 10 mg/kg of each formulation *via* oral gavage. The study resulted in more than 2.1-fold improvement in the AUC and 1.6-fold improvement in *C*max of cinnarizine of the functionalized NP-in-microparticle structure in comparison to the unformulated one and a 1.6-fold improvement in both AUC and *C*max in comparison with the nonfunctionalized formulation, resulting in an overall improved bioavailability of cinnarizine (Rao et al., 2015).

### Drug-Based NPs

Poor solubility, low bioavailability, and short stability *in vivo* are limiting problems in the development and delivery of new active ingredients (Liu et al., 2012). To overcome these issues, one can entrap the drug in a nanocarrier or reduce its particles to obtain a nanosized range, such as nanocrystals. Compared to NPs, drug-based particles offer higher drug loading (nearly 100%) with generally less excipients and uniform/stable physical nature (Liu et al., 2012).

Accordingly, indinavir nanonization increased absorption (from 0.83 to 18.16 µg∙h/mL) and *t*max (from 1.10 to 2.50 h) of the free base drug after administration of a single oral dose in mongrel dogs; however, NP-in-microparticle alginate/chitosan

particles performed much better than pure nanocrystals (AUC of 39.23 µg∙h/mL and *t*½ value of 76.3 h). This represented an increase of the oral bioavailability and the apparent *t*½ of 47 and 95 times, compared to the free drug. The increment in release time may improve HIV treatment, which demands long-term therapy and frequent dosing (Imperiale et al., 2015). Likewise, darunavir and ritonavir benefited from an NP-in-microparticle oral delivery system for nanocrystals. The difference was that NP-in-microparticle increased the oral bioavailability of the combined drugs by 2.3-fold compared with the NP only and the free drug. In this case, nanonization alone was not able to increase absorption (Augustine et al., 2018).

Nanocrystals vehicled in tablets and buccal films allow easier self-administration, increased dosage precision, and superior performance. Rebamipide presents only 10% of oral bioavailability in humans due to poor solubility, which is the reason Guo and coworkers formulated an NP-in-tablet version of the medicine. Drug nanocrystals were stabilized with HPMC and polyvinylpyrrolidone before tablet incorporation. The relative oral bioavailability of REB nanocrystal tablets was 256.8% in rats (reference Mucosta® tablets) (Guo et al., 2015). Nekkanti's group also used HPMC-stabilized nanocrystals for candesartan cilexetil delivery, further incorporated in mannitol-based tablets. This prodrug belongs to the low solubility/long-term therapy group, such as rebamipide. *In vivo* studies confirmed dosage form benefits: Wistar rats presented 2.51-fold increase in AUC, a 1.77-fold increase in *C*max, and a decreased *t*max (1.81–1.06 h) compared to the free prodrug (Nekkanti et al., 2009).

Rana and Murthy developed a three-layer buccal film: a mucoadhesive layer, a layer containing nanosuspension of carvedilol nanocrystals and a backing membrane. The structure aimed to prevent the first-pass metabolism, raising drug bioavailability. *C*max of the buccal patch was 7.3 times higher than that of the oral tablet, and *t*max exhibited by the patch was 4 h in comparison to 2h for oral tablet. The NP-in-microparticle structured buccal patch has been designed as a novel platform for potential buccal delivery of drugs having high first-pass metabolism (Rana and Murthy, 2013).

Another application of nanosized drugs is the development of better suitable dosage forms of existing drugs to attend the patient needs. Kevadiya et al. developed an oral strip-film containing nanocrystals of the cholesterol-reducing agent fenofibrate, a very low solubility prodrug, aiming to create a fast disintegrating solid dosage form ideal for emergency administration and for patients with swallowing difficulties. New Zealand white rabbits were divided in three groups: the first received the oral commercial formulation Tricor in suspension form; the second received the suspension of fenofibrate nanocrystals (pristine FNB) and

### REFERENCES


the third received the oral striped-films containing fenofibrate nanocrystals (OSF). Pharmacokinetics data showed a higher *C*max and lower *t*max of OSF when compared to the marketed Tricor formulation and pristine FNB. The AUC was also higher for OSF formulation (931.26 µg·h/mL) compared to Tricor (654.6 ± 251 µg·h/mL) and pristine FNB (514.8 ± 374 µg·h/mL); and the *t*max (h) of OSFs was found to be 2 h earlier than the *t*max of Tricor and pristine FNB (Kevadiya et al., 2018).

### CONCLUSION

Because most of the articles we gathered date from this decade, NP-in-matrix approach still has a long way for exploitation. In our findings, most matrices belong to the polymeric group, based on classical excipients in the market, like HPMC. However, they vary from buccal to oral delivery, with rapid or slow degradation to offer an immediate or modified release. The use of an external matrix to incorporate nanoparticles brought several advantages to formulations. This type of system reduced burst effect, avoided NP degradation in gastrointestinal tract, increased residence time in the mouth, modulated *t*max, and bypassed first-pass metabolism (buccal forms and some oral forms). Thus, the systemic bioavailability of the tested drugs was successfully enhanced.

Buccal release belongs to the trending strategies for drugs, so we expect an increase in pharmacokinetic studies concerning NP-in-matrix buccal delivery. Nanoparticle types will probably expand too, as the actual group is mainly PLGA, nanocrystals, and some lipid particles. In summary, we believe this article compiled several evidences and possible pitfalls of this strategy, which will help future developments on the field.

### AUTHOR CONTRIBUTIONS

RC: synthetic polymeric NP-in-matrix research and writing. DG— inorganic NP-in-matrix research and writing. VB-d-A: lipidic NP-in-matrix research and writing. JC: tables, figures and conclusion. LO-N: review design, introduction and pharmacokinetics writing, scientific revision.

### FUNDING

FAPESP 14/14457-5 - article charges funding CAPES 001 scholarships for graduate students (RF, DG, VB-d-A, JC). This study was financed by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001 and FAPESP project 2019/08281-5 and 2019/19696-1.

delivery of acyclovir. *Colloids and Surfaces. B. Biointerfaces* 136, 878–884. doi: 10.1016/j.colsurfb.2015.10.045


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Feitosa, Geraldes, Beraldo-de-Araújo, Costa and Oliveira-Nascimento. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis

*Clare Yuen Zen Lee1, Pairote Chakranon2 and Shaun Wen Huey Lee1,3,4\**

*1 School of Pharmacy, Monash University Malaysia, Bandar Sunway, Malaysia, 2 Faculty of Pharmacy, Silapakorn University, Pathom, Thailand, 3 Asian Centre for Evidence Synthesis in Population, Implementation and Clinical Outcomes (PICO), Health and Well-being Cluster, Global Asia in the 21st Century (GA21) Platform, Monash University Malaysia, Selangor, Malaysia, 4 School of Pharmacy, Taylor's University, Subang Jaya, Malaysia*

#### *Edited by:*

*Marcus Tolentino Silva, Universidade de Sorocaba, Brazil*

#### *Reviewed by:*

*Brian Godman, Karolinska Institute (KI), Sweden Karel Allegaert, University Hospitals Leuven, Belgium*

> *\*Correspondence: Shaun Wen Huey Lee shaun.lee@monash.edu*

*PROSPERO Registration: CRD42016053390*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 01 April 2019 Accepted: 23 September 2019 Published: 25 October 2019*

#### *Citation:*

*Lee CYZ, Chakranon P and Lee SWH (2019) Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis. Front. Pharmacol. 10:1221. doi: 10.3389/fphar.2019.01221*

Context: Several interventions are available for the management of hypoxic ischemic encephalopathy (HIE), but no studies have compared their relative efficacy in a single analysis. This study aims to compare and determine the effectiveness of available interventions for HIE using direct and indirect data.

Methods: Large randomized trials were identified from PubMed, EMBASE, CINAHL Plus, AMED, and Cochrane Library of Clinical Trials database from inception until June 30, 2018. Two independent reviewers extracted study data and performed quality assessment. Direct and network meta-analysis of randomized controlled trials was performed to obtained pooled results comparing the effectiveness of different therapies used in HIE on mortality, neurodevelopmental delay at 18 months, as well as adverse events. Their probability of having the highest efficacy and safety was estimated and ranked. The certainty of evidence for the primary outcomes of mortality and mortality or neurodevelopmental delay at 18 months was evaluated using GRADE criteria.

Results: Fifteen studies comparing five interventions were included in the network metaanalysis. Whole body cooling [Odds ratio: 0.62 (95% credible interval: 0.46–0.83); 8 trials, high certainty of evidence] was the most effective treatment in reducing the risk of mortality, followed by selective head cooling (0.73; 0.48–1.11; 2 trials, moderate certainty of evidence) and use of magnesium sulfate (0.79; 0.20–3.06; 2 trials, low certainty of evidence). Whole body hypothermia (0.48; 0.33–0.71; 5 trials), selective head hypothermia (0.54; 0.32–0.89; 2 trials), and erythropoietin (0.36; 0.19–0.66; 2 trials) were more effective for reducing the risk of mortality and neurodevelopmental delay at 18 months (moderate to high certainty). Among neonates treated for HIE, the use of erythropoietin (0.36; 0.18–0.74, 2 trials) and whole body hypothermia (0.61; 0.45–0.83; 7 trials) were associated with lower rates of cerebral palsy. Similarly, there were lower rates of seizures among neonates treated with erythropoietin (0.35; 0.13–0.94; 1 trial) and whole body hypothermia (0.64; 0.46–0.87, 7 trials).

**260**

Conclusion: The findings support current guidelines using therapeutic hypothermia in neonates with HIE. However, more trials are needed to determine the role of adjuvant therapy to hypothermia in reducing the risk of mortality and/or neurodevelopmental delay.

Keywords: hypoxic ischemic encephalopathy, neonatal, systematic review, meta-analysis, neuroprotective, perinatal

### INTRODUCTION

Neonatal hypoxic ischemic encephalopathy (HIE) is one of the most common causes of severe neurological deficit in children, affecting an estimated 15 per 10,000 live births (Graham et al., 2008; American College of Obstetricians and Gynecologists, and American Academy of Pediatrics, 2014). Several reviews have suggested that therapeutic hypothermia (both whole body and selective head) reduces mortality and improves survival with normal neurological outcome and is now a standard treatment protocol for most neonatal centers in developed countries (Edwards et al., 2010; Jacobs et al., 2013; Martinello et al., 2017). Despite this, their effectiveness is still limited, with mortality rates of approximately 10% −20% in several large trials (Jacobs et al., 2013). As a result, alternative strategies, including the use of adjuvant therapies such as xenon, allopurinol, erythropoietin, magnesium sulfate, and melatonin, have been suggested (Perrone et al., 2012). Other strategies suggested include cooling for longer periods of time, cooling at lower temperature or both (Shankaran et al., 2017).

Recently, several randomized controlled trials have examined the effects of such adjuvant therapy in neonates with HIE, yielding a complex evidence base that requires careful examination across different strategies (Bhat et al., 2009; Aly et al., 2015; Azzopardi et al., 2016; Filippi et al., 2017). However, most of these studies are relatively small and the results remains inconclusive and mixed. There are no reviews that attempted to summarize these data from large studies.

Furthermore, previous meta-analyses have often compared the efficacy of treatment within pairs of active treatment, which provides limited insights into the overall treatment hierarchy as treatment effects are estimated from two different treatment comparison only (Jacobs et al., 2013; Pauliah et al., 2013). Over the past few years, the use of network meta-analyses which allows for the simultaneous comparison of two or more interventions has increasingly been used (Lee et al., 2015; Lee et al., 2017; Bukhsh et al., 2018; Teoh et al., 2019). Network meta-analysis includes both direct and indirect comparison within a single analysis, thereby providing an integrated and more holistic conclusion, providing decision makers with a more complete evidence matrix (Watt et al., 2019). In this study, we aimed to estimate the efficacy and safety of available neuroprotective interventions for HIE who participated in randomized controlled studies.

### MATERIALS AND METHODS

### Search Strategy

A literature search was performed to identify for studies from inception to June 30, 2018, on the following databases: PubMed, EMBASE, CINAHL Plus, Allied and Complementary Medicines (AMED), and the Cochrane Central Register of Controlled Trials without any language restriction. We also obtained additional records by reviewing the reference list of the retrieved articles and other resources including Google Scholar, NDLTD database, and ClinicalTrials.gov. A full list of search terms can be found in **eText** in **Supplementary Material**.

### Study Selection

Studies were considered eligible for inclusion if they (1) were randomized controlled studies (RCT), (2) recruited term or preterm infants (gestational age >35 weeks) diagnosed with HIE (3) had a control group or comparison group, (4) sufficiently powered to detect differences in the outcome of death and/or disability, and (5) the infants were given any of the following as intervention: magnesium sulfate, deferoxamine, cannabinoids, melatonin, statin, topiramate, xenon, allopurinol, erythropoietin, N-acetylcysteine, or therapeutic hypothermia; either as single intervention or adjunct therapy. Studies which had significant methodological limitations such as poor description of inclusion/ exclusion criteria were excluded.

### Data Collection and Extraction

All identified records were screened independently by titles and abstracts by two reviewers (CL and PC) and validated by another reviewer (SL). The full texts of relevant articles were retrieved for further eligibility assessment, extracted, and any discrepancies were resolved through discussion. The information extracted included the author, study design and population, outcomes, and quality of the study using a standardized data extraction form. We subsequently assessed the study quality using the Cochrane risk of bias assessment tool (Higgins et al., 2011).

### Outcome Measures

The co-primary outcomes of interest were the composite of mortality or major neurodevelopmental disability and/or mortality assessed at least 18 months of age. Secondary outcomes include cerebral palsy, development delay based upon the mental development and psychomotor indices of the Bayley II scales of infant development (Bayley, 1993), seizures, quality of vision and hearing, and potential adverse effects caused by the treatment.

### Statistical Analyses

We used a stepwise approach whereby traditional meta-analysis was performed using the Mantel-Haenszel random-effects model since we expect the presence of heterogeneity. We calculated the risk ratio and risk difference for dichotomous outcomes, and its 95% confidence interval. To determine whether the benefit of treatment on outcomes was affected by the severity of encephalopathy, we examined subgroups for which the severity of encephalopathy was graded as moderate or severe on the basis of clinical examination and/or amplitude integrated electroencephalography. The consistency of the treatment effect across subgroups was explored by calculating the ratio of relative risks with 95% confidence interval. Potential small study publication bias was assessed using visual inspection of the funnel plot and Eggers test. Between studies heterogeneity was assessed using *I*<sup>2</sup> and Cochran's Q method.

We subsequently performed a network meta-analysis which combines the direct and indirect effects of treatment, allowing for simultaneous comparison of multiple treatments. These were ranked using the surface under the curve ranking (SUCRA). Inconsistency checks were performed for closed loop in the network (Higgins et al., 2012; Veroniki et al., 2013). Subsequently, we calculated the number needed to treat (NNT) or number needed to harm (NNH) to better understand the potential benefits of different treatments examined. We used the odds ratios (OR) derived from the usual care comparison in network meta-analysis for mortality outcome to estimate the absolute benefits (Dulai et al., 2016).

Preplanned sensitivity analyses include comparison between high versus low-middle income countries as well as severity of encephalopathy. All analyses were conducted using Stata version 13.0 (StataCorp, College TX). This study is registered with PROSPERO, number CRD42016053390.

### RESULTS

### Study Selection and Characteristics

The literature search identified a total of 1,731 studies and 71 full-text articles were assessed for eligibility (**Figure 1** and **eTables 1** and **2**). Fifteen studies enrolling 2,313 newborns were included in the review (Gluckman et al., 2005; Shankaran et al., 2005; Li et al., 2009; Zhu et al., 2009; Simbruner et al., 2010; Zhou et al., 2010; Joy et al., 2013; Azzopardi et al., 2016; Savitha and Prakash, 2016; Malla et al., 2017; Sreenivasa et al., 2017). Most of the included studies had a similar enrollment criteria and included infants with evidence of birth asphyxia as defined by the American College of Obstetrician and Gynecologists (American College of Obstetricians and Gynecologists, and American Academy of Pediatrics, 2014) with moderate to severe HIE. These studies had recruited newborns which were at least 35 weeks in 1 study, at least 36 weeks in 3 studies, and gestation weeks of more than or equal to 37 weeks in 9 studies. Interventions examined by studies were therapeutic hypothermia examined in 10 studies (8 whole body cooling and 2 selective head cooling), magnesium sulfate in 2 studies, erythropoietin in 2 studies, and use of xenon with therapeutic hypothermia in 1 study (**Table 1**). Eight studies (53.3%) were published in 2011 or later, and the studies were conducted in India (40%), China (20%), or were multicentered studies (20%). Nine studies (60%) had a duration (from recruitment to end of follow-up) of at least 18 months or more.

### Methodological Quality of Included Studies

Twelve (80.0%) studies had adequate reporting on sequence generation, 12 (80.0%) studies described the loss to follow-up, 10 (66.7%) studies described the selection concealment adequately, and 6 (40.0%) studies described the blinding of participants and outcome assessment. However, a high proportion [10 (66.7%)] had unclear risk of bias for blinding of outcome assessors (**eFigure 1**).

### Primary Outcomes Mortality

Fifteen RCTs reported the effectiveness of intervention in reducing the risk of mortality. Pairwise meta-analysis showed that whole body hypothermia was effective in reducing the risk of mortality (OR: 0.71; 0.52–0.92, *I*2 = 0%) compared to usual care (**eTable 3**).

### Mortality and/or Neurodevelopmental Delay at 18 Months

Nine studies reported the long term effects of pharmacotherapy or hypothermia on mortality and neurodevelopmental delay at 18 months (Gluckman et al., 2005; Shankaran et al., 2005; Azzopardi et al., 2009; Li et al., 2009; Zhu et al., 2009; Simbruner et al., 2010; Zhou et al., 2010; Jacobs et al., 2011; Malla et al., 2017). Pooled analysis of studies suggest that both erythropoietin (OR: 0.57; 0.36–0.91, *I*2 = 0%) and whole body hypothermia (0.74; 0.59–0.92, *I*2 = 0%) were effective in reducing the risk of mortality and neurodevelopmental delay at 18 months.

### Secondary Outcomes

Among infants who survived, 12 trials including 1,951 infants reported on cerebral palsy outcome. Whole body hypothermia was found to be statistically superior in reducing the odds developing cerebral palsy (0.70; 0.54–0.92, *I*2 = 0%) compared to usual care. With respect to seizures, 10 trials including 1,703 infants were included. Whole body hypothermia was superior in reducing the rates of seizure compared to usual care (0.73; 0.56– 0.96, *I*2 = 0%).

Five studies reported the neuromotor delay and neurodevelopmental delay among neonates using the Bayley II index, but results were not significant. For the other outcomes of renal failure, sepsis, hypotension, hypoglycemia, bradycardia, hearing loss, and blindness, the evidence base were sparse (**eTable 4**). Meta-analysis suggests that therapeutic hypothermia (both whole body and selective head cooling) were associated with an increased rate of normal survival, defined as survival without cerebral palsy, seizures, normal vision, and hearing.

### Effect of Severity of Encephalopathy

Most studies assessed the severity of encephalopathy by clinical assessment using Sarnat's criteria (Sarnat and Sarnat, 1976), which classifies the degree of encephalopathy to either stages I, II or III, correlating with mild, moderate or severe encephalopathy. Pooled analysis suggests that the relative odds of mortality or neurodevelopmental disability were lower among infants with moderate encephalopathy when treated with erythropoietin (0.27; 0.11–0.63, *I*2 = 0%) or whole body hypothermia (0.63; 0.44–0.89, *I*2 = 0%). Among infants with severe encephalopathy, all interventions including erythropoietin, whole body hypothermia or selective head hypothermia was ineffective in reducing the risk of combined outcomes of mortality or neurodevelopmental disability (**eTable 5**).

### Network Meta-Analysis

### Mortality or Mortality and Neurodevelopmental Disability at 18 Months

Overall, of the 15 unique pairwise comparisons that could be made, only 5 were studied head to head. The network meta-analysis gave an adequate fit to the data and designby-treatment model showed no evidence of inconsistency (**eFigure 2**).

#### TABLE 1 | Characteristics of included studies.


(*Continued*)

#### TABLE 1 | Continued


*AAP, American Association of Pediatrics; ACOG, American College of Obstetricians and Gynecologists; aEEG, amplitude-integrated continuous electroencephalopathy; C, control group; CBV, cerebral blood volume; EAA, excitatory amino acids; EPO, erythropoietin; 1st, first; I, intervention group; h, hours; HIE, hypoxic-ischemic encephalopathy; MgSO4, magnesium sulfate; min, minutes; m, months; NaCl, normal saline; NO, nitric oxide; rhEPO, recombinant human erythropoietin; subcut, subcutaneously.*

*\* Age at randomization; †Interquartile range.*


*Comparisons should be read from left to right. The estimate is located at the intersection of the column-defining treatment and the row-defining treatment. An OR value below 1 favors the column-defining treatment. To obtain ORs for comparisons in the opposing direction, reciprocals should be taken. Any significant results are in bold and underlined.*

All interventions examined showed a reduction in risk of mortality compared to usual care (**Figure 2**, **Table 2** and **eTable 6**). However, only whole body hypothermia was significantly better than usual care (0.62; 0.46–0.83), with an NNT of 11 (95% CI: 7–26) in reducing the risk of mortality (**eTable 7**). With respect to the composite mortality or neurodevelopmental disability outcome at 18 months or longer, erythropoietin was significantly better than usual care in patients with moderate encephalopathy (0.36; 0.19–0.66; **eFigure 3**). Therapeutic hypothermia (both whole body and selective head cooling) was the only treatment statistically superior to usual care in patients with moderate (whole body: 0.45; 0.31–0.66 and selective head: 0.51; 0.29–0.89) or severe encephalopathy (whole body: 0.32; 0.12–0.86).

### Secondary Outcomes

Network meta-analysis suggest that, compared to usual care, treatment with either erythropoietin or whole body hypothermia were associated with lower rates of cerebral palsy (0.36; 0.18–0.74 and 0.61; 0.45–0.83, respectively) and seizures (0.35; 0.13– 0.94 and 0.64; 0.46–0.87, **Figure 3**).

### Sensitivity Analyses

Comparison-adjusted funnel plots of the network metaanalysis for primary outcomes did not suggest any publication bias (**eFigure 4**). Ranking of treatment based upon cumulative probability plots (**eFigure 5**) and SUCRA showed that the most effective treatment for primary outcome mortality was whole body hypothermia (77.8%) and the least effective was usual care (24.5%). In terms of the composite outcome of mortality or neurodevelopmental disability at 18 months, the most effective treatment was erythropoietin (88.8%) and the least effective was usual care (0.2%). Using GRADE, the quality of evidence for primary outcomes were moderate to very low for most comparison (**Table 3** and **eTable 8**).

In the preplanned sensitivity analyses, we excluded studies which were conducted in low-middle income countries since previous meta-analysis have suggested that these study setting could influence the results (Pauliah et al., 2013). When studies from high-income countries were only included, only three comparisons were possible for the outcome of mortality. Results from the network meta-analysis was largely unchanged, and whole body cooling was found to reduce the risk of mortality by 36% (0.64; 0.46–0.90) as well as mortality or major neurodevelopmental disability at 18 months by 49% (0.51; 0.33–0.78) compared to usual care. These results were largely unchanged when we included studies which had examined full term neonates (>37 weeks), where whole body cooling reduced the risk of mortality (0.50; 0.31–0.81) as well as mortality or major neurodevelopmental disability at 18 months (0.44; 0.27– 0.71) compared to usual care (**eTable 9**). In addition, selective

Each rhombus represents the summary treatment effect estimated in the network meta-analysis on the odds ratio (OR) scale. The black horizontal lines represent the credible intervals (CrI) for the summary treatment effects; an OR >1 suggests that usual care is more effective to reduce the risk of mortality, whereas an OR<1 suggests that the comparable treatment is better. The vertical blue line corresponds to an OR=1.

FIGURE 3 | Network meta-analysis forest plots for each treatment versus usual care on secondary outcomes. Each rhombus represents the summary treatment effect estimated in the network meta-analysis on the odds ratio (OR) scale. The black horizontal lines represent the credible intervals (CrI) for the summary treatment effects; an odds ratio>1 suggests that usual care is more effective to reduce the risk of mortality, whereas an OR<1 suggests that the comparable treatment is better. The vertical blue line corresponds to an OR=1.

#### Estimates of effects, credible intervals, and certainty of the evidence for comparison of neuroprotective therapies for neonates with hypoxic ischemic encaphalopathy

Patient or population: Neonates with hypoxic ischemic encephalopathy

Interventions: Whole body cooling, selective head cooling, magnesium sulfate, erythropoietin, whole body cooling with xenon

Comparator (reference): Usual care

Outcome: Mortality; mortality or neurodevelopmental disability at 18 months or later

Setting(s): Inpatient and outpatient

Geometry of the Network

Lee et al.

Neuroprotective Therapies for Hypoxic Ischemic Encephalopathy


*1Study was downgraded due to imprecision and lack of direct RCTs contributing to direct evidence; 2Few event rates with wide confidence intervals leading to imprecision; 3Serious indirectness; 4Contributing direct evidence of moderate quality with inadequate concealment of allocation and blinding.*

head cooling was also noted to lower the risk of mortality or major neurodevelopmental disability at 18 months compared to usual care (0.54; 0.36–0.81).

### DISCUSSION

In this study, direct and indirect evidence from 15 RCTs was combined to compare the association of each therapy used for neuroprotection in HIE. The study has several key findings. Firstly, therapeutic hypothermia was significantly associated with lower odds of mortality and neurodevelopmental delays compared to usual care, with high confidence estimates and a number to treat of between 11 and 16. Secondly, whole body hypothermia should be offered to all infants with HIE, irrespective of the severity as it was effective in reducing the risk of morbidity as well as neurodevelopmental delays when compared to usual care.

Over the past decade, numerous drugs have been proven useful to be beneficial in animal models for neonates with HIE, but how these can be translated into clinical use remains unknown (Nair and Kumar, 2018). Our study provides novel insights into how different neuroprotective agents could be useful in patients with HIE especially when used in combination with hypothermia (Martinello et al., 2017). Although there is much progress, further studies are needed to determine the effectiveness of these adjuvant therapies. Several large clinical studies are underway to examine the benefits of these neuroprotective agents.

A previous review suggested that key differences exists in terms of efficacy between high-income and low-middle income countries, due to the use of low-technology devices, degree of encephalopathy, maternal pre-existing diseases, malnutrition status, infections, as well as study inclusion criteria (Pauliah et al., 2013). In this study, we showed that therapeutic hypothermia especially whole body hypothermia was the most effective intervention irrespective of study setting as well as device used. As HIE is the major cause of up to 23% of 2.8 million neonatal deaths especially in low-resource setting, our findings provide a further impetus for therapeutic hypothermia to be part of standard of care especially in low-middle income countries.

Our findings support the Cochrane review on hypothermia in patients with HIE for the primary outcome, where we found that cooling was beneficial in reducing the risk of mortality and disability (Jacobs et al., 2013). This has similarly been reported by other authors which concluded that hypothermia improves survival and neurodevelopmental delays in newborns (Tagin et al., 2012; Douglas-Escobar and Weiss, 2015). The current review was larger (an additional 4 RCTs and 808 infants) and includes information on the effectiveness of different neuroprotective agents. While the authors of a recent systematic review on hypothermia did not conduct network meta-analysis, the results presented were similar to those reported here and suggest that therapeutic hypothermia is effective.

### Study Strength

Strengths of our review are that we conducted a comprehensive search as well as the identification of new additional studies. We also used multiple approaches to assess the relationship of effects and performed a network meta-analysis, which provides added information on effects of different combination of interventions (drugs and non-drugs). The quality of the evidence generated were rated using the GRADE criterion (Puhan et al., 2014). We used a comprehensive search strategy and searched all pertinent sources for eligible studies, which reduces the possibility of missing any relevant studies. This study also included preterm infants >35 weeks as part of the inclusion criteria and provides a more holistic overview on the clinical safety and efficacy of therapeutic hypothermia in this group of infants where data on outcomes are sparse. Nevertheless, these results need to be confirmed from larger randomized controlled trials such as the Premmie Hypothermia for Neonatal Encephalopathy study which is currently in progress (NCT01793129). Until then, clinicians should take precaution especially when treating preterm neonates since evidence from a recent retrospective cohort analysis have suggested that a high incidence of complication and composite outcome of death and neurodevelopmental impairment (Herrera et al., 2018).

### Study Limitations

The limitations of our review are the inherent heterogeneity in terms of study design, intervention, as well as outcome assessment of included studies. We attempted to minimize this by using rigorous selection criteria and performing several sensitivity analyses to ensure the robustness of our results. Secondly, analysis for other outcomes such as neurodevelopmental outcomes, blindness, as well as adverse events should be interpreted with caution, owing to the few data points available. However, a recent high quality long-term study on effects of depth and duration of cooling showed that there is little to no effect of different hypothermia therapy or duration of therapy on outcome (Shankaran et al., 2014; Laptook et al., 2017). This independent analysis reinforces the case that hypothermia should be standard therapy, and additional policy options may be needed in low resource settings to improve outcomes. As with most network meta-analyses, there were only sparse data for some of the treatment comparison especially those related to erythropoietin, and thus, it is recommended that these treatment effect estimates be interpreted together with their precision.

Our study also revealed that additional studies are needed to further optimize cooling therapy as well as rewarming methods, such as the recently concluded NICHD funded study which examined the impact of different cooling depth and duration in neonates with HIE (Shankaran et al., 2014; Laptook et al., 2017). In addition, sufficiently powered studies which examine the use of adjuvant therapies in addition to hypothermia are needed. Another research goal is to reliably identify for subgroups of newborns who will go on to develop worsening encephalopathy and significant brain injury, possibly through the use of electroencephalography, or other biomarker to ensure that they benefit most from hypothermia. In summary, results of our analysis generally support current guidelines using hypothermia for neonates with HIE irrespective of setting. Our findings further support whole body hypothermia as first line, due to its ease of use, improving mortality and neurodevelopmental outcomes. However, further research is needed to determine if the use of additional adjuvant therapies could further improve outcomes of HIE.

### DATA AVAILABILITY STATEMENT

All datasets generated and analysed in this study are included in the article/**Supplementary Material**.

### AUTHOR CONTRIBUTIONS

SL conceptualized the study, designed the data collection instrument, carried out the statistical analysis, drafted the initial manuscript, and reviewed and revised the manuscript. CL and PC carried out the data acquisition and helped drafted out the

### REFERENCES


initial manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

### ACNOWLEDGMENTS

We wish to thank Dr. Nai Ming Lai of Taylor's University for helping edit the manuscript for clarity, Dr. Mohamed Tagin of University of Toronto, and Dr. Seetha Shankaran of Wayne State University School of Medicine for providing input and advice on this article.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01221/ full#supplementary-material

neurodevelopmental outcome among term neonates with perinatal asphyxia: a randomized controlled trial. *J. Trop. Pediatr*. 60(2), 134–140.


outcome in newborns with hypoxic-ischemic encephalopathy. *Hosp. Pract.* 37, 147–152. doi: 10.3810/hp.2009.12.269


**Conflict of Interest**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor and reviewer, BG, declared their involvement as co-editors in the Research Topic, and confirm the absence of any other collaboration.

*Copyright © 2019 Lee, Chakranon and Lee. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Expression of the Biologically Active Insulin Analog SCI-57 in Nicotiana Benthamiana

*Adriana Muñoz-Talavera1, Miguel Ángel Gómez-Lim2, Luis A. Salazar-Olivo3, Jörg Reinders4, Katharina Lim5, Abraham Escobedo-Moratilla6, Alberto Cristian López-Calleja2, María Cristina Islas-Carbajal1 and Ana Rosa Rincón-Sánchez7\**

#### Edited by:

Luciane Cruz Lopes, Universidade de Sorocaba, Brazil

#### Reviewed by:

Antonio Macho, University of Brasilia, Brazil Alexandra Castilho, University of Natural Resources and Life Sciences Vienna, Austria Ana Cecília Roque, New University of Lisbon, Portugal

#### \*Correspondence:

Ana Rosa Rincón-Sánchez anarosarincon@yahoo.com.mx

#### Specialty section:

This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology

Received: 01 February 2019 Accepted: 18 October 2019 Published: 14 November 2019

#### Citation:

Muñoz-Talavera A, Gómez-Lim MÁ, Salazar-Olivo LA, Reinders J, Lim K, Escobedo-Moratilla A, López-Calleja AC, Islas-Carbajal MC and Rincón-Sánchez AR (2019) Expression of the Biologically Active Insulin Analog SCI-57 in Nicotiana Benthamiana. Front. Pharmacol. 10:1335. doi: 10.3389/fphar.2019.01335

1 Department of Physiology, Institute of Experimental and Clinical Therapeutics, University Center for Health Sciences, University of Guadalajara, Guadalajara, Mexico, 2 Department of Genetic Engineering, Center for Research and Advanced Studies of the National Polytechnic Institute, Irapuato, Mexico, 3 Division of Molecular Biology, Institute for Scientific and Technological Research of San Luis Potosí, San Luis Potosí, Mexico, 4 Scientific Support Unit Analytical Chemistry, Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany, 5 Institute of Functional Genomics, University of Regensburg, Regensburg, Germany, 6 CONACYT-Consortium for Research, Innovation, and Development of the Drylands (CIIDZA), IPICYT, San Luis Potosí, Mexico, 7 Institute of Molecular Biology and Gene Therapy, Department of Molecular Biology and Genomic, University Center for Health Sciences, University of Guadalajara, Guadalajara, Mexico

Diabetes mellitus is a growing problem worldwide; however, only 23% of low-income countries have access to insulin, and ironically it costs higher in such countries than highincome ones. Therefore, new strategies for insulin and insulin analogs production are urgently required to improve low-cost access to therapeutic products, so as to contain the diabetes epidemic. SCI-57 is an insulin analog with a greater affinity for the insulin receptor and lower thermal degradation than native insulin. It also shows native mitogenicity and insulin-like biological activity. In this work, SCI-57 was transiently expressed in the Nicotiana benthamiana (Nb) plant, and we also evaluated some of its relevant biological effects. An expression plasmid was engineered to translate an N-terminal ubiquitin and C-terminal endoplasmic reticulum-targeting signal KDEL, in order to increase protein expression and stability. Likewise, the effect of co-expression of influenza M2 ion channel (M2) on the expression of insulin analog SCI-57 (SCI-57/M2) was evaluated. Although using M2 increases yield, it tends to alter the SCI-57 amino acid sequence, possibly promoting the formation of oligomers. Purification of SCI-57 was achieved by FPLC cation exchange and ultrafiltration of N. benthamiana leaf extract (NLE). SCI-57 exerts its anti-diabetic properties by stimulating glucose uptake in adipocytes, without affecting the lipid accumulation process. Expression of the insulin analog in agroinfiltrated plants was confirmed by SDS-PAGE, RP-HPLC, and MS. Proteome changes related to the expression of heterologous proteins on N. benthamiana were not observed; up-regulated proteins were related to the agroinfiltration process. Our results demonstrate the potential for producing a biologically active insulin analog, SCI-57, by transient expression in Nb.

Keywords: diabetes, insulin analog SCI-57, proteomic profile, Nicotiana benthamiana, transient expression, 3T3- L1 adipocytes

## INTRODUCTION

Diabetes is a public health problem, as it has been estimated that the number of patients with diabetes in 2040 will be 642 million, representing 10% of world population (Ogurtsova et al., 2017).

In order to manage the autoimmune destruction of insulinproducing pancreatic beta cells in type 1 diabetes (DM1), it is necessary to administer insulin *via* lifelong daily injection. Currently, worldwide, DM1 represents 5% to 10% of all diabetes cases (You and Henneberg, 2016), however, the use of insulin is not limited to only those people afflicted with DM1. A prospective study on diabetes in the UK (UKPDS) showed that to achieve better glycemic control, after five years, more than 50% of people with type 2 diabetes (DM2) required additional medication such as insulin (King et al., 1999). Similarly, general recommendations for patients with DM2, published by the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD), include insulin and insulin analogs as treatment options in dual and triple anti-hyperglycemic therapy (Inzucchi et al., 2015).

As a consequence of the increasing incidence of diabetes, the need for insulin and insulin analogs will augment exponentially. Therefore, new strategies for insulin and insulin analog production are urgently required, in order to increase the availability of therapeutic products for containing the diabetes epidemic. Likewise, globally, only 23% of low-income countries reported that insulin is generally available, although at higher costs than high-income countries (WHO, 2016). The average insulin cost increased three-fold from 2002 to 2013, with availability and rising costs having an impact on diabetes patients and health systems worldwide (Hua et al., 2016). For these reasons, the study of new platforms for the expression of insulin and insulin analogs costs is of utmost importance.

Recombinant human insulin is predominantly produced using *Escherichia coli* and yeast. *E. coli*, as expression system implicates several disadvantages; insulin is obtained *via* the production of insulin precursors (IP), therefore *in vitro* cleavage and oxidative refolding are necessary. In addition, some IP forms inclusion bodies requiring solubilization. With regard to yeast-based expression systems, although the insulin is correctly folded and directly secreted in the culture supernatant, the standard recovery and purification process may require numerous steps (Mollerup et al., 2002).

Plants are considered to be safe, effective, and affordable alternative systems for producing a wide variety of recombinant proteins such as enzymes, vaccines, and other biopharmaceuticals (Ma et al., 2005; Redwan, 2007; Castillo-Esparza, 2014). Their most important advantage over bacterial, yeast or mammal systems relates to the low cost of large-scale production (Twyman et al., 2003; Ma et al., 2005), partly because the necessary processes comprise existing agricultural systems, thus reducing operating and capital costs (Lico et al., 2008). Studies carried out by Tuse and collaborators have shown that costs can be as high as 1.00–2.00 dollars per kilogram of protein (Tuse et al., 2014). Plant-based expression systems hold tremendous potential for high-capacity production of insulin, in a very cost-effective manner, and may also contend with bacterial and yeast disadvantages such as extensive purification requirements (Baeshen et al., 2014). Recombinant human insulin has been successfully produced in transgenic *Arabidopsis thaliana* (Nykiforuk et al., 2006), and human proinsulin in transgenic tobacco and lettuce chloroplasts (Boyhan and Daniell, 2011), creating a natural cell store with long-term stability that provides storage until required.

Recombinant proteins can be produced in plants by genetic transformation or transient expression. Stable transformation involves the chromosomal integration of the gene of interest. This process can therefore take substantial time (often years) and require considerable resources. Likewise, the expressed protein yield is relatively low, and the release of a transgenic plant implies biosecurity problems (Edelbaum et al., 1992; Kusnadi et al., 1997). Recent studies have shown that proteins generated by transient expression (magnifection) have benefits, as human pathogens are not involved; thus, sterility is not required during production, meaning expression is rapid and high-level, with the potential to provide gram quantities of product in less than 4 weeks to use in clinical trials for FDA approval (Buyel, 2018). Phase I of the clinical study to produce immunoglobulins for the treatment of non-Hodgkin's lymphoma, by transient expression in *Nicotiana benthaminana*, has been completed (Klimyuk et al., 2014). Another recent development in this field is the production of ZMapp, a combination of three chimeric monoclonal antibodies destined to the treatment of Ebola virus disease. This combination is being developed as a product of molecular agriculture by the Leaf Biopharmaceutical Company and is produced by transient expression in *N. benthamiana* (Budzianowski, 2015; Hiatt et al., 2015). ZMapp was approved by the FDA and WHO during the Ebola outbreak in West Africa in 2014, because the transient expression system allowed rapid production, and previous studies had shown positive results among primates (Qiu et al., 2014). Subsequently, ZMapp began formal clinical development and recently completed phase II trials in Liberia, Sierra Leone, Guinea, and the United States (Na et al., 2015; Davey et al., 2016).

SCI-57 is an insulin analog that has a ten-fold affinity for the insulin receptor, higher resistance to thermal degradation than insulin, native mitogenicity and biological effect. The objective of this work was to transiently express the insulin analog SCI-57 in *N. benthamiana* leaves, purify, and characterize the protein by Reversed-Phase High-Performance Liquid Chromatography (RP-HPLC), mass spectrometry (MS), Diagonal two-dimensional electrophoresis (D-2DE) and enzyme-linked immunosorbent assay (ELISA). Here, we developed a new strategy for the production and purification of SCI-57 in plant leaves, evaluating some of its diabetes-relevant biological effects such as triglyceride accumulation and 2-NBDG uptake in 3T3-L1 adipocytes.

### MATERIALS AND METHODS

### Construction of Expression Vectors

The native version of the protein was obtained from the Protein Data Base (PDBID: 2JZQ). Three molecular strategies were used to increase the expression and accumulation of the insulin analog SCI-57: 1) The addition of a retention signal (KDEL peptide) at the carboxyl-terminus end to direct the recombinant protein to the endoplasmic reticulum, ensuring its correct folding and accumulation (Fischer and Emans, 2000; Cabrera et al., 2003; Conley et al., 2011; Giritch et al., 2015; Lacombe et al., 2018). 2) Ubiquitin fusion at amino-terminus to increase protein expression levels (Hondred et al., 1999; Streatfield, 2007; Benchabane et al., 2008); the ubiquitin at the plasmid construct was located on the 5' end, downstream of the translationally silent *BsaI* site, and 10 bp from the start codon (**Supplementary Figure S1**). 3) Co-expression of the ion channel of the influenza virus (M2) to increase the yield and quality of the recombinant protein (Sainsbury et al., 2013; Jutras et al., 2015); M2 was on the pMDC85 plasmid (PMDC85-M2). For optimal expression in plants, the gene encoding SCI-57 was codon optimized and synthesized (GenScript, USA). The 451 bp fragment corresponding to SCI-57 gene, KDEL peptide, and ubiquitin was digested with *Bsa*I and subcloned in the *Bsa*I site of pICH31070, in order to obtain the pICH31070-SCI-57 expression vector. pICH31070 containing green fluorescent protein (GFP) was used as positive control (pICH31070-GFP).

The vectors were introduced—the 5′ module (pICH15879), the 3′ module (pICH31070-SCI-57, pICH31070-GFP, PMDC85-M2), and the integrase module (pICH14011)—in *Agrobacterium tumefaciens* GV3101 strain by electroporation.

### Agroinfiltration of *N. Benthamiana* Plants

The infiltration procedure was carried out, following our laboratory procedure (Coconi-Linares et al., 2013). In brief, GV3101 cells were incubated at 28°C with constant agitation (180 rpm) in YEB liquid medium with 50 mg/l kanamycin until the OD 600nm = 1.0. Subsequently, cultures were diluted with infiltration buffer (10 mM MES (4-morpholineethanesulfonic acid) pH 5.5, 10 mM MgSO4 and 100 μM acetosyringone) to achieve an OD 600nm = 0.6.

The bacterial suspension containing pICH31070-GFP was mixed with equal volumes of the 5′ module and integrase module suspension. The bacterial suspension containing pICH31070- SCI-57 was mixed with equal volumes of the 5′ module and integrase module suspension, with or without pMDC85-M2 suspension to evaluate the M2 role in SCI-57 expression.

Bacterial suspension mixtures were infiltrated into plant leaves using syringes (**Figure 1A**). After infiltration, the plants were grown at 25°C during a 16-hour light/8-hour dark photoperiod, until GFP expression was observed over most of the leaf surface area (**Figure 1B**). After 4–5 days, the leaves were harvested and macerated using a mortar and pestle, in the presence of liquid nitrogen and stored at −80°C.

### Extraction of Total Soluble Proteins (TSP) and SDS-PAGE

The macerated tissue was mixed and homogenized with the extraction buffer: 100 mM citrate buffer pH 4.3, 1 μl of protease inhibitor cocktail (Sigma-Aldrich, USA), and 10 μl of cellulases per ml (Roche, USA) at a ratio of 1 g leaf tissue to 1.5 ml buffer. Subsequently, this was incubated at 37°C for 1 h and afterwards centrifuged at 10,000 rpm for 15 min at 4°C. The resulting solution was filtered with cellulose acetate membranes of 0.8, 0.45, and 0.22 μm (Millipore Corporation, USA) in order to remove solids and pigments. The NLE was stored at 4°C for subsequent analysis.

We used a low pH extraction buffer to precipitate contaminants such as cell debris and photosynthetic pigments present in green tissues like the leaves of *N. benthamiana*. In addition, this method was also useful for precipitating the protein ribulose-1,5-bisphosphate carboxylase/oxygenase (RuBisCO) which represents approximately 50% of the TSP present in the leaves and usually complicates the generation of highly pure recombinant proteins (Bendandi et al., 2010; Lai et al., 2010; Tuse et al., 2015).

In order to measure the TSP in NLE, we conducted the Bradford analysis. A standard protein curve was constructed using bovine serum albumin (Sigma-Aldrich, USA) from 0 to 1.5 mg/ml, using the extraction buffer as diluent. The absorbance of the standards and samples was measured at 595 nm.

TSP were separated by 13% Tricine–SDS–PAGE under reducing conditions (5% β-mercaptoethanol) and samples were heated for 5 min at 95°C. Electrophoresis was performed for 90 min at 120V. Gels were visualized with Coomassie blue stain (Sigma-Aldrich, USA).

### Extraction of Total RNA and cDNA Preparation

Total RNA was extracted from 100–200 mg of frozen agroinfiltrated leaves, following the manufacturer´s procedure for TRIzol (Invitrogen, USA). The quality of these samples was evaluated by the presence of ribosomal bands on the ethidium bromide-stained agarose gel electrophoresis. Accordingly, total RNA was treated with DNase (Invitrogen, USA). RNA samples (5 μg) were reverse transcribed to generate singlestranded cDNA using an oligo (dT)18 Primer and 200 units of SuperScript III reverse transcriptase, as described by the manufacturer (Invitrogen, USA).

### Gene Analysis by Endpoint PCR and Quantitative Real-Time PCR (qRT-PCR)

Primers were designed using the Primer select software (Graham and Holland, 2005) and validated with the help of the OligoAnalyzer program (Owczarzy et al., 2008) (data available in **Supplementary Table S1**). They were synthesized by the T4 Oligo company (T4 Oligo, México). Endpoint PCR was performed to amplify 18S rRNA, GFP and SCI-57, whereas qRT-PCR assays were performed to amplify 18S rRNA and SCI-57.

End-point PCR samples were amplified by applying the following program: initial denaturation at 98°C for 3 min, followed by 30 cycles of denaturation for 45 s at 94°C, annealing for 30 s at 60.4°C, and elongating for 30 s at 72°C. End-point PCR assays were performed on MultiGene™ OptiMax Thermal Cycler (Labnet International Inc., USA), and the enzyme Taq polymerase was used as described by the manufacturer (Invitrogen, USA) for a reaction volume of 25 μl with 250 ng of cDNA. No template controls (NTCs) were included with each instrument run for quality control, and 18S rRNA was chosen as the housekeeping gene. Samples were analyzed by 2% ethidium bromide-stained agarose gel electrophoresis.

qRT-PCR samples were diluted to a 100 ng/µl single-stranded cDNA concentration with sterile water. The CFX96 real-time PCR detection system (Bio-Rad, USA) was used for all qRT-PCRs. Samples were amplified using SYBR Green with the following program: initial denaturation at 95°C for 3 min, followed by 40 cycles of denaturation for 5 s at 98°C, annealing for 30 s at 60.4°C, and elongating for 30 s at 72°C. Amplification reactions were prepared using a total volume of 10 µl. PCR was followed by a standard melting curve analysis. All PCRs were run in triplicate, and control reactions without template were included in each assay. The 18S rRNA gene was used as a reference for normalization. The data was analyzed using the Bio-Rad CFX Manager 3.1 management software. Relative expression was determined by the evaluation of the expression by the 2−ΔΔCT (Livak and Schmittgen, 2001) method.

### Purification of SCI-57 Cation Exchange Fast Protein Liquid Chromatography (FPLC)

Analysis was carried out with a column of SP-Sepharose Fast Flow (GE Healthcare, Sweden) connected to the FPLC equipment (ÄKTA avant 25, Amersham Biosciences, USA). Recombinant human insulin (PISA pharmaceutical, México) on citrate buffer was used as positive control (1.75 mg/ml). The column was equilibrated with 10 ml of binding buffer (100 mM citrate buffer pH 4.3). Then, 5 ml of NLE or insulin solution was passed through the column. The column was washed with 5 ml of washing buffer (100 mM citrate buffer pH 4.3). The mobile phases were buffer A (100 mM citrate buffer pH 4.3) and buffer B (100 mM citrate buffer pH 4.3 with 1M NaCl). The elution profile to achieve the separation constituted a gradient of 100% B for 60 min with a continuous flow rate of 0.4 ml/min; samples were collected in 1 ml fractions. Elution fractions were monitored by a single path ultraviolet monitor at 280 nm. SCI-57 presence on the fractions was evaluated by 13% Tricine SDS-PAGE.

The fractions containing SCI-57 were filtrated through centrifugal filter devices with a cutoff of 3 kDa (Millipore, USA).

### Reversed-Phase High-Performance Liquid Chromatography (RP-HPLC) Analysis

Samples were analyzed using the HPLC System Agilent 12900 Infinity II (Agilent Technologies, USA). The diode array detector was set to collect signals within the spectral range of 200–400 nm. Chromatographic separation was performed on the chromatographic column (15 cm × 0.21 cm) Vydac 218MS C18 with a 5µ particle size and 300 A° pore size (Vydac, Hesperia, USA). The column was kept at 30°C in a column oven. During chromatographic separation, the mobile phase was in gradient elution from 100% A: H2O 0.1% trifluoroacetic acid (TFA) to 100% B: acetonitrile (ACN) 0.1% TFA over 1 h at a flow rate of 0.5 ml/min. The sample injection volume was 20 µL. Agilent ChemStation software was used for collecting and processing the data.

### Ionization Pattern and Exact Molecular Weight Determination SCI-57

A 30 µM purified SCI-57 sample obtained from A2–A7 fraction concentration and solvent exchange into formic acid 0.1% by 3 kDa filter was analyzed using the SYNAPT-HDMS system (Waters Corp., USA) with ESI-Lockspray interphase. Prior to the experiments, the system was calibrated with a sodium iodide standard solution (Waters Corp., USA), and the European Pharmacopoeia Reference standard insulin (EP insulin) sample was analyzed as a positive control (30 µM).

In the experiment, data was acquired by injecting the protein solution directly into the interphase. We used a scan interval of 50–2,000 m/z with an ionizing spray voltage of 3.2 kV in positive ionization mode and a desolvation temperature of 110°C to acquire data. Molecular weight estimations were generated by the UniDec GUI versión 1.1.10 (Marty et al., 2015).

### SCI-57 M2

The A2–A4 and A5–A7 fractions from SCI-57/M2 purification were concentrated by ultrafiltration with 3 kDa membranes and subsequently lyophilized. The resulting sample was dissolved in 0.1% TFA (16 μM), and the NaCl content was removed using C18 Zip-Tips (Millipore Corp., USA). The EP insulin standard (18 μM) was used as a positive control. Hence, the samples were dissolved in alpha-cyano-4-hydroxycinnamic acid (3.5 mg/ml in 50% acetonitrile, 0.1% TFA) and left to dry at room temperature. Molecular weight analysis was performed in the MALDI TOF/ TOF Analyzer 4800 plus mass spectrophotometer (Applied Biosystems/MDS SCIEX, USA) in linear mode and with a laser intensity of 3,800 Hz.

### Identification of SCI-57 and Proteomic Profile by Gel-Assisted Sample Preparation (GASP), Using Liquid Chromatography-Tandem Mass Spectrometry (LC-MS/MS) and Applying Sequential Windowed Acquisition of All Theoretical Fragment Ion Mass Spectra (SWATH)

We used the FluoroProfile protein quantification kit to determine concentration of samples (Sigma-Aldrich Corporation), with BSA as standard. Equal amounts of protein (50 µg) were prepared in compliance with the GASP protocol, which was published by Fischer and Kessler (2015). The sample was dissolved on GASP buffer (8M urea, 2M thiourea, and 0.1 M DTT). Acrylamide/ bis-acrylamide solution was added to reach a final concentration of 20%. The gel was formed by the TEMED and ammonium persulfate addition. Gel pieces were obtained by centrifugation through a plastic mesh and then fixed overnight by adding ethanol/acetic acid/water (40/10/50). In order to dehydrate the gel pieces, 1 ml of acetonitrile (ACN) was added, and then to rehydrate gel pieces, 0.5 ml 50 mM ammonium carbonate was added and shaken for 10 min. Gel pieces were dried in the speedvac for 2 h at 30°C. Trypsin solution 1:40 in 50 mM ammonium carbonate was added to the gel pieces at 37°C overnight for proteolysis. The addition of ACN and 5% formic acid were subsequently used to facilitate peptide extraction. The supernatant was dried using the speedvac and after the sample was resuspended in 5% formic acid.

From the sample containing the digested proteins, 8 μl were injected into the NanoLC 425 (AB Sciex) for peptide separation. Peptides were then separated on the YMC-Triart C18 1.9 μm, 3.0 × 150 mm column (YMC GmbH, Germany). The flow rate was set to 6 µL/min over 120-min multi-segment gradient: 0 min 98% A-2% B, 100 min 60% A-40% B, 102 min 20% A-80% B, 108 min 20% A-80% B, 110 min 98% A-2% B, 120 min 98% A-2% B (solvent A: 0.1% formic acid; solvent B: acetonitrile with 0.1% formic acid). The data set was generated when the fractionated peptides were transferred to the Q-TOF MS mass analyzer (TripleTOF 5600+, Sciex). The instrument was operated using positive ion with a mass resolution of ~30,000 for TOF MS scan (m/z 400–1,000) and ~15,000 for MS/MS; it was automatically calibrated after every two injections, using RePLiCal (Holman et al., 2016). A TOP20 method was used for the library runs, and variable SWATH-windows (60 windows from 400 to 1,000m/z) were used for SWATH-MS-runs. The accumulation time for IDA was set to 250 ms for an MS1 scan and 50 ms per MS2 scan; the total cycle time was approximately 1.3 s. The accumulation time for SWATH was set to 50 ms for an MS1 scan and 35 ms per MS2 scan; the total cycle time was approximately 1.3 s (Reinders et al., 2016; Feist et al., 2018). The SWATH-library was built using the NCBI database, trypsin as protease, fragment ion mass tolerance of 0.055 Da, maximum one missed cleavage site, oxidation of methionine, pGlu for N-terminal Gln as variable modifications, only doubly and triply charged ions with the Protein Pilot 4.5 software (Sciex GmbH, Germany), and employing a 1% false discovery rate. In Skyline, a scoring model using the second best peaks for each peptide was employed using MProphet (Reiter et al., 2011). Furthermore, peak intensity, retention time difference, retention time difference squared, library intensity dot product, weighted peak shape, weighted co-elution, co-elution count, signal to noise ratio, and product mass error were factored in.

For the comparison in protein expression of the different *N. benthamiana* leaf extract, skyline MSstats was used. *N. benthamiana* leaf extract from non-agroinfiltrated plants (NLEN) were determined as a control group. *N. benthamiana* leaf extract from plants expressing GFP (NLE-GFP), *N. benthamiana* leaf extract from plants expressing SCI-57 (NLE-SCI-57), *N. benthamiana* leaf extract from plants expressing SCI-57, and co-expressing M2 (NLE-SCI-57/M2) were compared to the control group at a confidence level of 95%, resulting in the generation of a file for comparing protein expression levels. Data from Skyline software (3.7) were analyzed using Microsoft Excel 2016.

### Diagonal Electrophoresis for the Detection of Disulfide Bridges

SDS-PAGE Tricine gel was run under non-reducing conditions (Schagger, 2006). The strip of gel containing the protein sample was cut from the gel slab of the first-dimensional electrophoresis. Following treatment with 10 mM DTT + 50% glycerol and 50 mM iodoacetamide, the band was embedded into another polyacrylamide gel slab for the second-dimensional electrophoresis, under reducing conditions. The gel was silver stained according to Blum et al. (1987). The EP insulin (1 µg) was used as positive control.

### SCI-57 Detection by ELISA

SCI-57 concentration was assessed with the help of the Enzyme-Linked Immunosorbent Assay (ELISA) on the strength of the double binding test (Sandwich ELISA), using DRG Iso-Insulin ELISA Kit (No cat. EIA-2336) DRG Instruments GmbH (Germany). All technical procedures, described by the manufacturer, were adhered to.

### 3T3-L1 Adipogenesis

Confluent cultures of 3T3-L1 preadipocytes were induced to adipose differentiation with adipogenic medium (AM; L15 medium added with 10% (v/v) fetal bovine serum (FBS), 0.25 μM dexamethasone, 0.1 mM 3-isobutyl-1-methylxanthine, 1 μM d-biotin, 80 U/ml penicillin, 80 μg/ml streptomycin and 5 μg/ml insulin) (Herrera-Herrera et al., 2009). After three days, the cells were fed with maintenance medium (MM; L15 added with 10% (v/v) FBS, 100 nM insulin, 1 μM d-biotin, 80 U/ml penicillin and 80 μg/ml streptomycin). 3T3-L1 preadipocytes were cultured with non-adipogenic medium to act as negative controls (NAM; L15 added to 10% (v/v) FBS).

### Effect of SCI-57 on 3T3-L1 Adipogenesis

3T3-L1 preadipocytes were incubated with insulin-lacking adipogenic medium (AMI−) in the presence or the absence of 50 µl/ml NLE-SCI-57 or NLE-SCI-57/M2 or NLEN. Three days later, the cells were refed with insulin-lacking maintenance medium added to the respective NLE preparations. As a positive control, 3T3-L1 preadipocytes were incubated with insulincontaining AM and MM. Parallel experiments evaluated the synergic effects of insulin and NLE, by adding NLE preparations to insulin-containing medium.

After seven days in MM, the extent of lipid accumulation was estimated by staining intracellular triglycerides with oil red O (Ramirez-Zacarias et al., 1992).

### Effect of SCI-57 on 2-NBDG Uptake by 3T3-L1 Adipocytes

Cell monolayers of terminally differentiated 3T3-L1 adipocytes cultured on 96-well fluorescence plates were incubated for 60 min with PBS containing BSA 1 mg/ml and 80 µM of the fluorescent glucose analog 2-NBDG (Zapata-Bustos et al., 2014b) in the presence of NLE-SCI-57, NLE-SCI-57/M2, NLEN (50 µl/ml each), or FPLC purified fractions (25 µl/ml). Positive controls received 100 nM insulin or 10 pM oral hypoglycemic rosiglitazone (RGZ). Afterward, cell monolayers were washed with PBS to remove free 2-NBDG, and the fluorescence retained in the cells was measured on a Tecan-GENios (Tecan, Austria) fluorescence reader at an excitation wavelength of 460 nm and emission at 540 nm, using the Magellan 4.0 program. The values of 2-NBDG incorporation in the absence of insulin were subtracted from those obtained with 100 nM insulin to establish 100% specific 2-NBDG incorporation.

### Statistical Analysis

The data from most of the experiments was expressed as the mean ± the standard deviation for each group. The significant differences between groups were evaluated using non-parametric statistics. Kruskal–Wallis and Mann–Whitney U test were performed using SPSS Statistics version 20 (SPSS Inc., Chicago, IL, USA) where a p-value < 0.05 was considered statistically significant. Statistical analysis was applied to results from TSP, quantitative real-time PCR, intracellular triglycerides and 2-NBDG uptake.

### RESULTS

### Expression of Recombinant Proteins in N. Benthamiana.

The 451 bp coding sequence for SCI-57 was cloned into the pICH31070 plasmid. The *Bgl*II restriction analysis confirmed their presence with a resulting plasmid of 5,393 bp in size, as anticipated (**Supplementary Figure S1**).

It was necessary to recombine pro-vector modules within the cell of the plant in order to produce SCI-57. It was thus essential to ascertain that pro-vector assembly was carried out correctly, and that the transient expression of recombinant proteins in *N. benthamiana* was not affected by external factors. GFP was used as a visible tool to confirm the provector assembly *in vivo* and therefore the SCI-57 expression. Four days after agroinfiltration (**Figure 1A**), GFP could be observed on the majority of leaf surfaces (**Figure 1B**). Leaves were thus harvested to obtain NLE. The quantity of TSP on NLE was determined by the Bradford assay. The Bradford assay revealed a TSP concentration which was at least 1.4 times higher in NLE from plants expressing heterologous proteins compared to NLEN. The greatest amount of TSP was observed on NLE-SCI-57/M2, followed by NLE-GFP and NLE-SCI-57. The results were presented as the mean ± S.D. of three independent experimental values for TPS (mg)/biomass (g) (**Figure 1C**).

To confirm the expression of SCI-57 in *Nb* plants, TSP extracts (TSPE) were separated by SDS-PAGE. A protein ~6.4 kDa in size, corresponding to SCI-57 molecular weight, was observed. The band was identified on NLE-SCI-57 and NLE-SCI-57/M2. However, band intensity was greater when M2 was co-expressed (**Figure 2**). The band was not detectable in NLEN or NLE-GFP (**Figure 2**).

### Gene Analysis by Endpoint PCR and Quantitative Real-Time PCR (qRT-PCR)

First, total RNA was extracted from the different *Nb* plants and reverse transcribed with random primers. The cDNA was then subjected to PCR with specific primer pairs: 18S rRNA (149 bp), GFP (158 bp) and SCI-57 (196 bp). The 18S rRNA was chosen as a housekeeping gene. The presence of 18S rRNA was confirmed in agroinfiltrated and non-agroinfiltrated plants, as expected. The presence of GFP was observed on lane 3 when GFP primers were used. This lane corresponds to *Nb* agroinfiltrated with pICH31070-GFP. Whereas SCI-57 was observed on lane 4 when SCI-57 primers were used, a lane which corresponds to *Nb* agroinfiltrated with pICH31070- SCI-57 (**Figure 3A**).

In **Figure 3B**, a slight difference in band intensity between SCI-57 expression in the presence and absence of M2 co-expression is apparent. Our observations indicate that M2 co-expression increases SCI-57 yield.

Relative expression quantitative real-time PCR was employed in order to quantify SCI-57, using cDNA from agroinfiltrated plants, both with and without co-expressed M2. Final relative quantification was carried out using the comparative 2−∆∆CT method. Expression levels were normalized using the 18S rRNA constitutive endogenous gene. The cDNA from agroinfiltrated plants with co-expressed M2 showed approximately two-fold change difference in SCI-57 level of expression compared with agroinfiltrated plants without co-expressed M2 (**Figure 3C**).

FIGURE 2 | TSP analysis from NLEN and NLE from agroinfiltrated plants by SDS-PAGE. Twenty micrograms of TSP were loaded per lane. Mw, Molecular weight marker; lane 1, Insulin 0.01 µg; lane 2, Insulin 0.001µg; lane 3, NLE-SCI-57/M2; lane 4, NLE-SCI-57; lane 5, NLE-GFP; lane 6, NLEN. Side numbers indicate molecular mass markers in kDa. Red Square and black arrow indicates the protein corresponding to SCI-57.

### SCI-57 Purification

Purification of SCI-57 was carried out with a cation exchange column from the NLE-SCI-57 and NLE-SCI-57/M2 samples. The equipment allowed us to monitor the relative intensity at 280 nm of each fraction obtained (**Figures 4A** and **5A**).

High purity of SCI-57 is crucial when administered through direct injection into the blood of the diabetes patient. Correspondingly, SCI-57 purification was accomplished by cation exchange FPLC. The recombinant human insulin as a positive control was purified using ÄKTA avant 25 with prepacked SP-Sepharose Fast Flow. Insulin was eluted, displaying one peak on the chromatogram, from the fraction A4 to fraction A11 (**Supplementary Figure S2**). The aforementioned fractions corresponded to 0.17 M to 0.42 M NaCl. This is consistent with previous publications which establish that, under similar experimental conditions, insulin captured by the column was eluded from 0.10 to 0.45 M NaCl, recovering up to 70% (Jagschies et al., 2018). It was thus probable that SCI-57 would be eluted with a similar concentration of NaCl.

With respect to NLE-SCI-57 purification, two peaks were observed: one from A2–A6 and another from A7–A10, both eluting at the same NaCl concentration range as the insulin peak (**Figure 4A**). SDS-PAGE analysis from fraction A2–A5 (**Figure 4B**) displays one band corresponding to the insulin analog molecular weight ~ 6.4 kDa. SDS-PAGE analysis from the fractions conforming to the second peak (A6–A10) show a band corresponding to SCI-57 molecular weight, as well as several bands that may represent higher molecular weight proteins (**Figure 4C**). The SCI-57 recovery was greater than 60% of the protein captured by the column (**Table 2**). Regarding NLE-SCI-57/M2 purification, two peaks were observed in the chromatograms (**Figure 5A**). The first peak goes from fraction A1–A3 and the second peak from A4–A10. The A1–A10 fraction analysis (**Figures 5B, C**) detected a band corresponding to a protein with the size ~6.4 kDa and the presence of proteins with a molecular weight greater than 14 kDa, which was similar to the proteins observed on A7–A10 NLE-SCI-57 purification. The SCI-57/M2 recovery was 46% of the protein captured by the column (**Table 2**).

After identifying the fractions containing SCI-57, the elution fractions in which the protein is present were combined, and the resultant solution was concentrated twenty-fold in a 3-kDa cutoff filter (Millipore, Bedford, MA, USA). The resultant solution contains pure SCI-57, as revealed by the gel (**Figure 4D**, lane 8).

### Reversed-Phase High-Performance Liquid Chromatography (RP-HPLC) Analysis

Once SCI-57 has been identified in the NLE-SCI-57, samples are evaluated by RP-HPLC. As a positive control, EP insulin, commercial insulin and bovine insulin were used. NLE-SCI-57 (**Figure 6D**) and NLE-SCI-57/M2 (**Figure 6E**) samples exhibit a peak with a retention time (RT) = 12.96 ± 1.30 and RT = 13.02 ± 1.30 min, which is similar to commercial insulin RT = 12.61 ± 1.26 (**Figure 6A**), EP insulin RT = 12.83± 1.28 (**Figure 6B**), and bovine insulin RT = 12.51± 1.25 (**Figure 6C**). The peak was not detectable in extracts of non-agroinfiltrated plants (**Figure 6F**).

After evaluating NLE-SCI-57, a purified SCI-57 sample from A2–A7 fractions was analyzed with the help of HPLC (**Figure 4D**, lane 8) to confirm that the peak observed on NLE-SCI-57 corresponds to the insulin analog SCI-57. The chromatogram (**Figure 6G**) displays a peak with a rt = 13.011, corresponding to the SCI-57 retention time previously observed on non-purified NLE-SCI-57.

### Ionization Pattern and Exact Molecular Weight Determination

An SCI-57 purified protein sample (A2–A7 fractions) was analyzed using a SYNAPT-HDMS system (Waters Corp.) with ESI-Lockspray interphase. Molecular weight estimations were generated by the UniDec GUI tool version 1.1.10 (Marty et al., 2015). Two SCI-57 proteins variants were identified from the sample (**Figure 7**). The protein variant with highest intensity had a molecular weight corresponding to SCI-57 (~6.4 kDa). The second protein variant had a molecular weight of ~ 7.1 kDa. The difference of mass units between the SCI-57 insulin analog and the protein mass identified is likely to refer to the addition of KDEL peptide to the protein sequence. The EP insulin experimental molecular weight was ~5.8 kDa (**Supplementary Figure S4**).

The samples from fractions A2–A4 and A5–A7 NLE-SCI-57/M2 and EP insulin were analyzed using a MALDI TOF/ TOF Analyzer 4800 plus mass spectrophotometer (Applied Biosystems/MDS SCIEX, USA), using a linear mode with a laser intensity of 3,800 Hz. On the A2–A4 purified fractions, the presence of three proteins was identified; the most abundant on the sample had a molecular weight of ~7.1 kDa (**Figure 8A**). The protein was previously identified on the purified protein sample from NLE-SCI-57, probably representing the one corresponding to a molecular weight of SCI-57 + KDEL peptide. On the A5– A7 purified sample, the ~7.1 kDa protein was also identified (**Figure 8B**). The EP insulin molecular weight was ~5.8 kDa (**Supplementary Figure S5**).

### Identification of SCI-57 and Proteomic Profile by Gel-Assisted Sample Preparation (GASP), Using Liquid Chromatography-Tandem Mass Spectrometry (LC-MS/MS) and Applying Sequential Windowed Acquisition of All Theoretical Fragment Ion Mass Spectra (SWATH)

We performed a differential proteomic NLE analysis on plants expressing heterologous proteins (GFP, SCI-57, and SCI-57/

M2) and NLEN to identify the proteomic changes related to heterologous protein expression on the TSP extract. A total of 280 plant proteins were detected with at least two unique peptides. The 280 protein intensities on NLEN sample were used to normalize protein intensities on NLE of plants expressing heterologous proteins in order to determine relative protein expression.

On a TSP basis, 102 proteins were up-regulated, and 82 down-regulated in NLE-GFP, compared to non-agroinfiltrated leaves; similar to NLE-SCI-57, with 89 up-regulated proteins and 102 down-regulated. NLE-SCI-57/M2 manifested 103 up-regulated and 44 down-regulated proteins. The plants expressing heterologous proteins (GFP, SCI-57 and SCI-57/M2) share 48 up-regulated and 23 down-regulated proteins by at least twofold (**Supplementary Tables S2, S3**), work is also underway to address these proteins to the most affected biological processes in leaves. Interestingly, all the plants expressing heterologous proteins showed up to five-fold up regulated pathogenesis-related and stress-inducible proteins (**Table 1**) which are related to agroinfiltration as previously reported (Pruss et al., 2008; Goulet et al., 2010; Robert et al., 2015); for example: protease inhibitors (Kunitz 2 trypsin inhibitor), cell wall modifying enzymes (Expansin) and proteins related to the pathogenesis (PR-) induced by *Agrobacterium* mainly PR-1 proteins (antimicrobial activity), PR-2 (β-glucanases) and PR-3 (chitinases).

Label-free analysis made it possible to identify the peptide FVNQHLCGSDLVEALYLVCERG on samples from plants expressing SCI-57 by measuring the intensity of the seven most intensive peptide fragment ions of the peptide. This peptide is also one of the identified peptides for the peptide map of insulin digestion with trypsin. The peptide was not detected on NLEN (**Supplementary Figure S6**).

### Diagonal Electrophoresis for Detecting Disulfide Bridges

**Figure 9** exhibited diagonal electrophoresis gels stained with silver, evaluating the presence of disulfide bridges in the NLE-SCI-57, NLE-SCI-57/M2, NLEN, and insulin samples. Spots or bands that ran off the diagonal and to the left of the slope indicated proteins containing disulfide bridges. Therefore, NLE-SCI-57 (**Figure 9A**) showed a protein corresponding to SCI-57 molecular weight, running off the diagonal to the left of the slope and indicating the presence of disulfide bridges. The NLE-SCI-57/M2 displayed

a protein corresponding to SCI-57 molecular weight, however, it did not run off the diagonal (**Figure 9C**).

### SCI-57 Detection by Anti-Insulin Antibodies

Quantitative determination of SCI-57 was accomplished by applying the enzyme-linked immunosorbent assay (ELISA) method. The response was measured using the commercially available insulin ELISA kit (DRG Insulin ELISA EIA-2935). The SCI-57 concentration was obtained by equation y = 38.867x − 26.925. The equation was obtained by plotting the absorbance values against insulin concentrations from the calibrator solutions provided in the kit, with a correlation coefficient of 0.9967. Purified SCI-57 obtained from NLE-SCI-57 A2–A7 fractions manifested a concentration of 1.16−6 mg/ml (26.65 mU/L). The sample was measured using the Bradford method with a concentration of 0.086 mg/ml. This concentration corresponds to 19.11% of total soluble protein (%TSP) or 0.1505 mg/g leaves fresh weight (LFW) (**Table 2**).

### SCI-57 Affects 3T3-L1 Adipogenesis

In order to determine whether *Nb* preparations affect the development of adipose tissue, we evaluated their effects on 3T3 adipogenesis. When added to preadipocytes exposed to insulin-lacking adipogenic medium (AMI−), NLE-SCI-57 stimulated the accumulation of lipids in these cells significantly compared to the insulin-lacking control (AMI−), although to a lesser extent than the positive control adipogenic medium (AM). This proadipogenic effect was stronger in NLE-SCI-57/ M2 treated cells and was not observed in cells treated with (NLEN) (**Figure 10**).

We then assayed the effects of NLE-SCI-57 and NLE-SCI-57/ M2 in insulin-containing medium or adipogenic medium (AM) to determine whether the extracts have synergic effect on adipogenesis stimulation (**Figure 10**). Under these conditions, NLE-SCI-57 and NLE-SCI-57/M2 increased the pro-adipogenic effect of insulin. As in the previous assay, NLEN did not show proadipogenic effects.

### SCI-57 Stimulates 2-NBDG Uptake

We evaluated the effect of SCI-57 on 2-NBDG incorporation in terminally differentiated 3T3-L1 cells to establish whether it stimulates glucose uptake by adipocytes (**Figure 11**). NLE-SCI-57 exerted a marked ~15.3-*fold* increase in the 2-*NBDG* uptake in comparison to insulin, whereas NLE-SCI-57/M2 exhibited a 5.3-fold increase. The NLEN stimulated 2-NBDG incorporation, causing a 6.5-fold increase.

A purified NLE-SCI-57 sample (A2–A7 fractions) stimulated 2-NBDG incorporation by a 6.4-fold increase; the sample was previously analyzed by HPLC (**Figure 6G**). The purified fraction A1–A3 from NLE-SCI-57/M2 had a lower 2-NBDG uptake (3.7 fold increase) compared to the A4–A9 fraction (5.1-fold increase).

### DISCUSSION

In this work, we have demonstrated that it is feasible to produce biologically active insulin analog SCI-57 by transient expression in *N. benthamiana* with a yield of 19.11% of TSP (**Table 2**). Furthermore, the number of plants for the production of one gram of the recombinant protein was calculated by considering leaf biomass yield per plant (Lai and Chen, 2012). Then, it will be necessary at least 2293 plants to get a gram of the insulin analog SCI-57, which is comparable to previous reports where the bestperforming plant-based platforms (Merlin et al., 2014).

A ~6.4 kDa band, which was expected to be the size of the insulin analog SCI-57, was detectable in the *N. benthamiana* leaf extract (NLE) from agro-infiltrated plants with SCI-57 constructions (**Figure 2**). Unpurified SCI-57 was readily detectable by SDS-PAGE according to Virgen-Ortiz and colleagues, although only abundant proteins are detected when using this technique (Virgen-Ortiz et al., 2013). Therefore, SCI-57 was expressed at significant levels. Furthermore, M2 co-expression tends to increase SCI-57 levels on leaf extracts. This was corroborated by gene analysis,

as twice as much SCI-57 seemed to be expressed when M2 is co-expressed (**Figure 3C**).

The endoplasmic reticulum (ER) of plant cells is able to accumulate a high concentration of proteins with the addition of a retention signal (KDEL) at the carboxyl terminal (C-terminal) end of the recombinant protein; this retention signal was included in the SCI-57 gene sequence. However, Okomoto et al. (Okomoto et al., 1999) found that, on the second day of infection, the recombinant protein still possessed the KDEL retention sequence. From the third day onwards, a protein mixture without KDEL and with KDEL was present. On day 4, the KDEL sequence was removed from the recombinant protein.

Based on Okomoto's findings and the MS analysis of the sample containing A2–A7 fractions from NLE-SCI-57 purification (**Figure 7**), we presume that the addition of a retention signal (KDEL) generates a mixture of two proteins: SCI-57 and SCI-57+KDEL. This concurs with the findings on the FPLC, where two peaks were observed (**Figure 4A**). Therefore, the band observed on the SDS-PAGE analysis (**Figure 4B**) from the fractions conforming to the first peak (A2-A6) corresponds to SCI-57 ~6.4 kDa, whereas the bands with higher molecular weight observed on A7-A10 may be SCI-57+KDEL ~7.1 kDa, which tends to form molecular aggregates (**Figure 4C**). Thus, SDS-PAGE was not capable of separating proteins with similar weights: SCI-57 and SCI-57+KDEL. Diagonal electrophoresis results (**Figure 9**) indicate that SCI-57 structure had disulfide bonds; hence, transient expression did not affect proper SCI-57 folding or disulfide bond formation.

M2 co-expression was used to enhance the yield and quality of SCI-57. However, the results from molecular weight determination (**Figure 8**) and diagonal electrophoresis suggested the presence of a 7.1 kDa protein with no disulfide bridges (**Figure 9**). Previous publications reported that M2 co-expression increases the pH in the Golgi apparatus (Jutras et al., 2015). Although SCI-57 mRNA was higher when M2 was co-expressed, it was believed that the use of M2 interferes in the binding of the KDEL protein to its receptor, neutralizing the acid pH necessary to carry out the binding. Basic pH inhibits the cleavage of the KDEL peptide (Reithmeier, 1996; Lodish et al., 2008) in the SCI-57 sequence. Thus, when M2 is co-expressed, SCI-57+KDEL production is enhanced, when compared to SCI-57 with no M2 co-expression. Therefore, the two peaks on the FPLC chromatogram may correspond to different degrees of SCI-57+KDEL aggregation. It is likely that on the A1–A4 fractions (first peak on the chromatogram, **Figure 5A**), the monomeric SCI-57+KDEL form was predominant; whereas on the A4–A9 fractions (second peak on the chromatogram, **Figure 5A**), oligomeric SCI-57+KDEL forms can be observed on the SDS-PAGE. Insulin self-assembly and aggregation were induced by our working conditions for purification (**Supplementary Figure S3**); previous publications reported that buffer citrate, chloride ions from NaCl, and the presence of zinc promote insulin aggregation (Carpenter, 1966; Nettleton et al., 2000). High NaCl concentration and zinc presence on the NLE-SCI-57/M2 could stimulate greater oligomeric formation on SCI-57+KDEL; thereby, interfering with the insulin analog SCI-57 recovery on the purification method.

From RP-HPLC, it was established that the band detected on SDS-PAGE gel from NLE-SCI-57 corresponded to a protein with similar retention time and UV spectra as the insulins that we evaluated (**Figure 6**). These findings were corroborated by the analysis of purified A2–A7 fractions (**Figure 6G**). In the case of the NLE-SCI-57/M2 sample, the presence of a peak with a similar retention time is also observed (**Figure 6E**). However, the UV spectrum differs slightly from the NLE-SCI-57 sample. Therefore, M2 protein interferes with the appropriate processing of the SCI-57 insulin analog.

The peptide FVNQHLCGSDLVEALYLVCGER was identified on NLE-SCI-57 (**Supplementary Figure S6**). This reinforces the evidence that SCI-57 is functionally expressed in *N. benthamiana*.

Recombinant proteins expressed in plants usually maintain their native structural properties, facilitating recognition by antibodies. Specific recognition of SCI-57 was observed by two

monoclonal antibodies, which were directed against separate antigenic determinants on the insulin molecule by ELISA. Although the ELISA test did not show a concentration similar to the Bradford method, results confirm that when SCI-57 is expressed without M2 and purified by FPLC, it is functionally active. Furthermore, most commercial insulin assays fail to detect

#### TABLE 1 | Relative abundance of up-regulated proteins1 by at least 5-fold in NLE GFP, NLE SCI-57 and NLE SCI-57/M2 compared with NLEN.


1Proteins were listed base on the n-fold expression normalized with NLEN. Ratios were inferred from MS/MS peptide abundance values determined for each protein. Red numbers indicate the group with the highest value and blue values indicate the group with the lowest value.

#### TABLE 2 | Yield of the SCI-57 expression and purification.


The expression levels are reported as mass of the insulin analog SCI-57 per unit of biomass (LFW, leaves fresh weight.). The recombinant protein productivity values were calculated by considering leaf biomass yield per plant [1 (Lai and Chen, 2012)]. ND, not determined.

recombinant insulin analogs (Heald et al., 2006); therefore, SCI-57 may tend to be underestimated.

As insulin and insulin analogs stimulate the development of adipose tissue (Hartman, 2008), we evaluated the effect of *Nb* preparations on the adipose differentiation of 3T3-L1 cells induced with an insulin-lacking adipogenic medium. Our results indicate that NLE-SCI-57 partially, and NLE-SCI-57/M2 advantageously, replace insulin's role on 3T3 adipogenesis, whereas NLEN did not stimulate lipid accumulation in these cells. When the effect of *Nb* preparations on lipid accumulation was evaluated in the presence of insulin, both NLE-SCI-57 and NLE-SCI-57/M2 samples showed a synergic effect with the hormone stimulating more than twice the lipid accumulation in 3T3-L1 cells. As in the insulin-lacking condition, NLEN sample did not stimulate 3T3 lipid accumulation in the presence of the hormone (**Figure 10**). These results prove that our agroinfiltrated *Nb* preparations possess insulin-mimetic properties.

Hypoglycemic drugs exert their effects by any of the following three mechanisms: by diminishing intestinal glucose, increasing insulin secretion, or stimulating glucose uptake by insulintargeted tissues such as adipose or skeletal muscle tissues. This last hypoglycemic mechanism is the most promising therapeutic target for both Type 1 and Type 2 diabetes.

In order to determine whether SCI-57 has an insulinmimetic effect by stimulating the incorporation of glucose in adipocytes, we evaluated its effects on the uptake of 2-NBDG in 3T3-L1 adipocytes (**Figure 11**). NLE-SCI-57 promoted 2-NBDG incorporation by adipocytes almost at the same level as RGZ. In contrast, NLE-SCI-57/M2 exhibit a lower 2-NBDG incorporation than RGZ. We also evaluated the effect of purified fractions from agroinfiltrated *Nb* preparations on the incorporation of glucose. The purified NLE-SCI-57 fractions (**HPLC**, **Figure 11**) stimulates the incorporation of 2-NBDG to a greater extent than insulin; whereas purified NLE-SCI-57/ M2 fractions (**A1–A3 and A4–A9**, **Figure 11**) are incorporated to a lesser extent than purified NLE-SCI-57 fractions. The high capacity of NLE-SCI-57 to stimulate the uptake of glucose in terminal adipocytes, and its low adipogenic capacity (proobesity), make it an optimal candidate for replacing native insulin. Whereas the NLE-SCI-57/M2 sample, has low (relative) incorporation of 2-NBDG and a marked pro-adipogenic effect (**Figure 10**).

Surprisingly, NLEN stimulated glucose uptake to the same extent as NLE-SCI-57/M2 (**Figure 11**), suggesting the *Nb* extract itself can incorporate glucose in 3T3-L1 adipocytes. Currently, we have no explanation for this effect. Although we and others

FIGURE 10 | Effect of SCI-57 on 3T3 adipogenesis. Adipose differentiation of 3T3-L1 preadipocytes was induced in the presence (AM) or the absence (AMI−) of 5 μg/ml insulin. Control cultures received a non-adipogenic medium (NAM). Effects of NLE preparations on 3T3-L1 adipogenesis were evaluated under both induction conditions by quantitation of intracytoplasmic lipid accumulation with oil red O. Results are presented as the mean ± S.D. of experiments in triplicate. Lowercase letters indicate significant differences according to the Mann-Whitney U test (p ≤ 0.05).

FIGURE 11 | Effect of NLEN, NLE SCI-57, NLE SCI-57/M2 and purified samples on 2-NBDG uptake in 3T3-L1 adipocytes. Mature 3T3-L1 adipocytes were incubated as described (Zapata-Bustos et al., 2014b). Control treatments were incubated with insulin 100 nM (INS) or rosiglitazone 10µM (RGZ). Results are presented as the mean ± S.D. of experiments in quadruplicate. Lowercase letters indicate significant differences according to the Mann-Whitney U test (p ≤ 0.05). HPLC: Purified NLE-SCI-57 sample (A2–A7 fractions) analyzed by HPLC (Figure 6G). A1–A3 and A4–A9: NLE-SCI-57/M2 fractions.

have documented insulino-mimetic effects in diverse plant extracts (Naowaboot et al., 2012; Ortiz-Andrade et al., 2012; Khattak et al., 2013; Eddouks et al., 2014; Zapata-Bustos et al., 2014a), to the best of our knowledge, this property has not yet been evaluated for *Nicotiana* species.

Plants expressing heterologous proteins manifested at least 1.4 greater quantity of TSP than non-agroinfiltrated plants (**Figure 1C**). We conducted a proteomic analysis of the NLE of the different protein expression systems (GFP and SCI-57) using as control a NLEN to estimate the overall impacts on the biochemical pathways and protein synthesis in order to characterize the specific effects of these treatments at the cellwide scale. However higher TSP content could be only related with pathogenesis and stress-related proteins produced as response of agroinfiltration (**Table 1**).

Targeted depletion pathogenesis-related and stress-inducible proteins may improve *N. benthamiana* as a protein expression platform and help identify the proteome changes in *N. benthamiana* when expressing different heterologous proteins with the aim to improve protein yield.

### CONCLUSIONS

For the first time, the results presented here show that *N. benthamiana* plants are capable of producing a biologically active insulin analog; SCI-57. The purification process enables us to extract pure SCI-57, in its active form, from a complex matrix of plant proteins, applying a time effective procedure. From the protein characterization experiments, we conclude that strategies to increase SCI-57 expression and accumulation may interfere with proper folding and the KDEL cleavage, generating the absence on disulfide bond formation and the KDEL peptide presence on the protein sequence. Even though SCI-57 in the lack of M2 co-expression produces a mixture of SCI-57 and SCI-57+KDEL, glycemic control was demonstrated through the 2-NBDG uptake by 3T3-L1 adipocytes, without any apparent pro-adipogenic or anti-adipogenic effects.

However, when SCI-57 is co-expressed with M2, it appears that M2 inhibits the cleavage of KDEL from SCI-57, and tends to encourage the formation of oligomeric forms, although further experiments are required to fully validate these findings.

When M2 is not co-expressed, the insulin analog SCI-57 displayed a lower pro-adipogenic effect and had a higher 2-NBDG uptake compared to when M2 is co-expressed. Overall our observations suggest that SCI-57 exerts its anti-diabetic properties, stimulating glucose uptake, without affecting the development of adipose tissue.

Proteome changes related to the expression of heterologous proteins on *N. benthamiana* were not observed; up-regulated proteins were related to the agroinfiltration process. However, further experiments are required, employing leaves agroinfiltrated with an empty vector.

### AUTHOR CONTRIBUTIONS

AM-T: Conceived of this idea, carried out the experiments, processed the experimental data, performed the analysis, drafted the manuscript, and designed the figures. JR and KL: Label-free proteomic analysis experimental design, assessment of the structural analyses by diagonal electrophoresis. MI-C: Contributed to the interpretation of the results and English revision, final approval of the version for submission. LS-O: Conceived, planned, financed the adipocyte experiments, and provided a critical review of the final manuscript. AE-M: Design and technical execution of MS analysis, interpretation of analytical assays, A critical review of the final manuscript. AL-C: Technical support for molecular cloning, transient expression in *N. benthamiana* and qRT-PCR for gene expression analysis, data acquisition and revision of the manuscript. MG-L: Key role in the study design, editing the manuscript and critical revision for important intellectual content, a critical review of the final manuscript. AR-S: Study design and concept, financial support for part of the study, drafting and editing the manuscript and critical revision for important intellectual content, final approval of the version for

### REFERENCES


submission. All authors discussed the results and commented on the manuscript.

### FUNDING

This work was supported in part by PROINPEP 2015, 2016 (AR-S), awarded to support Ph.D. students of the pharmacology program PROSNI 2016, 2017 to AR-S, and by own resources of the Doctors MG-L, LS-O, JR, and AR-S.

### ACKNOWLEDGMENTS

AM-T thanks CONACYT for national and international scholarship No. 398200. AM-T gratefully acknowledges the multidisciplinary Department of Genetic Engineering laboratory team for their support of this project. The authors would like to Francisco Pérez (Centro de Biociencias, UASLP, México) for his technical assistance in MS experiments.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01335/ full#supplementary-material


transcript profiling," in *Methods in enzymology.* (Elsevier BV: Academic Press), 544–553.


protein samples by rapid freeze centrifugation. *Anal. Biochem.* 443 (2), 249– 251. doi: 10.1016/j.ab.2013.09.015


**Conflict of Interest:** The authors declare that this research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Muñoz-Talavera, Gómez-Lim, Salazar-Olivo, Reinders, Lim, Escobedo-Moratilla, López-Calleja, Islas-Carbajal and Rincón-Sánchez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Efficacy and Safety in the Continued Treatment With a Biosimilar Drug in Patients Receiving Infliximab: A Systematic Review in the Context of Decision-Making From a Latin-American Country

#### *Edited by:*

*Marcus Tolentino Silva, Federal University of Amazonas, Brazil*

#### *Reviewed by:*

*Hillel Cohen, Sandoz Inc., United States Alexandros A. Drosos, University of Ioannina, Greece*

#### *\*Correspondence:*

*Edward Mezones-Holguin emezones@gmail.com; emezones@usil.edu.pe*

#### *Specialty section:*

*This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology*

*Received: 16 February 2019 Accepted: 08 August 2019 Published: 15 November 2019*

#### *Citation:*

*Mezones-Holguin E, Gamboa-Cardenas RV, Sanchez-Felix G, Chávez-Corrales J, Helguero-Santin LM, Laban Seminario LM, Burela-Prado PA, Castro-Reyes MM and Fiestas F (2019) Efficacy and Safety in the Continued Treatment With a Biosimilar Drug in Patients Receiving Infliximab: A Systematic Review in the Context of Decision-Making From a Latin-American Country. Front. Pharmacol. 10:1010. doi: 10.3389/fphar.2019.01010*

*Edward Mezones-Holguin1\*, Rocio Violeta Gamboa-Cardenas2, Gadwyn Sanchez-Felix3, José Chávez-Corrales4, Luis Miguel Helguero-Santin5, Luis Max Laban Seminario5, Paula Alejandra Burela-Prado6, Maribel Marilu Castro-Reyes6 and Fabian Fiestas6*

*1 Universidad San Ignacio de Loyola (USIL), Centro de Excelencia en Estudios Económicos y Sociales en Salud, Lima, Peru, 2 Seguro Social en Salud (EsSalud), Hospital Nacional Guillermo Almenara Irigoyen, Servicio de Reumatologia, Lima, Peru, 3 Seguro Social en Salud (EsSalud), Hospital Nacional Edgardo Rebagliati Martins, Servicio de Dermatología, Lima, Peru, 4 Seguro Social en Salud (EsSalud), Hospital Nacional Edgardo Rebagliati Martins, Servicio de Reumatologia, Lima, Peru, 5 Universidad Nacional de Piura (UNP), Facultad de Ciencias de la Salud, Sociedad Científica de Estudiantes de Medicina (SOCIEMUNP), Piura, Peru, 6 Seguro Social en Salud (EsSalud), Instituto de Evaluación de Tecnologías Sanitarias e Investigación (IETSI), Lima, Peru*

Introduction: Biological products, including infliximab (INF), are a therapeutic option for various medical conditions. In the Peruvian Social Security (EsSalud), infliximab is approved for the treatment of rheumatoid arthritis, psoriasis, psoriatic arthropathy, ankylosing spondylitis, ulcerative colitis and Crohn's disease (in cases refractory to conventional treatment). Biosimilars are a safe and effective alternative approved for these diseases in patients who start treatment with infliximab. Nevertheless, there are people in treatment with the biological reference product (BRP), in whom the continuing therapy with a biosimilar biological product (BBP) must be evaluated.

Objectives: To synthesize the best available evidence, calculate a preliminary financial impact and conduct technical discussions about the interchangeability into biosimilar in patients receiving treatment with original infliximab for medical conditions approved in EsSalud.

Methodology: We carried out a systematic review of controlled clinical trials. Primary search was performed in Pubmed- MEDLINE, SCOPUS, WOS, EMBASE, TRIPDATABASE, DARE, Cochrane Library, NICE, AHRQ, SMC, McMaster-PLUS, CADTH, and HSE until June-2018. We used the Cochrane Collaboration tool to assess the risk of bias. Also, we implemented a preliminary financial analysis about the impact of biosimilar introduction on institutional purchasing budget. Moreover, technical meetings with medical doctors specialized in rheumatology, gastroenterology and dermatology were held for discussing findings.

Results: In primary search, 1136 records were identified, and 357 duplicates were removed. From 799 records, we excluded 765 after title and abstract evaluation. From 14 full-text appraised documents, we included five clinical trials in the risk of bias assessment: four studies evaluated CTP-13 and one tested SB2. Two double-blind clinical trials reported no differences in efficacy and safety profiles between maintenance group (INF/ INF) and interchangeability group in all diseases included (INF/CTP-13) and rheumatoid arthritis (CTP13 and SB2). In the other three studies, open-label extension of primary clinical trials, no differences were founded in efficacy and safety profiles between CTP-13/ CTP-13 and INF/CTP-13 groups. In financial analysis, the inclusion of biosimilars implied savings around S/7´642,780.00 (1USD=S/3.30) on purchasing budget of EsSalud. In technical meetings, beyond certain concerns, specialists agreed with the findings.

Conclusions: Evidence from clinical trials support that there are no differences in efficacy or safety of continuing the treatment with Infliximab BRP or exchanging into its biosimilar in patients with medical conditions approved in EsSalud. Financial analysis shows that the biosimilar introduction produce savings in purchasing institutional budget. Therefore, based on cost-opportunity principle, exchanging into biosimilar in patients receiving the original Infliximab, is a valid therapeutic alternative in the Peruvian Social Security.

Keywords: Infliximab, biosimilar, interchangeability, decision making, Latin-American

### INTRODUCTION

Biological products are therapeutic options for different diseases. These drugs are molecules with a complex structure, large and often highly specific and are derived from living organisms (Pombo et al., 2009; Wang and Singh, 2013; Auclair, 2019). Biologic drugs are used to treat various diseases, including conditions that involve the immune system, randomized studies have shown their efficacy for reducing symptoms and improving the quality of life in people undergoing treatment (Wang and Singh, 2013; Zelikin et al., 2016). However, a significant number of patients do not respond, have an inadequate response to initial treatment (primary failure), lose response over time (secondary failure), or may develop adverse effects potentially limiting the therapy (Auclair, 2019). One of these drugs is infliximab (REMICADE®), a tumor necrosis factor alpha inhibitor (TNFa) (Ecker et al., 2015). Infliximab has been approved for the treatment of rheumatoid arthritis (RA), severe psoriasis, ankylosing spondylitis, Crohn's disease, and ulcerative colitis, among other diseases.(Acevedo and Gaitan, 2012; European Medicines Agency (EMA), 2018; Food and Drug Administration, 2018). Efficacy, effectiveness, and safety of this biological drug has been tested in different studies (Li et al., 2017). Hence, infliximab is currently included in the pharmacological petition of the Peruvian Social Security (EsSalud). (Seguro Social en Salud (EsSalud), 2017).

On the other hand, the biosimilar biological products (BBP) are an efficient treatment alternative to the biological reference products (BRP). They usually offering similar effects and lower cost (Declerck et al., 2017; Gutka et al., 2018). BBP contains the active component of BRP with similar characteristics in its pharmacological activity, efficacy and safety (Gamez-Belmonte et al., 2018; Gutka et al., 2018). The equivalence of BBP has been reported from comparison – in equal terms - with BRP in various randomized clinical trials (Portela et al., 2017; Uhlig and Goll, 2017), where infliximab is one the most studied drugs (Gutka et al., 2018). Based on this information, international guidelines for its regulation have been spread and adopted in several countries (Garcia and Araujo, 2016; Sheets, 2017; Tsai, 2017; Zahl, 2017). The Peruvian health system is fragmented, segmented and inequitable (Sánchez-Moreno, 2014), where around 25% of population are affiliated to EsSalud (Mezones-Holguin et al., 2019). The General Directorate of Medicines, Supplies and Drugs (DIGEMID, from Spanish Acronym) as the national health authority, approved the commercialization of some infliximab biosimilars in Peru (Ministerio de Salud, Dirección General de Medicamentos, Insumos y Drogas (DIGEMID), 2016). Therefore, certain public institutions, supported by the Peruvian Government contracting laws, including the Social Security, have purchased BBP. Currently, in EsSalud there are two kinds of patients: those who will start treatment with Infliximab and those who continue their therapy with Infliximab. In the first group, the use of biosimilar is accepted as valid; however, in the other group, there are certain concerns with respect to the continuation with BBP.

Based on the context described, a decision should be made regarding the continuation with a biosimilar in patients undergoing treatment with original infliximab in EsSalud. Although, there are several definitions on interchangeability, in our manuscript it means a transition from using BRP to BBP (Gutka et al., 2018; Trifirò et al., 2018). At the moment, there is an interesting debate about interchangeability with active participation of distinct actors from different health care systems; Mezones-Holguin et al. Biosimilar Interchangeability of Infliximab

thus, international regulations have been proposed to the use of BBP and the transition from its BRP (Portela et al., 2017; Tsai, 2017; Cohen et al., 2018; Niazi, 2018). Nevertheless, in Peru, and specifically in EsSalud, there is no explicit decree for it. Therefore, the Institute for Health Technology Assessment and Research (IETSI, from Spanish acronym) - as technical entity in EsSalud must evaluate the best available scientific evidence to inform decision-making in the Peruvian Social Security.

In light of the above mentioned, the aim of our study was to synthesize the best available evidence, calculate a preliminary financial impact, and conduct a technical discussion concerning the interchangeability into biosimilar in patients undergoing treatment with original infliximab for medical conditions approved in EsSalud. Although there are systematic reviews published (Chingcuanco et al., 2016; Cohen et al., 2018; McKinnon et al., 2018; Feagan et al., 2019), our study incorporates two key elements used in the decision-making process for health systems with limited resources: institutional budget and clinical experience. Consequently, our article is a description of this complexity in Peru and shows the use of the best scientific evidence in the real world.

### METHODS

In our manuscript, we describe the three main activities performed in order to inform the decision-making process in EsSalud regarding infliximab interchangeability:


### SYSTEMATIC REVIEW

### Clinical Question (PICOS)

The population(P) was circumscribed to adults with rheumatoid arthritis, psoriasis, ulcerative colitis, Crohn's disease and ankylosing spondylitis undergoing treatment with the original Infliximab. Intervention(I) was to exchange into a biosimilar, and comparison(C) was the continuation with original Infliximab. The outcomes(O) were efficacy and safety. In accordance with current legal regulations in EsSalud, we included only controlled clinical studies(S) in biosimilar drugs approved by DIGEMID for their commercialization in Peru (CTP-13 and SB2).

### Search Strategy and Selection of Study

We conducted a search without language restrictions until June 2018. Primary strategy formulation included controlled and free terms according to PICOS question. Studies were restricted to clinical trials in humans of any age, gender or nationality. We searched in: PubMed-MEDLINE, SCOPUS, Web of Science (WOS), Excerpta Medica (EMBASE), Translating Research into Practice (TRIPDATABASE), Database of Abstracts of Reviews of Effects (DARE), Cochrane Central Register of Controlled Trials (CENTRAL), National Institute for Health and Care Excellence (NICE), The Agency for Healthcare Research and Quality (AHRQ), The Scottish Medicines Consortium (SMC), McMaster PLUS, The Canadian Agency for Drugs and Technologies in Health (CADTH), and The Health Systems Evidence (HSE). Primary search strategies for each database are explicitly presented as annexes (A-N) (**Supplementary Table 1**). Additionally, we reviewed the list of references. Poster and oral presentations in scientific meetings were not considered.

### Article Selection

Records found were collected in an electronic folder using Mendeley® (Elsevier Inc, NY, USA) and we generated a Research Information Systems (RIS) file. Duplicates were removed by automatic and manual methods; then, we exported a new file to Rayyan® (Qatar Computer Research Institute, Doha, Qatar). Two authors (LHS and LLS) completed a blind and independent selection based on abstract and title, third author (EMH) had diriment decision. Then, two authors (LH-S and LL-S) selected articles in full-text evaluation with third author as diriment (EM-H). Afterward, two evaluators (LHS and LLS) codified the articles and uploaded them in Google Drive® folder (Google Inc, CA, USA).

### Risk of Bias Assessment

Two authors (LHS and LLS) acted upon blind and independent appraisal of clinical trials using the Cochrane Collaboration tool(Higgins et al., 2011). Disagreements were resolved by consensus and diriment participation (EMH).

### Statistical Synthesis

Although a meta-analysis was initially proposed, it was not performed due to clinical and methodological heterogeneity.

### Preliminary Financial Analysis

We implemented an analysis about the impact of biosimilar introduction in the institutional purchasing budget based on the official reports of EsSalud and Electronic Government Procurement System of Peru (SEACE, from Spanish Acronym).

### Technical Meeting

We held several face-to-face meetings to present and discuss the results of the systematic review and financial analysis. A group of rheumatologists, dermatologists and gastroenterologists working in hospitals of EsSalud in Lima, participated in these reunions.

## RESULTS

### Selection and Characteristics of Studies

We identified a total of 1136 records in the primary search, from which we removed 357 duplicates. From 799 screened records, we excluded 765 in title and abstract evaluation. Then, we appraised 14 full-text documents, and included five clinical trials for risk of bias assessment and data extraction (**Figure 1**).

We found five controlled studies, that corresponded to five publications, which evaluated the interchangeability between Infliximab and its PBB. Only one assessed SB2 biosimilar (Smolen et al., 2018), and the four remaining studies evaluated biosimilar CTP-13. Two publications were double-blind Randomized controlled studies (RCT), while the remaining three were openlabel continuation of clinical trials that initially compared the PBR with PBB. Three studies focused specifically on patients with rheumatoid arthritis, one in ankylosing spondylitis and another, in addition to these two diseases, included Crohn's disease, ulcerative colitis, psoriatic arthritis and chronic plaque psoriasis. In **Table 1** we present the general characteristics of trials included.

Only two articles respond directly to the PICO question, since they evaluated the exchange of the original Infliximab to the biosimilar compared to the maintenance of the original biotherapy: Smolen et al. (2018) and Jørgensen et al. (2017), who tested SB2 and CTP-13, respectively. In both cases, they did not find statistical differences in efficacy or safety between maintenance and exchanging groups.

The other three publications did not respond directly to PICOS question. These studies did not contain primary safety or efficacy data in a blind setting. Instead, they provided complementary

#### TABLE 1 | Characteristics of primary studies included in the analysis.


*\* The number of patients corresponds to exchanging started time.*

*\*\* Randomization was applied in patients who already had treatment with the original infliximab drug for a minimum of 6 months.*

*\*\*\* The initial phase of treatment ended at 54 weeks. The first dose of the second stage started eight weeks later in week 62.*

information with the purpose of expanding the perspective of clinical use in potential EsSalud scenarios. Those publications reported the evaluation of open-label continuation of primary clinical trials: Tanaka et al. (2017), Yoo et al. (2017), and Park

et al. (2017). In these publications no differences were found in efficacy or safety between patients who switched from original infliximab to biosimilar (INF/CTP-13), and maintained biosimilar treatment (CTP-13/CTP-13).

### Risk of Bias

In **Table 2**, we show the appraisal for each study included. Trials with direct response to PICOS question had lower risk of bias, mainly due to randomization and blinding.

### Description of Evidence

We briefly described efficacy and safety outcomes for each study: one for SB2 and four for CTP-13. We describe efficacy and safety outcomes.

### Biosimilar SB2

Smolen et al. (2018) "**Safety, Immunogenicity And Efficacy After Switching From Reference Infliximab To Biosimilar SB2 Compared With Continuing Reference Infliximab And SB2 In Patients With Rheumatoid Arthritis: Results Of A Randomized, Double-Blind, Phase III Transition Study.**" **Annals Of The Rheumatic Diseases; 7:234-40.**

A randomized, double-blind, phase 3 clinical study was carried out in people with rheumatoid arthritis. This study had two initial groups. Patients were randomized into two groups for 52 weeks: 293 were treated with Infliximab (INF) and 292 received biosimilar (SB2). Then, a new randomization was performed, INF group was divided into a maintenance group (INF/INF n=101) or exchanging group (INF/SB2 n=94). Meanwhile, the group initially assigned to SB2 continued with biosimilar (SB2/ SB2 n=201). Efficacy, safety and immunogenicity profiles were not different among the groups up to week 78.

### Efficacy

The major findings of this study are presented in **Table 3A**. We describe the findings according the clinical scale used.

### American College Of Rheumatology (ACR20, ACR50 And ACR70)

Authors found that the percentage of patients who showed a 20% improvement (ACR20) at week 78 of follow-up was not statistically different between the three groups: INF/INF (68.8%), SB2/SB2 (65.7%) and INF/SB2 (63.5%) (p-value:0.7316). Also, there was no statistically significant difference between groups in the proportion of patients with 50% improvement (ACR50)

(p-value:0.3249). Moreover, in 70% improvement (ACR70), no significant differences were found in (p-value: 0.3071): INF/INF group (31.2%), SB2/SB2 (25.6%) and INF/SB2 (22.4%).

### European League Against Rheumatology Score (EULAR)

EULAR response criteria scores were measured at week 78, no statistically significant differences were observed. Good or moderate responses were 84.9% in INF/INF group, 87.3% in SB2/SB2 group and 84.7% in INF/SB2 group (p-value: 0.8074). Regarding the proportion of patients with good response, there was no significant difference between groups: INF/INF (34.4%), SB2/SB2 (35.6%) and INF/SB2 (32.9%) (p= 0.8740).

### Diseases Activity Score 28 (DAS28), Simple Disease Activity Index (SDAI) And Clinical Diseases Activity Index (CDAI)

These three instruments were used to measure the activity of the disease and there were no significant statistical differences between randomized groups. DAS28 values were (mean±sd): INF/INF (4,1±1,5), SB2/SB2 (4,0±1,4), y INF/SB2 (3,9±1,3). SDAI score in each group were: INF/INF (15,2±12,0), SB2/SB2 (initial 14,6±12,2), and INF/SB2 (13,2±10,0). Regarding CDAI, patients obtained similar scores: INF/INF (15,2±12,0), SB2/SB2 (initial 14,6±12,2), and INF/SB2 (13,2±10,0). No point values were reported at the end of the follow-up at week 78; authors showed graphically the evolution of the scores during the post interchange period, there is no difference between the three groups evaluated (**Table 3A**).

## Safety

### Adverse Events

No differences were observed in the frequency of adverse events (AE) among the three post-exchange groups: Specifically, for any AE were: INF/INF (35,6%), SB2/SB2 (40,3%), and INF/SB2 (36,2%) of patients presenting any AE (p=0.546). Regarding serious AE post-exchange, frequencies were: 6.4% in INF/SB2, 3% in INF/INF and 3.5% in SB2/SB2 (p=0.456). Similarly, no differences were found in the frequency of discontinuation due to AE (p=0.625) (**Table 3E**).

TABLE 2 | Risk of bias assessment in each study according Cochrane Collaboration Tool.


*\* These articles respond directly PICOs question.*

#### TABLE 3A | Efficacy outcomes in patients with rheumatoid arthritis.


*ACR20, ACR50 y ACR70: Improvement in 20%, 50% and 70% according to the American College of Rheumatology criteria.*

*DAS28, Score of activity of the disease in 28 joints with reactive protein C (PCR).*

*EULAR, European League against Rheumatism.*

*\*Jorgensen et al. study included patients with rheumatoid arthritis, however, the random assignment and the sample calculation were for all pathologies. Because it was a subgroup analysis, no results were reported for RA in this table. Baseline corresponds to time of exchanging.*

*\*\*No differences were founded between the DAS28 indices for each group, no point values were reported at the end of follow-up. The article did not report any differences using graphic methods.*

*\*\*\*Comparison for moderate or good classification.*

#### Immunogenicity

Post-exchange immunogenicity levels were very similar among groups: INF/INF (14.9%), SB2/SB2 (14.1%) and INF/SB2 (14.6%) (p=0.98) (p=0.98) (**Table 3E**).

#### Biosimilar CTP-13

Jørgensen et al., 2017. **"Switching from Originator Infliximab to Biosimilar CT-P13 Compared with Maintained Treatment with Originator Infliximab (NOR-SWITCH): A 52-Week, Randomized, Double-Blind, Non-Inferiority Trial." Lancet 389 (10086): 2304–16** (Jørgensen et al., 2017).

The authors conducted a phase 4, randomized double-blind non-inferiority trial. Patients with Crohn's disease, ulcerative colitis, rheumatoid arthritis, spondylarthrosis, psoriatic arthritis and chronic plaque psoriasis receiving original infliximab were enrolled and randomized in two arms:: maintenance group with

the original biological component (INF/INF), and exchanging group from the original biological into its biosimilar (INF/ CT-P13). The exchange group showed non-inferiority to the ongoing treatment with INF on efficacy and safety for all diseases investigated. However, there was not enough statistical power to demonstrate the non-inferiority for each disease studied. This research was financed by the Norwegian Government.

### Efficacy

Different measurements were used according to clinical population studied. Authors define two main types of variables: a) *categorical (state):* percentage of patients with a specified condition (deterioration or remission) based on clinical scales, and b) *numerical (change)*: any variation in the score of clinical scales at the end of the follow-up with respect to the baseline (exchange time).



ASAS, *The Assessment of Spondylarthritis International Society;* PR, *Partial Remission;* BASDAI, *The Bath Ankylosing Spondylitis Disease Activity Index;*  BASFI, *Bath Ankylosing Spondylitis Functional Index;* ASDAS, *Ankylosing Spondylitis Disease Activity Score;* BASMI, *Bath Ankylosing Spondylitis Metrology Index;* 

ASDAS, *Ankylosing Spondylitis Disease Activity Score.* 

*\* Standard deviation was not reported. Authors only compared graphically.*

#### Deterioriation During Follow-Up

This was the primary outcome for all patients based on specific clinical scales for each of the six diseases studied. In the ITT analysis, frequency of deterioration in all diseases were 22.4% in the INF/INF group, and a 26.3% in the INF/CTP-13 group (p=0.3259). Although the frequency of decline of the six diseases was defined, there was not enough statistical power to test noninferiority of each disease; thus, we only report the frequencies for exploratory purposes (**Table 3C**).

#### Remission During Follow-Up

Approximately 60% of patients in each group achieved remission (p=0.8810) (**Table 3C**). There was not enough statistical power to evaluate the non-inferiority for each disease.

#### Quality of Life: SF36 and EQ5D

Health-related quality of life (QoL) for all diseases were assessed using SF36 and EQ5D; two validated and widely used instruments. In the first group, statistically significant differences during the follow-up period regarding physical limitations (p=0.0069) and emotional limitations were found (p=0.026); with a greater average of deterioration (decrease in score) in the maintenance group (-0.4) and exchange group (-1.1). There were not statistical differences in the others components. Meanwhile, there were different changes on the clinical global impressions scale of EQ5D in both groups (p=0.999) (**Table 3D**).

### Safety

There were no statistically significant differences between patients of two groups in safety variables.

#### Adverse Events

Frequencies of serious AE were 10% in maintenance patients and 9% in exchanging group. Discontinuation due to AE was 4% and 3%, respectively.

#### Immunogenecity

Frequency of patients with post transition ADA were: 7% (INF/ INF) and 8% (INF/CTP-13) (**Table 3E**).

Tanaka et al., 2017. **"Safety and Efficacy of CT-P13 in Japanese Patients with Rheumatoid Arthritis in an Extension Phase or after Switching from Infliximab." Modern Rheumatology 27 (2): 237–45** (Tanaka et al. 2017).

This open label study, RA patients were randomized in two arms: INF/CTP-13 and CTP-13/CTP-13. There were no statistical differences in efficacy and safety assessed by clinical scales.

### Efficacy

#### ACR20, ACR50 and ACR70

No differences were found in frequency of patients who improved in the three categories proposed by the American College of Rheumatology: ACR20%, ACR50% and ACR70%. In CTP-13/ CTP-13 (78.4%, 70.3% and 54.1%) and INF/CTP-13 (62.5%, 53.1% and 40.6%), respectively (**Table 3A**).

TABLE 3C | Frequency of deterioration and remission during follow-up in patients with Crohn's disease, ulcerative colitis, spondylarthritis, rheumatoid arthritis, psoriatic arthritis, and chronic plaque psoriasis.


*NE, Not estimated due to the low statistical power.*

TABLE 3D | Quality of life in patients with Crohn's disease, ulcerative colitis, spondylarthritis, rheumatoid arthritis, psoriatic arthritis, and chronic plaque psoriasis (SF36 and EQ5D).


*SF-36, 36-Item Short Form Health Survey; FF, Physical functioning; LRF, Limitation of physical roles; SG, General Health; BE, Emotional wellbeing; LRE, Limitation of emotional roles; FS, Social Functioning; EF, Energy or Fatigue; RCF, Physical component summary; RCM, Mental component summary; Baseline, exchanging time; End, Conclusion of follow-up.* 

#### DAS28

There were also no differences in the average scores at the end of the follow-up between maintenance group (2.78) and exchange group (2.03) (p=0.612) (**Table 3A**).

#### EULAR

Frequency of good or moderate response after the follow-up period did not show significant statistical difference between the two groups: 83% in maintenance patients and 68.8% in exchanging people (**Table 3A**).

### Safety

#### Adverse Events

In maintenance group, 5.3% of patients had serious AE and 10.5% discontinuing the prescription due to AE. Meanwhile, in CTP-13 exchanging group participants had 12.1% of serious AE and 24.2% discontinued the treatment due to AE. These differences were not statistically significant (**Table 3E**).

#### Immunogenicity

In post-transition stage, frequency of patients with ADA were 10.6% in maintenance group and 12.1% in exchanging group. (p=0.901) (**Table 3E**).

**"Efficacy and Safety of CT-P13 (Biosimilar Infliximab) in Patients with Rheumatoid Arthritis: Comparison between Switching from Reference Infliximab to CT-P13 and Continuing CT-P13 in the PLANETRA Extension Study." Annals of the Rheumatic Diseases 76 (2): 355–63** (Yoo et al., 2017).

Authors compared two groups of Rheumatoid arthritis patients: maintenance (CTP-13/CTP-13) and exchanging (INF/ CTP13). Efficacy, tolerability and safety observed were non different between groups.

### ACR20, ACR50 and ACR70

Frequencies of 20%, 50% and 70% responses according to the ACR criteria were: 74.1%, 49.4% and 24.7% in CTP-13/CTP-13, and 77.1%, 54.2% and 26.4% in INF/CTP-13 group. There was no evidence of statistically significant differences between groups (**Table 3A**).

### DAS28

The average final scores were not statistically different between maintenance (2.40) and exchanging (2.48) groups (**Table 3A**).

### EULAR

Frequency of patients with a good or moderate criterion according to EULAR were 80.9% and 81% in maintenance and exchanging arms, respectively. There were no statistical differences between groups (p=0.669) (**Table 3A**).

### Safety

#### Adverse Events

In maintenance group, 7.5% of patients had serious AE, and 10% discontinued their treatment due to AE. These frequencies did not differ than exchanging group (9% and 5.6%, respectively) (**Table 3E**).

Park et al., 2017. **"Efficacy and Safety of Switching from Reference Infliximab to CT-P13 Compared with Maintenance of CT-P13 in Ankylosing Spondylitis: 102-Week Data from the PLANETAS Extension Study" Annals of the Rheumatic Diseases 76 (2): 346–54** (Park et al., 2017).

This trial was carried out in patients with ankylosing spondylitis. Participants were randomized in two groups: maintenance (CTP-13/CTP-13) and exchanging (INF/CTP-13). No statistically significant differences were observed between group in terms of efficacy or safety.

### Efficacy

### Assessment of Spondylarthritis international Society (ASAS20, ASAS40 and ASAS PR)

Non statistical differences were found between groups according ASAS measurements for 20%, 40% and partial remission


TABLE 3E | Safety outcomes in all primary studies included.

*ADA, Anti-drug antibody; NR, Not reported; EA, Adverse Events. Primary outcome was cumulative incidence of AE.*

of disease: in maintenance (80.7%, 63.9% and 19.3%) and exchanging (76.9%, 61.5% and 23.1%) patients (**Table 3B**).

### Bath Ankylosing Spondylitis Disease Activity Index (BASDAI), Bath Ankylosing Spondylitis Functional Index (BASFI), Ankylosing Spondylitis Disease Activity Score (ASDAS) and Bath Ankylosing Spondylitis Metrology Index (BASMI)

Authors reported - using graphical methods - non differences in average at the end of follow-up between maintenance and exchanging groups: BASDAI (3.19 vs. 3.23), BASFI (3.1 vs. 3.25), ASDAS (1.86 vs. 1.97), and BASMI (2.4 vs. 2.6) (**Table 3B**).

### Safety

### Adverse Events

In maintenance group, 4.5% and 3.3% of allocated patients had serious AE and discontinued treatment due to AE, respectively. Meanwhile, in exchanging group frequencies were 4.6% in both measures. There are no evidence of statistical differences between arms (**Table 3E**).

### Immunogenicity

In post-exchange period, there were non statistical differences in proportion of patients with ADAs between groups. Authors reported 23.3% in CTP-13/CTP-13 and 27.4% in INF/CTP-13 groups (**Table 3E**).

### Preliminary Financial Analysis

First, we present the estimate of annual costs per patient based on price for each vial offered by each provider, S/2040.00 (S/: Peruvian soles) for BRP and S/857 for BBP; and number of vials required per patient (annual average). This implies annual savings around S/24843 per-patient with biosimilar. Secondly, we estimated the cost differences based on annual requirement of Infliximab from EsSalud (6460 vials); thus, the biosimilar introduction could produce savings around S/7´642,780.00 (1 USD: S/3.30) (**Table 4**).

TABLE 4 | Preliminary financial analysis about the cost related to treatment with infliximab an its biosimilar in EsSalud (1USD = S/3.30).


*\*Based on what was sold by the suppliers in the last purchase of the biological product registered in the SEACE platform for a vial of infliximab of 100 mg. \*\*Estimated for 60 kg person on average at a dose of 5mg / kg.*

*\*\*\*The application is every 8 weeks on average; the annual estimate has been rounded.*

*\*\*\*\*The total requirement corresponds to what was requested by EsSalud for the year 2018.*

### Technical Discussion With Medical Doctors

In first meeting, we received questions and feedback from rheumatologists, dermatologists and gastroenterologists. In second meeting, we discussed those questions an related legal aspects, and we also defined scope and limitations of analysis performed. At the last meeting, we presented and received the approbation of final technical document. The main concerns expressed by doctors were not being able to conclude for each disease separately, nocebo effect and using of generic questionnaires to assess the quality of life. We address them in the discussion.

### DISCUSSION

Our findings reflect the best primary evidence available related to continuation with a biological biosimilar drug in patients that receive Infliximab -as biological reference drug - in conditions approved by the Peruvian Social Security. While only two of the studies respond directly to PICOS question using Infliximab as original maintenance drug, we included all controlled trials that evaluated interchangeability from the original Infliximab into its biosimilar. All primary studies did not find statistical and clinical differences between maintenance and exchanging groups in efficacy and safety profiles. Moreover, in comparison with infliximab, the use of its biosimilar produce a substantial savings in EsSalud purchasing budget. In addition, both analyzes were discussed and accepted by rheumatologists, dermatologists and gastroenterologists working in EsSalud. In this sense, our manuscript is an integrated technical piece, which embraces scientific evidence, institutional budget and clinical experience about infliximab interchangeability in the complexity of Peruvian Health System, where EsSalud is one of the foremost public institutions with assurance, provision and health care functions (Sánchez-Moreno, 2014; Mezones-Holguin et al., 2019). Consequently, we described a mixed methodological approach to inform making-decisions with the best available evidence in low and middle-income countries context.

In the academic realm, other systematic reviews have addressed the interchangeability from original into biosimilar drugs. First, *Chingcuanco* et al., performed a SR in Pubmed, EMBASE, CENTRAL and LILACS until April-2016; they concluded that there is primary evidence that supports interchangeability from Biological reference products to biosimilar drugs in TNF-α family (Chingcuanco et al., 2016). Second, *Cohen et al*., carried out a SR including interventional and observational clinical studies in MEDLINE and EMBASE until June-2017; in this review the risk of events related to immunogenicity and declination of efficacy did not change after exchanging from original to biosimilar (Cohen et al., 2018). Third, *McKinnon et al.*, published an SR performed in Pubmed, EMBASE and Cochrane Library until June-2017 to evaluate the efficacy and safety of biosimilar interchangeability. There were still gaps to determining safety and efficacy of interchangeability of biosimilar was their conclusion, although they did not provide any specific conclusion about infliximab (McKinnon et al., 2018). Fourth, *Feagan et. al*, recently published a SR, they searched until January-2018 in Medline for articles and EMBASE for abstract congress. Six RCT and 64 observational studies were included. The authors described that "the evidence revealed no clinically important efficacy or safety signals associated with switching" (Feagan et al., 2019). Consequently, none of those synthesis studies reported differences in efficacy or safety between maintenance or exchanging into biosimilar, however there are different opinions in recommending the continuation with biosimilar drug.

In technical meetings with specialists, some concerns were exposed. The first was the inability to make specific comparisons for each disease separately- due to low statistical power- specifically in the clinical trial financed by the Government of Norway conducted in patients with rheumatoid arthritis, severe psoriasis, ulcerative colitis, Crohn's disease and spondylarthrosis. (Jørgensen et al., 2017). Although the authors performed subgroup analyses for each disease and reported findings with no statistically significant differences for several specific outcomes, their results were exploratory and could be affected by selection bias (Assmann et al., 2000; Brookes et al., 2004). However, valid conclusions were obtained for all diseases studied. In this regard, the authors report three main outcomes of efficacy (worsening of the disease, remission of the disease, and quality of life) and two safety outcomes (adverse effects and immunogenicity) for all diseases included. They observed non statistical differences between groups with adequate statistical power. (Jørgensen et al., 2017). Therefore, the first two outcomes of efficacy were the total percentage of patients who had worsening or remission of the disease in each group. Definition of state was based on medical evaluation supported by validated and accepted specific clinical scales for each disease (Jørgensen et al., 2017).

On the other hand, the use of generic questionnaires to assess quality of life (QoL) across the diseases was the second concern. *Jorgensen* et al. did not find differences in the SF-36 and EQ5D between the maintenance and switched groups (Jørgensen et al., 2017). QoL is widely recognized as a valid outcome in clinical studies and as basis for calculating utility measurements (Drummond et al., 2005; Bottomley et al., 2018). SF-36 and EQ5D can be used in any health conditions; both scales has been used by National Institute of Clinical Excellence (NICE) to assess efficacy of interventions in several diseases, including: rheumatological, dermatological and gastroenterological (Longworth et al., 2014). It is noteworthy that these two generic indices allow us to estimate utilities measures - as Quality of life adjusted life years (QALY) for comparing across different health conditions (Rabarison et al., 2015). In the following two paragraphs we provide a succinct description of those tools in relation to diseases evaluated.

The SF-36 is widely used worldwide and it has evidence of validity and reliability in Peru (Salazar and Bernabé, 2015). A series of SR show that this tool has adequate psychometric properties in patients with RA (Matcham et al., 2014), it allows to quantify worsening psoriasis in clinical trials (Ali et al., 2017) and it is a valid outcome in patients with psoriatic arthritis that receive biological drugs (Druyts et al., 2017). In addition, for inflammatory bowel disease, SR described that SF-36 is useful to assess the quality of life and it provides evidence of variations in stages of activity and inactivity of the disease (Knowles et al., 2018) and a Cochrane systematic review describe that SF-36 is a valid and reliable instrument to evaluate the effect of biological therapy. (LeBlanc et al., 2015). Moreover, other SRs argue that this questionnaire has psychometric validity (Yarlas et al., 2018a) and serves to estimate the burden of disease in patients with ulcerative colitis (Yarlas et al., 2018b). Also, SF-36 is useful to assess QoL in patients with ankylosing spondylitis (Yang et al., 2016).

The EQ5D is a tool developed by EUROQOL that, with appropriate contextualization, serves as the basis for the calculation of QALYs (Brazier et al., 2017; Dakin et al., 2018). There is a Peruvian version of EQ5D (Szende et al., 2007; Brooks et al., 2013). In patients with rheumatoid arthritis its use has been described in the estimation of utility measures (Boyadzieva et al., 2018), clinical practice (Hiligsmann et al., 2018), also it has a good correlation with disease activity (Skacelova et al., 2017). EQ-5D is a valid and reliable instrument in the assessment of worsening in clinical trials conducted in patients with psoriasis. (Ali et al., 2017). Also, it is used in patients with plaque psoriasis, psoriatic arthritis (Longworth et al., 2014; Yang et al., 2016) and multicenter studies of skin diseases (Balieva et al., 2017). A Cochrane SR described that EQ-5D was adequate in the evaluation of the effectiveness of treatment with biological drugs in patients with inflammatory bowel disease (LeBlanc et al., 2015). Likewise, EQ-5D is an adequate tool to measure quality of life in ankylosing spondylitis patients (Boonen et al., 2007) and high correlation with specific scales of the disease has been observed (Mlcoch et al., 2017).

Similarly, safety is a highly important outcome studied. We defined immunogenicity and adverse events as main safety results. Immunogenicity is a relevant marker in the biotherapeutics research, since the production of anti-drug antibodies is clearly associated with therapeutic failure and side effects of protein drugs (Ingrasciotta et al., 2018). Also, immunogenicity of Infliximab biosimilar can be extrapolated to the different diseases treated (Ben-Horin et al., 2015). Moreover, the adverse events, especially the serious ones, are valid safety outcomes for a biological drug in the context of clinical trials (Tridente, 2013). Subsequently, we incorporated two main safety measures in the biosimilars arena.

Our study has potential limitations. First, we did not include unpublished studies from the gray literature (reports, conference proceedings, doctoral theses/dissertations, etc.), which may imply a selection bias. But, critical appraisal of the evidence is essential for developing a SR, since, although the findings can be made known, we cannot evaluate their quality, which has repercussions on the validity and reliability of a synthesis study (Bolaños-Díaz et al., 2011). Second, non-inclusion of observational studies could be a selection bias source, even more when the academy recognizes them as a valid source of clinical evidence (Greenfield, 2017; Corrao and Cantarutti, 2018). Nevertheless, our manuscript is circumscribed in a specific decision-making environment, where there is an institutional regulatory framework. In EsSalud, IETSI has defined that– based on internal validity criterion- randomized clinical trials are the main source of evidence to inform makingdecisions; in addition, the overall results of SRs - that included observational studies - did not differ from RCT findings and provide consistency to our results. Third, we did not carry out a quantitative synthesis of the studies, due to the enormous clinical and methodological heterogeneity, but in this situation performing meta-analysis is not advisable (Melsen et al., 2014). Fourth, we have not considered drop-out rates and nocebo effect, which could potentially exist in switched patients (from original into biosimilar) (Kristensen et al., 2018; Odinet et al., 2018); education provided prior to switch -among other interventions - is a valuable tool that can greatly help overcome this effect (Pouillon et al., 2019). Fifth, in the financial analysis, we do not have the official information of short-term patients and long-term chronic patients in each disease approved; however, our estimation is valid since it was based on absolute institutional annual requirement of infliximab. Sixth, we did not have a national representative sample of physicians; however, participants were working in the main healthcare networks of EsSalud.

Beyond the limitations and based on cost-opportunity as a legitimate principle of collective health, our findings support the use of a biosimilar to continue the treatment in patients receiving infliximab in EsSalud. Therefore, biosimilar constitute a valid therapeutic alternative for the management of medical conditions approved in EsSalud. Access to biological drugs is a struggle for health care systems, especially in low and middle-income economies, where a key aspect is the price of these innovative medicines, which leads to a significant economic exertion from Governments and their public budgets. In this sense, infliximab biosimilars are an alternative that could be efficient in the Peruvian Social Security context.

### AUTHOR CONTRIBUTIONS

EM-H, FF, MC-R, PB-P, GS-F, RG-C, and JC-C participated in the conception of the study and the research question. EM-H, LL-S, and LH-S designed the systematic review, developed the search strategy, and selected the articles. EM-H, FF, PB-P, GS-F, MC-R, JC-C, and RG-C defined and discussed the outcomes of interest. EM-H, LL-S, and LH-S performed the extraction and preliminary drafting of the results. EM-H, FF, PB-P, and MC-R carried out the cost estimates. EM-H, FF, PB-P, GS-F, MC-R, JC-C, and RG-C reviewed the results and delineated

### REFERENCES


the discussion. EM-H, LL-S, and LH-S made the first version of the article. EM-H, FF, MC-R, PB-P, LH-S, LL-S, JC-C, GS-F, and RG-C made substantial contributions to the manuscript. All authors agreed with the published version of the article and assume responsibility for its content.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01010/ full#supplementary-material

biologics: a systematic review. *Ann. Intern. Med.* 165, 565–574. doi: 10.7326/ M16-0428


Food and Drug Administration (2018). Remicade (infliximab). Accessed.


Current Status and Future Direction. *Drug Saf.* 41, 1013–1022. doi: 10.1007/ s40264-018-0684-9


infliximab to CT-P13 and continuing CT-P13 in the PLANETRA extension study. *Ann. Rheum. Dis.* 76, 355–363. doi: 10.1136/annrheumdis-2015-208786 Zahl, A. (2017). *International Pharmaceutical Law and Practice*. Bolingbrook, IL,

USA: LexisNexis. Zelikin, A. N., Ehrhardt, C., and Healy, A. M. (2016). Materials and methods for delivery of biological drugs. *Nat. Chem.* 8, 997. doi: 10.1038/ nchem.2629

**Conflict of Interest:** RG-C has received funding for travel, accommodation, or expenses for congresses by Pfizer and Novartis, and has also received funding or contracts from Pfizer in health research projects. M-H has carried out academic activities financed indirectly by Jansen, a Johnson & Johnson company.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Mezones-Holguin, Gamboa-Cardenas, Sanchez-Felix, Chávez-Corrales, Helguero-Santin, Laban Seminario, Burela-Prado, Castro-Reyes and Fiestas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Robotic Therapy: Cost, Accuracy, and Times. New Challenges in the Neonatal Intensive Care Unit

*Ilaria Amodeo1,2, Nicola Pesenti3, Genny Raffaeli1,2, Gabriele Sorrentino1, Alessia Zorz1, Silvia Traina1, Silvia Magnani1, Maria Teresa Russo1, Salvatore Muscolo1, Laura Plevani1, Fabio Mosca1,2 and Giacomo Cavallaro1\**

1 Neonatal Intensive Care Unit, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy, 2 Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy, 3 Department of Statistics and Quantitative Methods, Division of Biostatistics, Epidemiology and Public Health, University of Milano-Bicocca, Milan, Italy

Background: The medication process in the Neonatal Intensive Care Unit (NICU), can be challenging in terms of costs, time, and the risk of errors. Newborns, especially if born preterm, are more vulnerable to medication errors than adults. Recently, robotic medication compounding has reportedly improved the safety and efficiency of the therapeutic process. In this study, we analyze the advantages of using the I.V. Station® system in our NICU, compared to the manual preparation of injectable drugs in terms of accuracy, cost, and time.

Edited by:

Cristiane De Cássia Bergamaschi, Universidade de Sorocaba, Brazil

#### Reviewed by:

Fathi M. Sherif, University of Tripoli, Libya Tauqeer Hussain Mallhi, University of Science Malaysia, Malaysia

\*Correspondence:

Giacomo Cavallaro giacomo.cavallaro@policlinico.mi.it

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 15 April 2019 Accepted: 08 November 2019 Published: 26 November 2019

#### Citation:

Amodeo I, Pesenti N, Raffaeli G, Sorrentino G, Zorz A, Traina S, Magnani S, Russo MT, Muscolo S, Plevani L, Mosca F and Cavallaro G (2019) Robotic Therapy: Cost, Accuracy, and Times. New Challenges in the Neonatal Intensive Care Unit. Front. Pharmacol. 10:1431. doi: 10.3389/fphar.2019.01431

Method: An in vitro experimental controlled study was conducted to analyze 10 injectable powdered or liquid drugs. Accuracy was calculated within a 5% difference of the bottle weight during different stages of preparation (reconstitution, dilution, and final product). The overall cost of manual and automated preparations were calculated and compared. Descriptive statistics for each step of the process are presented as mean ± standard deviation or median (range).

Results: The median error observed during reconstitution, dilution, and final therapy of the drugs prepared by the I.V. Station® ranged within ±5% accuracy, with narrower ranges of error compared to those prepared manually. With increasing preparations, the I.V. Station® consumed less materials, reduced costs, decreased preparation time, and optimized the medication process, unlike the manual method. In the 10 drugs analyzed, the time saved from using the I.V. Station® ranged from 16 s for acyclovir to 2 h 57 min for teicoplanin, and cost savings varied from 8% for ampicillin to 66% for teicoplanin. These advantages are also capable of continually improving as the total amount of final product increases.

Conclusions: The I.V. Station® improved the therapeutic process in our NICU. The benefits included increased precision in drug preparation, improved safety, lowered cost, and saved time. These advantages are particularly important in areas such as the NICU, where the I.V. Station® could improve the delivery of the high complexity of care and a large amount of intravenous therapy typically required. In addition, these benefits may lead to the reduction in medication errors and improve patient and family care; however, additional studies will be required to confirm this hypothesis.

Keywords: robotic therapy, newborn, safety therapy, patient safety, robotic cost, time, accuracy

## INTRODUCTION

A significant number of intravenous medications are administered in the Neonatal Intensive Care Unit (NICU) daily, and harmful medication errors are more likely to occur there compared to adult settings (Kaushal et al., 2001). Newborns are particularly vulnerable to medication errors, based on the peculiarity of the developmental pharmacotherapy (Kearns et al., 2003; Chedoe et al., 2007). Prescriptions are expressed per kilogram of body weight and require accurate calculation and multiple dilutions when administered to preterm low-birth-weight infants. Drug pharmacokinetics and pharmacodynamics change dynamically based on evolving systems and organs maturation (Tayman et al., 2011; Allegaert et al., 2013; Allegaert et al., 2014; Allegaert and Van Den Anker, 2014). The resulting patient-specific variability to drug exposure may threaten drug safety, particularly for compounds with a narrow therapeutic range (Samra et al., 2011). Lastly, most of the drugs prescribed are still off-label for neonates (Conroy et al., 1999).

Given the complexity of all these issues, the medication process can be challenging in terms of cost, time, and risk of errors, particularly for preterm newborns (Conroy et al., 1999; Chappell and Newman, 2004; Kugelman et al., 2008). In order to optimize the therapeutic process, many technologic solutions have been introduced in clinical practice in recent years, such as the use of a computerized physician order entry system (CPOE), bar-coded identification, and smart-infusion pumps (Myers et al., 1998; Bates, 2007; Vardi et al., 2007). Recently, robotic medication compounding has demonstrated an improvement in the safety and efficiency of the therapeutic process in different settings. Robotic devices have been shown to be useful in the preparation of chemotherapies, adjuvant medications, and cytotoxic drugs for adult patients (Seger et al., 2012; Nam et al., 2016; Iwamoto et al., 2017; Unluturk et al., 2018; Geersing et al., 2019). In all cases, the robot was handled by the hospital pharmacists. To date, there are no reports of robotic applications in the neonatal setting.

The I.V. Station® is an automated compounding robot that prepares ready-to-administer sterile medications through a fullyautomated process (Omnicell, inc. www.omnicell.com). This technology has been specifically developed for the automated individualized preparation and distribution of injectable drugs ready for use, including drugs requiring multiple dilutions (either powdered or in solution). Several studies have already

demonstrated that the use of the I.V. Station® reduces the rate of preparation errors and the waste of injectable drugs (Flynn et al., 1997). In this study, we compare the advantages obtained from the robot-assisted preparation of injectable drugs by the I.V. Station®, to the manual preparation in terms of accuracy, costs, and time in our NICU.

### MATERIALS AND METHODS

### Study Design

This study was conducted in the NICU of Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico in Milan in 2016 for 2 months after the introduction of the I.V. Station® technology. We performed an *in vitro* experimental controlled study to analyze the accuracy, cost, and preparation time of medication with robot-assistance compared to manual preparation.

The analysis included 10 injectable drugs that were either powdered or liquid. For each drug, 10 manual preparations and 10 automated preparations were compared, for a total of 200 samples (100 manual and 100 automated). Prescriptions have been made considering a hypothetical newborn patient of 1000 g of body weight. Dosages were prescribed based on the pediatric and neonatal therapeutic dosage handbook (Taketomo et al., 2018). The updated annual consumption of each drug calculated from January to December 2018 in listed in **Table 1**.

Samples intended for *in vitro* analyses were not utilized for clinical purposes. Since patients were not directly involved in the study, our investigation did not require ethical approval.

### Accuracy

Accuracy was calculated as <5% difference in the bottle weight during different stages of preparation (reconstitution, dilution, and final preparation) for both the manual and automated processes. This value reflects the accuracy of the concentration of the drug at the end of each step.

#### Automated-Preparation

Using the I.V. Station®, drugs were prepared and multiple controls were performed at different steps, specifically:

*• Reconstitution check*: for the powdered drugs (Acyclovir, Ampicillin, Ampicillin + Sulbactam, Amoxicillin + Clavulanic

#### TABLE 1 | Drugs, composition, therapeutic dose, and number of annual prescriptions.


Acid, Teicoplanin, Vancomycin), the I.V. Station® maintained accuracy within a range of ±5% by assessing the weight of the solvent injected.


### Manual Preparation

Drugs were prepared by 6 nurses with at least 5 years of experience working in our NICU. No specific training was conducted before starting the study since we considered a minimum of 5 years NICU expertise sufficient for the purpose of the study.

The following formula was applied as we were aware of the density of each drug:

•*Accuracy*% ( = *measured weight*- ) *ideal weight* / (*ideal weight*)×100

Multiple controls were performed at different steps, specifically:


### Costs

We calculated and compared the overall cost of the manual vs. automated drug preparation, considering a detailed list of items including bottles, syringes, needles, caps, solvents, gloves, sterile gauze, and stoppers. For manual preparations, costs were calculated based on a single dose of medication. For the IV Station®, since dilution takes place once, a higher number of vials are often needed to obtain the desired concentration of the stock solution, from which the IV Station® obtains the different doses of medication, but also has a larger amount of final product available. Hence, the costs included all the materials used to obtain the stock solution; however, multiple administrations can be obtained.

The costs of electricity, machine maintenance, days of detention due to possible damage or machine failure over the 2 month observation period were not considered.

### Preparation Time

For each manual preparation, we considered the time required to walk to the laminar flow hood at nurse's station, prepare a single dose of each drug, prime the intravenous line, and return to the patient to begin drug administration. For automated compounds, we considered the time required to walk to the I.V. Station® , withdraw all the drugs already prepared by the robot, return to the laminar flow hood to complete the preparation and priming of the intravenous line, and return to the patient to begin drug administration.

### Statistical Analysis

Descriptive statistics for each step of the process are presented as mean ± standard deviation or median (range) for both the I.V. Station® and manual preparations. The cost of each drug preparation is fixed, as a result of calculations based on the value of the drug itself and the overall material required. Therefore, a statistical comparison between the cost of the I.V. Station® and manual preparation cannot be conducted due to the absence of variability in the estimates. One-way ANOVA was used to compare time savings between the drugs for single preparations. Time differences between the I.V. Station® and manual preparations are reported. Statistical analysis is not informative when studying differences between the I.V. Station® and manual accuracy because all values lay in the range of ±5%.

Estimates of costs/savings and preparation times are presented using a heatmap plot. Boxplots are used to show the distribution of time for each drug analyzed. Statistical analyses were performed using R version 3.5.3 (R Foundation for Statistical Computing, Vienna, Austria).

### RESULTS

### Accuracy

The data concerning the accuracy of manual and robotic preparations are depicted in **Table 2**. For the I.V. Station® preparations, the median error observed during reconstitution, dilution, and final therapy ranged within ±5% accuracy. Narrow ranges of error were observed, and they were always included in the ±5% interval.

In the case of manual preparations, the median error lied in the range of ±5% accuracy, with the exception of the Ampicillin + Sulbactam, in which a median error 6.3% was observed during the dilution check. Moreover, higher variability and wider ranges of error were observed, which sometimes exceeded the ±5% limits. Indeed, in 5% of all manual preparations, the accuracy was outside the admitted interval between the range of ±5% (**Table 2**). A graphical representation of the accuracy in reconstitution, dilution, and final product for manual and robotic preparations is depicted in **Figures 1A**–**C**.

### Cost and Savings

The costs related to manual and automatic drug preparations and the number of vials needed to be prepared for a single dose of medication are listed in **Table 3**. A projection of the expected costs savings with the I.V. Station® compared to manual drug


TABLE 2 | The accuracy of manual and robotic preparations.

IV, I.V.Station®; M, manual.

preparation is displayed in **Figure 2**. For each drug, the number of drug preparations needed to amortize the robotic preparation is shown. In our study, when a low number of preparations was required, the robotic process was more expensive than the manual one (red boxes). As the number of preparations increased, the I.V. Station® optimized the materials consumed and cost, and eventually equaled the cost of manual preparation (white boxes) or became even less expensive (green boxes). As shown in the heatmap, if a single preparation was considered, the manual method was more cost-effective in four out of 10 cases. However, with the 10 drug preparations analyzed, the I.V. Station® led to substantial savings for all the cases considered. The expected savings ranged from 8% for ampicillin to 66% for teicoplanin and may continue to rise as the total amount of final product increased.

### Preparation Time

The time needed to prepare each drug separately is shown in **Figure 3**. In almost all cases, the time required to prepare a single manual dose was less than 4 min and ranged from 1 min 17 s for paracetamol to 3 min 18 s for Amoxicillin + Clavulanic Acid. The only exception was with teicoplanin, in which a single preparation took up to 18 min 43 s. In this case, the relevant difference was due to the difficulty associated with manual reconstitution and dilution of the powdered drug.

The median time required for a single I.V. Station® preparation was 2 min 59 s. This was in line with a majority of the manual results, although it was higher compared to paracetamol, fluconazole, and metronidazole (*p*-value < 0.001) and lower than amoxicillin + clavulanic acid (*p*-value = 0.007) and teicoplanin (*p*-value < 0.001).

**Figure 4** shows a projection of the expected time-savings associated with the I.V. Station® compared to manual preparation. For each drug, the number of administrations required to amortize the I.V. Station® preparation times are depicted. In our study, when a low number of doses were required, the I.V. Station® was more time-consuming than the manual method (red boxes) in some instances. For example, if we required a single preparation, the I.V. Station® was more expensive than the manual method in half of the cases analyzed. As the number of preparations increased, the I.V. Station® led to a progressive optimization of the preparation process (green boxes). The I.V. Station® led to substantial time-savings in all the cases analyzed. The time saved ranged from 16 s for acyclovir to 2 h and 57 min for teicoplanin and may continue to rise even as the total number of medications prepared increase.

### DISCUSSION

Medication errors are defined as any preventable event that can lead to inappropriate medication use or patient harm. Errors can occur at any stage in the medication-use process (prescribing, transcribing, dispensing, administering, monitoring) (Aronson, 2009). The therapeutic process includes several stages; each of which are potentially at risk for medication errors (Aronson, 2009). Infants who require more intensive levels of care are at a higher risk for medication errors and potentially fatal errors are three times more likely to occur in the NICU than in adult wards (Kaushal et al., 2001). Prescribing and drug administration challenges place newborns at risk of 10-fold and up to 100-fold potentially fatal dosing errors (Chappell and Newman, 2004; Taheri et al., 2013).

Preterm babies require a more intensive level of care and more complex therapy, which exposes them to a higher risk of



iatrogenic events. Thus, the incidence of medication errors that occur during the care of extremely preterm newborns is reported to be quite high near 57%, compared to 3% when caring for full-term infants (Kugelman et al., 2008). In this patient category, adequate drug dosing requires serial dilutions and manipulation of the solutions, thus increasing the odds of an incorrect dose, concentration, and contamination of the final solution (Kugelman et al., 2008). As previously demonstrated in the literature, manually-prepared drugs in the ICU setting frequently show significant deviations from the target concentration, while automated-prepared drugs show less variability (Allen et al., 1995; Parshuram et al., 2003; Wheeler et al., 2008; Dehmel et al., 2011; Seger et al., 2012). This wide variability can lead to harmful consequences, such as adverse reactions due to overdosing or loss of efficacy due to underdosing (Kugelman et al., 2008). In a recent study, Iwamoto et al. demonstrated that the robotic preparation of antineoplastic drugs using APOTECA-chemo had higher accuracy and a lower absolute dose error compared to manual preparation. The risk of overdose significantly reduced, resulting in safer cancer treatment (Iwamoto et al., 2017). Geersing et al. also demonstrated that APOTECAchemo preparations were microbiologically safe (Geersing et al., 2019).

Our data confirm that the I.V. Station® can reduce variability and thus improve accuracy at any step of the medication process (**Table 2**, **Figure 1**). Although manual method results in a tolerable median error, it shows wider variability among different preparations, with some dropping outside of the ±5% admitted interval. As recently reported in the literature, ward-based manually prepared solutions can deviate in concentration conformity more often than machine-made solutions (Kugelman et al., 2008). As can be expected in any operator-dependent process, the accuracy of the final therapy could not be guaranteed, without the possibility to identify and discard those preparations which do not respect the desired concentration. A centralized, automated preparation of standardized solutions has already been proposed as an effective means to reduce preparation error in everyday practice (Kugelman et al., 2008).

Unlike manual preparations, the I.V. Station® is set to automatically discard preparations that do not respect the predetermined range of accuracy. Hence, our data are consistent with this hypothesis. For this reason, the margin of error observed with the I.V. Station® never dropped outside the ±5% accuracy interval in any of the steps analyzed (reconstitution, dilution, final product). Therefore, the I.V. Station® may guarantee a high level of concentration conformity, and thus increased drug safety. Moreover, when combined with other strategies (i.e., electronic medical record, computerized order entry, and bar-code system) robotic technology is expected to reduce the risk of prescription and administration errors, improving safety and workflow efficiency (Dehmel et al., 2011).

Robotic preparation appears to be safer not only for the patient but also for the staff. Seger et al. observed a significant reduction in potentially harmful staff events after the

TAB

LE 3 | Costs of the single vial, number of vials and relative cost for the preparation of a single dose of medication (IV Station® and manual).

increases, I.V. Station® leads to an optimization of materials consumption and costs, becoming equal to manual preparation (white boxes) or even less expensive (green boxes).

introduction of robotic preparation of an antineoplastic drug (Seger et al., 2012). Although we did not evaluate staff events in relation to the therapeutic process, we could speculate that advantages similar to those reported by Seger et al. would occur in our NICU, where work-related risk is high. Robotic technology offers the opportunity not only for safer but also for a more cost-effective medication process. Moreover, Seger et al. found out that by introducing robotic preparation of an antineoplastic drug and adjuvant medications, they considerably reduced ancillary costs associated with several components. The savings accounted for 60% of the overall cost, and when annualized for the number of antineoplastics prepared in a year, they would have saved \$115,500 in material costs (Seger et al., 2012).

Our data confirm that robotic technology reduces the cost administering most drugs, especially when multiple preparations are needed. Benefits are expected to be even more remarkable when considering the huge number of medications prescribed in

FIGURE 4 | Estimated time savings. I.V. Station® expected time-saving compared to manual preparation. With a low number of doses, I.V. Station® could result in more time-consuming than the manual method (red boxes). With the increase of preparations, I.V. Station® leads to a progressive optimization of the preparation process (green boxes).

the NICU, since the more preparations of the same drug that are required within short period, the greater are the advantage, as the robot may use the same vial.

As shown in **Figure 2**, if the number of doses prepared through the I.V. Station® is low, the material consumption is high in a first step but could be subsequently amortized by an increase in the number of preparations. The greater is the number of patients requiring the same therapy in a given period, the more remarkable the advantage. Therefore, the I.V. Station® leads to greater savings in the long run, provided that the final product is consumed within the time frame in which the diluted solution remains stable.

Hence, the robotic process could be further optimized by consistently using the I.V. Station®, to expand production to other departments of the same hospital or other NICUs in the territory, building a distribution network with centralized production. Furthermore, automated-preparation of medications allow nurses to save time during the therapeutic process. Although manual preparation is rapid for some drugs (i.e., paracetamol, which is ready for use), some drugs require multiple dilutions and a significant amount of time to be prepared (i.e., teicoplanin) (**Figure 3**).

The I.V. Station® prepares multiple drugs in a sequence that can be withdrawn at once (**Figure 4**), thus inducing relevant time savings compared to manual preparation. For liquid drugs that are ready to be used (as paracetamol), the advantage commences when the number of doses to be prepared is more than nine. In analyzing the other compounds, savings begin with a lower number of preparations and is highest with teicoplanin, as robotic process saves a great deal of time from the first administration. With costs savings, the more intravenous therapy required, the greater the advantage in terms of procedural efficiency. The saved "drug preparation time" can, therefore, be used for the direct care of the neonate. While the robot is working, the nurse could remain at the

patient's bedside to better assist the baby wherever necessary, engage and educate the family, in an effort to provide the best care possible. As a result, the bond between the newborn and parents can be strongly enhanced. All these positive effects are not currently quantifiable but could represent a strong point in favor of smart robotics in the NICU setting.

Our study has some limitations. We analyzed a small number of injectable drugs, which only represents a small proportion of the intravenous therapy administered in the NICU. However, we have included the compounds that are the most commonly prescribed by neonatologists. Our analysis did not include costs concerning electricity, machine maintenance, or days of detention due to possible damage or failure of the robot. However, in 2018, the inactivity rate of the I.V. Station® was almost negligible (2.5% = 9.2/365 days), with a minimum time lag of 4.8 min up to 2.5 days of the stop. Seger et al. found some mechanical or software failure events associated with robotic preparations, which did not have harmful consequences on the patients but affected workflow efficiency and wasting of some medication (Seger et al., 2012). These are important limitations of robotic technology. The impact of ancillary costs, robot, or software failure, must be further characterized, and strategies for avoiding waste need to be implemented. Another limitation is that analysis of the microbiological safety of robotic compounds was not analyzed in this study. However, a preliminary microbiological analysis confirming the bacteriological safety was performed in 2013 before the implementation of the robot in our Unit. Based on the standard operating procedures of the I.V. Station®, the sterility of drug preparations is guaranteed for 24 h.

Based on our on-site microbiological surveillance, we have extended the sterility for liquid preparations for up to 72 h. Over 1 year, we analyzed microbiological cultures taken both from pharmaceutical preparations and the surface of the robot (daily during the first month, then weekly for three months and monthly for the rest of the year), which turned out to be negative (unpublished data). Lastly, the observation period was limited to 2 months after the introduction of the robot. To better define the advantages of the I.V. Station® in clinical practice, further analysis must be conducted on a greater number of compounds, and during a more extended period.

### CONCLUSIONS

Our data demonstrate that the I.V. Station® may support the therapeutic process in the NICU. Benefits are related to accuracy in drug preparation, cost, and time-saving. These advantages are particularly important in the NICU, where the I.V. Station® could facilitate the high complexity of care, nursing workload, and the significant amount of intravenous therapy typically administered. Robotics may positively impact the patients and their families, by allocating the human resources (i.e., nurses' time and effort) to neonatal care. A possible reduction in medication errors due to the introduction of automated procedures is also possible: however, additional studies are required to confirm this hypothesis. Efforts should be directed at drugs that are not currently available in vials, in order to extend the number of pharmaceutical drugs that can be prepared by the robot.

### REFERENCES


## DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

### AUTHOR CONTRIBUTIONS

GC, GR, GS, LP, and FM contributed conception and design of the study. GS, AZ, ST, SMa, MR, and SMu performed the experiments. IA, GC, GR, and NP wrote the first draft of the manuscript. All authors contributed to manuscript critical revision, read and approved the submitted version.

### ACKNOWLEDGMENTS

Vodafone Foundation and AISTMAROnlus bought and donated the robot to NICU of Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (Milan). Vodafone Foundation and AISTMAROnlus were not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication. The authors would like to thank all the healthcare providers involved in the Neonatal Intensive Care Unit of the Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico.


**Disclaimer:** None of the authors of the manuscript hold the copyright of I.V. Station. The robot was donated by the Vodafone Foundation, after a public money collection, to AISTMAR Onlus (https://www.aistmar.it/), a charity association, that donated it to NICU of Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan in 2013.

Our video showing the robot working in the Unit is available on the website: https://www.youtube.com/watch?v=ATDlPD-1j\_Q&t=10s. Furthermore, the robot manufacturer is aware of our willingness to carry out a comparative study even in the absence of funding, as the study was no profit. Therefore, being a product owned by NICU, we confirm that we have the correct authorizations to evaluate this product.

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Amodeo, Pesenti, Raffaeli, Sorrentino, Zorz, Traina, Magnani, Russo, Muscolo, Plevani, Mosca and Cavallaro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Fusogenic Liposomes Increase the Antimicrobial Activity of Vancomycin Against Staphylococcus aureus Biofilm

*Andreia Borges Scriboni1, Verônica Muniz Couto2, Lígia Nunes de Morais Ribeiro2, Irlan Almeida Freires3, Francisco Carlos Groppo1, Eneida de Paula2, Michelle Franz-Montan1\* and Karina Cogo-Müller1,4\**

#### Edited by:

Silvio Barberato-Filho, Universidade de Sorocaba, Brazil

#### Reviewed by:

Baolin Guo, Xi'an Jiaotong University, China Cristiane Duque, São Paulo State University, Brazil

#### \*Correspondence:

Michelle Franz-Montan michelle@fop.unicamp.br Karina Cogo-Müller karina.muller@fcf.unicamp.br

#### Specialty section:

This article was submitted to Translational Pharmacology, a section of the journal Frontiers in Pharmacology

Received: 22 February 2019 Accepted: 01 November 2019 Published: 29 November 2019

#### Citation:

Scriboni AB, Couto VM, Ribeiro LNdM, Freires IA, Groppo FC, de Paula E, Franz-Montan M and Cogo-Müller K (2019) Fusogenic Liposomes Increase the Antimicrobial Activity of Vancomycin Against Staphylococcus aureus Biofilm. Front. Pharmacol. 10:1401. doi: 10.3389/fphar.2019.01401

1 Department of Physiological Sciences, Piracicaba Dental School, University of Campinas, Piracicaba, Brazil, 2 Department of Biochemistry and Tissue Biology, Biology Institute, University of Campinas, Campinas, Brazil, 3 Department of Oral Biology, University of Florida College of Dentistry, Gainesville, FL, United States, 4 Faculty of Pharmaceutical Sciences, University of Campinas, Campinas, Brazil

Objective: The aim of the present study was to encapsulate vancomycin in different liposomal formulations and compare the in vitro antimicrobial activity against Staphylococcus aureus biofilms.

Methods: Large unilamellar vesicles of conventional (LUV VAN), fusogenic (LUVfuso VAN), and cationic (LUVcat VAN) liposomes encapsulating VAN were characterized in terms of size, polydispersity index, zeta potential, morphology, encapsulation efficiency (%EE) and in vitro release kinetics. The formulations were tested for their Minimum Inhibitory Concentration (MIC) and inhibitory activity on biofilm formation and viability, using methicillin-susceptible S. aureus ATCC 29213 and methicillin-resistant S. aureus ATCC 43300 strains.

Key Findings: LUV VAN showed better %EE (32.5%) and sustained release than LUVfuso VAN, LUVcat VAN, and free VAN. The formulations were stable over 180 days at 4°C, except for LUV VAN, which was stable up to 120 days. The MIC values for liposomal formulations and free VAN ranged from 0.78 to 1.56 µg/ml against both tested strains, with no difference in the inhibition of biofilm formation as compared to free VAN. However, when treating mature biofilm, encapsulated LUVfuso VAN increased the antimicrobial efficacy as compared to the other liposomal formulations and to free VAN, demonstrating a better ability to penetrate the biofilm.

Conclusion: Vancomycin encapsulated in fusogenic liposomes demonstrated enhanced antimicrobial activity against mature S. aureus biofilms.

Keywords: fusogenic liposomes, cationic liposomes, Staphylococcus aureus, vancomycin, biofilm

## INTRODUCTION

*Staphylococcus aureus* (*S. aureus*) is a Gram-positive microorganism responsible for the majority of nosocomial and community-acquired infections. Notably, *S. aureus* infections remain a global public health issue highly costly for the healthcare system, with increasing morbidity and mortality rates worldwide (Chakraborty et al., 2012; Honary et al., 2013; Elkhodairy et al., 2014; Holland et al., 2014). Today, over 90% of *S. aureus* strains are found to be resistant to methicillin (methicillin resistant *S. aureus*—MRSA), penicillin, aminoglycosides, macrolides, lincosamides, and other beta-lactams (Chakraborty et al., 2012; Muppidi et al., 2012; Sande et al., 2012; Elkhodairy et al., 2014; Shi et al., 2014).

In this scenario of microbial resistance, vancomycin (VAN) is considered a first-choice antibiotic for the treatment of methicillin-resistant *S. aureus* (MRSA) infections (Pumerantz et al., 2011; Honary et al., 2013; Holland et al., 2014; Men et al., 2016; Gudiol et al., 2017). While VAN remains a first-choice antibiotic for the treatment of MRSA infections, its therapeutic efficacy is limited due to its high molecular weight (1,449.2 g mol−1) and hydrophilicity restricting the drug interaction with bacterial cells and hindering its penetration into biofilms (Howden et al., 2010; Nicolosi et al., 2010; Butler et al., 2014; Moghadas-Sharif et al., 2015). In addition to that, VAN systemic side effects are another limiting factor, which include severe watery diarrhea, kidney failure (Pumerantz et al., 2011; Rose et al., 2012; Honary et al., 2013), ototoxicity, neutropenia, fever, anaphylaxis, thrombocytopenia, and phlebitis (McAuley, 2012).

Bacterial biofilms are characterized by the aggregation of specific bacterial species adhered to a substrate, forming highly organized microbial communities (Khameneh et al., 2014; McCarthy et al., 2015). Biofilm-forming bacteria display a differentiated phenotype compared to planktonic cells and have the ability to produce an extracellular polymeric matrix composed mainly of polysaccharides (Khameneh et al., 2014; Dong et al., 2015; McCarthy et al., 2015). This scaffold provides an extremely robust defense mechanism, which hinders antibiotic penetration into the biofilm structure, substantially reducing the susceptibility of bacterial cells to exogenous agents (Dong et al., 2015; McCarthy et al., 2015; Moghadas-Sharif et al., 2015).

The shortcomings of VAN traditional treatment along with the increased microbial resistance rates, and difficulty to treat biofilms have encouraged the development of drug-carrier systems such as VAN-loaded liposomal formulations (Kadry et al., 2004; Drulis-Kawa et al., 2009; Nicolosi et al., 2010; Lankalapalli et al., 2015). It has been shown that the liposomal sustained release of VAN (i) enhances antibacterial efficacy due to higher interaction of the antibiotic molecule with bacterial cells (Kim and Jones, 2004); (ii) improves pharmacokinetics (Ma et al., 2011); (iii) reduces toxicity (Sande et al., 2012); and (iv) increases the antimicrobial spectrum of action against Gram-negative bacteria (Nicolosi et al., 2010). Furthermore, liposomes can facilitate antibiotic penetration into bacterial cells and, therefore, increase drug concentration in the biofilm inner layers (Moghadas-Sharif et al., 2015). Despite these reports, only a few studies have evaluated the effects of liposomal formulations on the inhibition of biofilm development and viability, particularly *Staphylococcus* biofilms (Ma et al., 2011; Moghadas-Sharif et al., 2015).

The liposome composition can be specifically modulated in terms of morphology to favor the adsorption onto, or fusion with, the microbial cell membrane. Likewise, vesicle surfaces can be changed based on the characteristics of the infectious agent (Nicolosi et al., 2010). Among some types of liposomes with the ability of interacting with bacterial biofilm cells are fusogenic and cationic liposomes (Kim et al., 1999; Nicolosi et al., 2010). Fusogenic liposomes are vesicles that may fuse with biological membranes, thereby increasing drug contact and delivery into cells. They consist of lipids, such as dioleoyl-phosphatidylethanolamine (DOPE) and cholesterol hemisuccinate (CHEMS), which provide increased fluidity to the lipid bilayer and may destabilize biological membranes (Nicolosi et al., 2010; Aoki et al., 2015; Nicolosi et al., 2015). Because of their composition, fusogenic liposomes are normally in the liquid crystalline phase and, under specific chemical conditions, e.g., acidic milieu or in the presence of cations (Forier et al., 2014) they can lose the bilayer arrangement and fuse. Cationic liposomes are composed of lipids with a positive residual charge, such as stearylamine (SA), dimethyldioctadecylammonium bromide (DDBA), dimethylaminoethane carbamoyl cholesterol (DC-chol), and dioleoyltrimethylammoniumpropane (DOTAP), which provides specific electrostatic interaction with bacterial cell wall and biofilms, both negatively charged (Kim et al., 1999; Torchilin, 2012; Zhang et al., 2014; Moghadas-Sharif et al., 2015).

While fusogenic and cationic liposomes have proven advantages in interacting with bacterial cells and formed biofilms, there is still no consensus on the ideal composition of liposome-encapsulated VAN formulations able to prolong drug release and increase its antimicrobial efficacy. Thus, in the present study we developed and characterized large unilamellar vesicles of conventional (LUV VAN), fusogenic (LUVfuso VAN), and cationic (LUVcat VAN) liposomes encapsulating vancomycin hydrochloride. The *in vitro* antimicrobial activity of these formulations on *S. aureus* biofilms was further determined and compared.

### MATERIALS AND METHODS

### Materials

VAN hydrochloride was kindly provided by Teuto/Pfizer Laboratory (Anápolis, GO, Brazil). HEPES buffer, cholesterol (Chol), alpha-tocopherol (α-T) and egg phosphatidylcholine (EPC) were purchased from Sigma-Aldrich (St. Louis, MO, USA) and chloroform was obtained from Merck (Darmstadt, Germany). Dioleoylphosphatidylethanolamine (DOPE), dipalmitoylphosphatidylcholine (DPPC), cholesterol hemisuccinate (CHEMS) and stearylamine (Sa) were purchased from Avanti Polar Lipids Inc. (Alabaster, AL USA).

### Preparation of Liposomal Formulations

Conventional (LUV VAN), fusogenic (LUVfuso VAN) and cationic (LUVcat VAN) liposomal formulations were prepared containing 10 mg/ml VAN. Plain, VAN-free formulations were used as negative controls in the experiments (LUV, LUVfuso, and LUVcat). All liposomal formulations were prepared with 10 mM lipid concentration, with the following composition: LUV–EPC : Chol:α-T (4:3:0.07, mol%) (Cereda et al., 2006); LUVfuso–DOPE : DPPC : CHEMS:α-T (4:2:4:0.07, mol%) (Nicolosi et al., 2010); LUVcat–EPC : Sa:Chol:α-T (1:0.5:0.5:0.07, mol%) (Kadry et al., 2004), respectively. All formulations were prepared in HEPES buffer (80 mM) containing 150 mM NaCl (pH 7.4).

Preparation of liposomal formulations was carried out as previously described, with modifications (Cereda et al., 2006). Briefly, the lipids were dissolved in chloroform, evaporated under nitrogen flow to obtain the lipid film, and vacuumed for 2 h to ensure complete solvent removal. Subsequently, the film was hydrated in HEPES buffer with or without VAN hydrochloride solution. Then the suspension was vortexed for 5 min to form large multilamellar vesicles (MLVs). The suspensions were extruded under nitrogen flow at high pressure (Extruder Emulsiflex C5, Avestin, Inc., Ottawa, ON, Canada) 12 times using polycarbonate membrane initially with 400 nm pores, and then, with 100 nm pores, to obtain small unilamellar vesicles. The extrusion of LUVfuso formulation was performed in water bath at 50°C, which is higher than the DPPC phase transition temperature (Nicolosi et al., 2010).

### Characterization of Liposomal Formulations

#### Morphological Analysis

The morphology of the different types of VAN-containing liposomes or plain liposomes was analyzed by Transmission Electron Microscopy (TEM) (906 LEO-ZEISS, Jena, Germany) at 80 kV. Briefly, one drop of each formulation was added to a copper-coated grid with 200 mesh for 10 s (Electron Microscopy Sciences, Fort Washington, PA). Subsequently, uranyl acetate aqueous solution (2%, w/v) was added and kept at room temperature for 4 h.

### Determination of Size, Polydispersity Index, Zeta Potential and Stability of Liposomes

Liposomal vesicles were diluted in deionized water for evaluation of the average size (nm), polydispersity index (PDI), and zeta potential (mV) by the dynamic light scattering using Nano ZS equipment (Malvern Instruments Ltd., Worcestershire, UK, England) at 25°C in triplicate. To evaluate stability of liposomes, these parameters were monitored during 180 days at 4°C.

### Vancomycin Encapsulation Efficiency

The encapsulation efficiency (%EE) of VAN into liposomal formulations was determined by the ultrafiltration–centrifugation method (Da Silva et al., 2016) (35). Unencapsulated VAN was separated from encapsulated VAN by ultracentrifugation (Optima L-90K Ultracentrifuge, Beckman Coulter Inc. Pasadena, California, USA) at 120,000*g* for 2 h at 10°C. Aliquots from the supernatant were diluted in deionized water and analyzed spectrophotometrically at 280 nm (Varian Cary® 50 UV-vis, Varian Inc., Palo Alto, CA, USA). The %EE was calculated based on the concentration of unencapsulated VAN over the concentration of VAN in solution, using the formula as follows:

% [ ] [ ] [ EE VANsolution unencapsulated VAN VANso = − ×100 lution]

### Evaluation of Vancomycin Release *In Vitro*

The drug release assay was performed using the Franz vertical diffusion cell (Franz, 1975), which consists of two compartments—one donor and one receptor—separated by a regenerated cellulose membrane (Spectra/Por® 2) with molecular exclusion limit of 12,000–14,000 Da (Spectrum Laboratories Inc., Rancho Dominguez, CA, USA) (de Araujo et al., 2008; Da Silva et al., 2016). An aliquot of 1 ml of the liposomal suspensions was added to the donor compartment, while the receptor compartment was filled with 4 ml of buffer (pH 7.4), maintained at 37°C and 400 rpm agitation. Aliquots of the receptor medium were removed throughout the 10-hour experiment and analyzed by spectrophotometry at 280 nm (Varian Cary® 50 UV-Vis, Varian Inc., Palo Alto, CA, USA). The collected volume was replaced with fresh medium due to the dilution effect.

### Evaluation of Antimicrobial Activity Microorganisms and Growth Conditions

Methicillin-susceptible *S. aureus* (MSSA) ATCC 29213 and methicillin-resistant *S. aureus* (MRSA) ATCC 43300 strains were used in this study. Microorganisms were maintained in Tryptone Soy Broth (TSB) (Difco®, New Jersey, USA) with 20% glycerol at −80°C, and cultivated onto Tryptone Soy Agar (Difco®, New Jersey, USA) plates at 37°C. Mueller Hinton Broth (MHB) (Difco®, New Jersey, USA) was used in the MIC assay, while Brain Heart Infusion (Difco®, New Jersey, USA) plus 1% D-glucose (Sigma-Aldrich, St. Louis, MO, USA) was used in the biofilm killing assays.

### Experimental Groups

Test formulations consisted of VAN-containing and VAN-free LUV, LUVfuso and LUVcat. The experimental groups were set as follows: A—culture medium, test formulation and inoculum; B culture medium, control formulation and inoculum; C—culture medium, free VAN solution and inoculum; D—culture medium, HEPES buffer (vehicle) and inoculum; E—culture medium and test formulation; F—culture medium and inoculum; and G culture medium alone.

### Minimum Inhibitory Concentration (MIC)

The MIC was determined by the microdilution method, as previously described by the CLSI (2012), using Mueller-Hinton Broth. The formulations were added to 96-well microplates and serially diluted to obtain concentrations ranging from 0.025 to 50 µg/ml. From 18–24 h agar cultures, three to five colonies of *S. aureus* were dispersed into saline solution and bacterial inoculum was adjusted using a spectrophotometer (λ 625nm, OD 0.1, 1 to 2 × 108 CFU/ml). Then, the inoculum was diluted and transferred to the wells at a final concentration of 5 × 104 CFU/ ml. The plates were incubated at 37°C for 24 h and the absorbance was read at 620 nm (Biochrom ASYS UVM 340, Biochrom, Cambridge, England). The MIC was defined as the lowest concentration of the formulation which inhibited visible bacterial growth. The experiments were performed in six replicates.

#### Inhibitory Effects on Biofilm Formation

The liposomal formulations were tested for their ability to inhibit biofilm formation and adherence according to the protocol proposed by Graziano et al. (2015) and Wu et al. (2013). First, BHI medium supplemented with 1% glucose and *S. aureus* cell suspension (final concentration of 5 × 104 CFU/ml) were added to 96-well U-bottom microplates. Right after, the test formulations were added to the wells and plates were then incubated for 24 h, at 37°C. After this period, the supernatant was removed, and the wells were washed three times with distilled water to remove loosely bound or non-adhered cells. Biofilms were stained with 0.4% crystal violet, solubilized with 98% ethanol and read in a microplate reader at 575 nm (Asys UVM 340, Biochrom, Cambridge, England).

#### Inhibitory Effects on Biofilm Viability

The liposomal formulations were next tested for their inhibitory effects on biofilm viability, as previously described (Graziano et al., 2015) (39). Cellulose acetate membranes (25 mm diameter, 0.2 µM pores) (Sartorius Stedim GmbH, Guxhagen, Hessen, Germany) were used as substrates for *S. aureus* biofilm formation. The membranes were placed in 6-well plates containing BHI medium supplemented with 1% glucose and bacterial suspension (approximately 1 × 106 CFU/ml in each well). The plates were incubated at 37°C for 24 h. Then the membranes were transferred to new plates containing fresh BHI plus 1% glucose, and biofilms were treated with the formulations at 1 × MIC, 10 × MIC, and 50 × MIC for 24 h. Treated biofilm-coated membranes were gently washed (three times) through immersion into 5 ml of 0.9% NaCl. Then, the membranes were transferred to other tubes containing freshly 5 ml of 0.9% NaCl and then sonicated with six pulses of 9.9 s, 5 s time-interval, 5% amplitude (VibraCell 400W, Sonics & Materials Inc., Newtown, CT, USA) and vortexed at 3,800 rpm for 30 s. Ten-microliter aliquots were collected from each tube, serially diluted, and plated for CFUs onto TSA. The plates were incubated at 37°C for 24 h.

### Statistical Analysis

The data distribution was analyzed using the Shapiro–Wilks test. The variables size, PDI, zeta potential, and %EE, were compared using unpaired *t*-test. Stability parameters for liposomes and the biofilm data were compared using analysis of variance (ANOVA) followed by Tukey's post-hoc test. The drug release profile was analyzed by two-way ANOVA followed by Tukey's post-hoc test. Statistical analyses were performed on Origin 8.0 (Microcal TM Software Inc., EUA) and GraphPad Prism 6.0 (San Diego, California, USA). The data were presented as mean and standard deviation (SD), with a 5% significance level. All data are representative of three independent experiments.

### RESULTS

### Characterization of Liposomal Formulations

TEM images confirmed that the liposomal vesicles had spherical shape with clear edges. Vesicle size in all formulations ranged between 100 and 200 nm. Micrographs of all liposomal formulations are presented in **Figure 1**. As exemplified in **Figures 1C, D**, some fusogenic vesicles were found to merge with each other, which typically characterizes this type of liposome.

The means and standard deviations of size, PDI, zeta potential and %EE of the liposomal formulations are given in **Table 1**. Comparisons were made between plain and VAN-containing liposomes. No differences in size were found while PDI values increased for VAN-containing formulations in comparison to plain controls (*p <* 0.05). Moreover, as expected, the zeta potential values confirmed the presence of negative charges on LUV and LUVfuso liposomes and positive charges on LUVcat. The encapsulation decreased the negative zeta potentials of LUVfuso VAN liposomes (*p* < 0.05), while it increased the positive zeta potential in LUVcat VAN, as compared to their respective controls (*p* < 0.05). Higher %EE values were observed for LUV VAN, followed by LUVfuso VAN and LUVcat VAN.

The stability of the formulations was determined from measurements of size, PDI and zeta potential (**Figures 2A–C**) during storage at 4°C. In general, LUVcat VAN, LUVfuso VAN and LUV VAN kept their size during the 180-day experimental period (p > 0.05). LUV VAN showed an increase in size after 7 days of storage (p < 0.05) but kept their size in the other time points. LUVcat VAN also changed in size after 60 and 90 days (p < 0.05). However, these alterations were no greater than 10% of the initial size. No significant changes in PDI and Zeta values were found during the experiment (p > 0.05). It is also worth noting that, although with an increasingly trend, PDI values were found to be under 0.2, as required for a monodisperse size distribution.

The release kinetics of VAN in solution and encapsulated in the liposomal formulations was determined *in vitro*. As seen in **Figure 3**, encapsulated VAN formulations showed prolonged releases overtime as compared to free VAN (p < 0.05).

The LUV VAN formulation showed slower release profile than the other liposomes (p < 0.05), whereas LUVfuso VAN and LUVcat VAN were found to have very similar release kinetics (p > 0.05). As expected, VAN-free formulations showed greater percent release at all timepoints, with a significant difference from the other liposomal formulations (p < 0.05).

### Antimicrobial Activity

Free and encapsulated VAN LUV formulations affected bacterial growth in both MSSA (29213) and MRSA (43300) strains, with MIC values ranging between 0.78 and 1.56 µg/ml. These findings are in line with the information provided by the CLSI concerning *S. aureus* susceptibility to VAN (CLSI, 2012).

Next, the formulations were tested for their inhibitory effects on S*. aureus* ATCC 29213 biofilm adherence and formation. As shown in **Figure 4**, treatment with all formulations inhibited biofilm formation in a dose-dependent

follows: (A, B) LUV; (C, D) LUVfuso; and (E, F) LUVcat. Bars indicate 200 nm, with 100,000× magnification).

fashion. Free VAN was found to inhibit biofilm formation at MIC (1.56 µg/ml) and higher concentrations as compared to the untreated biofilm control, while the inhibitory effects of liposome-encapsulated VAN were only seen from 2 × MIC (3.13 µg/ml).

These results corroborate those of the *in vitro* release kinetics assay (**Figure 3**), in which encapsulated VAN showed a late release profile as compared to free VAN. These can be attributed to the % amount of VAN encapsulated into the liposomes, so that just a fraction amount of VAN is available to immediately TABLE 1 | Mean ( ± SD) of the size (nm), polydispersity index (PDI), zeta potential (mV) and encapsulation efficiency (%EE) of the liposomal formulations developed in this study.


The asterisk "\*" indicates statistically significant difference between plain (LUV, LUVFUSO, LUVCAT) and their respective vancomycin-containing liposomes (LUV VAN, LUVFUSO VAN, LUVCAT VAN), p < 0.05 (unpaired t-test).

act. Thus, it is likely that a lower amount of VAN molecules was initially released from the liposomal formulations, thereby slowing up their overall antimicrobial effects.

The inhibitory effects of the formulations on biofilm viability were also investigated. **Figure 5** shows the mean (± SD) CFU/ ml (Log10) of biofilms treated for 24 h at 1 × MIC, 10 × MIC, and 50 × MIC. The data was compared among treatment groups and the untreated control. At 1 × MIC, only LUVcat VAN caused a significant decrease in the number of viable biofilm cells (*p <*  0.01). Nevertheless, at 10 × MIC and 50 × MIC all formulations showed significant inhibitory effects as compared to the untreated control (*p <* 0.05). Free VAN was not able to affect biofilm viability significantly at 10 × MIC (p > 0.05), but it did at 50 × MIC (p < 0.05). When liposomal formulations were compared among themselves, we observed that LUVfuso VAN had the most noticeable inhibitory potential on mature biofilms, followed by LUVcat VAN and LUV VAN, with significant differences among them (p < 0.05).

The effects on mature biofilms treated with LUVcat VAN and free VAN were found to be similar at 50 × MIC (p > 0.05) and greater than those promoted by LUV VAN (*p <* 0.05). LUVfuso VAN was the most active formulation against *S. aureus* biofilm viability when compared to the other groups (*p <* 0.05). LUVfuso VAN reduced biofilm viability by 3.5 Log10 CFU/ml (35×); LUVcat VAN and free VAN caused a reduction of 2.5 Log10 CFU/ml (25×), while LUV VAN reduced biofilm viability by 1 log10 CFU/ ml (10×) in comparison to the control.

### DISCUSSION

Nosocomial and community-acquired MRSA infections remain a major concern in global health and have driven the adoption of public policies and medical research in this field (Honary et al., 2013; Elkhodairy et al., 2014; Holland et al., 2014). Evidence has shown the promising results of liposomal vesicles as drug carriers for pharmaceutical application (Kim et al., 1999; Nicolosi et al., 2010; Ma et al., 2011) (25, 26, 30). Herein, we report the development, characterization, and antimicrobial properties of experimental formulations containing VAN encapsulated into conventional, fusogenic and cationic liposomes. We compared the different formulations and demonstrated that the drugdelivery liposomes were more active than VAN in solution in reducing mature biofilm, with better efficacy for LUVfuso VAN.

Our goal when selecting the liposomal formulations was to achieve greater interaction with bacterial cells and, thereby, facilitate penetration into mature biofilms. Conventional (LUV)

liposomes contain a mixture of EPC and cholesterol, which increases the rigidity and stability of the vesicles (de Paula et al., 2012). LUVfuso liposomes contain DOPE in their composition, which promotes destabilization of the lipid bilayer (towards inverse hexagonal structures) at acidic pH as it occurs in infected tissue. The use of DPPC was required for stabilization of the lipid bilayers due to the presence of DOPE. Additionally, EPC and CHEMS contribute to greater stability of the formulation (Aoki et al., 2015; Nicolosi et al., 2015). LUVcat liposomes contained stearylamine, EPC, and cholesterol in their composition. Sa is a positively charged lipid that facilitates, through electrostatic interactions, adsorption in the negatively charged bacterial biofilm (Balazs and Godbey, 2011). In order to prevent lipid oxidation, the antioxidant alpha-tocopherol was added to all liposomal formulations (de Paula et al., 2012).

The effect of VAN encapsulation was observed in changes in the Zeta values and size distribution in comparison to controls (**Table 1**). Such changes may have occurred because encapsulated VAN has a tendency to be located in the aqueous core or adjacent lipid-water interface near the polar head groups (Bozzuto and Molinari, 2015). This molecular location of VAN in liposomes could contribute to the reduction of size distribution homogeneity and enhance electrostatic attraction among liposomes, as VAN is positively charged. Similar results concerning vesicle size and PDI were also found with tetraether lipid liposomes (Uhl et al., 2017). Moreover, after 180 days of storage (4°C) the structural properties of the liposomes were maintained after VAN encapsulation, presenting desirable size and monodisperse distribution, as required for a drug delivery system.

LUV VAN and LUVcat VAN showed higher %EE (32.5% and 10.1%, respectively) than those already reported (2.0% and 5.0%, respectively) using equivalent (conventional, cationic) liposomes, but prepared by sonication and containing 20 mg/ ml VAN (Kadry et al., 2004). On the other hand, Nicolosi et al. (2010) observed greater %EE (65.8%) for fusogenic liposomes (prepared by the reverse-phase evaporation method) as compared to our findings (11.4%) (Nicolosi et al., 2010). According to the authors, the preparation method and drug concentration in the liposomal suspension may have influenced the high upload (Muppidi et al., 2012).

In this study, no significant difference was observed in the release kinetics of VAN-containing LUVfuso and LUVcat. Both formulations released 12% of VAN after 1 h, whereas LUV VAN released 2% and free VAN 33%. The differences in the drug controlled release profile among the liposomal formulations may be a result of their diverse %EE (Lankalapalli et al., 2015; Liu et al., 2015). Recently (Lankalapalli et al., 2015), evaluated the release kinetics of VAN from conventional liposomes EPC : Chol liposomes with VAN (10 mg/ml), prepared by the ethanol injection method. The authors observed similar results to those found in our study regarding VAN release from LUV VAN liposomes, and different results with regard to release of free VAN, which was about 42% after 22 h. This divergence may be related to the free VAN concentration used in the donor compartment, which was 100 mg/ml in the study by Lankalapalli et al. and 10 mg/ml in our study.

It is also known that VAN exerts antibacterial action by inhibiting the synthesis of cell wall peptidoglycans (Howden et al., 2010). This drug has a high affinity to the D-Ala-D-Ala residue from the peptidoglycan precursor, lipid II, thereby blocking the addition of final precursors by transglycosylation and transpeptidation, which ultimately interrupts cell wall formation.

Tukey's post-hoc test).

In *S. aureus*, peptidoglycan biosynthesis takes place in the cell division septum in a specific site of the cytoplasmic membrane (Howden et al., 2010). Thus, in order to promote its effects on the cell wall, VAN molecules should penetrate approximately 20 layers of peptidoglycan to reach the division septum and bind to the protein fraction (L-lysine-D-alanyl-D-alanine) of murein monomers used as a substrate for glycosyltransferases. Depending on the bacterial cell cycle phase, the division septum can be completely formed or under formation (Nicolosi et al., 2010). Hence, the distance between the cell wall and the plasma membrane is shorter at the early phases of bacterial growth, which might have contributed to the bactericidal effects of free VAN. However, when bacterial growth reaches a final stage, the division septum is completely formed. As a result, the distance between the cell wall and the plasma membrane is wider, which may hinder the action of free VAN. In this case scenario, it is believed that encapsulated VAN could more effectively penetrate the cell wall and reach the periplasmic space, therefore promoting its antibacterial effects (Sande et al., 2012; Nicolosi et al., 2015). Such increased penetration can explain the improved antibiofilm activity observed in our study for the liposomal formulations.

The MIC values of liposome-encapsulated VAN on *S. aureus* ATCC 29213 observed in our study are in agreement with those found by Kadry et al. (2004). These authors reported MIC values of 0.75 µg/ml and 1.50 µg/ml for cationic and conventional liposomes, respectively. Another study found that encapsulation of VAN into conventional liposomes reduced by 2 the MIC against MRSA strains as compared to free VAN (Sande et al., 2012). This liposomal formulation was composed of DSPC : DCP:Chol (7:2:1, mol%) containing VAN at 50 mg/ml, which was 5 times higher than the VAN concentration used in our study.

Our findings indicate that free VAN at MIC had better inhibitory effects on early stages of biofilm formation than had the liposomal formulations. The latter inhibited biofilm adherence only from 2 × MIC, probably due to the encapsulation of vancomycin into the liposomes, with less free drug available to interact with forming biofilm. On the other hand, the liposomal formulations showed improved antibacterial activity than free VAN against mature biofilms, particularly LUVfuso VAN which was the most effective. Therefore, encapsulated VAN showed greater bactericidal effects on mature biofilms probably due to its increased ability to penetrate the peptidoglycan layers, whereas free VAN remained trapped in the cell wall.

Fusogenic liposomes have an increased potential to interact with extracellular matrix and cell wall due to their ability to merge with lipid membranes (Forier et al., 2014; Nicolosi et al., 2015). These vesicles are composed of lipids that promote destabilization of the lipid bilayers (Forier et al., 2014) and their fusion with the bacterial cell wall was previously proved through flow cytometry, lipid-mixing assay, electronic transmission microscopy and immunochemistry (Beaulac et al., 1998; Sachetelli et al., 2000; Forier et al., 2014; Wang et al., 2016). These vesicles can pass through the cell wall and deliver VAN into the periplasmic space, thereby making it easier for the drug to reach the division septum and block peptidoglycan biosynthesis (Howden et al., 2010; Sande et al., 2012; Nicolosi et al., 2015). Besides, cationic liposomes may have a higher affinity for negatively charged biofilms, which can decrease VAN delivery time into the infectious focus (Kim et al., 1999; Kadry et al., 2004). Accordingly, these liposomes probably release VAN in the vicinities of the bacterial cell wall due to the affinity with its negative charge, resulting in inhibition of cell wall biosynthesis.

There are other studies with VAN-loaded liposomal formulations (Ma et al., 2011; Barakat et al., 2014), but very few tested the ability in inhibit or eradicate *S. aureus* biofilm, which is a more resistant form of growth and much less sensitive to antibiotics. In the present study, we compared two formulations that are claimed to be effective against bacterial growth: fusogenic and cationic vesicles. Both formulations were effective in reducing mature biofilm, but with superiority to fusogenic vesicles.

To the best of the author´s knowledge, there are only two studies that encapsulate vancomycin into fusogenic liposomes (Nicolosi et al., 2010; Garcia et al., 2017), but none of them tested the activity against *S. aureus*. In addition, there are other non-fusogenic VAN-loaded liposomes that were tested against *S. aureus*, but very few aimed to test against biofilm (Ma et al., 2011; Barakat et al., 2014). Other drug delivery systems have also been proposed to improve drug delivery at sites of infection and to overcome antimicrobial resistance, such as injectable and biodegradable hydrogels (Zhao et al., 2017; Qu et al., 2018; Liang et al., 2019; Qu et al., 2019), polymeric nanoparticles (Lakshminarayanan et al., 2018), metal-based nanoparticles (Brown et al., 2012; Noronha et al., 2017), carbon-based nanoparticles (Zhao et al., 2017; Jiang et al., 2018), etc. Contributing to the development and comparison of antibiotics delivery systems, the present study showed that the liposomes here tested can reduce the formation and viability of mature biofilm, in a way superior to free vancomycin.

## CONCLUSION

We demonstrated the successful development, characterization and stability of LUV, LUVfuso and LUVcat encapsulated VAN formulations. Liposomes improved the antimicrobial activity of vancomycin against *S. aureus* biofilm, with better efficacy for fusogenic vesicles. Future studies are needed to validate this formulation as a candidate for *S. aureus* infection control.

## DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/ supplementary material.

## AUTHOR CONTRIBUTIONS

AS: Conception and design of the project. Acquisition of data. Analysis and interpretation of data. Writing and revision of the manuscript. Approval of the final version of the manuscript. VC: Acquisition of data. Analysis and interpretation of data. Final approval of the version to be published. LR: Acquisition of data. Analysis and interpretation of data. Writing and revision of the manuscript. Approval of the final version of the manuscript. IF: Writing and revision of the manuscript. Approval of the final version of the manuscript. FG: Conception and design of the project. Analysis and interpretation of data. Approval of the final version. EP: Conception and design of the project. Analysis and interpretation of data. Writing and revision of the manuscript. Approval of the final version. MF-M: Conception and design of the project. Analysis and interpretation of data. Writing and revision of the manuscript. Approval of the final version. KC-M: Conception and design of the project. Coordination and execution of the experiments. Analysis and interpretation

### REFERENCES


of data. Writing and revision of the manuscript. Approval of the final version of the manuscript.

## ACKNOWLEDGMENTS

The authors thank the São Paulo Research Foundation (FAPESP, grant no. 2014/14457-5) for financial support and Teuto/Pfizer for kindly donating the vancomycin.


dressings for wound healing. *Chem. Eng. J.* 362, 548–560. doi: 10.1016/j. cej.2019.01.028


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Scriboni, Couto, Ribeiro, Freires, Groppo, de Paula, Franz-Montan and Cogo-Müller. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Pain and Inflammation Management in Older Adults: A Brazilian Consensus of Potentially Inappropriate Medication and Their Alternative Therapies

#### *Fabiane Raquel Motter1,2\*, Sarah Nicole Hilmer2 and Vera Maria Vieira Paniz1*

Institute of Medical Research, Royal North Shore 83 Hospital, St. Leonards, NSW, Australia

inflammation in older people adapted to the Brazilian context.

#### Edited by:

Luciane Cruz Lopes, Universidade de Sorocaba, Brazil

#### Reviewed by:

Monique M. Elseviers, University of Antwerp, Belgium Marlon Aliberti, University of São Paulo, Brazil

\*Correspondence: Fabiane Raquel Motter fabianemotter@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 31 July 2019 Accepted: 05 November 2019 Published: 02 December 2019

#### Citation:

Motter FR, Hilmer SN and Paniz VMV (2019) Pain and Inflammation Management in Older Adults: A Brazilian Consensus of Potentially Inappropriate Medication and Their Alternative Therapies. Front. Pharmacol. 10:1408. doi: 10.3389/fphar.2019.01408

Purpose: The aim of the present study was to develop and validate a Potentially Inappropriate Medications (PIM) list and alternative therapies for treatment of pain and

1 Postgraduate Program in Collective Health, University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo, Brazil, 2 Kolling

Methods: A preliminary PIM list suitable for the Brazilian market was developed on the basis of three published international PIM lists [Beers 2015, Screening Tool of Older People's Potentially Inappropriate Prescriptions - 2015, European Union (7) PIM list]. We used the modified Delphi technique (two-round) to validate concerns of use and alternative therapies related to PIM for treatment of pain and inflammation in older adults ≥65 years in Brazil. The panel involved nine Brazilian experts in geriatric pharmacotherapy. All items with mean Likert scale score ≥4.0 (agree) and the lower limit of 95% confidence interval ≥4.0 were considered validated in this study.

Results: At the end of the consensus process, 94 (65.3%) items of 144 were validated. In total, consensus was reached for 33/35 (94.3%) concerns about drugs that should be avoided in older patients regardless of diagnosis, for 22/23 (95.7%) concerns about drugs that should be avoided in older patients with specific conditions or diseases, for 11/23 (47.8%) with special considerations of use, and for 28/63 (44.4%) of therapeutic alternatives.

Conclusion: Although these criteria are not designed to replace clinical judgement, PIM and alternative therapies lists can be useful to inform prescribers, pharmacists, and health care planners and may serve as a starting point for safe and effective use of medications in older people.

Keywords: inappropriate prescribing, potentially inappropriate medications list, pain management, deprescriptions, aged, Brazil

## INTRODUCTION

The process of aging in a population is accompanied by a rising prevalence of chronic and degenerative diseases and, consequently, a higher incidence of conditions characterised by pain and inflammation (Horgas, 2017). Studies show that the prevalence of chronic pain among elderly people in the community ranges from 21.5% to 65.0%, depending on the study population, the sampling method, the interview method, and the definition of "chronic pain" (Hairi et al., 2013; Eggermont et al., 2014; Jackson et al., 2015; Leao Ferreira et al., 2016; Larsson et al., 2017; Cimas et al., 2018; Liberman et al., 2018; Dahlhamer et al., 2018).

Pain and inflammation management in older people is a challenge for health professionals. Older persons often have age-related physiological changes and a high number of comorbidities, and undergo a number of therapies, which increase the risk of adverse drug effects, making it difficult to establish a balance between the benefits and risks of medications used in this population (Hilmer et al., 2007; McLachlan et al., 2011). In addition, some of the most commonly prescribed medications for the treatment of pain and inflammation can confer significant risks on older adults and have been associated with the occurrence of adverse events such as falls, fractures, gastrointestinal bleeding, worsening of heart failure, cognitive impairment, and renal failure (Marcum and Hanlon, 2010; O'Neil et al., 2012).

Potentially inappropriate medication (PIM) is a term used to describe a medicine for which the risk associated with its use outweighs the potential benefits, especially when there are more effective alternatives available (Gallagher et al., 2008; Renom-Guiteras et al., 2015). Several explicit criteria have identified medications that are considered inappropriate for the treatment of pain and inflammation in older people (McLeod et al., 1997; Gallagher et al., 2008; Winit-Watjana et al., 2008; Holt et al., 2010; Kim et al., 2010; American Geriatrics Society Beers Criteria Update Expert P, 2012; Chang et al., 2012; Mann et al., 2012; Clyne et al., 2013; Renom-Guiteras et al., 2015; American Geriatrics Society Beers Criteria Update Expert P., 2015; Kim et al., 2015; O'Mahony et al., 2015; Oliveira et al., 2016). These tools were developed by expert consensus and provide an accessible resource for health professionals in different settings.

Despite the availability of information, PIMs continue to be prescribed and used as first-line medications in older people. The frequency of PIM is high across a variety of healthcare settings, including Brazil. According to Brazilian studies, 42–59% of older people use at least one PIM (Baldoni et al., 2014; Martins et al., 2015; Lutz et al., 2017; Nascimento et al., 2017). Some of most commonly PIMs are used for treating pain and inflammation. These results illustrate that more work is needed to improve the use of appropriate medications in older adults.

The first Brazilian consensus on PIMs was published in 2016 (Oliveira et al., 2016). Limitations reported by the authors included that the criteria were based on previous versions of Beers (2012) (American Geriatrics Society Beers Criteria Update Expert P, 2012) and STOPP (2008) (Gallagher et al., 2008) and did not incorporate therapeutic alternatives. Evidence-based clinical practice guidelines including the list of PIMs should be continuously updated to incorporate emerging or changing evidence as well as newly approved drugs in order to remain current in line with current evidence (Shekelle et al., 2012). In 2015, updates of Beers (American Geriatrics Society Beers Criteria Update Expert P., 2015) and STOPP (O'Mahony et al., 2015) and a new European list [EU(7)] were published (Renom-Guiteras et al., 2015). This last PIM list was based on several international PIM lists and contains suggestions for dose adjustments, special considerations of use and therapeutic alternatives.

Therefore, the aim of the present study was to develop and validate a PIM list for the treatment of pain and inflammation and their respective alternative therapies based on the three international PIM lists recently updated [Beers, STOPP, and EU(7) PIM list], applicable to Brazilian elderly individuals (Renom-Guiteras et al., 2015; American Geriatrics Society Beers Criteria Update Expert P., 2015; O'Mahony et al., 2015).

## METHODS

We used the modified Delphi technique to validate concerns and alternative therapies related to PIMs for the treatment of pain and inflammation in older adults aged ≥65 years in Brazil. This process combines evidence from the literature and expert opinion, and has been successful in the development of previous explicit criteria for older people (Kim et al., 2010; Chang et al., 2012; Renom-Guiteras et al., 2015; Kim et al., 2015; Oliveira et al., 2016; Aliberti et al., 2018). In this method, the search for consensus is systematic. The experts assess the information, also called propositions, presented by the researcher(s) in the form of a previously formulated questionnaire. The questionnaire is based on the literature review of a research problem and presents a synthesis of the main discussions on the subject during rounds (Dalkey, 1969; Campbell and Cantrill, 2001).

The development of this study comprised of five steps: preparation of a preliminary PIM list, the plan for a modified Delphi study (elaboration of a data collection instrument and selection of an expert panel), two rounds of survey, and the summary of consensus.

### Preliminary List of PIM and Alternative Therapies for the Treatment of Pain and Inflammation

A systematic literature review was performed in order to identify possible screening tools for detection of PIMs published between January 1991 and April 2017 (Motter et al., 2018). After the review of 36 different tools, the list of PIMs related to pain and inflammation management was based on a combination of the relevant medicines from the updated Beers criteria (American Geriatrics Society Beers Criteria Update Expert P., 2015), updated STOPP criteria (O'Mahony et al., 2015), and EU(7) PIM list (Renom-Guiteras et al., 2015). These PIM lists are the most comprehensive and updated previously published lists. The original version of the PIM lists (in English) was translated into Brazilian Portuguese by two Brazilian researchers. The availability of medications and alternative therapies listed in the Brazilian market was confirmed by a medication database from the Brazilian National Health Surveillance Agency (National Agency of Sanitary Surveillance A, 2017). When a medication class was listed in a published PIM list, we identified all medications belonging to the class that were available in Brazil. The concerns about each medication/medication class were formulated using the information provided in the original list. The list of special considerations of use and alternative therapies was based on the EU(7) PIM lists (Renom-Guiteras et al., 2015).

The preliminary PIM list was organized by medication/ medication class. In total, the list of 12 PIMs contained 104 items which involved: 35 concerns about medications/medication classes that should be avoided in older people regardless of diagnosis, 20 concerns about medications/medication classes that should be avoided in specific diseases or conditions, 19 dose adjustments and special considerations of use, and 30 possible therapeutic alternatives. In the second round, items suggested by experts during the first round could be added to the preliminary PIM list.

### Selection of Expert Panel

The survey of experts was carried out through an initial search of the Lattes platform on the National Council for Scientific and Technological Development (CNPq) website (National Council of Technological and Scientific Development, 2016) on the 10th and 30th of August, 2016, using the following descriptors: "geriatric" and "medications." The members were primarily selected based on their expertise in the areas of geriatric medicine and/or clinical pharmacology. Additionally, their regional location was considered in order to provide national representation and to gain national perspective on these topics. We identified 47 potential participants (geriatricians and pharmacists) who were then invited by email which contained information on the study objective and a link to access the informed consent form. They were assured that participation in the consensus process was voluntary and confidential.

### First and Second Rounds

As the experts were based in disparate geographical locations across Brazil, an online two-round Delphi questionnaire was administered to facilitate efficient data collection; a link to the questionnaire on the Google Docs® website was provided. The first round took place between January and May 2017, and the second round between May and June 2017. We asked experts to assess each item of the preliminary list using a five-point Likert scale that ranged from one point (strongly disagree) to five points (strongly agree). The experts were also offered the opportunity to add items and to suggest alternative treatments. All the respondents in the first-round questionnaire were invited to participate in the second-round questionnaire. The second round included items for which no consensus had been reached in the first round (see *Data Analysis* section) and any new item suggested by experts in the first round. Quantitative feedback (percentage rating) from the first round of the Delphi process was incorporated into the survey questionnaire for the second round. The expert panel was instructed to consider the feedback provided while re-scoring the items contained in the secondround questionnaire. For both rounds, reminder emails were sent as necessary to encourage participation.

## Data Analysis

The collected data were organized in an Excel spreadsheet and subsequently imported into STATA 12.0 statistical software (StataCorp LP, College Station, TX, USA). A descriptive analysis was performed, and the absolute and relative frequencies, means and 95% confidence interval (CI) of the study items were evaluated. At the end of the consensus process, only items with mean Likert scale ≥4.0 (agree) and lower limit of CI ≥4.0 were considered validated in this study. This cut-off point is similar to that used by Oliveira et al. (Oliveira et al., 2016) and higher than that used in another study with similar scale (Chang et al., 2012).

## RESULTS

### Participants

Of the 13 experts who agreed to participate in the study, 10 were geriatricians and 3 were pharmacists. Among them, nine completed the first round while seven completed the second round. All respondents of the second round participated in first round. We were unable to identify which experts had left the panel, because the first round was conducted anonymously. At the end of the consensus process, all participants were geriatricians with more than 10 years of experience in geriatric medicine.

## First and Second Rounds

In the first round, experts reached consensus on 51/104 (49.0%) items in the preliminary tool. Among these, 26/51 (51,0%) items were concerns about medication/medication classes that should be avoided in older people regardless of diagnosis, 17/51 (33,3%) were concerns about medication/medication classes that should be avoided in specific diseases or conditions, and seven were special considerations of use. Only one possible alternative therapy reached consensus in this step.

After the first round, the items were revised in concordance with the comments from the experts. Four items were modified, and 40 new items were added. The majority (32/40; 82.5%) of items suggested by the experts were possible alternative therapies. Items for which consensus was not achieved were resubmitted for the second round (N = 53).

In the second round, experts evaluated 97 items. Among these, 44/97 (45.4%) reached consensus. One modified item was validated and replaced the original item. At the end of the consensus process, 94 (65.3%) items of 144 were validated. In total, consensus was reached for 33/35 (94.3%) concerns about drugs that should be avoided in older patients regardless of diagnosis, for 22/23 (95.7%) of concerns about drugs that should be avoided in older patients in specific conditions or diseases, for 11/23 (47.8%) of special considerations of use, and for 28/63 (44.4%) of therapeutic alternatives.

**Table 1** presents the medication/medication classes considered inappropriate for older people independent of their TABLE 1 | Potentially inappropriate drugs for the older patients independent of diagnosis validated by expert consensus.


Frontiers in Pharmacology

 |

www.frontiersin.org

#### TABLE 1 | Continued


aConfidence interval; bNon-steroidal anti-inflammatory drugs; cSuggestions from experts; dItems validated only in the second round; eProlonged use of NSAIDs nonselective COX-2 increases risk of GI bleeding and peptic ulcer disease in high-risk groups, including those aged >75 or taking oral or parenteral corticosteroids, anticoagulants, or antiplatelet agents. f Regular opiate use for more than 2 weeks in those with chronic constipation without concurrent use of laxatives; gRisk of falls, fractures, confusion, dependency and withdrawal syndrome; hNot effective oral analgesic in dosages commonly used; may have higher risk of neurotoxicity, including delirium, than other opioids; safer alternatives available; i All dose adjustment/special considerations of use and the alternative therapies described in this table were validated by expert consensus (lower limit of confidence interval ≥ 4.0); jScreening Tool to Alert doctors to Right Treatment; lEuropean Union.

diagnosis and their respective average Likert scales and CIs. The expert panel classified nonsteroidal anti-inflammatory drugs (NSAIDs), muscle relaxants, colchicine, and opioids as PIMs regardless of the diagnosis in the study. **Table 1** also provides information about dose adjustments and special considerations for medication use and alternative therapies validated by experts in this study.

A consensus also was reached for eight medication/ medication classes that should be avoided in older patients with 16 different conditions or diseases (**Table 2**). The use of NSAIDs was considered inappropriate for seven different conditions (long term use in osteoarthritis and gout, history of peptic ulcer disease or gastrointestinal bleeding, hypertension, heart disease, and chronic kidney disease stage IV and V or estimated glomerular filtration rate <50 ml/min per 1.73 m2 ) while the use of corticosteroids was considered inappropriate in five different conditions (rheumatoid arthritis, osteoarthritis, osteoporosis, diabetes, and delirium).

We presented the concerns also described in other PIM lists such as Beers (American Geriatrics Society Beers Criteria Update Expert P., 2015), STOPP criteria (O'Mahony et al., 2015) and the EU(7) PIM list (Renom-Guiteras et al., 2015) (**Tables 1** and **2**). Among all concerns validated in this study, about 65% were also reported in Beers consensus (American Geriatrics Society Beers Criteria Update Expert P, 2015). Lower consistency was observed when our list was compared with the STOPP criteria (47.2%) (O'Mahony et al., 2015) and the EU(7) PIM list (40.0%) (Renom-Guiteras et al., 2015).

In this study, the experts did not achieve consensus on 2/35 (5.7%) medication concerns regardless of diagnoses (phenylbutazone and tizanidine), on the use of strong opioids as first-line therapy for mild pain, and 12/23 (52.2%) dose adjustment/special considerations for medication use (i.e. special considerations for meperidine, indomethacin, and baclofen use) (**Appendix I**). The panel also did not reach a consensus about 35/63 (55.5%) alternatives therapies. These included medications such as ibuprofen, naproxen, and weak opioids (**Appendix II**).

### DISCUSSION

The consensus method allowed us to identify PIM criteria and alternative treatment options for the treatment of pain and inflammation in older people, adapted to the Brazilian context. The panel reached consensus on 94 items which contain important information about the use of medicines for the treatment of pain and inflammation in older adults. Although these criteria were not designed to replace clinical judgement, PIM, and alternative therapy lists can be useful in informing prescribers, pharmacists, and health care planners (Hanlon et al., 2015), and may serve as a starting point for the safe and effective use of medications to treat pain and inflammation in older people. To the best of our knowledge, this is the first study in which a consensus on alternative therapies to PIMs adapted to the Brazilian context was reached.

In the absence of a strong evidence to guide the optimization of medication regimens in elderly people, the consensus of experts has been used as a strategy to develop PIM lists in several countries. The elaboration of PIM lists consists of a complex, dynamic, and time-consuming process which involves the combination of systematic reviews and expert opinion. For these reasons, many researchers have combined two or more tools and added other medications that they considered were missing in order to develop and adapt existing criteria to different settings (Kim et al., 2010; Chang et al., 2012; Renom-Guiteras et al., 2015; Kim et al., 2015; Oliveira et al., 2016). Our list was based on the Beers criteria (American Geriatrics Society Beers Criteria Update Expert P, 2015), STOPP criteria (O'Mahony et al., 2015), and the EU(7) PIM lists published in 2015 (Renom-Guiteras et al., 2015). Although this last list has not been widely used compared with the Beers and STOPP criteria, it included some drugs rarely included in other PIM lists and therapeutic alternatives.

We have compared the results obtained in this consensus with previous lists: Beers criteria (American Geriatrics Society Beers Criteria Update Expert P, 2015), STOPP criteria (O'Mahony et al., 2015), EU(7) PIM list (Renom-Guiteras et al., 2015), and Brazilian PIM list (Oliveira et al., 2016). Although the list developed by Oliveira et al. (Oliveira et al., 2016) was based on previous versions of Beers (2012) (American Geriatrics Society Beers Criteria Update Expert P, 2012) and STOPP (2008) (Gallagher et al, 2008), it was the first Brazilian PIM list. Thus, the comparison between our list and previous explicit criteria may contribute to improving the knowledge on PIMs and alternative therapies which had not been previously investigated in the Brazilian setting.

In our study, consensus in the Delphi process was reached for 95% of the concerns related to the use of medications for the treatment of pain and inflammation in elderly individuals. In contrast with Beers criteria (American Geriatrics Society Beers Criteria Update Expert P., 2015) and the Brazilian PIM list (Oliveira et al., 2016), all NSAIDs were considered to be PIMs, regardless of the diagnosis, in our study. These results may be justified by the fact that our experts suggested and validated alternative therapies for these medications. They considered that one approach to reducing adverse drug reactions associated with NSAIDs is to avoid the use of these medications and use preferred alternative therapies, especially in those older adults with pre-existing diseases. There are few data informing the use of these medications in elderly who frequently have additional comorbidities and use multiple medications which increase the risk of adverse effects (Reid et al., 2011; Makris et al., 2017). On the other hand, for those patients that require an NSAID even after the use of alternative therapies, the experts also reached consensus that some NSAID should be used at the lowest effective dose. In addition, this panel incorporated two new concerns about the use of corticoids in older people who have diagnosed with osteoporosis and diabetes. These concerns were also presented in other previously published PIM lists (Winit-Watjana et al., 2008; Kim et al., 2010).

Some concerns about medications only achieved consensus in the second round of questioning. In these cases, there was doubt about the evidence concerning the increased risk of adverse effects in elderly patients. For example, we can cite the concern about the use of celecoxib in older patients diagnosed with cardiovascular diseases, where there is conflicting data from different randomised controlled trials (Solomon et al., 2005; Nissen, 2017).

#### TABLE 2 | Potentially inappropriate medication use in the older patients considering diagnoses or conditions.


(Continued)

Pain and Inflammation Management

#### TABLE 2 | Continued


Motter et al.

Pain and Inflammation Management

#### TABLE 2 | Continued


aConfidence interval; bNon-steroidal anti-inflammatory drugs; cSuggestions from experts; dItems validated only in the second round; eThe alternative therapies described in this table were validated by expert consensus (lower limit of confidence interval ≥ 4.0); dScreening Tool to Alert doctors to Right Treatment; eEuropean Union; f Non-COX-2 selective non-steroidal anti-inflammatory drug (NSAID) with history of peptic ulcer disease or gastrointestinal bleeding, unless with concurrent PPI or H2 antagonist (risk of peptic ulcer relapse; gMedications classified as anticholinergic drugs in the criteria; h Used the measure Creatinine Clearance <30 ml/min; i Long-term colchicine for treatment of gout where there is no contraindication to allopurinol. Allopurinol is first choice prophylactic drugs in gout; j Avoid the long-term use of Non-COX-2 selective non-steroidal anti-inflammatory (> 3 months).

Pain and Inflammation Management

A consensus could not be reached on two concerns (phenylbutazone and tizanidine) regardless of diagnosis even after the second round. These medications were also not described as PIMs independent of diagnosis in the Brazilian list published during the Delphi process (Oliveira et al., 2016). This result may be explained by the lack of experience regarding the use of these medications in the Brazilian context, since they are not included in the national list of essential medications (RENAME) available at no cost and are thus seldom prescribed in this setting (Ministry of Health, Secretariat of ScienceTechnology and Strategic Inputs. [National List of Essential Medicines: RENAME 2017]: Ministry of Health, 2017). Regarding PIM in specific disease or conditions, the concern about the use of strong oral or transdermal opioids as first-line therapy for mild pain, also did not achieve consensus; it received the lowest media in this study. The appropriate pain assessment in older patients may be complex. Barriers such as underreporting of pain by patients and the presence of cognitive deficits and comorbidities may complicate this evaluation in older people (Horgas, 2017; Schofield, 2018). Thus, some experts may have considered that the prescription of strong opioids should incorporate individualized clinical judgement, and that a generalised statement would not be appropriate.

In contrast to other lists, the recommendation about the combined use of NSAIDs and proton pump inhibitors (PPIs) did not reach a consensus in this study. This result is also because the experts did not think NSAIDs should be used in older people. Furthermore, during the consensus process, the experts were concerned that the prescription of drugs such as PPIs represents increasing drug burden which may also put individuals at risk of other adverse events such as fractures and *Clostridium difficile* infection. Studies showed that *the* co-administration of PPIs does not prevent NSAID-induced intestinal damage but might actually aggravate it (Scarpignato et al., 2016). Thus, this result suggests that specialists prefer to use alternative therapies rather than use PPIs to avoid problems related to the use of NSAIDs for the treatment of pain and inflammation.

The panel has also addressed possible alternative therapies which may be prescribed to treat pain and inflammation. Paracetamol (acetaminophen) and dipyrone (metamizole) were identified as possible alternative therapies by experts in several concerns. Although recent literature has demonstrated the limited efficacy of paracetamol (Marcum et al., 2016), it remains recommended in guidelines as a first-line pharmacologic treatment for older adults with mild-to-moderate pain (American Geriatrics Society Panel on Pharmacological Management of Persistent Pain in Older P, 2009; Abdulla et al., 2013). With its dose-dependent hepatotoxicity, the recommended maximum daily doses should not be exceeded (3250 mg) and it should be prescribed with caution (lower doses) in patients with pre-existing liver disease, malnutrition, anorexia, heavy alcohol intake, or in patients treated by hepatic enzyme inducers (rifampicin, phenytoin, carbamazepine, or barbiturates) (Yoon et al., 2016). Regarding the use of dypirone, studies have demonstrated that for short term use, this medication appeared to be a safe choice when compared to other analgesics. However, the intermediate and long term safety of dypirone are still not well documented (Kotter et al., 2015; Andrade et al., 2016).

Interestingly, the experts agree strongly about the use of nonpharmacological interventions such as physiotherapy, acupuncture, thermotherapy, electrostimulation, and therapeutic massage as alternative therapies for the treatment of pain and inflammation in older people. Nonpharmacological approaches can help avoid drugs that have high risks of causing adverse events. For this reason, the body of evidence about nonpharmacological approaches is growing in older adults, especially, in persons with dementia and delirium (Livingston et al., 2014; Resnick et al., 2014; Hshieh et al., 2015).

The internet-based Delphi method offers a practical and cost-effective approach to identifying areas of concordance and disagreement involving a geographically dispersed group of experts. Another advantage of this method is the anonymity that encourages experts to make statements on the basis of their personal knowledge and experience. Finally, a significant advantage of the Delphi method is that participants are very much aware at each stage of the results of the previous rounds, and there is scope for each expert to provide more detailed feedback on both the process and the results. In this study, the majority of alternative therapies were suggested by the experts. Some of these differ from other previously published PIM lists (Renom-Guiteras et al., 2015; American Geriatrics Society Beers Criteria Update Expert P, 2015; O'Mahony et al., 2015). This result demonstrated the importance of set-specific country lists. In this study, we selected the most comprehensive and updated lists published based on a literature review (Motter et al., 2018). To the best of our knowledge, to date, this study is the first to validate alternative therapies to PIMs in a Brazilian setting.

In order to minimise the inclusion of controversial PIMs or alternative therapies, we decided before the development of the consensus process that an item should be included in the final list only if the lower limit of the 95% CI was ≥ 4.0. Thus, not only the mean score was taken into consideration but also the degree of discord.

There are, however, some limitations which must be acknowledged in this study. Thus, the results must be interpreted with caution. Firstly, although we carefully searched for experts who comprised the consensus process panel, the limited participation of experts and the drop out of two participants in the second round may have compromised the representativeness of some areas of expertise and geographic regions. Secondly, the lack of engagement of experts in the second round may restrict the application of the Delphi method. Finally, the authors did not perform an evaluation of the quality and strength of evidence for each concern or alternative therapies presented in this study. The items were based on information available in some consensus publications referred to. These tools were published in 2015 and new findings from recent clinical trials or systematic review (Beers PIM list was updated in 2019) were not reviewed; thus, we recognize that some PIMs might have been added to or excluded from the next version.

### CONCLUSIONS

Explicit criteria summarized specific statements for identifying problems with medications in older people which make drugs easy to use by clinicians and health professionals. Condition-specific country lists are an important tool which may improve the rational use of medications among older people. In this study, we reached a consensus on 94 items that contain important information about the use of medicines used in the treatment of pain and inflammation in older adults. We believe that the application of our criteria combined with clinical judgement should contribute to helping physicians, pharmacists, and other health professionals to optimize the treatment of pain and inflammation in older patients. In this study, we also included special considerations of use and therapeutic alternatives; these may be an important addition to the screening process in caring for elderly Brazilian people. Future research should evaluate the implementation of the list among health professionals, including the usefulness of the suggestions for special considerations of medication use and alternative therapies, and apply this methodology to other therapeutic areas.

### DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/ **Supplementary Material**.

### ETHICS STATEMENT

The studies involving human participants were reviewed and approved by Ethics Committee of Universidade do Vale do Rio dos Sinos (UNISINOS) (number: 1.731.392). The patients/ participants provided their written informed consent to participate in this study.

### AUTHOR CONTRIBUTIONS

FM and VP participated in all stages of this research, from the design and interpretation of data to the final write-up. FM prepared the work documents during the development process, recruited the experts and coordinated the delphi survey. FM, VP, and SH assisted with interpretation of the data and preparation of the final version of PIM list. FM drafted the manuscript, supported by VP and SH. All the authors critically reviewed and approved the final manuscript.

### REFERENCES


### FUNDING

FM was supported by the coordination for the improvement of Higher Education Personnel – CAPES through a doctorate at University of Vale do Rio dos Sinos, Brazil (grant number: 88887159088/2017-00). FM was also supported by CAPES through a sandwich doctorate fellowship at The University of Sydney, Australia (grant number: 88881.134589/2016-01). This systematic review was funded by the National Council for Scientific and Technological Development-CNPQ (grant number: 426720/2016-4). The funders were not involved in the design or conduct of the study, collection, analysis, or interpretation of the data or preparation or approval of the manuscript.

### ACKNOWLEDGMENTS

We are very grateful to the members of the expert panel for their individual contributions to the validation of the Brazilian PIM list for the treatment of pain and inflammation. The following members of the expert panel who agreed to be acknowledged for their participation: Giancarlo Lucchetti, MD, PhD (School of Medicine, Federal University of Juiz de Fora, Juiz de Fora, MG, Brazil); Gislaine Bonardi, MD, PhD (Hospital Moinhos de Vento, HMV, Porto Alegre,RS, Brazil); Luciana Branco da Motta, MD, PhD (Rio de Janeiro State University, UERJ, Rio de Janeiro, RJ, Brazil) Milton Luiz Gorzoni, MD, PhD (Faculty of Medical Sciences of Santa Casa de São Paulo, São Paulo, SP, Brazil.), Roberta Rigo Dalla Corte, MD, PhD (Hospital das Clínicas de Porto Alegre /Federal University of Rio Grande do Sul, Porto Alegre, RS, Brazil) Thiago Junqueira Avelino da Silva, MD, PhD (Laboratory of Medical Research in Aging (LIM-66), Hospital das Clinicas, University of São Paulo Medical School, São Paulo, SP, Brazil). Furthermore, We thank to the Coordination for the Improvement of Higher Education Personnel, National Council for Scientific and Technological Development for the support that they are providing for development of this study.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01408/ full#supplementary-material


medication for elderly people]. *Geriatr. Gerontol. Aging* 10, 168–181. doi: 10.5327/Z2447-211520161600054


guideline implementability and accounting for comorbid conditions in guideline development. *Implement. Sci.: IS* 7, 62. doi: 10.1186/1748-5908-7-62


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Motter, Hilmer and Paniz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Treatment of Cystitis by Hungarian General Practitioners: A Prospective Observational Study

Ria Benko1\*, Maria Matuz <sup>1</sup> , Zoltan Juhasz <sup>2</sup> , Julia Bognar <sup>3</sup> , Reka Bordas <sup>1</sup> , Gyongyver Soos <sup>1</sup> , Edit Hajdu<sup>2</sup> and Zoltan Peto<sup>4</sup>

<sup>1</sup> Department of Clinical Pharmacy, University of Szeged, Szeged, Hungary, <sup>2</sup> First Department of Internal Medicine, Infectology Unit, University of Szeged, Szeged, Hungary, <sup>3</sup> Grove Lodge One, Norwich, United Kingdom, <sup>4</sup> Emergency Department, University of Szeged, Szeged, Hungary

Background: Lower urinary tract infections (LUTIs) are amongst the most common community acquired infections with frequent antibiotic prescribing.

#### Edited by:

Luciane Cruz Lopes, Universidade de Sorocaba, Brazil

### Reviewed by:

Tore Wentzel-Larsenm, Norwegian Centre for Violence and Traumatic Stress Studies, Norway Daniela Oliveira De Melo, Federal University of São Paulo, Brazil

#### \*Correspondence:

Ria Benko ria.benko@pharm.u-szeged.hu; benkoria@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 30 August 2019 Accepted: 19 November 2019 Published: 19 December 2019

#### Citation:

Benko R, Matuz M, Juhasz Z, Bognar J, Bordas R, Soos G, Hajdu E and Peto Z (2019) Treatment of Cystitis by Hungarian General Practitioners: A Prospective Observational Study. Front. Pharmacol. 10:1498. doi: 10.3389/fphar.2019.01498 Objectives: To assess empiric antibiotic choice in different types of lower urinary tract infections. We also aimed to identify determinants of fluoroquinolone prescribing, as well as to determine the rate of short antibiotic courses. The frequencies of executing laboratory tests and recommending analgesics/anti-inflammatory drugs were also assessed.

Methods: A prospective observational study was performed in 19 different Hungarian primary care practices. Participating general practitioners (GPs) filled out data sheets for each patient with a suspected urinary tract infection. Details of drug use were evaluated. Comparison of different LUTI groups were made by descriptive statistics and univariate analysis. Possible determinants of fluoroquinolone prescribing were assessed by logistic regression.

Results: Data sheets of 372 patients were analyzed. The majority of patients (68.82%) had acute uncomplicated cystitis. While antibiotics were prescribed for almost every patient (uncomplicated cases: 92.58%, complicated cases: 94.83%), analgesics/antiinflammatory drugs were recommended at a rate of 7.81% in uncomplicated, and 13.79% in complicated cystitis cases. Ciprofloxacin was the most commonly prescribed antibacterial agent in both types of cystitis. Short-term antibiotic therapy was prescribed in one third of relevant cases. Logistic regression found a weak association between fluoroquinolone use and patient's age and presence of complicating factors.

Conclusions: Many aspects of suboptimal cystitis management were identified (e.g. unnecessarily broad spectra agents, too long antibiotic courses). In this study, patient characteristics has weakly influenced fluoroquinolone prescribing. Based on these results there is considerable room for improvement in primary care antibiotic therapy of cystitis in Hungary.

Keywords: lower urinary tract infection, management, antibiotic choice, fluoroquinolone use, symptomatic treatment, antibiotic stewardship

### INTRODUCTION

To combat antibiotic resistance by lowering selection pressure, rationalizing antibiotic use for the most frequent community infections should be the key target of any antibiotic stewardship programmes. Lower urinary tract infections (LUTIs) are among the most commonly encountered illnesses in the ambulatory care (e.g. almost half of all women will experience at least one episode of cystitis during their lifetime), with consequently high antibiotic prescribing (Bonkat et al., 2017; Tyrstrup et al., 2017; Dumpis et al., 2018).

The classification of urinary tract infections is based on the affected anatomical site, and can be further stratified into uncomplicated or complicated types based on the presence or absence of complicating factors (e.g. diabetes mellitus) (Bonkat et al., 2017; Reynard and Biers, 2019). Compared to uncomplicated LUTI, the pathogen spectrum of complicated LUTIs is much wider, and these bacteria are more likely to be resistant to antibiotics, thus their empirical treatment tends to utilize broad-spectrum antibiotics (Bonkat et al., 2017). Unfortunately, the widely used coding system of diseases (International Statistical Classification of Diseases and Related Health Problems (ICD) version 10) does not follow this classification, which limits the capabilities of database studies. Previous drug utilisation studies based on the national health care database of Hungary revealed suboptimal antibiotic use in various infections (Juhasz et al., 2013; Matuz et al., 2015). The aim of this study was to overcome the limitations of database studies by exploiting clinical data that enables correct diagnosing and further stratification of cases. Based on these exact patientspecific data we have assessed the empiric antibiotic choice in complicated and uncomplicated LUTIs. Determinants of fluoroquinolone prescribing, the use of analgesics/antiinflammatory drugs and the frequency of laboratory urine analysis (microbiological and chemical) were also evaluated.

### METHODS

A prospective observational study was conducted. General practitioners (GPs) were invited and voluntarily participated, without any financial incentives. Twenty-five GPs accepted participation (response rate: 51%), and 19 GPs (i.e. 19 different primary care unit from various geographical location) contributed throughout the study. GPs were asked to fill in a data sheet for all consecutive patients who first presented with a suspected urinary tract infection. The following data were recorded: patient's demography, drug allergy, presence of complicating factors, clinical signs, history of urinary tract infections in the previous year, performed lab tests and treatments. Doctors' specialty and experience (number of practicing years) were derived from a public national database (National Health Registration and Training Centre).

All patients over 16 years of age were eligible for inclusion. Pregnant women, as well as cases with a suspected upper UTI (pyelonephritis), or genital or sexually transmitted infection (e.g. vaginal discharge) and cases where crucial data (e.g. prescribed/ recommended therapy) were missing were excluded. Based on the recorded signs/complaints and comorbidities we classified each case as an uncomplicated or complicated lower urinary tract infection according to the guideline of the European Urological Association (Bonkat et al., 2017). The followings were considered as complicating factors: male gender, diabetes mellitus, presence of indwelling catheter/stent/tube, reconstruction of the urinary tract, recent instrumentational intervention within the urinary tract, functional/anatomical defects (e.g. obstruction, incontinence), renal failure, kidney transplant, and immunosuppression. In accordance with the widely accepted definition, we defined recurrent cystitis as a minimum of 2 infectious episodes within 6 months, or 3 infectious episodes within a year (Bonkat et al., 2017). Antibiotics were classified according to the WHO ATC (Anatomical-Therapeutic-Chemical) index (version 2019). Short-term antibiotic course was defined as prescribing a single dose of fosfomycin, a 3-daycourse of fluoroquinolone, a 5-day-course of beta-lactam and 5–7 days of nitrofurantoin therapy for uncomplicated cystitis (Reynard and Biers, 2019).

Patient age, diagnostic measures and prescribed/ recommended therapy was compared in complicated and uncomplicated cystitis by descriptive methods and univariate analysis (Fischer's and Welch's tests). Potential influencing factors of fluoroquinolone prescribing (patient-specific characteristics: age, gender, recurrent cystitis, presence of complicating factors; GP-specific factors: years of practicing, and specializations) were analyzed by logistic regression. Statistical analyses were performed with the R statistical software (version 3.5).

The study protocol was approved by the Regional Human Medical Biology Research Ethical Board of the University of Szeged, Hungary. The ethical approval did not allowed the identification and follow-up of patients, hence the therapeutic outcomes could not be evaluated.

### RESULTS

Overall, 510 data sheets were collected. A total of 138 patients were excluded (because of pregnancy, n = 11; genital/sexually transmitted infection, n = 11; suspected kidney infection, n = 74; missing data, n = 42), thus data sheets of 372 patients were analyzed. Most patients were female (n = 342, 91.94%), diagnosed with uncomplicated cystitis (n = 256, 68.82%). Recurrent cystitis was present in one fifth of patients (n = 71, 19.07%).

The average age of those diagnosed with complicated cystitis was significantly higher (64.44 ± 16.30 years vs. 48.12 ± 19.56), and recurrent cystitis was also slightly more frequent in this patient group. (Table 1).

Diagnostic measures and details of prescribed/recommended therapies are summarized in Table 1 and Figure 1. Those with complicated cystitis had higher age and higher rate of recurrent infection (Table 1). With some exceptions, urine analysis was extensively performed (in over 85% of cases) in both cystitis types, while microbiological analysis of midstream urine was



1 Welch's two-sample t-test.

2 Fisher's exact test.

\*Considering only those patients who were prescribed antibiotics (Uncomplicated cystitis: 237; Complicated cystitis: 110).

performed only in a minority of cases (15.23% in uncomplicated and 21.55% in complicated cystitis, see Table 1). Antibiotic was prescribed for 347 patients (93.28%), however, antibiotic prescription rate of individual GPs ranged between 50.00% and 100.00%. Analgesics/anti-inflammatory drugs was recommended in 1 per 10 cases, while individual GP's recommendation rate ranged between 4.17% and 100.00%. Using non-antibiotic treatment only (analgesic/antiinflammatory agent OR cranberry product OR herbal tea) was recommended in as few as 11 cases (2.96%).

The most frequently prescribed antibiotics are shown in Table 2. Although their rankings differed, the top five antibiotics were the same in both types of cystitis. Cipro floxacin was the most widely used agent. Sulphamethoxazol-trimethoprim and fosfomycin were among the most frequently prescribed antibacterials. Nitrofurantoin was prescribed rarely (uncomplicated cystitis: 4 cases, complicated cystitis: 6 cases). Guideline-recommended short-term antibiotic course was initiated in ~30% of cases (Table 2).

Only evidence of a weak association was found between patient characteristics and fluoroquinolone prescribing (Table 3): younger adults and those with complicating factors tended to be treated with fluoroquinolones (odds ratio age: 0.98 yearly; odds ratio complicating factors: 1.80). The frequency of fluoroquinolone prescribing varied substantially: some of the GPs did not prescribe these agents at all, while a single GP prescribed this antibiotic in 85.71% of cystitis cases diagnosed in her practice.

### DISCUSSION

The burden of urinary tract infections is high worldwide (Foxman, 2014). However, field studies to provide more insight into the clinical practice of the outpatient management of UTIs in adults are scarce (Martinez et al., 2007; Llor et al., 2011; Denes et al., 2012; Butler et al., 2017; Dumpis et al., 2018). The main objective of our study was to fill in this gap by assessing empiric antibiotic choice in different lower urinary tract infections (i.e. complicated and uncomplicated cystitis). No significant difference has been observed in the antibiotic prescribing patterns for the two types of cystitis.

As antibiotics are unambiguously superior to placebo for urinary tract infections (Falagas et al., 2009), antibiotic use can be considered optimal in every LUTI case. The rate of antibiotic prescribing for LUTI in this study (over 90%) meets international quality indicators (Adriaenssens et al., 2011; Le Marechal et al., 2018) and correlates well with findings of antibiotic utilisation studies for other countries, except the Netherlands and Latvia where antibiotic prescription rate is

#### TABLE 2 | Most frequently prescribed systemic antibiotics.


AMC, amoxicillin-clavulanic acid.

SMX-TMP, sulphamethoxazol-trimethoprim.



OR, Odds ratio; 95% CI, confidence interval; logistic regression model constant: 0.52; p = 0.177.

reported to be less than 60% and around 70% in acute urinary tract infections in adults, respectively (Martinez et al., 2007; Llor et al., 2011; Butler et al., 2017). The lower prescription rate in these countries may be explained by the more frequent application of delayed or conditional antibiotic prescribing (Gagyor et al., 2012; Gagyor et al., 2015).

The use of analgesics/anti-inflammatory agents was found to be limited, which can be explained by the lack of such recommendations in national and international guidelines on urological infections (Ministry of Health, 2010; Bonkat et al., 2017). This is in contrast to the German guideline which only recommends symptomatic treatment in uncomplicated cystitis with mild to moderate symptoms (Kranz et al., 2017).

Short-course oral antibacterial treatment has been proven to be as effective as long courses of antibiotics in the ambulatory management of LUTIs (Dawson-Hahn et al., 2017). However, in our study only 1 in 3 patients with uncomplicated cystitis received a short-term antibiotic course, a finding similar to other studies (Llor et al., 2011; Denes et al., 2012; Hawker et al., 2014; Durkin et al., 2018; Phamnguyen et al., 2019). This may be explained by the fact that pharmacies can dispense only complete boxes of medicines instead of a certain number of tablets actually needed,

The GPs in this study performed urine analysis (i.e. a urine dipstick test) in the majority of cases in both types of cystitis. As dipstick analysis is considered to have little added value when typical urinary tract symptoms are present (Bonkat et al., 2017), unnecessary testing can be presumed in some cases. On the other hand, as dipstick analysis may also give information on the causative bacterial class (i.e nitrite test is positive if Enterobacteriaceae is present) its use can be justified.

Microbiological confirmation of UTI is recommended in all cases of complicated infections and recurrent urinary tract symptoms (Bonkat et al., 2017). Unfortunately, microbiological analysis was requested at a suboptimal rate (i.e only in 21.55% in complicated LUTI) which can be partly explained by logistic issues (lack of local laboratories and long distance between the GP's practice and laboratories) that hamper timely identification of the pathogens (Hajdu et al., 2009). The Flexicult system (Bates et al., 2014; Hullegie et al., 2017), aimed for point-of-care diagnoses of UTI and susceptibility testing of urinary pathogens is not used in Hungary due to the lack of reimbursement (and also this is the case for other point-ofcare tests utilized in other e.g. Scandinavian countries).

The pattern of antibiotic use was similar for both uncomplicated and complicated cystitis, and showed a high dominance of fluoroquinolones. The use of fluoroquinolones in UTI varies greatly in the literature, but none of the European countries (except for a non-recent publication from France)(Denes et al., 2012) reported such a high ratio of fluoroquinolone prescribing. In Norway, Denmark, Sweden and the Netherlands the use of fluoroquinolone agents were reported to be below 10%, thanks to strict and well-functioning antibiotic stewardships (Agdestein et al., 2011; Tyrstrup et al., 2017; Dumpis et al., 2018; Holm et al., 2019). In Latvia and Lithuania, fluoroquinolones are reported to be used in 23% and 25% of uncomplicated UTI cases, respectively (Dumpis et al., 2018). Similar rates for fluoroquinolone use in cystitis (22%) was reported for Belgium (Tyrstrup et al., 2017). In contrast, a relatively frequent fluoroquinolone use (30% of all cases) in uncomplicated UTI has been reported from the US recently, and this rate was even higher in the preceding years (Durkin et al., 2018; Cowart et al., 2019). The decreasing trend is explained by the publication of an FDA black box warning on fluoroquinolones in 2016 (Cowart et al., 2019). In Hungary no up-to-date guidelines are available for the antibiotic prescribing in UTIs, and the previous guidelines (that still pop-up as the first hit in a google search) recommended the use of fluoroquinolones in all types of UTIs (Nephrology, 2006; Ministry of Health, 2010). Thus, in fact, our findings are not surprising, but the high fluoroquinolone dominance is unwanted, especially if we consider the 2018 safety review-based restrictions (European Medicines Agency, 2019) of the European Commission on fluoroquinolone prescribing (valid since 11 March 2019 in all EU countries).

Only weak determinants of fluoroquinolone prescribing were identified: younger age and presence of complicating factors were found to influence the rate of fluoroquinolone therapy. As fluoroquinolones have a high potential to generate resistance (Schito et al., 2009), and the fluoroquinolone resistance of E.coli already exceeds 20% in urine samples in Hungary (Hungarian National Bacteriological Surveillance Management Team, National Centre for Epidemiology), the high use of fluoroquinolones demonstrated in our study is clearly worrisome. Qualitative studies are needed to better explore the high differences in fluoroquinolone prescribing rate of individual GPs.

Patients with uncomplicated cystitis were prescribed fosfomycin in 18.75% of all cases and nitrofurantoin in only 4 cases which is suboptimal according to international quality indicators (Adriaenssens et al., 2011). These data are also regarded as worrisome, as these two drugs have their renaissance in the treatment of uncomplicated LUTI due to their preserved effectiveness (Gardiner et al., 2019). On the other hand, these agents (i.e. fosfomycin and nitrofurantoin) should be avoided in complicated cystitis due to their lack of activity against the potential uropathogens in these cases (Bonkat et al., 2017). Nevertheless, fosfomycin was among the top five agents prescribed for complicated cystitis.

Regarding sulphamethoxazol-trimethoprim (SMX-TMP), its first line use should be limited to uncomplicated cases, and only when local resistance patterns permit this choice. In Hungary, the prevalence of E. coli strains resistant to sulphamethoxazoltrimethoprim is reported to be above 20% (Hungarian National Bacteriological Surveillance Management Team National Centre for Epidemiology), but considering that resistance surveillance systems may overestimate resistance patterns (Schmiemann et al., 2012), SMX-TMP use can be accepted as a rational choice in uncomplicated cystitis.

The strengths of our survey include the ability to exploit clinical data. Also, by applying common diagnostic and classification criteria, misclassification bias could be avoided, and the choice of antibiotic could be justified. A limitation of our study is the voluntary participation of GPs. As volunteering GPs may be more concerned about their rational antibiotic prescribing practices, the global prescribing patterns in Hungary might be more suboptimal than presented in this study.

### CONCLUSIONS

Our study has found similar patterns of antibiotic use in both types of cystitis (with high fluoroquinolone dominance), and identified suboptimal antibiotic use from various aspects. Patient characteristics has weakly influenced fluoroquinolone prescribing More prudent use of antibiotics in lower urinary tract infections is urgently needed.

### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

### ETHICS STATEMENT

The studies involving human participants were reviewed and approved by the Regional Human Medical Biology Research Ethical Board of the University of Szeged. The patients/participants provided their written informed consent to participate in this study.

### AUTHOR CONTRIBUTIONS

RBe, MM, ZJ, JB, EH, and ZP had the original idea for the manuscript. ZJ and EH organized data collection. RBo, MM, JB, GS, and ZP contributed to the analysis. RBo, MM, and JB drafted the manuscript, which was reviewed and approved by ZP, ZJ, GS, and EH.

### FUNDING

The study was funded by the University of Szeged.

### ACKNOWLEDGMENTS

We are grateful to all participating GPs. The authors thank Dora Bokor, PharmD, for proofreading the manuscript. Preliminary analysis of this work is present in a PhD thesis (Juhasz, 2017).

### REFERENCES


Schmiemann, G., Gagyor, I., Hummers-Pradier, E., and Bleidorn, J. (2012). Resistance profiles of urinary tract infections in general practice–an observational study. BMC Urol. 12, 33. doi: 10.1186/1471-2490-12-33

Tyrstrup, M., van der Velden, A., Engstrom, S., Goderis, G., Molstad, S., Verheij, T., et al. (2017). Antibiotic prescribing in relation to diagnoses and consultation rates in Belgium, the Netherlands and Sweden: use of European quality indicators. Scand. J. Prim. Health Care 35, 10–18. doi: 10.1080/02813432. 2017.1288680

Conflict of Interest: Author JB is employed at Grove Lodge One, though she was not at the time the study was conducted.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Benko, Matuz, Juhasz, Bognar, Bordas, Soos, Hajdu and Peto. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Knowledge Translation for Improving the Care of Deinstitutionalized People With Severe Mental Illness in Health Policy

*Izabela Fulone1, Jorge Otavio Maia Barreto2, Silvio Barberato-Filho1, Marcel Henrique de Carvalho3 and Luciane Cruz Lopes1\**

1Pharmaceutical Sciences Graduate Course, University of Sorocaba, UNISO, Sorocaba, Brazil, 2 Fiocruz School of Government, Fiocruz Brasília, Oswaldo Cruz Foundation, Brasília, Brazil, 3Veredas Institute, Brasília, Brazil

#### Edited by:

Iris Hoxha, University of Medicine, Tirana, Albania

#### Reviewed by:

Adina Turcu-Stiolica, University of Medicine and Pharmacy of Craiova, Romania Kurt Neumann, Independent Researcher, Kerékteleki, Hungary

#### \*Correspondence:

Luciane Cruz Lopes luciane.lopes@prof.uniso.br; luslopesbr@gmail.com

#### Specialty section:

This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology

Received: 02 August 2019 Accepted: 13 November 2019 Published: 21 January 2020

#### Citation:

Fulone I, Barreto JOM, Barberato-Filho S, Carvalho MH, and Lopes LC (2020) Knowledge Translation for Improving the Care of Deinstitutionalized People With Severe Mental Illness in Health Policy. Front. Pharmacol. 10:1470. doi: 10.3389/fphar.2019.01470

Background: Knowledge translation (KT) is an effective strategy that uses the best available research evidence to bring stakeholders together to develop solutions and improve public health policy-making. Despite progress, the process of deinstitutionalization in Brazil is still undergoing consolidation, and the changes and challenges that are involved in this process are complex and necessitate evidence-informed decision-making. Accordingly, this study used KT tools to support efforts that aim to improve the care that is available to deinstitutionalized people with severe mental disorders in Brazil.

Methods: We used the Supporting Policy Relevant Reviews and Trials tools for evidenceinformed health policymaking and followed eight steps: 1) capacity building; 2) identification of a priority policy issue within a Brazilian public health system; 3) meetings with policymakers, researchers and stakeholders; 4) development of an evidence brief (EB) that addresses the problem of deinstitutionalization; 5) facilitating policy dialogue (PD); 6) the evaluation of the EB and PD; 7) post-dialogue mini-interviews; and 8) dissemination of the findings.

Results: Capacity building and meetings with key informants promoted awareness about the gap between research and practice. Local findings were used to define the problem and develop the EB. Twenty-four individuals (policy-makers, stakeholders, researchers, representatives of the civil society, and public defense) participated in the PD. They received the EB to subsidise their deliberations during the PD, which in turn were used to validate and improve the EB. The PD achieved the objective of promoting an exhaustive discussion about the problem and proposed options and improved communication and interaction among those who are involved in mental health care. The features of both the EB and PD were considered to be favorable and helpful.

Conclusions: The KT strategy helped participants understand different perspectives and values, the interpersonal tensions that exist among those who are involved in the field of mental health, and the strategies that can bridge the gap between research and policymaking. The present findings suggest that PDs can influence practice by promoting greater engagement among stakeholders who formulate or revise mental health policies.

Keywords: evidence-informed policy, knowledge translation, health policy, policy-making, deinstitutionalization, mental health

### BACKGROUND

Knowledge translation (KT) is a dynamic and interactive process that uses evidence to make decisions and take actions that can improve health outcomes and reduce health inequities, particularly in low- and middle-income countries (LMICs) (Boyko et al., 2012).

Overall, there are different complexities and barriers that impede the application of KT for public health action in LMICs: deficits in knowledge production, the application of the available knowledge, and the use of strategies that are based on the best available evidence (Malla et al., 2018). When resources are scarce and there are strong sociocultural interferences, the translation and dissemination of knowledge can be adversely affected by contextual and local limiting factors (Newlin and Webber, 2015).

In order to promote the appropriate use of scientific evidence in the development and implementation of public health policies, KT platforms such as the Evidence-informed Policy Network (EVIPNet), which is supported by the World Health Organization (WHO), have been established to support health policy-making in Africa, Asia, and the Americas (Moat et al., 2014). The main objective of the EVIPNet is to facilitate the use of scientific knowledge in the formulation and implementation of health policies. Specifically, it focuses on the preparation of evidence briefs and policy dialogues, and adopts an approach that is similar to the Supporting Policy Relevant Reviews and Trials (SUPPORT) method (Moat and Lavis, 2014).

KT platforms are change agents that have a positive impact on policy decisions, interest group interactions, and health systems (Ongolo-Zogo et al., 2018). The use of KT platforms in Uganda, Cameroon, and Lebanon demonstrate the positive impact of such platforms: the promotion of awareness, acceptance, and adoption of research-based knowledge, achievement of the health goals, reallocation of resources, and identification of the sources of conflicts (Yehia and El Jardali, 2015; Ongolo-Zogo et al., 2018).

Evidence briefs should rely on the best available systematic reviews to delineate the important aspects of the issue in question. It must integrate global evidences and local knowledge to inform deliberations about health policies among policy-makers and stakeholders (Lavis et al., 2009a). Policy dialogues use the evidence brief as primary input to subsidise the deliberations followed by the views, experiences and tacit knowledge of different actors, who will be affect or involved by future decisions (Lavis et al., 2009b; El-Jardali et al., 2014; Yehia and El Jardali, 2015).

Since its inception in Brazil in 2007, EVIPINet has been focusing on promoting the use of scientific knowledge in the decision-making processes of the Brazilian Health System, the development of innovative strategies in health management, and the facilitation of technical cooperation regarding KT among the participant countries (Evipnet-Brazil, 2019). The Brazilian network consists of the representatives of different institutions and subject-matter experts (Dias et al., 2014).

Accordingly, in response to the need for and challenges in the promotion of evidence-informed health policy-making in the largest city in the state of São Paulo (Sorocaba), a working group was constituted at the University of Sorocaba in 2016. This team, which consisted of researchers, doctoral students, and health professionals, was denominated as *Seriema* (Evidence Services for Monitoring & Evaluation in Health Policy).

The Seriema group aims to suggest and contribute to health initiatives and formulate evidence-based public policies. This group works collaboratively with the Health Department of Sorocaba, which oversees 48 additional cities in São Paulo that are together inhabited by more than three million individuals (Brazil, 2018).

This group seeks to design research studies in accordance with the needs of Brazilian policy-makers specially supporting deinstitutionalization in Brazil (mainly in region of Sorocaba).

### Mental Health in the Region of Sorocaba

In the 1980s, the history of Brazilian mental health was marked by serious denunciations of mistreatment, lack of hygiene and care for patients with mental disorders who lived in psychiatric hospitals, mainly in the region of Sorocaba (São Paulo), Rio de Janeiro (Rio de Janeiro), and Barbacena (Minas Gerais) (Vidal et al., 2008; Emerich and Yasui, 2016). Social and political mobilizations that advocated for psychiatric reform and the approval of Federal law no. 10216 in 2001 accelerated the process of deinstitutionalization. It also led to the understanding that hospitalization must be the last treatment option for patients with mental disorders. Consequently, the right to receive community care services was promulgated (Brazil, 2001; Silva and Rosa, 2014).

Sorocaba has a population of approximately 671,186 inhabitants and a high human development index (0.8), and its economy is based on industries and commerce (Brazil, 2019). The city has an adequate health-care infrastructure, and its hospitals provide services to the (almost three million) inhabitants of the tertiary care level of 48 municipalities in southwest São Paulo (Brazil, 2018). These municipalities are smaller than Sorocaba, their economies are diversified, and their high human development index ranges from 0.6 to 0.8 (Brazil, 2019). Mental health care services are not available in all 48 municipalities. Therefore, these municipalities belong to a network of mental health care institutions that are connected at the primary, secondary, and tertiary level (Brazil, 2019).

The Sorocaba region housed the largest mental asylum in the country (i.e. high number of psychiatric beds) (Cayres, 2015). The seven asylums in this region were among the ten largest Brazilian asylums that had the highest mortality rate between 2004 and 2011. Most of these deaths were due to an unknown cause, and they were especially common during the colder months of the year; the age of the youngest patient who died under these circumstances was approximately 53 years (Garcia, 2012; Cayres, 2015). In addition, there was a high number of resident patients who did not have the requisite civil documentation, and the number of mental health professionals was less than half of the number that was specified by the federal legislation (Garcia, 2012; Emerich and Yasui, 2016).

During the second half of the 1990s, there were 72,514 psychiatric beds in the Brazilian public health sector. In Brazil, the number of beds had reduced to 52,962 in 2001; in 2014, there were 25,988 psychiatric beds across the 167 psychiatric hospitals that were located in the 116 municipalities of the 23 states (Brazil, 2005; Brazil, 2015). In 2014, the Psychosocial Census of the State of São Paulo identified 53 psychiatric hospitals across 39 municipalities, seven of which were located within the Sorocaba region and together housed 2,273 patients (Cayres, 2015).

On the basis of the aforementioned census data, the federal, state, and municipal bodies signed an agreement that they would ensure the gradual deinstitutionalization of patients with mental disorders and the closure of the seven asylums in the region (Brazil, 2012). However, the deinstitutionalization process did not proceed in the same manner across the different regions of Brazil. Specifically, in regions where the number of patients that were admitted to the hospitals was very high, the institutions were underequipped to provide ambulatories and community services. This demonstrated the insufficiency and fragility of the services that were available to meet the demands of the patients (Vidal et al., 2008). However, a few community mental health care services (e.g. Psychosocial Care Center, Therapeutic Residential Service, and the Back Home Federal Program) have been found to be effective (Brazil, 2015). Nevertheless, some of the key principles that have been recommended by the WHO are not adhered to, primarily due to the following reasons: insufficient funding, qualitative and quantitative human resource deficiency, poor infrastructure, a lack of political resources and intensive follow-up care, the absence of an integration between services and fragile social mobilization (WHO, 2014; Brazil, 2015).

In October 2016, the Seriema organised the first workshop on evidence-based health policy during which the deinstitutionalization of patients with mental disorders was ascribed the highest priority among all other health policy-related issues. Subsequently, the State Health Department of the Sorocaba region contacted the Seriema group with the objective of signing a partnership and helping them formulate public policies that are related to deinstitutionalization. This represented an important opportunity to subsidise the policy and collaborate with the State Health Department. This allowed them to adapt their actions and strategies to improve the care of deinstitutionalized individuals with mental disorders in Sorocaba and the neighboring regions.

Since the use of KT is one of the challenges that is currently faced by the health systems in LMICs, the present study investigated the means by which the care of deinstitutionalized individuals with severe mental disorders can be enhanced using KT tools.

### METHODS

We used the SUPPORT Tools (Lavis et al., 2009a; Lavis et al., 2009b) for evidence-informed health Policymaking, which includes the following eight steps (**Figure 1**) for KT:

#### 1) Capacity building

There was a need to conduct capacity building workshops that addressed evidence-informed policy-making and provided technical training on the use of SUPPORT tools for relevant stakeholders. Therefore, in 2016 and 2017, three workshops were conducted to provide training and raise awareness. In addition, there was the possibility of addressing topics of interest.

2) Prioritizing and supporting evidence-informed policy-making

The first step was to prioritise policy-related issues. The Seriema group provided a set of criteria that were to be used to

select important topics, and it included questions about public perceptions and the impact of the problem (see **Supplementary Materials—Table S1**).

In the first workshop, 40 participants fulfilled the criteria and they discussed their most pressing issues. Deinstitutionalization was identified as the most important health policy-related issue by the workshop participants. With regard to the means by which the care of deinstitutionalized people with severe mental disorders can be improved, the participants underscored the need for further evidence and to address policy-related challenges at both the national and regional levels. The chronology of events that have led to the current state of the mental health care systems in Brazil can be summarised as follows: (i) asylums provided inadequate services to their patients with mental disorders; (ii) there was immense pressure to shut down the seven psychiatric hospitals in the region; and (iii) important changes have been made to Brazilian mental health policies.

3) Meetings with policy-makers researchers and stakeholders

A number of meetings were organised with policymakers and stakeholders to clarify and define the problem, gather information about the status quo that could promote dialogue, and identify other key informants who could provide further insights.

4) The development of a policy brief that addresses the problem of deinstitutionalization

Once the issue of deinstitutionalization was prioritised, the focus was geared towards gathering a wide range of evidence on the various aspects of the issue. Therefore, a systematic review of literature was undertaken. First, a well-defined search strategy was used to retrieve relevant research articles from research databases. The search focused elements for policies that were related to the care of deinstitutionalized patients with mental disorders (see **Supplementary Materials Data Sheet 1**).

Between March and May 2018, we prepared a policy brief, which defined the problem and five evidence-based options to address the issue of deinstitutionalization. The evidence was contextualized to the Brazilian scenario, based on the recommendations of the policy-makers, subject-matter experts, and experts in the field of mental health.

#### 5) Facilitating policy dialogue

The policy brief was circulated to the participants 30 days prior to the dialogue to inform them of the deliberations of the meeting. A group of 24 individuals, which entailed an equal representation of policy-makers, health-care providers, researchers, and representatives of the community and public defense sectors, participated in the policy dialogue (see **Table 1**).

The dialogue was conducted in accordance with the method that has been described by the SUPPORT tools and Chatham House rules. It was intended to achieve the following: participant commitment and transparency, an appropriate duration of dialogue, adequate group size and representation of the participants, skilful facilitation of problem-focused discussions (i.e. five options to address the policy issue), equity, key implementation considerations, and role distribution.

TABLE 1 | A profile of the stakeholders who participated in the policy dialogue.


aPolicy-makers at the federal, state, and municipal level; bHealth care providers included mental health specialists, public health specialists, psychologists, psychiatrists, occupational therapists, nurses, and social workers; cResearchers from Brazilian public and private universities, EVIPNet-Brazil members, and Seriema members; dThe Brazilian anti-asylum movement; ePublic defense representative from the state of São Paulo who was involved in mental healthrelated legislations.

#### 6) The evaluation of the evidence brief and policy dialogue

The evaluation of the evidence brief and the policy dialogue was based on an adapted version of Lavis (2009) (Lavis et al., 2009a; Lavis et al., 2009b). Specifically, two surveys were administered to the participants (i.e. prior to dialogue and during the dialogue for those who did not complete it the first time). It consisted of items that required the respondent to assess the evidence brief and indicate the extent to which the policy dialogue was helpful on a rating scale that ranged from 1 (very unhelpful) to 7 (very helpful).

#### 7) Post-dialogue mini-interviews

During the policy dialogue, the stakeholders were invited to participate in a video-recorded interview. In this interview, they were required to describe the insights that they gained from the dialogue. For this purpose, we posed the following two questions: a) How did the policy dialogue change your perspective about the problem in question? and b) What actions should be taken to address the problem in question?

### 8) Dissemination of the findings

The evidence brief was uploaded to the EVIPNet-Brazil secretariat webpage (http://brasil.evipnet.org/), where it is currently available for free download by all who are interested. A summary of the evidence brief and the policy dialogue will also be made available. Further, the federal government will order 100 prints of the evidence brief.

### RESULTS

The results that are presented in the following sections summarise the main findings that pertain to the evidence brief; this section is followed by a discussion of the results that belong to the policy dialogue.

### Defining the Problem

*What are the most important challenges that impede the improvement of mental health care that is available to deinstitutionalized people with severe mental disorders in Brazil?*

The participants reviewed the findings that were presented in the evidence brief, highlighted what is already known about the problem, and provided an enriching analysis of the brief; this process consumed the most time. They individually and collectively focused on the prominent challenges: (i) insufficient and fragile community care services to meet patient needs; (ii) unequal access to community care across the different regions of Brazil; (iii) insufficient funding and a lack of political resources; (iv) qualitative and quantitative human resource deficiencies; (v) a lack of intensive follow-up care; and (vi) the absence of integration and communication between services.

All the participants agreed that it is necessary to expand and strengthen community care services for all Brazilians. Indeed, the process of deinstitutionalization did not progress in the same manner across different Brazilian regions. In some of them, such as region of Sorocaba (main manicomial pole), where the number of patients admitted to hospital beds was very high, the deinstitutionalization process exceeded the capacity of assimilation of services offered in community.

Some participants observed that, despite progress, community care services are still precarious with regard to a wide range of issues (i.e. from physical infrastructure to human resources). They noted that many professionals still retain an "asylum mentality," and that there is insufficient communication among mental health professionals and services, and between the municipal, state, and federal governmental bodies. The participants contended that the lack of communication and continued education adversely affects the follow-up care and rehabilitation of patients with mental disorders.

The participants expressed their concerns about the process of deinstitutionalization (i.e. the withdrawal of patients from psychiatric hospitals) and trans-institutionalization (i.e. the transfer of patients from asylums to other inappropriate institutions). Indeed, these can lead to social neglect and have profound repercussions for the community, such as increased rates of homelessness, incarceration, drug addiction (primarily, cocaine), depression, suicide, and an overloading of emergency services. All participants agreed that deinstitutionalization requires efforts that extend beyond deinstitutionalization and trans-institutionalization. They also agreed that the "Ministry of Health must have a serious commitment to those patients who leave the psychiatric hospitals."

The participants contended that the issue of deinstitutionalization is also complicated by financial conflicts of interests that pertain to psychiatric hospitalizations. This suggests that there is a "mercantilization of life of an especially vulnerable population." Finally, the participants also recognised that the health care that is available to deinstitutionalized individuals has significantly advanced across the years; however, the socio-cultural treatment of these individuals remains problematic.

### Options to Address the Problem of Deinstitutionalization

The five mutually non-exclusive options to address the problem of deinstitutionalization that was articulated in the evidence brief are presented in **Table 2**.

TABLE 2 | The definitions of the options to address deinstitutionalization that were presented in the evidence brief.


The deliberations that pertained to the options are summarised in the following sections.

### Option 1: Expand and Improve the Implementation of a Psychiatric Hospital Day

This option caused much polemic and controversy among the participants of the policy dialogue, possibly because of a misunderstanding of the option. A majority of the participants opposed this option because they considered traditional psychiatric hospitals to be regressive: "*something that did not work in the past, which isolates and excludes*." At the same time that the policy dialogue was conducted, the national policy on mental health was being reformulated with a strong aim to reopen the psychiatric hospitals; evidently, many of the participants were aware of this. However, other participants understood this option more accurately and were in favor of such an approach because it entails the early discharge policy of psychiatric day hospitals. However, they suggested that the name of the option be changed to "Strengthening interventions for acute psychiatric episodes" in order to convey that this option endorses institutions that provide humane treatment to individuals who present with acute psychiatric episodes, and hospitalize briefly such individuals only when necessary.

### Option 2: Provide Psychoeducational Programs

This option was wholeheartedly supported and endorsed by the participants. Further, a majority of the studies that were reviewed supported the effectiveness of this option. The Brazilian Health System does not offer psychoeducational programs. According to some of the participants, this may have been attributable to the preconceived notions that managers hold about mental health professionals. They also recommended the implementation of a few psychoeducational techniques.

#### Option 3: Develop Community Mental Health Teams

Only one of the systematic reviews (Malone et al., 2007) addressed this option. Nevertheless, the conclusions of the review suggested that this option promotes greater acceptance of the treatment and greater patient satisfaction, when compared to standard treatment paradigms. In addition, the hospitalization rate was significantly lower; this suggests that the number of suicides and deaths under suspicious circumstances was also lower. The participants considered this option to be interesting and promising. However, the Brazilian mental health policy does not have provisions for such community mental health teams. Although Brazil does have other community teams, they comply with only a few of the principles of the proposed team.

### Option 4: Implement and Monitor the Practice of Intensive Case Management

Model that is similar to those of intensive case management are practiced in some communities in Brazil. Every participant considered this model to be extremely important to all Brazilian cities. However, several small towns do not comply with this model. Therefore, the participants highlighted the importance of expanding and strengthening this model.

### Option 5: Promote Assisted Living

The participants underscored the importance of and challenges that are involved in implementing assisted living in such a manner that it does not result in trans-institutionalization.

Two participants observed issues that pertained to the inadequacy of housing, infrastructure, and food, and the absence of leisure-time activities.

Many participants agreed that cohabitating a space with individuals who differ in age, diagnosis, and the severity of the diagnosis facilitates social reintegration: "caring and helping each other are positive factors observed in their daily lives."

### The Evidence Brief and Policy Dialogue: Evaluation Results

Eight and nine individuals out of the 24 participants completed the evaluation surveys for the evidence brief and policy dialogue, respectively. The response rate was low despite repeated attempts to administer the survey, and it can be attributed to time limitations and the busy lives that the participants led.

Despite the low response rate, the average item scores were positive, and they ranged from 5.0 to 7.0 for the evidence brief evaluation survey. The features that received the highest ranking (i.e. very helpful) were as follows: employ a graded-entry format

and use systematic and transparent methods to identify, select, and assess synthesized research evidence (see **Supplementary Materials—Table S2**).

The results of the policy dialogue evaluation were also positive, and the scores ranged from 4.6 to 6.6. The following features were considered to be very useful: rely on a facilitator to assist with the deliberation, address high-priority policy issues, do not aim for consensus, provide an exhaustive discussion, and ensure a fair representation of those who will be involved in or affected by future decisions that are related to the respective issue (see **Supplementary Materials—Table S3**).

### Post-Dialogue Mini-Interviews

Approximately 10 individuals agreed to participate in the postdialogue mini-interviews, which were video-recorded. The findings of the study suggest that many participants demonstrated the positive insights that they gained during the policy dialogue (see **Table 3**).

## DISCUSSION

### Main Findings

The application of KT tools to support efforts to improve the care of deinstitutionalized patients with mental disorders and to contribute to the promulgation of evidence-informed mental health policies was a promising and innovative experience in Sorocaba. This experience entailed eight steps, and it demonstrated to policy-makers that the process of KT can bridge the gap between research and practice.

TABLE 3 | Participant opinions (insights) about the policy dialogue.


The application of evidence in mental health practice and the exchange of knowledge between health-care providers, researchers, and community representatives were positively appraised. The entire process also helped those who are likely to be involved in or be affected by future policy-related decisions gain valuable insights. The features of the evidence brief and policy dialogue were considered to be very helpful, and they believed that it promoted an exhaustive discussion about the issue of deinstitutionalization.

### A Comparison of the Present and Past Findings

Capacity building, which was the first step of the process, made the participants aware of the importance of the following: using KT tools to make evidence-informed policy decisions, align research at the University of Sorocaba with policy priorities, and build partnerships between policy-makers, stakeholders, and researchers. Training workshops have been found to improve knowledge and comprehension about the use of evidence in policy decision making in other countries as well (Uneke et al., 2012; Waqa et al., 2013; El-Jardali et al., 2014). The workshops also strengthened partnerships and enhanced the interaction between the Seriema group and the Health Departments of Sorocaba and the neighboring regions.

The evidence brief was prepared based on the best evidence available on the issue at hand. However, a majority of the systematic reviews focused on high-income countries (e.g. the United States of America, the United Kingdom, Canada, Australia), and none of them were conducted in Brazil. This demonstrated a knowledge gap regarding mental health care in Brazil (Amaral et al., 2018; Votruba et al., 2018). This led to many difficulties because the relationship between evidence and policy-making depends on country-specific features (e.g. social, organizational, and public factors), the specific policy issue, resources allocation, and contextual factors, which are very different (and in some cases, deficient) in LMICs (Tricco et al., 2013; Votruba et al., 2018). This difference can be attributed to the following features that characterise LMICs: low research capacity, an obscure policy-making process, a high risk of political instability, limited financial resources, a lack of interaction between researchers and policy-makers, and lack of empowerment of civil society (Young, 2005).

Furthermore, our findings corroborate the gap between research and practice that has been observed in LMICs, as well as the difficulties and complexities that mental health care entails. Despite the global burden of mental disorders (e.g. disability and lower disability-adjusted life years), mental health is not a policy priority in LMICs (Patel, 2007; Votruba et al., 2018). Mental health policy issues differ from other policy issues because they pertain to a highly heterogeneous set of conditions (i.e. mental, behavioral, or neurodevelopmental disorders), the presence of comorbidities, a lack of consensus on the best possible approach to treatment and care, a high rate of untreated patients, and the incumbent stigma (Votruba et al., 2018).

The definition of the problem and the options were discussed exhaustively, without the aim of reaching a consensus. The problem was perceived to be critical, and many of the participants (policymakers, health-care providers, researchers, and representative of civil society, and public defense) conceptualized the problem based on their rich practical experience, and they echoed a majority of the challenges that were already presented in the evidence brief. In other words, the policy dialogue deliberations validate the evidence brief (Yehia and El Jardali, 2015). Thus, it is noteworthy that option 2 (*Provide psychoeducational programs*) was strongly supported by findings as well as the participants. On the other hand, option 1 (*Expand and improve the implementation of a Psychiatric Day Hospital*) was strongly opposed by a majority of the participants due to local findings; further, there were differences of opinion between international and local researches. Many of the participants were aware of the grave and inhumane treatment that patients with mental disorders had been subjected to in psychiatric hospitals in this region; they were also cognisant of the struggles that were required to shut down all the hospitals. The regulation of care with regard to crisis management and the treatment of acute episodes appear to be the most unclear albeit critical aspects of mental health care in Brazil (Amaral et al., 2018).

There is no KT strategy that is singularly effective across all contexts. Therefore, it is important to report about the contextspecific utility of each strategy, so that they can be modified and utilised by other interested decision makers (Larocca et al., 2012). In this study, the participants provided positive evaluations of the evidence brief and of policy dialogue; they considered it to be favorable and useful, and these results corroborate past findings (Yehia and El Jardali, 2015; Boyko et al., 2016; Mc Sween-Cadieux et al., 2018). Similar findings emerged from the miniinterviews that were conducted at the end of the policy dialogue; specifically, all participant opinions were positive in tone. The use of a facilitator be to assist with the deliberation was considerate the most helpful feature of the policy dialogue. Past findings corroborate these results and emphasize the role of the facilitator as an unbiased agent which support KT platform (El-Jardali et al., 2014; Yehia and El Jardali, 2015).

Evidence briefs and summaries of policy dialogues (i.e. products of KT) can be used in public health policy-making only if the local and federal authorities are receptive to such efforts; unfortunately, often not the case (Cabieses and Espinoza, 2011).

Although the application of KT in public health policy-making is relatively new in LMCIs, the situation is changing. There is an increased use of evidence-informed policy frameworks (Cabieses and Espinoza, 2011; Votruba et al., 2018) and an increased demand for KT products from policy-makers. This has been proven by the EVIPNet-Brazil, which has expanded and consolidated its network (Dias et al., 2014). This practice needs to become a priority for Brazilian policy-makers because evidencebased public health models are powerful frameworks that can be used to identify the most effective health strategies and ensure that the resources are spent appropriately (Milat and Li, 2017).

### Limitations and Strengths

The present study was the first attempt to use KT tools to improve some aspects of mental health care in Brazil (e.g. deinstitutionalization), which is a priority topic of regional and national importance. The policy dialogue brought together stakeholders who are involved in the process of deinstitutionalization (e.g. researchers, policy-makers, healthcare providers, and representatives from public defense and civil society), which enriched the deliberations and provided the participants with an opportunity to acquire new knowledge and learn from each other.

The present study has a few limitations. A large part of the KT framework and the best evidence available were developed in high-income countries (e.g. the United Kingdom, Canada, Australia) that's can bring indirectness evidence. Further, we could not examine budgetary impact because the studies did not present cost analyses. Additionally, some of the options that were identified were difficult to understand because they were articulated using obscure terminologies. The variability in the quality of the reviewed studies and the lack of information about the options that can be implemented are a few other limitations. The low response rate that was evidenced for the evidence brief and policy dialogue evaluation surveys was attributed to time limitations and the busy lives that our participants led; therefore, some of our results may be underestimated. Although we have conducted an exhaustive and in-depth discussion, some topics that pertained to implementation were not discussed due to the paucity of time. However, since some aspects of implementation vary across communities, they should be discussed in accordance with the conditions of each municipality.

### CONCLUSIONS

The KT process that was adopted was considered to be a useful means to discuss important policy issues, bring together policymakers, health care providers, researchers, and representatives of civil society and public defense, enhance interaction and partnerships between evidence-producers and evidence-users, and promote the dissemination and application of global and local evidence in practice.

### REFERENCES


The present study did not seek to examine causal relationships. Nevertheless, a longer study period will allow future researchers to capture the positive changes in mental health care that result from KT. Future investigations are required to understand whether and how evidence briefs and policy dialogue can be used to improve the care of deinstitutionalized people with severe mental disorders and their contributions to Brazilian mental health policy.

Researchers and other stakeholders who are interested in using KT tools should consider the lessons that were learnt during the course of our study.

### AUTHOR CONTRIBUTIONS

LL conceptualized the study. IF, JB, LL designed the study. IF, SB, MC and LL led data collection, carried out the analysis and drafted the initial manuscript. All authors read (IF, LL, JB, SB and MC) provided critical revision and approved the final manuscript.

## FUNDING

This article was funded by FAPESP grants 2017/20668-7 and EVIPNet Brazil/Ministry of Health SCON2017-02502.

### ACKNOWLEDGMENTS

The authors thank the Health Departments of the Sorocaba region.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01470/ full#supplementary-material

Reforma psiquiátrica e política de saúde mental no Brasil. Documento apresentado à Conferência Regional de Reforma dos Serviços de Saúde Mental: 15 anos depois de Caracas. OPAS. Brasília: DF, 2005.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2020 Fulone, Barreto, Barberato-Filho, Carvalho and Lopes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*