Original Research ARTICLE
Post-marketing health technology monitoring. The analysis of an experience from a clinical perspective
- 1 Basque Office for Health Technology Assessment (Osteba), Department of Health and Consumer Affairs, Basque Government, Vitoria-Gasteiz, Spain
- 2 Department of Gastroenterology, Hospital Galdakao-Usansolo, Galdakao, Spain
- 3 Department of Gastroenterology, Complexo Hospitalario Universitario de Vigo, Vigo, Spain
- 4 Department of Gastroenterology, Hospital Manises, Manises, Spain
- 5 Department of Gastroenterology, Hospital Universitario Germans Trias I Pujol, Badalona, Spain
- 6 Department of Gastroenterology, Hospital Son Espases, Palma de Mallorca, Spain
- 7 Patient Safety Promotion Service, Catalan Department of Health, Barcelona, Spain
- 8 AVALIA-T, Galician Agency for Health Technology Assessment, Department of Health, Galician Government, Santiago de Compostela, Spain
Introduction: A system for monitoring the use of aphaeresis in the treatment of ulcerative colitis (UC), named system for monitoring aphaeresis in ulcerative colitis (SiMAC), was designed in 2006 in the Basque Country. In the present study, the opinion of the clinicians who participated in SiMAC was evaluated, in order to identify the barriers and gather suggestions that could improve implementation of this kind of system. Methods: A mixed questionnaire was designed, in order to gather clinicians’ assessments of the SiMAC monitoring system. Results: The response rate was 73.9% (17/23). The data from 40.96% (159/388) of patients with UC treated with aphaeresis was recorded. The main reasons for not including the data from all treated patients were a lack of required data or not meeting the study inclusion criteria. Positive aspects of the SiMAC were identified, as the simplicity of data collection and its systematic, multi-center approach. The negative aspects mentioned were the use of a local computer application and the lack of time for health professionals to enter data. Discussion: The use of monitoring systems helps to formalize the introduction of technologies of little-known effectiveness; involve clinicians and medical societies in coming to agreement and obtaining information about the safety, effectiveness or efficiency of new technologies; and provide relevant information to healthcare administrations for making decisions about the introduction of new technologies into healthcare practice. In order for a monitoring system to work, the process must be straightforward. A minimum set of key variables that are easy to collect must be selected, and an effort made to involve a range of stakeholders, especially institutions and scientific societies, to support the work group.
Health technology assessment (HTA) is designed to provide information, based on scientific knowledge and a real-life context, to inform decision-making at various levels in the health system. However, when making decisions about the introduction of new technologies, there may be doubts due to a lack of information about effectiveness or because, when using the technology under normal conditions, problems, or side effects arise that were not predicted in the preliminary studies (Varela Lema et al., 2007).
In the case of medical devices, the CE mark guarantees that the product meets the requirements of the European Directives designed to ensure their safety and performance. However, the CE mark does not guarantee the clinical effectiveness or cost effectiveness of these products. For this reason, various initiatives have been implemented in the past decade with the aim of observing when and how technology is used in real conditions, or in order to carry out studies in which the safety and effectiveness of the technology is assessed within a specific population or scenario (Frønsdal et al., 2010). In Spain, a procedure called “Coverage under review,” has been designed for assessing technologies with high potential for producing substantial improvements in health or in quality of life, but for which there is not sufficient information to make pronouncements about their safety, effectiveness, and efficiency. This procedure is used before the technologies can be funded in general terms by the public health system.
In the Basque Country and Galicia, specific regulations determine that, depending on the available scientific evidence, some new technologies may be considered as subject to special limitations and controls or to special monitoring. This means that, following authorization for use, they remain subject to the specific indication, implementation, and assessment criteria established by the relevant bodies.
Under this law, the Basque Office for Health Technology Assessment (Osteba) produced an assessment report on the use of aphaeresis systems in treating inflammatory bowel disease (IBD; Gutiérrez-Ibarluzea and Arcelay, 2004), technology that has borne the CE mark since 19991. This report suggested that the technology be monitored for treating ulcerative colitis (UC), given the lack of scientific evidence found about its effectiveness and the short monitoring periods previously implemented, which are inappropriate for detecting safety problems in the medium- to long-term. A research project was consequently designed, using a computer application known as the system for monitoring aphaeresis in ulcerative colitis (SiMAC, abbreviated from the Spanish; Ibargoyen-Roteta et al., 2006). This project was based on the prior related recommendations of the Spanish Work Group on Crohn’s Disease and Ulcerative Colitis (GETECCU); Cabriada et al., 2006).
Twenty-three hospitals from all over Spain participated in this project and data were collected from 195 patients with UC treated with this type of system.
Because this was the first system designed to monitor the use of a new technology, the present work aims to reflect the opinions of the clinicians who participated in data collection via SiMAC, with the aim of identifying the difficulties that were encountered and gathering suggestions about aspects that could improve the implementation and participation of clinicians in data collection of other monitoring systems.
Materials and Methods
Clinicians responsible for collecting data from the 23 hospitals participating in SiMAC.
Structure of the Questionnaire
A mixed questionnaire was designed, with two sections (see Figure 1). The first section, labeled “Data of interest,” contained questions related to the hospital, the situation in terms of IBD and UC, and the use of aphaeresis systems in each of the hospitals.
Figure 1. System for monitoring aphaeresis in ulcerative colitis assessment questionnaire. The outcomes of the questionnaire had not been validated.
The second part of the questionnaire was designed in order to evaluate the SiMAC by the participating clinicians. This section dealt with topics such as the suitability of the variables defined for treatment monitoring, the most appreciated aspects of the system, the difficulties encountered, and the measures that could improve participation of health professionals in this type of monitoring system.
Transversal, descriptive study, carried out at the end of the SiMAC data collection by sending an assessment questionnaire via email to the representative health practitioner at each hospital. The email was used because of difficulties to contact health professionals by telephone. The questionnaire was first sent on 15.12.2009. Two weeks later, a reminder was sent to those hospitals that had not answered. A certificate accrediting participation in the assessment study was sent to all practitioners who had responded to the questionnaire.
The numerical data related to the hospital and the number of registered patients with IBD and UC, as well as the number of patients treated with aphaeresis, have been summarized in the form of absolute and relative frequencies. The answers to the open questions have been described in the text.
Participation in the Study
The response rate to the questionnaire was 73.9% (17/23).
Figure 2 shows the location of the hospitals that participated in the SiMAC, with those that responded to the assessment questionnaire shown in black.
Figure 2. Distribution of participating hospitals according to whether or not they responded to the questionnaire.
Data of Interest about Participating Hospitals
These are tertiary hospitals, for which the population coverage, the number of registered patients with IBD and specifically, with UC, are presented in Table 1.
Table 2 shows the year in which aphaeresis treatment was first used in each hospital, the number of patients with treated IBD and UC and the number of treated UC patients included in SiMAC. It can be observed that there are six hospitals that began using aphaeresis systems before 2004.
SiMAC Assessment by Clinicians
Table 2 reveals that 8 of the 17 hospitals included less than 50% of the UC patients that they had treated with aphaeresis. When asked why they had not included all the patients in SiMAC, of the 14 centers that answered this question, eight blamed a lack of data about these patients; three explained that they were patients with Crohn’s disease or indeterminate colitis; another four indicated that it was due to the inclusion criteria not being met (fundamentally, because no endoscopy had been performed, which was necessary for calculating the level of UC activity within the stated time periods); and one cited a work overload and not knowing how to involve the other department which was responsible for the treatment, in the data collection.
Six clinicians answered the question about the existence of specific indications for the use of aphaeresis, with each citing different indications: steroid-dependent or -resistant IBD; UC in general; steroid-dependent UC following the failure of immunosuppressive agents; steroid-dependent or resistant UC; as a treatment prior to surgery; or in mild to moderate and steroid-dependent UC.
Only two participants considered that the variables that had been defined for treatment monitoring were not the right ones. One justified this by saying that there are too many and that some do not provide information that is of interest to the study, which hinders data collection.
Of the 15 participants who considered the defined variables to have been well-chosen, three suggested making some changes related to defining collection times, with a clearer definition of the possible indications of aphaeresis and the use of an activity index that does not require endoscopy. With regard to those aspects of the SiMAC that received the best feedback, five participants appreciated the design of a systematic collection method for patient treatment, monitoring, and control; three referred to the simplicity of data collection and use of the computer application; four highlighted the multi-center nature of the study; and finally, two participants referred to the persistence of the sponsor group and the ease of communication with the same.
The difficulties encountered by participants were the following: seven had some kind of problem installing the application or with data-sending; three stated that they did not have time to enter the data; another three participants pointed to their lack of information about patients for whom the data was collected before the start of the project; two indicated that they had too many variables to collect; and one claimed to have had sparse information about the results of their patients in the medium- and long-term.
Incentives or measures pointed out to improve participation in a monitoring system were the following: provide a data manager or person to collect the required data (four participants); have a permanent database to facilitate publications (three participants); facilitate the inclusion of more patients in the database (three participants); prioritize one or more meetings between researchers and participants (three participants); reduce the work load of healthcare professionals (one participant); financial and curricular incentives (one participant); and the creation of an online (rather than local) database for data entry (one participant).
Of the 14 participants who answered the question on the participation of industry, one believed that industry players should not participate or, if they did, should do as little as possible; five participants considered that they could provide the material (the aphaeresis columns, in this case) or finance treatment in centers that are without coverage; four suggested that they could provide data managers; another participant suggested that they could take charge of maintaining the project database and webpage; and another that they could send reminders to participants to collect the pending data. One participant pointed out that the legal framework would pose barriers to industry players providing this type of help.
With regard to how to improve data collection for technology monitoring, six participants suggested that an online database could be designed, easier to use and with less variables; three proposed that the care load of the healthcare professional could be reduced or that the research could be included as part of their professional responsibilities; one indicated that having an electronic patient record that already includes these data would facilitate data collection; another that it would be improved by creating databases into which all treated patients could be entered (and not only some cases); and finally, one considered that the process would improve if it was done prospectively, using a data collection program with an easier interface. All these results are presented in Table 3.
Some options for obtaining the necessary information for monitoring health technology include the use of administrative databases, surveillance activities, or databases obtained from more specific activities, such as records of particular technologies or diseases (Frønsdal et al., 2010). In the present case, in order to monitor the effectiveness and safety of aphaeresis systems in treating UC, the patients treated with these systems were specifically registered. An installable application was designed for this purpose, with variables defined in advance by a group of IBD experts from the GETECCU group (Ibargoyen-Roteta et al., 2006). The reason for developing an installable application was to provide every participating center with the option of maintaining and exploiting their own local database. It was considered that this might be an incentive for increasing the level of participation in the SiMAC. However, once the evaluation study had been completed, it became clear that participants had had some problems, specifically with installing the application and sending data to the central database. This is a very important point, as it may have influenced the data collection and the ease of the whole process. In fact, as indicated by some participants, the use of an online application, as well as the existence of the electronic patient record (Varela Lema et al., 2007) are options that could have simplified data collection. The ease of data collection has a major impact on whether or not participants meet the deadline for inclusion of data from any study of this type (Ministerio de Sanidad y Consumo, 2007).
Even though initially SiMAC was designed for collecting data in a prospective manner, it was decided that the inclusion of data from those patients who had started aphaeresis treatment before the start of the study should be allowed, since many hospitals had already started to use these systems prior to 2004.
When working with technologies used in pathologies or conditions with low incidence, or with applications in a small number of patients, national and international collaborations could facilitate the data collection process. In the present case, an attempt was made to involve the greatest possible number of Spanish hospitals in order to collect data from the greatest possible number of patients, and thus build a sufficiently large sample as to be able to draw conclusions that are relevant to clinical practice.
This type of collaboration facilitates mechanisms for sharing protocols, and even data on results from individual patients, which makes it possible to draw more reliable conclusions within the shortest possible time. In this respect, it is worth mentioning the effort carried out at European level by the EUnetHTA network in order to describe the situation in Europe in this area, and search for parameters for standardizing the design and operation of a monitoring system2.
With regard to the low percentages of UC patients treated with aphaeresis systems that were included by some centers, clinicians state that this was mainly due to the impossibility of filling in the data for some of the variables required in SiMAC. According to opinions presented in other studies, focusing the monitoring on a minimum set of key data is essential in order to be able to meet the proposed target (Frønsdal et al., 2010). In fact, the clinicians who participated in SiMAC proposed reducing the number of variables to be collected for each patient, and replacing the index used for measuring UC activity with another that would not require endoscopy.
Another of the issues flagged by participants as a factor hindering data collection is their lack of time for dedicating themselves to this type of task. Reducing the care load for professionals participating in this type of initiatives, or including the research as an activity that is part of standard hospital tasks, are some of the suggestions made to remedy this. In this regard, the potential role of the electronic patient record should not be overlooked. This is starting to be used, and should be sufficiently flexible so as to be able to include the data necessary for monitoring studies, and from which it should be easy to extract data for analysis, without overloading medical practitioners even more. Study participants also suggested contracting data managers as another possible solution, although this would pose a funding problem for healthcare organizations, since the funds dedicated to these ends are limited (Frønsdal et al., 2010).
The funding obtained for carrying out the present work was restricted, since the design and setting up of the SiMAC monitoring system was funded through the Spanish national health system’s Quality Plan 2006, but not the monitoring of the data collection by the hospitals participating in the study. The project was able to be completed thanks to the involvement of GETECCU, which enabled closer contact with clinicians, provided information about the project in its annual meeting and promoted data collection. Increased funding would have made it possible to hold regular meetings between the principal researchers and the participating professionals, which the latter raised as an aspect that could improve the functioning of this type of monitoring system.
Industry players also participated in completing the study to a certain extent, by personally reminding the participants, during their visits, to complete the monitoring of the patients that they had included in the records. As a possible solution to the funding problems, the European Commission signaled that technology monitoring could be funded by the industry in general (European Commission, 2004, 2009), despite the fact that it has already invested large amounts of money in the process of technology acceptance by the competent authorities and in safety monitoring systems. In the present case, only one participant held the opinion that industry should not participate, or that it should do so as little as possible. However, most participants offered differing visions of how industry could participate, such as facilitating the contracting of data managers or funding the columns or the treatment. Whatever the case, it would be fundamental to recognize whether industry participation influenced data collection in any way, disclosing the existence or absence of conflicts of interest. There are currently a variety of initiatives for bridging the “know-do” gap and for implementing health technologies in which agreements are reached between service providers and technology producers. In this vein, NICE (the UK’s National Institute for health and Clinical Excellence) has proposed mechanisms for improving access to new treatments (Raftery, 2001); and joint risk agreements with industry have now been developed in order for health technologies to be introduced in a prudent manner, while simultaneously reducing timescales (Hutton et al., 2007; Klemp et al., 2011; US Department of Health and Human Services, 2011).
Finally, while keeping in mind the small sample size (despite having included the opinions of almost 74% of the healthcare professionals), we can conclude that the use of monitoring systems can help us to: (a) create protocols for the introduction of technologies of little-known effectiveness, so that it is better controlled and has a better guarantee in terms of patient benefits; (b) involve clinicians and medical societies in agreements about and obtainment of information on new technologies; (c) obtain corroborated, objective information about the safety, effectiveness, and efficiency of new technologies; and (d) provide relevant information to the health administrations for making decisions about the introduction of new technologies into medical practice.
However, in order for a monitoring system to work, the following points need to be taken into account: (a) the necessity of facilitating the process; (b) the selection of a minimum set of relevant variables that are easy to collect [methods for identifying outcomes of interest could be used, such as the one developed by GRADE (Ibargoyen-Roteta et al., 2010)]; and (c) that the involvement in these systems of different stakeholders is fundamental, particularly institutions (recognition of healthcare professionals’ work and the necessity of funding the procedure) and scientific societies (support for the work group), an example of which is the role played by GETECCU in this project.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors would like to thank all the clinicians who helped with data collection and all those who responded to the evaluation questionnaire on the system for monitoring aphaeresis in treating ulcerative colitis (SiMAC): Benito González, from the Hospital Juan Canalejo; Antonio Arín, from the Hospital de Navarra; Mariam Aguas, from the Hospital Universitario La Fe de Valencia; Jesús Barrio, from the Hospital Río Hortega; Beatriz Merino, from the Hospital Gregorio Marañón; Nieves Martínez, from the Complejo Hospitalario Universitario de Albacete; Antonio Pereda, from the Hospital Universitario Infantil La Fe de Valencia, Oscar Roncero, from the Hospital La Mancha Centro; Ingrid Ordás, from the Hospital Clinic; Esther García, from the Hospital Santa Creu i Sant Pau; Fernando Muñoz, from the Hospital de León; Gonzalo Gómez, from the Hospital Doce de Octubre; María Esteve, from the Hospital Mutua de Terrasa and Yamile Zabana, from the Hospital Germans Trias i Pujol.
Nora Ibargoyen-Roteta participated in designing the questionnaire, collecting and analyzing data, and in writing the article. José Luis Cabriada-Nuño participated in revising and completing the questionnaire and in writing the article. Iñaki Gutiérrez-Ibarluzea participated in designing the questionnaire and writing the article. Vicent Hernández-Ramírez and Daniel Ginard-Vicens participated in completing the questionnaire and writing the article. Eugeni Doménech-Morral, Juan Clofent Vilaplana, Gloria Oliva Oliva, and Teresa Queiro Verdes participated in writing the article.
Cabriada, J. L., Doménech, E., Gomollón, F., González-Carro, P., González-Lara, V., Hinojosa, J., Jiménez-López, C. E., Nos, P., Obrador, A., Panès, J., Saro, C., Varea, V., Lafuente, R., and Guilera, M. (2006). Documento de consenso en el uso de la granulocitoaféresis en pacientes con enfermedad inflamatoria intestinal. Gastroenterol. Hepatol. 29, 85–92.
European Commission. (2004). DG Enterprise. Guidelines on Post-Market Clinical Follow-up. MEDDEV 2. 12-2. May. Available at: http://ec.europa.eu/consumers/ sectors/medical-devices/files/medde v/2_12-2_05-2004_en.pdf [accessed 28 February, 2011].
European Commission. (2009). DG Enterprise, and Industry. Guidelines on a Medical Devices Vigilance System. MEDDEV 2. 12 rev 6. December. Avaialble at: http://ec.europa.eu/consumers/sect ors/medical-devices/files/meddev/2_12_1-rev_6-12-2009_en.pdf [accessed 28 February, 2011].
Frønsdal, K. B., Facey, K., Klemp, M., Norderhaug, I. N., Mørland, B., and Røttingen, J. A. (2010). Health technology assessment to optimize health technology utilization: using implementation initiatives and monitoring processes. Int. J. Technol. Assess. Health Care 26, 309–316.
Gutiérrez-Ibarluzea, I., and Arcelay, A. (2004). “Leucocitoaféresis para el tratamiento de la enfermedad inflamatoria intestinal.” Informe de Revisión. Vitoria-Gasteiz. Departamento de Sanidad. Informe n Osteba IR-04–3. Gobierno Vasco.
Ibargoyen-Roteta, N., Gutiérrez-Ibarluzea, I., Cabriada-Nuñez, J. L., Clofent-Vilaplana, J., Ginard-Vicens, D., Doménech-Morral, E., Oliva-Oliva, G., and Queiro-Verdes, T. (2006). Establecimiento de Estándares, registro y análisis de casos de tratamiento de la colitis ulcerosa mediante sistemas de aféresis. Informes de Evaluación de Tecnologías Sanitarias: OSTEBA N 2006/05. Madrid: Plan Nacional para el SNS del MSC, Agencia de Evaluación de Tecnologías Sanitarias del País Vasco. Available at: http://www9.euskadi.net/sanidad/osteba/datos/e_06_05_Aferesis.pdf [accessed 28 February, 2011].
Ibargoyen-Roteta, N., Gutiérrez-Ibarluzea, I., Rico-Iturrioz, R., López-Argumedo, M., Reviriego-Rodrigo, E., Cabriada-Nuño, J. L., and Schünemann, H. J. (2010). The GRADE approach for assessing new technologies as applied to apheresis devices in ulcerative colitis. Implement. Sci. 5, 48.
Ministerio de Sanidad y Consumo. (2007). El uso tutelado como mecanismo de actualización de las prestaciones: resultados de la experiencia piloto. Madrid: Ministerio de Sanidad y Consumo. Available at: http://www.msps.es/profesionales/ prestacionesSanitarias/CarteraDeSe rvicios/ActualizacionCS/docs/UsoT uteladoExpPiloto.pdf [accessed 28 February, 2011].
US Department of Health and Human Services. (2011). Coverage with Evidence Development. Available at: http://www.cms.hhs.gov/Cover ageGenInfo/03_CED.asp [accessed 28 February, 2011].
Lema, L., Ruano Raviña, A., Cerdá Mota, T., Blasco Amaro, J. A., Gutiérrez-Ibarluzea, I., Ibargoyen_Roteta, N., Blasco-Amaro, J. A., Imaz, I., Sampietro-Colom, M., Soto, P., and Villegas-Portero, R. (2007). Observación post-introducción de tecnologías sanitarias. Guía metodológica. Plan de Calidad para el Sistema Nacional de Salud del Ministerio de Sanidad y Política Social. Axencia de Avaliación de Tecnoloxías Sanitarias de Galicia. Informes de Evaluación de Tecnologías Sanitarias: avalia-t No. 2007/02. Available at: http://aunets.isciii.es/ficherosproductos/120/avalia07-02-3.pdf [accessed February 28].
Keywords: monitoring systems, product surveillance, post-marketing, technology assessment, biomedical, diffusion of innovation, coverage with evidence, managed entry agreements
Citation: Ibargoyen-Roteta N, Cabriada-Nuño JL, Gutiérrez-Ibarluzea I, Hernández-Ramírez V, Clofent-Vilaplana J, Domènech-Morral E, Ginard-Vicens D, Oliva-Oliva G and Queiro-Verdes T (2011) Post-marketing health technology monitoring. The analysis of an experience from a clinical perspective. Front. Pharmacol. 2:45. doi: 10.3389/fphar.2011.00045
Received: 12 April 2011;
Paper pending published: 30 May 2011;
Accepted: 19 July 2011; Published online: 29 July 2011.
Edited by:Mark J. C. Nuijten, Institute for Medical Technology Assessment, Netherlands
Reviewed by:Sirajudheen Anwar, University of Messina, Italy
Hans-Peter Dauben, German Institute for Medical Documentation and Information, Germany
Copyright: © 2011 Ibargoyen-Roteta, Cabriada-Nuño, Gutiérrez-Ibarluzea, Hernández-Ramírez, Clofent-Vilaplana, Domènech-Morral, Ginard-Vicens, Oliva-Oliva and Queiro-Verdes. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Iñaki Gutiérrez-Ibarluzea, Basque Office for Health Technology Assessment (Osteba), Department of Health and Consumer Affairs, Basque Government, Calle Donostia-San Sebastian, 01010 Vitoria-Gasteiz, Araba, Spain. e-mail: firstname.lastname@example.org