Systematic review on the implementation of metrological assurance systems for medical devices in Latin America

Abstract

Background:

Metrology plays a crucial role in small healthcare service businesses to ensure the quality of products and services. While legal metrology in healthcare exists in some regions, it lacks harmonization. In other countries, there is limited presence of metrology in medical and biomedical engineering. We aimed to evaluate the implementation of metrological assurance systems for medical devices in Latin America.

Methods:

A systematic review was conducted following PRISMA 2020 guidelines and registered with PROSPERO (CRD42022359284). Searches were performed across 13 databases from October 30th to November 3rd, 2022. The search equation was “(((quality assurance) AND (metrology)) AND (medical devices)).” A total of 7,789 documents were identified, of which only 16 met the inclusion criteria.

Results:

The majority of studies (75%) were conducted in Colombia, with a significant portion being undergraduate theses. The primary normative references used in the analyzed studies were ISO 10012 and ISO 17025, with the majority (68.75%) relying on national legislation for their approach. One study in Colombia referenced eight standards, and one in Brazil analyzed user involvement in medical device management. Among the included studies, 56.25% were conducted in healthcare institutions, mainly clinics. Most studies provided implementation guidelines, with ISO 10012 being prominent, alongside ISO 17025, which implicitly addresses ISO 9001 elements. Global bias was low across all studies.

Conclusion:

Our results underscore the importance of metrological assurance in managing medical devices in Latin America. The utilization of international standards and national legislation illustrates the diverse approaches adopted by different institutions. Future research should focus on optimizing metrological practices to enhance quality and safety in healthcare.

1 Introduction

The rise of small enterprises is a global trend, and they encounter challenges in service delivery and product quality (1), and the healthcare service sector is not the exception. In this context, metrology plays a pivotal role, as modern professional development relies on accurate measurements to ensure high-quality products and services (2). Metrology is the science responsible for making measurements, assuring and maintaining units of measurement, as well as verifying periodic calibration of equipment (3). Within this science, there are divisions like legal metrology, which has legal implications, industrial metrology, used in manufacturing and production, and scientific metrology, focused on research and development of measurements (4).

Legal Metrology in Healthcare is a challenge present in several countries, including those within the European Union. Bošnjaković and Džemić (5), from the Institute of Metrology of Bosnia and Herzegovina, highlight that this area is not harmonized among different Union members, except for equipment covered by directives NAWI 2014/31/EU and MID 2014/32/EU (6). Thus, the implementation of a Legal Metrology system is left to the discretion of each member country. This situation was also observed in Portugal in 2011 (7). Nevertheless, it is noted that the European Union only allows the commercialization of medical equipment with the CE marking, guaranteeing compliance with metrological and quality standards upon factory exit, in compliance with prevailing regulations, especially 2007/47/EC. This measure was relevant in 2011 as the medical equipment industry employed nearly half a million people. However, Do Ceu Ferreira also states that, in her specific case, the requirements for quality metrological traceability are often disregarded.

However, the results presented by Ferreira and Matos (8) for the case of Portugal shed light on the concerning disparity between budgetary matters and metrology in public hospitals. Among the evaluated hospitals (35 responses received), only 34% performed some form of calibration, either internally or through an outsourced service. Additionally, hospitals and private clinics showed a 1.5 times higher prevalence in pursuing quality, opting for in-house calibration services. A concerning fact is that, in hospitals with over 1,100 installed devices, only around 40 units were calibrated in the last year. Furthermore, the question of dubious metrological traceability in calibrations conducted by external entities arises due to inadequate oversight. Despite Do Céu Ferreira’s assertion (7) about the strengths of the European Union in manufacturing and ensuring the metrological chain from the factory, it has been demonstrated that this continuity does not persist post-sale and deployment, as evidenced by Ferreira and Matos (8) and Bošnjaković and Džemić (5).

In the Latin American context, Andres et al. (9) address metrology applied in the medical field in Colombia, where a regulatory framework has been implemented to ensure quality in healthcare service provision. Colombian legislation stipulates requirements for registration, marketing, and health surveillance of medical equipment (10). It is precisely this approach, in the Colombian context, that infuses calibrations and metrology oversight with a sense of obligation. However, the study also illuminates the absence of standardized criteria and guidelines for conducting this monitoring effectively.

In Cuba, a different approach to metrological management is adopted, with large-scale integration supervised by the government, as indicated in the study by Albert and Téllez (11). The Ministry of Public Health in Cuba has implemented metrology by establishing the Calibration Network in 2019 and calibration laboratories in electromedicine. The plan incorporates international standards like ISO 9001 for document management and metrological control, aiming for future accreditations and a patient-centered approach. Additionally, the acquisition of national metrology standards is suggested if necessary. On the other hand, thanks to appropriate political and legislative support, Mejías et al. (12) presented their report on the metrology situation to the National Health System and the National Bureau of Standards. The report emphasized the importance of strengthening metrology as support for the National Service of Metrology and the network of electromedicine services, which already dealt with the maintenance, repair, and calibration of medical equipment but faced limitations due to a lack of professionals and resources (13).

In Peru, metrology is still lagging behind, and in fields such as medicine and biomedical engineering, it is nearly nonexistent (14). Despite the development of appropriate infrastructure and the presence of the National Institute of Quality (INACAL) as the regulatory and oversight body for metrology in Peru (15), there is an urgent need for businesses to achieve desired competitiveness by demonstrating high-quality standards (16). A set of factors, including the absence of regulations enforcing metrological control, the lack of companies providing biomedical calibration services, and the limited number of INACAL-accredited tests. For small healthcare service companies, metrology offers the potential to enhance measurement accuracy, thus ensuring the quality of the services and products they offer. However, it is crucial to understand quality assurance processes in metrology on a country-by-country basis in order to determine whether metrological practices are adequate to meet quality standards and fulfill patients’ needs.

The objective of this study was to evaluate the implementation of metrological assurance systems for medical devices in Latin America.

2 Materials and methods

2.1 Study design, data sources, and search strategy

This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines (17) and was registered with the International Prospective Register of Systematic Reviews (PROSPERO) under code: CRD42022359284. Searches were conducted across 13 databases (PubMed, Scopus, Web of Science, ScienceDirect, JSTOR, EMBASE, Scielo, Latindex, LILIACS, and EBSCO), Google Scholar, Dimensions, and ALICIA CONCyTec (Peruvian Thesis Repository) from October 30th to November 3rd, 2022. The search equation used was “(((quality assurance) AND (metrology)) AND (medical devices))” without limitations on time or language (Supplementary Data Sheet 1).

2.2 Inclusion and exclusion criteria

Included studies met the following criteria: (i) original studies, clinical trials, validation studies, exploratory studies, commentaries, proceedings, and theses; (ii) publications between 2012 and 2022; (iii) articles in English, Portuguese, and Spanish; (iv) articles about medical devices; (v) studies in Latin American countries. Studies outside the clinical scope, systematic reviews, reflection articles, opinion letters, editorials, meta-analyses, documents focusing solely on calibration or medical equipment management were excluded (Figure 1).

Figure 1

2.3 Document selection and extraction

Two authors (HMF-V and JM-S) independently evaluated abstracts and excluded those not meeting inclusion criteria. Then, full-text reviews were performed according to a defined protocol for final analysis. Disagreements were resolved through consensus, following a previously reported procedure (18). Cohen’s Kappa correlation analysis was conducted to determine overall agreement between reviewers.

2.4 Data extraction, quality assessment, and data analysis

Data were extracted using the CASPe (Critical Appraisal Skills Programme) data matrix template in MS-Excel 2013 (Microsoft Corp., Redmond, WA, United States) to collect necessary information from systematic reviews. Cochrane’s risk of bias tool (Robvis 2.0) was used to assess bias, with studies not contributing to the study objective (analyzing at least one variable) considered to have a high risk of confusion (Figure 1). Disagreements among authors were resolved through consensus. Descriptive analysis of included studies was conducted using IBM SPSS version 23.0 (Armonk, NY, United States), estimating frequencies for categorical data and measures of central tendency for continuous data.

3 Results

Throughout the research process, a total of 7,789 documents were found in Scopus (n = 9), PubMed (n = 23), JSTOR (n = 795), ScienceDirect (n = 489), Dimensions (n = 541), Alicia CONCyTec (n = 5), and Google Scholar (n = 5,893). No studies were identified in Scielo and LILACS, and 790 duplicate files were removed. Additionally, 5,571 documents not closely related to the topic were eliminated, leaving 209 documents for further analysis. However, it was not possible to retrieve 40 of these documents, resulting in the inclusion of 16 documents in this systematic review (19–34). Bias analysis is shown in Figure 2.

Figure 2

3.1 Document characteristics

Out of the 16 documents, 75% (12/16) were studies conducted in Colombia, with comparable numbers between Bogotá, Cali, and Medellín (19–21, 23–25, 27–30, 32, 33), and of these, 50% (6/12) were undergraduate theses (20, 21, 28–30, 33). Also, 18.7% (3/16) of the documents were published conference papers from Brazil (26), Colombia (23), and Cuba (29). A quarter of the documents were research articles (19, 24, 27, 31), and the majority (3/6) were published in 2021 (32–34) (Table 1).

3.2 Normative references used in different studies

Among the analyzed documents, various applicable standards for medical devices are mentioned (Table 2), such as IEC 60601–1-6 (standard to ensure basic safety and essential performance requirements for electromedical devices) and IEC 62366 (standard for applying usability engineering to medical devices) (26). Additionally, standards applicable to quality management like ISO 9000 (19, 25, 33), ISO 9001 (29–31), and ISO 15189 (specific to clinical laboratories) (29) are referenced, along with ISO 17000, which establishes vocabulary (33), ISO 17025 (specific to calibration and testing laboratories) (19, 21, 23, 25, 29–31, 33), and ISO 10012 (applied to measurement management systems) (19, 22, 23, 25, 27, 29, 31, 33).

Furthermore, ANSI/AAMI EQ56 2008 R (2013), a standard providing recommendations for implementing a medical equipment management program, is mentioned in half of the documents (24). Half of the documents (8/16) mentions the use of ISO 10012 (19, 22, 23, 25, 27, 29, 31, 33) and the ISO 17025 standard (19, 21, 23, 25, 29–31, 33). The fourth part of the documents mentions the use of de ISO 9001 (29–31), 18.75% (3/16) identify the use of ISO 9000 (19, 25, 33) and ISO 14971 (20, 26, 32). Out of the 16 documents, 68.75% (11/16) uses the national legislation for their framework (20, 22, 23, 26–30, 32–34) and only a study (6.25%) mentions international legislation (24).

A significant work by Torres (33) stands out due to its reference to eight standards, including ISO 5725, ISO 9000, ISO 9001, ISO 10012, ISO 13485, ISO 17000, ISO 17025, and IEC 62353. This study focuses on optimizing the metrological assurance plan for the Departmental Public Health Laboratory of Antioquia, Colombia, and stands out for utilizing the highest number of international and national standards in its analysis. The work conducted by Campese and Costa (26) addresses user involvement in medical device management in small enterprises, particularly emphasizing safety factors for users. While briefly mentioning the effects of metrology in this context, its primary focus is on technical standards like IEC 60601–1-6 and IEC 62366, as well as the ergonomics standard ISO 9241-11.

The study by Gaviria and Romero (28) is notable for its use of national legislation instead of international standards, although paradoxically, it references images from ISO 17025 in its theoretical framework. The study entitled “Diseño e implementación de un plan de aseguramiento metrológico para equipos biomédicos de la Clínica Colsubsidio Calle 100 y el Centro Médico de Especialistas de la Calle 63” (Design and Implementation of a Metrological Assurance Plan for Biomedical Equipment at Clínica Colsubsidio Calle 100 and Centro Médico de Especialistas de la Calle 63) is interesting due to its focus on defining metrological needs for each listed piece of equipment and selecting patterns for calibration.

3.3 Analysis by type of organization

Table 3 describes the types of organizations in the included studies. Out of all the studies, 56.25% (9/16) were conducted in healthcare institutions (20, 21, 23, 28–30, 32–34), with the majority of these being carried out in clinics (20, 23, 28, 30, 32, 34), two in clinical laboratories (21, 33), and one in a university laboratory (29).

3.4 Studies presenting implementation guidelines

Of the total, 31.3% (5/16) of the studies showed guidelines for implementing assurance systems (19, 22, 25–27). Four out of these five studies provided guides based on ISO 10012 and focused on metrological assurance (19, 22, 25, 27), while one presented a guide based on usability and electrical safety, focusing on users in small enterprises (26). Of these studies, three were conducted in Colombia (19, 25, 27), one in Costa Rica (22), and one in Brazil (26).

ISO 10012 is widely mentioned in the implementation guidelines for metrological assurance systems. From the analysis of Tables 2, 4, the relevance of ISO 17025 is also highlighted, implicitly incorporating certain aspects of ISO 9001. The latter standard proposes a process-based approach to address all organizational procedures to ensure quality.

3.5 Bias analysis

Two types of bias risk variables were not applicable to the selected documents, as they did not involve randomization or double-blind procedures, hence listed as “No information.” The majority (12/16) of the included studies had a low risk of bias, while three had insufficient information (Figure 1). The inter-observer agreement analysis was optimal (k=0.899) according to previous recommendations (35).

4 Discussion

This review of medical devices has showed the use of different standards, such as IEC 60601–1-6 for electromedical safety and IEC 62366 for usability. ISO 9000, ISO 9001, and ISO 15189 standards were mentioned for quality management by a third of the studies. Half of the studies were conducted in healthcare institutions, with only one study describing the use of eight national and international standards in Colombia, and discrepancies were identified in the use of national legislation and ISO 17025.

4.1 Strengths

This research stands out as the first systematic review comprehensively addressing metrological assurance systems for medical devices in Latin America. Unlike other studies that focus on specific aspects or evaluate institutions or countries (5–12), this analysis considers the various standards used in regional institutions. Understanding metrological assurance systems opens new opportunities to improve institutional design and functionality, promoting the creation of protocols or manuals that benefit the quality assurance of processes. Another strength of this research is its thorough information search, including grey literature, thereby expanding the database used for analysis.

4.2 Main findings

Based on the evidence found in Latin America, recommendations are made, such as using a reference standard to develop a metrological assurance system. In addition, fundamental characteristics for such a system are considered, including necessary input documentation, defining metrological needs, and outlining protocols (19, 22, 25, 27). These recommendations provide a solid foundation for the effective implementation of metrological assurance, ensuring the quality and accuracy of measurements in devices used in the healthcare field.

Across the reviewed studies, the implementation of metrological assurance systems typically occurred in institutions with a high number of equipment, such as clinics, laboratories, or universities. Due to this characteristic, there are few investigations that address this issue in smaller businesses, with Campese and Costa’s work (26) being an exception, aiming to involve users of small enterprises in the assurance process. However, it is noteworthy that ISO 9001 and ISO 10012 standards are easily adaptable to smaller organizations, including micro and small businesses. After analyzing the body of studies, fundamental components necessary for a metrological assurance system were identified, including proper identification of equipment used, recognition of their metrological needs (19, 25, 27), and the creation of a metrological assurance plan (19, 21, 25, 27). These components are tailored to meet the particularities of a small business, ensuring quality and precision in measurements within the healthcare field.

Our findings indicate that the most recommended standard for metrological assurance implementation is ISO 10012, and its citation frequency is comparable to that of ISO 17025 (36). ISO 17025 focuses on ensuring the technical competence of testing laboratories, assuring accuracy and reliability of test and calibration data. The standard encompasses key aspects of laboratory management, such as quality management system requirements, personnel competence, equipment calibration and maintenance, test methodology, and reporting results. However, this standard is designed for organizations with calibration laboratories and would only be applicable if a company can conduct internal calibrations on its devices (37). On the other hand, ISO 10012 (38) can be used to manage measurement processes and metrological confirmation of measurement equipment used to support and demonstrate compliance with metrological requirements in the context of medical device metrology. The standard specifies the quality management requirements of a measurement management system that an organization can employ as part of its overall management system, ensuring compliance with metrological requirements.

With the large number of small and microenterprises in the healthcare sector in Latin America, the implementation of standards and metrological assurance systems becomes even more relevant. This is essential to ensure measurement quality and, through that, the safety of diagnoses and patients, while also enhancing the competitiveness of these companies with procedures that can later be certified or accredited with ISO 9001 (16) or ISO 10012 (38).

Undoubtedly, the COVID-19 pandemic has emerged as an unparalleled event, delivering a seismic impact on the healthcare sector across various Latin American nations. In Peru, a country that has witnessed a surge in public health research over the past 50 years (39), there exists a palpable need for initiatives aimed at overseeing adherence to medical standards. This necessity arises as many research endeavors confront inherent limitations in their methodological approaches and outcome analyses (40). Simultaneously, the healthcare sector grapples with challenges such as a shortage of adequately trained personnel and constraints in the utilization of available technologies integral to daily clinical practice (41). Notably, certain technologies, including telemedicine, have demonstrated successful applications during the lockdown, holding the potential to enhance Clinical Risk Management, thereby elevating the Quality of Care and ensuring Patient Safety (42, 43). Nevertheless, it becomes imperative to diligently monitor the post-COVID landscape to propose regulatory frameworks that can effectively govern the seamless technological integration into comprehensive healthcare processes.

4.3 Limitations

Firstly, it is noted that the analysis was limited to four countries due to evidence availability. That did not allow us to perform a regional comparison with nations such as Chile, Argentina, Bolivia, Uruguay, or Paraguay, which lack available research on metrology. Secondly, the number of studies found (12/16) on metrology in Colombia might have influenced the overall results. While it is commendable that Colombia has a considerable number of biomedical engineering schools in nine universities (44), other countries like Peru have a more limited offering, with only two recent universities offering this program (45). This could impact the generalization of results, potentially predominantly reflecting the situation in Colombia. Therefore, it is essential to encourage research on metrology (46) across various Latin American countries to understand their realities and promote the development of metrological assurance systems for medical devices, thus ensuring quality in healthcare.

5 Conclusions and future directions

This study provides valuable insights into the implementation of metrology and quality control in the context of medical devices. The analysis of various applicable standards for medical devices, such as IEC 60601-1-6 and IEC 62366, along with ISO 9000, ISO 9001, ISO 17025, and ISO 10012, highlights the multifaceted approach necessary to ensure safety, accuracy, and reliability in healthcare technology. The research emphasizes the importance of incorporating both international and national legislation, and the inclusion of implementation guides based on ISO 10012, as well as the focus on small businesses, further contribute to the practicality and relevance of this research.

Addressing the challenges and requirements of metrology and quality assurance in medical devices, this study presents a valuable resource for advancing healthcare technology and improving patient care. Furthermore, it is crucial for both national and private institutions in each Latin American country to establish education and training programs focused on medical device metrology. This could ensure the quality of diagnoses in both larger hospitals and smaller organizations, benefiting patients. On the other hand, it is essential for academia and research centers to work on providing evidence and information availability, as well as conducting reference studies on medical device management and applicable metrological assurance plans. This will foster continuous advancement and improvement in the implementation of practices that ensure quality and accuracy in healthcare.

References

• 1.

  María Luisa SG María Elena CA Sánchez BT . Quality for competitiveness micro, small and medium enterprises of Mexico City. Rev Ven Gerencia. (2017) 22:551–75. doi: 10.37960/revista.v22i80.23174

  • CrossRef
  • Google Scholar

• 2.

  Soler-López ML Regla Hernández-Leonard A Martínez-Barragán ND . Programa tecnológico en metrologia promotor de calidad en productos y servicios. Boletín Científico Técnico INIMET, (2015), 2, 13–19. Available at: https://www.redalyc.org/articulo.oa?id=223043197003

  • Google Scholar

• 3.

  Wade HA The ASQ . Metrology handbook. 3th ed. Milwaukee: ASQ Quality Press (2023).

  • Google Scholar

• 4.

  Gao W . Metrology. 1st ed. Singapore: Springer (2019).

  • Google Scholar

• 5.

  Bošnjaković A Džemić Z . Legal metrology: medical devices. IFMBE Proc. (2017) 62:583–8. doi: 10.1007/978-981-10-4166-2_88

  • CrossRef
  • Google Scholar

• 6.

  European Union. Regulation (EU) . 2017/745 of the European Parliament and of the council on medical devices, amending directive 2001/83/EC, regulation (EC) no 178/2002 and regulation (EC) no 1223/2009 and repealing council directives 90/385/EEC and 93/42/EEC. EUR-Lex. Off J Eur Union. (2017) 117:1–175. Available at:https://eur-lex.europa.eu/eli/reg/2017/745/2020-04-24

  • Google Scholar

• 7.

  do Céu Ferreira M . The role of metrology in the field of medical devices. Int J Metrol Quality Eng. (2011) 2:135–40. doi: 10.1051/ijmqe/2011101

  • CrossRef
  • Google Scholar

• 8.

  Ferreira M Matos A . Metrology in health: a pilot study. J Phys. (2015) 588:012018. doi: 10.1088/1742-6596/588/1/012018

  • CrossRef
  • Google Scholar

• 9.

  Andres RBF Jhon FMG Gonzalo NB . Caracterización de la gestión metrológica en instituciones sanitarias. Rev Ingen Biomed. (2015) 9:57–64. doi: 10.14508/rbme.2015.9.18.57-64

  • CrossRef
  • Google Scholar

• 10.

  Ministerio de comercio , Industria y turismo de Colombia. N.° 1471 Por el cual se reorganiza el Subsistema Nacional de la Calidad y se modifica el Decreto 2269 de 1993. Bogota D.C.: Función Pública, Colombia; (2014). Available at:https://www.funcionpublica.gov.co/eva/gestornormativo/norma.php?i=58845

  • Google Scholar

• 11.

  Albert DOL Tellez SI . Gestión para implementación de la Metrología en institución de Salud Pública. Ciencias Holguín. (2020) 26:97–195.

  • Google Scholar

• 12.

  Mejías SY Morales SI Perera VS . Bases legales y proyecciones de desarrollo de la metrología en el Sistema Nacional de Salud. Rev Cubana Salud Pública. (2015) 41:172–7.

  • Google Scholar

• 13.

  Ministerio De Economía y Planificación. Plan Nacional de Desarrollo Económico y Social hasta el (2030). La Habana: Observatorio Regional de Planificación Para El Desarrollo de América Latina y El Caribe, Presidencia y Gobierno de Cuba; 2019. Available at: https://www.mep.gob.cu/es/pndesods-2030/plan-nacional-de-desarrollo-economico-y-social-2030

  • Google Scholar

• 14.

  Acuña FJ . Metrología Biomédica. Boletín INS. (2015) 21:11–5.

  • Google Scholar

• 15.

  Instituto Nacional de Calidad . Normas y documentos legales. Lima: Gobierno del Perú. Disponible en: Available at: https://www.gob.pe/institucion/inacal/normas-legales

• 16.

  International Organization for Standardization . Standard quality management systems-requirements, ISO 9001:2015. Geneva: ISO (2015).

  • Google Scholar

• 17.

  Page MJ McKenzie JE Bossuyt PM Boutron I Hoffmann TC Mulrow CD et al . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. (2021) 372:n71. doi: 10.1136/bmj.n71

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  Moya-Salazar J Nuñez E Jaime-Quispe A Zuñiga N Loaiza-Barboza IL Balabarca EA et al . Substance use in healthcare professionals during the COVID-19 pandemic in Latin America: a systematic review and a call for reports. Subst Abuse. (2022) 16:11782218221085592. doi: 10.1177/11782218221085592

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  Rodríguez-Denis EB . Método Para La Implementación de un Plan de Aseguramiento Metrológico Para Dispositivos Médicos En Servicios de Salud. Bogota D.C.: Congreso Internacional de Metrologia (2012) Available at:https://inm.gov.co/web/wp-content/uploads/2021/10/METODO-PARA-LA-IMPLEMENTACION-DE-UN-PLAN-DE-ASEGURAMIENTO-METROLOGICO-PARA-DISPOSITIVOS-MEDICOS-EN-SERVICIOS-DE-SALUD.pdf.

  • Google Scholar

• 20.

  Montenegro AM . Desarrollo de una guía metodológica para la evaluación de equipos biomédicos instalados en una IPS en la ciudad de Santiago de Cali. [Bachellor thesis]. Cali: Universidad Autónoma de Occidente (2014).

  • Google Scholar

• 21.

  Calambás DM . Manual de procedimientos para la verificación/calibración de instrumentos y equipos del laboratorio clínico. [Bachellor thesis]. Cali: Universidad Autónoma de Occidente (2013).

  • Google Scholar

• 22.

  Pacheco R . Un modelo para asegurar las mediciones en el sistema de salud pública de Costa Rica. Rev Ingeniería. (2014) 24:69–80. doi: 10.15517/ring.v24i1.10251

  • CrossRef
  • Google Scholar

• 23.

  García JH Hernandez AM Barreneche JG . Methodology for metrological confirmation in health care institutions. (2015) Pan American health care exchanges (PAHCE), Santiago, Chile, (2015), 1–7.

  • Google Scholar

• 24.

  Trujillo M Salazar FE Andrade ME . Politics for the integration of the activities of insurance metrological to the process of maintenance of biomedical Equipments. Rev Ingen Biomed. (2015) 9:33–40. doi: 10.24050/19099762.n18.2015.764

  • CrossRef
  • Google Scholar

• 25.

  Rodríguez EB Obando FM Villa JD Villar MC Calvo PC Ordeñez S . Recomendaciones para la implementación de un plan de aseguramiento metrológico para dispositivos médicos en entidades prestadoras de servicios de salud. Cali: Fondo Editorial Universidad Autónoma de Occidente (2016).

  • Google Scholar

• 26.

  Campese C Costa JMH . O Envolvimento Do Usuário No Processo De Desenvolvimento De Eletromédicos De Pequenas Empresas. 11o Congresso Brasileiro de Inovação e Gestão de Desenvolvimento Do Produto. (2017) 3:563–71. doi: 10.5151/cbgdp2017-057

  • CrossRef
  • Google Scholar

• 27.

  Ramírez Barrera AF Martínez GJF Hidalgo VE . Modelo de gestión para la aplicación del control metrológico legal y la evaluación de la conformidad en equipos biomédicos. Rev Ingen Biomed. (2017) 11:73–80. doi: 10.24050/19099762.n21.2017.1175

  • CrossRef
  • Google Scholar

• 28.

  Gaviria L Romero IP . Diseño e implementación de un plan de aseguramiento metrológico para equipos biomédicos de la clínica Colsubsidio calle 100 y el Centro Médico de Especialistas de la calle 63. [Bachellor thesis]. Rosario: Universidad del Rosario (2016).

  • Google Scholar

• 29.

  León-Ramentol CC Menéndez-Cabezas A Rodríguez-Socarrás IP López-Estrada B García-González MC Fernández-Torres S . Importancia de un sistema de gestión de la calidad en la Universidad de Ciencias Médicas. Rev Arch Med Camagüey. (2018) 22:843–57.

  • Google Scholar

• 30.

  Rivera LC Toloza JN . Propuesta de plan de aseguramiento metrológico en la IPS Oftalmosanitas S.A. [Bachellor thesis]. Bogota D.C.: Universidad ECCI (2019).

  • Google Scholar

• 31.

  Pacheco SJ Perafán JA . Propuesta de un plan de gestión administrativa metrológica basado en el apartado 7.1.5 de la norma ISO 9001:2015. [Bachellor thesis]. Guayaquil: Universidad de Guayaquil (2020).

  • Google Scholar

• 32.

  Duque DM . Propuesta de mejora para la gestión de tecnología biomédica, aplicando la metodología AMFE en una Institución de Salud. En Universidad ECCI. [Bachellor thesis]. Bogota D.C.: Universidad ECCI (2020).

  • Google Scholar

• 33.

  Torres VC . Optimización del plan de aseguramiento metrológico del Laboratorio Departamental de Salud Pública de Antioquia. [Bachellor thesis]. Antioquía: Universidad de Antioquía (2021).

  • Google Scholar

• 34.

  León PC . Plan de aseguramiento metrológico en los servicios de UCI y salas de cirugía para la Clínica Universidad de la Sabana. [Bachellor thesis]. Rosario: Universidad del Rosario (2021).

  • Google Scholar

• 35.

  Viera AJ Garrett JM . Understanding interobserver agreement: the kappa statistic. Fam Med. (2005) 37:360–3. PMID:

  • Pubmed Abstract
  • Google Scholar

• 36.

  International Standard Organization . Medical laboratories—requirements for quality and competence ISO 15189. Geneva: ISO (2012).

  • Google Scholar

• 37.

  Joseph L Ramesh P Remya NS Arumugham V Rajesh RP . Significance of metrological tools in an ISO 17025 accredited quality system for a biological evaluation facility. Mapan. (2022) 37:683–91. doi: 10.1007/s12647-021-00517-2

  • CrossRef
  • Google Scholar

• 38.

  International Organization for Standardization . Measurement management systems-requirements for measurement processes and measuring equipment ISO 10012. Geneva: ISO (2003).

  • Google Scholar

• 39.

  Sevillano-Jimenez J Carrión-Chambilla M Espinoza-Lecca E Mori-Quispe E Contreras-Pulache H Moya-Salazar J . A bibliometric analysis of 47-years of research on public health in Peru. Electron J Gen Med. (2023) 20:em488. doi: 10.29333/ejgm/13103

  • CrossRef
  • Google Scholar

• 40.

  Moya-Salazar C-PH . Reply to the letter: The evidence of Bayesian a/B testing in the contrast of clinical events by COVID-19. Infectio. (2022) 26:101. doi: 10.22354/in.v26i1.1004

  • CrossRef
  • Google Scholar

• 41.

  Moya-Salazar J . Una agenda pendiente sobre la calidad de vida post-COVID-19: comentarios sobre el estudio de García et al., (2022). J Healthc Qual Res. (2023) 38:61–2. doi: 10.1016/j.jhqr.2022.08.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  De Micco F Fineschi V Banfi G Frati P Oliva A Travaini GV et al . From COVID-19 pandemic to patient safety: a new "spring" for telemedicine or a boomerang effect?Front Med. (2022) 9:901788. doi: 10.3389/fmed.2022.901788

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  De Micco F De Benedictis A Fineschi V Frati P Ciccozzi M Pecchia L et al . From Syndemic lesson after COVID-19 pandemic to a "systemic clinical risk management" proposal in the perspective of the ethics of job well done. Int J Environ Res Public Health. (2021) 19:15. doi: 10.3390/ijerph19010015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  Universia . Universidades para estudiar Ingeniería Biomédica en Colombia. Available at: https://orientacion.universia.net.co/infodetail/orientacion/orientacion_vocacional/universidades-para-estudiar-ingenieria-biomedica-en-colombia-8570.html

• 45.

  Universia . Estudiar Igeniería Biomédica en Perú. Available at: https://orientacion.universia.edu.pe/infodetail/orientacion/orientacion_vocacional/estudiar-igenieria-biomedica-en-peru-3292.html

• 46.

  Sené M Gilmore I Janssen JT . Metrology is key to reproducing results. Nature. (2017) 547:397–9. doi: 10.1038/547397a

  • Pubmed Abstract
  • CrossRef
  • Google Scholar