Systematic review on the implementation of metrological assurance systems for medical devices in Latin America

Background Metrology plays a crucial role in small healthcare service businesses to ensure the quality of products and services. While legal metrology in healthcare exists in some regions, it lacks harmonization. In other countries, there is limited presence of metrology in medical and biomedical engineering. We aimed to evaluate the implementation of metrological assurance systems for medical devices in Latin America. Methods A systematic review was conducted following PRISMA 2020 guidelines and registered with PROSPERO (CRD42022359284). Searches were performed across 13 databases from October 30th to November 3rd, 2022. The search equation was “(((quality assurance) AND (metrology)) AND (medical devices)).” A total of 7,789 documents were identified, of which only 16 met the inclusion criteria. Results The majority of studies (75%) were conducted in Colombia, with a significant portion being undergraduate theses. The primary normative references used in the analyzed studies were ISO 10012 and ISO 17025, with the majority (68.75%) relying on national legislation for their approach. One study in Colombia referenced eight standards, and one in Brazil analyzed user involvement in medical device management. Among the included studies, 56.25% were conducted in healthcare institutions, mainly clinics. Most studies provided implementation guidelines, with ISO 10012 being prominent, alongside ISO 17025, which implicitly addresses ISO 9001 elements. Global bias was low across all studies. Conclusion Our results underscore the importance of metrological assurance in managing medical devices in Latin America. The utilization of international standards and national legislation illustrates the diverse approaches adopted by different institutions. Future research should focus on optimizing metrological practices to enhance quality and safety in healthcare.


Introduction
The rise of small enterprises is a global trend, and they encounter challenges in service delivery and product quality (1), and the healthcare service sector is not the exception.In this context, metrology plays a pivotal role, as modern professional development relies on accurate measurements to ensure high-quality products and services (2).Metrology is the science responsible for making measurements, assuring and maintaining units of measurement, as well as verifying periodic calibration of equipment (3).Within this science, there are divisions like legal metrology, which has legal implications, industrial metrology, used in manufacturing and production, and scientific metrology, focused on research and development of measurements (4).
Legal Metrology in Healthcare is a challenge present in several countries, including those within the European Union.Bošnjaković and Džemić (5), from the Institute of Metrology of Bosnia and Herzegovina, highlight that this area is not harmonized among different Union members, except for equipment covered by directives NAWI 2014/31/EU and MID 2014/32/EU (6).Thus, the implementation of a Legal Metrology system is left to the discretion of each member country.This situation was also observed in Portugal in 2011 (7).Nevertheless, it is noted that the European Union only allows the commercialization of medical equipment with the CE marking, guaranteeing compliance with metrological and quality standards upon factory exit, in compliance with prevailing regulations, especially 2007/47/EC.This measure was relevant in 2011 as the medical equipment industry employed nearly half a million people.However, Do Ceu Ferreira also states that, in her specific case, the requirements for quality metrological traceability are often disregarded.
However, the results presented by Ferreira and Matos (8) for the case of Portugal shed light on the concerning disparity between budgetary matters and metrology in public hospitals.Among the evaluated hospitals (35 responses received), only 34% performed some form of calibration, either internally or through an outsourced service.Additionally, hospitals and private clinics showed a 1.5 times higher prevalence in pursuing quality, opting for in-house calibration services.A concerning fact is that, in hospitals with over 1,100 installed devices, only around 40 units were calibrated in the last year.Furthermore, the question of dubious metrological traceability in calibrations conducted by external entities arises due to inadequate oversight.Despite Do Céu Ferreira's assertion (7) about the strengths of the European Union in manufacturing and ensuring the metrological chain from the factory, it has been demonstrated that this continuity does not persist post-sale and deployment, as evidenced by Ferreira and Matos (8) and Bošnjaković and Džemić (5).
In the Latin American context, Andres et al. (9) address metrology applied in the medical field in Colombia, where a regulatory framework has been implemented to ensure quality in healthcare service provision.Colombian legislation stipulates requirements for registration, marketing, and health surveillance of medical equipment (10).It is precisely this approach, in the Colombian context, that infuses calibrations and metrology oversight with a sense of obligation.However, the study also illuminates the absence of standardized criteria and guidelines for conducting this monitoring effectively.
In Cuba, a different approach to metrological management is adopted, with large-scale integration supervised by the government, as indicated in the study by Albert and Téllez (11).The Ministry of Public Health in Cuba has implemented metrology by establishing the Calibration Network in 2019 and calibration laboratories in electromedicine.The plan incorporates international standards like ISO 9001 for document management and metrological control, aiming for future accreditations and a patient-centered approach.Additionally, the acquisition of national metrology standards is suggested if necessary.On the other hand, thanks to appropriate political and legislative support, Mejías et al. (12) presented their report on the metrology situation to the National Health System and the National Bureau of Standards.The report emphasized the importance of strengthening metrology as support for the National Service of Metrology and the network of electromedicine services, which already dealt with the maintenance, repair, and calibration of medical equipment but faced limitations due to a lack of professionals and resources (13).
In Peru, metrology is still lagging behind, and in fields such as medicine and biomedical engineering, it is nearly nonexistent (14).Despite the development of appropriate infrastructure and the presence of the National Institute of Quality (INACAL) as the regulatory and oversight body for metrology in Peru (15), there is an urgent need for businesses to achieve desired competitiveness by demonstrating high-quality standards (16).A set of factors, including the absence of regulations enforcing metrological control, the lack of companies providing biomedical calibration services, and the limited number of INACAL-accredited tests.For small healthcare service companies, metrology offers the potential to enhance measurement accuracy, thus ensuring the quality of the services and products they offer.However, it is crucial to understand quality assurance processes in metrology on a country-by-country basis in order to determine whether metrological practices are adequate to meet quality standards and fulfill patients' needs.
The objective of this study was to evaluate the implementation of metrological assurance systems for medical devices in Latin America.

Study design, data sources, and search strategy
This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines (17) and was registered with the International Prospective Register of Systematic Reviews (PROSPERO) under code: CRD42022359284.Searches were conducted across 13 databases (PubMed, Scopus, Web of Science, ScienceDirect, JSTOR, EMBASE, Scielo, Latindex, LILIACS, and EBSCO), Google Scholar, Dimensions, and ALICIA CONCyTec (Peruvian Thesis Repository) from October 30th to November 3rd, 2022.The search equation used was "(((quality assurance) AND (metrology)) AND (medical devices))" without limitations on time or language (Supplementary Data Sheet 1).

Inclusion and exclusion criteria
Included studies met the following criteria: (i) original studies, clinical trials, validation studies, exploratory studies, (iv) articles about medical devices; (v) studies in Latin American countries.Studies outside the clinical scope, systematic reviews, reflection articles, opinion letters, editorials, meta-analyses, documents focusing solely on calibration or medical equipment management were excluded (Figure 1).

Document selection and extraction
Two authors (HMF-V and JM-S) independently evaluated abstracts and excluded those not meeting inclusion criteria.Then, fulltext reviews were performed according to a defined protocol for final analysis.Disagreements were resolved through consensus, following a previously reported procedure (18).Cohen's Kappa correlation analysis was conducted to determine overall agreement between reviewers.

Data extraction, quality assessment, and data analysis
Data were extracted using the CASPe (Critical Appraisal Skills Programme) data matrix template in MS-Excel 2013 (Microsoft Corp., Redmond, WA, United States) to collect necessary information from systematic reviews.Cochrane's risk of bias tool (Robvis 2.0) was used to assess bias, with studies not contributing to the study objective (analyzing at least one variable) considered to have a high risk of confusion (Figure 1).Disagreements among authors were resolved

FIGURE 2
Bias analysis of the selected studies.
A significant work by Torres (33) stands out due to its reference to eight standards, including ISO 5725, ISO 9000, ISO 9001, ISO 10012, ISO 13485, ISO 17000, ISO 17025, and IEC 62353.This study focuses on optimizing the metrological assurance plan for the Departmental Public Health Laboratory of Antioquia, Colombia, and stands out for utilizing the highest number of international and national standards in its analysis.The work conducted by Campese and Costa (26) addresses user involvement in medical device management in small enterprises, particularly emphasizing safety factors for users.While briefly mentioning the effects of metrology in this context, its primary focus is on technical standards like IEC 60601-1-6 and IEC 62366, as well as the ergonomics standard ISO 9241-11.
The study by Gaviria and Romero (28) is notable for its use of national legislation instead of international standards, although paradoxically, it references images from ISO 17025 in its theoretical framework.The study entitled "Diseño e implementación de un plan de aseguramiento metrológico para equipos biomédicos de la Clínica Colsubsidio Calle 100 y el Centro Médico de Especialistas de la Calle 63" (Design and Implementation of a Metrological Assurance Plan for Biomedical Equipment at Clínica Colsubsidio Calle 100 and Centro Médico de Especialistas de la Calle 63) is interesting due to its focus on defining metrological needs for each listed piece of equipment and selecting patterns for calibration.

Studies presenting implementation guidelines
Of the total, 31.3%(5/16) of the studies showed guidelines for implementing assurance systems (19,22,(25)(26)(27).Four out of these five studies provided guides based on ISO 10012 and focused on metrological assurance (19,22,25,27), while one presented a guide based on usability and electrical safety, focusing on users in small enterprises (26).Of these studies, three were conducted in Colombia (19,25,27), one in Costa Rica (22), and one in Brazil (26).ISO 10012 is widely mentioned in the implementation guidelines for metrological assurance systems.From the analysis of Tables 2, 4, the relevance of ISO 17025 is also highlighted, implicitly incorporating certain aspects of ISO 9001.The latter standard proposes a processbased approach to address all organizational procedures to ensure quality.

Bias analysis
Two types of bias risk variables were not applicable to the selected documents, as they did not involve randomization or double-blind procedures, hence listed as "No information." The majority (12/16) of the included studies had a low risk of bias, while three had insufficient information (Figure 1).The inter-observer agreement analysis was optimal (k=0.899)according to previous recommendations (35).

Discussion
This review of medical devices has showed the use of different standards, such as IEC 60601-1-6 for electromedical safety and IEC 62366 for usability.ISO 9000, ISO 9001, and ISO 15189 standards were mentioned for quality management by a third of the studies.Half of the studies were conducted in healthcare institutions, with only one study describing the use of eight national and international standards in Colombia, and discrepancies were identified in the use of national legislation and ISO 17025.

Strengths
This research stands out as the first systematic review comprehensively addressing metrological assurance systems for medical devices in Latin America.Unlike other studies that focus on specific aspects or evaluate institutions or countries (5)(6)(7)(8)(9)(10)(11)(12), this analysis considers the various standards used in regional institutions.Understanding metrological assurance systems opens new opportunities to improve institutional design and functionality, promoting the creation of protocols or manuals that benefit the quality assurance of processes.Another strength of this research is its thorough information search, including grey literature, thereby expanding the database used for analysis.

Main findings
Based on the evidence found in Latin America, recommendations are made, such as using a reference standard to develop a metrological assurance system.In addition, fundamental characteristics for such a system are considered, including necessary input documentation, defining metrological needs, and outlining protocols (19,22,25,27).These recommendations provide a solid foundation for the effective implementation of metrological assurance, ensuring the quality and accuracy of measurements in devices used in the healthcare field.
Across the reviewed studies, the implementation of metrological assurance systems typically occurred in institutions with a high number of equipment, such as clinics, laboratories, or universities.Due to this characteristic, there are few investigations that address this issue in smaller businesses, with Campese and Costa's work (26) being an exception, aiming to involve users of small enterprises in the assurance process.However, it is noteworthy that ISO 9001 and ISO 10012 standards are easily adaptable to smaller organizations, including micro and small businesses.After analyzing the body of studies, fundamental components necessary for a metrological assurance system were identified, including proper identification of equipment used, recognition of their metrological needs (19,25,27), and the creation of a metrological assurance plan (19,21,25,27).These components are tailored to meet the particularities of a small business, ensuring quality and precision in measurements within the healthcare field.
Our findings indicate that the most recommended standard for metrological assurance implementation is ISO 10012, and its citation frequency is comparable to that of ISO 17025 (36).ISO 17025 focuses on ensuring the technical competence of testing laboratories, assuring accuracy and reliability of test and calibration data.The standard encompasses key aspects of laboratory management, such as quality management system requirements, personnel competence, equipment calibration and maintenance, test methodology, and reporting results.However, this standard is designed for organizations with calibration laboratories and would only be applicable if a company can conduct internal calibrations on its devices (37).On the other hand, ISO 10012 (38) can be used to manage measurement processes and metrological confirmation of measurement equipment used to support and demonstrate compliance with metrological requirements in the context of medical device metrology.The standard specifies the quality management requirements of a measurement management system that an organization can employ as part of its overall management system, ensuring compliance with metrological requirements.
With the large number of small and microenterprises in the healthcare sector in Latin America, the implementation of standards and metrological assurance systems becomes even more relevant.This is essential to ensure measurement quality and, through that, the safety of diagnoses and patients, while also enhancing the competitiveness of these companies with procedures that can later be certified or accredited with ISO 9001 (16) or ISO 10012 (38).
Undoubtedly, the COVID-19 pandemic has emerged as an unparalleled event, delivering a seismic impact on the healthcare sector across various Latin American nations.In Peru, a country that has witnessed a surge in public health research over the past 50 years (39), there exists a palpable need for initiatives aimed at overseeing adherence to medical standards.This necessity arises as many research endeavors confront inherent limitations in their methodological approaches and outcome analyses (40).Simultaneously, the healthcare sector grapples with challenges such as a shortage of adequately trained personnel and constraints in the utilization of available technologies integral to daily clinical practice (41).Notably, certain technologies, including telemedicine, have demonstrated successful applications during the lockdown, holding the potential to enhance Clinical Risk Management, thereby elevating the Quality of Care and ensuring Patient Safety (42,43).Nevertheless, it becomes imperative to diligently monitor the post-COVID landscape to propose regulatory frameworks that can effectively govern the seamless technological integration into comprehensive healthcare processes.

Limitations
Firstly, it is noted that the analysis was limited to four countries due to evidence availability.That did not allow us to perform a regional comparison with nations such as Chile, Argentina, Bolivia, Uruguay, or Paraguay, which lack available research on metrology.Secondly, the number of studies found (12/16) on metrology in Colombia might have influenced the overall results.While it is commendable that Colombia has a considerable number of biomedical engineering schools in nine universities (44), other countries like Peru have a more limited offering, with only two recent universities offering this program (45).This could impact the generalization of results, potentially predominantly reflecting the situation in Colombia.Therefore, it is essential to encourage research on metrology (46) across various Latin American countries to understand their realities and promote the development of metrological assurance systems for medical devices, thus ensuring quality in healthcare.

Conclusions and future directions
This study provides valuable insights into the implementation of metrology and quality control in the context of medical devices.The analysis of various applicable standards for medical devices, such as IEC 60601-1-6 and IEC 62366, along with ISO 9000, ISO 9001, ISO 17025, and ISO 10012, highlights the multifaceted approach necessary to ensure safety, accuracy, and reliability in healthcare technology.The research emphasizes the importance of incorporating both international and national legislation, and the inclusion of implementation guides based on ISO 10012, as well as the focus on small businesses, further contribute to the practicality and relevance of this research.Addressing the challenges and requirements of metrology and quality assurance in medical devices, this study presents a valuable resource for advancing healthcare technology and improving patient care.Furthermore, it is crucial for both national and private institutions in each Latin American country to establish education and training programs focused on medical device metrology.This could ensure the quality of diagnoses in both larger hospitals and smaller organizations, benefiting patients.On the other hand, it is essential for academia and research centers to work on providing evidence and information availability, as well as conducting reference studies on medical device management and applicable metrological assurance plans.This will foster continuous advancement and improvement in the implementation of practices that ensure quality and accuracy in healthcare.

FIGURE 1 PRISMA
FIGURE 1PRISMA flowchart of the selected studies.

TABLE 1
Bibliometric characteristics of the included studies.

TABLE 2
Standards cited in each included study.

TABLE 3
Description of the work performed according to the institution where the study was conducted.assurance plan for the Intensive Care Unit and Surgical Ward of Clínica Universidad de La Sabana.The study shows flowcharts used for conformity assessment.the importance of metrological assurance when implementing a quality management system at the Center for Immunology and Biological Products at the University of Medical Sciences of Camagüey.enabling the design and control of metrology processes in healthcare institutions by evaluating the calibration index, which determines the minimum calibration frequency required for various medical equipment.The model comprises three stages: analysis of a range of medical devices across eight healthcare entities, formulation of a metrological assurance plan, and establishment of necessary control mechanisms for these purposes.

TABLE 4
Studies on guidelines for the implementation of metrological assurance systems.Guideline Formulate a methodology for the implementation of a metrological assurance plan for medical devices in healthcare institutions following the ISO 10012 framework.Colombia Rodríguez et al. (25) Guideline Develop a set of recommendations to follow during the implementation of a metrological assurance plan for medical devices in healthcare service providers, based on ISO 10012:2003.facilitating the implementation of measurement assurance throughout the lifecycle of medical devices, in accordance with the ISO 10012:2003 standard and compliance with Costa Rican national legislation.Brazil Campese and Costa (26) Guideline Construct a guide for involving users in the verification process of medical devices, based on usability and electrical safety standards, tailored for small businesses.