Evaluation of emergency drills effectiveness by center of disease prevention and control staff in Heilongjiang Province, China: an empirical study using the logistic-ISM model

Introduction Emergency drills are critical practices that can improve the preparedness for crisis situations. This study aims to comprehend the evaluation of emergency drill effectiveness by the staff at the Centers for Disease Control and Prevention (CDC) in Heilongjiang Province, China. It identifies potential factors that could influence the personnel’s appraisal of outcomes throughout the emergency drill procedure. Methods A cross-sectional survey was conducted among public health professionals from various CDCs in Heilongjiang, a northeastern Chinese province. The binary logistic regression analysis identified the factors associated with the CDC staff’s assessment of emergency drill efficacy, while the Interpretative Structural Modeling (ISM) elucidated the hierarchical structure among the influencing factors. Results 53.3% (95% CI = 50.6–55.4) of participants perceived the emergency drills’ effectiveness as low. Binary logistic regression analysis revealed that the following adverse factors associated with the emergency drills increased the risk of a lower evaluation: lack of equipment and poor facilities (OR = 2.324, 95% CI = 1.884–2.867), poor training quality (OR = 1.765, 95% CI = 1.445–2.115), low leadership focus (OR = 1.585, 95% CI = 1.275–1.971), insufficient training frequency (OR = 1.539, 95% CI = 1.258–1.882), low skill in designing emergency drill plans (OR = 1.494, 95% CI = 1.180–1.890), lack of funding (OR = 1.407, 95% CI = 1.111–1.781), and poor coordination between departments (OR = 1.335, 95% CI = 1.085–1.641). The ISM revealed the hierarchical relationship of the influential factors, which were classified into three levels: Surface, Middle and Bottom. The Surface Level factors were training frequency, training quality, leaders’ focus, and inter-departmental coordination. The Middle Level factors were equipment availability and skill in designing emergency drill plans. The Bottom Level factor was funding guarantee. Discussion This survey revealed that over half of the CDC staff rated the effectiveness of public health emergency drills as low. The Logistic-ISM Model results indicated that the evaluation of drill effectiveness was negatively influenced by insufficient facility and equipment support, financial constraints, lack of departmental coordination, and inadequate leadership attention. Among these factors, funding guarantee was the most fundamental one. Therefore, this calls for strategic decisions to increase funding for equipment, leadership training support, and effective emergency coordination.


Introduction
Many disasters and emergencies are unpredictable and need to be tested through practical simulations to enhance the preparedness of the personnel involved.Emergency drills are useful tools for this purpose (1)(2)(3).Emergency drills allow departments and individual practitioners to evaluate their ability to respond quickly and effectively to real-world crises (4).Since these drills often occur without prior notice, responders must show rapid and accurate responses (5,6).Simulated emergency drills are essential components of any robust emergency preparedness framework (7).They aim to be part of a continuous cycle that includes planning, training, practicing, identifying weaknesses, improving areas, and implementing corrective actions.
According to the "Technical Guidelines for Health Emergency Drills" published by the Chinese Center for Disease Control and Prevention in 2013, emergency drills can be understood as a simulated training and experiential learning process for responding to sudden public health events (8)."How to deal with accidents, how to conduct drills, " just like "how to fight, how to train soldiers." In summary, emergency drills are by no means a simple repetition of past emergency practices.They emphasize the exploration of standardized emergency response or the simulation of "worst-case scenarios" (9).Furthermore, drills are training, which emphasizes "scenario-driven, role-playing, teaching interaction, and joint evaluation (10)." After the devastating SARS outbreak in 2003 and the Wenchuan earthquake in 2008, the Chinese Central Government shifted its focus to establishing an integrated national emergency response system that can anticipate, assess, mitigate, and manage potential disasters whenever possible (11).Amid the escalating COVID-19 crisis in 2019, the Central Government gave more priority to the role of CDCs in protecting public health and intensifying health emergency drills.However, for emergency drill projects, people's evaluation of their effectiveness is often unsatisfactory.
Studies have shown that the design, implementation and organization of drills can affect the testing and improvement of emergency drill procedures, which in turn affect the evaluation of the effectiveness of emergency drill (4,(12)(13)(14).Enough funding, equipment, and logistical support are needed to make the simulations mimic life-threatening situations and prepare for actual events (15).For example, a four-day emergency exercise in New South Wales, Australia, simulated a new strain of the flu virus, with enough equipment backup and technical support and keeping it authentic through strict monitoring.The responders did well when H1N1 hit just 8 months later (16).This resulted in efficient emergency response efforts and less economic damage.However, after the SARS outbreak in China, the main form of training for two decades was short lectures that lacked specificity and novelty (17).The participants were diverse and not very engaged, making it hard for the training to meet their needs.Some of the other limitations are lack of funding, unclear or vague guidelines, lack of interest from senior leaders, coordination problems among departments, lack of institutional security frameworks, no direct link to employee evaluations, and low motivation among workers (18)(19)(20)(21)(22).All these factors will have a negative impact on people's evaluation of emergency drills.While emergency drills are becoming more popular worldwide because they help gain practical skills quickly, there is not much research on their use, suitability, and efficiency in China, even though they are getting more recognition.
To evaluate the relevant factors for the effectiveness evaluation of emergency drills, the Logistic regression model can be used.However, the Logistic regression model cannot reveal the hierarchical relationship among these relevant factors (23,24).To do this, the ISM method can be further used to analyze the significant factors in the Logistic regression model, and thus identify the surface, intermediate and bottom factors.ISM is a technique for analyzing complex problems by creating structural models, developed by Professor Warfield in 1973 (23).It uses a matrix model to decompose the complex relationship among relevant factors into a clear multi-level structure through a grouping and ranking process (24).
In summary, this study aimed to understand how CDC personnel in Heilongjiang Province evaluated the effectiveness of emergency drills based on their feedback and insights, using the Logistic-Ism method, to identify the obstacles and potential factors that influenced their appraisal of outcomes, and to display the hierarchical relationship between these factors.The results of this study not only help to gain a deeper understanding of the current situation of the effectiveness evaluation of emergency drill projects by the Heilongjiang CDC staff, but also provide an important basis for optimizing the ways to improve the effectiveness of emergency drills.

Data collection
This study conducted a cross-sectional anonymous survey of disease prevention and control practitioners at the municipal, district, and county levels in Heilongjiang, China.We used a multistage stratified cluster sampling method to select the participants.We randomly sampled seven administrative regions out of the 13 in Heilongjiang province in the first stage.In the second stage, we performed cluster sampling of the 40 municipal, district, and county-level disease prevention and control institutions within these seven regions.We distributed paper questionnaires to all staff members in each institution.The inclusion criteria for this study were as follows: (1) voluntary participation, (2) signing an informed consent form, (3) having participated or known someone who had participated in at least one emergency drill project, and (4) providing personal assurance of the accuracy of the information.The exclusion criteria were as follows: (1) missing key items in the questionnaire, (2) contradictory answers, and (3) too short or too long response time.

Questionnaire design
To evaluate the effectiveness of emergency drills, this survey questionnaire was designed based on the 'Technical Guide for Health Emergency Drills' issued by the Chinese Center for Disease Control and Prevention (8).It enumerated several evaluation criteria for the participants' reference.The participants were encouraged to consider factors such as financial support, facilities and equipment, departmental coordination, and leadership attention to ultimately provide an overall assessment score.The content for validity assessment could be divided into the following aspects: 1. Exercise purpose: Although emergency drills can achieve many specific objectives, they can be generally summarized into two aspects: First, whether the staff were trained, enabling the participants to access their respective emergency functions and roles, and thus gain more skills and experience.Second, whether the system was improved, by refining the health emergency plan through the drill, and further promoting the comprehensive improvement of the health emergency management system.2. Exercise objectives: First, whether the existing health emergency plans, implementation schemes, and operating procedures were tested and evaluated, and whether the shortcomings were revealed.Second, whether the responsibilities of each department and organization were clarified, and whether the coordination and communication among them were strengthened.Third, whether the personnel with corresponding emergency functions and roles were trained, and whether their abilities and levels were improved.Fourth, whether the lack of resources for health emergency work was revealed.Fifth, whether the recognition and support for the emergency drill planning were enhanced in the follow-up.3. Exercise content: Whether the relevant emergency functions were selected for content development according to the purpose and objectives of the drill, such as monitoring and early warning, risk assessment, field epidemiological investigation, field rapid testing, laboratory testing and identification, etc. 4. Exercise evaluation: whether there is a scientific and rigorous evaluation mechanism, specific evaluation indicators and quantitative standards, and whether corresponding improvement measures have been taken after the drill (Figure 1).

Variable 2.3.1 Outcome variable
The outcome measure was the overall evaluation of the effectiveness of emergency drills by CDC personnel.This was assessed Reference content framework for evaluation of the effectiveness of emergency drills.These scores were divided into two categories for logistic regression modeling: "high effectiveness" (scores above the average of 3.16) and "low effectiveness" (scores equal or below the average of 3.16).

Explanatory variable
The explanatory variables were divided into several categories according to previous studies, such as demographics (gender, age, education, work tenure), previous training experience (frequency and quality of training sessions), components of emergency drills (funding, personnel, site, equipment, security, leadership, and coordination), types of exercises conducted in the past 5 years (tabletop, operation-based, functional, and comprehensive), and emergency drill disposal skills (plan designing, script writing, form designing, and needs assessment).
Age was divided into four groups, namely, less than or equal to 30 years, 30-39 years, 40-49 years, and greater than or equal to 50 years; work tenure was also divided into four categories, namely, 0-9 years, 10-19 years, 20-29 years, and greater than or equal to 30 years.The number of training sessions was binary ("none" or "≥1"), the quality of training was either "low quality "or "high quality ", the components of emergency drills were either "yes" or "no, " and the answers for emergency drill disposal skills were rated on a 5-point scale, ranging from 1 (very poor) to 5 (very good).In order to apply the binary logistic regression method, the answers were finally coded as two-dimensional, including bad (0) and good (1). 1, 2, and 3 were classified as bad, and 4 and 5 were classified as good.
Among the four types of emergency drills used in the past, a tabletop drill is a discussion of different measures based on a hypothetical scenario, an operation-based drill is a practice of actual combat in a real-life setting, a functional drill is a drill based on an emergency response function or some of its actions, a comprehensive drill is a drill that covers all or most of the emergency response functions in an emergency plan.

Statistical analysis
This study applied two methods to analyze the survey data: binary logistic regression, which examined the effect of individual predictors on the main outcome variable, and Interpretative Structural Modeling (ISM), which explored the interconnections and hierarchies of factors.The data were organized and analyzed using SPSS V.19.0 and MATLAB V.9.0.First, univariate descriptive statistics were obtained for all variables considered in the study.Then, Pearson's χ 2 was used to test the association between all variables and self-assessment of the effectiveness of emergency drills, with a two-tailed value of p < 0.05 indicating statistical significance.Factors that were significant in the χ 2 test were included in the regression analysis.Then, the significant factors were further screened by using binary logistic regression analysis, and the hierarchical structure among them was revealed by using ISM method.
The specific steps of ISM analysis are as follows: Step 1: Judge the logical relationship between the influencing factors and construct the adjacency matrix R.
The adjacency matrix is a way of representing the matrix of relationships between factors.It provides a clear description of the direct relationship between factors.In this study, S 0 is used to refer to the selection of the effectiveness of emergency drills for CDC responders.On the other hand，S i (=1,2,…,k) is used to denote the k significant factors that affect the evaluation of the effectiveness of emergency drills.The adjacency matrix R is constructed based on the logical relationship between the factors, which is defined as follows: 1, Among them,i 0,1, 2 , k; j 0,1, 2 , k. 0, The adjacency matrix R ij represents the binary direct relationship between the factors within the system through the form of a 0-1 matrix.Through extensive information gathering and consultation with experts, we can determine the direct relationship between factors that affect the evaluation of the effectiveness of emergency drills for CDC personnel.If S i directly affects S j ,we assign a value of 1 to R ij .
If not, we assign a value of 0. By doing so, we can generate the adjacency matrix, R.
Step 2: Based on the adjacency matrix R, create the reachable matrix M.
The reachable matrix describes the relationship between factors' reachability in the system.Where I is the unit matrix, 2 ≤ λ ≤ k, and the Boolean operator is used for the power operation of the matrix.
Step 3: Determine the hierarchical structure L between the factors.
where R S i ( ) is the reachable set and A S i ( ) represents the prior set.After obtaining the elements contained in the highest layer L, the rows corresponding to the factors in L are rounded off from the original reachable matrix M to obtain the matrix M 2 , which is repeated to obtain L 2 , and so on the factors contained in all layers are obtained, and the hierarchical structure of the factors influencing the evaluation of the effectiveness of emergency drills is finally obtained.

Ethical considerations
This study adhered to the Declaration of Helsinki principles and received approval from the Institutional Review Board ethics committee at Harbin Medical University.The study subjects gave informed consent before participating.The information collected was anonymous and confidential to safeguard the privacy of the subjects.

Characteristics of participants and self-rated effectiveness of emergency drills
The survey sample consisted of 1,859 subjects, of whom 59.1% were female.The majority of the participants (84%) had a Bachelor  1).
The training methods differed in frequency and type.Operations-based drills were the most common methodology reported by participants, accounting for 36% of the total training sessions.The next most common method was tabletop drills (31%), followed by function-oriented drills (20%) and comprehensive drills (13%).More than half (53.3, 95% CI = 50.6 to 55.4) of the 1,859 participants rated the overall effectiveness of the emergency drills as low.

Chi-square test and binary logistic regression results
Using χ 2 test we identified the statistically significant association between the perceived effectiveness of emergency drills and gender, educational background, training frequency, training quality, fund guarantee, equipment and facilities availability, leaders' focus, inter-departmental coordination, tabletop drill exercises, operation-based drill exercises, and skills in emergency drill plan design, script writing, form design, and needs assessment (Table 2).
Significant contributors from Chi-squared investigations filtered into the logistic regression equation.In the results of the binary logistic regression model, the Hausman test indicated that the model was well fitted (p = 0.745>0.05).The logistic regression tests revealed seven major predictors of the evaluation of the effectiveness of emergency drills.These predictors were: equipment and facilities availability, training quality, leaders' involvement, number of training sessions, skill in emergency drill plan design, fund adequacy, and interdepartmental coordination.
The results showed that the likelihood of obtaining a lower effectiveness rating for emergency drills with insufficient equipment and facilities was 2.324 times (95% CI = 1.884-2.867)that of emergency drills with adequate equipment and facilities.Compared with participants who received high-quality training, participants who received low-quality training were 1.765 times (95% CI = 1.445-2.155)more likely to negatively evaluate the effectiveness of emergency drills.Emergency drills without leader attention were more likely to receive lower effectiveness ratings than those with leader attention (OR = 1.585).In addition, training frequency (OR = 1.539), emergency drill plan designing skill (OR = 1.494), funding guarantee (OR = 1.407), and inter-departmental coordination (OR = 1.335) were also important predictors of the effectiveness evaluation of emergency drills (Table 3)

ISM analysis results
The specific results and the process of their analysis are described below: 1. Construct the adjacency matrix R: The seven significant influencing factors from the results of the binary logistic model were labeled as S1-S7.We consulted experts and conducted extensive research to establish the adjacency matrix R for the influencing factor system.We used an ISM group to analyze and discuss the matrix.0 1 0 0 0 1 0 0 0 0 0 1 0 0 1 1 0 1 1 1 1 0 1 0 0 0 1 0 1 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 1 1 0 0 0 1 0 1 1 0 0 0 1 0 The set of system factors for the columns corresponding to the matrix elements with value 1 in the ith row of the reachable matrix, i.e., the reachable set R (S i ).The set of system factors for the row corresponding to the matrix element with value 1 in column i of the reachable matrix is the prior set A (S i ).The intersection of the reachable set and the prior set is the common set, i.e., T (S i ) = R (S i ) ∩ A (S i ).Accordingly, based on the reachability matrix M, the above set of each influencing factor is obtained as shown in the following table.
Then, by comparing the relationship between the intersection set T (S i ) and the reachable set R (S i ), the first layer of the set of factors, i.e., the factors that satisfy T (S i ) = R (S i ), is determined.Next, the  stratified factors are eliminated from the reachable matrix M to obtain the new matrix M 1 , and the previous operation is repeated until all the factors are stratified (Table 4).
3. Determining the hierarchical structure L: Our model aims to identify the variables that influence the evaluation of the effectiveness of emergency drills for CDC personnel.We based our model on the stratification results and influence paths from the reachability matrix.We classified the system of influence factors into three levels: surface, middle, and bottom (Figure 2).Surface level factors.These factors have the most direct impact on the evaluation of the effectiveness of emergency drills.They include four items: training frequency (S1), training quality (S2), leaders' focus (S5), and inter-departmental coordination (S6).
Middle level factors.These factors are the core of the system.They include two items: equipment and facilities availability (S4) and skill in emergency drill plan design (S7).
Bottom level factor.This factor is the foundation of the system that determines the overall performance.It includes one item: fund guarantee (S3).

Having financial guarantee is crucial to support emergency drills
This study identified fund guarantee as the bottom-level factor that influences the evaluation of the effectiveness of emergency drills for CDC personnel.Lack of fund guarantee led to poor equipment and facilities for emergency response and ineffective emergency measures, thus compromising the evaluation of the drills.
Funding is a vital factor for conducting emergency drills in an organized and efficient manner.Without sufficient financial support, the drill may not be executed as planned.Emergency drill funding covers the expenses of materials, site rental, and personnel training needed for the drill process (25).First, equipment procurement and maintenance are essential to ensure the availability and functionality of the required tools and equipment, such as alarm systems, road signs, and rescue equipment.Adequate funds can facilitate smooth equipment procurement and maintenance for the drill process.Second, funding can be used to pay for site rental fees, environmental improvement, and temporary facilities.This ensures the site is appropriate and well-prepared.Third, personnel training and incentives are necessary for the participation and coordination of government personnel at various levels.Providing subsidies can motivate the personnel involved in the drill process.
Many developed countries assign great importance to emergency drills in the face of public health crises and have allocated significant funds toward their implementation (26).For instance, the US government has invested $21 million in drills for senior officials, highlighting the priority given to emergency preparedness (27).However, emergency drills are still in their infancy stage in China, despite substantial investments in the public health system since the SARS outbreak in 2003 (28).Therefore, it is imperative for the state and health-related financial sectors to increase their support for emergency drills.Relevant departments should include emergency drill funds in their financial budgets, establish economic evaluation and compensation mechanisms, and enhance funding mechanisms to Factor R (Si) T (Si) 10.3389/fpubh.2024.1305426 Frontiers in Public Health 08 frontiersin.orgensure the smooth implementation of emergency drills.Moreover, it is recommended that the state establish and refine corresponding laws and regulations, prioritize emergency drills in the legal management track, and establish a comprehensive emergency drill system.

Adequate equipment and facilities are the basic requirements for conducting emergency drills
This study identified two intermediate-level factors that influence the evaluation of the effectiveness of emergency drills for CDC personnel: equipment and facilities availability, and skill in emergency drill plan design.The availability of adequate equipment and facilities helped the stakeholders perform their tasks confidently and comprehensively during the simulation runs, and improved their overall emergency response capabilities in areas such as communication command, emergency rescue, medical rescue, and subsequent disposal (29).
Emergency drills require various resources, such as essential personal protective equipment, mobile laboratories for pathogen detection, medical treatment facilities and equipment, proper medical waste disposal methods, and vehicles for swift mobilization in case of infectious disease outbreaks.Moreover, for catastrophic events like earthquakes, rescue and rehabilitation sites must be established.All these measures depend on an adequate supply of equipment and facilities.However, China currently faces a shortage of several categories of emergency supplies, and some of the existing emergency materials do not meet the required standards (30).Therefore, it is crucial to improve the quantity and quality of emergency stockpiles, and provide adequate facilities and equipment for emergency drills to enhance their overall performance (31).
Computer-assisted simulation technology can help address these issues (6,32).In China, the "13th Five-Year Plan for the Construction of a National Emergency Response System" advocates for the creation of a national public safety emergency experience base.The utilization of virtual simulation technology to replicate various disaster scenes, characterized by minimal space requirements, reduced costs, and reusability, has a vast potential in the emergency management domain (33).In this modern technological era, cost-effective emergency drills would be challenging to implement without the support of advanced electronic technologies such as 3D simulation systems or VR systems (34).

A qualified design of emergency drill plan is the key
The success of emergency drills depends largely on the design of the emergency drill plan, which serves as a blueprint for conducting the drills.Therefore, meticulous planning is essential, as it directly affects the drill's impact and management (35,36).A well-designed emergency drill plan can enhance the leadership's ability to prioritize and coordinate the response efforts, and improve the flow and sequence of the processes for successful deployment.Hence, creating hypothetical scenarios requires a comprehensive and thoughtful approach.
Fundamentally, drills are about "answering questions." Every scenario within the drill atmosphere requires a response, and if these "scenarios" are viewed as "questions, " then responding to the scenarios equates to "answering questions (10)." The scope of "problems" is broad, encompassing all situations related to activating emergency responses before, during, and after an incident.When the scenario construction and drill scene settings are rigorous, every sentence, phrase, word group, and even punctuation in the scenario description can be a "problem" that necessitates a thoughtful answer (10,37,38).All of this relies on the meticulous advance design of the emergency drill plan.However, at present, there are significant shortcomings in the overall design of emergency drill plans.Many designs are too ordinary and disregard the unexpected nature of public health emergencies.This makes the emergency drills lack authenticity and a scientific basis.
To address this issue, the design of the rehearsal plot should be as realistic and diverse as possible to ensure the quality of emergency drills.Additionally, in the planning of drills, it is possible to deconstruct the functions of contingency plans according to certain standards, breaking them down to the smallest executable units.This involves specific emergency functions such as the roles and responsibilities outlined in the plan, emergency mobilization, resource management, facility and equipment support, early warning and announcements, traffic control, management of evacuated personnel, medical services, external reinforcement, situation control and on-site recovery, documentation and investigation (10).The aim is to achieve the exercise objectives for each module as much as possible.

Establish multi-sectorial coordination mechanism and strengthen the leadership's attention
This study showed that the evaluation of the effectiveness of emergency drills is directly influenced by the surface level factors, which include four items: training frequency, training quality, interdepartmental coordination, and leaders' focus.Some units did not organize or conduct the drills effectively or regularly, resulting in low attention and participation from the personnel.This made it difficult to identify and solve problems, exercise the team, and enhance the overall capabilities, thus reducing the evaluation of the effectiveness of emergency drills (28).Moreover, poor training quality prevented the full realization of the functions of the drills.
Notably, insufficient leaders' involvement led to superficial drills with low-quality implementation.In China, the government budget allocation authority is primarily controlled by the central, provincial, municipal, and county-level financial departments.Due to decades of decentralization reforms, local governments are responsible for financing local health and other social services.Although it is a national policy that requires local CDCs to undergo emergency training, only wealthier provinces and municipalities with more funds and resources, where leaders prioritize it, provide sufficient support for emergency drills.Compared to traditional knowledge-based training methods such as lectures, courses, or workshops -often funded by the central government as a nationwide earmarked program -skill-based emergency drills rarely receive funds from the center.Therefore, the financing responsibility mainly falls on the shoulders of local governments, implying the need for extra human, material, and financial resources.If leaders in poor areas fail to recognize the importance and irreplaceability of emergency drills and consider them a waste of resources, they may be reluctant to allocate extra funds.To improve this situation, various methods such as financial support, technology transfer, leadership training, and legal enforcement should be adopted.
Finally, departmental coordination was a vital aspect of the drills.Without close collaboration between departments, it was hard to achieve rapid response and synergy.However, currently, emergency drills mainly consist of individual departmental activities, with administrative units operating independently and lacking a multiagency coordination mechanism (39).The absence of coordination, coupled with conflicts among agencies, can often result in extensive time loss, resource waste, duplication, and uncoordinated and inappropriate responses (40).Therefore, to improve effectiveness, teams should undergo comprehensive training to develop knowledge, skills, and attitudes; enhance coordination; foster a shared mental model and accurate expectations of team requirements; and encourage adaptability and flexibility (41).Multi-agency emergency drills can establish and strengthen relationships, bring together individuals from different backgrounds to work as a team, clarify goals, understand roles and responsibilities, and appreciate each agency's strengths and weaknesses (42, 43).

Limitations of this study
Although this study used the Logistic-ISM method to identify the factors related to the CDC staff 's evaluation of the emergency drill outcomes, and further derived the hierarchical structure of the related factors, some limitations still need to be addressed.
First, this survey may not capture all the potential factors that may cause reasoning bias, and the presence of omitted variables remains an area for future research.Second, this study used cross-sectional data for analysis, which cannot obtain causal effects.Future studies can use longitudinal data sets to capture more accurate relationships between the research variables and the CDC staff 's evaluation of the emergency drill outcomes.Third, this study only surveyed the data from Heilongjiang Province, which limits the generalizability of the research results.Future studies can expand the sample size, include more provinces in China, and explore the potential differences in the results obtained in different regions.Finally, this study's sampling design used a multistage cluster sampling method, but did not account for this in the statistical analysis, which could affect the accuracy of parameter estimation and hypothesis testing.Future research should use more appropriate statistical methods to deal with the multistage sampling issue.

Conclusion
This paper explores the factors that affect the evaluation of the effectiveness of emergency drills and their hierarchical structure based on actual research data.The paper uses Logistic model and ISM analysis to identify and analyze the factors.The results of the study show that the evaluation of the effectiveness of emergency drills is the result of a combination of factors.Firstly, seven factors have significant influence on the effectiveness of emergency drills.They are equipment and facilities guarantee, training effectiveness, leaders' focus, training frequency, skill in emergency drill plans design, fund guarantee, and inter-departmental coordination.Secondly, the seven dominant factors are both independent and interrelated, and are at three different levels.Among them, training frequency, training quality, leaders' focus, and coordination between departments are factors that directly affect the effectiveness of emergency drills for public health emergencies; they are the surface influences.The intermediate influences are equipment and facilities guarantee and skill in emergency drill plans design.Fund guarantee is the deep-rooted factor.

FIGURE 1
FIGURE 1 . Construct the reachable matrix M: Calculate the reachable matrix M from the adjacency matrix R of the influencing factor system.

FIGURE 2
FIGURE 2 Hierarchical structure diagram of factors influencing effectiveness evaluation of emergency drills.
, and 37.7% had more than 20 years of work experience.Most participants were in the age range of 30 to 49 years (Table

TABLE 2
Chi-square test results of all explanatory variables (N = 1859).

TABLE 3
Factors influencing effectiveness evaluation of emergency drills.