Diagnostic Accuracy of Oxygen Desaturation Index for Sleep-Disordered Breathing in Patients With Diabetes

Background Polysomnography (PSG) is the gold standard for diagnosis of sleep-disordered breathing (SDB). But it is impractical to perform PSG in all patients with diabetes. The objective was to develop a clinically easy-to-use prediction model to diagnosis SDB in patients with diabetes. Methods A total of 440 patients with diabetes were recruited and underwent overnight PSG at West China Hospital. Prediction algorithms were based on oxygen desaturation index (ODI) and other variables, including sex, age, body mass index, Epworth score, mean oxygen saturation, and total sleep time. Two phase approach was employed to derivate and validate the models. Results ODI was strongly correlated with apnea-hypopnea index (AHI) (rs = 0.941). In the derivation phase, the single cutoff model with ODI was selected, with area under the receiver operating characteristic curve (AUC) of 0.956 (95%CI 0.917–0.994), 0.962 (95%CI 0.943–0.981), and 0.976 (95%CI 0.956–0.996) for predicting AHI ≥5/h, ≥15/h, and ≥30/h, respectively. We identified the cutoff of ODI 5/h, 15/h, and 25/h, as having important predictive value for AHI ≥5/h, ≥15/h, and ≥30/h, respectively. In the validation phase, the AUC of ODI was 0.941 (95%CI 0.904–0.978), 0.969 (95%CI 0.969–0.991), and 0.949 (95%CI 0.915–0.983) for predicting AHI ≥5/h, ≥15/h, and ≥30/h, respectively. The sensitivity of ODI ≥5/h, ≥15/h, and ≥25/h was 92%, 90%, and 93%, respectively, while the specificity was 73%, 89%, and 85%, respectively. Conclusions ODI is a sensitive and specific tool to predict SDB in patients with diabetes.


INTRODUCTION
Sleep-disordered breathing (SDB) is characterized by episodic sleep state dependent collapse of the upper airway, resulting in periodic reductions or cessations in ventilation, with consequent hypoxia, hypercapnia, or arousals from sleep (1). SDB is a common disease with a prevalence of 3% among women and 10% among men 30 to 49 years of age and 9% among women and 17% among men 50 to 70 years of age in the general population (2). It has been estimated that there is a high prevalence of undiagnosed SDB in patients with type 2 diabetes (24-86%) (3,4). Moreover, the prevalence of moderate-to-severe SDB was more than 50% in patients with diabetic foot ulcers (DFUs) (5). Patients with SDB are more likely to have hypertension, stroke, heart failure, and depression (6). Furthermore, SDB is associated with increased risk of complications of diabetes, such as diabetic polyneuropathy, peripheral artery disease, and ulcer development (7,8). Thus, early diagnosis of SDB in patients with diabetes is essential.
The overnight polysomnography (PSG) is the gold standard for the diagnosis of SDB. However, PSG is a time-consuming and costly procedure. In addition, it is a rather uncomfortable experience for some patients. In our previous study, nearly one-third of the patients with diabetic foot ulcers did not complete the process of the test because of the uncomfortable experience (5). Home sleep apnea testing can also be used for diagnosing sleep apnea in uncomplicated patients. However, patients with diabetes often suffer from diabetes related complications, such as diabetic neuropathy, which might compromise the result of home sleep apnea testing (9). Thus, it is necessary to develop an easy-to-use method based on clinical features and accessible metrics to predict the likelihood of SDB.
Many screening tools have been developed, such as the oftenutilized Epworth Sleepiness Scale and "STOP-BANG". However, they have minimal predictive value for SDB (10)(11)(12). Multistep screening approaches, such as using a questionnaire or prediction tool followed by overnight home-based testing, did not have adequate discrimination (13). Machine learning, such as Neural Network and Support Vector Machine, have been utilized to predict the SDB using clinical features (e.g. age, sex, and smoking) and physiologic measurements (e.g. blood pressure, overnight pulse oximetry, and pulmonary function) (14,15). The predictive performance varied among studies with sensitivity from 66% to 100%, and specificity from 30.8% to 76.2% to predict AHI ≥5/h. However, the predictive models derived from machine learning are sometimes difficult to understand and hard to use in clinical practice (16). Furthermore, most studies were carried out at sleep centers in the general population with possible SDB. Given the high prevalence of SDB in patients with diabetes and the bidirectional association between SDB and diabetes (17), it is crucial to validate the criminative performance of models in patients with diabetes. However, there was lack of study that investigated the predictive models in patients with diabetes.
SDB is often accompanied with hypoxemia. Oxymetric measures can be extracted from pulse oximetry, including mean oxygen saturation, percentage of time with oxygen saturation < 90%, and oxygen desaturation index (ODI). A systematic review evaluated the diagnostic accuracy of ODI for SDB, significant heterogeneity exists among studies with sensitivities ranged from 32% to 98.5%, while specificities ranged from 47.7% to 98% (18). Furthermore, studies have revealed that hypoxemia, rather than AHI, were well associated with cardiovascular risk (19,20). Till now, no study has been published to evaluate the diagnostic performance of ODI in patients with diabetes. The high prevalence of SDB in patients with diabetes and associated diabetic complications such as diabetic neuropathy might influence the predictive accuracy of ODI (4,21). Thus, it is imperative to evaluate the diagnostic performance of ODI in patients with diabetes.
The purpose of the study was to develop a model based on ODI to identify patients with SDB at three AHI cutoffs (≥5/h, ≥15/h, and ≥30/h) in patients with diabetes.

Patients
In this study, a total of 440 patients with diabetes were recruited and underwent overnight PSG at the ward of Endocrinology and Metabolism of West China Hospital, Sichuan University between July 2013 and June 2019. Patients who had a diagnosis of specific types of diabetes due to other causes, such as Cushing syndrome, Acromegaly, and those who were being treated with corticosteroids, immunosuppressive drugs, or chemotherapy, were excluded. Patients who experienced severe hypoglycemia were excluded. No participant had prior diagnosis of sleep disorders or was being treated with continuous positive airway pressure. This study was approved by the Biomedical Research Ethics Committee of West China Hospital of Sichuan University, and all participants provided written informed consent.
Demographics, comorbidities, and laboratory data were obtained from the patient's medical record at the date of admission. The demographics included age, sex, smoking, BMI, duration of diabetes. Daytime sleepiness was assessed by the Epworth Sleepiness Scale (ESS), with excessive daytime sleepiness defined as ESS ≥10 (22).

Procedures
An overnight unattended portable PSG recording (Somte, Compumedics, Australia) was performed inpatient, five to seven days following hospitalization. Subjects were allowed to sleep based on their habitual sleep time. The apparatus included electroencephalography, bilateral electrooculography, electrocardiography, electromyography (submental and anterior tibialis), nasal pressure and thermal airflow, thoracoabdominal movements, and pulse oxygen saturation. Sleep recordings were scored by certified sleep technologists, who were blind to any diagnosis.
Respiratory events were manually scored according to 2012 AASM recommendations (23). Apnea was defined as a drop of at least 90% of airflow from baseline lasting 10 s or longer. Hypopnea was defined as a reduced airflow of 30% from baseline for more than 10 s, in association with either a 3% oxygen desaturation or an arousal. AHI was calculated as number of apnea and hypopnea events per hour of sleep. SDB severity was graded as follows: normal (AHI <5 events/hour), mild (AHI 5-14.9 events/hour), moderate (AHI 15-29.9 events/ hour), and severe (AHI ≥30 events/hour). The three cutoffs of AHI (≥5/h, ≥15/h, and ≥30/h) for SDB were regarded as the reference standard.
Because oximetry is one of the PSG channels, we derived oxymetric measures, including mean oxygen saturation and ODI (number of events per hour in which oxygen saturation decreased by ≥3% from baseline), from the PSG test.

Statistical Analysis
We summarized data as either the number of patients (%), mean (SD), or median (IQR). We did the bivariate analysis with Chisquare test, Student's t test, or Mann-Whitney U test. Random split-sample function was used to randomly split the full data set into two data sets: 2/3 of the patients was used for derivation (derivation data set), and the remaining 1/3 was used for validation (validation data set).
In the derivation phase, spearman correlation between ODI and AHI was analyzed. Scatter plot and simple linear regression between ODI and AHI were depicted and explored. Prediction algorithms were based on oxygen desaturation index (ODI) and other variables, including sex, age, body mass index, Epworth score, mean oxygen saturation, and total sleep time. Three models were applied for each prediction (≥5/h, ≥15/h, and ≥30/h): a model with ODI; a model with ODI and Epworth score; a model with sex, age, BMI, Epworth score, ODI, oxygen saturation, and total sleep time (full model). K-Fold Cross-Validation (k=10) was used to generate the model. DeLong's test was carried out to compare the area under the receiver operating characteristic curve (AUC) among three models. The model with ODI alone was selected because the AUCs were similar among three models. The Youden's index was employed to find the optimal cutoff. McNemar's test was employed to compare the sensitivities and specificities between the derived optimal cutoff and a simplified cutoff point (5/h, 15/h, and 25/h) (24). The cutoff points of 5/h, 15/h, and 25/h for predicting AHI ≥5/h, ≥15/h, and ≥30/h was preferred for reasons of simplicity and ease of use.
For validation, sensitivity, specificity, negative predictive value, positive predictive value, and AUC, with corresponding 95% confidence intervals, were assessed at chosen cutoffs in the validation data set. Predictive performance was also estimated in the full data set. Statistical analysis was performed with Stata version 13 (Stata Corp.). A p value below 0.05 was considered statistically significant.

RESULTS
The baseline characteristics are listed in Table 1 and Supplementary Table S1 (derivation data set) and Table S2 (validation data set) (categorized according to SDB severity). The average age was 58 ± 14 (range 18-89) years, and 66% of patients were men. Most of the patients included were patients with type 2 diabetes, with only three cases of type 1 diabetes. The median duration of diabetes was 9 (3-15) years, with a mean glycated hemoglobin (HbA1c) level of 8.8 ± 2.2%. The median AHI was 23.9 (9.7-41.7)/h, with a prevalence of 90.5%, 62.7%, and 39% for AHI≥5/h, ≥15/h, and ≥30/h, respectively. The clinical demographics and comorbidities were similar between patients in the validation group and those in the derivation group. Overall, the patients above the AHI cutoff had lower mean oxygen saturation, higher T90%, and higher ODI, while there was no significant difference in Epworth score.

Derivation Phase
The median ODI was 26.2/h, 36.1/h, and 49.3/h, respectively, much higher among patients with AHI ≥5/h, ≥15/h, and ≥30/h. For the corresponding patients who had an AHI value below the cutoffs, the median ODI was 2.8/h, 9.1/h, and 12.7/h, respectively ( Figure 1). There was a strong correlation between ODI and AHI, with Spearman coefficient of 0.941. The relationship between ODI and AHI is further depicted in Supplementary Figure S1. Thus, three models based on ODI were set up to predict SDB.
The AUCs were similar in the three models for predicting SDB (Supplementary Table S3). Given the simplicity, the model which included only ODI was selected. The cutoffs identified using Youden's index in the model were 7.4, 16.8, and 26.8.    Figure 2). The sensitivity of ODI ≥5/h, ≥15/h, and ≥25/h was 92%, 90%, and 93% for AHI ≥5/h, ≥15/h, and ≥30/h, respectively, while the specificity was 73%, 89%, and 85%, respectively ( Table 2). Results of AUCs, sensitivity, specificity, negative and positive predictive values with the use of the full data set also showed good diagnostic performance (Supplementary Figure S2, and Figure S3).
To our knowledge, this is the first study to assess the diagnostic accuracy of ODI for SDB in patients with diabetes. The high prevalence of SDB in patients with diabetes and the bidirectional association make it necessary to screen SDB in patients with diabetes (5,17,25). However, because of the lack of a convenient and inexpensive tool, it has been a challenge to precisely identify patients with SDB in the diabetes population in clinical practice. The results of this study suggested that ODI could be a useful tool to assist recognizing patients with SDB. The high sensitivity of 93% of ODI ≥5/h enables us to exclude the possibility of SDB if a patient with diabetes had an ODI ≤5/h. Thus, utility of ODI could reduce the number of PSG tests.
On the other hand, the specificity of ODI ≥15/h and ODI ≥25/h to predict moderate and severe SDB was as high as 86% and 94%, respectively. Therefore, we would be confident to consider a patient with an ODI ≥15/h as having moderate to severe SDB, and a patient with ODI ≥25/h as having severe SDB. Given the high prevalence of SDB in patients with diabetes (4), an ODI more than 25/h would have a high positive predictive value of 83% for severe SDB, as stated in this study. This would prompt clinicians to take measures and avoid delayed detection and therapy of severe SDB for those patients with high ODI, particularly in situations where waiting list for PSG is too long.
Using nocturnal oximetry to screen patients with SDB has been exploited in the general population in previous studies. However, significant heterogeneity exists with sensitivities ranged from 32% to 98.5%, whereas specificities ranged from 47.7% to 98% (18). Chiner et al., Lin et al., Hang et al. found that ODI was a good predictor for AHI, with sensitivities more than 84.0% and specificities more than 87.8% for predicting SDB (26)(27)(28). Chung et al. assessed the predictive value of ODI in patients who were scheduled for inpatient surgery for SDB. They found that ODI was a good predictor for AHI, with AUC for ODI to predict AHI ≥5/h, ≥15/h, and ≥30/h, was 0.908 (95%CI 0.880-0.936), 0.931 (95%CI 0.090-0.952), and 0.958 (95%CI 0.937-0.979), respectively (29). These results are consistent with the high diagnostic accuracy of our study.
However, other studies did not found that ODI provided a satisfactory diagnostic accuracy. Alvarez et al., Golpe et al., and Gyulay et al. found that ODI had a high specificity of 94.3%, 97%, and 98%, with a low sensitivity of 58.7%, 32%, and 40%, respectively (30)(31)(32). The high specificity suggested that ODI was a better rule in tool, rather than a rule out tool. Several possible reasons can be put forward to explain these discrepancies. First, in our study, the pre-test probability of SDB in patients with diabetes is higher than that in the general population. This might lead to the relatively high diagnostic accuracy. Second, the population heterogeneity might contribute the difference of the sensitivity and specificity of ODI, because the accuracy of ODI lies in the detection of apnea while hypopnea are underdiagnosed (78% of apneas and 54% hypopneas caused desaturation) (33). Third, different oximeters might have great difference in calculating the number of desaturations. Fourth, Different criteria for grading SDB severity and different definitions of hypopnea, oxygen desaturation, baseline airflow and baseline saturation might also account for the variations. In our study, neither the inclusion of Epworth score nor excessive daytime sleepiness (Epworth score ≥10) had better predictive effect than ODI alone. Furthermore, the predictive value of Epworth score for SDB was very low with the AUC between 0.5 and 0.6 (Supplementary Table S5). This was consistent with previous studies (10,12). It may be because that sleepiness is not necessarily concordant with SDB severity.  There are a few limitations in this study. First, due to the relatively high age (average age 58 ± 14 years) and high prevalence of commodities of the patients with diabetes in this study, further research should be done to verify whether the results could be generalized to the whole diabetes population. Second, the prevalence of SDB in this study was as high as 90.5%, this may affect the positive and negative predictive value of ODI. However, the sensitivities and specificities should not be affected. Third, the prediction of ODI was based on the AHI cutoffs. Although AHI was regarded as the diagnostic standard of SDB, AHI was not the best predictor of SDB-related morbidity (34). In the future, the morbidity and outcomes, such as cardiovascular mortality, should be considered as the targets of prediction. Fourth, Chronic obstructive pulmonary disease (COPD) or other pulmonary disorders which could influence the oxygen desaturation was not considered in this study. Fifth, although ODI is a good predictor of SDB, it only reflects the oxygen saturation change and does not monitor nasal flow and respiratory effort. Thus, it is not able to distinguish obstructive sleep apnea and central sleep apnea.
The results in our study have implications for clinical practice. The high diagnostic accuracy of ODI for SDB suggests that ODI could be used to reduce the number of PSG test for those with low ODI, and avoid delayed detection and therapy of severe SDB for those patients with high ODI. However, it is worth noting that for patients who are suffering from COPD or other pulmonary disorders, oximetry alone is not an appropriate method to predict SDB. What is more, because the oxygen signal is derived from the PSG system in this study, the predictive activity of ODI has to be validated in pulse oximetry devices. The loss of signal during test of oxygen signal should also be paid attention during clinical practice. Furthermore, in addition to pulse oximetry, oxygen saturation metrics could be derived from other accessible devices, such as smartphones and wearable devices (35). Thus, a simple overnight screening and everyday monitoring could become possible. However, these devices are often not certified to give medical accurate information on oxygen saturation (36). Signal quality, signal treatment, and averaging over several pulses may vary and differ largely compared to oximetry integrated in a polysomnography as used here. Thus, the accuracy of these devices should be evaluated carefully before they can be used to make medical predictions.

CONCLUSION
In conclusion, ODI was strongly correlated with AHI in patients with diabetes. Based on maximal accuracy and simplicity of clinical use, ODI ≥5/h, ODI ≥15/h, ODI ≥25/h were good predictors for mild, moderate, and severe SDB for patients with diabetes, respectively. Further studies should be carried out to validate the diagnostic accuracy of ODI in the communitybased diabetes population for greater clinical application.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Biomedical Research Ethics Committee of West China Hospital of Sichuan University. The patients/participants provided their written informed consent to participate in this study.

AUTHOR CONTRIBUTIONS
LC and WT designed the study, collected data, did statistical analysis, and drafted the manuscript. CW, DC, YG, WM, and PZ designed the study and collected data and contributed to the statistical analysis. FL and XT participated in the study design, data collection and data analysis. XR designed and coordinated the study, acquired funding, and participated in writing and editing the final manuscript. All authors contributed to the article and approved the submitted version.