ORIGINAL RESEARCH article
A U-Shaped Relationship Between Selenium Concentrations and All-Cause or Cardiovascular Mortality in Patients With Hypertension
- 1School of Biology and Biological Engineering, South China University of Technology, Guangzhou, China
- 2Department of Cardiology, Guangdong Cardiovascular Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
- 3Department of Epidemiology, Centre for Global Cardio-metabolic Health, Brown University, Providence, RI, United States
Background: Given the antioxidant activity of selenium, it has been reported benefits for blood pressure control and hypertension prevention, but few studies have investigated the association between serum selenium with mortality in hypertensive population.
Methods: All participants with hypertension aged ≥18 years at baseline were recruited from the National Health and Nutritional Examination Surveys (NHANES) 2003–2004, and followed for mortality through December 31, 2015. Subjects were categorized by quartiles of serum selenium (Q1: ≤124 μg/L, Q2: 125–135 μg/L, Q3: 136–147 μg/L, Q4: ≥148 μg/L). Multivariate Cox regression were implemented to estimate hazard ratios (HRs) and 95% confidence intervals (CIs). Restricted cubic spline analysis and two-piecewise linear regression were used to evaluate the relationship of serum selenium with mortality. Survival curves were used to depict cause-specific mortalities.
Results: A total of 929 participants (52.53% were male) were eligible for the current study with the average age of 63.10 ± 12.59 years. There were 307 deaths occurred including 56 cardiovascular death events during the mean follow-up time of 121.05 ± 40.85 months. A U-shaped association was observed between serum selenium and all-cause or cardiovascular mortality. In fully adjusted model, comparisons among quartiles revealed that risks of all-cause [HR (95%CI), 0.57 (0.39–0.81)] and cardiovascular death [HR (95%CI), 0.33 (0.13–0.86)] were lower in Q3. The nadir mortality of all-cause and cardiovascular was occurred at the serum selenium level of 136 μg/L and 130 μg/L, respectively.
Conclusion: Serum selenium concentration showed a U-shaped association with all-cause and cardiovascular mortality.
Selenium is one of the essential trace element for human beings that play a role via selenoproteins (1). It has for long been suggested as a protective factor for cancer, critical illness, diabetes, and cardiovascular disease through functions of antioxidant, anti-inflammation and reducing platelet aggregation (2, 3). Given oxidative stress is showed to be a primary contributor to hypertension development (4), sufficient selenium may benefit blood pressure control and hypertension prevention (5, 6). Deficient or excessive concentration of serum selenium has been reported closely correlated to elevated prevalence of hypertension (6–8), especially for subjects in the selenium-replete region (9). Because of the different contents of selenium in the environment, the concentration of selenium varies greatly worldwide. Compared with Europe, the United States demonstrated a higher level of selenium in soil and food intake (10). As a consequence, the average serum selenium concentration in most American adults was above 95 ng/mL (11), while Europeans ranged from 50 to 90 ng/mL (12).
Published literatures presented controversial conclusions of the efficacy of selenium concentration on mortality. Several studies reported a negative correlation between selenium status and risk of all-cause or cardiovascular death (13–15). However, the Third National Health and Nutrition Examination Survey (NHANES III; 1988–1994) reported that no association was found between serum selenium and cardiovascular mortality (16). Another prospective study conducted in China showed that no association was observed between serum selenium and overall mortality (17). To our knowledge, there was no investigation about the relationship of selenium status with mortality in hypertensive population. Thus, this study sought to evaluate the association between serum selenium concentration with all-cause and cardiovascular mortality in patients with hypertension.
Materials and Methods
Data for the current study were abstracted from the NHANES database, which aimed at evaluating the health and nutritional status in the US population. It was conducted by the National Center for Health Statistics (NCHS) within the United States Centers for Disease Control and Prevention (CDC). The study survey was administrated through face-to-face interviews and physical examination in the mobile examination center (MEC), and the resulting data release by a 2-year cycle. Further information about NHANES was available on the website (https://www.cdc.gov/nchs/nhanes/index.htm).
In total, 7,564 participants were enrolled from NHANES 2003–2004. Subjects younger than 18 years of age (n = 1,944), without hypertension (n = 3,538), had missing data on selenium (n = 471), blood pressure (n = 669), height and weight (n = 10), or lost to follow-up (n = 3) were excluded. Thus, 929 participants were included in the final list (Figure 1). The survey was approved by the Institutional Review Board of the CDC (ethical approval code: Protocol #98-12). Signed informed consent was obtained from each subject.
Assessment of Serum Selenium
Measurements of serum selenium was obtained at baseline. Blood drawn must be performing in the fasting state. After centrifugation, serum specimens should be stored at 4°C for transport and frozen at −20°C or at −70°C until time for analysis. Inductively Coupled Plasma-Dynamic Reaction Cell-Mass Spectrometry (ICP-DRC-MS) method was used to detect serum selenium concentration at the Trace Elements Laboratory, State of New York Department of Health, Wadsworth Center. More detail about the laboratory analyses procedures were described elsewhere (18).
Covariate Data Collection
Information on sex, age, race, alcohol consumption, education level, smoking behavior, marital status, history of cardiovascular disease and cancer, medication history (including antihypertensive drugs, hypoglycemic agents, antiplatelet drugs and lipid-lowering drugs) were obtained through face-to-face questionnaire. Hypertension was defined as the examined systolic blood pressures (SBP) ≥ 130 mmHg or/and diastolic blood pressure (DBP) ≥ 80 mmHg (19), confirmed to be taking antihypertensive medications, or self-reported history of hypertension. Diabetes was defined as fasting blood glucose (FBG) ≥ 126 mg/dL, self-report, hemoglobin A1c(HbA1C) ≥6.5% (20), or using hypoglycaemic drugs. SBP, DBP, height and weight were measured in accordance with laboratory procedures in MEC. Body mass index (BMI) was calculated as weight (kilograms) divided by height (meters) squared (kg/m2) (21). Total cholesterol (TC, mg/dL), high-density lipoprotein cholesterol (HDL-C, mg/dL), and C-reactive protein (CRP, mg/dL) were assessed under standardized procedure and protocol. The estimated glomerular filtration rate (eGFR, mL/min/1.73 m2) was calculated using the Chronic Kidney Disease Epidemiology equation (22).
The primary outcome of the current study was all-cause and cardiovascular mortality. Mortality data were obtained from the NHANES Public-use Linked Mortality Files through December 31, 2015, provided through the CDC. All-cause mortality was defined as death resulting from any cause. Cardiovascular mortality was considered when I00-I09, I11, I13, I20-I51, or I60-I69 were recorded as the underlying cause of death, based on the International Classification of Diseases (ICD-10) codes.
Participants were categorized by serum selenium quartiles (Q1: ≤124 μg/L, Q2: 125–135 μg/L, Q3: 136–147 μg/L, Q4: ≥148 μg/L). Descriptive data are expressed as mean ± standard deviation (SD) for continuous variables or as proportions for categorical variables. The one-way ANOVA and chi-square were used to assess differences among selenium quartiles or mortality and survival group. Cox proportional hazard regression models were performed to calculate hazard ratios (HRs) and 95% confidence intervals (CIs) for all-cause and cardiovascular mortality. Crude analysis adjust for none; adjusted model adjust for: age, sex, and race; fully adjusted model adjust for: age, sex, race, education level, marital status, smoking, BMI, SBP, TC, HDL-C, CRP, alcohol consumption, eGFR, comorbidities (cardiovascular disease, diabetes, and cancer), and medication use (lipid-lowering drugs, hypoglycemic agents, antiplatelet drugs, and antihypertensive drugs). Restricted cubic spline analysis was performed to examine the non-linearity of serum selenium and mortality. Three knots (10th, 50th, 90th percentiles of serum selenium distribution, corresponding to 115,135, and 159 μg/L, respectively) were used for restricted cubic spline modeling, with the lowest level of risk as the reference value. If a non-linear correlation was detected, a recursive algorithm was conducted to calculate the inflection point and a two-piecewise Cox proportional hazards model on both sides of the inflection point was then performed. Threshold level was defined by choosing the inflection point with maximum model likelihood, along with a log-likelihood ratio test to examine the statistical difference with one-line Cox proportional hazards model (23). Survival probability of serum selenium quartiles was presented by Kaplan–Meier curves and the log-rank test. All analyses were performed with R version 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria).
Baseline Demographic and Clinical Parameters
The current study enrolled 929 participants with mean age of 63.10 ± 12.59 years, 488 (52.53%) of them were males. The mean level of selenium was 136.78 ± 20.32 μg/L. Participants with higher selenium levels were more prone to be male, diabetics, having hypoglycemic agents, elevated TC, and reduced CRP. During the mean follow-up time of 121.05 ± 40.85 months, 307 (33.05%) deaths were recorded including 56 (6.03%) cardiovascular deaths (Table 1). Participants in survival group were on average younger in age, higher proportion being female, lower prevalence of smoking, diabetes, cancer, and cardiovascular disease, and had higher TC, DBP, eGFR, BMI, and education level, lower CRP level than participants in other two groups (cardiovascular death group and non-cardiovascular death group) (Table 2).
Table 2. Baseline demographic and clinical parameters among participants between mortality and survival group.
Association of Serum Selenium With All-Cause and Cardiovascular Mortality
The results of the Cox regression analysis were listed in Table 3. Taking the lowest quartiles (Q1) as reference, the full-adjusted HRs for all-cause mortality among groups (Q2, Q3, and Q4) were 0.74 (95%CI, 0.54–1.03), 0.57 (95%CI, 0.39–0.81), and 0.74 (95%CI, 0.53–1.04), respectively. For cardiovascular mortality, these numbers were 0.38 (95%CI, 0.16–0.90), 0.33 (95%CI, 0.13–0.86), and 0.94 (95%CI, 0.46–1.90), respectively. Thus, risk for all-cause or cardiovascular death by categorized selenium level suggesting a U-shaped association, which was lowest in Q3, and the beneficial effects could be attenuated in Q4.
The Analyses of U-Shaped Relationship
As depicted in Figure 2, results of restricted cubic spline regression found a U-shaped relationship between serum selenium with all-cause and cardiovascular mortality (Non-linear p < 0.001 and Non-linear p = 0.005, respectively). Furthermore, the two-piecewise linear regression revealed that the risk of all-cause mortality decreased up to a minimum at the selenium level of 136 μg/L, and then increased with higher selenium level [HR (95%CI): 0.75 (0.66–0.85), 1.08 (1.01–1.16) per 10 μg/L selenium increment in less or more than 136 μg/L, respectively]. Similarly, risk of cardiovascular death fell with rising serum selenium up to 130 μg/L, then ascended with greater level of selenium [HR (95%CI): 0.60 (0.42–0.86), 1.15 (1.02–1.31) per 10 μg/L selenium increased in less or more than 130 μg/L, respectively] (Table 4). Therefore, a U-shaped association between serum selenium and all-cause or cardiovascular mortality was noted, and the nadir mortality was occurred at the serum selenium level of 136 and 130 μg/L, respectively.
Figure 2. Association of selenium levels with the all-cause (A) and cardiovascular mortality (B) performed by restricted cubic spline analysis.
Table 4. The results of two-piecewise linear regression model between selenium levels and all-cause or cardiovascular mortality.
Kaplan-Meier survival curves of all patients stratified by selenium were demonstrated in Figure 3. It showed that participants in Q3 had the lowest risk of all-cause or cardiovascular mortality (Log-rank P = 0.023 and Log-rank P = 0.028, respectively).
This study found that the higher the selenium concentration, it more likely to accompanied by higher TC, and diabetes. Meanwhile, a U-shaped relationship between serum selenium with all-cause and cardiovascular mortality was observed in patients with hypertension, and the optimal cut-off value was 136 and 130 μg/L, respectively.
It has been showed that the relationship between selenium and death has been draw different conclusions in different populations. The AtheroGene study (24) followed up 879 participants with acute coronary syndrome during 6.1 years of median follow-up in Koblenz, Germany. In their research, the HR (95%CI) was 0.38 (0.16–0.91) for cardiovascular mortality comparing the highest (>84.4 μg/L) with the lowest (<64.0 μg/L) tertile of serum selenium, demonstrated a negative trend across the tertiles. Another prospective study of 347 community-dwelling older conducted in Italy found that after 10 years of follow-up time, compared with low serum selenium (≤105.3 μg/L), participants with high serum selenium (>105.3 μg/L) had a lower risk of all-cause death [HR(95%), 0.71(0.54–0.92)] (13). Interestingly, the two studies mentioned above were conducted in Europe, where the selenium intake and average serum selenium levels were lower than the United States (11, 12). Thus, their results for the lower level of selenium related to a higher risk of all-cause or cardiovascular death could compatible with our findings. However, a cohort study followed up with 1,103 adults during 15 years in China showed that no association was observed between serum selenium and total death (17), which was inconsistent with our finding. The reason for discrepancy may due to different adjustment factors. The previous study only adjusted for sex, age, smoking, drinking, and serum cholesterol, whereas our study adjusted for more potential confounding variables, such as anthropometric and demographic features, CRP, eGFR, comorbidities, and medication use.
The results of our study showed that too low and too high level of serum selenium may be linked to elevated all-cause and cardiovascular mortality were partially consistent with several previous research. The NHANES III (1988–1994) study (16) found that a U-shaped relationship was confirmed between serum selenium and all-cause mortality, instead, no association was observed between serum selenium and cardiovascular mortality. This discrepancy could be explained by the different characteristics of study population and detection methods. In the present research, the study population comprised people with hypertension were found to be older than the previous study involved a general population. Moreover, the ICP-DRC-MS method was used to assay selenium in our research, which reaching a high degree of precision demonstrated by a lower inter-assay coefficient of variation than the traditional method in the previous research. Li et al. (25) reported that the non-linear association was found between serum selenium and risks of all-cause death, however, the relationship between serum selenium with cardiovascular mortality was markedly in females only. The disparate results could be explained by the application of different versions and the range of ICD code, which may impact the recording of cardiovascular mortality. Death from cardiovascular disease was determined by ICD-10 (version presently applied, I00-I09, I11, I13, I20-I51, or I60-I69) in the current study, whereas previous studies used a broader set of ICD codes to define cardiovascular mortality, which may overestimate the prevalence of cardiovascular death.
Several potential mechanisms underlying our findings explained below. During the hypertensive state, bioavailability of antioxidants decreases and excessive reactive oxygen species (ROS) production eventually led to oxidative stress, cellular and tissue damage (26). Selenium is known to defense against oxidative stress through selenoproteins [including glutathione peroxidases (GPx) and thioredoxin reductases (TrxR)] (27). Low level of selenium could be limiting the synthesis of selenoproteins, leading to blunted effect of antioxidant and anti-inflammatory (11) and resulting in elevated risk of death (28). Nevertheless, with increasing selenium concentration, the GPx activity increases directly until reaching a plateau (29), and depletion of selenoproteins may occur when selenium oversupply resulting in health problem (30). In animal model, wistar rats with selenium supplement for 85 days showed elevation of blood pressure, indicating that longer-term selenium supplement may affect cardiovascular health (31). Furthermore, excessive exposure of selenium may also relate to harmful effects (32), including preventing proper protein folding (33), inducing the unfolded protein response (UPR) (34), causing production of superoxide and angiogenesis damage (30, 35).
The strengths of our study are the large sample size from a nationally representative population with strict adherence to protocol, and the use of a national register for identification of deaths. However, some limitations still exist. First, cardiovascular mortality is a complex phenomenon, and although we have adjusted for most relevant confounders, we cannot rule out residual or unknown confounding factors. Second, the exclusion of these patients who had missing data on selenium, blood pressure, height and weight may lead to selection bias. Third, serum selenium was only measured at baseline, and selenium exposure may have changed over time, which might have resulted in misclassification. Forth, some factors may affect selenium metabolism including supplement intake, living environment and dietary behaviors were not collected in our study.
In conclusion, serum selenium concentrations had a U-shaped relationship with all-cause and cardiovascular mortality in patients with hypertension, and the cut-off values were showed to be 136 and 130 μg/L, respectively. This suggested that too low or too high concentration of serum selenium might be concomitant with poor outcomes. The exact reaction mechanism of selenium in human in biological systems require further study.
Data Availability Statement
Publicly available datasets were analyzed in this study. This data can be found here: https://www.cdc.gov/nchs/nhanes/index.htm.
The studies involving human participants were reviewed and approved by Institutional Review Board of the Centers for Disease Control and Prevention. The patients/participants provided their written informed consent to participate in this study.
Y-qF, Y-qH, KL, and Q-hT: conceptualization and methodology. Y-qH, Q-hT, LL, and X-cL: formal analysis. Y-qH, J-yC, and Y-qF: supervision and validation. Q-hT and KL: writing and revision. All authors contributed to the article and approved the submitted version.
This research was supported by the Science and Technology Plan Program of Guangzhou (No. 201803040012), the Key Area R&D Program of Guangdong Province (No. 2019B020227005), Guangdong Provincial People's Hospital Clinical Research Fund (Y012018085), the Fundamental and Applied Basic Research Foundation Project of Guangdong Province (2020A1515010738), and the Climbing Plan of Guangdong Provincial People's Hospital (DFJH2020022).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
We sincerely thank all the staff for their important contributions and all the enrolled participants.
6. Nawrot TS, Staessen JA, Roels HA, Den Hond E, Thijs L, Fagard RH, et al. Blood pressure and blood selenium: a cross-sectional and longitudinal population study. Eur Heart J. (2007) 28:628–33. doi: 10.1093/eurheartj/ehl479
8. Laclaustra M, Navas-Acien A, Stranges S, Ordovas JM, Guallar E. Serum selenium concentrations and hypertension in the US Population. Circ Cardiovasc Qual Outcomes. (2009) 2:369–76. doi: 10.1161/CIRCOUTCOMES.108.831552
9. Wu G, Li Z, Ju W, Yang X, Fu X, Gao X. Cross-sectional Study: relationship between serum selenium and hypertension in the Shandong Province of China. Biol Trace Elem Res. (2018) 185:295–301. doi: 10.1007/s12011-018-1272-7
11. Institute of Medicine Panel on Dietary A Related C. Dietary Reference Intakes for Vitamin C, Vitamin E, Selenium, and Carotenoids. Washington (DC): National Academies Press (US). Copyright 2000 by the National Academy of Sciences. (2000). Available online at: http://www.nap.edu/catalog/9810.html
13. Giovannini S, Onder G, Lattanzio F, Bustacchini S, di Stefano G, Moresi R, et al. Selenium concentrations and mortality among community-dwelling older adults: results from ilSIRENTE study. J Nutr Health Aging. (2018) 22:608–12. doi: 10.1007/s12603-018-1021-9
14. Alehagen U, Johansson P, Björnstedt M, Rosén A, Post C, Aaseth J. Relatively high mortality risk in elderly Swedish subjects with low selenium status. Eur J Clin Nutr. (2016) 70:91–6. doi: 10.1038/ejcn.2015.92
15. Salonen J, Alfthan G, Huttunen J, Pikkarainen J, Puska P. Association between cardiovascular death and myocardial infarction and serum selenium in a matched-pair longitudinal study. Lancet. (1982) 320:175–9. doi: 10.1016/S0140-6736(82)91028-5
16. Bleys J, Navas-Acien A, Guallar E. Serum selenium levels and all-cause, cancer, and cardiovascular mortality among US adults. Arch Intern Med. (2008) 168:404–10. doi: 10.1001/archinternmed.2007.74
17. Wei WQ, Abnet CC, Qiao YL, Dawsey SM, Dong ZW, Sun XD, et al. Prospective study of serum selenium concentrations and esophageal and gastric cardia cancer, heart disease, stroke, and total death. Am J Clin Nutr. (2004) 79:80–5. doi: 10.1093/ajcn/79.1.80
18. NHANES. Laboratory Procedure Manual of Serum Selenium. (2004). Available online at: https://wwwn.cdc.gov/nchs/data/nhanes/2003-2004/labmethods/l39_c_met_selenium.pdf [accessed February 23, 2021).
19. Whelton PK, Carey RM, Aronow WS, Casey DE, Jr., Collins KJ, Dennison Himmelfarb C, et al. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. (2018) 138:e426–83. doi: 10.1161/HYP.0000000000000066
22. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF III, Feldman HI, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. (2009) 150:604–12. doi: 10.7326/0003-4819-150-9-200905050-00006
24. Lubos E, Sinning CR, Schnabel RB, Wild PS, Zeller T, Rupprecht HJ, et al. Serum selenium and prognosis in cardiovascular disease: results from the AtheroGene study. Atherosclerosis. (2010) 209:271–7. doi: 10.1016/j.atherosclerosis.2009.09.008
25. Li J, Lo K, Shen G, Feng YQ, Huang YQ. Gender difference in the association of serum selenium with all-cause and cardiovascular mortality. Postgrad Med. (2020) 132:148–55. doi: 10.1080/00325481.2019.1701864
28. Dragan S, Buleu F, Christodorescu R, Cobzariu F, Iurciuc S, Velimirovici D, et al. Benefits of multiple micronutrient supplementation in heart failure: a comprehensive review. Crit Rev Food Sci Nutr. (2019) 59:965–81. doi: 10.1080/10408398.2018.1540398
29. Hurst R, Armah CN, Dainty JR, Hart DJ, Teucher B, Goldson AJ, et al. Establishing optimal selenium status: results of a randomized, double-blind, placebo-controlled trial. Am J Clin Nutr. (2010) 91:923–31. doi: 10.3945/ajcn.2009.28169
30. Rayman MP, Winther KH, Pastor-Barriuso R, Cold F, Thvilum M, Stranges S, et al. Effect of long-term selenium supplementation on mortality: results from a multiple-dose, randomised controlled trial. Free Rad Biol Med. (2018) 127:46–54. doi: 10.1016/j.freeradbiomed.2018.02.015
31. Grotto D, Carneiro MFH, de Castro MM, Garcia SC, Barbosa Junior F. Long-term excessive selenium supplementation induces hypertension in rats. Biol Trace Elem Res. (2018) 182:70–7. doi: 10.1007/s12011-017-1076-1
32. Lippman SM, Klein EA, Goodman PJ, Lucia MS, Thompson IM, Ford LG, et al. Effect of selenium and vitamin e on risk of prostate cancer and other cancers: the selenium and vitamin e cancer prevention trial (SELECT). JAMA. (2009) 301:39–51. doi: 10.1016/S0084-3873(09)79550-1
34. Wallenberg M, Misra S, Wasik AM, Marzano C, Björnstedt M, Gandin V, et al. Selenium induces a multi-targeted cell death process in addition to ROS formation. J Cell Mol Med. (2014) 18:671–84. doi: 10.1111/jcmm.12214
Keywords: selenium, hypertension, all-cause mortality, cardiovascular mortality, risk factors
Citation: Tan QH, Huang YQ, Liu XC, Liu L, Lo K, Chen JY and Feng YQ (2021) A U-Shaped Relationship Between Selenium Concentrations and All-Cause or Cardiovascular Mortality in Patients With Hypertension. Front. Cardiovasc. Med. 8:671618. doi: 10.3389/fcvm.2021.671618
Received: 24 February 2021; Accepted: 06 July 2021;
Published: 30 July 2021.
Edited by:Michel Burnier, Centre Hospitalier Universitaire Vaudois (CHUV), Switzerland
Reviewed by:Tommaso Filippini, University of Modena and Reggio Emilia, Italy
Ronny Lesmana, Universitas Padjadjaran, Indonesia
Copyright © 2021 Tan, Huang, Liu, Liu, Lo, Chen and Feng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ying-qing Feng, email@example.com
†These authors have contributed equally to this work