Melehat Gezer
Yıldız Yıldırım
Mustafa İlhan- 1Faculty of Education, Social Studies Education, Dicle University, Diyarbakir, Türkiye
- 2Faculty of Education, Measurement and Evaluation in Education, Aydin Adnan Menderes University, Aydin, Türkiye
- 3Faculty of Education, Department of Mathematics and Science Education, Dicle University, Diyarbakir, Türkiye
Climate change worry is an increasingly critical issue in eco-psychology literature. A commonly used instrument for measuring this construct is the Climate Change Worry Scale (CCWS), developed by Stewart. This Likert-type scale assesses individuals' climate change worry through 10 items clustered under a single factor. It has been adapted for multiple cultures and utilized in numerous studies conducted across various countries. Nevertheless, no study has synthesized the reliability values obtained from individual studies for the scale. The purpose of the current meta-analysis was to perform a reliability generalization for the CCWS. To this end, an exhaustive literature search was conducted from July 14 to November 17, 2024, in the EBSCO, ERIC, Taylor & Francis, PubMed, and Web of Science databases, as well as Google Scholar, using the keyword “Climate Change Worry Scale.” After scrutinizing the identified studies for duplicates and applying inclusion and exclusion criteria, the research focused on the 40 Cronbach's alpha coefficients acquired from 37 papers. The results of the analysis, which involved running the random effects model and the Bonnet transformation, indicated that the pooled Cronbach's alpha was 0.932 (95% CI = 0.919–0.942). The results of the moderator analysis revealed that the sample descriptors and study characteristics included in the meta-analysis did not significantly affect the reliability estimates. Accordingly, the CCWS was found to be an instrument that produces highly reliable measurements regardless of factors such as region, language, participants' age, and the total number of items answered during administration. Finally, the reliability induction rate was determined to be 29.41%. However, the high heterogeneity observed among the reliability estimates of the primary studies exposed the limitations of generalizing the reliability of CCWS scores across different populations and research conditions. This situation also emphasized the importance of providing detailed information about the scale's sample demographics and administration conditions when reporting reliability.
Introduction
Climate change is considered the major and most widespread challenge that the natural environment and humanity have ever faced (United Nations, 2022). Its effects on both ecosystems and humans make the issue even more important. This concept is defined as statistically significant changes in the average condition of the climate lasting for decades or a more extended period (Türkeş, 2008). The climate is constantly changing due to many different natural factors. However, a crucial new factor that has been affecting the Earth's climate more and more in the last two centuries is human activity (Mikhaylov et al., 2020). Human-induced climate change, known as anthropogenic influences, causes many weather and climate extremes in almost every geographical area of the world (Trenberth, 2018). The human impact on extreme changes observed in the atmosphere, ocean, and land, such as warming, heatwaves, glacier loss, heavy rainfall, droughts, and tropical cyclones, has been demonstrated with evidence (IPCC., 2023).
The potentially devastating influences of climate change and its accompanying extreme weather events on people, the environment, and the economy have been widely documented (Sümen et al., 2025). For example, alterations in climate have provoked migrations in many locations from Africa to Asia and from North America to Central and South America (IPCC., 2023), to the extent that the term “climate refugees” has entered the literature due to these migrations (Lister, 2014). In addition to the notable economic losses identified in climate-sensitive sectors such as agriculture, forestry, fisheries, shellfish aquaculture, energy, and tourism in our times (IPCC., 2023), climate-related natural disasters have incurred substantial costs for countries (European Environment Agency, 2024). Moreover, numerous species are at risk of extinction due to climate change (McElwee, 2021), posing a potential threat to the Earth's biodiversity (Kappelle et al., 1999).
The impacts of climate change on physical and mental health
Increases in extreme temperatures trigger various physical illnesses in humans, such as heat stroke, hypothermia, and diarrhea, as well as cardiovascular, respiratory, infectious, and pathogenic diseases (Avan and Vural, 2025; Comrie, 2007; D'Amato et al., 2014; Franchini and Mannucci, 2015; Kim et al., 2014), and can even result in fatalities (Arnold, 2022). Additionally, climate change may lead to a range of adverse effects related to mental health, including psychological distress, anxiety disorders, worry, acute stress, depression, aggression, insomnia, suicidal ideation, and a diminished sense of self and identity (Charlson et al., 2021; DeJarnett et al., 2025; Kumar et al., 2024; Palinkas and Wong, 2020). In a large-scale cross-cultural study by Hickman et al. (2021) involving over 10,000 participants aged 16–25 years, it was found that 84% of the respondents are at least moderately concerned about climate change. Furthermore, more than 45% of the participants reported that their feelings regarding climate change negatively impacted their daily functioning. It is anticipated that the negative effects of climate change on mental health will become increasingly pronounced in the future as climate-related stressors affect a larger number of people (Taylor, 2020). The impact of environmental degradation on emotional responses has also been highlighted in the eco-psychology literature, introducing new climate-specific mental health phenomena such as solastalgia [a state of feeling lonely, insecure, and powerless caused by significant changes in one's immediate environment due to the acute effects of climate change, including sea level rise, flooding, and forest fires] (Albrecht et al., 2007), ecological grief (Cunsolo and Ellis, 2018), eco-guilt (Cianconi et al., 2020), eco-anxiety, climate change anxiety (Albrecht, 2011; Clayton, 2020), and climate change worry (Stewart, 2021).
Climate change worry
The number of studies on the psychological effects of climate change is rapidly increasing. Various concepts have been proposed in these studies to describe individual reactions to climate change. Anxiety and worry are two concepts frequently encountered in the literature on this subject. Clayton (2020) explained the effects of climate change on mental health by focusing on the concept of anxiety (a psychological state that can lead to restlessness and sleep disturbances), while Stewart (2021) focused on worry (verbal-linguistic thoughts about potential alterations in the climate system and their possible consequences). Although these two concepts are related, they are not identical, and it is incorrect to use them interchangeably (Innocenti et al., 2022). Anxiety is a broader construct expressed through affective (such as nervousness and fear), cognitive (difficulty concentrating), physiological (nausea and sweating), and behavioral (sleep disturbances, dysfunction) indicators (van Valkengoed et al., 2023). In contrast, worry pertains specifically to cognitive content, particularly excessive concern regarding future events (Ojala et al., 2021; Zebb and Beck, 1998).
Climate change worry is rooted in the perceived threat of climate change; a greater perceived threat can drive more worry (Reser and Swim, 2011). Worry about climate change forms the initial stage of all emotions related to this issue and has the potential to regulate emotions (Kurt and Akdur, 2024), serving as a motivating factor for individuals to take action on the matter (Bouman et al., 2020) and to engage in pro-environmental behaviors such as recycling and reducing their ecological footprint (Donati et al., 2025; Kabasakal Çetin et al., 2025). If this worry becomes maladaptive and chronic in an individual's life, particularly with increases in autonomic arousal (like elevated heart rate, respiration, sweating, and tension), it can lead to climate anxiety (Innocenti et al., 2022; Stewart et al., 2024). Worry also relates to perceived risk and fear, yet it is less cognitive than perceived risk and less overwhelming than fear. Climate change worry reflects an individual's active emotional engagement with the issue and personal concern regarding its consequences (Bouman et al., 2020). As long as climate change worry remains adaptive and does not become pathological, it enables individuals to pay attention to ongoing climate events, making it a healthier response than denial or disavowal (Dodds, 2021). In other words, while severe worry related to the topic can be extremely debilitating (Whitmarsh et al., 2022), milder worry is a rational and potentially functional reaction to the awareness of the serious threat posed to the planet (Martin et al., 2024). All these points highlight the importance of investigating climate change worry and assessing its level.
Climate Change Worry Scale
Despite the increasing attention to the psychological effects of climate change, the number of valid and reliable instruments addressing these phenomena is relatively scarce. Studies generally use items and inventories designed for other applications (Cruz and High, 2022). The Climate Change Anxiety Scale (CCAS) (Clayton and Karazsia, 2020), the Climate Change Worry Scale (CCWS) (Stewart, 2021), and the Hogg Eco-Anxiety Scale (HEAS) (Hogg et al., 2021) are among the scales most frequently used by researchers. Of the mentioned scales, CCAS and HEAS are widely utilized to measure anxiety, while CCWS is employed to assess climate change worry. This study focused on CCWS rather than CCAS and HEAS, considering that while CCAS and HEAS are valuable tools, they prioritize emotional and anxious responses to climate change and may not capture cognitive aspects as precisely as CCWS. One reason for conducting the current RG meta-analysis on CCWS instead of CCAS was its generally higher predictive value in healthy, general samples (Plohl et al., 2023). Other reasons for choosing CCWS over CCAS and HEAS include its ease of application due to its brevity, likely ensuring more accurate and complete responses by reducing participant burden and possible respondent fatigue, streamlining data processing and analysis, thereby being a practical option even for large-scale studies.
The CCWS, developed by Stewart (2021) in English, has been adapted into multiple languages and cultures, including Turkish, Spanish, Polish, French, and Hebrew. Despite being recently developed, a total of 96 citations were found in Web of Science, and 218 results were identified in Google Scholar when the scale was searched in the literature. The original form of the CCWS consists of 10 items with a five-point Likert-type rating scale. It has a unidimensional structure that explains 73.60% of the total variance, with factor loadings of the items in this single-factor structure ranging from 0.73 to 0.90. In the original version of the scale, Cronbach's alpha and McDonald's omega coefficients were calculated to assess reliability, both showing internal consistency coefficients of 0.95 (Stewart, 2021). In addition to providing valid and reliable measurements, the CCWS's concise nature, consisting of only 10 items, makes it a valuable instrument for researchers studying climate change worry.
The purpose and importance of the research
Numerous studies have utilized the CCWS across various geographical locations and cultures, involving different age groups and diverse sample sizes with varying gender distributions. The existing studies report differing reliability coefficients for the measurements obtained through the CCWS. For example, while there is a study that calculated a reliability value slightly above 0.70 for the data collected with the CCWS (Duran and Kaynak, 2024), there are also studies that reach reliability coefficients above 0.95 (e.g., Özbay and Alci, 2021; Plohl et al., 2023). Additionally, a review of conducted studies revealed instances of reliability induction (Vacha-Haase et al., 2000); that is, some researchers cited reliability values from prior studies related to the development or adaptation of the CCWS without providing the reliability of their own data. However, reliability is not a fixed attribute of the measurement tool; it varies according to sample characteristics, administration conditions, and other factors (Bandalos, 2018; Crocker and Algina, 1986). Therefore, researchers must not only report the reliability of the studies from which their scales were developed or adapted but also disclose the reliability of their own data. Given these considerations, it is essential to conduct a reliability generalization (RG) meta-analysis for the CCWS to obtain a pooled reliability coefficient based on the coefficients calculated across various cultures and samples and to investigate possible variables that may account for the heterogeneity in the reported reliability coefficients. In this context, the current study posed the following questions:
1) What is the overall Cronbach's alpha coefficient for the CCWS?
2) How do the sample descriptors and study characteristics (region, language scale, gender distribution, age, and the total number of items respondents answered during the administration of the CCWS) affect the pooled reliability coefficient?
3) What is the reliability induction rate in studies using the CCWS?
Based on the research questions outlined above, the current meta-analysis of individual empirical studies utilizing the CCWS will provide insights into the percentage of studies in which reliability was misinterpreted due to reliability induction. Furthermore, as Sen and Yörük (2023) noted, the presence of variables that may cause systematic errors in scale scores will be revealed through moderator analysis. Specifically, this study will enhance our understanding of whether errors in scale scores are random (related to unpredictable factors such as respondents' low motivation, inattention, etc.) or systematic, stemming from variables such as gender, culture, scale language, and the total number of items answered by respondents. In this regard, this investigation has the potential to provide more comprehensive information regarding the reliability of the CCWS compared to individual studies, allowing researchers to use the scale more effectively and interpret the reliability of its scores accurately. To date, numerous RG meta-analyses have been conducted on various measurement tools across different disciplines. A search for the keyword “reliability generalization meta-analysis” in Google Scholar on March 3, 2025, yielded 2,430 results. To the best of our knowledge, no effort has been made to consolidate the individual reliability evidence of the CCWS. Thus, the current meta-analysis represents an original study that could enhance the evidence regarding the psychometric properties of the CCWS. Since it was not possible to access factor loadings or inter-item correlation matrices from individual studies that employed the scale, except for the papers where the CCWS was developed or adapted, this meta-analysis focuses solely on RG, a meta-analytic factor analysis regarding the validity of the CCWS was not conducted. Given that reliability is a prerequisite for validity (Morrison et al., 2011), it is anticipated that this research will establish a foundation for future meta-analytic factor analysis studies on the CCWS. In this respect, the present study remains highly valuable even without a meta-analysis of validity.
Method
The current RG meta-analysis was conducted by adhering to the REGEMA checklist developed by Sánchez-Meca et al. (2021), which is based on previous guidelines proposed in the field of meta-analysis.
Data sources and literature search
A comprehensive literature search was conducted in the EBSCO, ERIC, Taylor & Francis, PubMed, and Web of Science databases using the keyword “Climate Change Worry Scale.” The search was performed from July 14, 2024, to November 17, 2024, without any restrictions on publication types. Since the CCWS was published in 2021, the relevant studies covered a period from 2021 to November 2024, inclusive. To ensure that no eligible studies were overlooked, the reference lists of retrieved studies were manually reviewed (backward search), and Google Scholar, identified as a gray literature repository by Jahrami et al. (2024), was also searched.
Selection of the studies
The identified studies were reviewed based on the following inclusion and exclusion criteria. Empirical studies were included in the RG meta-analysis if they (1) administered the full 10-item version of the CCWS to participants, (2) reported the Cronbach's alpha coefficient of their data along with the sample size, and (3) were published in Turkish or English. Conversely, studies were excluded if they (1) were theoretical or qualitative studies, meta-analyses, or systematic reviews, (2) utilized a scale other than the CCWS, (3) used only partial CCWS items in their instrument, (4) did not report the sample size or Cronbach's alpha coefficient of the available data and did not respond to the authors' email request regarding this issue, and (5) were published in a language other than Turkish or English. In calculating the reliability induction rate, studies that met the first and third inclusion criteria but did not present either the Cronbach's alpha coefficient or any other reliability estimation for their data were considered papers that induced reliability (even if the Cronbach's alpha coefficient was provided in response to the authors' email request).
Figure 1 illustrates the flowchart summarizing the selection process, including both elimination and inclusion, of the studies incorporated in the RG meta-analysis. As shown in Figure 1, 289 references were initially identified, of which 99 were duplicates, and 63 were excluded for various reasons. When the full text of the remaining 127 references was reviewed, it was found that 69 employed a scale other than the CCWS, and 7 included only some of the CCWS items in their data collection tools. By excluding these 76 studies, the number of empirical references implementing the full 10-item version of the CCWS was reduced to 51. While one of the 51 studies indicated that it did not calculate a reliability coefficient due to its small sample size, the other 14 studies did not report any reliability estimates without explanation. For studies that applied the CCWS but did not provide Cronbach's alpha coefficient regarding scale data, an email was sent to the corresponding authors, leading to the acquisition of reliability estimations for three studies. Thus, the number of references implementing the CCWS and reporting reliability estimates based on the indexed sample increased from 36 to 39. However, since two of the 39 references provided a reliability estimate other than Cronbach's alpha, which is the target reliability index of this investigation, the RG meta-analysis was conducted on the 40 Cronbach's alpha estimates obtained from 37 studies.
Figure 1. REGEMA flowchart for CCWS.
Data extraction
Initially, a coding schema was developed to extract relevant study characteristics and reliability coefficients. The extracted data included study metadata, scale language, sample characteristics (size, region, age, female ratio), reported reliability estimates, and the total number of items in the instrument applied to the participants. Pentapati et al. (2025) indicated that language and geographical location could influence the participants' definitions and perceptions of conceptual words in the scale items; therefore, these two variables were included among the moderator variables of this meta-analysis. The ratio of females to the total sample size was selected as a moderator because studies show that women have significantly higher climate worry scores than men (e.g., Clayton et al., 2023; Verplanken et al., 2020), suggesting that gender distribution may affect internal reliability by altering sample heterogeneity. Given the significant relationship between age and climate change anxiety, as well as perceived climate change threat (Searle and Gow, 2010; Whitmarsh, 2008), the sample's age (mean and standard deviation) was also included as a moderator variable. The reason for incorporating the variable of “total number of items” in the coding form was as follows: CCWS was administered alongside other scales in some studies, leading to variations in the number of items answered by participants from one study to another. It was thought that an increase in the number of responded items could degrade data quality due to factors such as fatigue and inattention, which could serve as a moderator variable generating heterogeneity in the reliability coefficient.
All 37 studies included in the meta-analysis were coded independently by two raters with double-blinding. One coder was an expert in social studies education, while the other specialized in measurement and evaluation. After the two researchers completed the coding, the reliability of the data extraction process was assessed. The agreement between coders was calculated using the formula proposed by Miles and Huberman (1994), resulting in 97.22%. This finding indicated a highly satisfactory consensus. Discrepancies between coders were primarily related to the variable of “total number of items” and were resolved through discussion. Table 1 summarizes the coding schema for the studies.
Table 1. Coding schema for the studies.
The coders implemented the coding directly using the category names provided in Table 1. In studies with multiple samples that reported a separate alpha coefficient for each sample, a number was appended next to the author and publication details when coding the study tag (e.g., Stewart, 2021_1; Stewart, 2021_2; Stewart, 2021_3). As shown in Table 1, the numerical values specified in the original studies were used directly when coding continuous variables. Some adjustments were made to the categorical variables during data preparation for analysis, considering the number of studies in each subcategory. In the country title, the subcategories were coded as USA = (1), Turkey = (2), and a Europe subcategory was created by grouping other European countries, which were coded as (3). Countries (e.g., Vietnam, Israel) that could not be classified into the first three subcategories were coded as Others (4). After this change, the region name replaced the country name for the relevant variable. Given the limited number of studies in languages other than English and Turkish, the moderator analysis for the variable of scale language was conducted as English, Turkish, and other languages, coded as 1, 2, and 3, respectively.
Data analysis
Vacha-Haase's (1998) meta-analytic RG was used in the data analysis. Only Cronbach's alpha was considered in the current investigation because the alpha coefficient was the most frequently used reliability index in individual studies. Prior to analysis, conversion was applied to the raw Cronbach's alpha values to normalize their negatively skewed distribution and stabilize the variance (Rodriguez and Maeda, 2006; Semma et al., 2018). Several methods (Fisher's z, Hakstian and Whalen's, and Bonett's transformation) have been introduced for transformation in RG meta-analysis. The most commonly used one in the studies has been Fisher z. However, this transformation is only suitable for correlation-based reliability coefficients (e.g., test–retest, parallel forms) (Sánchez-Meca et al., 2013). Since Cronbach's alpha is not a correlation coefficient, either Hakstian and Whalen's or Bonett's transformations should be employed in RG meta-analyses performed on the alpha coefficient (López-Ibáñez et al., 2024). Taking as reference the suggestion of Sánchez-Meca et al. (2013), Bonett (2010) 's formula was preferred for the transformation in this meta-analysis.
Along with the transformation method, another critical decision for the analysis involves selecting the appropriate model: fixed effects vs. random effects. The random effects model is generally the better choice because (a) it is more likely to align with the actual sampling distribution, (b) it does not impose a common effect size restriction, (c) it yields the same estimates as the fixed-effect model in the absence of heterogeneity, and (d) it enables the results to be generalized to a broader range of research than those included in the meta-analysis (Borenstein et al., 2010). Similarly, Field and Gillett (2010) argued that random effects are more realistic than the fixed-effect approach for real-world data in the social sciences and should be the norm in that context. In this regard, while calculating pooled alphas, the meta-analysis was conducted according to the random effects model, using the inverse variance method to weight the coefficients.
The heterogeneity among the studies was assessed using several statistics. The first statistic was the variance between studies (τ2), which was checked to see if it was different from zero. The restricted maximum likelihood method served as the estimator for the τ2 statistic. The second statistic was the Q-test, and its statistical significance served as evidence of heterogeneity. The third heterogeneity indicator analyzed was the I2 index. This index provides information about the magnitude of heterogeneity, unlike the Q-test, which produces a binary decision regarding the presence or absence of heterogeneity (Lin, 2019). Values of 75% or greater were interpreted as considerable heterogeneity when evaluating the I2 index (Deeks et al., 2008).
Due to heterogeneity, moderator analyses were conducted to identify the study descriptors and sample characteristics that may be sources of this heterogeneity. These analyses, performed for both continuous and categorical variables, utilized a mixed-effects model. The scale language and region variables, for which the number of primary studies in the subcategory was sufficient, were identified as categorical moderator variables. In contrast, the mean and standard deviation of age (in years), the female ratio in the sample, and the total number of items responded to by participants during the administration of the CCWS were included as continuous moderator variables in the meta-analysis. While meta-regression was executed for continuous moderator variables, an analog ANOVA was conducted for the categorical ones.
To audit the risk of publication bias, a funnel plot was created using Bonett's transformation values of Cronbach's alphas and standard errors. Additionally, statistical methods such as Rosenthal (1979)'s Fail-safe N, Begg and Mazumdar (1994)'s rank correlation, Egger's regression test (Egger et al., 1997), and Duval and Tweedie (2000)'s trim and fill were evaluated. The criterion for Rosenthal's Fail-safe N approach used the equation N/(5k+10), where N and k represent the number of missing studies and the number of studies included in the meta-analysis, respectively, and a value >1 indicates no publication bias (Rosenthal, 1979). Finally, both types of reliability induction—“by omission (i.e., failing to mention reliability throughout the manuscript)” and “by the report (i.e., presenting the reliability estimates of previous studies)” (Sánchez-Meca et al., 2021)—were considered while calculating the reliability induction rate. All statistical analyses were conducted with the meta package (Balduzzi et al., 2019) and the metafor package (Viechtbauer, 2010) in the R Studio environment.
Results
Within the scope of the research, the characteristics of the studies included in the meta-analysis were examined first, and the findings obtained are displayed in Table 2. It is apparent from the table that the studies incorporated in this meta-analysis were published between 2020 and 2024. At this point, the question may arise: “If the CCWS was first published in 2021, how is it that the scale was used in an article in 2020?” In the study by Orr et al. (2020), the study that CCWS published was cited as “submitted for publication.” Therefore, the publication date of the relevant article is before the CCWS' publication year. A closer inspection of the table demonstrates that a single alpha coefficient was calculated in 35 of the 37 studies, while the studies by Stewart (2021) and Hamama-Raz and Shinan-Altman (2024) reported three and two separate Cronbach's alpha estimates, respectively. Table 2 also shows that the studies were conducted in different regions; nevertheless, most of them were based in Türkiye, followed by the US. Hence, the scale was predominantly applied to the Turkish and English languages. The table shows that the female ratio to the total sample size ranged from 0.324 to 1.00. Accordingly, among the studies using CCWS, one was conducted only with women, but none included only men. The sample size of the studies selected for analysis varied from 54 to 3,412, totaling 18,067 participants. The mean and standard deviation of the participants' ages were between 20.56 and 44.87 and 1.06 and 16.78, respectively. When the number of items answered by the participants (k) in the studies was examined, it was inferred that in some studies, only the CCWS was applied; thus, the number of items answered was 10, while in other studies, several instruments were administered to the sample simultaneously, and the number of items answered reached up to 140. Since a five-point rating scale was employed in all research included in the meta-analysis and almost all studies had a mixed composition concerning participants' education levels, no descriptions related to these variables were included in the table.
Table 2. Summary of the 37 included studies with 40 Cronbach's alpha estimates.
After examining the descriptive characteristics of the studies, publication bias was assessed. The funnel plot created for this purpose is shown in Figure 2.
Figure 2. Funnel plot.
Figure 2 shows that the funnel plot has a distribution close to symmetrical. This implies the absence of publication bias. However, merely examining the funnel plot is not sufficient to settle the question of publication bias. For this reason, along with the funnel plot, results based on statistical methods were also examined. Firstly, Duval and Tweedie's (2000) trim and fill method result was scrutinized, which is an approach based on a funnel plot. Accordingly, the number of studies to be added to the right of the plot to ensure full symmetry of the funnel plot is 9, indicating publication bias. When the results of the Rosenthal Fail-safe N method were examined, it was found that the number of missing studies required to render the average effect size statistically insignificant was 743,191. Since N = 743,191 and k = 40, the value of N/(5k+10) was >1, which asserted that there was no publication bias. Lastly, Begg and Mazumdar (1994)'s rank correlation and Egger's regression test (Egger et al., 1997) were explored. The z values of Kendall's τ and Egger's regression test were −0.107 and −0.524, respectively, and both were found to be statistically insignificant. Taken together, all bias statistics suggest that there is no publication bias. After concluding that there was no publication bias, the meta-analytic RG and heterogeneity results of the CCWS were exhibited in Table 3.
Table 3. Results of overall Cronbach's alpha and heterogeneity.
According to Table 3, the statistically significant Q-test indicated heterogeneity among the reliability estimates. The I2 value (98.12%), which is >75%, indicated a high level of heterogeneity. The fact that the τ2 value (0.268) was significantly different from 0 provided further evidence of the variance among studies. Although it is a relatively subjective argument, the presence of heterogeneity can also be observed in the forest plot in Figure 3, where the lowest Cronbach's alpha coefficient was 0.74 (Duran and Kaynak, 2024), and the highest was 0.98 (Ecer et al., 2023). Overall, all evidence supported the existence of heterogeneity.
Figure 3. Forest plot.
The pooled Cronbach's alpha reliability coefficient of the CCWS was found to be 0.932 [95% Confidence Interval (CI) = 0.919–0.942]. This coefficient was statistically significant and exceeded the widely accepted threshold of 0.70 for the reliability index. It was also very close to Cronbach's alpha value of 0.95, which was calculated by Stewart in his study developing the CCWS. Due to the high heterogeneity observed in the RG meta-analysis, it became essential to investigate the sources of this variance. Therefore, the variation in the Cronbach's alpha coefficient based on the moderator variables was assessed. Moderator analysis was primarily conducted for the categorical variables of region and scale language. Table 4 presents the analog ANOVA results applied for this purpose.
Table 4. Results of the analog ANOVA using categorical moderator variables as independent variables.
It is evident from Table 4 that Cronbach's alpha coefficients for all subcategories related to the regional variable exceeded 0.90 and were statistically significant. The lowest and highest overall alpha coefficients were recorded for Europe (0.926 [95% CI 0.895–0.948]) and the USA (0.935 [95% CI 0.899–0.958]), respectively. However, the differences observed between the pooled internal consistency estimates of Türkiye, Europe, the USA, and other regions were not statistically significant. A similar trend was observed for the scale language variable, where Cronbach's alpha values in all subcategories were >0.90 and were statistically significant. The pooled alpha values for both the English (0.932 [95% CI 0.908–0.950]) and Turkish (0.932 [95% CI 0.914–0.946]) versions of the CCWS were identical, while the overall alpha of other languages was lower (0.928 [95% CI 0.895–0.950]). Following the categorical variables, a continuous moderator analysis was performed using the mean age, standard deviation of age, female ratio, and the total number of items applied to the sample. Table 5 presents the meta-regression results with continuous moderator variables as predictors.
Table 5. Results of meta-regression using continuous moderator variables as predictors.
As shown in Table 5, the slopes of the continuous variables were not statistically significant. Based on this result, we can conclude that the changes in the analyzed continuous variables did not produce a statistically significant difference in Cronbach's alphas. Furthermore, the variance explained by these variables was found to be 0.000%. Conversely, the fact that all QE residual variance values were statistically significant indicates that the variability in Cronbach's alpha values can be attributed to other variables.
Eventually, the reliability induction rate was calculated. From Figure 1, it can be observed that one primary study could not be included in the meta-analysis due to omission. Conversely, the number of studies that induced reliability from prior research was 14. In this context, since CCWS was applied in 51 studies, the induction rate was 29.41%. When the reliability induction through omission and reporting was calculated separately, it was determined that these values were 1.96% and 27.45%, respectively.
Discussion and conclusion
Among the various psychological impacts of climate change, the ones that empirical research has primarily focused on are eco-anxiety and worry (Regnoli et al., 2024). Despite the growing interest in anxiety and worry related to climate change, relatively few validated scales address the structures involved (Plohl et al., 2023). A promising new instrument for assessing climate change worry is the CCWS (Larionow et al., 2024; Shepherd et al., 2024). This tool, which evaluates disturbing thoughts about climate change through 10 items, was initially developed by Stewart (2021). After the development of its original version in English, the CCWS was adapted into Turkish (Gezer and Ilhan, 2021; Özbay and Alci, 2021), Italian (Donati et al., 2025; Innocenti et al., 2022), Polish (Larionow et al., 2024), Slovenian (Plohl et al., 2023), and French (Shepherd et al., 2024) and has been used in many studies. The published research has provided substantial information regarding the reliability of the measures obtained from the CCWS, but it remains at the individual level. Therefore, a study designed to provide more comprehensive evidence about the scale's reliability was deemed important, leading to the present RG meta-analysis.
This study combined the Cronbach's alpha coefficients calculated in individual studies utilizing the CCWS in a statistically rigorous manner and tested various moderator variables that could be effective in estimating differing internal consistency coefficients for scale scores from one administration to another. When examining the descriptive properties of the analyzed studies, it was found that more than half of them were conducted with a Turkish sample. The CCWS was adapted into Turkish in the same year its original English form was published. The fact that the CCWS has been deployed in Turkey for about 4 years may have contributed to the increased number of studies conducted with the Turkish sample compared to the cultures to which it was adapted later. An additional factor influencing the extensive use of CCWS in the Turkish sample may be that Türkiye is located in a geographical area highly susceptible to climate change (Sen, 2013), which could prompt research focused on worry about climate change.
It was found that Cronbach's alpha coefficients for the measurements obtained from the CCWS in individual studies ranged from 0.74 to 0.98; however, in most cases, they were 0.90 or greater. This finding indicates that the CCWS generally produced measures with analogous internal consistency, although some individual studies had Cronbach's alpha estimates that were significantly divergent. The RG meta-analysis revealed a pooled Cronbach's alpha of 0.932 for the CCWS. Hogan (2019) describes reliability coefficients exceeding 0.90 as excellent and further states that measurements with this level of reliability can be used to make critical decisions about individuals, such as placement on an occupational licensing examination or diagnosing someone as mentally impaired in a forensic case. Therefore, the measurements yielded from CCWS demonstrate very high reliability, and there is strong agreement among the items of the scale.
The findings from meta-regression and analog ANOVA indicated that none of the examined variables moderated the psychometric properties of the CCWS. This result suggests that the CCWS is a robust measurement tool that is not influenced by potential confounders. To clarify, the CCWS produces scores with high internal consistency regardless of language, geographical location, age, gender, or the total number of items answered during the administration. The fact that gender and age were not significant moderators of the internal reliability of the CCWS aligns with the findings of the study conducted by Donati et al. (2025), which verified the measurement invariance of the CCWS across biological sex and age groups. Conversely, the results regarding the moderator variable of the total number of items answered were somewhat contrary to expectations. When participants respond to a large number of items, factors such as fatigue and inattention may decrease the reliability of the data. However, the results of the moderator analysis revealed that even when participants answered an excessive number of items due to the simultaneous use of the CCWS with other instruments, the reliability coefficient remained as high as when the scale was administered alone. Another important finding from the moderator analysis worth highlighting is the absence of region- and language-related differences in the reliability of CCWS scores, supporting the fact that adaptation studies for the scale have generally yielded similar results to their original form. It is certainly possible that more than half of the studies included in the meta-analysis were conducted in Türkiye using the Turkish version of the CCWS, which may have contributed to this outcome.
The results indicate that the addressed moderator variables did not affect the internal consistency of the CCWS, which could be attributed to various factors. Firstly, climate change is a global issue that can generate similar worries in all individuals, regardless of their demographics. In other words, people of all ages, genders, and geographic backgrounds may worry equally about the effects of climate change, resulting in minimal variability in the internal reliability of the CCWS across the listed variables. Another possible explanation for the moderator variables being non-significant predictors of the CCWS's reliability is that the scale was designed to capture broad, collective, and universally shared concerns that extend beyond age, gender-based perceptions, or culturally specific thoughts. In summary, the moderator analysis results indicate that factors related to the administration conditions of the scale and/or other moderators beyond the variables examined in this study may have contributed to the variability of the CCWS score reliability across the primary studies.
The study also aimed to determine the ratio of reliability induction in the research utilizing the CCWS. It was found that 29.41% of the studies included in the meta-analysis exhibited reliability induction, either by reporting a reliability estimate from previous applications of the scale or by not mentioning reliability at all. This percentage was similar to the rates of reliability induction reported by López-Nicolás et al. (2021), Vicent et al. (2019), and Demir and Demircioglu (2024) in their RG meta-analysis studies. Conversely, it was considerably lower than the average reliability induction rate of 78.6% reported by Sánchez-Meca et al. (2015), who systematically reviewed RG meta-analysis research (cited in López-Nicolás et al., 2021), the 67.95% reported by Correia et al. (2025) in their manuscript on the RG of the pre-sleep arousals scale, and the 76.10% found by Alcocer-Bruno et al. (2020) in the RG of the Medical Outcome Study-HIV Health Survey. As the increase in RG meta-analyses has heightened researchers' awareness of the need to report reliability estimates with the available data rather than inferring them from prior research (López-Nicolás et al., 2021), the relatively low-reliability induction rate observed for the CCWS may be due to the scale being relatively new. Indeed, Sánchez-Meca et al. (2015) identified a positive correlation between the proportion of reliability reporting and the publication year of 100 RG meta-analyses (cited in López-Nicolás et al., 2021). Nevertheless, it should also be noted that the reliability induction rate established in the current study remains a significant finding. It is hoped that this article, which addresses reliability induction as a sub-issue, will act as a caution for researchers planning to utilize the CCWS in the future and will contribute to further decreasing the reliability induction of the scale.
Consequently, CCWS scores demonstrated excellent internal consistency regardless of sample and study characteristics. Given this favorable result, it can be said that CCWS accurately measures worry about climate change, and researchers can confidently utilize the scale. Moreover, the fact that the CCWS can produce highly reliable scores with only 10 items shows that it can be administered quickly and is a powerful instrument even for large survey studies, promoting its continued use. However, researchers should be cautious about the following matter when employing the scale: the high heterogeneity discovered in the reliability of CCWS scores among primary studies confirmed that its reliability cannot be generalized across different population characteristics and study conditions and that inducing reliability is an approach that needs to be eradicated. The variation in the reliability of CCWS from one study to another indicates that when presenting the reliability analysis results, the sample and study characteristics under which reliability was calculated should also be described in detail.
Limitations and future directions
The current study has certain limitations that suggest several areas for future research. Firstly, this RG meta-analysis was confined to Cronbach's alpha coefficient; thus, the reliability considered here pertains only to internal consistency. Because test-retest reliability, a measure of stability, and omega, another internal consistency coefficient, were rarely reported in the available studies employing the CCWS, these coefficients were excluded from the current RG research. In light of this limitation, it is recommended that RG meta-analyses also consider these coefficients, as the number of studies reporting the test-retest reliability and omega coefficient of CCWS scores increases in the future. Furthermore, future RG meta-analyses conducted on the CCWS can explore whether variables beyond the moderators examined in the current study influence the reliability estimates. Finally, as studies using the CCWS and reporting factor loadings of its items or inter-item correlation matrices increase in the literature, meta-analytic factor analysis research can also be pursued.
Data availability statement
Publicly available datasets were analyzed in this study. This data can be found here: http://bit.ly/4i8j9LT.
Author contributions
MG: Conceptualization, Data curation, Investigation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing. YY: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. Mİ: Conceptualization, Data curation, Investigation, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declare that no Gen AI was used in the creation of this manuscript.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Albrecht, G. (2011). “Chronic environmental change: emerging 'psychoterratic' syndromes,” in Climate change and human well-being: Global challenges and opportunities, ed. I. Weissbecker (Cham: Springer Science + Business Media), 43–56. doi: 10.1007/978-1-4419-9742-5_3
Albrecht, G., Sartore, G. M., Connor, L., and Higginbotham, N. (2007). Solastalgia: the distress caused by environmental change. Austral. Psychiat. 15, 95–98. doi: 10.1080/10398560701701288
Alcocer-Bruno, C., Ferrer-Cascales, R., Rubio-Aparicio, M., and Ruiz-Robledillo, N. (2020). The medical outcome study-HIV health survey: a systematic review and reliability generalization meta-analysis. Res. Nurs. Health 43, 610–620. doi: 10.1002/nur.22070
Arnold, C. (2022). Death by climate change. Nat. Clim. Chang. 12, 607–609. doi: 10.1038/s41558-022-01411-8
Aslan, R., Köçer, M. S., and Mizrak, S. (2023). Worry about climate change of outdoor recreation participants: a case study in Türkiye. Weather, Clim. Soc. 15, 801–812. doi: 10.1175/WCAS-D-23-0039.1
Avan, H., and Vural, B. (2025). The relationship between climate change anxiety and depression and insomnia in adolescents. Child. Youth Serv. Rev. 170:108138. doi: 10.1016/j.childyouth.2025.108138
Ayar, D., Gözüaçik, B., and Sen, B. (2024). Effect of global climate change concern on environmental sensitivity of nursing students. Public Health Nurs. 42, 334–340. doi: 10.1111/phn.13434
Aydemir, A. A. (2022). Relationship between climate change worry, religious coping and religiosity among adults [Master thesis, Marmara University]. Istanbul, Türkiye.
Balduzzi, S., Rücker, G., and Schwarzer, G. (2019). How to perform a meta-analysis with R: a practical tutorial. Evid. Based Ment. Health 22, 153–160. doi: 10.1136/ebmental-2019-300117
Bandalos, D. L. (2018). Measurement Theory and Applications for the Social Sciences. New York: The Guilford.
Baykara Mat, S. T., Çalişkan, B. B., and Baştarcan, Ç. (2023). Worry and hope levels of nursing students about climate change: a cross-sectional study. J. Psychiatr. Nurs. 15, 9–16. doi: 10.14744/phd.2023.00187
Begg, C. B., and Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics 12, 1088–1101. doi: 10.2307/2533446
Bonett, D. G. (2010). Varying coefficient meta-analytic methods for alpha reliability. Psychol. Methods 15, 368–385. doi: 10.1037/a0020142
Borenstein, M., Hedges, L. V., Higgins, J. P. T., and Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Res. Synth. Methods 1, 97–111. doi: 10.1002/jrsm.12
Bouman, T., Verschoor, M., Albers, C. J., Böhm, G., Fisher, S. D., Poortinga, W., et al. (2020). When worry about climate change leads to climate action: how values, worry and personal responsibility relate to various climate actions. Global Environ. Change 62:102061. doi: 10.1016/j.gloenvcha.2020.102061
Ceylan, H., Sevgili, A., and Aslan, S. (2024). Determination of climate change awareness and anxiety levels of social work students. J. Univ. Res. 7, 131–143. doi: 10.32329/uad.1442686
Charlson, F., Ali, S., Benmarhnia, T., Pearl, M., Massazza, A., Augustinavicius, J., et al. (2021). Climate change and mental health: a scoping review. Int. J. Environ. Res. Public Health 18:4486. doi: 10.3390/ijerph18094486
Cianconi, P., Betrò, S., and Janiri, L. (2020). The impact of climate change on mental health: a systematic descriptive review. Front. Psychiat. 11:74. doi: 10.3389/fpsyt.2020.00074
Clayton, S. (2020). Climate anxiety: psychological responses to climate change. J. Anxiety Disord. 74:102263. doi: 10.1016/j.janxdis.2020.102263
Clayton, S., and Karazsia, B. T. (2020). Development and validation of a measure of climate change anxiety. J. Environ. Psychol. 69:101434. doi: 10.1016/j.jenvp.2020.101434
Clayton, S. D., Pihkala, P., Wray, B., and Marks, E. (2023). Psychological and emotional responses to climate change among young people worldwide: differences associated with gender, age, and country. Sustainability 15:3540. doi: 10.3390/su15043540
Comrie, A. (2007). Climate change and human health. Geogr. Compass 1, 246–712. doi: 10.1111/j.1749-8198.2007.00037.x
Correia, I. L., Husain, W., Gonçalves, M. T., Pires, L., Trabelsi, K., Achraf, A., et al. (2025). A reliability generalization meta-analysis of the internal consistency and test-retest reliability of the pre-sleep arousal scale (PSAS). Sleep Med. 126, 290–299. doi: 10.1016/j.sleep.2024.12.030
Crocker, L., and Algina, J. (1986). Introduction to Classical and Modern Test Theory. San Diego: Harcourt Brace Jovanovich.
Cruz, S. M., and High, A. C. (2022). Psychometric properties of the climate change anxiety scale. J. Environ. Psychol. 84, 1–6. doi: 10.1016/j.jenvp.2022.101905
Cunsolo, A., and Ellis, N. R. (2018). Ecological grief as a mental health response to climate change-related loss. Nat. Clim. Chang. 8, 275–281. doi: 10.1038/s41558-018-0092-2
D'Amato, G., Cecchi, L., and D'Amato, M. (2014). Climate change and respiratory diseases. Eur. Respir. Rev. 23, 161–169. doi: 10.1183/09059180.00001714
Deeks, J. J., Higgins, J. P. T., and Altman, D. G. (2008). “Analysis data and undertaking meta-analyses,” in Cochrane handbook for systematic reviews of interventions, eds. J. Higgins and S. Green (New York: Wiley), 243–296. doi: 10.1002/9780470712184.ch9
DeJarnett, N., Jennings, V., and Pathak, N. (2025). “Confronting the realities of climate change,” in Environmental health: Foundations for public health, eds. N. Sampson, L. Tallon and N. DeJarnett (Cham; Springer), 43–71. doi: 10.1891/9780826183538.0003
Demir, E., and Demircioglu, H. (2024). A reliability generalization meta-analysis of the family resilience assessment scale (FRAS). J. Pediatr. Nurs. 77, 150–157. doi: 10.1016/j.pedn.2024.04.003
Demir, R., Ölmez Yalaz,i, R., and Din,ç, A. (2023). The relationship between women's climate change awareness and concerns about climate change in Türkiye. Public Health Nurs. 41, 215–220. doi: 10.1111/phn.13269
Dodds, J. (2021). The psychology of climate anxiety. BJPsych Bull. 45, 222–226. doi: 10.1192/bjb.2021.18
Dogan, H., and Buz, S. (2024). The effect of climate change anxiety on social work students in Turkey. Int. Soc. Work 67, 1045–1058. doi: 10.1177/00208728231208007
Donati, M. A., Santisi, S., Di Leonardo, L., and Primi, C. (2025). How to measure climate change worry in adolescents? Psychometric properties of the Climate Change Worry Scale. Int. J. Behav. Dev. 49, 195–203. doi: 10.1177/01650254241266119
Duckwitz, D. (2024). Understanding climate change threat perception: exploring implications for everyday altruistic behavior [Master thesis, Utrecht University]. Utrecht, Netherland.
Dugstad, A., Grimsrud, K., Kipperberg, G., Lindhjem, H., and Navrud, S. (2023). “The impact of choice architecture on preferences for public goods management regimes: an experimental study of attribute translations and signposting,” in Paper presented at the 28th Annual Conference of the European Association of Environmental and Resource Economists, University of Cyprus.
Duran, S., and Kaynak, S. (2024). Climate change worry among nurses and their hope levels for climate change prevention. BMC Nurs. 23:419. doi: 10.1186/s12912-024-02067-9
Duval, S., and Tweedie, R. (2000). Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56, 455–463. doi: 10.1111/j.0006-341X.2000.00455.x
Ecer, K., Varol Ülker, S., and Hizli Sayar, G. (2023). Toward a more circular economy: Would climate change worry affect preference for recycled products? A psychophysiological study. Sustain. Clim. Change 16, 437–448. doi: 10.1089/scc.2023.0089
Egger, M., Smith, G. D., Schneider, M., and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315, 629–634. doi: 10.1136/bmj.315.7109.629
European Environment Agency (2024). 8th Environment Action Programme Economic losses from weather- and climate-related extremes in Europe. Available online at: https://www.eea.europa.eu/publications/european-union-8th-environment-action-programme/indicators/03-economic-losses-from-weather (accessed December 20, 2024).
Field, A. P., and Gillett, R. (2010). How to do a meta-analysis. Br. J. Math. Stat. Psychol. 63, 665–694. doi: 10.1348/000711010X502733
Franchini, M., and Mannucci, P. M. (2015). Impact on human health of climate changes. Eur. J. Intern. Med. 26, 1–5. doi: 10.1016/j.ejim.2014.12.008
Geraci, A., Giordano, G., Cucinella, N., Cannav,ò, M., Cavarretta, M. V., Alesi, M., et al. (2024). Psychological dimensions associated with youth engagement in climate change issues: a person-centered approach. Curr. Psychol. 43, 25836–25846. doi: 10.1007/s12144-024-06263-3
Gezer, M., and Ilhan, M. (2021). Climate change worry scale: Turkish adaptation study. Aegean Geogr. J. 30, 195–204. doi: 10.51800/ecd.932817
Gottwlad, I. (2024). A new climate narrative: The potential of fictional storytelling in climate change communication [Master thesis, Universität Wien]. Vienna.
Hamama-Raz, Y., and Shinan-Altman, S. (2024). Does climate change worry decrease during armed conflicts? Climate 12:162. doi: 10.3390/cli12100162
Hickman, C., Marks, E., Pihkala, P., Clayton, S., Lewandowski, R. E., Mayall, E. E., et al. (2021). Climate anxiety in children and young people and their beliefs about government responses to climate change: A global survey. Lancet Planet Health 5, 863–873. doi: 10.1016/S2542-5196(21)00278-3
Hogg, T. L., Stanley, S. K., O'Brien, L. V., Wilson, M. S., and Watsford, C. R. (2021). The Hogg eco-anxiety scale: development and validation of a multidimensional scale. Global Environ. Change 71, 1–10. doi: 10.1016/j.gloenvcha.2021.102391
Ilaslan, N., and Orak, N. S. (2024). Relationship between nursing students' global climate change awareness, climate change anxiety and sustainability attitudes in nursing: a descriptive and cross-sectional study. BMC Nurs. 23:573. doi: 10.1186/s12912-024-02252-w
Innocenti, M., Santarelli, G., Faggi, V., Ciabini, L., Castellini, G., Galassi, F., et al. (2022). Psychometric properties of the Italian version of the climate change worry scale. J. Clim. Change Health 6:100140. doi: 10.1016/j.joclim.2022.100140
IPCC. (2023). “Summary for policymakers,” in Climate Change 2023: Synthesis Report. Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, eds. Core Writing Team, H. Lee and J. Romero (IPCC).
Jahrami, H., Husain, W., Lin, C. Y., Björling, G., Potenza, M. N., and Pakpour, A. (2024). Reliability generalization meta-analysis and psychometric review of the gaming disorder test (GDT): Evaluating internal consistency. Addict. Behav. Rep. 20:100563. doi: 10.1016/j.abrep.2024.100563
Kabasakal Çetin, A., Sen, G., and Aksaray, B. (2025). Examining the association of climate change worry and awareness of ecological footprint reduction behaviours with Mediterranean diet adherence and climate-friendly food choices. Br. Food J. 127, 168–181. doi: 10.1108/BFJ-06-2024-0577
Kappelle, M., Van Vuuren, M. M., and Baas, P. (1999). Effects of climate change on biodiversity: a review and identification of key research issues. Biodivers. Conserv. 8, 1383–1397. doi: 10.1023/A:1008934324223
Kars Fertelli, T. (2023). Awareness, worry, and hope regarding climate change among nurses: a cross-sectional study. Arch. Environ. Occupat. Health 78, 413–422. doi: 10.1080/19338244.2023.2278521
Kim, K. H., Kabir, E., and Ara Jahan, S. (2014). A review of the consequences of global climate change on human health. J. Environ. Sci. Health, Part C 32, 299–318. doi: 10.1080/10590501.2014.941279
Kumar, S., Ghosh, F. C. M. N. A., and Khanna, S. (2024). “The mental health climate crisis: Unveiling the hidden consequences of global warming,” in The impact of climate change on social and mental well-being, eds. M. Garg, D. Kumar, D. Samanta and A. Sathiyaseelan (London: Academic), 285–298. doi: 10.1016/B978-0-443-23788-1.00016-6
Kurt, G., and Akdur, R. (2024). Under what conditions does climate change worry contribute to climate action in Turkey: what moderates this relationship? Sustainability 16:2269. doi: 10.3390/su16062269
Larionow, P., Gawrych, M., Mackiewicz, J., Michalak, M., Mudło-Głagolska, K., Preece, D. A., et al. (2024). The climate change worry scale (CCWS) and its links with demographics and mental health outcomes in a Polish sample. Healthcare 12:1128. doi: 10.3390/healthcare12111128
Le, N., Lam, P. H., Tuyet, C. H., and Hoa, N. T. L. (2024). Impact of emotional perceptions and social influences on green consumption practices in Vietnam. Challenges Sustain. 12, 34–51. doi: 10.56578/cis120103
Lin, L. (2019). Comparison of four heterogeneity measures for meta-analysis. J. Eval. Clin. Pract. 26, 376–384. doi: 10.1111/jep.13159
Lister, M. (2014). Climate change refugees. Crit. Rev. Int. Soc. Polit. Philos. 17, 618–634. doi: 10.1080/13698230.2014.919059
López-Ibáñez, C., López-Nicolás, R., Blázquez-Rincón, D. M., and Sánchez-Meca, J. (2024). Reliability generalization meta-analysis: comparing different statistical methods. Curr. Psychol. 43, 18275–18293 doi: 10.1007/s12144-023-05604-y
López-Nicolás, R., Rubio-Aparicio, M., López-Ibáñez, C., and Sánchez-Meca1, J. (2021). A reliability generalization meta-analysis of the dimensional obsessive-compulsive scale. Psicothema 33, 481–489. doi: 10.7334/psicothema2020.455
Martin, G., Roswell, T., and Cosma, A. (2024). Exploring the relationships between worry about climate change, belief about personal responsibility, and mental wellbeing among adolescents and young adults. Wellbeing, Space Soc. 6:100198. doi: 10.1016/j.wss.2024.100198
McElwee, P. (2021). Climate change and biodiversity loss: two sides of the same coin. Curr. History 120, 295–300. doi: 10.1525/curh.2021.120.829.295
Mikhaylov, A., Moiseev, N., Aleshin, K., and Burkhardt, T. (2020). Global climate change and greenhouse effect. Entrepr. Sustain. Issues 7, 2897–2913. doi: 10.9770/jesi.2020.7.4(21)
Miles, M. B., and Huberman, A. M. (1994). Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks: SAGE Publications, Inc.
Morrison, G. R., Ross, S. M., Kalman, H. K., and Kemp, J. E. (2011). Designing Effective Instruction (6th ed.). New York: John Wiley and Sons.
Ojala, M., Cunsolo, A., Ogunbode, C. A., and Middleton, J. (2021). Anxiety, worry, and grief in a time of environmental and climate crisis: a narrative review. Annu. Rev. Environ. Resour. 46, 35–58. doi: 10.1146/annurev-environ-012220-022716
Orr, M., Stewart, A., and Grundstein, A. (2020). Investigating connections between need for cognitive closure and climate change concern in college students. Int. J. Environ. Res. Public Health 17:5619. doi: 10.3390/ijerph17155619
Özbay, S., and Alci, B. (2021). Climate change worry scale: adaptation to Turkish, validity and reliability study. Res. Stud. Anatolia J. 4, 183–193. doi: 10.33723/rs.958016
Palinkas, L. A., and Wong, M. (2020). Global climate change and mental health. Curr. Opin. Psychol. 32, 12–16. doi: 10.1016/j.copsyc.2019.06.023
Pentapati, K. C., Chenna, D., Kumar, V. S., and Kumar, N. (2025). Reliability generalization meta-analysis of Cronbach's alpha of the oral impacts on daily performance (OIDP) questionnaire. BMC Oral Health 25:220. doi: 10.1186/s12903-025-05496-3
Pereira, A. T., Cabaços, C., Marques, C. C., Araújo, A. I., and Macedo, A. (2024). Perfectionism and cognitive and emotional reactions to climate change and psychological distress. Sustainability 16:9221. doi: 10.3390/su16219221
Plohl, N., Mlakar, I., Musil, B., and Smrke, U. (2023). Measuring young individuals' responses to climate change: validation of the Slovenian versions of the climate anxiety scale and the climate change worry scale. Front. Psychol. 14:1297782. doi: 10.3389/fpsyg.2023.1297782
Qi, B., Presseller, E. K., Cooper, G. E., Kapadia, A., Dumain, A. S., Jayawickreme, S. M., et al. (2022). Development and validation of an eating-related eco-concern questionnaire. Nutrients 14:4517. doi: 10.3390/nu14214517
Regnoli, G. M., Tiano, G., and De Rosa, B. (2024). Is climate change worry fostering young Italian adults' psychological distress? An Italian exploratory study on the mediation role of intolerance of uncertainty and future anxiety. Climate 12:118. doi: 10.3390/cli12080118
Reser, J. P., and Swim, J. K. (2011). Adapting to and coping with the threat and impacts of climate change. Am. Psychol. 66, 277–289. doi: 10.1037/a0023412
Rodriguez, M. C., and Maeda, Y. (2006). Meta-analysis of coefficient alpha. Psychol. Methods 11, 306–322. doi: 10.1037/1082-989X.11.3.306
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychol. Bull. 86, 638–641. doi: 10.1037/0033-2909.86.3.638
Sánchez-Meca, J., López-López, J. A., and López-Pina, J. A. (2013). Some recommended statistical analytic practiceswhen reliability generalization studies are conducted. Br. J. Math. Stat. Psychol. 66, 402–425. doi: 10.1111/j.2044-8317.2012.02057.x
Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., et al. (2021). Improving the reporting quality of reliability generalization meta-analyses: the REGEMA checklist. Res. Synthesis Methods 12, 407–570. doi: 10.1002/jrsm.1487
Sánchez-Meca, J., Rubio-Aparicio, M., Marin-Martinez, F., and Lopez-Pina, J. A. (2015). “The phenomenon of reliability induction in social and health sciences,” in Paper presented Fifteenth Congress of Methodology of Behavioral Science (Palma de Mallorca: Spanish Association of Behavioral Sciences Methodology).
Searle, K., and Gow, K. (2010). Do concerns about climate change lead to distress? Int. J. Clim. Change Strat. Manage. 2, 362–379. doi: 10.1108/17568691011089891
Semenderoglu, A., Sanl,i, V., and Arslan, K. (2024). Anxiety levels regarding global climate change among students of Buca Faculty of Education. Int. J. Geogr. Geogr. Educ. 53, 39–58. doi: 10.32003/igge.1550018
Semma, B., Henri, M., Luo, W., and Thompson, C. G. (2018). Reliability generalization of the meaning in life questionnaire subscales. J. Psychoeduc. Assess. 37, 837–851. doi: 10.1177/0734282918800739
Sen, Ö. L. (2013). Holistic picture of climate change in Turkey. III. Turkey Climate Change Congress, Istanbul Technical University (TIKDEK 2013).
Sen, S., and Yörük, S. (2023). A reliability generalization meta-analysis of the Kaufman domains of creativity scale. J. Creat. Behav. 57, 467–837. doi: 10.1002/jocb.620
Servan, M. (2024). Examining the relationship between middle school students' climate literacy levels and their concerns about climate change [Master thesis, Aydin Adnan Menderes University]. National Thesis Center.
Shepherd, S., Raynal, P., and Guedj, M. (2024). Psychometric properties of the French version of the climate change worry scale. J. Clim. Change Health 20:100361. doi: 10.1016/j.joclim.2024.100361
Shinan-Altman, S., and Hamama-Raz, Y. (2023). The Association between climate change exposure and climate change worry among Israeli adults: the interplay of risk appraisal, collective efficacy, age, and gender. Sustainability 15, 1–14. doi: 10.3390/su151813689
Smith, C., Allen, A., Schaffer, V., and Kannis-Dymand, L. (2023). Nature relatedness may play a protective role and contribute to eco-distress. Ecopsychology 16, 1–12. doi: 10.1089/eco.2023.0004
Stewart, A. E. (2021). Psychometric properties of the climate change worry scale. Int. J. Environ. Res. Public Health 18:494. doi: 10.3390/ijerph18020494
Stewart, A. E., Chapman, H. E., and Davis, J. B. L. (2024). Anxiety and worry about six categories of climate change impacts. Int. J. Environ. Res. Public Health 21:23. doi: 10.3390/ijerph21010023
Sümen, A., Kublay, Y., and Adibelli, S. (2025). Evaluation of adult individuals' climate change concern and mental well-being levels in Türkiye: a descriptive and correlational study. Public Health Nurs. 42, 655–664. doi: 10.1111/phn.13493
Taylor, S. (2020). Anxiety disorders, climate change, and the challenges ahead: Introduction to the special issue. J. Anxiety Disord. 76:102313. doi: 10.1016/j.janxdis.2020.102313
Trenberth, K. E. (2018). Climate change caused by human activities is happening and it already has major consequences. J. Energy Nat. Resour. Law 36, 463–481. doi: 10.1080/02646811.2018.1450895
Tümer, A., Ipek, M., and Ercan, Z. (2024). Nursing students' awareness, worry and hope levels regarding climate change: a cross-sectional and correlational study. J. Public Health Nurs. 6, 29–38. doi: 10.54061/jphn.1396915
Türe, K. (2024). The impact of climate change worry on sustainable consumption behaviours [Master thesis, Marmara University]. Istanbul, Türkiye.
Türkarslan, K. K., Kozak, E. D., and Yildirim, J. C. (2023). Psychometric properties of the Hogg Eco-Anxiety Scale (HEAS-13) and the prediction of pro-environmental behavior. J. Environ. Psychol. 92:102147. doi: 10.1016/j.jenvp.2023.102147
Türkeş, M. (2008). What is climate change? Basic definition, causes, observed and predicted results of climate change. Clim. Change Environ. 1, 26–37.
United Nations (2022). Climate change the greatest threat the world has ever faced, UN expert warns. United NationsHuman Rights: Office of the High Commissioner. Available online at: https://www.ohchr.org/en/press-releases/2022/10/climate-change-greatest-threat-world-has-ever-faced-un-expert-warns
Usta, G. (2023). Determining the climate change anxiety levels of university students. J. Selçuk Univ. Soc. Sci. Inst. 51, 159–167. doi: 10.52642/susbed.1282642
Vacha-Haase, T. (1998). Reliability generalization: exploring variance in measurement error afecting score reliability across studies. Educ. Psychol. Meas. 58, 6–20. doi: 10.1177/0013164498058001002
Vacha-Haase, T., Kogan, L. R., and Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: validity of score reliability inductions. Educ. Psychol. Meas. 60, 509–522. doi: 10.1177/00131640021970682
van Valkengoed, A. M., Steg, L., and de Jonge, P. (2023). Climate anxiety: a research agenda inspired by emotion research. Emot. Rev. 15, 258–262. doi: 10.1177/17540739231193752
Verplanken, B., Marks, E., and Dobromir, A. I. (2020). On the nature of eco-anxiety: how constructive or unconstructive is habitual worry about global warming? J. Environ. Psychol. 72:101528. doi: 10.1016/j.jenvp.2020.101528
Vicent, M., Rubio-Aparicio, M., Sánchez-Meca, J., and Gonzálvez, C. (2019). A reliability generalization meta-analysis of the child and adolescent perfectionism scale. J. Affect. Disord. 245, 533–544. doi: 10.1016/j.jad.2018.11.049
Viechtbauer, W. (2010). Metafor meta-analysis package for R, Version 1.4-0 [Computer software]. Available online at: https://cran.r-project.org/web/packages/metafor/index.html (accessed February 15, 2025). doi: 10.32614/CRAN.package.metafor
Whitmarsh, L. (2008). Are flood victims more concerned about climate change than other people? The role of direct experience in risk perception and behavioural response. J. Risk Res. 11, 351–374. doi: 10.1080/13669870701552235
Whitmarsh, L., Player, L., Jiongco, A., James, M., Williams, M., Marks, E., et al. (2022). Climate anxiety: what predicts it and how is it related to climate action? J. Environ. Psychol. 83:101866. doi: 10.1016/j.jenvp.2022.101866
Keywords: Climate Change Worry Scale, internal consistency, meta-analysis, reliability generalization, reliability induction
Citation: Gezer M, Yıldırım Y and İlhan M (2025) Reliability generalization meta-analysis of the Climate Change Worry Scale. Front. Psychol. 16:1590126. doi: 10.3389/fpsyg.2025.1590126
Received: 08 March 2025; Accepted: 07 April 2025;
Published: 25 April 2025.
Edited by:
Francisco Sampaio, Escola Superior de Enfermagem do Porto, PortugalReviewed by:
Mohamed Ali, Sapienza University of Rome, ItalyGiulia Rocchi, Sapienza University of Rome, Italy
Ida Bagus Ari Arjaya, Universitas Mahasaraswati Denpasar, Indonesia
Bartolomeus Adiwena, Soegijapranata Catholic University, Indonesia
Copyright © 2025 Gezer, Yıldırım and İlhan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Melehat Gezer, bWVsZWhhdGdlemVyQGdtYWlsLmNvbQ==